2024-09-17T02:25:51.593 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T02:25:51.615 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T02:25:51.697 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354 branch: wip-pdonnell-testing-20240916.200549-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/forward-scrub} email: pdonnell@redhat.com first_in_suite: false job_id: '7908354' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240916.200549-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected - object missing on disk - Invalid tag char sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240916.200549-debug sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 476 sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf sleep_before_teardown: 0 subset: 320/512 suite: fs suite_branch: wip-pdonnell-testing-20240916.200549-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEVJGVwn+bFMVh1aaHw3OXUAPc4kNfoLIOKr/OL4XxkyscHKWWoihRgeFVXdf/cbNESwKJ0DzyTVZPsyZFrdhCQ= smithi189.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLRkqELI6tuDKtYxD0/Pwx0mR+tqTRrgAidtPxFlaUpbYnbT4T48yM71nEYZTodL8HnZDNwln0GVfnNoOTwEow= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_02:09:51 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T02:25:51.698 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa; will attempt to use it 2024-09-17T02:25:51.698 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks 2024-09-17T02:25:51.699 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T02:25:51.701 INFO:teuthology.task.internal:Checking packages... 2024-09-17T02:25:51.724 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf' 2024-09-17T02:25:51.724 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T02:25:51.724 INFO:teuthology.packaging:ref: None 2024-09-17T02:25:51.724 INFO:teuthology.packaging:tag: None 2024-09-17T02:25:51.725 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240916.200549-debug 2024-09-17T02:25:51.725 INFO:teuthology.packaging:sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T02:25:51.725 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240916.200549-debug 2024-09-17T02:25:51.879 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4989-gce1b342b-1jammy 2024-09-17T02:25:51.880 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T02:25:51.886 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T02:25:51.886 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T02:25:51.892 INFO:teuthology.task.internal:Saving configuration 2024-09-17T02:25:51.909 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T02:25:51.915 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T02:25:51.937 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 02:19:57.019332', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAID2soQnZbvbc7YOU16G/OHyHNGYb3+VyI+CjBKtb4JOE'} 2024-09-17T02:25:51.963 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi189.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 02:19:57.017919', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIA5RhIhSmMyVSG0ohYkcs2Ohv5+rJbv+Ji9UTRLpPCRc'} 2024-09-17T02:25:51.963 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T02:25:51.970 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-17T02:25:51.970 INFO:teuthology.task.internal:roles: ubuntu@smithi189.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-17T02:25:51.970 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T02:25:52.032 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4af9067ac0>, signals=[15]) 2024-09-17T02:25:52.032 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T02:25:52.038 INFO:teuthology.task.internal:Opening connections... 2024-09-17T02:25:52.038 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:25:52.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T02:25:52.121 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:25:52.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T02:25:52.199 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T02:25:52.205 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-09-17T02:25:52.211 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-09-17T02:25:52.211 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-09-17T02:25:52.258 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T02:25:52.259 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="Ubuntu" 2024-09-17T02:25:52.259 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="22.04" 2024-09-17T02:25:52.259 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T02:25:52.259 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_CODENAME=jammy 2024-09-17T02:25:52.259 INFO:teuthology.orchestra.run.smithi131.stdout:ID=ubuntu 2024-09-17T02:25:52.259 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE=debian 2024-09-17T02:25:52.259 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T02:25:52.259 INFO:teuthology.orchestra.run.smithi131.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T02:25:52.259 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T02:25:52.259 INFO:teuthology.orchestra.run.smithi131.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T02:25:52.259 INFO:teuthology.orchestra.run.smithi131.stdout:UBUNTU_CODENAME=jammy 2024-09-17T02:25:52.260 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-09-17T02:25:52.286 DEBUG:teuthology.orchestra.run.smithi189:> uname -m 2024-09-17T02:25:52.292 INFO:teuthology.orchestra.run.smithi189.stdout:x86_64 2024-09-17T02:25:52.292 DEBUG:teuthology.orchestra.run.smithi189:> cat /etc/os-release 2024-09-17T02:25:52.339 INFO:teuthology.orchestra.run.smithi189.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-17T02:25:52.339 INFO:teuthology.orchestra.run.smithi189.stdout:NAME="Ubuntu" 2024-09-17T02:25:52.339 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_ID="22.04" 2024-09-17T02:25:52.339 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-17T02:25:52.339 INFO:teuthology.orchestra.run.smithi189.stdout:VERSION_CODENAME=jammy 2024-09-17T02:25:52.339 INFO:teuthology.orchestra.run.smithi189.stdout:ID=ubuntu 2024-09-17T02:25:52.339 INFO:teuthology.orchestra.run.smithi189.stdout:ID_LIKE=debian 2024-09-17T02:25:52.339 INFO:teuthology.orchestra.run.smithi189.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-17T02:25:52.339 INFO:teuthology.orchestra.run.smithi189.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-17T02:25:52.339 INFO:teuthology.orchestra.run.smithi189.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-17T02:25:52.340 INFO:teuthology.orchestra.run.smithi189.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-17T02:25:52.340 INFO:teuthology.orchestra.run.smithi189.stdout:UBUNTU_CODENAME=jammy 2024-09-17T02:25:52.340 INFO:teuthology.lock.ops:Updating smithi189.front.sepia.ceph.com on lock server 2024-09-17T02:25:52.361 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T02:25:52.369 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T02:25:52.374 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T02:25:52.374 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-09-17T02:25:52.377 DEBUG:teuthology.orchestra.run.smithi189:> test '!' -e /home/ubuntu/cephtest 2024-09-17T02:25:52.386 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T02:25:52.392 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T02:25:52.392 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-09-17T02:25:52.424 DEBUG:teuthology.orchestra.run.smithi189:> test -z $(ls -A /var/lib/ceph) 2024-09-17T02:25:52.447 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T02:25:52.495 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T02:25:52.512 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T02:25:52.512 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T02:25:52.512 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T02:25:52.513 DEBUG:teuthology.orchestra.run.smithi189:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T02:25:52.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:25:52.518 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-09-17T02:25:52.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:25:52.519 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-17T02:25:52.563 INFO:teuthology.orchestra.run.smithi131.stdout:5.15.0-56-generic 2024-09-17T02:25:52.563 INFO:teuthology.task.kernel:Running kernel on smithi131: 5.15.0-56-generic 2024-09-17T02:25:52.564 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-get clean 2024-09-17T02:25:52.568 INFO:teuthology.orchestra.run.smithi189.stdout:5.15.0-56-generic 2024-09-17T02:25:52.569 INFO:teuthology.task.kernel:Running kernel on smithi189: 5.15.0-56-generic 2024-09-17T02:25:52.569 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-get clean 2024-09-17T02:25:52.695 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-get update 2024-09-17T02:25:52.721 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-get update 2024-09-17T02:25:52.828 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T02:25:52.844 INFO:teuthology.orchestra.run.smithi131.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-17T02:25:52.844 INFO:teuthology.orchestra.run.smithi131.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-17T02:25:52.908 INFO:teuthology.orchestra.run.smithi131.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-17T02:25:52.995 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T02:25:52.995 INFO:teuthology.orchestra.run.smithi189.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-17T02:25:53.042 INFO:teuthology.orchestra.run.smithi131.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-17T02:25:53.061 INFO:teuthology.orchestra.run.smithi189.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-17T02:25:53.120 INFO:teuthology.orchestra.run.smithi131.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-17T02:25:53.120 INFO:teuthology.orchestra.run.smithi131.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-17T02:25:53.156 INFO:teuthology.orchestra.run.smithi131.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-17T02:25:53.159 INFO:teuthology.orchestra.run.smithi131.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-17T02:25:53.159 INFO:teuthology.orchestra.run.smithi131.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-17T02:25:53.175 INFO:teuthology.orchestra.run.smithi131.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-17T02:25:53.175 INFO:teuthology.orchestra.run.smithi131.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-17T02:25:53.178 INFO:teuthology.orchestra.run.smithi131.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-17T02:25:53.178 INFO:teuthology.orchestra.run.smithi131.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-17T02:25:53.181 INFO:teuthology.orchestra.run.smithi131.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-17T02:25:53.185 INFO:teuthology.orchestra.run.smithi131.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-17T02:25:53.190 INFO:teuthology.orchestra.run.smithi131.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-17T02:25:53.205 INFO:teuthology.orchestra.run.smithi131.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-17T02:25:53.206 INFO:teuthology.orchestra.run.smithi131.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-17T02:25:53.206 INFO:teuthology.orchestra.run.smithi131.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-17T02:25:53.208 INFO:teuthology.orchestra.run.smithi131.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-17T02:25:53.209 INFO:teuthology.orchestra.run.smithi131.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-17T02:25:53.209 INFO:teuthology.orchestra.run.smithi131.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-17T02:25:53.209 INFO:teuthology.orchestra.run.smithi131.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-17T02:25:53.210 INFO:teuthology.orchestra.run.smithi131.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-17T02:25:53.211 INFO:teuthology.orchestra.run.smithi131.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-17T02:25:53.220 INFO:teuthology.orchestra.run.smithi131.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-17T02:25:53.222 INFO:teuthology.orchestra.run.smithi131.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-17T02:25:53.223 INFO:teuthology.orchestra.run.smithi131.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-17T02:25:53.228 INFO:teuthology.orchestra.run.smithi131.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-17T02:25:53.228 INFO:teuthology.orchestra.run.smithi131.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-17T02:25:53.229 INFO:teuthology.orchestra.run.smithi131.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-17T02:25:53.229 INFO:teuthology.orchestra.run.smithi131.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-17T02:25:53.244 INFO:teuthology.orchestra.run.smithi131.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-17T02:25:53.244 INFO:teuthology.orchestra.run.smithi131.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-17T02:25:53.248 INFO:teuthology.orchestra.run.smithi131.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-17T02:25:53.248 INFO:teuthology.orchestra.run.smithi131.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-17T02:25:53.255 INFO:teuthology.orchestra.run.smithi131.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-17T02:25:53.259 INFO:teuthology.orchestra.run.smithi131.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-17T02:25:53.261 INFO:teuthology.orchestra.run.smithi131.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-17T02:25:53.261 INFO:teuthology.orchestra.run.smithi131.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-17T02:25:53.261 INFO:teuthology.orchestra.run.smithi131.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-17T02:25:53.273 INFO:teuthology.orchestra.run.smithi131.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-17T02:25:53.273 INFO:teuthology.orchestra.run.smithi131.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-17T02:25:53.416 INFO:teuthology.orchestra.run.smithi189.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-17T02:25:53.435 INFO:teuthology.orchestra.run.smithi189.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-17T02:25:53.554 INFO:teuthology.orchestra.run.smithi189.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-17T02:25:53.779 INFO:teuthology.orchestra.run.smithi189.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-17T02:25:53.863 INFO:teuthology.orchestra.run.smithi189.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-17T02:25:53.879 INFO:teuthology.orchestra.run.smithi189.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-17T02:25:53.887 INFO:teuthology.orchestra.run.smithi189.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-17T02:25:53.888 INFO:teuthology.orchestra.run.smithi189.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-17T02:25:53.910 INFO:teuthology.orchestra.run.smithi189.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-17T02:25:53.922 INFO:teuthology.orchestra.run.smithi189.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-17T02:25:53.923 INFO:teuthology.orchestra.run.smithi189.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-17T02:25:53.985 INFO:teuthology.orchestra.run.smithi189.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-17T02:25:53.985 INFO:teuthology.orchestra.run.smithi189.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-17T02:25:53.995 INFO:teuthology.orchestra.run.smithi189.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-17T02:25:54.034 INFO:teuthology.orchestra.run.smithi189.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-17T02:25:54.034 INFO:teuthology.orchestra.run.smithi189.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-17T02:25:54.034 INFO:teuthology.orchestra.run.smithi189.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-17T02:25:54.042 INFO:teuthology.orchestra.run.smithi189.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-17T02:25:54.042 INFO:teuthology.orchestra.run.smithi189.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-17T02:25:54.048 INFO:teuthology.orchestra.run.smithi189.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-17T02:25:54.051 INFO:teuthology.orchestra.run.smithi189.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-17T02:25:54.056 INFO:teuthology.orchestra.run.smithi189.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-17T02:25:54.058 INFO:teuthology.orchestra.run.smithi189.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-17T02:25:54.059 INFO:teuthology.orchestra.run.smithi189.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-17T02:25:54.059 INFO:teuthology.orchestra.run.smithi189.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-17T02:25:54.066 INFO:teuthology.orchestra.run.smithi189.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-17T02:25:54.069 INFO:teuthology.orchestra.run.smithi189.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-17T02:25:54.075 INFO:teuthology.orchestra.run.smithi189.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-17T02:25:54.076 INFO:teuthology.orchestra.run.smithi189.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-17T02:25:54.125 INFO:teuthology.orchestra.run.smithi189.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-17T02:25:54.126 INFO:teuthology.orchestra.run.smithi189.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-17T02:25:54.151 INFO:teuthology.orchestra.run.smithi189.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-17T02:25:54.151 INFO:teuthology.orchestra.run.smithi189.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-17T02:25:54.151 INFO:teuthology.orchestra.run.smithi189.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-17T02:25:54.152 INFO:teuthology.orchestra.run.smithi189.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-17T02:25:54.153 INFO:teuthology.orchestra.run.smithi189.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-17T02:25:54.156 INFO:teuthology.orchestra.run.smithi189.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-17T02:25:54.157 INFO:teuthology.orchestra.run.smithi189.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-17T02:25:54.157 INFO:teuthology.orchestra.run.smithi189.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-17T02:25:54.238 INFO:teuthology.orchestra.run.smithi189.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-17T02:25:54.238 INFO:teuthology.orchestra.run.smithi189.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-17T02:25:59.608 INFO:teuthology.orchestra.run.smithi131.stdout:Fetched 16.1 MB in 2s (7,706 kB/s) 2024-09-17T02:26:00.194 INFO:teuthology.orchestra.run.smithi189.stdout:Fetched 16.1 MB in 3s (5,824 kB/s) 2024-09-17T02:26:01.025 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2024-09-17T02:26:01.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T02:26:01.107 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2024-09-17T02:26:01.317 INFO:teuthology.orchestra.run.smithi131.stdout:Building dependency tree... 2024-09-17T02:26:01.317 INFO:teuthology.orchestra.run.smithi131.stdout:Reading state information... 2024-09-17T02:26:01.506 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T02:26:01.507 INFO:teuthology.orchestra.run.smithi131.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T02:26:01.507 INFO:teuthology.orchestra.run.smithi131.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T02:26:01.507 INFO:teuthology.orchestra.run.smithi131.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T02:26:01.507 INFO:teuthology.orchestra.run.smithi131.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-17T02:26:01.508 INFO:teuthology.orchestra.run.smithi131.stdout: python2.7-minimal 2024-09-17T02:26:01.508 INFO:teuthology.orchestra.run.smithi131.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T02:26:01.509 INFO:teuthology.orchestra.run.smithi131.stdout:The following additional packages will be installed: 2024-09-17T02:26:01.509 INFO:teuthology.orchestra.run.smithi131.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T02:26:01.509 INFO:teuthology.orchestra.run.smithi131.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-17T02:26:01.509 INFO:teuthology.orchestra.run.smithi131.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-17T02:26:01.511 INFO:teuthology.orchestra.run.smithi131.stdout:Suggested packages: 2024-09-17T02:26:01.511 INFO:teuthology.orchestra.run.smithi131.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-17T02:26:01.511 INFO:teuthology.orchestra.run.smithi131.stdout:Recommended packages: 2024-09-17T02:26:01.511 INFO:teuthology.orchestra.run.smithi131.stdout: thermald 2024-09-17T02:26:01.551 INFO:teuthology.orchestra.run.smithi131.stdout:The following NEW packages will be installed: 2024-09-17T02:26:01.551 INFO:teuthology.orchestra.run.smithi131.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T02:26:01.551 INFO:teuthology.orchestra.run.smithi131.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-17T02:26:01.552 INFO:teuthology.orchestra.run.smithi131.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-17T02:26:01.553 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages will be upgraded: 2024-09-17T02:26:01.554 INFO:teuthology.orchestra.run.smithi131.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-17T02:26:01.573 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-17T02:26:01.591 DEBUG:teuthology.orchestra.run.smithi189:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T02:26:01.650 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-17T02:26:01.734 INFO:teuthology.orchestra.run.smithi131.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T02:26:01.734 INFO:teuthology.orchestra.run.smithi131.stdout:Need to get 113 MB of archives. 2024-09-17T02:26:01.735 INFO:teuthology.orchestra.run.smithi131.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-17T02:26:01.735 INFO:teuthology.orchestra.run.smithi131.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-17T02:26:01.860 INFO:teuthology.orchestra.run.smithi189.stdout:Building dependency tree... 2024-09-17T02:26:01.861 INFO:teuthology.orchestra.run.smithi189.stdout:Reading state information... 2024-09-17T02:26:02.043 INFO:teuthology.orchestra.run.smithi189.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T02:26:02.043 INFO:teuthology.orchestra.run.smithi189.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T02:26:02.043 INFO:teuthology.orchestra.run.smithi189.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T02:26:02.044 INFO:teuthology.orchestra.run.smithi189.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T02:26:02.044 INFO:teuthology.orchestra.run.smithi189.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-17T02:26:02.044 INFO:teuthology.orchestra.run.smithi189.stdout: python2.7-minimal 2024-09-17T02:26:02.044 INFO:teuthology.orchestra.run.smithi189.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T02:26:02.045 INFO:teuthology.orchestra.run.smithi189.stdout:The following additional packages will be installed: 2024-09-17T02:26:02.045 INFO:teuthology.orchestra.run.smithi189.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T02:26:02.045 INFO:teuthology.orchestra.run.smithi189.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-17T02:26:02.045 INFO:teuthology.orchestra.run.smithi189.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-17T02:26:02.047 INFO:teuthology.orchestra.run.smithi189.stdout:Suggested packages: 2024-09-17T02:26:02.047 INFO:teuthology.orchestra.run.smithi189.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-17T02:26:02.047 INFO:teuthology.orchestra.run.smithi189.stdout:Recommended packages: 2024-09-17T02:26:02.047 INFO:teuthology.orchestra.run.smithi189.stdout: thermald 2024-09-17T02:26:02.079 INFO:teuthology.orchestra.run.smithi189.stdout:The following NEW packages will be installed: 2024-09-17T02:26:02.080 INFO:teuthology.orchestra.run.smithi189.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-17T02:26:02.080 INFO:teuthology.orchestra.run.smithi189.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-17T02:26:02.080 INFO:teuthology.orchestra.run.smithi189.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-17T02:26:02.081 INFO:teuthology.orchestra.run.smithi189.stdout:The following packages will be upgraded: 2024-09-17T02:26:02.081 INFO:teuthology.orchestra.run.smithi189.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-17T02:26:02.260 INFO:teuthology.orchestra.run.smithi189.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T02:26:02.260 INFO:teuthology.orchestra.run.smithi189.stdout:Need to get 113 MB of archives. 2024-09-17T02:26:02.260 INFO:teuthology.orchestra.run.smithi189.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-17T02:26:02.260 INFO:teuthology.orchestra.run.smithi189.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-17T02:26:03.122 INFO:teuthology.orchestra.run.smithi131.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-17T02:26:03.470 INFO:teuthology.orchestra.run.smithi131.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-17T02:26:03.662 INFO:teuthology.orchestra.run.smithi189.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-17T02:26:04.012 INFO:teuthology.orchestra.run.smithi189.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-17T02:26:05.666 INFO:teuthology.orchestra.run.smithi131.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-17T02:26:05.666 INFO:teuthology.orchestra.run.smithi131.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-17T02:26:05.667 INFO:teuthology.orchestra.run.smithi131.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-17T02:26:06.082 INFO:teuthology.orchestra.run.smithi131.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-17T02:26:06.135 INFO:teuthology.orchestra.run.smithi131.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-17T02:26:06.232 INFO:teuthology.orchestra.run.smithi189.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-17T02:26:06.232 INFO:teuthology.orchestra.run.smithi189.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-17T02:26:06.233 INFO:teuthology.orchestra.run.smithi189.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-17T02:26:06.538 INFO:teuthology.orchestra.run.smithi131.stdout:Fetched 113 MB in 5s (24.8 MB/s) 2024-09-17T02:26:06.579 INFO:teuthology.orchestra.run.smithi189.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-17T02:26:06.657 INFO:teuthology.orchestra.run.smithi189.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-17T02:26:06.700 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-17T02:26:07.098 INFO:teuthology.orchestra.run.smithi189.stdout:Fetched 113 MB in 5s (24.8 MB/s) 2024-09-17T02:26:07.298 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-17T02:26:08.666 INFO:teuthology.orchestra.run.smithi131.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-17T02:26:08.670 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T02:26:08.731 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:09.312 INFO:teuthology.orchestra.run.smithi189.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-17T02:26:09.316 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T02:26:09.396 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:10.708 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-17T02:26:10.730 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T02:26:10.814 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:11.244 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-17T02:26:11.265 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T02:26:11.303 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:11.588 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-17T02:26:11.609 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T02:26:11.716 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:12.233 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-17T02:26:12.256 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T02:26:12.291 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:17.631 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T02:26:17.740 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T02:26:18.039 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T02:26:18.152 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T02:26:18.413 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-17T02:26:18.424 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-17T02:26:18.463 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T02:26:19.202 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T02:26:19.321 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T02:26:19.603 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T02:26:19.699 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T02:26:19.994 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-17T02:26:20.016 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-17T02:26:20.060 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T02:26:23.715 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-17T02:26:23.752 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T02:26:23.789 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:25.531 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-17T02:26:25.567 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-17T02:26:25.605 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:26.478 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T02:26:26.627 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T02:26:26.952 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T02:26:27.090 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:27.217 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-17T02:26:27.345 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:28.264 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-17T02:26:28.729 INFO:teuthology.orchestra.run.smithi131.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-17T02:26:28.729 INFO:teuthology.orchestra.run.smithi131.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-17T02:26:28.730 INFO:teuthology.orchestra.run.smithi131.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-17T02:26:28.730 INFO:teuthology.orchestra.run.smithi131.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-17T02:26:29.252 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-17T02:26:29.281 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:29.602 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-17T02:26:29.723 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:29.849 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-17T02:26:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:30.691 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-17T02:26:30.817 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-17T02:26:30.943 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:31.343 INFO:teuthology.orchestra.run.smithi189.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-17T02:26:31.343 INFO:teuthology.orchestra.run.smithi189.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-17T02:26:31.343 INFO:teuthology.orchestra.run.smithi189.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-17T02:26:31.343 INFO:teuthology.orchestra.run.smithi189.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-17T02:26:31.496 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:32.353 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:32.417 INFO:teuthology.orchestra.run.smithi131.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-17T02:26:32.417 INFO:teuthology.orchestra.run.smithi131.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-17T02:26:32.939 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-17T02:26:33.064 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-17T02:26:33.564 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:35.434 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-17T02:26:35.479 INFO:teuthology.orchestra.run.smithi189.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-17T02:26:35.480 INFO:teuthology.orchestra.run.smithi189.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-17T02:26:46.326 INFO:teuthology.orchestra.run.smithi131.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-17T02:26:46.326 INFO:teuthology.orchestra.run.smithi131.stdout:Sourcing file `/etc/default/grub' 2024-09-17T02:26:46.346 INFO:teuthology.orchestra.run.smithi131.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T02:26:46.370 INFO:teuthology.orchestra.run.smithi131.stdout:Generating grub configuration file ... 2024-09-17T02:26:46.623 INFO:teuthology.orchestra.run.smithi131.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T02:26:46.640 INFO:teuthology.orchestra.run.smithi131.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T02:26:46.919 INFO:teuthology.orchestra.run.smithi131.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T02:26:46.924 INFO:teuthology.orchestra.run.smithi131.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T02:26:46.998 INFO:teuthology.orchestra.run.smithi131.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T02:26:47.003 INFO:teuthology.orchestra.run.smithi131.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T02:26:47.153 INFO:teuthology.orchestra.run.smithi131.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T02:26:47.153 INFO:teuthology.orchestra.run.smithi131.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T02:26:47.153 INFO:teuthology.orchestra.run.smithi131.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T02:26:47.196 INFO:teuthology.orchestra.run.smithi131.stdout:done 2024-09-17T02:26:47.692 DEBUG:teuthology.orchestra.run.smithi131:> dpkg -s linux-image-generic 2024-09-17T02:26:47.712 INFO:teuthology.orchestra.run.smithi131.stdout:Package: linux-image-generic 2024-09-17T02:26:47.712 INFO:teuthology.orchestra.run.smithi131.stdout:Status: install ok installed 2024-09-17T02:26:47.712 INFO:teuthology.orchestra.run.smithi131.stdout:Priority: optional 2024-09-17T02:26:47.712 INFO:teuthology.orchestra.run.smithi131.stdout:Section: kernel 2024-09-17T02:26:47.713 INFO:teuthology.orchestra.run.smithi131.stdout:Installed-Size: 21 2024-09-17T02:26:47.713 INFO:teuthology.orchestra.run.smithi131.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T02:26:47.713 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: amd64 2024-09-17T02:26:47.713 INFO:teuthology.orchestra.run.smithi131.stdout:Source: linux-meta 2024-09-17T02:26:47.713 INFO:teuthology.orchestra.run.smithi131.stdout:Version: 5.15.0.122.122 2024-09-17T02:26:47.713 INFO:teuthology.orchestra.run.smithi131.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T02:26:47.713 INFO:teuthology.orchestra.run.smithi131.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T02:26:47.713 INFO:teuthology.orchestra.run.smithi131.stdout:Recommends: thermald 2024-09-17T02:26:47.713 INFO:teuthology.orchestra.run.smithi131.stdout:Description: Generic Linux kernel image 2024-09-17T02:26:47.713 INFO:teuthology.orchestra.run.smithi131.stdout: This package will always depend on the latest generic kernel image 2024-09-17T02:26:47.713 INFO:teuthology.orchestra.run.smithi131.stdout: available. 2024-09-17T02:26:47.714 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-17T02:26:47.714 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T02:26:47.714 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T02:26:47.714 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T02:26:47.714 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T02:26:47.714 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-get clean 2024-09-17T02:26:47.892 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-get update 2024-09-17T02:26:48.038 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T02:26:48.047 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T02:26:48.065 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T02:26:48.139 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T02:26:49.420 INFO:teuthology.orchestra.run.smithi189.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-17T02:26:49.420 INFO:teuthology.orchestra.run.smithi189.stdout:Sourcing file `/etc/default/grub' 2024-09-17T02:26:49.435 INFO:teuthology.orchestra.run.smithi189.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T02:26:49.453 INFO:teuthology.orchestra.run.smithi189.stdout:Generating grub configuration file ... 2024-09-17T02:26:49.469 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2024-09-17T02:26:49.488 DEBUG:teuthology.orchestra.run.smithi131:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T02:26:49.548 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2024-09-17T02:26:49.743 INFO:teuthology.orchestra.run.smithi189.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T02:26:49.760 INFO:teuthology.orchestra.run.smithi131.stdout:Building dependency tree... 2024-09-17T02:26:49.761 INFO:teuthology.orchestra.run.smithi131.stdout:Reading state information... 2024-09-17T02:26:49.770 INFO:teuthology.orchestra.run.smithi189.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T02:26:49.937 INFO:teuthology.orchestra.run.smithi131.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-17T02:26:49.937 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T02:26:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T02:26:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T02:26:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T02:26:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T02:26:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T02:26:49.939 INFO:teuthology.orchestra.run.smithi131.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T02:26:49.979 INFO:teuthology.orchestra.run.smithi131.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T02:26:49.981 DEBUG:teuthology.orchestra.run.smithi131:> dpkg -s linux-image-generic 2024-09-17T02:26:50.000 INFO:teuthology.orchestra.run.smithi131.stdout:Package: linux-image-generic 2024-09-17T02:26:50.000 INFO:teuthology.orchestra.run.smithi131.stdout:Status: install ok installed 2024-09-17T02:26:50.000 INFO:teuthology.orchestra.run.smithi131.stdout:Priority: optional 2024-09-17T02:26:50.000 INFO:teuthology.orchestra.run.smithi131.stdout:Section: kernel 2024-09-17T02:26:50.000 INFO:teuthology.orchestra.run.smithi131.stdout:Installed-Size: 21 2024-09-17T02:26:50.000 INFO:teuthology.orchestra.run.smithi131.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T02:26:50.000 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: amd64 2024-09-17T02:26:50.000 INFO:teuthology.orchestra.run.smithi131.stdout:Source: linux-meta 2024-09-17T02:26:50.000 INFO:teuthology.orchestra.run.smithi131.stdout:Version: 5.15.0.122.122 2024-09-17T02:26:50.000 INFO:teuthology.orchestra.run.smithi131.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T02:26:50.000 INFO:teuthology.orchestra.run.smithi131.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T02:26:50.001 INFO:teuthology.orchestra.run.smithi131.stdout:Recommends: thermald 2024-09-17T02:26:50.001 INFO:teuthology.orchestra.run.smithi131.stdout:Description: Generic Linux kernel image 2024-09-17T02:26:50.001 INFO:teuthology.orchestra.run.smithi131.stdout: This package will always depend on the latest generic kernel image 2024-09-17T02:26:50.001 INFO:teuthology.orchestra.run.smithi131.stdout: available. 2024-09-17T02:26:50.001 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-09-17T02:26:50.050 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.zutj1Ho4GU 2024-09-17T02:26:50.050 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub/grub.cfg /tmp/tmp.zutj1Ho4GU 2024-09-17T02:26:50.073 INFO:teuthology.orchestra.run.smithi189.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T02:26:50.079 INFO:teuthology.orchestra.run.smithi189.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T02:26:50.106 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.zutj1Ho4GU 2024-09-17T02:26:50.144 INFO:teuthology.orchestra.run.smithi189.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T02:26:50.148 INFO:teuthology.orchestra.run.smithi189.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T02:26:50.225 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.zutj1Ho4GU is 10KB 2024-09-17T02:26:50.238 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.zutj1Ho4GU 2024-09-17T02:26:50.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-17T02:26:50.298 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:26:50.298 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-17T02:26:50.298 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-17T02:26:50.312 INFO:teuthology.orchestra.run.smithi189.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T02:26:50.313 INFO:teuthology.orchestra.run.smithi189.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T02:26:50.313 INFO:teuthology.orchestra.run.smithi189.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T02:26:50.351 INFO:teuthology.orchestra.run.smithi189.stdout:done 2024-09-17T02:26:50.366 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-17T02:26:50.367 DEBUG:teuthology.orchestra.run.smithi131:> sudo update-grub 2024-09-17T02:26:50.913 DEBUG:teuthology.orchestra.run.smithi189:> dpkg -s linux-image-generic 2024-09-17T02:26:50.940 INFO:teuthology.orchestra.run.smithi189.stdout:Package: linux-image-generic 2024-09-17T02:26:50.940 INFO:teuthology.orchestra.run.smithi189.stdout:Status: install ok installed 2024-09-17T02:26:50.940 INFO:teuthology.orchestra.run.smithi189.stdout:Priority: optional 2024-09-17T02:26:50.940 INFO:teuthology.orchestra.run.smithi189.stdout:Section: kernel 2024-09-17T02:26:50.940 INFO:teuthology.orchestra.run.smithi189.stdout:Installed-Size: 21 2024-09-17T02:26:50.940 INFO:teuthology.orchestra.run.smithi189.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T02:26:50.940 INFO:teuthology.orchestra.run.smithi189.stdout:Architecture: amd64 2024-09-17T02:26:50.940 INFO:teuthology.orchestra.run.smithi189.stdout:Source: linux-meta 2024-09-17T02:26:50.940 INFO:teuthology.orchestra.run.smithi189.stdout:Version: 5.15.0.122.122 2024-09-17T02:26:50.940 INFO:teuthology.orchestra.run.smithi189.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T02:26:50.940 INFO:teuthology.orchestra.run.smithi189.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T02:26:50.941 INFO:teuthology.orchestra.run.smithi189.stdout:Recommends: thermald 2024-09-17T02:26:50.941 INFO:teuthology.orchestra.run.smithi189.stdout:Description: Generic Linux kernel image 2024-09-17T02:26:50.941 INFO:teuthology.orchestra.run.smithi189.stdout: This package will always depend on the latest generic kernel image 2024-09-17T02:26:50.941 INFO:teuthology.orchestra.run.smithi189.stdout: available. 2024-09-17T02:26:50.942 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-17T02:26:50.942 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T02:26:50.942 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T02:26:50.942 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T02:26:50.942 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi189.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T02:26:50.942 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-get clean 2024-09-17T02:26:51.129 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-get update 2024-09-17T02:26:51.245 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T02:26:51.366 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T02:26:51.450 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T02:26:51.528 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T02:26:51.529 INFO:teuthology.orchestra.run.smithi131.stderr:Sourcing file `/etc/default/grub' 2024-09-17T02:26:51.531 INFO:teuthology.orchestra.run.smithi131.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T02:26:51.533 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2024-09-17T02:26:51.716 INFO:teuthology.orchestra.run.smithi131.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T02:26:51.722 INFO:teuthology.orchestra.run.smithi131.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T02:26:52.025 INFO:teuthology.orchestra.run.smithi131.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T02:26:52.030 INFO:teuthology.orchestra.run.smithi131.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T02:26:52.071 INFO:teuthology.orchestra.run.smithi131.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T02:26:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T02:26:52.199 INFO:teuthology.orchestra.run.smithi131.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T02:26:52.200 INFO:teuthology.orchestra.run.smithi131.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T02:26:52.200 INFO:teuthology.orchestra.run.smithi131.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T02:26:52.214 INFO:teuthology.orchestra.run.smithi131.stderr:done 2024-09-17T02:26:52.216 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2024-09-17T02:26:52.722 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-17T02:26:52.741 DEBUG:teuthology.orchestra.run.smithi189:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-17T02:26:52.794 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-17T02:26:52.983 INFO:teuthology.orchestra.run.smithi189.stdout:Building dependency tree... 2024-09-17T02:26:52.983 INFO:teuthology.orchestra.run.smithi189.stdout:Reading state information... 2024-09-17T02:26:53.155 INFO:teuthology.orchestra.run.smithi189.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-17T02:26:53.155 INFO:teuthology.orchestra.run.smithi189.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T02:26:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T02:26:53.155 INFO:teuthology.orchestra.run.smithi189.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T02:26:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T02:26:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T02:26:53.156 INFO:teuthology.orchestra.run.smithi189.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T02:26:53.156 INFO:teuthology.orchestra.run.smithi189.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T02:26:53.195 INFO:teuthology.orchestra.run.smithi189.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-17T02:26:53.197 DEBUG:teuthology.orchestra.run.smithi189:> dpkg -s linux-image-generic 2024-09-17T02:26:53.256 INFO:teuthology.orchestra.run.smithi189.stdout:Package: linux-image-generic 2024-09-17T02:26:53.256 INFO:teuthology.orchestra.run.smithi189.stdout:Status: install ok installed 2024-09-17T02:26:53.256 INFO:teuthology.orchestra.run.smithi189.stdout:Priority: optional 2024-09-17T02:26:53.256 INFO:teuthology.orchestra.run.smithi189.stdout:Section: kernel 2024-09-17T02:26:53.256 INFO:teuthology.orchestra.run.smithi189.stdout:Installed-Size: 21 2024-09-17T02:26:53.256 INFO:teuthology.orchestra.run.smithi189.stdout:Maintainer: Ubuntu Kernel Team 2024-09-17T02:26:53.256 INFO:teuthology.orchestra.run.smithi189.stdout:Architecture: amd64 2024-09-17T02:26:53.256 INFO:teuthology.orchestra.run.smithi189.stdout:Source: linux-meta 2024-09-17T02:26:53.256 INFO:teuthology.orchestra.run.smithi189.stdout:Version: 5.15.0.122.122 2024-09-17T02:26:53.256 INFO:teuthology.orchestra.run.smithi189.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-17T02:26:53.257 INFO:teuthology.orchestra.run.smithi189.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-17T02:26:53.257 INFO:teuthology.orchestra.run.smithi189.stdout:Recommends: thermald 2024-09-17T02:26:53.257 INFO:teuthology.orchestra.run.smithi189.stdout:Description: Generic Linux kernel image 2024-09-17T02:26:53.257 INFO:teuthology.orchestra.run.smithi189.stdout: This package will always depend on the latest generic kernel image 2024-09-17T02:26:53.257 INFO:teuthology.orchestra.run.smithi189.stdout: available. 2024-09-17T02:26:53.257 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-09-17T02:26:53.303 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.gcevz2TUfD 2024-09-17T02:26:53.304 DEBUG:teuthology.orchestra.run.smithi189:> sudo cp /boot/grub/grub.cfg /tmp/tmp.gcevz2TUfD 2024-09-17T02:26:53.358 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.gcevz2TUfD 2024-09-17T02:26:53.484 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.gcevz2TUfD is 10KB 2024-09-17T02:26:53.497 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.gcevz2TUfD 2024-09-17T02:26:53.502 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-17T02:26:53.555 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:26:53.555 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-17T02:26:53.555 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-17T02:26:53.621 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-17T02:26:53.622 DEBUG:teuthology.orchestra.run.smithi189:> sudo update-grub 2024-09-17T02:26:54.833 INFO:teuthology.orchestra.run.smithi189.stderr:Sourcing file `/etc/default/grub' 2024-09-17T02:26:54.834 INFO:teuthology.orchestra.run.smithi189.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-17T02:26:54.845 INFO:teuthology.orchestra.run.smithi189.stderr:Generating grub configuration file ... 2024-09-17T02:26:55.028 INFO:teuthology.orchestra.run.smithi189.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-17T02:26:55.033 INFO:teuthology.orchestra.run.smithi189.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-17T02:26:55.276 INFO:teuthology.orchestra.run.smithi189.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-17T02:26:55.281 INFO:teuthology.orchestra.run.smithi189.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-17T02:26:55.323 INFO:teuthology.orchestra.run.smithi189.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-17T02:26:55.328 INFO:teuthology.orchestra.run.smithi189.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-17T02:26:55.456 INFO:teuthology.orchestra.run.smithi189.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-17T02:26:55.456 INFO:teuthology.orchestra.run.smithi189.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-17T02:26:55.456 INFO:teuthology.orchestra.run.smithi189.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-17T02:26:55.488 INFO:teuthology.orchestra.run.smithi189.stderr:done 2024-09-17T02:26:55.494 DEBUG:teuthology.orchestra.run.smithi189:> sudo shutdown -r now 2024-09-17T02:27:22.245 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T02:27:22.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-17T02:27:22.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T02:27:25.497 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T02:27:25.497 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-17T02:27:25.498 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T02:27:40.737 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-09-17T02:27:44.061 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-17T02:27:49.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-17T02:27:49.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T02:27:52.801 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.131 2024-09-17T02:27:53.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-17T02:27:53.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T02:27:56.125 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.189 2024-09-17T02:28:04.809 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-17T02:28:04.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T02:28:08.129 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-17T02:28:08.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T02:28:36.642 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-09-17T02:28:37.135 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-17T02:28:37.135 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-17T02:28:37.136 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-09-17T02:28:37.181 INFO:teuthology.orchestra.run.smithi131.stdout:5.15.0-122-generic 2024-09-17T02:28:37.181 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-17T02:28:37.182 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T02:28:37.182 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T02:28:38.182 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T02:28:38.182 DEBUG:teuthology.orchestra.run.smithi131:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T02:28:38.256 INFO:teuthology.orchestra.run.smithi131.stdout:ttyS1 2024-09-17T02:28:38.283 DEBUG:teuthology.parallel:result is None 2024-09-17T02:29:08.133 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T02:29:23.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-17T02:29:23.136 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T02:29:23.435 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-17T02:29:23.987 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-17T02:29:23.987 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-17T02:29:23.987 DEBUG:teuthology.orchestra.run.smithi189:> uname -r 2024-09-17T02:29:24.034 INFO:teuthology.orchestra.run.smithi189.stdout:5.15.0-122-generic 2024-09-17T02:29:24.034 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-17T02:29:24.034 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T02:29:24.034 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-17T02:29:25.035 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T02:29:25.035 DEBUG:teuthology.orchestra.run.smithi189:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T02:29:25.403 INFO:teuthology.orchestra.run.smithi189.stdout:ttyS1 2024-09-17T02:29:25.424 DEBUG:teuthology.parallel:result is None 2024-09-17T02:29:25.424 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T02:29:25.432 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T02:29:25.432 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T02:29:25.435 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T02:29:25.440 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T02:29:25.455 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T02:29:25.467 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T02:29:25.467 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T02:29:25.482 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T02:29:25.562 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T02:29:25.569 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T02:29:25.569 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T02:29:25.571 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T02:29:25.589 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T02:29:25.590 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T02:29:25.596 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T02:29:25.598 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T02:29:25.600 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T02:29:25.606 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T02:29:25.606 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T02:29:25.641 DEBUG:teuthology.orchestra.run.smithi189:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T02:29:25.732 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T02:29:25.742 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T02:29:25.742 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T02:29:25.745 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T02:29:25.750 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T02:29:25.801 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T02:29:25.849 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:29:25.849 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T02:29:25.914 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T02:29:25.920 DEBUG:teuthology.orchestra.run.smithi189:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T02:29:25.967 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:29:25.967 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T02:29:26.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-09-17T02:29:26.034 DEBUG:teuthology.orchestra.run.smithi189:> sudo service rsyslog restart 2024-09-17T02:29:26.104 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T02:29:26.111 INFO:teuthology.task.internal:Starting timer... 2024-09-17T02:29:26.111 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T02:29:26.135 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T02:29:26.144 DEBUG:teuthology.task.selinux:Excluding smithi131: OS 'ubuntu' does not support SELinux 2024-09-17T02:29:26.144 DEBUG:teuthology.task.selinux:Excluding smithi189: OS 'ubuntu' does not support SELinux 2024-09-17T02:29:26.144 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T02:29:26.144 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-17T02:29:26.144 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T02:29:26.144 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T02:29:26.154 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T02:29:26.165 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T02:29:26.166 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com,smithi189.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T02:36:55.483 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi189.front.sepia.ceph.com')] 2024-09-17T02:36:55.484 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-17T02:36:55.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T02:36:55.571 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-09-17T02:36:55.649 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-17T02:36:55.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-17T02:36:55.650 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi189.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T02:36:55.730 DEBUG:teuthology.orchestra.run.smithi189:> true 2024-09-17T02:36:55.810 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi189.front.sepia.ceph.com' 2024-09-17T02:36:55.810 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T02:36:55.820 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T02:36:55.820 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T02:36:55.821 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T02:36:55.823 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T02:36:55.824 DEBUG:teuthology.orchestra.run.smithi189:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T02:36:55.852 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T02:36:55.852 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: Command line: ntpd -gq 2024-09-17T02:36:55.852 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: ---------------------------------------------------- 2024-09-17T02:36:55.852 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T02:36:55.852 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T02:36:55.852 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: corporation. Support and training for ntp-4 are 2024-09-17T02:36:55.852 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: available at https://www.nwtime.org/support 2024-09-17T02:36:55.852 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: ---------------------------------------------------- 2024-09-17T02:36:55.854 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: proto: precision = 0.045 usec (-24) 2024-09-17T02:36:55.854 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: basedate set to 2022-02-04 2024-09-17T02:36:55.854 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: gps base set to 2022-02-06 (week 2196) 2024-09-17T02:36:55.854 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T02:36:55.854 INFO:teuthology.orchestra.run.smithi131.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T02:36:55.854 INFO:teuthology.orchestra.run.smithi131.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T02:36:55.856 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T02:36:55.856 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T02:36:55.856 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T02:36:55.856 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T02:36:55.856 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: Listen normally on 3 enp3s0f1 172.21.15.131:123 2024-09-17T02:36:55.856 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: Listen normally on 4 lo [::1]:123 2024-09-17T02:36:55.856 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:72f9%5]:123 2024-09-17T02:36:55.856 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:36:55 ntpd[18468]: Listening on routing socket on fd #22 for interface updates 2024-09-17T02:36:55.877 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-17T02:36:55.877 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: Command line: ntpd -gq 2024-09-17T02:36:55.877 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: ---------------------------------------------------- 2024-09-17T02:36:55.877 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: ntp-4 is maintained by Network Time Foundation, 2024-09-17T02:36:55.877 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-17T02:36:55.877 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: corporation. Support and training for ntp-4 are 2024-09-17T02:36:55.877 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: available at https://www.nwtime.org/support 2024-09-17T02:36:55.877 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: ---------------------------------------------------- 2024-09-17T02:36:55.878 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: proto: precision = 0.059 usec (-24) 2024-09-17T02:36:55.878 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: basedate set to 2022-02-04 2024-09-17T02:36:55.878 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: gps base set to 2022-02-06 (week 2196) 2024-09-17T02:36:55.878 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T02:36:55.879 INFO:teuthology.orchestra.run.smithi189.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-17T02:36:55.879 INFO:teuthology.orchestra.run.smithi189.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-17T02:36:55.880 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: restrict ::: KOD does nothing without LIMITED. 2024-09-17T02:36:55.880 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: Listen and drop on 0 v6wildcard [::]:123 2024-09-17T02:36:55.880 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-17T02:36:55.880 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: Listen normally on 2 lo 127.0.0.1:123 2024-09-17T02:36:55.880 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: Listen normally on 3 enp3s0f1 172.21.15.189:123 2024-09-17T02:36:55.880 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: Listen normally on 4 lo [::1]:123 2024-09-17T02:36:55.880 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6df1%5]:123 2024-09-17T02:36:55.880 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:36:55 ntpd[18482]: Listening on routing socket on fd #22 for interface updates 2024-09-17T02:36:56.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:56 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:56.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:56 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:56.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:56 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:56.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:56 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:56.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:56 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:56.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:56 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:56.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:56 ntpd[18468]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:36:56.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:56 ntpd[18468]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:36:56.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:56.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:56.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:56.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:56.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:56.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:56.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:36:56.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:36:56.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:56.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:56.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:36:56.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:56 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:36:58.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:58 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:58.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:58 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:58.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:58 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:58.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:58 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:58.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:58 ntpd[18468]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:36:58.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:58 ntpd[18468]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:36:58.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:58.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:58.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:58.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:58.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:58.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:58.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:36:58.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:36:58.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:58.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:58.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:36:58.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:58 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:36:59.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:59.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:36:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:36:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:36:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:36:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:36:59 ntpd[18468]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:36:59.877 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:59 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:36:59.877 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:36:59 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:00 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:00 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:00 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:00 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:00 ntpd[18468]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:37:00.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:00 ntpd[18468]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:37:00.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:00.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:00.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:00.880 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:37:00.880 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:00 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:37:01.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:01.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:01.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:01.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:01.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:01.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:01.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:37:01.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:37:01.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:01.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:01.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:37:01.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:01 ntpd[18468]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:37:02.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:02 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:02.853 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:02 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:02 ntpd[18468]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:02 ntpd[18468]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:02 ntpd[18468]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:37:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:02 ntpd[18468]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:37:02.855 INFO:teuthology.orchestra.run.smithi131.stdout:17 Sep 02:37:02 ntpd[18468]: ntpd: time slew +0.002507 s 2024-09-17T02:37:02.855 INFO:teuthology.orchestra.run.smithi131.stdout:ntpd: time slew +0.002507s 2024-09-17T02:37:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:02 ntpd[18468]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T02:37:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:17 Sep 02:37:02 ntpd[18468]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T02:37:02.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:02.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:02.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:02.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:02.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:02.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:02.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-17T02:37:02.878 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: can't open /var/log/ntpstats/rawstats.20240917: Permission denied 2024-09-17T02:37:02.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-17T02:37:02.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: can't open /var/log/ntpstats/peerstats.20240917: Permission denied 2024-09-17T02:37:02.879 INFO:teuthology.orchestra.run.smithi189.stdout:17 Sep 02:37:02 ntpd[18482]: ntpd: time slew -0.000234 s 2024-09-17T02:37:02.879 INFO:teuthology.orchestra.run.smithi189.stdout:ntpd: time slew -0.000234s 2024-09-17T02:37:02.879 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-17T02:37:02.880 INFO:teuthology.orchestra.run.smithi189.stderr:17 Sep 02:37:02 ntpd[18482]: can't open /var/log/ntpstats/loopstats.20240917: Permission denied 2024-09-17T02:37:02.934 INFO:teuthology.orchestra.run.smithi189.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T02:37:02.935 INFO:teuthology.orchestra.run.smithi189.stdout:============================================================================== 2024-09-17T02:37:02.935 INFO:teuthology.orchestra.run.smithi189.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T02:37:02.935 INFO:teuthology.orchestra.run.smithi189.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T02:37:02.935 INFO:teuthology.orchestra.run.smithi189.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T02:37:02.935 INFO:teuthology.orchestra.run.smithi189.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T02:37:02.988 INFO:teuthology.orchestra.run.smithi131.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T02:37:02.988 INFO:teuthology.orchestra.run.smithi131.stdout:============================================================================== 2024-09-17T02:37:02.988 INFO:teuthology.orchestra.run.smithi131.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T02:37:02.988 INFO:teuthology.orchestra.run.smithi131.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T02:37:02.989 INFO:teuthology.orchestra.run.smithi131.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T02:37:02.989 INFO:teuthology.orchestra.run.smithi131.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-17T02:37:02.989 INFO:teuthology.run_tasks:Running task install... 2024-09-17T02:37:02.998 DEBUG:teuthology.task.install:project ceph 2024-09-17T02:37:02.998 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}} 2024-09-17T02:37:02.998 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'} 2024-09-17T02:37:02.998 INFO:teuthology.task.install:Using flavor: default 2024-09-17T02:37:03.004 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T02:37:03.004 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-17T02:37:03.005 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-key list | grep Ceph 2024-09-17T02:37:03.005 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-key list | grep Ceph 2024-09-17T02:37:03.061 INFO:teuthology.orchestra.run.smithi189.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T02:37:03.086 INFO:teuthology.orchestra.run.smithi189.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T02:37:03.087 INFO:teuthology.orchestra.run.smithi189.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T02:37:03.087 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-17T02:37:03.087 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-17T02:37:03.087 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T02:37:03.092 INFO:teuthology.orchestra.run.smithi131.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-17T02:37:03.122 INFO:teuthology.orchestra.run.smithi131.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-17T02:37:03.122 INFO:teuthology.orchestra.run.smithi131.stdout:uid [ unknown] Ceph.com (release key) 2024-09-17T02:37:03.122 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-17T02:37:03.122 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-17T02:37:03.123 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T02:37:03.235 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default/ 2024-09-17T02:37:03.235 INFO:teuthology.task.install.deb:Package version is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T02:37:03.274 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default/ 2024-09-17T02:37:03.274 INFO:teuthology.task.install.deb:Package version is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T02:37:03.357 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:37:03.357 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T02:37:03.371 DEBUG:teuthology.orchestra.run.smithi189:> sudo apt-get update 2024-09-17T02:37:03.383 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:37:03.384 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-17T02:37:03.398 DEBUG:teuthology.orchestra.run.smithi131:> sudo apt-get update 2024-09-17T02:37:03.567 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T02:37:03.575 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T02:37:03.593 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T02:37:03.595 INFO:teuthology.orchestra.run.smithi189.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T02:37:03.617 INFO:teuthology.orchestra.run.smithi189.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-17T02:37:03.619 INFO:teuthology.orchestra.run.smithi131.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy InRelease 2024-09-17T02:37:03.640 INFO:teuthology.orchestra.run.smithi131.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-17T02:37:03.641 INFO:teuthology.orchestra.run.smithi189.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T02:37:03.663 INFO:teuthology.orchestra.run.smithi131.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-17T02:37:03.671 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-17T02:37:03.671 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T02:37:03.671 INFO:teuthology.orchestra.run.smithi189.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-17T02:37:03.685 INFO:teuthology.orchestra.run.smithi131.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-17T02:37:03.690 INFO:teuthology.orchestra.run.smithi131.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-17T02:37:03.701 INFO:teuthology.orchestra.run.smithi189.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,420 B] 2024-09-17T02:37:03.712 INFO:teuthology.orchestra.run.smithi131.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-17T02:37:03.742 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-17T02:37:03.823 INFO:teuthology.orchestra.run.smithi189.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-17T02:37:03.926 INFO:teuthology.orchestra.run.smithi131.stdout:Fetched 29.2 kB in 0s (84.1 kB/s) 2024-09-17T02:37:04.047 INFO:teuthology.orchestra.run.smithi189.stdout:Fetched 29.2 kB in 0s (58.8 kB/s) 2024-09-17T02:37:04.960 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2024-09-17T02:37:04.978 DEBUG:teuthology.orchestra.run.smithi131:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4989-gce1b342b-1jammy cephadm=19.3.0-4989-gce1b342b-1jammy ceph-mds=19.3.0-4989-gce1b342b-1jammy ceph-mgr=19.3.0-4989-gce1b342b-1jammy ceph-common=19.3.0-4989-gce1b342b-1jammy ceph-fuse=19.3.0-4989-gce1b342b-1jammy ceph-test=19.3.0-4989-gce1b342b-1jammy ceph-volume=19.3.0-4989-gce1b342b-1jammy radosgw=19.3.0-4989-gce1b342b-1jammy python3-rados=19.3.0-4989-gce1b342b-1jammy python3-rgw=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy python3-rbd=19.3.0-4989-gce1b342b-1jammy libcephfs2=19.3.0-4989-gce1b342b-1jammy libcephfs-dev=19.3.0-4989-gce1b342b-1jammy librados2=19.3.0-4989-gce1b342b-1jammy librbd1=19.3.0-4989-gce1b342b-1jammy rbd-fuse=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy cephfs-shell=19.3.0-4989-gce1b342b-1jammy cephfs-top=19.3.0-4989-gce1b342b-1jammy cephfs-mirror=19.3.0-4989-gce1b342b-1jammy 2024-09-17T02:37:05.043 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2024-09-17T02:37:05.067 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-17T02:37:05.084 DEBUG:teuthology.orchestra.run.smithi189:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4989-gce1b342b-1jammy cephadm=19.3.0-4989-gce1b342b-1jammy ceph-mds=19.3.0-4989-gce1b342b-1jammy ceph-mgr=19.3.0-4989-gce1b342b-1jammy ceph-common=19.3.0-4989-gce1b342b-1jammy ceph-fuse=19.3.0-4989-gce1b342b-1jammy ceph-test=19.3.0-4989-gce1b342b-1jammy ceph-volume=19.3.0-4989-gce1b342b-1jammy radosgw=19.3.0-4989-gce1b342b-1jammy python3-rados=19.3.0-4989-gce1b342b-1jammy python3-rgw=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy python3-rbd=19.3.0-4989-gce1b342b-1jammy libcephfs2=19.3.0-4989-gce1b342b-1jammy libcephfs-dev=19.3.0-4989-gce1b342b-1jammy librados2=19.3.0-4989-gce1b342b-1jammy librbd1=19.3.0-4989-gce1b342b-1jammy rbd-fuse=19.3.0-4989-gce1b342b-1jammy python3-cephfs=19.3.0-4989-gce1b342b-1jammy cephfs-shell=19.3.0-4989-gce1b342b-1jammy cephfs-top=19.3.0-4989-gce1b342b-1jammy cephfs-mirror=19.3.0-4989-gce1b342b-1jammy 2024-09-17T02:37:05.150 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-17T02:37:05.254 INFO:teuthology.orchestra.run.smithi131.stdout:Building dependency tree... 2024-09-17T02:37:05.255 INFO:teuthology.orchestra.run.smithi131.stdout:Reading state information... 2024-09-17T02:37:05.361 INFO:teuthology.orchestra.run.smithi189.stdout:Building dependency tree... 2024-09-17T02:37:05.362 INFO:teuthology.orchestra.run.smithi189.stdout:Reading state information... 2024-09-17T02:37:05.452 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T02:37:05.452 INFO:teuthology.orchestra.run.smithi131.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T02:37:05.452 INFO:teuthology.orchestra.run.smithi131.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T02:37:05.452 INFO:teuthology.orchestra.run.smithi131.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T02:37:05.452 INFO:teuthology.orchestra.run.smithi131.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T02:37:05.453 INFO:teuthology.orchestra.run.smithi131.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T02:37:05.453 INFO:teuthology.orchestra.run.smithi131.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T02:37:05.453 INFO:teuthology.orchestra.run.smithi131.stdout:The following additional packages will be installed: 2024-09-17T02:37:05.454 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T02:37:05.454 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T02:37:05.454 INFO:teuthology.orchestra.run.smithi131.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T02:37:05.454 INFO:teuthology.orchestra.run.smithi131.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T02:37:05.454 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T02:37:05.455 INFO:teuthology.orchestra.run.smithi131.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T02:37:05.455 INFO:teuthology.orchestra.run.smithi131.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T02:37:05.455 INFO:teuthology.orchestra.run.smithi131.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T02:37:05.455 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T02:37:05.455 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-17T02:37:05.455 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-17T02:37:05.455 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-17T02:37:05.455 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-17T02:37:05.456 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-17T02:37:05.456 INFO:teuthology.orchestra.run.smithi131.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-17T02:37:05.456 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-17T02:37:05.456 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-17T02:37:05.456 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-17T02:37:05.456 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-17T02:37:05.456 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-17T02:37:05.456 INFO:teuthology.orchestra.run.smithi131.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-17T02:37:05.456 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-17T02:37:05.456 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-17T02:37:05.456 INFO:teuthology.orchestra.run.smithi131.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-17T02:37:05.457 INFO:teuthology.orchestra.run.smithi131.stdout:Suggested packages: 2024-09-17T02:37:05.457 INFO:teuthology.orchestra.run.smithi131.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T02:37:05.457 INFO:teuthology.orchestra.run.smithi131.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T02:37:05.457 INFO:teuthology.orchestra.run.smithi131.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T02:37:05.457 INFO:teuthology.orchestra.run.smithi131.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T02:37:05.457 INFO:teuthology.orchestra.run.smithi131.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T02:37:05.458 INFO:teuthology.orchestra.run.smithi131.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T02:37:05.458 INFO:teuthology.orchestra.run.smithi131.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T02:37:05.458 INFO:teuthology.orchestra.run.smithi131.stdout:Recommended packages: 2024-09-17T02:37:05.458 INFO:teuthology.orchestra.run.smithi131.stdout: btrfs-tools 2024-09-17T02:37:05.565 INFO:teuthology.orchestra.run.smithi131.stdout:The following NEW packages will be installed: 2024-09-17T02:37:05.565 INFO:teuthology.orchestra.run.smithi131.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T02:37:05.565 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T02:37:05.565 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-17T02:37:05.565 INFO:teuthology.orchestra.run.smithi131.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-17T02:37:05.565 INFO:teuthology.orchestra.run.smithi131.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T02:37:05.566 INFO:teuthology.orchestra.run.smithi131.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-17T02:37:05.566 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-17T02:37:05.568 INFO:teuthology.orchestra.run.smithi131.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-17T02:37:05.568 INFO:teuthology.orchestra.run.smithi131.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T02:37:05.569 INFO:teuthology.orchestra.run.smithi131.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T02:37:05.569 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-17T02:37:05.569 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-17T02:37:05.569 INFO:teuthology.orchestra.run.smithi131.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-17T02:37:05.569 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T02:37:05.569 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T02:37:05.569 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T02:37:05.569 INFO:teuthology.orchestra.run.smithi131.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T02:37:05.569 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T02:37:05.569 INFO:teuthology.orchestra.run.smithi131.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T02:37:05.570 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-17T02:37:05.570 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-17T02:37:05.570 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-17T02:37:05.570 INFO:teuthology.orchestra.run.smithi131.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T02:37:05.570 INFO:teuthology.orchestra.run.smithi131.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T02:37:05.570 INFO:teuthology.orchestra.run.smithi131.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T02:37:05.570 INFO:teuthology.orchestra.run.smithi131.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T02:37:05.570 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-17T02:37:05.570 INFO:teuthology.orchestra.run.smithi131.stdout: socat unzip xmlstarlet zip 2024-09-17T02:37:05.571 INFO:teuthology.orchestra.run.smithi189.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T02:37:05.571 INFO:teuthology.orchestra.run.smithi189.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T02:37:05.571 INFO:teuthology.orchestra.run.smithi189.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T02:37:05.572 INFO:teuthology.orchestra.run.smithi189.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T02:37:05.572 INFO:teuthology.orchestra.run.smithi189.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T02:37:05.572 INFO:teuthology.orchestra.run.smithi189.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T02:37:05.572 INFO:teuthology.orchestra.run.smithi189.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T02:37:05.573 INFO:teuthology.orchestra.run.smithi189.stdout:The following additional packages will be installed: 2024-09-17T02:37:05.573 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-17T02:37:05.573 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-17T02:37:05.574 INFO:teuthology.orchestra.run.smithi189.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T02:37:05.574 INFO:teuthology.orchestra.run.smithi189.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-17T02:37:05.574 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-17T02:37:05.575 INFO:teuthology.orchestra.run.smithi189.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-17T02:37:05.575 INFO:teuthology.orchestra.run.smithi189.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T02:37:05.575 INFO:teuthology.orchestra.run.smithi189.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T02:37:05.575 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-17T02:37:05.575 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-17T02:37:05.575 INFO:teuthology.orchestra.run.smithi189.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-17T02:37:05.575 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-17T02:37:05.575 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-17T02:37:05.576 INFO:teuthology.orchestra.run.smithi189.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-17T02:37:05.576 INFO:teuthology.orchestra.run.smithi189.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-17T02:37:05.576 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-17T02:37:05.576 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-17T02:37:05.576 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-17T02:37:05.576 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-17T02:37:05.576 INFO:teuthology.orchestra.run.smithi189.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-17T02:37:05.576 INFO:teuthology.orchestra.run.smithi189.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-17T02:37:05.576 INFO:teuthology.orchestra.run.smithi189.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-17T02:37:05.576 INFO:teuthology.orchestra.run.smithi189.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-17T02:37:05.576 INFO:teuthology.orchestra.run.smithi189.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-17T02:37:05.577 INFO:teuthology.orchestra.run.smithi189.stdout:Suggested packages: 2024-09-17T02:37:05.577 INFO:teuthology.orchestra.run.smithi189.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-17T02:37:05.577 INFO:teuthology.orchestra.run.smithi189.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-17T02:37:05.577 INFO:teuthology.orchestra.run.smithi189.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-17T02:37:05.577 INFO:teuthology.orchestra.run.smithi189.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-17T02:37:05.577 INFO:teuthology.orchestra.run.smithi189.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-17T02:37:05.577 INFO:teuthology.orchestra.run.smithi189.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-17T02:37:05.577 INFO:teuthology.orchestra.run.smithi189.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-17T02:37:05.577 INFO:teuthology.orchestra.run.smithi189.stdout:Recommended packages: 2024-09-17T02:37:05.577 INFO:teuthology.orchestra.run.smithi189.stdout: btrfs-tools 2024-09-17T02:37:05.628 INFO:teuthology.orchestra.run.smithi131.stdout:0 upgraded, 125 newly installed, 0 to remove and 331 not upgraded. 2024-09-17T02:37:05.628 INFO:teuthology.orchestra.run.smithi131.stdout:Need to get 239 MB of archives. 2024-09-17T02:37:05.628 INFO:teuthology.orchestra.run.smithi131.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-17T02:37:05.629 INFO:teuthology.orchestra.run.smithi131.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-17T02:37:05.646 INFO:teuthology.orchestra.run.smithi131.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T02:37:05.656 INFO:teuthology.orchestra.run.smithi131.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-17T02:37:05.685 INFO:teuthology.orchestra.run.smithi189.stdout:The following NEW packages will be installed: 2024-09-17T02:37:05.685 INFO:teuthology.orchestra.run.smithi189.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-17T02:37:05.685 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-17T02:37:05.685 INFO:teuthology.orchestra.run.smithi189.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-17T02:37:05.685 INFO:teuthology.orchestra.run.smithi189.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-17T02:37:05.686 INFO:teuthology.orchestra.run.smithi189.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-17T02:37:05.686 INFO:teuthology.orchestra.run.smithi189.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-17T02:37:05.686 INFO:teuthology.orchestra.run.smithi189.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-17T02:37:05.687 INFO:teuthology.orchestra.run.smithi189.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-17T02:37:05.688 INFO:teuthology.orchestra.run.smithi189.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-17T02:37:05.688 INFO:teuthology.orchestra.run.smithi189.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-17T02:37:05.688 INFO:teuthology.orchestra.run.smithi189.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-17T02:37:05.688 INFO:teuthology.orchestra.run.smithi189.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-17T02:37:05.688 INFO:teuthology.orchestra.run.smithi189.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-17T02:37:05.688 INFO:teuthology.orchestra.run.smithi189.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-17T02:37:05.688 INFO:teuthology.orchestra.run.smithi189.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-17T02:37:05.688 INFO:teuthology.orchestra.run.smithi189.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-17T02:37:05.688 INFO:teuthology.orchestra.run.smithi189.stdout: socat unzip xmlstarlet zip 2024-09-17T02:37:05.696 INFO:teuthology.orchestra.run.smithi131.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T02:37:05.711 INFO:teuthology.orchestra.run.smithi131.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4989-gce1b342b-1jammy [3,591 kB] 2024-09-17T02:37:05.717 INFO:teuthology.orchestra.run.smithi131.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T02:37:05.720 INFO:teuthology.orchestra.run.smithi131.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T02:37:05.731 INFO:teuthology.orchestra.run.smithi131.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T02:37:05.744 INFO:teuthology.orchestra.run.smithi189.stdout:0 upgraded, 125 newly installed, 0 to remove and 331 not upgraded. 2024-09-17T02:37:05.744 INFO:teuthology.orchestra.run.smithi189.stdout:Need to get 239 MB of archives. 2024-09-17T02:37:05.744 INFO:teuthology.orchestra.run.smithi189.stdout:After this operation, 986 MB of additional disk space will be used. 2024-09-17T02:37:05.744 INFO:teuthology.orchestra.run.smithi189.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-17T02:37:05.762 INFO:teuthology.orchestra.run.smithi189.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-17T02:37:05.772 INFO:teuthology.orchestra.run.smithi189.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-17T02:37:05.778 INFO:teuthology.orchestra.run.smithi131.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T02:37:05.782 INFO:teuthology.orchestra.run.smithi131.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T02:37:05.793 INFO:teuthology.orchestra.run.smithi131.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T02:37:05.797 INFO:teuthology.orchestra.run.smithi131.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T02:37:05.798 INFO:teuthology.orchestra.run.smithi131.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T02:37:05.799 INFO:teuthology.orchestra.run.smithi131.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T02:37:05.807 INFO:teuthology.orchestra.run.smithi131.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T02:37:05.809 INFO:teuthology.orchestra.run.smithi131.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T02:37:05.811 INFO:teuthology.orchestra.run.smithi131.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T02:37:05.812 INFO:teuthology.orchestra.run.smithi131.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T02:37:05.813 INFO:teuthology.orchestra.run.smithi189.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-17T02:37:05.816 INFO:teuthology.orchestra.run.smithi131.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T02:37:05.818 INFO:teuthology.orchestra.run.smithi131.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T02:37:05.820 INFO:teuthology.orchestra.run.smithi131.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T02:37:05.826 INFO:teuthology.orchestra.run.smithi189.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4989-gce1b342b-1jammy [3,591 kB] 2024-09-17T02:37:05.833 INFO:teuthology.orchestra.run.smithi131.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T02:37:05.834 INFO:teuthology.orchestra.run.smithi131.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T02:37:05.834 INFO:teuthology.orchestra.run.smithi131.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T02:37:05.834 INFO:teuthology.orchestra.run.smithi189.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-17T02:37:05.835 INFO:teuthology.orchestra.run.smithi131.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T02:37:05.835 INFO:teuthology.orchestra.run.smithi131.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T02:37:05.835 INFO:teuthology.orchestra.run.smithi131.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T02:37:05.836 INFO:teuthology.orchestra.run.smithi131.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T02:37:05.838 INFO:teuthology.orchestra.run.smithi189.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-17T02:37:05.849 INFO:teuthology.orchestra.run.smithi189.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-17T02:37:05.850 INFO:teuthology.orchestra.run.smithi131.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T02:37:05.850 INFO:teuthology.orchestra.run.smithi131.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T02:37:05.850 INFO:teuthology.orchestra.run.smithi131.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T02:37:05.867 INFO:teuthology.orchestra.run.smithi131.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T02:37:05.867 INFO:teuthology.orchestra.run.smithi131.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T02:37:05.868 INFO:teuthology.orchestra.run.smithi131.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T02:37:05.868 INFO:teuthology.orchestra.run.smithi131.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T02:37:05.869 INFO:teuthology.orchestra.run.smithi131.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T02:37:05.869 INFO:teuthology.orchestra.run.smithi131.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T02:37:05.870 INFO:teuthology.orchestra.run.smithi131.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T02:37:05.884 INFO:teuthology.orchestra.run.smithi131.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T02:37:05.885 INFO:teuthology.orchestra.run.smithi131.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T02:37:05.885 INFO:teuthology.orchestra.run.smithi131.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T02:37:05.893 INFO:teuthology.orchestra.run.smithi131.stdout:Get:42 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4989-gce1b342b-1jammy [3,368 kB] 2024-09-17T02:37:05.897 INFO:teuthology.orchestra.run.smithi189.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-17T02:37:05.901 INFO:teuthology.orchestra.run.smithi131.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T02:37:05.901 INFO:teuthology.orchestra.run.smithi189.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-17T02:37:05.902 INFO:teuthology.orchestra.run.smithi131.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T02:37:05.902 INFO:teuthology.orchestra.run.smithi131.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T02:37:05.902 INFO:teuthology.orchestra.run.smithi131.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T02:37:05.903 INFO:teuthology.orchestra.run.smithi131.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T02:37:05.905 INFO:teuthology.orchestra.run.smithi131.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T02:37:05.906 INFO:teuthology.orchestra.run.smithi131.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T02:37:05.912 INFO:teuthology.orchestra.run.smithi189.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-17T02:37:05.915 INFO:teuthology.orchestra.run.smithi189.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-17T02:37:05.917 INFO:teuthology.orchestra.run.smithi189.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-17T02:37:05.917 INFO:teuthology.orchestra.run.smithi189.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-17T02:37:05.918 INFO:teuthology.orchestra.run.smithi131.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T02:37:05.918 INFO:teuthology.orchestra.run.smithi131.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T02:37:05.921 INFO:teuthology.orchestra.run.smithi131.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T02:37:05.925 INFO:teuthology.orchestra.run.smithi189.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-17T02:37:05.927 INFO:teuthology.orchestra.run.smithi189.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-17T02:37:05.929 INFO:teuthology.orchestra.run.smithi189.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-17T02:37:05.930 INFO:teuthology.orchestra.run.smithi189.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-17T02:37:05.934 INFO:teuthology.orchestra.run.smithi131.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T02:37:05.935 INFO:teuthology.orchestra.run.smithi131.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T02:37:05.936 INFO:teuthology.orchestra.run.smithi189.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-17T02:37:05.937 INFO:teuthology.orchestra.run.smithi189.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-17T02:37:05.938 INFO:teuthology.orchestra.run.smithi131.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T02:37:05.939 INFO:teuthology.orchestra.run.smithi189.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-17T02:37:05.949 INFO:teuthology.orchestra.run.smithi131.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4989-gce1b342b-1jammy [733 kB] 2024-09-17T02:37:05.953 INFO:teuthology.orchestra.run.smithi189.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-17T02:37:05.954 INFO:teuthology.orchestra.run.smithi189.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-17T02:37:05.954 INFO:teuthology.orchestra.run.smithi189.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-17T02:37:05.955 INFO:teuthology.orchestra.run.smithi189.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-17T02:37:05.955 INFO:teuthology.orchestra.run.smithi189.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-17T02:37:05.956 INFO:teuthology.orchestra.run.smithi189.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-17T02:37:05.956 INFO:teuthology.orchestra.run.smithi189.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-17T02:37:05.971 INFO:teuthology.orchestra.run.smithi189.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-17T02:37:05.971 INFO:teuthology.orchestra.run.smithi189.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-17T02:37:05.972 INFO:teuthology.orchestra.run.smithi189.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-17T02:37:05.973 INFO:teuthology.orchestra.run.smithi131.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4989-gce1b342b-1jammy [356 kB] 2024-09-17T02:37:05.980 INFO:teuthology.orchestra.run.smithi131.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4989-gce1b342b-1jammy [32.8 kB] 2024-09-17T02:37:05.980 INFO:teuthology.orchestra.run.smithi131.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4989-gce1b342b-1jammy [180 kB] 2024-09-17T02:37:05.984 INFO:teuthology.orchestra.run.smithi131.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4989-gce1b342b-1jammy [71.3 kB] 2024-09-17T02:37:05.985 INFO:teuthology.orchestra.run.smithi131.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4989-gce1b342b-1jammy [337 kB] 2024-09-17T02:37:05.989 INFO:teuthology.orchestra.run.smithi189.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-17T02:37:05.989 INFO:teuthology.orchestra.run.smithi131.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T02:37:05.990 INFO:teuthology.orchestra.run.smithi189.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-17T02:37:05.990 INFO:teuthology.orchestra.run.smithi189.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-17T02:37:05.990 INFO:teuthology.orchestra.run.smithi131.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T02:37:05.991 INFO:teuthology.orchestra.run.smithi131.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4989-gce1b342b-1jammy [8,847 kB] 2024-09-17T02:37:05.991 INFO:teuthology.orchestra.run.smithi189.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-17T02:37:05.991 INFO:teuthology.orchestra.run.smithi189.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-17T02:37:05.992 INFO:teuthology.orchestra.run.smithi131.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T02:37:05.992 INFO:teuthology.orchestra.run.smithi131.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T02:37:05.992 INFO:teuthology.orchestra.run.smithi189.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-17T02:37:05.992 INFO:teuthology.orchestra.run.smithi189.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-17T02:37:05.993 INFO:teuthology.orchestra.run.smithi131.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T02:37:06.006 INFO:teuthology.orchestra.run.smithi189.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-17T02:37:06.007 INFO:teuthology.orchestra.run.smithi189.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-17T02:37:06.007 INFO:teuthology.orchestra.run.smithi189.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-17T02:37:06.010 INFO:teuthology.orchestra.run.smithi131.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T02:37:06.012 INFO:teuthology.orchestra.run.smithi131.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T02:37:06.012 INFO:teuthology.orchestra.run.smithi131.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T02:37:06.023 INFO:teuthology.orchestra.run.smithi189.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-17T02:37:06.024 INFO:teuthology.orchestra.run.smithi189.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-17T02:37:06.024 INFO:teuthology.orchestra.run.smithi189.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-17T02:37:06.025 INFO:teuthology.orchestra.run.smithi189.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-17T02:37:06.026 INFO:teuthology.orchestra.run.smithi189.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-17T02:37:06.027 INFO:teuthology.orchestra.run.smithi131.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T02:37:06.027 INFO:teuthology.orchestra.run.smithi131.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T02:37:06.027 INFO:teuthology.orchestra.run.smithi131.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T02:37:06.028 INFO:teuthology.orchestra.run.smithi189.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-17T02:37:06.028 INFO:teuthology.orchestra.run.smithi131.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T02:37:06.028 INFO:teuthology.orchestra.run.smithi131.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T02:37:06.029 INFO:teuthology.orchestra.run.smithi189.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-17T02:37:06.029 INFO:teuthology.orchestra.run.smithi131.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T02:37:06.040 INFO:teuthology.orchestra.run.smithi189.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-17T02:37:06.041 INFO:teuthology.orchestra.run.smithi189.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-17T02:37:06.042 INFO:teuthology.orchestra.run.smithi131.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T02:37:06.043 INFO:teuthology.orchestra.run.smithi131.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T02:37:06.043 INFO:teuthology.orchestra.run.smithi189.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-17T02:37:06.046 INFO:teuthology.orchestra.run.smithi131.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T02:37:06.057 INFO:teuthology.orchestra.run.smithi189.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-17T02:37:06.058 INFO:teuthology.orchestra.run.smithi189.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-17T02:37:06.059 INFO:teuthology.orchestra.run.smithi131.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T02:37:06.060 INFO:teuthology.orchestra.run.smithi189.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-17T02:37:06.061 INFO:teuthology.orchestra.run.smithi131.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T02:37:06.061 INFO:teuthology.orchestra.run.smithi131.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T02:37:06.065 INFO:teuthology.orchestra.run.smithi131.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T02:37:06.067 INFO:teuthology.orchestra.run.smithi131.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-17T02:37:06.067 INFO:teuthology.orchestra.run.smithi131.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T02:37:06.068 INFO:teuthology.orchestra.run.smithi131.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T02:37:06.077 INFO:teuthology.orchestra.run.smithi131.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T02:37:06.081 INFO:teuthology.orchestra.run.smithi131.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T02:37:06.082 INFO:teuthology.orchestra.run.smithi131.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T02:37:06.094 INFO:teuthology.orchestra.run.smithi131.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T02:37:06.095 INFO:teuthology.orchestra.run.smithi131.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T02:37:06.095 INFO:teuthology.orchestra.run.smithi131.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T02:37:06.096 INFO:teuthology.orchestra.run.smithi131.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T02:37:06.097 INFO:teuthology.orchestra.run.smithi131.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T02:37:06.098 INFO:teuthology.orchestra.run.smithi131.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T02:37:06.104 INFO:teuthology.orchestra.run.smithi131.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T02:37:06.111 INFO:teuthology.orchestra.run.smithi189.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-17T02:37:06.112 INFO:teuthology.orchestra.run.smithi189.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-17T02:37:06.112 INFO:teuthology.orchestra.run.smithi131.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T02:37:06.113 INFO:teuthology.orchestra.run.smithi131.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T02:37:06.113 INFO:teuthology.orchestra.run.smithi189.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-17T02:37:06.113 INFO:teuthology.orchestra.run.smithi189.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-17T02:37:06.114 INFO:teuthology.orchestra.run.smithi131.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T02:37:06.115 INFO:teuthology.orchestra.run.smithi189.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-17T02:37:06.126 INFO:teuthology.orchestra.run.smithi189.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4989-gce1b342b-1jammy [3,368 kB] 2024-09-17T02:37:06.129 INFO:teuthology.orchestra.run.smithi131.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T02:37:06.132 INFO:teuthology.orchestra.run.smithi189.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-17T02:37:06.134 INFO:teuthology.orchestra.run.smithi189.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-17T02:37:06.134 INFO:teuthology.orchestra.run.smithi189.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-17T02:37:06.148 INFO:teuthology.orchestra.run.smithi189.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-17T02:37:06.148 INFO:teuthology.orchestra.run.smithi189.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-17T02:37:06.149 INFO:teuthology.orchestra.run.smithi189.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-17T02:37:06.149 INFO:teuthology.orchestra.run.smithi189.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-17T02:37:06.150 INFO:teuthology.orchestra.run.smithi189.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-17T02:37:06.150 INFO:teuthology.orchestra.run.smithi189.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-17T02:37:06.166 INFO:teuthology.orchestra.run.smithi189.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-17T02:37:06.166 INFO:teuthology.orchestra.run.smithi189.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-17T02:37:06.169 INFO:teuthology.orchestra.run.smithi189.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-17T02:37:06.182 INFO:teuthology.orchestra.run.smithi189.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-17T02:37:06.184 INFO:teuthology.orchestra.run.smithi189.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-17T02:37:06.184 INFO:teuthology.orchestra.run.smithi189.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-17T02:37:06.187 INFO:teuthology.orchestra.run.smithi189.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-17T02:37:06.190 INFO:teuthology.orchestra.run.smithi189.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-17T02:37:06.190 INFO:teuthology.orchestra.run.smithi189.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-17T02:37:06.191 INFO:teuthology.orchestra.run.smithi189.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-17T02:37:06.199 INFO:teuthology.orchestra.run.smithi189.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-17T02:37:06.203 INFO:teuthology.orchestra.run.smithi189.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-17T02:37:06.204 INFO:teuthology.orchestra.run.smithi189.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-17T02:37:06.204 INFO:teuthology.orchestra.run.smithi131.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4989-gce1b342b-1jammy [112 kB] 2024-09-17T02:37:06.206 INFO:teuthology.orchestra.run.smithi131.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4989-gce1b342b-1jammy [478 kB] 2024-09-17T02:37:06.214 INFO:teuthology.orchestra.run.smithi131.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4989-gce1b342b-1jammy [29.1 MB] 2024-09-17T02:37:06.217 INFO:teuthology.orchestra.run.smithi189.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-17T02:37:06.217 INFO:teuthology.orchestra.run.smithi189.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-17T02:37:06.217 INFO:teuthology.orchestra.run.smithi189.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-17T02:37:06.218 INFO:teuthology.orchestra.run.smithi189.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-17T02:37:06.221 INFO:teuthology.orchestra.run.smithi189.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-17T02:37:06.221 INFO:teuthology.orchestra.run.smithi189.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-17T02:37:06.227 INFO:teuthology.orchestra.run.smithi189.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-17T02:37:06.234 INFO:teuthology.orchestra.run.smithi189.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-17T02:37:06.234 INFO:teuthology.orchestra.run.smithi189.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-17T02:37:06.236 INFO:teuthology.orchestra.run.smithi189.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-17T02:37:06.251 INFO:teuthology.orchestra.run.smithi189.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-17T02:37:06.300 INFO:teuthology.orchestra.run.smithi189.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4989-gce1b342b-1jammy [733 kB] 2024-09-17T02:37:06.315 INFO:teuthology.orchestra.run.smithi189.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4989-gce1b342b-1jammy [356 kB] 2024-09-17T02:37:06.357 INFO:teuthology.orchestra.run.smithi189.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4989-gce1b342b-1jammy [32.8 kB] 2024-09-17T02:37:06.358 INFO:teuthology.orchestra.run.smithi189.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4989-gce1b342b-1jammy [180 kB] 2024-09-17T02:37:06.363 INFO:teuthology.orchestra.run.smithi189.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4989-gce1b342b-1jammy [71.3 kB] 2024-09-17T02:37:06.365 INFO:teuthology.orchestra.run.smithi189.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4989-gce1b342b-1jammy [337 kB] 2024-09-17T02:37:06.372 INFO:teuthology.orchestra.run.smithi189.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4989-gce1b342b-1jammy [8,847 kB] 2024-09-17T02:37:06.812 INFO:teuthology.orchestra.run.smithi189.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4989-gce1b342b-1jammy [112 kB] 2024-09-17T02:37:06.861 INFO:teuthology.orchestra.run.smithi189.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4989-gce1b342b-1jammy [478 kB] 2024-09-17T02:37:06.870 INFO:teuthology.orchestra.run.smithi189.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4989-gce1b342b-1jammy [29.1 MB] 2024-09-17T02:37:07.024 INFO:teuthology.orchestra.run.smithi131.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4989-gce1b342b-1jammy [5,020 kB] 2024-09-17T02:37:07.100 INFO:teuthology.orchestra.run.smithi131.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4989-gce1b342b-1jammy [251 kB] 2024-09-17T02:37:07.102 INFO:teuthology.orchestra.run.smithi131.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4989-gce1b342b-1jammy [126 kB] 2024-09-17T02:37:07.104 INFO:teuthology.orchestra.run.smithi131.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4989-gce1b342b-1jammy [1,551 kB] 2024-09-17T02:37:07.154 INFO:teuthology.orchestra.run.smithi131.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4989-gce1b342b-1jammy [6,200 kB] 2024-09-17T02:37:07.336 INFO:teuthology.orchestra.run.smithi131.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4989-gce1b342b-1jammy [22.9 MB] 2024-09-17T02:37:08.024 INFO:teuthology.orchestra.run.smithi189.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4989-gce1b342b-1jammy [5,020 kB] 2024-09-17T02:37:08.093 INFO:teuthology.orchestra.run.smithi131.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4989-gce1b342b-1jammy [14.0 kB] 2024-09-17T02:37:08.094 INFO:teuthology.orchestra.run.smithi131.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4989-gce1b342b-1jammy [865 kB] 2024-09-17T02:37:08.104 INFO:teuthology.orchestra.run.smithi189.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4989-gce1b342b-1jammy [251 kB] 2024-09-17T02:37:08.107 INFO:teuthology.orchestra.run.smithi189.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4989-gce1b342b-1jammy [126 kB] 2024-09-17T02:37:08.109 INFO:teuthology.orchestra.run.smithi189.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4989-gce1b342b-1jammy [1,551 kB] 2024-09-17T02:37:08.111 INFO:teuthology.orchestra.run.smithi131.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4989-gce1b342b-1jammy [2,386 kB] 2024-09-17T02:37:08.145 INFO:teuthology.orchestra.run.smithi189.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4989-gce1b342b-1jammy [6,200 kB] 2024-09-17T02:37:08.150 INFO:teuthology.orchestra.run.smithi131.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4989-gce1b342b-1jammy [814 kB] 2024-09-17T02:37:08.161 INFO:teuthology.orchestra.run.smithi131.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4989-gce1b342b-1jammy [160 kB] 2024-09-17T02:37:08.163 INFO:teuthology.orchestra.run.smithi131.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4989-gce1b342b-1jammy [61.5 MB] 2024-09-17T02:37:08.385 INFO:teuthology.orchestra.run.smithi189.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4989-gce1b342b-1jammy [22.9 MB] 2024-09-17T02:37:09.297 INFO:teuthology.orchestra.run.smithi189.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4989-gce1b342b-1jammy [14.0 kB] 2024-09-17T02:37:09.298 INFO:teuthology.orchestra.run.smithi189.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4989-gce1b342b-1jammy [865 kB] 2024-09-17T02:37:09.315 INFO:teuthology.orchestra.run.smithi189.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4989-gce1b342b-1jammy [2,386 kB] 2024-09-17T02:37:09.385 INFO:teuthology.orchestra.run.smithi189.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4989-gce1b342b-1jammy [814 kB] 2024-09-17T02:37:09.419 INFO:teuthology.orchestra.run.smithi189.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4989-gce1b342b-1jammy [160 kB] 2024-09-17T02:37:09.423 INFO:teuthology.orchestra.run.smithi189.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4989-gce1b342b-1jammy [61.5 MB] 2024-09-17T02:37:10.173 INFO:teuthology.orchestra.run.smithi131.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4989-gce1b342b-1jammy [8,625 kB] 2024-09-17T02:37:10.348 INFO:teuthology.orchestra.run.smithi131.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4989-gce1b342b-1jammy [14.2 kB] 2024-09-17T02:37:10.348 INFO:teuthology.orchestra.run.smithi131.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4989-gce1b342b-1jammy [40.9 MB] 2024-09-17T02:37:11.872 INFO:teuthology.orchestra.run.smithi189.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4989-gce1b342b-1jammy [8,625 kB] 2024-09-17T02:37:11.887 INFO:teuthology.orchestra.run.smithi131.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4989-gce1b342b-1jammy [130 kB] 2024-09-17T02:37:11.890 INFO:teuthology.orchestra.run.smithi131.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4989-gce1b342b-1jammy [237 kB] 2024-09-17T02:37:11.894 INFO:teuthology.orchestra.run.smithi131.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4989-gce1b342b-1jammy [33.9 kB] 2024-09-17T02:37:11.895 INFO:teuthology.orchestra.run.smithi131.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4989-gce1b342b-1jammy [23.4 kB] 2024-09-17T02:37:11.947 INFO:teuthology.orchestra.run.smithi131.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4989-gce1b342b-1jammy [39.0 kB] 2024-09-17T02:37:11.948 INFO:teuthology.orchestra.run.smithi131.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4989-gce1b342b-1jammy [17.6 MB] 2024-09-17T02:37:11.983 INFO:teuthology.orchestra.run.smithi189.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4989-gce1b342b-1jammy [14.2 kB] 2024-09-17T02:37:11.983 INFO:teuthology.orchestra.run.smithi189.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4989-gce1b342b-1jammy [40.9 MB] 2024-09-17T02:37:12.450 INFO:teuthology.orchestra.run.smithi131.stdout:Get:125 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4989-gce1b342b-1jammy [93.1 kB] 2024-09-17T02:37:13.048 INFO:teuthology.orchestra.run.smithi189.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4989-gce1b342b-1jammy [130 kB] 2024-09-17T02:37:13.050 INFO:teuthology.orchestra.run.smithi189.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4989-gce1b342b-1jammy [237 kB] 2024-09-17T02:37:13.054 INFO:teuthology.orchestra.run.smithi189.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4989-gce1b342b-1jammy [33.9 kB] 2024-09-17T02:37:13.055 INFO:teuthology.orchestra.run.smithi189.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4989-gce1b342b-1jammy [23.4 kB] 2024-09-17T02:37:13.055 INFO:teuthology.orchestra.run.smithi189.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4989-gce1b342b-1jammy [39.0 kB] 2024-09-17T02:37:13.056 INFO:teuthology.orchestra.run.smithi189.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4989-gce1b342b-1jammy [17.6 MB] 2024-09-17T02:37:13.088 INFO:teuthology.orchestra.run.smithi131.stdout:Fetched 239 MB in 7s (34.8 MB/s) 2024-09-17T02:37:13.227 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-17T02:37:13.325 INFO:teuthology.orchestra.run.smithi131.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-17T02:37:13.330 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-17T02:37:13.375 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-17T02:37:13.381 INFO:teuthology.orchestra.run.smithi189.stdout:Get:125 https://2.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4989-gce1b342b-1jammy [93.1 kB] 2024-09-17T02:37:13.653 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T02:37:13.679 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T02:37:13.719 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T02:37:14.013 INFO:teuthology.orchestra.run.smithi189.stdout:Fetched 239 MB in 8s (31.1 MB/s) 2024-09-17T02:37:14.048 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-cmd2. 2024-09-17T02:37:14.074 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-17T02:37:14.114 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-17T02:37:14.151 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-17T02:37:14.246 INFO:teuthology.orchestra.run.smithi189.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-17T02:37:14.251 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-17T02:37:14.291 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-17T02:37:14.442 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T02:37:14.468 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T02:37:14.551 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T02:37:14.593 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-17T02:37:14.620 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-17T02:37:14.660 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T02:37:14.905 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T02:37:14.931 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T02:37:14.971 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T02:37:14.979 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-cmd2. 2024-09-17T02:37:15.005 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-17T02:37:15.046 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-17T02:37:15.283 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T02:37:15.309 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T02:37:15.358 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T02:37:15.416 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-17T02:37:15.442 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-17T02:37:15.541 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T02:37:15.795 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T02:37:15.821 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T02:37:15.869 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:37:15.936 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-17T02:37:15.962 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-17T02:37:16.013 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T02:37:16.231 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T02:37:16.247 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T02:37:16.281 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:37:16.381 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-17T02:37:16.407 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-17T02:37:16.448 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T02:37:16.610 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T02:37:16.637 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T02:37:16.677 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:37:16.901 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-17T02:37:16.927 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T02:37:16.976 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:37:17.041 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T02:37:17.064 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T02:37:17.104 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T02:37:17.408 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package librados2. 2024-09-17T02:37:17.434 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../010-librados2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:17.454 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-17T02:37:17.475 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:17.482 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T02:37:17.529 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:37:17.883 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-17T02:37:17.908 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-17T02:37:17.949 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:37:17.961 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libnbd0. 2024-09-17T02:37:17.987 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T02:37:18.027 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T02:37:18.286 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-17T02:37:18.312 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-17T02:37:18.339 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package librbd1. 2024-09-17T02:37:18.360 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T02:37:18.365 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../012-librbd1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:18.405 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:18.755 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package librados2. 2024-09-17T02:37:18.781 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../010-librados2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:18.830 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:18.884 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T02:37:18.910 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:18.950 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:19.236 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-rados. 2024-09-17T02:37:19.251 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:19.294 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:19.391 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libnbd0. 2024-09-17T02:37:19.418 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-17T02:37:19.458 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-17T02:37:19.597 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T02:37:19.623 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:19.655 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:19.769 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package librbd1. 2024-09-17T02:37:19.795 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../012-librbd1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:19.836 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:19.875 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T02:37:19.901 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:19.941 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:20.253 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T02:37:20.279 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:20.305 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libcephfs2. 2024-09-17T02:37:20.319 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:20.331 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:20.372 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:20.648 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T02:37:20.674 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T02:37:20.674 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-rados. 2024-09-17T02:37:20.689 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:20.714 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T02:37:20.733 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:20.976 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T02:37:21.002 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:21.019 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-17T02:37:21.042 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:21.045 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:21.077 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:21.329 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-cephfs. 2024-09-17T02:37:21.345 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:21.378 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T02:37:21.387 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:21.405 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T02:37:21.437 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T02:37:21.690 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T02:37:21.699 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-17T02:37:21.705 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T02:37:21.726 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:21.748 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T02:37:21.765 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:22.060 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T02:37:22.085 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-prettytable. 2024-09-17T02:37:22.086 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T02:37:22.112 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-17T02:37:22.126 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T02:37:22.152 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-17T02:37:22.396 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-rbd. 2024-09-17T02:37:22.419 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:22.454 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:22.455 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package lua5.1. 2024-09-17T02:37:22.490 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T02:37:22.522 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T02:37:22.782 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-17T02:37:22.808 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package lua-any. 2024-09-17T02:37:22.808 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-17T02:37:22.834 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-17T02:37:22.849 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T02:37:22.874 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T02:37:23.144 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package zip. 2024-09-17T02:37:23.151 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-17T02:37:23.159 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-17T02:37:23.178 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-17T02:37:23.202 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T02:37:23.218 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T02:37:23.521 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-17T02:37:23.522 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package unzip. 2024-09-17T02:37:23.536 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-17T02:37:23.548 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T02:37:23.579 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T02:37:23.589 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T02:37:23.874 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package lua5.1. 2024-09-17T02:37:23.901 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-17T02:37:23.932 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-17T02:37:24.260 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package lua-any. 2024-09-17T02:37:24.276 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-17T02:37:24.319 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-17T02:37:24.437 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package luarocks. 2024-09-17T02:37:24.463 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T02:37:24.503 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T02:37:24.563 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package zip. 2024-09-17T02:37:24.589 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-17T02:37:24.629 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking zip (3.0-12build2) ... 2024-09-17T02:37:24.845 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package librgw2. 2024-09-17T02:37:24.860 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../028-librgw2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:24.895 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:24.958 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package unzip. 2024-09-17T02:37:24.985 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-17T02:37:25.024 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-17T02:37:25.378 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package luarocks. 2024-09-17T02:37:25.405 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-17T02:37:25.436 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-17T02:37:25.589 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T02:37:25.604 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:25.647 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:25.794 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package librgw2. 2024-09-17T02:37:25.821 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../028-librgw2_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:25.861 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:25.984 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T02:37:26.010 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T02:37:26.067 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T02:37:26.387 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T02:37:26.413 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:26.454 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:26.563 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-rgw. 2024-09-17T02:37:26.579 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:26.621 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:26.790 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-common. 2024-09-17T02:37:26.817 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:26.857 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:27.000 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-17T02:37:27.026 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-17T02:37:27.066 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T02:37:27.319 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libradosstriper1. 2024-09-17T02:37:27.334 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:27.369 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:27.789 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-common. 2024-09-17T02:37:27.816 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:27.855 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:28.579 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-base. 2024-09-17T02:37:28.606 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:28.655 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:29.234 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T02:37:29.249 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T02:37:29.284 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T02:37:29.579 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T02:37:29.605 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T02:37:29.645 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T02:37:29.819 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-base. 2024-09-17T02:37:29.845 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:29.895 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:29.974 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T02:37:30.000 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T02:37:30.040 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T02:37:30.318 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T02:37:30.344 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T02:37:30.376 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T02:37:30.424 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-17T02:37:30.450 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-17T02:37:30.490 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-17T02:37:30.655 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T02:37:30.681 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T02:37:30.721 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T02:37:30.818 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-cheroot. 2024-09-17T02:37:30.845 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-17T02:37:30.884 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T02:37:31.024 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T02:37:31.051 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T02:37:31.090 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T02:37:31.196 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-17T02:37:31.222 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-17T02:37:31.262 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-17T02:37:31.335 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-portend. 2024-09-17T02:37:31.362 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-17T02:37:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T02:37:31.549 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-17T02:37:31.576 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-17T02:37:31.615 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-17T02:37:31.705 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T02:37:31.732 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T02:37:31.771 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T02:37:31.893 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-17T02:37:31.920 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-17T02:37:31.960 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-17T02:37:32.075 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T02:37:32.102 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T02:37:32.141 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T02:37:32.263 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-tempora. 2024-09-17T02:37:32.289 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-17T02:37:32.329 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-17T02:37:32.529 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T02:37:32.555 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T02:37:32.595 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T02:37:32.617 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-portend. 2024-09-17T02:37:32.642 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-17T02:37:32.682 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-17T02:37:32.907 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T02:37:32.933 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T02:37:32.943 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-17T02:37:32.958 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-17T02:37:32.973 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T02:37:32.993 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-17T02:37:33.235 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T02:37:33.261 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T02:37:33.279 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-17T02:37:33.301 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T02:37:33.306 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-17T02:37:33.346 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-17T02:37:33.554 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-mako. 2024-09-17T02:37:33.581 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T02:37:33.621 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T02:37:33.683 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-natsort. 2024-09-17T02:37:33.710 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-17T02:37:33.749 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-17T02:37:33.983 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T02:37:34.009 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T02:37:34.049 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T02:37:34.061 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-logutils. 2024-09-17T02:37:34.087 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-17T02:37:34.127 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-17T02:37:34.514 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-17T02:37:34.540 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-17T02:37:34.544 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T02:37:34.571 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T02:37:34.580 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-17T02:37:34.611 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T02:37:34.850 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-mako. 2024-09-17T02:37:34.876 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-17T02:37:34.897 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-webob. 2024-09-17T02:37:34.908 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T02:37:34.923 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T02:37:34.955 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T02:37:35.220 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-17T02:37:35.249 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-17T02:37:35.250 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T02:37:35.277 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T02:37:35.286 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-17T02:37:35.319 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T02:37:35.572 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-17T02:37:35.595 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T02:37:35.599 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-17T02:37:35.611 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T02:37:35.639 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-17T02:37:35.645 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T02:37:35.898 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-paste. 2024-09-17T02:37:35.913 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T02:37:35.925 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-webob. 2024-09-17T02:37:35.948 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T02:37:35.951 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-17T02:37:35.992 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T02:37:36.259 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T02:37:36.275 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T02:37:36.309 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T02:37:36.328 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-waitress. 2024-09-17T02:37:36.354 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-17T02:37:36.398 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T02:37:36.604 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T02:37:36.630 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T02:37:36.671 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T02:37:36.706 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-tempita. 2024-09-17T02:37:36.732 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-17T02:37:36.772 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T02:37:36.991 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T02:37:37.017 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T02:37:37.057 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T02:37:37.059 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-paste. 2024-09-17T02:37:37.085 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-17T02:37:37.117 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T02:37:37.344 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T02:37:37.370 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T02:37:37.410 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T02:37:37.437 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-17T02:37:37.463 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-17T02:37:37.503 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T02:37:37.791 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T02:37:37.814 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-17T02:37:37.817 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T02:37:37.841 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-17T02:37:37.857 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T02:37:37.881 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-17T02:37:38.177 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-idna. 2024-09-17T02:37:38.203 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T02:37:38.234 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-webtest. 2024-09-17T02:37:38.243 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T02:37:38.261 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-17T02:37:38.301 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-17T02:37:38.517 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T02:37:38.540 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T02:37:38.570 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pecan. 2024-09-17T02:37:38.580 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T02:37:38.597 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-17T02:37:38.637 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T02:37:39.301 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-requests. 2024-09-17T02:37:39.328 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T02:37:39.742 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T02:37:39.817 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-certifi. 2024-09-17T02:37:39.844 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-17T02:37:39.883 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-17T02:37:40.079 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T02:37:40.105 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T02:37:40.145 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T02:37:40.203 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-idna. 2024-09-17T02:37:40.230 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-17T02:37:40.270 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T02:37:40.440 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T02:37:40.467 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:40.507 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:40.531 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-urllib3. 2024-09-17T02:37:40.546 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-17T02:37:40.581 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T02:37:40.887 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T02:37:40.909 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-requests. 2024-09-17T02:37:40.914 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:40.936 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-17T02:37:40.954 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:40.977 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T02:37:41.265 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T02:37:41.287 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-17T02:37:41.292 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T02:37:41.314 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-17T02:37:41.332 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T02:37:41.353 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T02:37:41.623 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-17T02:37:41.638 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:41.652 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T02:37:41.678 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T02:37:41.681 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:41.718 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T02:37:42.013 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T02:37:42.028 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-17T02:37:42.040 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:42.055 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:42.080 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:42.094 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:42.447 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-17T02:37:42.474 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-17T02:37:42.474 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T02:37:42.501 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:42.514 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-17T02:37:42.541 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:42.817 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-openssl. 2024-09-17T02:37:42.843 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-17T02:37:42.883 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-17T02:37:43.170 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr. 2024-09-17T02:37:43.170 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T02:37:43.197 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:43.197 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:43.228 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:43.236 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:43.598 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mon. 2024-09-17T02:37:43.625 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:43.664 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:44.265 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph. 2024-09-17T02:37:44.277 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../069-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:44.301 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-osd. 2024-09-17T02:37:44.314 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:44.328 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:44.368 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:44.834 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T02:37:44.860 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:44.901 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:45.296 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T02:37:45.322 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:45.362 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:45.404 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph. 2024-09-17T02:37:45.420 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../069-ceph_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:45.454 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:45.724 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package cephadm. 2024-09-17T02:37:45.739 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../072-cephadm_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:45.741 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-fuse. 2024-09-17T02:37:45.767 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:45.773 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:45.806 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:46.110 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T02:37:46.137 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T02:37:46.176 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T02:37:46.193 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mds. 2024-09-17T02:37:46.220 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:46.259 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:46.513 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T02:37:46.528 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T02:37:46.571 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T02:37:46.671 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package cephadm. 2024-09-17T02:37:46.697 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../072-cephadm_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:37:46.737 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:46.999 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-17T02:37:47.025 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-17T02:37:47.056 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-17T02:37:47.351 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-17T02:37:47.378 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T02:37:47.417 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T02:37:47.576 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-babel. 2024-09-17T02:37:47.603 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T02:37:47.634 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T02:37:47.913 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T02:37:47.928 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T02:37:47.971 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T02:37:48.324 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T02:37:48.351 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:48.391 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:48.439 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-babel. 2024-09-17T02:37:48.467 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-17T02:37:48.505 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T02:37:48.727 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T02:37:48.755 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T02:37:48.794 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T02:37:48.834 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-jinja2. 2024-09-17T02:37:48.860 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-17T02:37:48.900 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T02:37:49.055 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-routes. 2024-09-17T02:37:49.071 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T02:37:49.105 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T02:37:49.237 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-17T02:37:49.264 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:49.303 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:49.417 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T02:37:49.444 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:49.475 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:49.640 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-17T02:37:49.667 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-17T02:37:49.706 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-17T02:37:50.034 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-routes. 2024-09-17T02:37:50.062 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-17T02:37:50.101 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T02:37:50.412 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-17T02:37:50.439 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:50.487 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:52.348 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T02:37:52.365 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T02:37:52.406 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T02:37:52.893 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T02:37:52.921 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T02:37:53.046 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T02:37:53.153 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-17T02:37:53.170 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-17T02:37:53.211 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T02:37:53.425 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T02:37:53.453 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T02:37:53.491 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T02:37:53.706 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-joblib. 2024-09-17T02:37:53.734 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-17T02:37:53.795 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T02:37:53.822 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T02:37:53.861 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T02:37:53.883 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T02:37:54.204 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-17T02:37:54.231 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-17T02:37:54.262 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-17T02:37:54.546 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T02:37:54.556 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-sklearn. 2024-09-17T02:37:54.573 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:54.584 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-17T02:37:54.612 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:54.622 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T02:37:55.515 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-17T02:37:55.543 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:55.581 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:55.774 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T02:37:55.802 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T02:37:55.841 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T02:37:56.119 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T02:37:56.143 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T02:37:56.177 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T02:37:56.473 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T02:37:56.493 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-cachetools. 2024-09-17T02:37:56.501 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T02:37:56.520 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-17T02:37:56.539 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T02:37:56.568 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-17T02:37:56.859 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T02:37:56.871 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-17T02:37:56.886 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-17T02:37:56.898 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-17T02:37:56.925 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T02:37:56.937 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-17T02:37:57.241 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-17T02:37:57.246 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T02:37:57.268 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-17T02:37:57.274 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T02:37:57.307 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T02:37:57.313 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T02:37:57.608 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T02:37:57.610 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-rsa. 2024-09-17T02:37:57.635 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T02:37:57.638 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-17T02:37:57.666 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T02:37:57.693 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-17T02:37:57.953 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T02:37:57.980 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T02:37:57.989 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-google-auth. 2024-09-17T02:37:58.016 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-17T02:37:58.019 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T02:37:58.055 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-17T02:37:58.356 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T02:37:58.384 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T02:37:58.384 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-17T02:37:58.411 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-17T02:37:58.422 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T02:37:58.450 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T02:37:58.711 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-websocket. 2024-09-17T02:37:58.739 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-17T02:37:58.769 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-17T02:37:59.062 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T02:37:59.064 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-17T02:37:59.078 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:59.092 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-17T02:37:59.120 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:59.131 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T02:37:59.407 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T02:37:59.434 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T02:37:59.473 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T02:37:59.787 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-17T02:37:59.804 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:37:59.818 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T02:37:59.846 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T02:37:59.853 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:37:59.884 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T02:38:00.122 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-17T02:38:00.138 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-17T02:38:00.172 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T02:38:00.213 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package jq. 2024-09-17T02:38:00.241 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T02:38:00.271 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T02:38:00.482 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package socat. 2024-09-17T02:38:00.498 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T02:38:00.540 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T02:38:00.834 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-17T02:38:00.862 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T02:38:00.900 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T02:38:01.027 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T02:38:01.052 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T02:38:01.093 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T02:38:01.170 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package jq. 2024-09-17T02:38:01.198 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-17T02:38:01.237 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-17T02:38:01.448 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-test. 2024-09-17T02:38:01.465 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:38:01.492 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package socat. 2024-09-17T02:38:01.506 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:01.519 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-17T02:38:01.572 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-17T02:38:01.876 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package xmlstarlet. 2024-09-17T02:38:01.903 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-17T02:38:01.942 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-17T02:38:02.271 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-test. 2024-09-17T02:38:02.288 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:38:02.330 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:03.569 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T02:38:03.585 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:38:03.627 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:04.000 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-17T02:38:04.028 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:38:04.067 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:04.334 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ceph-volume. 2024-09-17T02:38:04.336 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-colorama. 2024-09-17T02:38:04.362 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:38:04.364 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-17T02:38:04.392 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:04.403 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-17T02:38:04.655 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package cephfs-shell. 2024-09-17T02:38:04.671 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:38:04.730 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:04.772 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-17T02:38:04.799 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:38:04.838 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:04.957 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package cephfs-top. 2024-09-17T02:38:04.974 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:38:05.016 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:05.166 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-colorama. 2024-09-17T02:38:05.194 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-17T02:38:05.232 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-17T02:38:05.268 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T02:38:05.296 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:38:05.334 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:05.504 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package cephfs-shell. 2024-09-17T02:38:05.517 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:38:05.551 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:05.662 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T02:38:05.691 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T02:38:05.754 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T02:38:05.804 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package cephfs-top. 2024-09-17T02:38:05.820 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4989-gce1b342b-1jammy_all.deb ... 2024-09-17T02:38:05.862 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:06.084 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T02:38:06.112 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T02:38:06.114 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-17T02:38:06.131 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:38:06.150 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T02:38:06.173 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:06.487 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T02:38:06.492 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-17T02:38:06.515 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T02:38:06.520 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-17T02:38:06.553 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T02:38:06.559 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T02:38:06.855 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-17T02:38:06.883 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-17T02:38:06.921 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-17T02:38:06.959 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package pkg-config. 2024-09-17T02:38:06.988 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T02:38:07.025 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T02:38:07.259 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package nvme-cli. 2024-09-17T02:38:07.270 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T02:38:07.287 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-17T02:38:07.298 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T02:38:07.325 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T02:38:07.336 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T02:38:07.747 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package pkg-config. 2024-09-17T02:38:07.756 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T02:38:07.776 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-17T02:38:07.784 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T02:38:07.814 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T02:38:07.822 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T02:38:08.075 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-17T02:38:08.091 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-17T02:38:08.108 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T02:38:08.133 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-17T02:38:08.137 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T02:38:08.175 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T02:38:08.486 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T02:38:08.515 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T02:38:08.553 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T02:38:08.561 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-17T02:38:08.590 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-17T02:38:08.629 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-17T02:38:08.805 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T02:38:08.822 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T02:38:08.864 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T02:38:08.931 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pastescript. 2024-09-17T02:38:08.959 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-17T02:38:08.997 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-17T02:38:09.166 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-py. 2024-09-17T02:38:09.192 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-17T02:38:09.225 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T02:38:09.342 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pluggy. 2024-09-17T02:38:09.370 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-17T02:38:09.409 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-17T02:38:09.520 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T02:38:09.548 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T02:38:09.578 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T02:38:09.729 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-psutil. 2024-09-17T02:38:09.757 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-17T02:38:09.795 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-17T02:38:10.032 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T02:38:10.061 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T02:38:10.098 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T02:38:10.123 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-py. 2024-09-17T02:38:10.151 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-17T02:38:10.189 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-17T02:38:10.409 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-toml. 2024-09-17T02:38:10.438 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-17T02:38:10.484 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T02:38:10.484 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pygments. 2024-09-17T02:38:10.513 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-17T02:38:10.550 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T02:38:10.762 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T02:38:10.790 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T02:38:10.828 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T02:38:10.997 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-17T02:38:11.025 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-17T02:38:11.063 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-17T02:38:11.174 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T02:38:11.203 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T02:38:11.240 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T02:38:11.374 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-toml. 2024-09-17T02:38:11.403 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-17T02:38:11.440 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-17T02:38:11.509 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T02:38:11.526 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T02:38:11.567 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T02:38:11.710 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-pytest. 2024-09-17T02:38:11.739 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-17T02:38:11.776 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T02:38:12.122 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package python3-simplejson. 2024-09-17T02:38:12.150 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-17T02:38:12.188 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-17T02:38:12.271 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package radosgw. 2024-09-17T02:38:12.300 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../123-radosgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:38:12.329 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:12.466 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-17T02:38:12.483 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-17T02:38:12.524 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-17T02:38:13.237 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package radosgw. 2024-09-17T02:38:13.265 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../123-radosgw_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:38:13.295 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:13.483 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T02:38:13.511 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:38:13.549 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:13.845 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T02:38:14.129 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T02:38:14.255 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T02:38:14.439 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package rbd-fuse. 2024-09-17T02:38:14.464 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4989-gce1b342b-1jammy_amd64.deb ... 2024-09-17T02:38:14.506 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:14.518 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T02:38:14.811 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-17T02:38:14.860 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T02:38:15.070 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-17T02:38:15.196 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-17T02:38:15.270 INFO:teuthology.orchestra.run.smithi131.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T02:38:15.357 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:15.454 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-17T02:38:15.808 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-17T02:38:15.831 INFO:teuthology.orchestra.run.smithi131.stdout:Adding system user cephadm....done 2024-09-17T02:38:15.987 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T02:38:16.255 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T02:38:16.283 INFO:teuthology.orchestra.run.smithi189.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-17T02:38:16.431 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:16.515 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T02:38:16.641 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T02:38:16.909 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T02:38:16.973 INFO:teuthology.orchestra.run.smithi189.stdout:Adding system user cephadm....done 2024-09-17T02:38:17.168 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T02:38:17.180 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-17T02:38:17.294 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T02:38:17.587 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T02:38:17.674 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-17T02:38:17.917 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-17T02:38:17.948 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T02:38:18.034 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-17T02:38:18.199 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-17T02:38:18.286 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-17T02:38:18.442 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T02:38:18.545 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-17T02:38:18.671 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-17T02:38:18.726 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T02:38:18.945 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T02:38:18.973 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-17T02:38:19.187 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T02:38:19.317 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-17T02:38:19.430 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T02:38:19.551 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-17T02:38:19.681 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:19.794 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-17T02:38:19.924 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T02:38:20.050 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T02:38:20.062 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-17T02:38:20.176 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T02:38:20.322 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-17T02:38:20.427 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-17T02:38:20.573 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-17T02:38:20.677 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T02:38:20.804 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T02:38:20.841 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-17T02:38:21.063 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T02:38:21.092 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-ceph-argparse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:21.368 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-17T02:38:21.435 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T02:38:21.470 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T02:38:21.486 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-17T02:38:21.533 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T02:38:21.612 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-17T02:38:21.659 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T02:38:21.721 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-17T02:38:21.964 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-17T02:38:22.018 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T02:38:22.090 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-17T02:38:22.144 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T02:38:22.349 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-17T02:38:22.403 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T02:38:22.646 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T02:38:22.829 INFO:teuthology.orchestra.run.smithi189.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-17T02:38:22.864 INFO:teuthology.orchestra.run.smithi189.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-17T02:38:22.936 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-17T02:38:22.956 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T02:38:23.053 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-17T02:38:23.107 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T02:38:23.413 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-17T02:38:23.539 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-17T02:38:23.767 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T02:38:23.807 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-17T02:38:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T02:38:24.066 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-17T02:38:24.144 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T02:38:24.245 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T02:38:24.384 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up zip (3.0-12build2) ... 2024-09-17T02:38:24.501 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-17T02:38:24.529 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T02:38:24.889 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T02:38:25.148 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T02:38:25.245 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-17T02:38:25.505 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-17T02:38:25.566 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T02:38:25.631 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-17T02:38:25.757 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-17T02:38:25.851 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-17T02:38:26.074 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-17T02:38:26.152 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T02:38:26.459 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-17T02:38:26.710 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-17T02:38:26.711 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T02:38:26.954 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T02:38:27.072 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:27.095 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-17T02:38:27.356 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T02:38:27.388 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-17T02:38:27.649 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T02:38:27.681 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-17T02:38:28.032 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-17T02:38:28.283 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-17T02:38:28.426 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:28.736 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-17T02:38:28.837 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T02:38:29.053 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-17T02:38:29.081 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:38:29.190 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T02:38:29.441 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T02:38:29.567 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T02:38:29.696 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T02:38:29.986 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T02:38:30.237 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T02:38:30.267 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-17T02:38:30.471 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:38:30.527 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:38:30.573 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T02:38:30.637 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-17T02:38:30.849 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T02:38:30.896 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-17T02:38:31.022 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-17T02:38:31.100 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T02:38:31.148 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-17T02:38:31.376 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T02:38:31.450 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-17T02:38:31.635 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T02:38:31.709 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-17T02:38:31.886 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T02:38:31.960 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:38:32.069 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-17T02:38:32.129 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T02:38:32.255 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T02:38:32.362 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-17T02:38:32.531 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T02:38:32.614 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-17T02:38:32.632 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T02:38:32.873 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-17T02:38:32.917 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T02:38:33.132 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-17T02:38:33.226 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T02:38:33.383 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-17T02:38:33.477 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T02:38:33.603 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T02:38:33.651 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-17T02:38:33.737 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T02:38:33.777 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-17T02:38:33.826 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T02:38:34.045 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-17T02:38:34.064 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T02:38:34.154 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-17T02:38:34.190 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T02:38:34.439 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-17T02:38:34.583 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T02:38:34.732 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-17T02:38:34.859 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T02:38:34.982 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-17T02:38:34.985 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T02:38:35.109 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-17T02:38:35.234 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-17T02:38:35.236 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:38:35.325 INFO:teuthology.orchestra.run.smithi189.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-17T02:38:35.337 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T02:38:35.563 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-17T02:38:35.638 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T02:38:35.689 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-17T02:38:35.939 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T02:38:36.082 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-17T02:38:36.292 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T02:38:36.358 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-17T02:38:36.418 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T02:38:36.484 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-17T02:38:36.678 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T02:38:36.744 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-17T02:38:36.861 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-17T02:38:36.968 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T02:38:37.179 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-17T02:38:37.247 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T02:38:37.481 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-17T02:38:37.616 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T02:38:37.834 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-17T02:38:37.892 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T02:38:37.960 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-17T02:38:38.219 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T02:38:38.227 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-17T02:38:38.345 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:38.471 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:38.496 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-17T02:38:38.597 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T02:38:38.747 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-17T02:38:38.883 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:39.009 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:39.137 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T02:38:39.165 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-17T02:38:39.403 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:39.450 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-17T02:38:39.513 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:39.639 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:39.794 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-17T02:38:39.847 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T02:38:39.847 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T02:38:39.945 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up librados2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:40.071 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:40.198 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-17T02:38:40.290 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:40.391 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:40.509 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libcephfs2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:40.517 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:40.618 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libradosstriper1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:40.643 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:40.744 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-17T02:38:40.769 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T02:38:41.037 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up librbd1 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:41.163 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr-modules-core (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:41.289 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:41.505 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T02:38:41.505 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-17T02:38:41.923 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:42.008 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libcephfs-dev (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:42.049 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:42.133 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-rados (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:42.175 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:42.259 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up librgw2 (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:42.300 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:42.385 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up cephfs-top (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:42.426 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:42.503 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-17T02:38:42.536 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:43.028 INFO:teuthology.orchestra.run.smithi131.stdout:Adding group ceph....done 2024-09-17T02:38:43.405 INFO:teuthology.orchestra.run.smithi131.stdout:Adding system user ceph....done 2024-09-17T02:38:43.556 INFO:teuthology.orchestra.run.smithi131.stdout:Setting system user ceph properties....done 2024-09-17T02:38:43.570 INFO:teuthology.orchestra.run.smithi131.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T02:38:43.711 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T02:38:43.733 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-rbd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:43.857 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up rbd-fuse (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:43.983 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-rgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:44.055 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T02:38:44.109 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up python3-cephfs (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:44.235 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up cephfs-shell (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:44.353 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-common (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:44.586 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:44.779 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T02:38:44.779 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T02:38:44.871 INFO:teuthology.orchestra.run.smithi189.stdout:Adding group ceph....done 2024-09-17T02:38:45.206 INFO:teuthology.orchestra.run.smithi189.stdout:Adding system user ceph....done 2024-09-17T02:38:45.279 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:45.333 INFO:teuthology.orchestra.run.smithi189.stdout:Setting system user ceph properties....done 2024-09-17T02:38:45.353 INFO:teuthology.orchestra.run.smithi189.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-17T02:38:45.405 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:45.490 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-17T02:38:45.831 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-17T02:38:45.847 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T02:38:45.847 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T02:38:46.307 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:46.329 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up cephfs-mirror (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:46.536 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T02:38:46.537 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-17T02:38:46.644 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T02:38:47.131 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-test (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:47.439 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:47.569 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up radosgw (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:47.638 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T02:38:47.638 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T02:38:48.023 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T02:38:48.024 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-17T02:38:48.149 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:48.363 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T02:38:48.363 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T02:38:48.492 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-base (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:48.849 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-17T02:38:48.901 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:49.281 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T02:38:49.281 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T02:38:49.327 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mds (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:49.558 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T02:38:49.558 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-17T02:38:49.769 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:49.895 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:50.037 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:50.046 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:50.253 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T02:38:50.254 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-17T02:38:50.254 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T02:38:50.254 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T02:38:50.705 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-osd (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:50.781 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:50.932 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:50.970 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T02:38:50.971 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-17T02:38:51.058 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:51.184 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:51.449 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:51.520 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T02:38:51.558 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:51.621 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T02:38:51.717 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mon (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:51.765 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T02:38:51.928 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T02:38:51.929 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-17T02:38:52.385 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr-cephadm (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:52.536 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:52.550 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T02:38:52.662 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-mgr-dashboard (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:52.831 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ceph-volume (19.3.0-4989-gce1b342b-1jammy) ... 2024-09-17T02:38:53.183 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T02:38:53.193 INFO:teuthology.orchestra.run.smithi131.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T02:38:53.197 DEBUG:teuthology.orchestra.run.smithi131:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-17T02:38:53.251 INFO:teuthology.orchestra.run.smithi131.stdout:Reading package lists... 2024-09-17T02:38:53.276 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T02:38:53.428 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T02:38:53.468 INFO:teuthology.orchestra.run.smithi131.stdout:Building dependency tree... 2024-09-17T02:38:53.469 INFO:teuthology.orchestra.run.smithi131.stdout:Reading state information... 2024-09-17T02:38:53.679 INFO:teuthology.orchestra.run.smithi131.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-17T02:38:53.679 INFO:teuthology.orchestra.run.smithi131.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-17T02:38:53.679 INFO:teuthology.orchestra.run.smithi131.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-17T02:38:53.679 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T02:38:53.679 INFO:teuthology.orchestra.run.smithi131.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T02:38:53.679 INFO:teuthology.orchestra.run.smithi131.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T02:38:53.679 INFO:teuthology.orchestra.run.smithi131.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T02:38:53.680 INFO:teuthology.orchestra.run.smithi131.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T02:38:53.680 INFO:teuthology.orchestra.run.smithi131.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T02:38:53.680 INFO:teuthology.orchestra.run.smithi131.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T02:38:53.681 INFO:teuthology.orchestra.run.smithi131.stdout:The following additional packages will be installed: 2024-09-17T02:38:53.681 INFO:teuthology.orchestra.run.smithi131.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-17T02:38:53.681 INFO:teuthology.orchestra.run.smithi131.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-17T02:38:53.682 INFO:teuthology.orchestra.run.smithi131.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-17T02:38:53.682 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-17T02:38:53.682 INFO:teuthology.orchestra.run.smithi131.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T02:38:53.683 INFO:teuthology.orchestra.run.smithi131.stdout:Suggested packages: 2024-09-17T02:38:53.683 INFO:teuthology.orchestra.run.smithi131.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-17T02:38:53.683 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-17T02:38:53.683 INFO:teuthology.orchestra.run.smithi131.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-17T02:38:53.730 INFO:teuthology.orchestra.run.smithi131.stdout:The following NEW packages will be installed: 2024-09-17T02:38:53.731 INFO:teuthology.orchestra.run.smithi131.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-17T02:38:53.731 INFO:teuthology.orchestra.run.smithi131.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-17T02:38:53.731 INFO:teuthology.orchestra.run.smithi131.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-17T02:38:53.732 INFO:teuthology.orchestra.run.smithi131.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-17T02:38:53.732 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-17T02:38:53.732 INFO:teuthology.orchestra.run.smithi131.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-17T02:38:53.732 INFO:teuthology.orchestra.run.smithi131.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T02:38:53.733 INFO:teuthology.orchestra.run.smithi131.stdout:The following packages will be upgraded: 2024-09-17T02:38:53.734 INFO:teuthology.orchestra.run.smithi131.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-17T02:38:53.935 INFO:teuthology.orchestra.run.smithi131.stdout:4 upgraded, 39 newly installed, 0 to remove and 327 not upgraded. 2024-09-17T02:38:53.936 INFO:teuthology.orchestra.run.smithi131.stdout:Need to get 55.8 MB of archives. 2024-09-17T02:38:53.936 INFO:teuthology.orchestra.run.smithi131.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-17T02:38:53.936 INFO:teuthology.orchestra.run.smithi131.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-17T02:38:54.488 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T02:38:54.560 INFO:teuthology.orchestra.run.smithi131.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-17T02:38:54.576 INFO:teuthology.orchestra.run.smithi131.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-17T02:38:54.701 INFO:teuthology.orchestra.run.smithi131.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-17T02:38:54.702 INFO:teuthology.orchestra.run.smithi131.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-17T02:38:54.703 INFO:teuthology.orchestra.run.smithi131.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-17T02:38:54.731 INFO:teuthology.orchestra.run.smithi131.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-17T02:38:54.733 INFO:teuthology.orchestra.run.smithi131.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-17T02:38:54.737 INFO:teuthology.orchestra.run.smithi131.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-17T02:38:54.739 INFO:teuthology.orchestra.run.smithi131.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-17T02:38:54.740 INFO:teuthology.orchestra.run.smithi131.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-17T02:38:54.743 INFO:teuthology.orchestra.run.smithi131.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-17T02:38:54.804 INFO:teuthology.orchestra.run.smithi131.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-17T02:38:54.805 INFO:teuthology.orchestra.run.smithi131.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-17T02:38:54.805 INFO:teuthology.orchestra.run.smithi131.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-17T02:38:54.886 INFO:teuthology.orchestra.run.smithi131.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-17T02:38:55.115 INFO:teuthology.orchestra.run.smithi189.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T02:38:55.118 DEBUG:teuthology.orchestra.run.smithi189:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-17T02:38:55.176 INFO:teuthology.orchestra.run.smithi189.stdout:Reading package lists... 2024-09-17T02:38:55.388 INFO:teuthology.orchestra.run.smithi189.stdout:Building dependency tree... 2024-09-17T02:38:55.389 INFO:teuthology.orchestra.run.smithi189.stdout:Reading state information... 2024-09-17T02:38:55.595 INFO:teuthology.orchestra.run.smithi189.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-17T02:38:55.595 INFO:teuthology.orchestra.run.smithi189.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-17T02:38:55.596 INFO:teuthology.orchestra.run.smithi189.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-17T02:38:55.596 INFO:teuthology.orchestra.run.smithi189.stdout:The following packages were automatically installed and are no longer required: 2024-09-17T02:38:55.596 INFO:teuthology.orchestra.run.smithi189.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-17T02:38:55.596 INFO:teuthology.orchestra.run.smithi189.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-17T02:38:55.596 INFO:teuthology.orchestra.run.smithi189.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-17T02:38:55.596 INFO:teuthology.orchestra.run.smithi189.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-17T02:38:55.596 INFO:teuthology.orchestra.run.smithi189.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-17T02:38:55.596 INFO:teuthology.orchestra.run.smithi189.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-17T02:38:55.597 INFO:teuthology.orchestra.run.smithi189.stdout:The following additional packages will be installed: 2024-09-17T02:38:55.597 INFO:teuthology.orchestra.run.smithi189.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-17T02:38:55.597 INFO:teuthology.orchestra.run.smithi189.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-17T02:38:55.598 INFO:teuthology.orchestra.run.smithi189.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-17T02:38:55.598 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-17T02:38:55.598 INFO:teuthology.orchestra.run.smithi189.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T02:38:55.599 INFO:teuthology.orchestra.run.smithi189.stdout:Suggested packages: 2024-09-17T02:38:55.599 INFO:teuthology.orchestra.run.smithi189.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-17T02:38:55.599 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-17T02:38:55.600 INFO:teuthology.orchestra.run.smithi189.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-17T02:38:55.642 INFO:teuthology.orchestra.run.smithi189.stdout:The following NEW packages will be installed: 2024-09-17T02:38:55.642 INFO:teuthology.orchestra.run.smithi189.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-17T02:38:55.642 INFO:teuthology.orchestra.run.smithi189.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-17T02:38:55.643 INFO:teuthology.orchestra.run.smithi189.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-17T02:38:55.643 INFO:teuthology.orchestra.run.smithi189.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-17T02:38:55.643 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-17T02:38:55.643 INFO:teuthology.orchestra.run.smithi189.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-17T02:38:55.643 INFO:teuthology.orchestra.run.smithi189.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-17T02:38:55.644 INFO:teuthology.orchestra.run.smithi189.stdout:The following packages will be upgraded: 2024-09-17T02:38:55.645 INFO:teuthology.orchestra.run.smithi189.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-17T02:38:55.673 INFO:teuthology.orchestra.run.smithi131.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-17T02:38:55.677 INFO:teuthology.orchestra.run.smithi131.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-17T02:38:55.678 INFO:teuthology.orchestra.run.smithi131.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-17T02:38:55.684 INFO:teuthology.orchestra.run.smithi131.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-17T02:38:55.684 INFO:teuthology.orchestra.run.smithi131.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-17T02:38:55.693 INFO:teuthology.orchestra.run.smithi131.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-17T02:38:55.697 INFO:teuthology.orchestra.run.smithi131.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-17T02:38:55.712 INFO:teuthology.orchestra.run.smithi131.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-17T02:38:55.712 INFO:teuthology.orchestra.run.smithi131.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-17T02:38:55.775 INFO:teuthology.orchestra.run.smithi131.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-17T02:38:55.798 INFO:teuthology.orchestra.run.smithi131.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-17T02:38:55.820 INFO:teuthology.orchestra.run.smithi131.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-17T02:38:55.825 INFO:teuthology.orchestra.run.smithi189.stdout:4 upgraded, 39 newly installed, 0 to remove and 327 not upgraded. 2024-09-17T02:38:55.825 INFO:teuthology.orchestra.run.smithi189.stdout:Need to get 55.8 MB of archives. 2024-09-17T02:38:55.825 INFO:teuthology.orchestra.run.smithi189.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-17T02:38:55.825 INFO:teuthology.orchestra.run.smithi189.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-17T02:38:55.887 INFO:teuthology.orchestra.run.smithi131.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-17T02:38:55.893 INFO:teuthology.orchestra.run.smithi131.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-17T02:38:55.893 INFO:teuthology.orchestra.run.smithi131.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-17T02:38:55.894 INFO:teuthology.orchestra.run.smithi131.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-17T02:38:55.895 INFO:teuthology.orchestra.run.smithi131.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-17T02:38:55.931 INFO:teuthology.orchestra.run.smithi131.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-17T02:38:55.934 INFO:teuthology.orchestra.run.smithi131.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-17T02:38:56.445 INFO:teuthology.orchestra.run.smithi189.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-17T02:38:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-17T02:38:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-17T02:38:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-17T02:38:56.461 INFO:teuthology.orchestra.run.smithi189.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-17T02:38:56.496 INFO:teuthology.orchestra.run.smithi131.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-17T02:38:56.496 INFO:teuthology.orchestra.run.smithi131.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-17T02:38:56.496 INFO:teuthology.orchestra.run.smithi131.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-17T02:38:56.527 INFO:teuthology.orchestra.run.smithi131.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-17T02:38:56.528 INFO:teuthology.orchestra.run.smithi131.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-17T02:38:56.585 INFO:teuthology.orchestra.run.smithi189.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-17T02:38:56.586 INFO:teuthology.orchestra.run.smithi189.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-17T02:38:56.587 INFO:teuthology.orchestra.run.smithi189.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-17T02:38:56.615 INFO:teuthology.orchestra.run.smithi189.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-17T02:38:56.616 INFO:teuthology.orchestra.run.smithi189.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-17T02:38:56.620 INFO:teuthology.orchestra.run.smithi189.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-17T02:38:56.622 INFO:teuthology.orchestra.run.smithi189.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-17T02:38:56.623 INFO:teuthology.orchestra.run.smithi189.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-17T02:38:56.626 INFO:teuthology.orchestra.run.smithi189.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-17T02:38:56.688 INFO:teuthology.orchestra.run.smithi189.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-17T02:38:56.688 INFO:teuthology.orchestra.run.smithi189.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-17T02:38:56.689 INFO:teuthology.orchestra.run.smithi189.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-17T02:38:56.769 INFO:teuthology.orchestra.run.smithi189.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-17T02:38:56.943 INFO:teuthology.orchestra.run.smithi131.stdout:Preconfiguring packages ... 2024-09-17T02:38:57.104 INFO:teuthology.orchestra.run.smithi131.stdout:Fetched 55.8 MB in 3s (20.0 MB/s) 2024-09-17T02:38:57.405 INFO:teuthology.orchestra.run.smithi131.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176353 files and directories currently installed.) 2024-09-17T02:38:57.411 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T02:38:57.556 INFO:teuthology.orchestra.run.smithi189.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-17T02:38:57.556 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T02:38:57.560 INFO:teuthology.orchestra.run.smithi189.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-17T02:38:57.561 INFO:teuthology.orchestra.run.smithi189.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-17T02:38:57.567 INFO:teuthology.orchestra.run.smithi189.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-17T02:38:57.568 INFO:teuthology.orchestra.run.smithi189.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-17T02:38:57.579 INFO:teuthology.orchestra.run.smithi189.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-17T02:38:57.583 INFO:teuthology.orchestra.run.smithi189.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-17T02:38:57.599 INFO:teuthology.orchestra.run.smithi189.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-17T02:38:57.600 INFO:teuthology.orchestra.run.smithi189.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-17T02:38:57.660 INFO:teuthology.orchestra.run.smithi189.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-17T02:38:57.685 INFO:teuthology.orchestra.run.smithi189.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-17T02:38:57.707 INFO:teuthology.orchestra.run.smithi189.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-17T02:38:57.775 INFO:teuthology.orchestra.run.smithi189.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-17T02:38:57.780 INFO:teuthology.orchestra.run.smithi189.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-17T02:38:57.781 INFO:teuthology.orchestra.run.smithi189.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-17T02:38:57.782 INFO:teuthology.orchestra.run.smithi189.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-17T02:38:57.783 INFO:teuthology.orchestra.run.smithi189.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-17T02:38:57.806 INFO:teuthology.orchestra.run.smithi189.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-17T02:38:57.809 INFO:teuthology.orchestra.run.smithi189.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-17T02:38:58.195 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-17T02:38:58.366 INFO:teuthology.orchestra.run.smithi189.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-17T02:38:58.366 INFO:teuthology.orchestra.run.smithi189.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-17T02:38:58.366 INFO:teuthology.orchestra.run.smithi189.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-17T02:38:58.366 INFO:teuthology.orchestra.run.smithi189.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-17T02:38:58.367 INFO:teuthology.orchestra.run.smithi189.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-17T02:38:58.367 INFO:teuthology.orchestra.run.smithi189.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-17T02:38:58.395 INFO:teuthology.orchestra.run.smithi189.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-17T02:38:58.396 INFO:teuthology.orchestra.run.smithi189.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-17T02:38:58.810 INFO:teuthology.orchestra.run.smithi189.stdout:Preconfiguring packages ... 2024-09-17T02:38:58.958 INFO:teuthology.orchestra.run.smithi189.stdout:Fetched 55.8 MB in 3s (20.3 MB/s) 2024-09-17T02:38:59.168 INFO:teuthology.orchestra.run.smithi131.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-17T02:38:59.301 INFO:teuthology.orchestra.run.smithi189.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176353 files and directories currently installed.) 2024-09-17T02:38:59.307 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T02:38:59.437 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package flex. 2024-09-17T02:38:59.452 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T02:38:59.539 INFO:teuthology.orchestra.run.smithi131.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176353 files and directories currently installed.) 2024-09-17T02:38:59.544 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-17T02:38:59.585 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-17T02:39:00.090 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-17T02:39:00.127 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T02:39:00.292 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-17T02:39:00.685 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T02:39:01.043 INFO:teuthology.orchestra.run.smithi189.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-17T02:39:01.214 INFO:teuthology.orchestra.run.smithi131.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176426 files and directories currently installed.) 2024-09-17T02:39:01.219 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T02:39:01.307 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package flex. 2024-09-17T02:39:01.341 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T02:39:01.378 INFO:teuthology.orchestra.run.smithi189.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176353 files and directories currently installed.) 2024-09-17T02:39:01.384 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-17T02:39:01.431 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-17T02:39:01.759 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T02:39:01.889 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T02:39:01.931 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T02:39:02.112 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-17T02:39:02.217 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T02:39:02.506 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T02:39:02.574 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package bison. 2024-09-17T02:39:02.679 INFO:teuthology.orchestra.run.smithi131.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176426 files and directories currently installed.) 2024-09-17T02:39:02.684 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-17T02:39:02.734 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T02:39:03.135 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-17T02:39:03.163 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-17T02:39:03.202 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T02:39:03.203 INFO:teuthology.orchestra.run.smithi189.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176426 files and directories currently installed.) 2024-09-17T02:39:03.209 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T02:39:03.346 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T02:39:03.497 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package dump. 2024-09-17T02:39:03.524 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-17T02:39:03.555 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking dump (0.4b47-3) ... 2024-09-17T02:39:03.764 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-17T02:39:03.834 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package indent. 2024-09-17T02:39:03.850 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:03.892 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T02:39:03.902 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-17T02:39:04.153 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-17T02:39:04.170 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-17T02:39:04.211 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T02:39:04.229 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T02:39:04.490 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-17T02:39:04.507 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-17T02:39:04.540 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T02:39:04.544 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package bison. 2024-09-17T02:39:04.641 INFO:teuthology.orchestra.run.smithi189.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176426 files and directories currently installed.) 2024-09-17T02:39:04.647 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-17T02:39:04.688 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T02:39:04.835 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-17T02:39:04.864 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-17T02:39:04.901 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T02:39:05.039 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-17T02:39:05.067 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-17T02:39:05.097 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T02:39:05.238 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-17T02:39:05.267 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-17T02:39:05.305 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T02:39:05.434 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package dump. 2024-09-17T02:39:05.462 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-17T02:39:05.517 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking dump (0.4b47-3) ... 2024-09-17T02:39:05.532 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-17T02:39:05.549 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-17T02:39:05.582 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T02:39:05.813 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package indent. 2024-09-17T02:39:05.819 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libjson-perl. 2024-09-17T02:39:05.835 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-17T02:39:05.841 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:05.877 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-17T02:39:05.879 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T02:39:06.189 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-17T02:39:06.215 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-17T02:39:06.217 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-17T02:39:06.232 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-17T02:39:06.255 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T02:39:06.265 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T02:39:06.594 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-17T02:39:06.622 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-17T02:39:06.661 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T02:39:06.939 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-17T02:39:06.967 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-17T02:39:07.005 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T02:39:07.309 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-17T02:39:07.337 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-17T02:39:07.367 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T02:39:07.603 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-17T02:39:07.619 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-17T02:39:07.661 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T02:39:07.922 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libjson-perl. 2024-09-17T02:39:07.925 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-17T02:39:07.939 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-17T02:39:07.953 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T02:39:07.981 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-17T02:39:07.992 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T02:39:08.228 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-17T02:39:08.256 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T02:39:08.294 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T02:39:08.318 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-17T02:39:08.346 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-17T02:39:08.384 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T02:39:08.622 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-17T02:39:08.650 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T02:39:08.689 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T02:39:09.034 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-17T02:39:09.062 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-17T02:39:09.109 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T02:39:09.412 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-17T02:39:09.439 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T02:39:09.478 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T02:39:09.823 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-17T02:39:09.851 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:09.889 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:09.979 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-17T02:39:10.007 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T02:39:10.059 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T02:39:10.235 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-17T02:39:10.263 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:10.290 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-17T02:39:10.301 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T02:39:10.306 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-17T02:39:10.348 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T02:39:10.612 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-17T02:39:10.641 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:10.679 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T02:39:10.692 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-17T02:39:10.721 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T02:39:10.759 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T02:39:10.923 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-17T02:39:10.939 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T02:39:10.981 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T02:39:11.054 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-17T02:39:11.082 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-17T02:39:11.120 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T02:39:11.440 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-17T02:39:11.452 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-17T02:39:11.468 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-17T02:39:11.469 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T02:39:11.502 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T02:39:11.506 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T02:39:11.838 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package modemmanager. 2024-09-17T02:39:11.851 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-17T02:39:11.866 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T02:39:11.880 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:11.905 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T02:39:11.918 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:12.238 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-17T02:39:12.266 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:12.304 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T02:39:12.579 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package network-manager. 2024-09-17T02:39:12.607 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T02:39:12.624 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-17T02:39:12.652 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:12.653 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-17T02:39:12.690 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T02:39:12.936 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-17T02:39:12.962 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-17T02:39:13.001 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T02:39:13.345 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package ppp. 2024-09-17T02:39:13.373 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-17T02:39:13.581 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-17T02:39:13.609 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-17T02:39:13.647 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T02:39:13.648 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T02:39:13.967 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package modemmanager. 2024-09-17T02:39:13.995 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-17T02:39:14.008 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package pptp-linux. 2024-09-17T02:39:14.033 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T02:39:14.036 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-17T02:39:14.075 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-17T02:39:14.328 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-17T02:39:14.344 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-17T02:39:14.386 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-17T02:39:14.689 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-17T02:39:14.691 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package network-manager. 2024-09-17T02:39:14.717 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-17T02:39:14.719 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-17T02:39:14.756 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-client-common (238) ... 2024-09-17T02:39:14.766 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-17T02:39:15.068 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-17T02:39:15.097 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:15.135 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:15.340 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package ppp. 2024-09-17T02:39:15.368 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-17T02:39:15.645 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T02:39:15.740 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-common. 2024-09-17T02:39:15.768 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-17T02:39:15.825 INFO:teuthology.orchestra.run.smithi131.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-17T02:39:15.854 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-common (238) ... 2024-09-17T02:39:15.970 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package pptp-linux. 2024-09-17T02:39:15.998 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-17T02:39:16.036 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-17T02:39:16.306 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-17T02:39:16.322 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-17T02:39:16.330 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-14. 2024-09-17T02:39:16.356 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-17T02:39:16.359 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:16.392 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:16.600 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-17T02:39:16.617 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-17T02:39:16.650 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking postgresql-client-common (238) ... 2024-09-17T02:39:16.954 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-17T02:39:16.982 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:17.020 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:17.584 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package postgresql-common. 2024-09-17T02:39:17.612 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-17T02:39:17.668 INFO:teuthology.orchestra.run.smithi189.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-17T02:39:17.692 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql. 2024-09-17T02:39:17.698 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking postgresql-common (238) ... 2024-09-17T02:39:17.722 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-17T02:39:17.767 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql (14+238) ... 2024-09-17T02:39:18.149 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package postgresql-14. 2024-09-17T02:39:18.178 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-17T02:39:18.219 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:18.453 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-client. 2024-09-17T02:39:18.482 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-17T02:39:18.519 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-client (14+238) ... 2024-09-17T02:39:18.763 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-17T02:39:18.780 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-17T02:39:18.822 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-17T02:39:19.141 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package tcl8.6. 2024-09-17T02:39:19.170 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-17T02:39:19.207 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T02:39:19.536 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package tcl. 2024-09-17T02:39:19.564 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-17T02:39:19.607 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-17T02:39:19.920 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package postgresql. 2024-09-17T02:39:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package wpasupplicant. 2024-09-17T02:39:19.950 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-17T02:39:19.967 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-17T02:39:19.986 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking postgresql (14+238) ... 2024-09-17T02:39:20.005 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T02:39:20.213 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package postgresql-client. 2024-09-17T02:39:20.230 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-17T02:39:20.274 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking postgresql-client (14+238) ... 2024-09-17T02:39:20.377 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-17T02:39:20.405 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-17T02:39:20.443 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-17T02:39:20.509 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-17T02:39:20.526 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-17T02:39:20.567 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-17T02:39:20.837 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package tcl8.6. 2024-09-17T02:39:20.838 INFO:teuthology.orchestra.run.smithi131.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-17T02:39:20.866 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-17T02:39:20.866 INFO:teuthology.orchestra.run.smithi131.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-17T02:39:20.903 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T02:39:20.914 INFO:teuthology.orchestra.run.smithi131.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T02:39:21.189 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package tcl. 2024-09-17T02:39:21.209 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-client-common (238) ... 2024-09-17T02:39:21.218 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-17T02:39:21.261 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-17T02:39:21.428 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-17T02:39:21.554 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T02:39:21.609 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package wpasupplicant. 2024-09-17T02:39:21.638 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-17T02:39:21.655 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:21.675 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T02:39:21.756 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T02:39:21.882 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T02:39:22.005 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-17T02:39:22.008 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:22.021 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-17T02:39:22.063 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-17T02:39:22.475 INFO:teuthology.orchestra.run.smithi189.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-17T02:39:22.503 INFO:teuthology.orchestra.run.smithi189.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-17T02:39:22.512 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-17T02:39:22.550 INFO:teuthology.orchestra.run.smithi189.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T02:39:22.606 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T02:39:22.838 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up postgresql-client-common (238) ... 2024-09-17T02:39:22.983 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T02:39:22.983 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T02:39:23.047 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-17T02:39:23.165 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-17T02:39:23.283 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:23.425 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-17T02:39:23.492 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T02:39:23.551 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-17T02:39:23.686 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:24.044 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T02:39:24.170 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T02:39:24.211 INFO:teuthology.orchestra.run.smithi189.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-17T02:39:24.296 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-17T02:39:24.308 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-17T02:39:24.422 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T02:39:24.547 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T02:39:24.665 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T02:39:24.688 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T02:39:24.688 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-17T02:39:24.766 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T02:39:25.194 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-17T02:39:25.321 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-17T02:39:25.405 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T02:39:25.446 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-17T02:39:25.531 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up dump (0.4b47-3) ... 2024-09-17T02:39:25.572 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-17T02:39:25.625 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-17T02:39:25.682 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-17T02:39:25.691 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T02:39:25.792 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T02:39:25.808 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-17T02:39:25.918 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-17T02:39:25.934 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-17T02:39:26.044 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-17T02:39:26.085 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-17T02:39:26.170 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T02:39:26.260 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-17T02:39:26.322 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T02:39:26.448 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T02:39:26.616 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T02:39:26.717 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T02:39:26.723 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-17T02:39:26.818 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-17T02:39:26.841 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up dump (0.4b47-3) ... 2024-09-17T02:39:26.935 INFO:teuthology.orchestra.run.smithi189.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-17T02:39:26.995 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-client (14+238) ... 2024-09-17T02:39:27.001 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T02:39:27.121 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-17T02:39:27.127 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-17T02:39:27.254 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-17T02:39:27.379 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-17T02:39:27.505 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-17T02:39:27.581 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:27.582 INFO:teuthology.orchestra.run.smithi131.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-17T02:39:27.582 INFO:teuthology.orchestra.run.smithi131.stdout:which means they are currently configured by ifupdown: 2024-09-17T02:39:27.582 INFO:teuthology.orchestra.run.smithi131.stdout:- enp3s0f1 2024-09-17T02:39:27.582 INFO:teuthology.orchestra.run.smithi131.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-17T02:39:27.582 INFO:teuthology.orchestra.run.smithi131.stdout:remove their configuration from /etc/network/interfaces. 2024-09-17T02:39:27.582 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:27.596 INFO:teuthology.orchestra.run.smithi189.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-17T02:39:27.657 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-17T02:39:27.693 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-17T02:39:27.775 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-17T02:39:27.951 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-17T02:39:28.015 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-17T02:39:28.077 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T02:39:28.203 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-17T02:39:28.340 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-17T02:39:28.371 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up postgresql-client (14+238) ... 2024-09-17T02:39:28.497 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-17T02:39:28.831 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T02:39:28.983 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T02:39:29.109 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-17T02:39:29.110 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:29.110 INFO:teuthology.orchestra.run.smithi189.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-17T02:39:29.110 INFO:teuthology.orchestra.run.smithi189.stdout:which means they are currently configured by ifupdown: 2024-09-17T02:39:29.110 INFO:teuthology.orchestra.run.smithi189.stdout:- enp3s0f1 2024-09-17T02:39:29.110 INFO:teuthology.orchestra.run.smithi189.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-17T02:39:29.110 INFO:teuthology.orchestra.run.smithi189.stdout:remove their configuration from /etc/network/interfaces. 2024-09-17T02:39:29.110 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:29.245 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-17T02:39:29.252 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-common (238) ... 2024-09-17T02:39:29.566 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-17T02:39:29.881 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-17T02:39:30.120 INFO:teuthology.orchestra.run.smithi131.stdout:Adding user postgres to group ssl-cert 2024-09-17T02:39:30.333 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:30.333 INFO:teuthology.orchestra.run.smithi131.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-17T02:39:30.408 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-17T02:39:30.518 INFO:teuthology.orchestra.run.smithi131.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-17T02:39:30.519 INFO:teuthology.orchestra.run.smithi131.stdout:Removing obsolete dictionary files: 2024-09-17T02:39:30.543 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-17T02:39:30.669 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-17T02:39:30.795 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up postgresql-common (238) ... 2024-09-17T02:39:31.085 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-17T02:39:31.462 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:31.773 INFO:teuthology.orchestra.run.smithi131.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-17T02:39:31.796 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-17T02:39:31.837 INFO:teuthology.orchestra.run.smithi131.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-17T02:39:31.838 INFO:teuthology.orchestra.run.smithi131.stdout:This user must also own the server process. 2024-09-17T02:39:31.838 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:31.838 INFO:teuthology.orchestra.run.smithi131.stdout:The database cluster will be initialized with locale "en_US". 2024-09-17T02:39:31.838 INFO:teuthology.orchestra.run.smithi131.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-17T02:39:31.838 INFO:teuthology.orchestra.run.smithi131.stdout:The default text search configuration will be set to "english". 2024-09-17T02:39:31.838 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:31.838 INFO:teuthology.orchestra.run.smithi131.stdout:Data page checksums are disabled. 2024-09-17T02:39:31.838 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:31.838 INFO:teuthology.orchestra.run.smithi131.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-17T02:39:31.838 INFO:teuthology.orchestra.run.smithi131.stdout:creating subdirectories ... ok 2024-09-17T02:39:31.839 INFO:teuthology.orchestra.run.smithi131.stdout:selecting dynamic shared memory implementation ... posix 2024-09-17T02:39:31.853 INFO:teuthology.orchestra.run.smithi131.stdout:selecting default max_connections ... 100 2024-09-17T02:39:31.872 INFO:teuthology.orchestra.run.smithi131.stdout:selecting default shared_buffers ... 128MB 2024-09-17T02:39:31.877 INFO:teuthology.orchestra.run.smithi189.stdout:Adding user postgres to group ssl-cert 2024-09-17T02:39:31.883 INFO:teuthology.orchestra.run.smithi131.stdout:selecting default time zone ... Etc/UTC 2024-09-17T02:39:31.884 INFO:teuthology.orchestra.run.smithi131.stdout:creating configuration files ... ok 2024-09-17T02:39:31.997 INFO:teuthology.orchestra.run.smithi131.stdout:running bootstrap script ... ok 2024-09-17T02:39:32.099 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:32.099 INFO:teuthology.orchestra.run.smithi189.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-17T02:39:32.296 INFO:teuthology.orchestra.run.smithi189.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-17T02:39:32.296 INFO:teuthology.orchestra.run.smithi189.stdout:Removing obsolete dictionary files: 2024-09-17T02:39:32.648 INFO:teuthology.orchestra.run.smithi131.stdout:performing post-bootstrap initialization ... ok 2024-09-17T02:39:32.848 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-17T02:39:33.230 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-17T02:39:33.241 INFO:teuthology.orchestra.run.smithi131.stdout:syncing data to disk ... ok 2024-09-17T02:39:33.559 INFO:teuthology.orchestra.run.smithi189.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-17T02:39:33.579 INFO:teuthology.orchestra.run.smithi189.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-17T02:39:33.624 INFO:teuthology.orchestra.run.smithi189.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-17T02:39:33.624 INFO:teuthology.orchestra.run.smithi189.stdout:This user must also own the server process. 2024-09-17T02:39:33.624 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:33.624 INFO:teuthology.orchestra.run.smithi189.stdout:The database cluster will be initialized with locale "en_US". 2024-09-17T02:39:33.624 INFO:teuthology.orchestra.run.smithi189.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-17T02:39:33.624 INFO:teuthology.orchestra.run.smithi189.stdout:The default text search configuration will be set to "english". 2024-09-17T02:39:33.624 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:33.624 INFO:teuthology.orchestra.run.smithi189.stdout:Data page checksums are disabled. 2024-09-17T02:39:33.625 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:33.625 INFO:teuthology.orchestra.run.smithi189.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-17T02:39:33.625 INFO:teuthology.orchestra.run.smithi189.stdout:creating subdirectories ... ok 2024-09-17T02:39:33.625 INFO:teuthology.orchestra.run.smithi189.stdout:selecting dynamic shared memory implementation ... posix 2024-09-17T02:39:33.628 INFO:teuthology.orchestra.run.smithi131.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-17T02:39:33.637 INFO:teuthology.orchestra.run.smithi189.stdout:selecting default max_connections ... 100 2024-09-17T02:39:33.658 INFO:teuthology.orchestra.run.smithi189.stdout:selecting default shared_buffers ... 128MB 2024-09-17T02:39:33.667 INFO:teuthology.orchestra.run.smithi189.stdout:selecting default time zone ... Etc/UTC 2024-09-17T02:39:33.668 INFO:teuthology.orchestra.run.smithi189.stdout:creating configuration files ... ok 2024-09-17T02:39:33.782 INFO:teuthology.orchestra.run.smithi189.stdout:running bootstrap script ... ok 2024-09-17T02:39:34.418 INFO:teuthology.orchestra.run.smithi189.stdout:performing post-bootstrap initialization ... ok 2024-09-17T02:39:35.040 INFO:teuthology.orchestra.run.smithi189.stdout:syncing data to disk ... ok 2024-09-17T02:39:35.442 INFO:teuthology.orchestra.run.smithi189.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-17T02:39:36.404 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T02:39:36.522 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-17T02:39:36.649 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up postgresql (14+238) ... 2024-09-17T02:39:36.774 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T02:39:36.925 INFO:teuthology.orchestra.run.smithi131.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T02:39:37.185 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T02:39:37.185 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T02:39:37.744 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T02:39:37.845 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-17T02:39:38.131 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T02:39:38.228 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T02:39:38.291 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T02:39:38.354 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-17T02:39:38.480 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up postgresql (14+238) ... 2024-09-17T02:39:38.597 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-17T02:39:38.732 INFO:teuthology.orchestra.run.smithi189.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-17T02:39:38.976 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T02:39:38.976 INFO:teuthology.orchestra.run.smithi189.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-17T02:39:39.518 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-17T02:39:39.602 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-17T02:39:39.610 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-17T02:39:39.711 INFO:teuthology.orchestra.run.smithi131.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T02:39:39.921 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-17T02:39:40.073 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-17T02:39:40.209 INFO:teuthology.orchestra.run.smithi131.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T02:39:40.214 DEBUG:teuthology.parallel:result is None 2024-09-17T02:39:41.351 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-17T02:39:41.459 INFO:teuthology.orchestra.run.smithi189.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-17T02:39:41.920 INFO:teuthology.orchestra.run.smithi189.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-17T02:39:41.925 DEBUG:teuthology.parallel:result is None 2024-09-17T02:39:41.925 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T02:39:42.069 DEBUG:teuthology.orchestra.run.smithi131:> dpkg-query -W -f '${Version}' ceph 2024-09-17T02:39:42.088 INFO:teuthology.orchestra.run.smithi131.stdout:19.3.0-4989-gce1b342b-1jammy 2024-09-17T02:39:42.089 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T02:39:42.089 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989-gce1b342b-1jammy is installed. 2024-09-17T02:39:42.091 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T02:39:42.232 DEBUG:teuthology.orchestra.run.smithi189:> dpkg-query -W -f '${Version}' ceph 2024-09-17T02:39:42.251 INFO:teuthology.orchestra.run.smithi189.stdout:19.3.0-4989-gce1b342b-1jammy 2024-09-17T02:39:42.251 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989-gce1b342b-1jammy 2024-09-17T02:39:42.251 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989-gce1b342b-1jammy is installed. 2024-09-17T02:39:42.253 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T02:39:42.254 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:39:42.254 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T02:39:42.268 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:39:42.268 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T02:39:42.306 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T02:39:42.306 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:39:42.306 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T02:39:42.325 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T02:39:42.380 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:39:42.380 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T02:39:42.394 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T02:39:42.449 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T02:39:42.449 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:39:42.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T02:39:42.462 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T02:39:42.517 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:39:42.517 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T02:39:42.529 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T02:39:42.585 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T02:39:42.585 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:39:42.585 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T02:39:42.598 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T02:39:42.653 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:39:42.653 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T02:39:42.667 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T02:39:42.720 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T02:39:42.800 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T02:39:42.801 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /var/log/ceph 2024-09-17T02:39:42.803 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 777 /var/log/ceph 2024-09-17T02:39:42.816 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T02:39:42.816 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T02:39:42.859 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T02:39:42.871 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T02:39:42.872 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T02:39:42.918 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T02:39:42.936 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-17T02:39:42.936 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-17T02:39:42.937 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-17T02:39:42.937 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:39:42.937 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T02:39:42.977 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T02:39:43.054 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:39:43.054 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T02:39:43.060 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T02:39:43.138 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T02:39:43.138 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T02:39:43.138 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354', 'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/forward-scrub}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7908354', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240916.200549-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 476, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'sleep_before_teardown': 0, 'subset': '320/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240916.200549-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi131.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEVJGVwn+bFMVh1aaHw3OXUAPc4kNfoLIOKr/OL4XxkyscHKWWoihRgeFVXdf/cbNESwKJ0DzyTVZPsyZFrdhCQ=', 'smithi189.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDLRkqELI6tuDKtYxD0/Pwx0mR+tqTRrgAidtPxFlaUpbYnbT4T48yM71nEYZTodL8HnZDNwln0GVfnNoOTwEow='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_02:09:51', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T02:39:43.139 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T02:39:43.141 DEBUG:teuthology.orchestra.run.smithi189:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T02:39:43.182 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T02:39:43.190 DEBUG:teuthology.orchestra.run.smithi189:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T02:39:43.238 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:39:43.238 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T02:39:43.249 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T02:39:43.250 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-09-17T02:39:43.297 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T02:39:43.297 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T02:39:43.298 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 877 Links: 1 2024-09-17T02:39:43.298 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T02:39:43.298 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-17 02:33:26.339317856 +0000 2024-09-17T02:39:43.298 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-17 02:33:26.083320141 +0000 2024-09-17T02:39:43.298 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-17 02:33:26.083320141 +0000 2024-09-17T02:39:43.298 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2024-09-17T02:39:43.298 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T02:39:43.352 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-17T02:39:43.352 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-17T02:39:43.352 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000370531 s, 1.4 MB/s 2024-09-17T02:39:43.353 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T02:39:43.402 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-09-17T02:39:43.450 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T02:39:43.450 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T02:39:43.450 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 895 Links: 1 2024-09-17T02:39:43.450 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T02:39:43.450 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-17 02:33:26.795313783 +0000 2024-09-17T02:39:43.450 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-17 02:33:26.575315748 +0000 2024-09-17T02:39:43.450 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-17 02:33:26.575315748 +0000 2024-09-17T02:39:43.450 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2024-09-17T02:39:43.450 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T02:39:43.503 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-17T02:39:43.503 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-17T02:39:43.503 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000354235 s, 1.4 MB/s 2024-09-17T02:39:43.504 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T02:39:43.554 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-09-17T02:39:43.601 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T02:39:43.601 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T02:39:43.601 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 914 Links: 1 2024-09-17T02:39:43.601 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T02:39:43.601 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-17 02:33:27.287309388 +0000 2024-09-17T02:39:43.601 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-17 02:33:27.051311496 +0000 2024-09-17T02:39:43.601 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-17 02:33:27.051311496 +0000 2024-09-17T02:39:43.602 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2024-09-17T02:39:43.602 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T02:39:43.655 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-17T02:39:43.656 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-17T02:39:43.656 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000402683 s, 1.3 MB/s 2024-09-17T02:39:43.657 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T02:39:43.706 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-09-17T02:39:43.753 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T02:39:43.753 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T02:39:43.753 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 929 Links: 1 2024-09-17T02:39:43.753 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T02:39:43.753 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-17 02:33:27.779304992 +0000 2024-09-17T02:39:43.753 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-17 02:33:27.535307173 +0000 2024-09-17T02:39:43.753 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-17 02:33:27.535307173 +0000 2024-09-17T02:39:43.754 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: - 2024-09-17T02:39:43.754 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T02:39:43.806 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-17T02:39:43.806 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-17T02:39:43.807 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000312929 s, 1.6 MB/s 2024-09-17T02:39:43.808 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T02:39:43.854 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T02:39:43.854 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:39:43.854 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T02:39:43.860 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T02:39:43.861 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_1 2024-09-17T02:39:43.906 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T02:39:43.906 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T02:39:43.906 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 880 Links: 1 2024-09-17T02:39:43.907 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T02:39:43.907 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-17 02:33:25.262757358 +0000 2024-09-17T02:39:43.907 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-17 02:33:25.018759537 +0000 2024-09-17T02:39:43.907 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-17 02:33:25.018759537 +0000 2024-09-17T02:39:43.907 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-09-17T02:39:43.907 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T02:39:43.961 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-17T02:39:43.961 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-17T02:39:43.961 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000425408 s, 1.2 MB/s 2024-09-17T02:39:43.962 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T02:39:44.012 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_2 2024-09-17T02:39:44.062 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T02:39:44.062 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T02:39:44.062 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 896 Links: 1 2024-09-17T02:39:44.062 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T02:39:44.062 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-17 02:33:25.502755214 +0000 2024-09-17T02:39:44.063 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-17 02:33:25.498755250 +0000 2024-09-17T02:39:44.063 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-17 02:33:25.498755250 +0000 2024-09-17T02:39:44.063 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-09-17T02:39:44.063 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T02:39:44.117 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-17T02:39:44.117 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-17T02:39:44.117 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000451283 s, 1.1 MB/s 2024-09-17T02:39:44.118 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T02:39:44.168 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_3 2024-09-17T02:39:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T02:39:44.214 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T02:39:44.215 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 914 Links: 1 2024-09-17T02:39:44.215 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T02:39:44.215 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-17 02:33:26.254748496 +0000 2024-09-17T02:39:44.215 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-17 02:33:25.990750854 +0000 2024-09-17T02:39:44.215 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-17 02:33:25.990750854 +0000 2024-09-17T02:39:44.215 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-09-17T02:39:44.215 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T02:39:44.268 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-17T02:39:44.268 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-17T02:39:44.269 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000313149 s, 1.6 MB/s 2024-09-17T02:39:44.270 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T02:39:44.319 DEBUG:teuthology.orchestra.run.smithi189:> stat /dev/vg_nvme/lv_4 2024-09-17T02:39:44.366 INFO:teuthology.orchestra.run.smithi189.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T02:39:44.366 INFO:teuthology.orchestra.run.smithi189.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T02:39:44.366 INFO:teuthology.orchestra.run.smithi189.stdout:Device: 5h/5d Inode: 931 Links: 1 2024-09-17T02:39:44.366 INFO:teuthology.orchestra.run.smithi189.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T02:39:44.366 INFO:teuthology.orchestra.run.smithi189.stdout:Access: 2024-09-17 02:33:26.738744174 +0000 2024-09-17T02:39:44.366 INFO:teuthology.orchestra.run.smithi189.stdout:Modify: 2024-09-17 02:33:26.486746424 +0000 2024-09-17T02:39:44.366 INFO:teuthology.orchestra.run.smithi189.stdout:Change: 2024-09-17 02:33:26.486746424 +0000 2024-09-17T02:39:44.367 INFO:teuthology.orchestra.run.smithi189.stdout: Birth: - 2024-09-17T02:39:44.367 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T02:39:44.421 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records in 2024-09-17T02:39:44.421 INFO:teuthology.orchestra.run.smithi189.stderr:1+0 records out 2024-09-17T02:39:44.421 INFO:teuthology.orchestra.run.smithi189.stderr:512 bytes copied, 0.000388441 s, 1.3 MB/s 2024-09-17T02:39:44.422 DEBUG:teuthology.orchestra.run.smithi189:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T02:39:44.471 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T02:39:44.471 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi189.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T02:39:44.471 INFO:tasks.ceph:Generating config... 2024-09-17T02:39:44.473 INFO:tasks.ceph:[client] pid file = 2024-09-17T02:39:44.473 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T02:39:44.473 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T02:39:44.473 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T02:39:44.473 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T02:39:44.473 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T02:39:44.473 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-17T02:39:44.473 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T02:39:44.473 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T02:39:44.473 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-17T02:39:44.474 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-17T02:39:44.474 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T02:39:44.474 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T02:39:44.474 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-17T02:39:44.474 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T02:39:44.474 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T02:39:44.474 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T02:39:44.474 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T02:39:44.474 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T02:39:44.474 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T02:39:44.474 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T02:39:44.541 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T02:39:44.561 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T02:39:44.627 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T02:39:44.637 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.131'), ('mon.b', '172.21.15.189'), ('mon.c', '[v2:172.21.15.189:3301,v1:172.21.15.189:6790]')] 2024-09-17T02:39:44.637 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.131,172.21.15.189,[v2:172.21.15.189:3301,v1:172.21.15.189:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-17T02:39:44.638 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:39:44.638 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T02:39:44.684 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.131 --add b 172.21.15.189 --addv c '[v2:172.21.15.189:3301,v1:172.21.15.189:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:39:44.793 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:39:44.793 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: generated fsid f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:39:44.794 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = quincy 2024-09-17T02:39:44.794 INFO:teuthology.orchestra.run.smithi131.stdout:epoch 0 2024-09-17T02:39:44.794 INFO:teuthology.orchestra.run.smithi131.stdout:fsid f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:39:44.794 INFO:teuthology.orchestra.run.smithi131.stdout:last_changed 2024-09-17T02:39:44.800110+0000 2024-09-17T02:39:44.794 INFO:teuthology.orchestra.run.smithi131.stdout:created 2024-09-17T02:39:44.800110+0000 2024-09-17T02:39:44.794 INFO:teuthology.orchestra.run.smithi131.stdout:min_mon_release 17 (quincy) 2024-09-17T02:39:44.794 INFO:teuthology.orchestra.run.smithi131.stdout:election_strategy: 1 2024-09-17T02:39:44.794 INFO:teuthology.orchestra.run.smithi131.stdout:0: [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] mon.a 2024-09-17T02:39:44.794 INFO:teuthology.orchestra.run.smithi131.stdout:1: [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] mon.b 2024-09-17T02:39:44.794 INFO:teuthology.orchestra.run.smithi131.stdout:2: [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] mon.c 2024-09-17T02:39:44.794 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T02:39:44.799 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T02:39:44.804 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f03002b8-e723-4674-9385-a37ff83176c9... 2024-09-17T02:39:44.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T02:39:44.846 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T02:39:44.872 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: chdir = "" 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: auth supported = cephx 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: filestore xattr use omap = true 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: mon clock drift allowed = 1.000 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: osd crush chooseleaf type = 0 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: auth debug = true 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.873 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on old message = true 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on bug = true 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: mon pg warn max object skew = 0 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default size = 2 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow primary affinity = true 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow pg remap = true 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on legacy crush tunables = false 2024-09-17T02:39:44.874 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on crush straw calc version zero = false 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on no sortbitwise = false 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on osd down out interval zero = false 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on too few osds = false 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: mon_allow_pool_size_one = true 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: osd default data pool replay window = 5 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: mon allow pool delete = true 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.875 INFO:teuthology.orchestra.run.smithi131.stdout: mon cluster log file level = debug 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: debug asserts on shutdown = true 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: mon health detail to clog = false 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: mon host = "172.21.15.131,172.21.15.189,[v2:172.21.15.189:3301,v1:172.21.15.189:6790]" 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout:[osd] 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: osd journal size = 100 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub load threshold = 5.0 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub max interval = 600 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: osd mclock profile = high_recovery_ops 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: osd recover clone overlap = true 2024-09-17T02:39:44.876 INFO:teuthology.orchestra.run.smithi131.stdout: osd recovery max chunk = 1048576 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug shutdown = true 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug op order = true 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify stray on activate = true 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug trim objects = true 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: osd open classes on start = true 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug pg log writeout = true 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: osd deep scrub update digest min age = 30 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: osd map max advance = 10 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.877 INFO:teuthology.orchestra.run.smithi131.stdout: journal zero on create = true 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: filestore ondisk finisher threads = 3 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: filestore apply finisher threads = 3 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: bdev debug aio = true 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug misdirected ops = true 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore block size = 96636764160 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore fsck on mount = True 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluefs = 20 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluestore = 20 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: debug osd = 20 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: debug rocksdb = 10 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T02:39:44.878 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd full ratio = 0.9 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd nearfull ratio = 0.8 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: osd failsafe full ratio = 0.95 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: osd objectstore = bluestore 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr] 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: debug mgr = 20 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: mgr/telemetry/nag = false 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.879 INFO:teuthology.orchestra.run.smithi131.stdout:[mon] 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: debug paxos = 20 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: mon data avail warn = 5 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: mon mgr mkfs grace = 240 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd reporter subtree level = osd 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd prime pg temp = true 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T02:39:44.880 INFO:teuthology.orchestra.run.smithi131.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: # 1m isn't quite enough 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: mon_down_mkfs_grace = 2m 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_filestore_osds = false 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: mon down mkfs grace = 300 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout:[client] 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: rgw cache enabled = true 2024-09-17T02:39:44.881 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable ops log = true 2024-09-17T02:39:44.882 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable usage log = true 2024-09-17T02:39:44.882 INFO:teuthology.orchestra.run.smithi131.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T02:39:44.882 INFO:teuthology.orchestra.run.smithi131.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T02:39:44.882 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = "" 2024-09-17T02:39:44.882 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.a] 2024-09-17T02:39:44.882 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.b] 2024-09-17T02:39:44.882 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.c] 2024-09-17T02:39:44.882 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2024-09-17T02:39:44.882 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2024-09-17T02:39:44.882 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2024-09-17T02:39:44.882 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2024-09-17T02:39:44.883 INFO:teuthology.orchestra.run.smithi189.stdout:[global] 2024-09-17T02:39:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: chdir = "" 2024-09-17T02:39:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T02:39:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: auth supported = cephx 2024-09-17T02:39:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: filestore xattr use omap = true 2024-09-17T02:39:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: mon clock drift allowed = 1.000 2024-09-17T02:39:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: osd crush chooseleaf type = 0 2024-09-17T02:39:44.883 INFO:teuthology.orchestra.run.smithi189.stdout: auth debug = true 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on old message = true 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on bug = true 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: mon pg warn max object skew = 0 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default size = 2 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow primary affinity = true 2024-09-17T02:39:44.884 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow pg remap = true 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on legacy crush tunables = false 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on crush straw calc version zero = false 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on no sortbitwise = false 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on osd down out interval zero = false 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on too few osds = false 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: mon_allow_pool_size_one = true 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: osd default data pool replay window = 5 2024-09-17T02:39:44.885 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: mon allow pool delete = true 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: mon cluster log file level = debug 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: debug asserts on shutdown = true 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: mon health detail to clog = false 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: mon host = "172.21.15.131,172.21.15.189,[v2:172.21.15.189:3301,v1:172.21.15.189:6790]" 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: fsid = f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout:[osd] 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: osd journal size = 100 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub load threshold = 5.0 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub max interval = 600 2024-09-17T02:39:44.886 INFO:teuthology.orchestra.run.smithi189.stdout: osd mclock profile = high_recovery_ops 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: osd recover clone overlap = true 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: osd recovery max chunk = 1048576 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug shutdown = true 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug op order = true 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug verify stray on activate = true 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug trim objects = true 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: osd open classes on start = true 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug pg log writeout = true 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: osd deep scrub update digest min age = 30 2024-09-17T02:39:44.887 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: osd map max advance = 10 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: journal zero on create = true 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: filestore ondisk finisher threads = 3 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: filestore apply finisher threads = 3 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: bdev debug aio = true 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug misdirected ops = true 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore block size = 96636764160 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore fsck on mount = True 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluefs = 20 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluestore = 20 2024-09-17T02:39:44.888 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: debug osd = 20 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: debug rocksdb = 10 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd full ratio = 0.9 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd nearfull ratio = 0.8 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: osd failsafe full ratio = 0.95 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: osd objectstore = bluestore 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr] 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: debug mgr = 20 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:39:44.889 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:39:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: mgr/telemetry/nag = false 2024-09-17T02:39:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.890 INFO:teuthology.orchestra.run.smithi189.stdout:[mon] 2024-09-17T02:39:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:39:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-17T02:39:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: debug paxos = 20 2024-09-17T02:39:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-17T02:39:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: mon data avail warn = 5 2024-09-17T02:39:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: mon mgr mkfs grace = 240 2024-09-17T02:39:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:39:44.890 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd reporter subtree level = osd 2024-09-17T02:39:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd prime pg temp = true 2024-09-17T02:39:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:39:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T02:39:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T02:39:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T02:39:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T02:39:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T02:39:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T02:39:44.891 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: # 1m isn't quite enough 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon_down_mkfs_grace = 2m 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_filestore_osds = false 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: mon down mkfs grace = 300 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout:[client] 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: rgw cache enabled = true 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable ops log = true 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable usage log = true 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T02:39:44.892 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = "" 2024-09-17T02:39:44.893 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.a] 2024-09-17T02:39:44.893 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.b] 2024-09-17T02:39:44.893 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.c] 2024-09-17T02:39:44.893 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.a] 2024-09-17T02:39:44.893 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.c] 2024-09-17T02:39:44.893 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.b] 2024-09-17T02:39:44.893 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.d] 2024-09-17T02:39:44.894 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T02:39:44.894 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T02:39:45.018 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T02:39:45.018 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:39:45.018 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T02:39:45.023 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:39:45.023 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T02:39:45.069 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:39:45.069 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:39:45.069 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T02:39:45.069 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T02:39:45.133 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:39:45.133 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:39:45.181 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:39:45.181 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:39:45.181 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T02:39:45.182 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T02:39:45.205 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:39:45.205 DEBUG:teuthology.orchestra.run.smithi189:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:39:45.250 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T02:39:45.250 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T02:39:45.250 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T02:39:45.324 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T02:39:45.331 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T02:39:45.405 INFO:teuthology.orchestra.run.smithi189.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T02:39:45.428 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T02:39:45.428 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T02:39:45.503 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T02:39:45.509 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-17T02:39:45.519 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T02:39:45.622 INFO:teuthology.orchestra.run.smithi131.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T02:39:45.629 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-17T02:39:45.640 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T02:39:45.716 INFO:teuthology.orchestra.run.smithi189.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T02:39:45.723 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-17T02:39:45.735 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T02:39:45.845 INFO:teuthology.orchestra.run.smithi189.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T02:39:45.851 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-17T02:39:45.863 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T02:39:45.863 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T02:39:45.927 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T02:39:45.941 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-17T02:39:46.002 INFO:teuthology.orchestra.run.smithi131.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-17T02:39:46.018 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T02:39:46.084 INFO:teuthology.orchestra.run.smithi189.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T02:39:46.098 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-17T02:39:46.158 INFO:teuthology.orchestra.run.smithi189.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-17T02:39:46.171 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T02:39:46.171 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi189.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T02:39:46.172 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T02:39:46.185 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T02:39:46.185 INFO:tasks.ceph:role: osd.0 2024-09-17T02:39:46.185 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:39:46.186 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T02:39:46.249 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T02:39:46.249 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T02:39:46.249 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T02:39:46.249 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T02:39:46.249 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T02:39:46.249 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-09-17T02:39:46.249 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T02:39:46.249 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T02:39:46.249 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T02:39:46.250 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T02:39:46.707 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-09-17T02:39:46.709 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-09-17T02:39:46.709 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T02:39:46.808 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T02:39:46.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:39:46.818 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T02:39:46.819 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T02:39:46.870 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T02:39:46.870 INFO:tasks.ceph:role: osd.1 2024-09-17T02:39:46.870 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:39:46.870 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T02:39:46.932 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T02:39:46.932 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T02:39:46.932 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T02:39:46.932 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T02:39:46.932 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T02:39:46.932 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-09-17T02:39:46.932 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T02:39:46.933 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T02:39:46.933 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T02:39:46.933 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T02:39:47.404 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-09-17T02:39:47.406 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-09-17T02:39:47.406 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T02:39:47.560 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T02:39:47.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:39:47.572 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T02:39:47.572 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T02:39:47.629 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T02:39:47.629 INFO:tasks.ceph:role: osd.2 2024-09-17T02:39:47.630 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:39:47.630 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T02:39:47.693 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T02:39:47.694 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T02:39:47.694 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T02:39:47.694 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T02:39:47.694 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T02:39:47.695 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-09-17T02:39:47.695 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T02:39:47.695 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T02:39:47.695 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T02:39:47.696 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T02:39:48.159 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-09-17T02:39:48.161 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-09-17T02:39:48.161 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T02:39:48.305 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T02:39:48.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:39:48.317 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T02:39:48.317 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T02:39:48.373 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T02:39:48.373 INFO:tasks.ceph:role: osd.3 2024-09-17T02:39:48.373 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:39:48.373 DEBUG:teuthology.orchestra.run.smithi131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T02:39:48.436 INFO:teuthology.orchestra.run.smithi131.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T02:39:48.437 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T02:39:48.437 INFO:teuthology.orchestra.run.smithi131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T02:39:48.437 INFO:teuthology.orchestra.run.smithi131.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T02:39:48.437 INFO:teuthology.orchestra.run.smithi131.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T02:39:48.437 INFO:teuthology.orchestra.run.smithi131.stdout: = sunit=0 swidth=0 blks 2024-09-17T02:39:48.437 INFO:teuthology.orchestra.run.smithi131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T02:39:48.437 INFO:teuthology.orchestra.run.smithi131.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T02:39:48.437 INFO:teuthology.orchestra.run.smithi131.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T02:39:48.437 INFO:teuthology.orchestra.run.smithi131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T02:39:48.868 INFO:teuthology.orchestra.run.smithi131.stdout:Discarding blocks...Done. 2024-09-17T02:39:48.870 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi131.front.sepia.ceph.com -o noatime 2024-09-17T02:39:48.870 DEBUG:teuthology.orchestra.run.smithi131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T02:39:48.963 DEBUG:teuthology.orchestra.run.smithi131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T02:39:48.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:39:48.973 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T02:39:48.973 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:39:49.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:49.119+0000 7f1abb21a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T02:39:49.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:49.123+0000 7f1abb21a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T02:39:49.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:49.123+0000 7f1abb21a7c0 -1 bdev(0x5644ac1c2a00 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T02:39:49.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:49.123+0000 7f1abb21a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T02:39:52.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T02:39:52.286 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:39:52.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:52.403+0000 7f3b9a4d77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T02:39:52.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:52.403+0000 7f3b9a4d77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T02:39:52.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:52.403+0000 7f3b9a4d77c0 -1 bdev(0x55c041dc6a00 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T02:39:52.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:52.403+0000 7f3b9a4d77c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T02:39:55.548 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T02:39:55.561 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:39:55.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:55.675+0000 7f889cc637c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T02:39:55.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:55.675+0000 7f889cc637c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T02:39:55.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:55.675+0000 7f889cc637c0 -1 bdev(0x557458dc6a00 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T02:39:55.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:55.675+0000 7f889cc637c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T02:39:58.805 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T02:39:58.818 DEBUG:teuthology.orchestra.run.smithi131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:39:58.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:58.927+0000 7fb28e69e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T02:39:58.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:58.927+0000 7fb28e69e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T02:39:58.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:58.927+0000 7fb28e69e7c0 -1 bdev(0x55d1b45c4a00 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T02:39:58.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:39:58.927+0000 7fb28e69e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T02:40:02.065 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T02:40:02.078 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T02:40:02.093 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T02:40:02.094 INFO:tasks.ceph:role: osd.4 2024-09-17T02:40:02.094 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:40:02.094 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T02:40:02.157 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T02:40:02.157 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T02:40:02.157 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T02:40:02.157 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T02:40:02.157 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T02:40:02.157 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-17T02:40:02.157 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T02:40:02.158 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T02:40:02.158 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T02:40:02.158 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T02:40:02.608 INFO:teuthology.orchestra.run.smithi189.stdout:Discarding blocks...Done. 2024-09-17T02:40:02.610 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-17T02:40:02.611 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T02:40:02.703 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T02:40:02.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:40:02.713 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T02:40:02.714 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T02:40:02.767 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T02:40:02.767 INFO:tasks.ceph:role: osd.5 2024-09-17T02:40:02.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:40:02.768 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T02:40:02.830 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T02:40:02.830 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T02:40:02.830 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T02:40:02.830 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T02:40:02.830 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T02:40:02.830 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-17T02:40:02.831 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T02:40:02.831 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T02:40:02.831 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T02:40:02.831 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T02:40:03.300 INFO:teuthology.orchestra.run.smithi189.stdout:Discarding blocks...Done. 2024-09-17T02:40:03.302 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-17T02:40:03.303 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T02:40:03.465 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T02:40:03.475 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T02:40:03.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:40:03.477 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T02:40:03.529 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T02:40:03.529 INFO:tasks.ceph:role: osd.6 2024-09-17T02:40:03.530 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:40:03.530 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T02:40:03.594 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T02:40:03.594 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T02:40:03.594 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T02:40:03.594 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T02:40:03.594 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T02:40:03.594 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-17T02:40:03.594 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T02:40:03.594 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T02:40:03.594 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T02:40:03.595 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T02:40:04.076 INFO:teuthology.orchestra.run.smithi189.stdout:Discarding blocks...Done. 2024-09-17T02:40:04.078 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-17T02:40:04.078 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T02:40:04.230 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T02:40:04.241 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T02:40:04.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:40:04.242 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T02:40:04.297 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T02:40:04.297 INFO:tasks.ceph:role: osd.7 2024-09-17T02:40:04.298 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:40:04.298 DEBUG:teuthology.orchestra.run.smithi189:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T02:40:04.362 INFO:teuthology.orchestra.run.smithi189.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T02:40:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T02:40:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T02:40:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-17T02:40:04.362 INFO:teuthology.orchestra.run.smithi189.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T02:40:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: = sunit=0 swidth=0 blks 2024-09-17T02:40:04.362 INFO:teuthology.orchestra.run.smithi189.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T02:40:04.362 INFO:teuthology.orchestra.run.smithi189.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-17T02:40:04.362 INFO:teuthology.orchestra.run.smithi189.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T02:40:04.362 INFO:teuthology.orchestra.run.smithi189.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T02:40:04.788 INFO:teuthology.orchestra.run.smithi189.stdout:Discarding blocks...Done. 2024-09-17T02:40:04.791 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi189.front.sepia.ceph.com -o noatime 2024-09-17T02:40:04.791 DEBUG:teuthology.orchestra.run.smithi189:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T02:40:04.887 DEBUG:teuthology.orchestra.run.smithi189:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T02:40:04.896 INFO:teuthology.orchestra.run.smithi189.stderr:sudo: /sbin/restorecon: command not found 2024-09-17T02:40:04.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:40:04.897 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:40:05.057 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:05.060+0000 7f507a4337c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T02:40:05.060 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:05.060+0000 7f507a4337c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T02:40:05.061 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:05.060+0000 7f507a4337c0 -1 bdev(0x55cabfec4a00 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T02:40:05.061 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:05.060+0000 7f507a4337c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T02:40:08.204 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T02:40:08.219 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:40:08.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:08.332+0000 7fb1eafa77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T02:40:08.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:08.332+0000 7fb1eafa77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T02:40:08.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:08.332+0000 7fb1eafa77c0 -1 bdev(0x555a01566a00 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T02:40:08.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:08.332+0000 7fb1eafa77c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T02:40:11.468 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T02:40:11.479 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:40:11.590 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:11.592+0000 7fa91a3977c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T02:40:11.590 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:11.592+0000 7fa91a3977c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T02:40:11.590 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:11.592+0000 7fa91a3977c0 -1 bdev(0x560261c84a00 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T02:40:11.591 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:11.592+0000 7fa91a3977c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T02:40:13.140 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:40:13.143 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:40:14.723 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T02:40:14.737 DEBUG:teuthology.orchestra.run.smithi189:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:40:14.850 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:14.852+0000 7fa73641c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T02:40:14.850 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:14.852+0000 7fa73641c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T02:40:14.851 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:14.852+0000 7fa73641c7c0 -1 bdev(0x561e9f110a00 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T02:40:14.851 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:14.852+0000 7fa73641c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T02:40:17.987 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T02:40:18.001 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T02:40:18.001 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:18.001 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T02:40:18.014 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:18.014 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-17T02:40:18.069 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:18.069 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-17T02:40:18.128 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:18.128 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T02:40:18.184 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:18.184 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T02:40:18.240 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:18.240 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T02:40:18.300 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:18.300 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T02:40:18.357 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:18.357 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T02:40:18.371 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:18.371 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-17T02:40:18.425 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:18.425 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-17T02:40:18.482 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:18.482 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T02:40:18.538 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:18.538 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T02:40:18.593 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:18.593 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T02:40:18.648 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:18.648 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T02:40:18.701 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:18.701 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T02:40:18.708 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:18.708 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-17T02:40:18.758 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:18.758 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T02:40:18.765 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:18.765 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-17T02:40:18.810 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T02:40:18.810 DEBUG:teuthology.orchestra.run.smithi131:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T02:40:18.813 DEBUG:teuthology.orchestra.run.smithi189:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T02:40:18.856 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr.y] 2024-09-17T02:40:18.856 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hmrTq3ExAAuR8+2bItlpHqqFzlvPxASA== 2024-09-17T02:40:18.856 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2024-09-17T02:40:18.856 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hm4U1cHhAATu86VrwT2vMjL/cW2SICMw== 2024-09-17T02:40:18.856 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2024-09-17T02:40:18.857 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hmwHuCJRAAlOj+za8ZmqMI55PsShjKDw== 2024-09-17T02:40:18.857 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.0] 2024-09-17T02:40:18.857 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD16+hmV9xkBxAAT+IDf2J4awJfqYcx2xDoNA== 2024-09-17T02:40:18.857 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.1] 2024-09-17T02:40:18.857 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD46+hmOhJsGBAAdlA4BSdOPk8BySUTfo1ghg== 2024-09-17T02:40:18.857 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.2] 2024-09-17T02:40:18.857 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD76+hm3mSoKBAAQPoGFy816yWZQW6vPIK6bA== 2024-09-17T02:40:18.857 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.3] 2024-09-17T02:40:18.857 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQD+6+hmpZaPNxAAp0EV6bn5SugDMwA4RnnJwg== 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr.x] 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hmnWmJGBAA6fzewrm8ELqUAz/MczTOGA== 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hmVpUSKxAA4l5zGHXg0VRasiyBI/j4ig== 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hm9f+9MhAAS8z9jZFYon3nrmu9/4JMPg== 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.4] 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAF7OhmgDfVAxAALACK3bk+PK3UhG20b498HQ== 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.5] 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAI7Ohm6w44FBAA90GkelUAbiNjAwn+6IJWcg== 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.6] 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAL7Ohm6S6TIxAArISJW//rmEONG2bC/sfkXA== 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout:[osd.7] 2024-09-17T02:40:18.858 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQAO7OhmICwTMxAArAr7DhcO8A0FDSu7VEBehw== 2024-09-17T02:40:18.859 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-09-17T02:40:18.859 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA== 2024-09-17T02:40:18.859 INFO:teuthology.orchestra.run.smithi131.stdout:[client.2] 2024-09-17T02:40:18.859 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDy6+hmBBCDABAAMw9581QyaOxJk5fs35o+ng== 2024-09-17T02:40:18.859 INFO:teuthology.orchestra.run.smithi131.stdout:[client.1] 2024-09-17T02:40:18.859 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDy6+hmduVoBRAAwr50jSXw6sCyo+5t+3iT7A== 2024-09-17T02:40:18.859 INFO:teuthology.orchestra.run.smithi131.stdout:[client.3] 2024-09-17T02:40:18.859 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDy6+hmgznRCRAA1Z8d32ADoSG+7RHtdPGmTQ== 2024-09-17T02:40:18.864 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr.y] 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQDx6+hmrTq3ExAAuR8+2bItlpHqqFzlvPxASA== 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.a] 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQDx6+hm4U1cHhAATu86VrwT2vMjL/cW2SICMw== 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.c] 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQDx6+hmwHuCJRAAlOj+za8ZmqMI55PsShjKDw== 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.0] 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQD16+hmV9xkBxAAT+IDf2J4awJfqYcx2xDoNA== 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.1] 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQD46+hmOhJsGBAAdlA4BSdOPk8BySUTfo1ghg== 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.2] 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQD76+hm3mSoKBAAQPoGFy816yWZQW6vPIK6bA== 2024-09-17T02:40:18.865 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.3] 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQD+6+hmpZaPNxAAp0EV6bn5SugDMwA4RnnJwg== 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr.x] 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQDx6+hmnWmJGBAA6fzewrm8ELqUAz/MczTOGA== 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.b] 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQDx6+hmVpUSKxAA4l5zGHXg0VRasiyBI/j4ig== 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.d] 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQDx6+hm9f+9MhAAS8z9jZFYon3nrmu9/4JMPg== 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.4] 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAF7OhmgDfVAxAALACK3bk+PK3UhG20b498HQ== 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.5] 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAI7Ohm6w44FBAA90GkelUAbiNjAwn+6IJWcg== 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.6] 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAL7Ohm6S6TIxAArISJW//rmEONG2bC/sfkXA== 2024-09-17T02:40:18.866 INFO:teuthology.orchestra.run.smithi189.stdout:[osd.7] 2024-09-17T02:40:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQAO7OhmICwTMxAArAr7DhcO8A0FDSu7VEBehw== 2024-09-17T02:40:18.867 INFO:teuthology.orchestra.run.smithi189.stdout:[client.0] 2024-09-17T02:40:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA== 2024-09-17T02:40:18.867 INFO:teuthology.orchestra.run.smithi189.stdout:[client.2] 2024-09-17T02:40:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQDy6+hmBBCDABAAMw9581QyaOxJk5fs35o+ng== 2024-09-17T02:40:18.867 INFO:teuthology.orchestra.run.smithi189.stdout:[client.1] 2024-09-17T02:40:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQDy6+hmduVoBRAAwr50jSXw6sCyo+5t+3iT7A== 2024-09-17T02:40:18.867 INFO:teuthology.orchestra.run.smithi189.stdout:[client.3] 2024-09-17T02:40:18.867 INFO:teuthology.orchestra.run.smithi189.stdout: key = AQDy6+hmgznRCRAA1Z8d32ADoSG+7RHtdPGmTQ== 2024-09-17T02:40:18.868 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T02:40:18.906 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T02:40:18.988 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T02:40:18.991 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T02:40:19.068 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T02:40:19.070 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T02:40:19.140 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.143 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.214 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.216 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.284 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.287 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.354 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.356 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.429 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T02:40:19.431 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T02:40:19.499 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T02:40:19.501 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T02:40:19.573 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T02:40:19.575 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T02:40:19.652 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.655 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.722 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.725 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.792 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.794 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.862 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.865 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T02:40:19.938 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T02:40:19.940 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T02:40:20.012 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T02:40:20.014 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T02:40:20.091 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T02:40:20.093 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T02:40:20.169 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T02:40:20.172 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T02:40:20.243 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T02:40:20.243 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T02:40:20.254 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T02:40:20.373 DEBUG:teuthology.orchestra.run.smithi131:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T02:40:20.383 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T02:40:20.395 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T02:40:20.523 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T02:40:20.534 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T02:40:20.585 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T02:40:20.705 DEBUG:teuthology.orchestra.run.smithi189:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T02:40:20.715 DEBUG:teuthology.orchestra.run.smithi131:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:40:20.718 DEBUG:teuthology.orchestra.run.smithi189:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T02:40:20.761 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T02:40:20.762 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T02:40:20.762 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T02:40:20.765 INFO:tasks.ceph.mon.a:Started 2024-09-17T02:40:20.766 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T02:40:20.766 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T02:40:20.811 INFO:tasks.ceph.mon.b:Started 2024-09-17T02:40:20.811 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T02:40:20.811 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T02:40:20.813 INFO:tasks.ceph.mon.c:Started 2024-09-17T02:40:20.813 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T02:40:20.814 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T02:40:20.814 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T02:40:20.816 INFO:tasks.ceph.mgr.y:Started 2024-09-17T02:40:20.816 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T02:40:20.816 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T02:40:20.819 INFO:tasks.ceph.mgr.x:Started 2024-09-17T02:40:20.819 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-17T02:40:20.819 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-17T02:40:20.822 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-17T02:40:20.822 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-17T02:40:20.869 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-17T02:40:20.869 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-17T02:40:20.913 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-17T02:40:20.913 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-17T02:40:20.961 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-17T02:40:20.961 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-17T02:40:21.004 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-17T02:40:21.005 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-17T02:40:21.049 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-17T02:40:21.049 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-17T02:40:21.093 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-17T02:40:21.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-17T02:40:21.137 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-17T02:40:21.137 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-17T02:40:21.181 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-17T02:40:21.181 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-17T02:40:21.225 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-17T02:40:21.225 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-17T02:40:21.269 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-17T02:40:21.269 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-17T02:40:21.313 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-17T02:40:21.313 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-17T02:40:21.357 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-17T02:40:21.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-17T02:40:21.405 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-17T02:40:21.405 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-17T02:40:21.449 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-17T02:40:21.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-17T02:40:21.493 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-17T02:40:21.493 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-17T02:40:21.540 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-17T02:40:21.541 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-17T02:40:21.585 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-17T02:40:21.585 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-17T02:40:21.629 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-17T02:40:21.630 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-17T02:40:21.673 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-17T02:40:21.673 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-17T02:40:21.722 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-17T02:40:21.722 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-17T02:40:21.769 DEBUG:tasks.ceph:set 22 configs 2024-09-17T02:40:21.769 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi131') 2024-09-17T02:40:26.549 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi131') 2024-09-17T02:40:26.595 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.170+0000 7f2175f94640 1 -- 172.21.15.131:0/334867398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f215c004080 msgr2=0x7f2170163a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.170+0000 7f2175f94640 1 --2- 172.21.15.131:0/334867398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f215c004080 0x7f2170163a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2160006c60 tx=0x7f2160014830 comp rx=0 tx=0).stop 2024-09-17T02:40:27.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.170+0000 7f2175f94640 1 -- 172.21.15.131:0/334867398 shutdown_connections 2024-09-17T02:40:27.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.170+0000 7f2175f94640 1 --2- 172.21.15.131:0/334867398 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f215c009410 0x7f2170153e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.170+0000 7f2175f94640 1 --2- 172.21.15.131:0/334867398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f215c006640 0x7f21701f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.170+0000 7f2175f94640 1 --2- 172.21.15.131:0/334867398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f215c004080 0x7f2170163a60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.170+0000 7f2175f94640 1 -- 172.21.15.131:0/334867398 >> 172.21.15.131:0/334867398 conn(0x7f21700cf150 msgr2=0x7f2170155600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.170+0000 7f2175f94640 1 -- 172.21.15.131:0/334867398 shutdown_connections 2024-09-17T02:40:27.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.170+0000 7f2175f94640 1 -- 172.21.15.131:0/334867398 wait complete. 2024-09-17T02:40:27.187 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.206+0000 7f2296fef640 1 -- 172.21.15.131:0/3564758808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f2290071210 con 0x7f227400aea0 2024-09-17T02:40:27.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.210+0000 7f228d7fa640 1 -- 172.21.15.131:0/3564758808 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7f2288005370 con 0x7f227400aea0 2024-09-17T02:40:27.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.214+0000 7f228d7fa640 1 -- 172.21.15.131:0/3564758808 <== mon.0 v2:172.21.15.131:3300/0 9 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22880056e0 con 0x7f227400aea0 2024-09-17T02:40:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.214+0000 7f2296fef640 1 -- 172.21.15.131:0/3564758808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f227400aea0 msgr2=0x7f229015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.214+0000 7f2296fef640 1 --2- 172.21.15.131:0/3564758808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f227400aea0 0x7f229015ba80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2274006190 tx=0x7f2288013ba0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.214+0000 7f2296fef640 1 -- 172.21.15.131:0/3564758808 shutdown_connections 2024-09-17T02:40:27.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.214+0000 7f2296fef640 1 --2- 172.21.15.131:0/3564758808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f227400aea0 0x7f229015ba80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.214+0000 7f2296fef640 1 --2- 172.21.15.131:0/3564758808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2274006700 0x7f229015b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.214+0000 7f2296fef640 1 --2- 172.21.15.131:0/3564758808 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2274003b60 0x7f229013f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.214+0000 7f2296fef640 1 -- 172.21.15.131:0/3564758808 >> 172.21.15.131:0/3564758808 conn(0x7f22900d1130 msgr2=0x7f229015dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.214+0000 7f2296fef640 1 -- 172.21.15.131:0/3564758808 shutdown_connections 2024-09-17T02:40:27.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.214+0000 7f2296fef640 1 -- 172.21.15.131:0/3564758808 wait complete. 2024-09-17T02:40:27.228 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.228 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.228 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.242+0000 7f4a988cd640 1 -- 172.21.15.131:0/3932147511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a740066c0 msgr2=0x7f4a74009280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.242+0000 7f4a988cd640 1 --2- 172.21.15.131:0/3932147511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a740066c0 0x7f4a74009280 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4a74006100 tx=0x7f4a8401d980 comp rx=0 tx=0).stop 2024-09-17T02:40:27.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.242+0000 7f4a988cd640 1 -- 172.21.15.131:0/3932147511 shutdown_connections 2024-09-17T02:40:27.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.242+0000 7f4a988cd640 1 --2- 172.21.15.131:0/3932147511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a740066c0 0x7f4a74009280 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.242+0000 7f4a988cd640 1 -- 172.21.15.131:0/3932147511 >> 172.21.15.131:0/3932147511 conn(0x7f4a940d2130 msgr2=0x7f4a940d2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.242+0000 7f4a988cd640 1 -- 172.21.15.131:0/3932147511 shutdown_connections 2024-09-17T02:40:27.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.242+0000 7f4a988cd640 1 -- 172.21.15.131:0/3932147511 wait complete. 2024-09-17T02:40:27.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a988cd640 1 Processor -- start 2024-09-17T02:40:27.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a988cd640 1 -- start start 2024-09-17T02:40:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a988cd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a74003b90 0x7f4a941ec0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a988cd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a740066c0 0x7f4a940cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a988cd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7400af10 0x7f4a940ced60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a988cd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941607e0 con 0x7f4a7400af10 2024-09-17T02:40:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a988cd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94160660 con 0x7f4a740066c0 2024-09-17T02:40:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a988cd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4a94160960 con 0x7f4a74003b90 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a92d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7400af10 0x7f4a940ced60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a92d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7400af10 0x7f4a940ced60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49792/0 (socket says 172.21.15.131:49792) 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a92d9d640 1 -- 172.21.15.131:0/2124304727 learned_addr learned my addr 172.21.15.131:0/2124304727 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a8bfff640 1 --2- 172.21.15.131:0/2124304727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a740066c0 0x7f4a940cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a9259c640 1 --2- 172.21.15.131:0/2124304727 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a74003b90 0x7f4a941ec0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a92d9d640 1 -- 172.21.15.131:0/2124304727 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a74003b90 msgr2=0x7f4a941ec0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a92d9d640 1 --2- 172.21.15.131:0/2124304727 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a74003b90 0x7f4a941ec0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a92d9d640 1 -- 172.21.15.131:0/2124304727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a740066c0 msgr2=0x7f4a940cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a92d9d640 1 --2- 172.21.15.131:0/2124304727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a740066c0 0x7f4a940cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a92d9d640 1 -- 172.21.15.131:0/2124304727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a941eb020 con 0x7f4a7400af10 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a8bfff640 1 --2- 172.21.15.131:0/2124304727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a740066c0 0x7f4a940cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a92d9d640 1 --2- 172.21.15.131:0/2124304727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7400af10 0x7f4a940ced60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a94074d00 tx=0x7f4a8c00c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.246+0000 7f4a8affd640 1 -- 172.21.15.131:0/2124304727 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c018070 con 0x7f4a7400af10 2024-09-17T02:40:27.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.250+0000 7f4a8affd640 1 -- 172.21.15.131:0/2124304727 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a8c013790 con 0x7f4a7400af10 2024-09-17T02:40:27.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.250+0000 7f4a8affd640 1 -- 172.21.15.131:0/2124304727 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c013be0 con 0x7f4a7400af10 2024-09-17T02:40:27.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.250+0000 7f4a988cd640 1 -- 172.21.15.131:0/2124304727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a941eb300 con 0x7f4a7400af10 2024-09-17T02:40:27.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.250+0000 7f4a988cd640 1 -- 172.21.15.131:0/2124304727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a941eb8c0 con 0x7f4a7400af10 2024-09-17T02:40:27.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.250+0000 7f4a8affd640 1 -- 172.21.15.131:0/2124304727 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4a8c01b070 con 0x7f4a7400af10 2024-09-17T02:40:27.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.250+0000 7f4a8affd640 1 -- 172.21.15.131:0/2124304727 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4a8c012850 con 0x7f4a7400af10 2024-09-17T02:40:27.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.250+0000 7f4a988cd640 1 -- 172.21.15.131:0/2124304727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a94143760 con 0x7f4a7400af10 2024-09-17T02:40:27.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.250+0000 7f4a8affd640 1 -- 172.21.15.131:0/2124304727 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4a8c022030 con 0x7f4a7400af10 2024-09-17T02:40:27.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.306+0000 7f4a988cd640 1 -- 172.21.15.131:0/2124304727 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f4a94070d60 con 0x7f4a7400af10 2024-09-17T02:40:27.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.314+0000 7f4a8affd640 1 -- 172.21.15.131:0/2124304727 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v17)=0 v17) ==== 117+0+0 (secure 0 0 0) 0x7f4a940cd0d0 con 0x7f4a7400af10 2024-09-17T02:40:27.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.314+0000 7f4a988cd640 1 -- 172.21.15.131:0/2124304727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7400af10 msgr2=0x7f4a940ced60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.314+0000 7f4a988cd640 1 --2- 172.21.15.131:0/2124304727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7400af10 0x7f4a940ced60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a94074d00 tx=0x7f4a8c00c610 comp rx=0 tx=0).stop 2024-09-17T02:40:27.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.314+0000 7f4a988cd640 1 -- 172.21.15.131:0/2124304727 shutdown_connections 2024-09-17T02:40:27.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.314+0000 7f4a988cd640 1 --2- 172.21.15.131:0/2124304727 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a7400af10 0x7f4a940ced60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.314+0000 7f4a988cd640 1 --2- 172.21.15.131:0/2124304727 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a740066c0 0x7f4a940cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.314+0000 7f4a988cd640 1 --2- 172.21.15.131:0/2124304727 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a74003b90 0x7f4a941ec0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.314+0000 7f4a988cd640 1 -- 172.21.15.131:0/2124304727 >> 172.21.15.131:0/2124304727 conn(0x7f4a940d2130 msgr2=0x7f4a9415f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.314+0000 7f4a988cd640 1 -- 172.21.15.131:0/2124304727 shutdown_connections 2024-09-17T02:40:27.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.314+0000 7f4a988cd640 1 -- 172.21.15.131:0/2124304727 wait complete. 2024-09-17T02:40:27.335 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.335 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.335 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.374+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/1700629035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af4009310 msgr2=0x7f7af400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.374+0000 7f7b0ddbd640 1 --2- 172.21.15.131:0/1700629035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af4009310 0x7f7af400b730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7af8006fd0 tx=0x7f7af801d670 comp rx=0 tx=0).stop 2024-09-17T02:40:27.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.374+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/1700629035 shutdown_connections 2024-09-17T02:40:27.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.374+0000 7f7b0ddbd640 1 --2- 172.21.15.131:0/1700629035 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af4009310 0x7f7af400b730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.374+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/1700629035 >> 172.21.15.131:0/1700629035 conn(0x7f7b080d1130 msgr2=0x7f7b080d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.374+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/1700629035 shutdown_connections 2024-09-17T02:40:27.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.374+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/1700629035 wait complete. 2024-09-17T02:40:27.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.382+0000 7f7b0ddbd640 1 Processor -- start 2024-09-17T02:40:27.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.382+0000 7f7b0ddbd640 1 -- start start 2024-09-17T02:40:27.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.382+0000 7f7b0ddbd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7af40052e0 0x7f7b08152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.382+0000 7f7b0ddbd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40066e0 0x7f7b08149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.382+0000 7f7b0ddbd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7af4009310 0x7f7b080b2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.382+0000 7f7b0ddbd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7b080cde80 con 0x7f7af40066e0 2024-09-17T02:40:27.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b0ddbd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7b080cdd00 con 0x7f7af40052e0 2024-09-17T02:40:27.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b0ddbd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7b080ce000 con 0x7f7af4009310 2024-09-17T02:40:27.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b06ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40066e0 0x7f7b08149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b06ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40066e0 0x7f7b08149400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49798/0 (socket says 172.21.15.131:49798) 2024-09-17T02:40:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b06ffd640 1 -- 172.21.15.131:0/745024214 learned_addr learned my addr 172.21.15.131:0/745024214 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b07fff640 1 --2- 172.21.15.131:0/745024214 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7af4009310 0x7f7b080b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b077fe640 1 --2- 172.21.15.131:0/745024214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7af40052e0 0x7f7b08152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b06ffd640 1 -- 172.21.15.131:0/745024214 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7af4009310 msgr2=0x7f7b080b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b06ffd640 1 --2- 172.21.15.131:0/745024214 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7af4009310 0x7f7b080b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b06ffd640 1 -- 172.21.15.131:0/745024214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7af40052e0 msgr2=0x7f7b08152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b06ffd640 1 --2- 172.21.15.131:0/745024214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7af40052e0 0x7f7b08152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b06ffd640 1 -- 172.21.15.131:0/745024214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b081f6330 con 0x7f7af40066e0 2024-09-17T02:40:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b06ffd640 1 --2- 172.21.15.131:0/745024214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40066e0 0x7f7b08149400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7afc004d90 tx=0x7f7afc014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:27.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7aeffff640 1 -- 172.21.15.131:0/745024214 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc012860 con 0x7f7af40066e0 2024-09-17T02:40:27.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7aeffff640 1 -- 172.21.15.131:0/745024214 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7afc012a00 con 0x7f7af40066e0 2024-09-17T02:40:27.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7aeffff640 1 -- 172.21.15.131:0/745024214 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc012e50 con 0x7f7af40066e0 2024-09-17T02:40:27.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/745024214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b081f67a0 con 0x7f7af40066e0 2024-09-17T02:40:27.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/745024214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b081f6d60 con 0x7f7af40066e0 2024-09-17T02:40:27.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7aeffff640 1 -- 172.21.15.131:0/745024214 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7afc026440 con 0x7f7af40066e0 2024-09-17T02:40:27.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7aeffff640 1 -- 172.21.15.131:0/745024214 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7afc01dbc0 con 0x7f7af40066e0 2024-09-17T02:40:27.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.386+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/745024214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b08146430 con 0x7f7af40066e0 2024-09-17T02:40:27.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.390+0000 7f7aeffff640 1 -- 172.21.15.131:0/745024214 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7b08146430 con 0x7f7af40066e0 2024-09-17T02:40:27.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.410+0000 7fca52efa640 1 -- 172.21.15.131:0/3458718302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca3000ae90 msgr2=0x7fca3000d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.410+0000 7fca52efa640 1 --2- 172.21.15.131:0/3458718302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca3000ae90 0x7fca3000d330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0734a0 tx=0x7fca4401d880 comp rx=0 tx=0).stop 2024-09-17T02:40:27.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.410+0000 7fca52efa640 1 -- 172.21.15.131:0/3458718302 shutdown_connections 2024-09-17T02:40:27.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.410+0000 7fca52efa640 1 --2- 172.21.15.131:0/3458718302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca3000ae90 0x7fca3000d330 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.410+0000 7fca52efa640 1 --2- 172.21.15.131:0/3458718302 >> v2:172.21.15.189:3300/0 conn(0x7fca300066a0 0x7fca3000a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.410+0000 7fca52efa640 1 -- 172.21.15.131:0/3458718302 >> 172.21.15.131:0/3458718302 conn(0x7fca4c0cd6a0 msgr2=0x7fca4c0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.410+0000 7fca52efa640 1 -- 172.21.15.131:0/3458718302 shutdown_connections 2024-09-17T02:40:27.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.410+0000 7fca52efa640 1 -- 172.21.15.131:0/3458718302 wait complete. 2024-09-17T02:40:27.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca52efa640 1 Processor -- start 2024-09-17T02:40:27.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca52efa640 1 -- start start 2024-09-17T02:40:27.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca52efa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30003b10 0x7fca4c14f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca52efa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca300066a0 0x7fca4c0c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca52efa640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fca3000ae90 0x7fca4c153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca52efa640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c15f410 con 0x7fca30003b10 2024-09-17T02:40:27.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca52efa640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c15f290 con 0x7fca300066a0 2024-09-17T02:40:27.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca52efa640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fca4c15f590 con 0x7fca3000ae90 2024-09-17T02:40:27.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca50c96640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30003b10 0x7fca4c14f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca50c96640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30003b10 0x7fca4c14f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49820/0 (socket says 172.21.15.131:49820) 2024-09-17T02:40:27.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca50c96640 1 -- 172.21.15.131:0/1926946467 learned_addr learned my addr 172.21.15.131:0/1926946467 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca4bfff640 1 --2- 172.21.15.131:0/1926946467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca300066a0 0x7fca4c0c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca51497640 1 --2- 172.21.15.131:0/1926946467 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fca3000ae90 0x7fca4c153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca50c96640 1 -- 172.21.15.131:0/1926946467 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fca3000ae90 msgr2=0x7fca4c153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca50c96640 1 --2- 172.21.15.131:0/1926946467 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fca3000ae90 0x7fca4c153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca50c96640 1 -- 172.21.15.131:0/1926946467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca300066a0 msgr2=0x7fca4c0c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca50c96640 1 --2- 172.21.15.131:0/1926946467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca300066a0 0x7fca4c0c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca50c96640 1 -- 172.21.15.131:0/1926946467 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c0ca780 con 0x7fca30003b10 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca50c96640 1 --2- 172.21.15.131:0/1926946467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30003b10 0x7fca4c14f570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca40004cc0 tx=0x7fca40014830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca497fa640 1 -- 172.21.15.131:0/1926946467 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40012880 con 0x7fca30003b10 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca52efa640 1 -- 172.21.15.131:0/1926946467 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4c1f6b70 con 0x7fca30003b10 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca497fa640 1 -- 172.21.15.131:0/1926946467 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca40012b10 con 0x7fca30003b10 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca497fa640 1 -- 172.21.15.131:0/1926946467 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca400113d0 con 0x7fca30003b10 2024-09-17T02:40:27.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca52efa640 1 -- 172.21.15.131:0/1926946467 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4c0c9f90 con 0x7fca30003b10 2024-09-17T02:40:27.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca497fa640 1 -- 172.21.15.131:0/1926946467 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fca40011570 con 0x7fca30003b10 2024-09-17T02:40:27.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca497fa640 1 -- 172.21.15.131:0/1926946467 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fca4001dd50 con 0x7fca30003b10 2024-09-17T02:40:27.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.414+0000 7fca52efa640 1 -- 172.21.15.131:0/1926946467 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca4c0ca4a0 con 0x7fca30003b10 2024-09-17T02:40:27.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.418+0000 7fca497fa640 1 -- 172.21.15.131:0/1926946467 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fca4001d430 con 0x7fca30003b10 2024-09-17T02:40:27.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.446+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/745024214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f7b081f6500 con 0x7f7af40066e0 2024-09-17T02:40:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.450+0000 7f7aeffff640 1 -- 172.21.15.131:0/745024214 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f7b081f6500 con 0x7f7af40066e0 2024-09-17T02:40:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.450+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/745024214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40066e0 msgr2=0x7f7b08149400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.450+0000 7f7b0ddbd640 1 --2- 172.21.15.131:0/745024214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40066e0 0x7f7b08149400 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7afc004d90 tx=0x7f7afc014830 comp rx=0 tx=0).stop 2024-09-17T02:40:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.450+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/745024214 shutdown_connections 2024-09-17T02:40:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.450+0000 7f7b0ddbd640 1 --2- 172.21.15.131:0/745024214 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7af4009310 0x7f7b080b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.450+0000 7f7b0ddbd640 1 --2- 172.21.15.131:0/745024214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7af40066e0 0x7f7b08149400 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.450+0000 7f7b0ddbd640 1 --2- 172.21.15.131:0/745024214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7af40052e0 0x7f7b08152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.450+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/745024214 >> 172.21.15.131:0/745024214 conn(0x7f7b080d1130 msgr2=0x7f7b080b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.450+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/745024214 shutdown_connections 2024-09-17T02:40:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.450+0000 7f7b0ddbd640 1 -- 172.21.15.131:0/745024214 wait complete. 2024-09-17T02:40:27.472 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.478+0000 7fca52efa640 1 -- 172.21.15.131:0/1926946467 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fca4c0b6420 con 0x7fca30003b10 2024-09-17T02:40:27.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.486+0000 7fca497fa640 1 -- 172.21.15.131:0/1926946467 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fca400119f0 con 0x7fca30003b10 2024-09-17T02:40:27.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.486+0000 7fca52efa640 1 -- 172.21.15.131:0/1926946467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30003b10 msgr2=0x7fca4c14f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.486+0000 7fca52efa640 1 --2- 172.21.15.131:0/1926946467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30003b10 0x7fca4c14f570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fca40004cc0 tx=0x7fca40014830 comp rx=0 tx=0).stop 2024-09-17T02:40:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.486+0000 7fca52efa640 1 -- 172.21.15.131:0/1926946467 shutdown_connections 2024-09-17T02:40:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.486+0000 7fca52efa640 1 --2- 172.21.15.131:0/1926946467 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fca3000ae90 0x7fca4c153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.486+0000 7fca52efa640 1 --2- 172.21.15.131:0/1926946467 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fca300066a0 0x7fca4c0c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.486+0000 7fca52efa640 1 --2- 172.21.15.131:0/1926946467 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fca30003b10 0x7fca4c14f570 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.486+0000 7fca52efa640 1 -- 172.21.15.131:0/1926946467 >> 172.21.15.131:0/1926946467 conn(0x7fca4c0cd6a0 msgr2=0x7fca4c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.486+0000 7fca52efa640 1 -- 172.21.15.131:0/1926946467 shutdown_connections 2024-09-17T02:40:27.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.486+0000 7fca52efa640 1 -- 172.21.15.131:0/1926946467 wait complete. 2024-09-17T02:40:27.501 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.501 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.501 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.501 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.594+0000 7ff7353ea640 1 -- 172.21.15.131:0/3920968165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c007d80 msgr2=0x7ff71c008190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.594+0000 7ff7353ea640 1 --2- 172.21.15.131:0/3920968165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c007d80 0x7ff71c008190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff724006fd0 tx=0x7ff72401d730 comp rx=0 tx=0).stop 2024-09-17T02:40:27.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.594+0000 7ff7353ea640 1 -- 172.21.15.131:0/3920968165 shutdown_connections 2024-09-17T02:40:27.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.594+0000 7ff7353ea640 1 --2- 172.21.15.131:0/3920968165 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff71c008820 0x7ff71c00d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.594+0000 7ff7353ea640 1 --2- 172.21.15.131:0/3920968165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c007d80 0x7ff71c008190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.594+0000 7ff7353ea640 1 -- 172.21.15.131:0/3920968165 >> 172.21.15.131:0/3920968165 conn(0x7ff73006fdb0 msgr2=0x7ff7300701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.594+0000 7ff7353ea640 1 -- 172.21.15.131:0/3920968165 shutdown_connections 2024-09-17T02:40:27.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.594+0000 7ff7353ea640 1 -- 172.21.15.131:0/3920968165 wait complete. 2024-09-17T02:40:27.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.598+0000 7ff7353ea640 1 Processor -- start 2024-09-17T02:40:27.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.598+0000 7ff7353ea640 1 -- start start 2024-09-17T02:40:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff7353ea640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c003bb0 0x7ff730158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff7353ea640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff71c007d80 0x7ff7300ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff7353ea640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c008820 0x7ff7300ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff7353ea640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff73015d780 con 0x7ff71c003bb0 2024-09-17T02:40:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff7353ea640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff73015d600 con 0x7ff71c008820 2024-09-17T02:40:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff7353ea640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff730071210 con 0x7ff71c007d80 2024-09-17T02:40:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c003bb0 0x7ff730158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c003bb0 0x7ff730158dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49840/0 (socket says 172.21.15.131:49840) 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72effd640 1 -- 172.21.15.131:0/734559559 learned_addr learned my addr 172.21.15.131:0/734559559 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72e7fc640 1 --2- 172.21.15.131:0/734559559 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff71c007d80 0x7ff7300ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72f7fe640 1 --2- 172.21.15.131:0/734559559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c008820 0x7ff7300ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72effd640 1 -- 172.21.15.131:0/734559559 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff71c007d80 msgr2=0x7ff7300ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72effd640 1 --2- 172.21.15.131:0/734559559 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff71c007d80 0x7ff7300ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72effd640 1 -- 172.21.15.131:0/734559559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c008820 msgr2=0x7ff7300ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72effd640 1 --2- 172.21.15.131:0/734559559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c008820 0x7ff7300ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72effd640 1 -- 172.21.15.131:0/734559559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7300adf70 con 0x7ff71c003bb0 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72f7fe640 1 --2- 172.21.15.131:0/734559559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c008820 0x7ff7300ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72e7fc640 1 --2- 172.21.15.131:0/734559559 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff71c007d80 0x7ff7300ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff72effd640 1 --2- 172.21.15.131:0/734559559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c003bb0 0x7ff730158dc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff720011a90 tx=0x7ff720011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:27.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff7177fe640 1 -- 172.21.15.131:0/734559559 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720010c00 con 0x7ff71c003bb0 2024-09-17T02:40:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff7177fe640 1 -- 172.21.15.131:0/734559559 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff720010e90 con 0x7ff71c003bb0 2024-09-17T02:40:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff7353ea640 1 -- 172.21.15.131:0/734559559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7300ae250 con 0x7ff71c003bb0 2024-09-17T02:40:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff7177fe640 1 -- 172.21.15.131:0/734559559 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720004260 con 0x7ff71c003bb0 2024-09-17T02:40:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.602+0000 7ff7353ea640 1 -- 172.21.15.131:0/734559559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7300ae810 con 0x7ff71c003bb0 2024-09-17T02:40:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.606+0000 7ff7177fe640 1 -- 172.21.15.131:0/734559559 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff720024030 con 0x7ff71c003bb0 2024-09-17T02:40:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.606+0000 7ff7177fe640 1 -- 172.21.15.131:0/734559559 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff7200269a0 con 0x7ff71c003bb0 2024-09-17T02:40:27.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.606+0000 7ff7353ea640 1 -- 172.21.15.131:0/734559559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff730071770 con 0x7ff71c003bb0 2024-09-17T02:40:27.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.610+0000 7ff7177fe640 1 -- 172.21.15.131:0/734559559 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff72002b020 con 0x7ff71c003bb0 2024-09-17T02:40:27.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.666+0000 7ff7353ea640 1 -- 172.21.15.131:0/734559559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7ff7300b08c0 con 0x7ff71c003bb0 2024-09-17T02:40:27.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.670+0000 7ff7177fe640 1 -- 172.21.15.131:0/734559559 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v20)=0 v20) ==== 120+0+0 (secure 0 0 0) 0x7ff720026ba0 con 0x7ff71c003bb0 2024-09-17T02:40:27.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.670+0000 7ff7353ea640 1 -- 172.21.15.131:0/734559559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c003bb0 msgr2=0x7ff730158dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.670+0000 7ff7353ea640 1 --2- 172.21.15.131:0/734559559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c003bb0 0x7ff730158dc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff720011a90 tx=0x7ff720011f50 comp rx=0 tx=0).stop 2024-09-17T02:40:27.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.670+0000 7ff7353ea640 1 -- 172.21.15.131:0/734559559 shutdown_connections 2024-09-17T02:40:27.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.670+0000 7ff7353ea640 1 --2- 172.21.15.131:0/734559559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff71c008820 0x7ff7300ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.670+0000 7ff7353ea640 1 --2- 172.21.15.131:0/734559559 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff71c007d80 0x7ff7300ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.670+0000 7ff7353ea640 1 --2- 172.21.15.131:0/734559559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff71c003bb0 0x7ff730158dc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.670+0000 7ff7353ea640 1 -- 172.21.15.131:0/734559559 >> 172.21.15.131:0/734559559 conn(0x7ff73006fdb0 msgr2=0x7ff73015a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.670+0000 7ff7353ea640 1 -- 172.21.15.131:0/734559559 shutdown_connections 2024-09-17T02:40:27.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.670+0000 7ff7353ea640 1 -- 172.21.15.131:0/734559559 wait complete. 2024-09-17T02:40:27.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.686+0000 7f25d0666640 1 -- 172.21.15.131:0/925516181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b4004810 msgr2=0x7f25b4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.686+0000 7f25d0666640 1 --2- 172.21.15.131:0/925516181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25b4004810 0x7f25b4004c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f25ac006650 tx=0x7f25b801ec10 comp rx=0 tx=0).stop 2024-09-17T02:40:27.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.686+0000 7f25cec03640 1 -- 172.21.15.131:0/925516181 reap_dead start 2024-09-17T02:40:27.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.686+0000 7f25d0666640 1 -- 172.21.15.131:0/925516181 shutdown_connections 2024-09-17T02:40:27.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.686+0000 7f25d0666640 1 -- 172.21.15.131:0/925516181 >> 172.21.15.131:0/925516181 conn(0x7f25c80d1130 msgr2=0x7f25c80d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.686+0000 7f25d0666640 1 -- 172.21.15.131:0/925516181 shutdown_connections 2024-09-17T02:40:27.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.686+0000 7f25d0666640 1 -- 172.21.15.131:0/925516181 wait complete. 2024-09-17T02:40:27.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.686+0000 7f25d0666640 1 Processor -- start 2024-09-17T02:40:27.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.686+0000 7f25d0666640 1 -- start start 2024-09-17T02:40:27.684 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.684 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25d0666640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25b4004810 0x7f25c81e6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25d0666640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c80b55b0 0x7f25c8145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25d0666640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c81587f0 0x7f25c81f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25d0666640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25c80715a0 con 0x7f25c81587f0 2024-09-17T02:40:27.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25d0666640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f25c8071420 con 0x7f25c80b55b0 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25d0666640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f25c80cd900 con 0x7f25b4004810 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25cec03640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c81587f0 0x7f25c81f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25cec03640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c81587f0 0x7f25c81f56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49862/0 (socket says 172.21.15.131:49862) 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25cec03640 1 -- 172.21.15.131:0/3452272719 learned_addr learned my addr 172.21.15.131:0/3452272719 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25cdc01640 1 --2- 172.21.15.131:0/3452272719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c80b55b0 0x7f25c8145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25ce402640 1 --2- 172.21.15.131:0/3452272719 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25b4004810 0x7f25c81e6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25cec03640 1 -- 172.21.15.131:0/3452272719 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25b4004810 msgr2=0x7f25c81e6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25cec03640 1 --2- 172.21.15.131:0/3452272719 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25b4004810 0x7f25c81e6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25cec03640 1 -- 172.21.15.131:0/3452272719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c80b55b0 msgr2=0x7f25c8145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25cec03640 1 --2- 172.21.15.131:0/3452272719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c80b55b0 0x7f25c8145c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25cec03640 1 -- 172.21.15.131:0/3452272719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c80b4510 con 0x7f25c81587f0 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25ce402640 1 --2- 172.21.15.131:0/3452272719 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25b4004810 0x7f25c81e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:40:27.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25cec03640 1 --2- 172.21.15.131:0/3452272719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c81587f0 0x7f25c81f56e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0061b0 tx=0x7f25c000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:27.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25beffd640 1 -- 172.21.15.131:0/3452272719 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c0018070 con 0x7f25c81587f0 2024-09-17T02:40:27.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25beffd640 1 -- 172.21.15.131:0/3452272719 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25c0003b80 con 0x7f25c81587f0 2024-09-17T02:40:27.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25beffd640 1 -- 172.21.15.131:0/3452272719 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c000c070 con 0x7f25c81587f0 2024-09-17T02:40:27.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25d0666640 1 -- 172.21.15.131:0/3452272719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c80b47f0 con 0x7f25c81587f0 2024-09-17T02:40:27.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25d0666640 1 -- 172.21.15.131:0/3452272719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c80b4db0 con 0x7f25c81587f0 2024-09-17T02:40:27.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25beffd640 1 -- 172.21.15.131:0/3452272719 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f25c001b070 con 0x7f25c81587f0 2024-09-17T02:40:27.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25beffd640 1 -- 172.21.15.131:0/3452272719 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f25c001dc00 con 0x7f25c81587f0 2024-09-17T02:40:27.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.690+0000 7f25d0666640 1 -- 172.21.15.131:0/3452272719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c80cde60 con 0x7f25c81587f0 2024-09-17T02:40:27.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.694+0000 7f25beffd640 1 -- 172.21.15.131:0/3452272719 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f25c0022020 con 0x7f25c81587f0 2024-09-17T02:40:27.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.750+0000 7f25d0666640 1 -- 172.21.15.131:0/3452272719 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f25c815e010 con 0x7f25c81587f0 2024-09-17T02:40:27.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.754+0000 7f25beffd640 1 -- 172.21.15.131:0/3452272719 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v21)=0 v21) ==== 111+0+0 (secure 0 0 0) 0x7f25c80cff70 con 0x7f25c81587f0 2024-09-17T02:40:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.758+0000 7f25d0666640 1 -- 172.21.15.131:0/3452272719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c81587f0 msgr2=0x7f25c81f56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.758+0000 7f25d0666640 1 --2- 172.21.15.131:0/3452272719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c81587f0 0x7f25c81f56e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0061b0 tx=0x7f25c000e970 comp rx=0 tx=0).stop 2024-09-17T02:40:27.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.758+0000 7f25d0666640 1 -- 172.21.15.131:0/3452272719 shutdown_connections 2024-09-17T02:40:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.758+0000 7f25d0666640 1 --2- 172.21.15.131:0/3452272719 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c81587f0 0x7f25c81f56e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.758+0000 7f25d0666640 1 --2- 172.21.15.131:0/3452272719 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c80b55b0 0x7f25c8145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.758+0000 7f25d0666640 1 --2- 172.21.15.131:0/3452272719 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25b4004810 0x7f25c81e6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.758+0000 7f25d0666640 1 -- 172.21.15.131:0/3452272719 >> 172.21.15.131:0/3452272719 conn(0x7f25c80d1130 msgr2=0x7f25c815f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.758+0000 7f25d0666640 1 -- 172.21.15.131:0/3452272719 shutdown_connections 2024-09-17T02:40:27.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.758+0000 7f25d0666640 1 -- 172.21.15.131:0/3452272719 wait complete. 2024-09-17T02:40:27.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.774+0000 7f151c9f2640 1 -- 172.21.15.131:0/506969689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f8006640 msgr2=0x7f14f8003b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.774+0000 7f151c9f2640 1 --2- 172.21.15.131:0/506969689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f8006640 0x7f14f8003b00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f14f8005cd0 tx=0x7f150801d550 comp rx=0 tx=0).stop 2024-09-17T02:40:27.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.774+0000 7f151c9f2640 1 -- 172.21.15.131:0/506969689 shutdown_connections 2024-09-17T02:40:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.774+0000 7f151c9f2640 1 --2- 172.21.15.131:0/506969689 >> v2:172.21.15.189:3300/0 conn(0x7f14f800b300 0x7f14f800d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.774+0000 7f151c9f2640 1 --2- 172.21.15.131:0/506969689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f8006640 0x7f14f8003b00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.774+0000 7f151c9f2640 1 -- 172.21.15.131:0/506969689 >> 172.21.15.131:0/506969689 conn(0x7f15180d1130 msgr2=0x7f15180d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.774+0000 7f151c9f2640 1 -- 172.21.15.131:0/506969689 shutdown_connections 2024-09-17T02:40:27.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.774+0000 7f151c9f2640 1 -- 172.21.15.131:0/506969689 wait complete. 2024-09-17T02:40:27.773 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.773 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.778+0000 7f151c9f2640 1 Processor -- start 2024-09-17T02:40:27.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.778+0000 7f151c9f2640 1 -- start start 2024-09-17T02:40:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f151c9f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14f8004040 0x7f1518152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f151c9f2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14f8006640 0x7f151814d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f151c9f2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f800b300 0x7f1518159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f151c9f2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f15180cd900 con 0x7f14f800b300 2024-09-17T02:40:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f151c9f2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1518071480 con 0x7f14f8004040 2024-09-17T02:40:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f151c9f2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f15180cda80 con 0x7f14f8006640 2024-09-17T02:40:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1516d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f800b300 0x7f1518159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1516d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f800b300 0x7f1518159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49886/0 (socket says 172.21.15.131:49886) 2024-09-17T02:40:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1516d9d640 1 -- 172.21.15.131:0/3053580406 learned_addr learned my addr 172.21.15.131:0/3053580406 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:27.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f151659c640 1 --2- 172.21.15.131:0/3053580406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14f8004040 0x7f1518152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1515d9b640 1 --2- 172.21.15.131:0/3053580406 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14f8006640 0x7f151814d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1516d9d640 1 -- 172.21.15.131:0/3053580406 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14f8006640 msgr2=0x7f151814d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1516d9d640 1 --2- 172.21.15.131:0/3053580406 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14f8006640 0x7f151814d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1516d9d640 1 -- 172.21.15.131:0/3053580406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14f8004040 msgr2=0x7f1518152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1516d9d640 1 --2- 172.21.15.131:0/3053580406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14f8004040 0x7f1518152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1516d9d640 1 -- 172.21.15.131:0/3053580406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151814e500 con 0x7f14f800b300 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1516d9d640 1 --2- 172.21.15.131:0/3053580406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f800b300 0x7f1518159310 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f14f8006160 tx=0x7f1510001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1506ffd640 1 -- 172.21.15.131:0/3053580406 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510010890 con 0x7f14f800b300 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1506ffd640 1 -- 172.21.15.131:0/3053580406 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1510017070 con 0x7f14f800b300 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f151c9f2640 1 -- 172.21.15.131:0/3053580406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151814e7a0 con 0x7f14f800b300 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1506ffd640 1 -- 172.21.15.131:0/3053580406 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1510010da0 con 0x7f14f800b300 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f151c9f2640 1 -- 172.21.15.131:0/3053580406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1518151740 con 0x7f14f800b300 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1506ffd640 1 -- 172.21.15.131:0/3053580406 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f151001c460 con 0x7f14f800b300 2024-09-17T02:40:27.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f1506ffd640 1 -- 172.21.15.131:0/3053580406 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1510011870 con 0x7f14f800b300 2024-09-17T02:40:27.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.782+0000 7f151c9f2640 1 -- 172.21.15.131:0/3053580406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15181514b0 con 0x7f14f800b300 2024-09-17T02:40:27.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.786+0000 7f1506ffd640 1 -- 172.21.15.131:0/3053580406 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f15180cff70 con 0x7f14f800b300 2024-09-17T02:40:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.818+0000 7f971392e640 1 -- 172.21.15.131:0/2679667948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f0006650 msgr2=0x7f96f0003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.818+0000 7f971392e640 1 --2- 172.21.15.131:0/2679667948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f0006650 0x7f96f0003b30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9708006bf0 tx=0x7f970801d650 comp rx=0 tx=0).stop 2024-09-17T02:40:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.818+0000 7f971392e640 1 -- 172.21.15.131:0/2679667948 shutdown_connections 2024-09-17T02:40:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.818+0000 7f971392e640 1 --2- 172.21.15.131:0/2679667948 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96f0005c50 0x7f96f000d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.818+0000 7f971392e640 1 --2- 172.21.15.131:0/2679667948 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f0006650 0x7f96f0003b30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.818+0000 7f971392e640 1 -- 172.21.15.131:0/2679667948 >> 172.21.15.131:0/2679667948 conn(0x7f970c0d1130 msgr2=0x7f970c0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.818+0000 7f971392e640 1 -- 172.21.15.131:0/2679667948 shutdown_connections 2024-09-17T02:40:27.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.818+0000 7f971392e640 1 -- 172.21.15.131:0/2679667948 wait complete. 2024-09-17T02:40:27.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f971392e640 1 Processor -- start 2024-09-17T02:40:27.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f971392e640 1 -- start start 2024-09-17T02:40:27.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f971392e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f0004070 0x7f970c0c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f971392e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96f0005c50 0x7f970c150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f971392e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96f0006650 0x7f970c1514d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f971392e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f970c160e00 con 0x7f96f0004070 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f971392e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f970c160c80 con 0x7f96f0005c50 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f971392e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f970c160f80 con 0x7f96f0006650 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f97116ca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f0004070 0x7f970c0c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f97116ca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f0004070 0x7f970c0c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49912/0 (socket says 172.21.15.131:49912) 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f97116ca640 1 -- 172.21.15.131:0/4107245018 learned_addr learned my addr 172.21.15.131:0/4107245018 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f9711ecb640 1 --2- 172.21.15.131:0/4107245018 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96f0006650 0x7f970c1514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f9710ec9640 1 --2- 172.21.15.131:0/4107245018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96f0005c50 0x7f970c150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f97116ca640 1 -- 172.21.15.131:0/4107245018 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96f0006650 msgr2=0x7f970c1514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f97116ca640 1 --2- 172.21.15.131:0/4107245018 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96f0006650 0x7f970c1514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f97116ca640 1 -- 172.21.15.131:0/4107245018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96f0005c50 msgr2=0x7f970c150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f97116ca640 1 --2- 172.21.15.131:0/4107245018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96f0005c50 0x7f970c150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f97116ca640 1 -- 172.21.15.131:0/4107245018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c153cb0 con 0x7f96f0004070 2024-09-17T02:40:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f97116ca640 1 --2- 172.21.15.131:0/4107245018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f0004070 0x7f970c0c6050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f970801def0 tx=0x7f97080ac510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f9701ffb640 1 -- 172.21.15.131:0/4107245018 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97080bb070 con 0x7f96f0004070 2024-09-17T02:40:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f9701ffb640 1 -- 172.21.15.131:0/4107245018 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9708004410 con 0x7f96f0004070 2024-09-17T02:40:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f971392e640 1 -- 172.21.15.131:0/4107245018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c153f90 con 0x7f96f0004070 2024-09-17T02:40:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f9701ffb640 1 -- 172.21.15.131:0/4107245018 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97080acaf0 con 0x7f96f0004070 2024-09-17T02:40:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.822+0000 7f971392e640 1 -- 172.21.15.131:0/4107245018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c154550 con 0x7f96f0004070 2024-09-17T02:40:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.826+0000 7f9701ffb640 1 -- 172.21.15.131:0/4107245018 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f97080c6060 con 0x7f96f0004070 2024-09-17T02:40:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.826+0000 7f9701ffb640 1 -- 172.21.15.131:0/4107245018 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9708003b80 con 0x7f96f0004070 2024-09-17T02:40:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.826+0000 7f971392e640 1 -- 172.21.15.131:0/4107245018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970c06e6f0 con 0x7f96f0004070 2024-09-17T02:40:27.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.826+0000 7f9701ffb640 1 -- 172.21.15.131:0/4107245018 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f97080e0860 con 0x7f96f0004070 2024-09-17T02:40:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.870+0000 7f151c9f2640 1 -- 172.21.15.131:0/3053580406 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f15180cdfe0 con 0x7f14f800b300 2024-09-17T02:40:27.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.874+0000 7f1506ffd640 1 -- 172.21.15.131:0/3053580406 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v22)=0 v22) ==== 106+0+0 (secure 0 0 0) 0x7f1510011a70 con 0x7f14f800b300 2024-09-17T02:40:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.874+0000 7f151c9f2640 1 -- 172.21.15.131:0/3053580406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f800b300 msgr2=0x7f1518159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.874+0000 7f151c9f2640 1 --2- 172.21.15.131:0/3053580406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f800b300 0x7f1518159310 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f14f8006160 tx=0x7f1510001aa0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.874+0000 7f151c9f2640 1 -- 172.21.15.131:0/3053580406 shutdown_connections 2024-09-17T02:40:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.874+0000 7f151c9f2640 1 --2- 172.21.15.131:0/3053580406 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14f800b300 0x7f1518159310 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.874+0000 7f151c9f2640 1 --2- 172.21.15.131:0/3053580406 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14f8006640 0x7f151814d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.874+0000 7f151c9f2640 1 --2- 172.21.15.131:0/3053580406 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14f8004040 0x7f1518152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.874+0000 7f151c9f2640 1 -- 172.21.15.131:0/3053580406 >> 172.21.15.131:0/3053580406 conn(0x7f15180d1130 msgr2=0x7f15180b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.874+0000 7f151c9f2640 1 -- 172.21.15.131:0/3053580406 shutdown_connections 2024-09-17T02:40:27.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.874+0000 7f151c9f2640 1 -- 172.21.15.131:0/3053580406 wait complete. 2024-09-17T02:40:27.897 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.922+0000 7f971392e640 1 -- 172.21.15.131:0/4107245018 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f970c156b40 con 0x7f96f0004070 2024-09-17T02:40:27.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.926+0000 7f9701ffb640 1 -- 172.21.15.131:0/4107245018 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v23)=0 v23) ==== 125+0+0 (secure 0 0 0) 0x7f97080ce020 con 0x7f96f0004070 2024-09-17T02:40:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.926+0000 7f971392e640 1 -- 172.21.15.131:0/4107245018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f0004070 msgr2=0x7f970c0c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.926+0000 7f971392e640 1 --2- 172.21.15.131:0/4107245018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f0004070 0x7f970c0c6050 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f970801def0 tx=0x7f97080ac510 comp rx=0 tx=0).stop 2024-09-17T02:40:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.926+0000 7f971392e640 1 -- 172.21.15.131:0/4107245018 shutdown_connections 2024-09-17T02:40:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.926+0000 7f971392e640 1 --2- 172.21.15.131:0/4107245018 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96f0006650 0x7f970c1514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.926+0000 7f971392e640 1 --2- 172.21.15.131:0/4107245018 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96f0005c50 0x7f970c150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.926+0000 7f971392e640 1 --2- 172.21.15.131:0/4107245018 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96f0004070 0x7f970c0c6050 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.926+0000 7f971392e640 1 -- 172.21.15.131:0/4107245018 >> 172.21.15.131:0/4107245018 conn(0x7f970c0d1130 msgr2=0x7f970c15f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.926+0000 7f971392e640 1 -- 172.21.15.131:0/4107245018 shutdown_connections 2024-09-17T02:40:27.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:27.926+0000 7f971392e640 1 -- 172.21.15.131:0/4107245018 wait complete. 2024-09-17T02:40:27.941 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.941 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi131') 2024-09-17T02:40:27.941 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config dump 2024-09-17T02:40:28.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.126+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/1163594383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc0d0710 msgr2=0x7ff4cc0d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:28.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.126+0000 7ff4d2cb7640 1 --2- 172.21.15.131:0/1163594383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc0d0710 0x7ff4cc0d0b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc072ae0 tx=0x7ff4c001c6e0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.126+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/1163594383 shutdown_connections 2024-09-17T02:40:28.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.126+0000 7ff4d2cb7640 1 --2- 172.21.15.131:0/1163594383 >> v2:172.21.15.189:3300/0 conn(0x7ff4cc0cd900 0x7ff4cc0cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.126+0000 7ff4d2cb7640 1 --2- 172.21.15.131:0/1163594383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc0d0710 0x7ff4cc0d0b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.126+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/1163594383 >> 172.21.15.131:0/1163594383 conn(0x7ff4cc0d1130 msgr2=0x7ff4cc0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.126+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/1163594383 shutdown_connections 2024-09-17T02:40:28.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.126+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/1163594383 wait complete. 2024-09-17T02:40:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d2cb7640 1 Processor -- start 2024-09-17T02:40:28.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d2cb7640 1 -- start start 2024-09-17T02:40:28.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d2cb7640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4cc071210 0x7ff4cc1f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d2cb7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc0cd900 0x7ff4cc1f93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d2cb7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4cc166c30 0x7ff4cc0b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d2cb7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc169840 con 0x7ff4cc0cd900 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d2cb7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff4cc1696c0 con 0x7ff4cc166c30 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d2cb7640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff4cc1699c0 con 0x7ff4cc071210 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4cbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc0cd900 0x7ff4cc1f93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4cbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc0cd900 0x7ff4cc1f93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49932/0 (socket says 172.21.15.131:49932) 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4cbfff640 1 -- 172.21.15.131:0/2442577896 learned_addr learned my addr 172.21.15.131:0/2442577896 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d0a53640 1 --2- 172.21.15.131:0/2442577896 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4cc071210 0x7ff4cc1f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d1254640 1 --2- 172.21.15.131:0/2442577896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4cc166c30 0x7ff4cc0b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4cbfff640 1 -- 172.21.15.131:0/2442577896 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4cc071210 msgr2=0x7ff4cc1f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:28.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4cbfff640 1 --2- 172.21.15.131:0/2442577896 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4cc071210 0x7ff4cc1f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4cbfff640 1 -- 172.21.15.131:0/2442577896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4cc166c30 msgr2=0x7ff4cc0b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4cbfff640 1 --2- 172.21.15.131:0/2442577896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4cc166c30 0x7ff4cc0b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4cbfff640 1 -- 172.21.15.131:0/2442577896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4cc165b90 con 0x7ff4cc0cd900 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d1254640 1 --2- 172.21.15.131:0/2442577896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4cc166c30 0x7ff4cc0b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4cbfff640 1 --2- 172.21.15.131:0/2442577896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc0cd900 0x7ff4cc1f93b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800c940 tx=0x7ff4b800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4c97fa640 1 -- 172.21.15.131:0/2442577896 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8004220 con 0x7ff4cc0cd900 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4c97fa640 1 -- 172.21.15.131:0/2442577896 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4b80079b0 con 0x7ff4cc0cd900 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4c97fa640 1 -- 172.21.15.131:0/2442577896 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80044b0 con 0x7ff4cc0cd900 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/2442577896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4cc165e70 con 0x7ff4cc0cd900 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/2442577896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4cc166430 con 0x7ff4cc0cd900 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4c97fa640 1 -- 172.21.15.131:0/2442577896 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff4b80053b0 con 0x7ff4cc0cd900 2024-09-17T02:40:28.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.130+0000 7ff4c97fa640 1 -- 172.21.15.131:0/2442577896 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff4b80038b0 con 0x7ff4cc0cd900 2024-09-17T02:40:28.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.134+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/2442577896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4cc1658e0 con 0x7ff4cc0cd900 2024-09-17T02:40:28.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.134+0000 7ff4c97fa640 1 -- 172.21.15.131:0/2442577896 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7ff4b8014030 con 0x7ff4cc0cd900 2024-09-17T02:40:28.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.190+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/2442577896 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7ff4cc0d0b20 con 0x7ff4cc0cd900 2024-09-17T02:40:28.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.190+0000 7ff4c97fa640 1 -- 172.21.15.131:0/2442577896 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7ff4b8007de0 con 0x7ff4cc0cd900 2024-09-17T02:40:28.188 INFO:teuthology.orchestra.run.smithi131.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T02:40:28.188 INFO:teuthology.orchestra.run.smithi131.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced client_mount_timeout 30 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_client 20/20 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_mgr 20/20 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_ms 1/1 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds 20/20 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_ms 2/2 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_frag true 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_scatterstat true 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_verify_scatter true 2024-09-17T02:40:28.189 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T02:40:28.190 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T02:40:28.190 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T02:40:28.190 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced client_mount_timeout 600 2024-09-17T02:40:28.190 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_client 20/20 2024-09-17T02:40:28.190 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_ms 1/1 2024-09-17T02:40:28.190 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T02:40:28.190 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T02:40:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.194+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/2442577896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc0cd900 msgr2=0x7ff4cc1f93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.194+0000 7ff4d2cb7640 1 --2- 172.21.15.131:0/2442577896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc0cd900 0x7ff4cc1f93b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800c940 tx=0x7ff4b800ce00 comp rx=0 tx=0).stop 2024-09-17T02:40:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.194+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/2442577896 shutdown_connections 2024-09-17T02:40:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.194+0000 7ff4d2cb7640 1 --2- 172.21.15.131:0/2442577896 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4cc166c30 0x7ff4cc0b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.194+0000 7ff4d2cb7640 1 --2- 172.21.15.131:0/2442577896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4cc0cd900 0x7ff4cc1f93b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.194+0000 7ff4d2cb7640 1 --2- 172.21.15.131:0/2442577896 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4cc071210 0x7ff4cc1f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.194+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/2442577896 >> 172.21.15.131:0/2442577896 conn(0x7ff4cc0d1130 msgr2=0x7ff4cc06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.194+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/2442577896 shutdown_connections 2024-09-17T02:40:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.194+0000 7ff4d2cb7640 1 -- 172.21.15.131:0/2442577896 wait complete. 2024-09-17T02:40:28.208 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T02:40:28.208 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T02:40:28.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.358+0000 7fd155734640 1 -- 172.21.15.131:0/1655880456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd15015a470 msgr2=0x7fd15015a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:28.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.358+0000 7fd155734640 1 --2- 172.21.15.131:0/1655880456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd15015a470 0x7fd15015a880 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd1500ce860 tx=0x7fd14801cd50 comp rx=0 tx=0).stop 2024-09-17T02:40:28.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.358+0000 7fd155734640 1 -- 172.21.15.131:0/1655880456 shutdown_connections 2024-09-17T02:40:28.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.358+0000 7fd155734640 1 --2- 172.21.15.131:0/1655880456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd15015a470 0x7fd15015a880 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.358+0000 7fd155734640 1 --2- 172.21.15.131:0/1655880456 >> v2:172.21.15.189:3300/0 conn(0x7fd1500cc3b0 0x7fd1500cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.358+0000 7fd155734640 1 --2- 172.21.15.131:0/1655880456 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd15015f010 0x7fd15006e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.358+0000 7fd155734640 1 -- 172.21.15.131:0/1655880456 >> 172.21.15.131:0/1655880456 conn(0x7fd1500cd6a0 msgr2=0x7fd1500cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:28.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.358+0000 7fd155734640 1 -- 172.21.15.131:0/1655880456 shutdown_connections 2024-09-17T02:40:28.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.358+0000 7fd155734640 1 -- 172.21.15.131:0/1655880456 wait complete. 2024-09-17T02:40:28.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.358+0000 7fd155734640 1 Processor -- start 2024-09-17T02:40:28.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.358+0000 7fd155734640 1 -- start start 2024-09-17T02:40:28.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd155734640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd1500cc3b0 0x7fd150153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd155734640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd15015a470 0x7fd1501fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd155734640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd15015f010 0x7fd1501fc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd155734640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd1500cb800 con 0x7fd15015a470 2024-09-17T02:40:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd155734640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd1500cb680 con 0x7fd15015f010 2024-09-17T02:40:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd155734640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd1500cb980 con 0x7fd1500cc3b0 2024-09-17T02:40:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd15015a470 0x7fd1501fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd15015a470 0x7fd1501fbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49956/0 (socket says 172.21.15.131:49956) 2024-09-17T02:40:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14e7fc640 1 -- 172.21.15.131:0/4067042042 learned_addr learned my addr 172.21.15.131:0/4067042042 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14effd640 1 --2- 172.21.15.131:0/4067042042 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd1500cc3b0 0x7fd150153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14f7fe640 1 --2- 172.21.15.131:0/4067042042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd15015f010 0x7fd1501fc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14e7fc640 1 -- 172.21.15.131:0/4067042042 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd1500cc3b0 msgr2=0x7fd150153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14e7fc640 1 --2- 172.21.15.131:0/4067042042 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd1500cc3b0 0x7fd150153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14e7fc640 1 -- 172.21.15.131:0/4067042042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd15015f010 msgr2=0x7fd1501fc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14e7fc640 1 --2- 172.21.15.131:0/4067042042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd15015f010 0x7fd1501fc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14e7fc640 1 -- 172.21.15.131:0/4067042042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd150153940 con 0x7fd15015a470 2024-09-17T02:40:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14effd640 1 --2- 172.21.15.131:0/4067042042 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd1500cc3b0 0x7fd150153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:40:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd14e7fc640 1 --2- 172.21.15.131:0/4067042042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd15015a470 0x7fd1501fbeb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd14400c9a0 tx=0x7fd14400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd1377fe640 1 -- 172.21.15.131:0/4067042042 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144007bb0 con 0x7fd15015a470 2024-09-17T02:40:28.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd1377fe640 1 -- 172.21.15.131:0/4067042042 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd144007e40 con 0x7fd15015a470 2024-09-17T02:40:28.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd155734640 1 -- 172.21.15.131:0/4067042042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1501fcc30 con 0x7fd15015a470 2024-09-17T02:40:28.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd1377fe640 1 -- 172.21.15.131:0/4067042042 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd144005260 con 0x7fd15015a470 2024-09-17T02:40:28.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd155734640 1 -- 172.21.15.131:0/4067042042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1500b1e50 con 0x7fd15015a470 2024-09-17T02:40:28.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd1377fe640 1 -- 172.21.15.131:0/4067042042 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd1440047b0 con 0x7fd15015a470 2024-09-17T02:40:28.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd1377fe640 1 -- 172.21.15.131:0/4067042042 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd14400fc40 con 0x7fd15015a470 2024-09-17T02:40:28.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.362+0000 7fd155734640 1 -- 172.21.15.131:0/4067042042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1500b2090 con 0x7fd15015a470 2024-09-17T02:40:28.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.366+0000 7fd1377fe640 1 -- 172.21.15.131:0/4067042042 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd144014030 con 0x7fd15015a470 2024-09-17T02:40:28.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.426+0000 7fd155734640 1 -- 172.21.15.131:0/4067042042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd15015a880 con 0x7fd15015a470 2024-09-17T02:40:28.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.430+0000 7fd1377fe640 1 -- 172.21.15.131:0/4067042042 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fd144004c80 con 0x7fd15015a470 2024-09-17T02:40:28.425 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2024-09-17T02:40:28.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.430+0000 7fd155734640 1 -- 172.21.15.131:0/4067042042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd15015a470 msgr2=0x7fd1501fbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:28.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.430+0000 7fd155734640 1 --2- 172.21.15.131:0/4067042042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd15015a470 0x7fd1501fbeb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd14400c9a0 tx=0x7fd14400ce60 comp rx=0 tx=0).stop 2024-09-17T02:40:28.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.430+0000 7fd155734640 1 -- 172.21.15.131:0/4067042042 shutdown_connections 2024-09-17T02:40:28.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.430+0000 7fd155734640 1 --2- 172.21.15.131:0/4067042042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd15015f010 0x7fd1501fc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.430+0000 7fd155734640 1 --2- 172.21.15.131:0/4067042042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd15015a470 0x7fd1501fbeb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.430+0000 7fd155734640 1 --2- 172.21.15.131:0/4067042042 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd1500cc3b0 0x7fd150153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:28.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.430+0000 7fd155734640 1 -- 172.21.15.131:0/4067042042 >> 172.21.15.131:0/4067042042 conn(0x7fd1500cd6a0 msgr2=0x7fd1501599b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:28.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.430+0000 7fd155734640 1 -- 172.21.15.131:0/4067042042 shutdown_connections 2024-09-17T02:40:28.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:28.430+0000 7fd155734640 1 -- 172.21.15.131:0/4067042042 wait complete. 2024-09-17T02:40:28.445 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T02:40:28.445 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T02:40:28.445 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:28.445 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T02:40:28.456 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:28.456 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T02:40:28.509 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:28.509 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T02:40:28.564 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:40:28.565 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T02:40:28.619 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:28.620 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T02:40:28.634 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:28.634 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T02:40:28.687 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:28.687 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T02:40:28.772 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:40:28.772 DEBUG:teuthology.orchestra.run.smithi189:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T02:40:28.825 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 5ed37453-7b40-4153-8943-f14db057d578 0 2024-09-17T02:40:29.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.023+0000 7f2649e2a640 1 -- 172.21.15.189:0/57381404 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26440b41a0 msgr2=0x7f26440b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.023+0000 7f2649e2a640 1 --2- 172.21.15.189:0/57381404 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26440b41a0 0x7f26440b45b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2638009840 tx=0x7f263801c820 comp rx=0 tx=0).stop 2024-09-17T02:40:29.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.023+0000 7f2649e2a640 1 -- 172.21.15.189:0/57381404 shutdown_connections 2024-09-17T02:40:29.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.023+0000 7f2649e2a640 1 --2- 172.21.15.189:0/57381404 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26440b41a0 0x7f26440b45b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.023+0000 7f2649e2a640 1 --2- 172.21.15.189:0/57381404 >> v2:172.21.15.131:3300/0 conn(0x7f2644156bd0 0x7f26440cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.023+0000 7f2649e2a640 1 -- 172.21.15.189:0/57381404 >> 172.21.15.189:0/57381404 conn(0x7f26440cf150 msgr2=0x7f26440cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:29.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.023+0000 7f2649e2a640 1 -- 172.21.15.189:0/57381404 shutdown_connections 2024-09-17T02:40:29.021 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.023+0000 7f2649e2a640 1 -- 172.21.15.189:0/57381404 wait complete. 2024-09-17T02:40:29.023 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2649e2a640 1 Processor -- start 2024-09-17T02:40:29.023 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2649e2a640 1 -- start start 2024-09-17T02:40:29.026 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2649e2a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26440b41a0 0x7f26440b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.027 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2649e2a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2644156bd0 0x7f264415c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.027 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2649e2a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26441601a0 0x7f264415c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.027 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2649e2a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2644169ee0 con 0x7f26441601a0 2024-09-17T02:40:29.027 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2649e2a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2644169d60 con 0x7f2644156bd0 2024-09-17T02:40:29.027 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2649e2a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f264416a060 con 0x7f26440b41a0 2024-09-17T02:40:29.027 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f26437fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26440b41a0 0x7f26440b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.027 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2642ffd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2644156bd0 0x7f264415c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.027 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f26437fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26440b41a0 0x7f26440b79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.189:37456/0 (socket says 172.21.15.189:37456) 2024-09-17T02:40:29.027 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f26437fe640 1 -- 172.21.15.189:0/1029496704 learned_addr learned my addr 172.21.15.189:0/1029496704 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-17T02:40:29.027 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2643fff640 1 --2- 172.21.15.189:0/1029496704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26441601a0 0x7f264415c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.027 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f26437fe640 1 -- 172.21.15.189:0/1029496704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2644156bd0 msgr2=0x7f264415c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f26437fe640 1 --2- 172.21.15.189:0/1029496704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2644156bd0 0x7f264415c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f26437fe640 1 -- 172.21.15.189:0/1029496704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26441601a0 msgr2=0x7f264415c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f26437fe640 1 --2- 172.21.15.189:0/1029496704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26441601a0 0x7f264415c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f26437fe640 1 -- 172.21.15.189:0/1029496704 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26440b7f30 con 0x7f26440b41a0 2024-09-17T02:40:29.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2643fff640 1 --2- 172.21.15.189:0/1029496704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26441601a0 0x7f264415c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:40:29.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f26437fe640 1 --2- 172.21.15.189:0/1029496704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26440b41a0 0x7f26440b79f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d950 tx=0x7f263c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:29.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f262bfff640 1 -- 172.21.15.189:0/1029496704 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c017070 con 0x7f26440b41a0 2024-09-17T02:40:29.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f262bfff640 1 -- 172.21.15.189:0/1029496704 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f263c009d20 con 0x7f26440b41a0 2024-09-17T02:40:29.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2649e2a640 1 -- 172.21.15.189:0/1029496704 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26440d1a60 con 0x7f26440b41a0 2024-09-17T02:40:29.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f262bfff640 1 -- 172.21.15.189:0/1029496704 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c0106c0 con 0x7f26440b41a0 2024-09-17T02:40:29.028 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2649e2a640 1 -- 172.21.15.189:0/1029496704 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f26441f70f0 con 0x7f26440b41a0 2024-09-17T02:40:29.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f262bfff640 1 -- 172.21.15.189:0/1029496704 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f263c01a070 con 0x7f26440b41a0 2024-09-17T02:40:29.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f262bfff640 1 -- 172.21.15.189:0/1029496704 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f263c01cb30 con 0x7f26440b41a0 2024-09-17T02:40:29.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.027+0000 7f2649e2a640 1 -- 172.21.15.189:0/1029496704 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26441f6c00 con 0x7f26440b41a0 2024-09-17T02:40:29.029 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.031+0000 7f262bfff640 1 -- 172.21.15.189:0/1029496704 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f263c010860 con 0x7f26440b41a0 2024-09-17T02:40:29.100 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.103+0000 7f2649e2a640 1 -- 172.21.15.189:0/1029496704 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5ed37453-7b40-4153-8943-f14db057d578", "id": 0} v 0) -- 0x7f26440b45b0 con 0x7f26440b41a0 2024-09-17T02:40:29.105 INFO:tasks.ceph.mgr.x.smithi189.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T02:40:29.105 INFO:tasks.ceph.mgr.x.smithi189.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T02:40:29.105 INFO:tasks.ceph.mgr.x.smithi189.stderr: from numpy import show_config as show_numpy_config 2024-09-17T02:40:29.105 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.107+0000 7f262bfff640 1 -- 172.21.15.189:0/1029496704 <== mon.2 v2:172.21.15.189:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5ed37453-7b40-4153-8943-f14db057d578", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f263c010a40 con 0x7f26440b41a0 2024-09-17T02:40:29.106 INFO:teuthology.orchestra.run.smithi189.stdout:0 2024-09-17T02:40:29.107 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.107+0000 7f2649e2a640 1 -- 172.21.15.189:0/1029496704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26440b41a0 msgr2=0x7f26440b79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.107 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.107+0000 7f2649e2a640 1 --2- 172.21.15.189:0/1029496704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26440b41a0 0x7f26440b79f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f263c00d950 tx=0x7f263c00de10 comp rx=0 tx=0).stop 2024-09-17T02:40:29.107 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.107+0000 7f2649e2a640 1 -- 172.21.15.189:0/1029496704 shutdown_connections 2024-09-17T02:40:29.107 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.107+0000 7f2649e2a640 1 --2- 172.21.15.189:0/1029496704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26441601a0 0x7f264415c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.107 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.107+0000 7f2649e2a640 1 --2- 172.21.15.189:0/1029496704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2644156bd0 0x7f264415c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.107 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.107+0000 7f2649e2a640 1 --2- 172.21.15.189:0/1029496704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26440b41a0 0x7f26440b79f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.107+0000 7f2649e2a640 1 -- 172.21.15.189:0/1029496704 >> 172.21.15.189:0/1029496704 conn(0x7f26440cf150 msgr2=0x7f26440d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:29.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.107+0000 7f2649e2a640 1 -- 172.21.15.189:0/1029496704 shutdown_connections 2024-09-17T02:40:29.108 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.107+0000 7f2649e2a640 1 -- 172.21.15.189:0/1029496704 wait complete. 2024-09-17T02:40:29.124 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new fdcf031e-6ed3-418a-b404-6fda63045e6a 1 2024-09-17T02:40:29.251 INFO:tasks.ceph.mgr.y.smithi131.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-17T02:40:29.251 INFO:tasks.ceph.mgr.y.smithi131.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-17T02:40:29.251 INFO:tasks.ceph.mgr.y.smithi131.stderr: from numpy import show_config as show_numpy_config 2024-09-17T02:40:29.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.275+0000 7f39d723c640 1 -- 172.21.15.189:0/1554113658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39d00cca70 msgr2=0x7f39d0167a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.273 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.275+0000 7f39d723c640 1 --2- 172.21.15.189:0/1554113658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39d00cca70 0x7f39d0167a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009bc0 tx=0x7f39c801cb70 comp rx=0 tx=0).stop 2024-09-17T02:40:29.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.275+0000 7f39d723c640 1 -- 172.21.15.189:0/1554113658 shutdown_connections 2024-09-17T02:40:29.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.275+0000 7f39d723c640 1 --2- 172.21.15.189:0/1554113658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39d00cca70 0x7f39d0167a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.275+0000 7f39d723c640 1 -- 172.21.15.189:0/1554113658 >> 172.21.15.189:0/1554113658 conn(0x7f39d00cf150 msgr2=0x7f39d00cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:29.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.275+0000 7f39d723c640 1 -- 172.21.15.189:0/1554113658 shutdown_connections 2024-09-17T02:40:29.274 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.275+0000 7f39d723c640 1 -- 172.21.15.189:0/1554113658 wait complete. 2024-09-17T02:40:29.275 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.275+0000 7f39d723c640 1 Processor -- start 2024-09-17T02:40:29.275 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d723c640 1 -- start start 2024-09-17T02:40:29.280 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d723c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39d00cca70 0x7f39d00b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.280 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d723c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f39d0156bd0 0x7f39d015c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.280 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d723c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39d015ca80 0x7f39d0157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.280 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d723c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f39d016ab30 con 0x7f39d015ca80 2024-09-17T02:40:29.281 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d723c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39d016a9b0 con 0x7f39d00cca70 2024-09-17T02:40:29.281 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d723c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f39d016acb0 con 0x7f39d0156bd0 2024-09-17T02:40:29.281 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d4fd8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39d00cca70 0x7f39d00b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.281 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39cffff640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f39d0156bd0 0x7f39d015c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.281 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d4fd8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39d00cca70 0x7f39d00b7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:40822/0 (socket says 172.21.15.189:40822) 2024-09-17T02:40:29.281 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39cffff640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f39d0156bd0 0x7f39d015c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.189:37468/0 (socket says 172.21.15.189:37468) 2024-09-17T02:40:29.282 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d4fd8640 1 -- 172.21.15.189:0/3733270278 learned_addr learned my addr 172.21.15.189:0/3733270278 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-17T02:40:29.282 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d57d9640 1 --2- 172.21.15.189:0/3733270278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39d015ca80 0x7f39d0157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.282 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d4fd8640 1 -- 172.21.15.189:0/3733270278 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f39d0156bd0 msgr2=0x7f39d015c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.282 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d4fd8640 1 --2- 172.21.15.189:0/3733270278 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f39d0156bd0 0x7f39d015c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.282 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d4fd8640 1 -- 172.21.15.189:0/3733270278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39d015ca80 msgr2=0x7f39d0157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.282 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d4fd8640 1 --2- 172.21.15.189:0/3733270278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39d015ca80 0x7f39d0157270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.283 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d4fd8640 1 -- 172.21.15.189:0/3733270278 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d00c4970 con 0x7f39d00cca70 2024-09-17T02:40:29.283 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39cffff640 1 --2- 172.21.15.189:0/3733270278 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f39d0156bd0 0x7f39d015c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T02:40:29.283 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d4fd8640 1 --2- 172.21.15.189:0/3733270278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39d00cca70 0x7f39d00b7ba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f39c000c3c0 tx=0x7f39c000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:29.283 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39cd7fa640 1 -- 172.21.15.189:0/3733270278 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c00052d0 con 0x7f39d00cca70 2024-09-17T02:40:29.283 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39cd7fa640 1 -- 172.21.15.189:0/3733270278 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39c0007020 con 0x7f39d00cca70 2024-09-17T02:40:29.283 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d723c640 1 -- 172.21.15.189:0/3733270278 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d0160960 con 0x7f39d00cca70 2024-09-17T02:40:29.284 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39cd7fa640 1 -- 172.21.15.189:0/3733270278 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c0007470 con 0x7f39d00cca70 2024-09-17T02:40:29.284 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d723c640 1 -- 172.21.15.189:0/3733270278 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d00c5530 con 0x7f39d00cca70 2024-09-17T02:40:29.284 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39cd7fa640 1 -- 172.21.15.189:0/3733270278 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f39c00067e0 con 0x7f39d00cca70 2024-09-17T02:40:29.284 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39cd7fa640 1 -- 172.21.15.189:0/3733270278 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f39c0003bc0 con 0x7f39d00cca70 2024-09-17T02:40:29.284 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.279+0000 7f39d723c640 1 -- 172.21.15.189:0/3733270278 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d00b77e0 con 0x7f39d00cca70 2024-09-17T02:40:29.285 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.283+0000 7f39cd7fa640 1 -- 172.21.15.189:0/3733270278 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f39c0003e10 con 0x7f39d00cca70 2024-09-17T02:40:29.351 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.355+0000 7f39d723c640 1 -- 172.21.15.189:0/3733270278 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fdcf031e-6ed3-418a-b404-6fda63045e6a", "id": 1} v 0) -- 0x7f39d00b41a0 con 0x7f39d00cca70 2024-09-17T02:40:29.357 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.359+0000 7f39cd7fa640 1 -- 172.21.15.189:0/3733270278 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fdcf031e-6ed3-418a-b404-6fda63045e6a", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f39c0007610 con 0x7f39d00cca70 2024-09-17T02:40:29.357 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-09-17T02:40:29.359 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.359+0000 7f39d723c640 1 -- 172.21.15.189:0/3733270278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39d00cca70 msgr2=0x7f39d00b7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.359 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.359+0000 7f39d723c640 1 --2- 172.21.15.189:0/3733270278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39d00cca70 0x7f39d00b7ba0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f39c000c3c0 tx=0x7f39c000c880 comp rx=0 tx=0).stop 2024-09-17T02:40:29.359 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.359+0000 7f39d723c640 1 -- 172.21.15.189:0/3733270278 shutdown_connections 2024-09-17T02:40:29.359 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.359+0000 7f39d723c640 1 --2- 172.21.15.189:0/3733270278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39d015ca80 0x7f39d0157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.359 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.359+0000 7f39d723c640 1 --2- 172.21.15.189:0/3733270278 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f39d0156bd0 0x7f39d015c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.359 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.359+0000 7f39d723c640 1 --2- 172.21.15.189:0/3733270278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f39d00cca70 0x7f39d00b7ba0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.359 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.359+0000 7f39d723c640 1 -- 172.21.15.189:0/3733270278 >> 172.21.15.189:0/3733270278 conn(0x7f39d00cf150 msgr2=0x7f39d015fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:29.359 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.359+0000 7f39d723c640 1 -- 172.21.15.189:0/3733270278 shutdown_connections 2024-09-17T02:40:29.359 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.359+0000 7f39d723c640 1 -- 172.21.15.189:0/3733270278 wait complete. 2024-09-17T02:40:29.377 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new d004f0b0-433d-46db-a270-e6f5f1f6e851 2 2024-09-17T02:40:29.517 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.519+0000 7f0d486d9640 1 -- 172.21.15.189:0/1286375249 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0d40152550 msgr2=0x7f0d4006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.518 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.519+0000 7f0d486d9640 1 --2- 172.21.15.189:0/1286375249 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0d40152550 0x7f0d4006e840 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0d4006e530 tx=0x7f0d3c01c7c0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.518 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.519+0000 7f0d486d9640 1 -- 172.21.15.189:0/1286375249 shutdown_connections 2024-09-17T02:40:29.518 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.519+0000 7f0d486d9640 1 --2- 172.21.15.189:0/1286375249 >> v2:172.21.15.131:3300/0 conn(0x7f0d40071920 0x7f0d400b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.518 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.519+0000 7f0d486d9640 1 --2- 172.21.15.189:0/1286375249 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0d40152550 0x7f0d4006e840 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.518 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.519+0000 7f0d486d9640 1 -- 172.21.15.189:0/1286375249 >> 172.21.15.189:0/1286375249 conn(0x7f0d400d1130 msgr2=0x7f0d400d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:29.518 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.519+0000 7f0d486d9640 1 -- 172.21.15.189:0/1286375249 shutdown_connections 2024-09-17T02:40:29.518 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.519+0000 7f0d486d9640 1 -- 172.21.15.189:0/1286375249 wait complete. 2024-09-17T02:40:29.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d486d9640 1 Processor -- start 2024-09-17T02:40:29.519 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d486d9640 1 -- start start 2024-09-17T02:40:29.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d486d9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d40152550 0x7f0d4014a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d486d9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d4015ba70 0x7f0d401584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d486d9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0d400d01f0 0x7f0d400d0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d486d9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401690b0 con 0x7f0d40152550 2024-09-17T02:40:29.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d486d9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0d40168f30 con 0x7f0d4015ba70 2024-09-17T02:40:29.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d486d9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0d40169230 con 0x7f0d400d01f0 2024-09-17T02:40:29.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d46c76640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0d400d01f0 0x7f0d400d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d45c74640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d4015ba70 0x7f0d401584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d46c76640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0d400d01f0 0x7f0d400d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.189:37484/0 (socket says 172.21.15.189:37484) 2024-09-17T02:40:29.522 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d46c76640 1 -- 172.21.15.189:0/2566508915 learned_addr learned my addr 172.21.15.189:0/2566508915 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d46475640 1 --2- 172.21.15.189:0/2566508915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d40152550 0x7f0d4014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d46c76640 1 -- 172.21.15.189:0/2566508915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d4015ba70 msgr2=0x7f0d401584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d46c76640 1 --2- 172.21.15.189:0/2566508915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d4015ba70 0x7f0d401584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d46c76640 1 -- 172.21.15.189:0/2566508915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d40152550 msgr2=0x7f0d4014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d46c76640 1 --2- 172.21.15.189:0/2566508915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d40152550 0x7f0d4014a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d46c76640 1 -- 172.21.15.189:0/2566508915 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d400d0b40 con 0x7f0d400d01f0 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d46c76640 1 --2- 172.21.15.189:0/2566508915 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0d400d01f0 0x7f0d400d0600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800cae0 tx=0x7f0d3800a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d32ffd640 1 -- 172.21.15.189:0/2566508915 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3800ae30 con 0x7f0d400d01f0 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d32ffd640 1 -- 172.21.15.189:0/2566508915 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d3800f560 con 0x7f0d400d01f0 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d486d9640 1 -- 172.21.15.189:0/2566508915 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d401560a0 con 0x7f0d400d01f0 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d32ffd640 1 -- 172.21.15.189:0/2566508915 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3800f9b0 con 0x7f0d400d01f0 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d486d9640 1 -- 172.21.15.189:0/2566508915 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d401fa4b0 con 0x7f0d400d01f0 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d32ffd640 1 -- 172.21.15.189:0/2566508915 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d3800fb50 con 0x7f0d400d01f0 2024-09-17T02:40:29.523 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d32ffd640 1 -- 172.21.15.189:0/2566508915 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f0d38022810 con 0x7f0d400d01f0 2024-09-17T02:40:29.524 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.523+0000 7f0d486d9640 1 -- 172.21.15.189:0/2566508915 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d40156270 con 0x7f0d400d01f0 2024-09-17T02:40:29.524 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.527+0000 7f0d32ffd640 1 -- 172.21.15.189:0/2566508915 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0d38038860 con 0x7f0d400d01f0 2024-09-17T02:40:29.591 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.591+0000 7f0d486d9640 1 -- 172.21.15.189:0/2566508915 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d004f0b0-433d-46db-a270-e6f5f1f6e851", "id": 2} v 0) -- 0x7f0d400b40e0 con 0x7f0d400d01f0 2024-09-17T02:40:29.596 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.599+0000 7f0d32ffd640 1 -- 172.21.15.189:0/2566508915 <== mon.2 v2:172.21.15.189:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d004f0b0-433d-46db-a270-e6f5f1f6e851", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f0d38026030 con 0x7f0d400d01f0 2024-09-17T02:40:29.597 INFO:teuthology.orchestra.run.smithi189.stdout:2 2024-09-17T02:40:29.599 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.599+0000 7f0d486d9640 1 -- 172.21.15.189:0/2566508915 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0d400d01f0 msgr2=0x7f0d400d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.599 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.599+0000 7f0d486d9640 1 --2- 172.21.15.189:0/2566508915 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0d400d01f0 0x7f0d400d0600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800cae0 tx=0x7f0d3800a250 comp rx=0 tx=0).stop 2024-09-17T02:40:29.600 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.599+0000 7f0d486d9640 1 -- 172.21.15.189:0/2566508915 shutdown_connections 2024-09-17T02:40:29.600 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.599+0000 7f0d486d9640 1 --2- 172.21.15.189:0/2566508915 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0d400d01f0 0x7f0d400d0600 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.600 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.599+0000 7f0d486d9640 1 --2- 172.21.15.189:0/2566508915 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0d4015ba70 0x7f0d401584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.600 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.599+0000 7f0d486d9640 1 --2- 172.21.15.189:0/2566508915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0d40152550 0x7f0d4014a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.600 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.599+0000 7f0d486d9640 1 -- 172.21.15.189:0/2566508915 >> 172.21.15.189:0/2566508915 conn(0x7f0d400d1130 msgr2=0x7f0d40071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:29.600 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.599+0000 7f0d486d9640 1 -- 172.21.15.189:0/2566508915 shutdown_connections 2024-09-17T02:40:29.600 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.599+0000 7f0d486d9640 1 -- 172.21.15.189:0/2566508915 wait complete. 2024-09-17T02:40:29.616 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new ae5eb561-b2d1-4e1f-93db-a2076d46bf5c 3 2024-09-17T02:40:29.757 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.759+0000 7facf3078640 1 -- 172.21.15.189:0/30717537 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec0cca70 msgr2=0x7facec167a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.757 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.759+0000 7facf3078640 1 --2- 172.21.15.189:0/30717537 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec0cca70 0x7facec167a40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7facec0d22a0 tx=0x7face401cd70 comp rx=0 tx=0).stop 2024-09-17T02:40:29.757 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.759+0000 7facf3078640 1 -- 172.21.15.189:0/30717537 shutdown_connections 2024-09-17T02:40:29.757 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.759+0000 7facf3078640 1 --2- 172.21.15.189:0/30717537 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec0cca70 0x7facec167a40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.757 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.759+0000 7facf3078640 1 -- 172.21.15.189:0/30717537 >> 172.21.15.189:0/30717537 conn(0x7facec0cf150 msgr2=0x7facec0cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:29.758 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.759+0000 7facf3078640 1 -- 172.21.15.189:0/30717537 shutdown_connections 2024-09-17T02:40:29.758 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.759+0000 7facf3078640 1 -- 172.21.15.189:0/30717537 wait complete. 2024-09-17T02:40:29.758 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.759+0000 7facf3078640 1 Processor -- start 2024-09-17T02:40:29.758 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.759+0000 7facf3078640 1 -- start start 2024-09-17T02:40:29.761 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf3078640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec0cca70 0x7facec155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.761 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf3078640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec156bd0 0x7facec0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.761 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf3078640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec155f90 0x7facec0c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:29.761 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf3078640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7facec16ab30 con 0x7facec0cca70 2024-09-17T02:40:29.761 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf3078640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facec16a9b0 con 0x7facec155f90 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf3078640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7facec16acb0 con 0x7facec156bd0 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf1615640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec155f90 0x7facec0c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facebfff640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec156bd0 0x7facec0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf1615640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec155f90 0x7facec0c6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:54028/0 (socket says 172.21.15.189:54028) 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf1615640 1 -- 172.21.15.189:0/1902390560 learned_addr learned my addr 172.21.15.189:0/1902390560 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facebfff640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec156bd0 0x7facec0c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.189:54584/0 (socket says 172.21.15.189:54584) 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf0e14640 1 --2- 172.21.15.189:0/1902390560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec0cca70 0x7facec155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf1615640 1 -- 172.21.15.189:0/1902390560 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec156bd0 msgr2=0x7facec0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf1615640 1 --2- 172.21.15.189:0/1902390560 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec156bd0 0x7facec0c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf1615640 1 -- 172.21.15.189:0/1902390560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec0cca70 msgr2=0x7facec155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf1615640 1 --2- 172.21.15.189:0/1902390560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec0cca70 0x7facec155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.762 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf1615640 1 -- 172.21.15.189:0/1902390560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec155c90 con 0x7facec155f90 2024-09-17T02:40:29.763 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf1615640 1 --2- 172.21.15.189:0/1902390560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec155f90 0x7facec0c6eb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7face4098920 tx=0x7face4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:29.763 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7face97fa640 1 -- 172.21.15.189:0/1902390560 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face40b2040 con 0x7facec155f90 2024-09-17T02:40:29.763 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7face97fa640 1 -- 172.21.15.189:0/1902390560 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7face40bb070 con 0x7facec155f90 2024-09-17T02:40:29.763 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf3078640 1 -- 172.21.15.189:0/1902390560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facec06f290 con 0x7facec155f90 2024-09-17T02:40:29.763 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7face97fa640 1 -- 172.21.15.189:0/1902390560 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4004370 con 0x7facec155f90 2024-09-17T02:40:29.763 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf3078640 1 -- 172.21.15.189:0/1902390560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facec0c9f30 con 0x7facec155f90 2024-09-17T02:40:29.763 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7face97fa640 1 -- 172.21.15.189:0/1902390560 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7face4004510 con 0x7facec155f90 2024-09-17T02:40:29.763 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7face97fa640 1 -- 172.21.15.189:0/1902390560 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7face40c0c80 con 0x7facec155f90 2024-09-17T02:40:29.764 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.763+0000 7facf3078640 1 -- 172.21.15.189:0/1902390560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facec0ca190 con 0x7facec155f90 2024-09-17T02:40:29.764 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.767+0000 7face97fa640 1 -- 172.21.15.189:0/1902390560 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7face4016990 con 0x7facec155f90 2024-09-17T02:40:29.829 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.831+0000 7facf3078640 1 -- 172.21.15.189:0/1902390560 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ae5eb561-b2d1-4e1f-93db-a2076d46bf5c", "id": 3} v 0) -- 0x7facec0b41a0 con 0x7facec155f90 2024-09-17T02:40:29.834 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.835+0000 7face97fa640 1 -- 172.21.15.189:0/1902390560 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ae5eb561-b2d1-4e1f-93db-a2076d46bf5c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7face40ad440 con 0x7facec155f90 2024-09-17T02:40:29.834 INFO:teuthology.orchestra.run.smithi189.stdout:3 2024-09-17T02:40:29.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.839+0000 7facf3078640 1 -- 172.21.15.189:0/1902390560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec155f90 msgr2=0x7facec0c6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.839+0000 7facf3078640 1 --2- 172.21.15.189:0/1902390560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec155f90 0x7facec0c6eb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7face4098920 tx=0x7face4005c00 comp rx=0 tx=0).stop 2024-09-17T02:40:29.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.839+0000 7facf3078640 1 -- 172.21.15.189:0/1902390560 shutdown_connections 2024-09-17T02:40:29.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.839+0000 7facf3078640 1 --2- 172.21.15.189:0/1902390560 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec155f90 0x7facec0c6eb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.839+0000 7facf3078640 1 --2- 172.21.15.189:0/1902390560 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec156bd0 0x7facec0c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.839+0000 7facf3078640 1 --2- 172.21.15.189:0/1902390560 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec0cca70 0x7facec155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.839+0000 7facf3078640 1 -- 172.21.15.189:0/1902390560 >> 172.21.15.189:0/1902390560 conn(0x7facec0cf150 msgr2=0x7facec15dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:29.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.839+0000 7facf3078640 1 -- 172.21.15.189:0/1902390560 shutdown_connections 2024-09-17T02:40:29.836 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:29.839+0000 7facf3078640 1 -- 172.21.15.189:0/1902390560 wait complete. 2024-09-17T02:40:29.853 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new a25e80d7-ac62-4a4c-a40f-95d4b43d985b 4 2024-09-17T02:40:29.999 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 -- 172.21.15.189:0/4176016137 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f100cfe00 msgr2=0x7f4f10070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:29.999 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 --2- 172.21.15.189:0/4176016137 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f100cfe00 0x7f4f10070cb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f1006e2c0 tx=0x7f4f0c01cd10 comp rx=0 tx=0).stop 2024-09-17T02:40:29.999 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 -- 172.21.15.189:0/4176016137 shutdown_connections 2024-09-17T02:40:29.999 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 --2- 172.21.15.189:0/4176016137 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f100cfe00 0x7f4f10070cb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:29.999 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 --2- 172.21.15.189:0/4176016137 >> v2:172.21.15.131:3300/0 conn(0x7f4f100cae70 0x7f4f100cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.000 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 -- 172.21.15.189:0/4176016137 >> 172.21.15.189:0/4176016137 conn(0x7f4f100cd6a0 msgr2=0x7f4f100cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:30.000 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 -- 172.21.15.189:0/4176016137 shutdown_connections 2024-09-17T02:40:30.000 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 -- 172.21.15.189:0/4176016137 wait complete. 2024-09-17T02:40:30.000 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 Processor -- start 2024-09-17T02:40:30.000 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 -- start start 2024-09-17T02:40:30.003 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f100cae70 0x7f4f101f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f100cf2e0 0x7f4f1015e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f100cfe00 0x7f4f1015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f10168d30 con 0x7f4f100cf2e0 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f10168bb0 con 0x7f4f100cae70 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f173fc640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4f10168eb0 con 0x7f4f100cfe00 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f15999640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f100cfe00 0x7f4f1015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f15198640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f100cae70 0x7f4f101f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f15999640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f100cfe00 0x7f4f1015e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.189:54604/0 (socket says 172.21.15.189:54604) 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f15999640 1 -- 172.21.15.189:0/859650090 learned_addr learned my addr 172.21.15.189:0/859650090 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f15198640 1 --2- 172.21.15.189:0/859650090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f100cae70 0x7f4f101f98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.189:54044/0 (socket says 172.21.15.189:54044) 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f14997640 1 --2- 172.21.15.189:0/859650090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f100cf2e0 0x7f4f1015e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.004 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f15999640 1 -- 172.21.15.189:0/859650090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f100cae70 msgr2=0x7f4f101f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f15999640 1 --2- 172.21.15.189:0/859650090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f100cae70 0x7f4f101f98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f15999640 1 -- 172.21.15.189:0/859650090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f100cf2e0 msgr2=0x7f4f1015e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f15999640 1 --2- 172.21.15.189:0/859650090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f100cf2e0 0x7f4f1015e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f15999640 1 -- 172.21.15.189:0/859650090 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f101fc910 con 0x7f4f100cfe00 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.003+0000 7f4f15198640 1 --2- 172.21.15.189:0/859650090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f100cae70 0x7f4f101f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.007+0000 7f4f15999640 1 --2- 172.21.15.189:0/859650090 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f100cfe00 0x7f4f1015e6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009bd0 tx=0x7f4f0c003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.007+0000 7f4f05ffb640 1 -- 172.21.15.189:0/859650090 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c0b2040 con 0x7f4f100cfe00 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.007+0000 7f4f173fc640 1 -- 172.21.15.189:0/859650090 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f101fcbf0 con 0x7f4f100cfe00 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.007+0000 7f4f05ffb640 1 -- 172.21.15.189:0/859650090 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f0c004200 con 0x7f4f100cfe00 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.007+0000 7f4f05ffb640 1 -- 172.21.15.189:0/859650090 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c004630 con 0x7f4f100cfe00 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.007+0000 7f4f173fc640 1 -- 172.21.15.189:0/859650090 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f101fd1b0 con 0x7f4f100cfe00 2024-09-17T02:40:30.005 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.007+0000 7f4f05ffb640 1 -- 172.21.15.189:0/859650090 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4f0c0ad440 con 0x7f4f100cfe00 2024-09-17T02:40:30.006 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.007+0000 7f4f05ffb640 1 -- 172.21.15.189:0/859650090 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f4f0c0bf910 con 0x7f4f100cfe00 2024-09-17T02:40:30.006 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.007+0000 7f4f173fc640 1 -- 172.21.15.189:0/859650090 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f100cc900 con 0x7f4f100cfe00 2024-09-17T02:40:30.008 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.011+0000 7f4f05ffb640 1 -- 172.21.15.189:0/859650090 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f4f0c016990 con 0x7f4f100cfe00 2024-09-17T02:40:30.074 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.075+0000 7f4f173fc640 1 -- 172.21.15.189:0/859650090 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a25e80d7-ac62-4a4c-a40f-95d4b43d985b", "id": 4} v 0) -- 0x7f4f100b61d0 con 0x7f4f100cfe00 2024-09-17T02:40:30.080 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.083+0000 7f4f05ffb640 1 -- 172.21.15.189:0/859650090 <== mon.2 v2:172.21.15.189:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a25e80d7-ac62-4a4c-a40f-95d4b43d985b", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f4f0c0bfb60 con 0x7f4f100cfe00 2024-09-17T02:40:30.081 INFO:teuthology.orchestra.run.smithi189.stdout:4 2024-09-17T02:40:30.082 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.083+0000 7f4f173fc640 1 -- 172.21.15.189:0/859650090 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f100cfe00 msgr2=0x7f4f1015e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.082 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.083+0000 7f4f173fc640 1 --2- 172.21.15.189:0/859650090 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f100cfe00 0x7f4f1015e6c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009bd0 tx=0x7f4f0c003f80 comp rx=0 tx=0).stop 2024-09-17T02:40:30.082 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.083+0000 7f4f173fc640 1 -- 172.21.15.189:0/859650090 shutdown_connections 2024-09-17T02:40:30.083 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.083+0000 7f4f173fc640 1 --2- 172.21.15.189:0/859650090 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f100cfe00 0x7f4f1015e6c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.083 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.083+0000 7f4f173fc640 1 --2- 172.21.15.189:0/859650090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f100cf2e0 0x7f4f1015e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.083 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.083+0000 7f4f173fc640 1 --2- 172.21.15.189:0/859650090 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f100cae70 0x7f4f101f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.083 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.083+0000 7f4f173fc640 1 -- 172.21.15.189:0/859650090 >> 172.21.15.189:0/859650090 conn(0x7f4f100cd6a0 msgr2=0x7f4f100b4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:30.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.083+0000 7f4f173fc640 1 -- 172.21.15.189:0/859650090 shutdown_connections 2024-09-17T02:40:30.084 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.083+0000 7f4f173fc640 1 -- 172.21.15.189:0/859650090 wait complete. 2024-09-17T02:40:30.098 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 1246ef1d-3575-4bbd-9cad-c79513551797 5 2024-09-17T02:40:30.247 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 -- 172.21.15.189:0/4136118949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb415a470 msgr2=0x7fadb415a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 --2- 172.21.15.189:0/4136118949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb415a470 0x7fadb415a880 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fadb40734a0 tx=0x7fada401cb50 comp rx=0 tx=0).stop 2024-09-17T02:40:30.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 -- 172.21.15.189:0/4136118949 shutdown_connections 2024-09-17T02:40:30.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 --2- 172.21.15.189:0/4136118949 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb415a470 0x7fadb415a880 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 --2- 172.21.15.189:0/4136118949 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadb40cc3b0 0x7fadb40cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 --2- 172.21.15.189:0/4136118949 >> v2:172.21.15.131:3300/0 conn(0x7fadb415f010 0x7fadb406e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 -- 172.21.15.189:0/4136118949 >> 172.21.15.189:0/4136118949 conn(0x7fadb40cd6a0 msgr2=0x7fadb40cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:30.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 -- 172.21.15.189:0/4136118949 shutdown_connections 2024-09-17T02:40:30.248 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 -- 172.21.15.189:0/4136118949 wait complete. 2024-09-17T02:40:30.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 Processor -- start 2024-09-17T02:40:30.249 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 -- start start 2024-09-17T02:40:30.251 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadb40cc3b0 0x7fadb4157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadb415a470 0x7fadb41581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb415f010 0x7fadb4158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fadb40caff0 con 0x7fadb40cc3b0 2024-09-17T02:40:30.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fadb40cae70 con 0x7fadb415f010 2024-09-17T02:40:30.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadbaa6b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fadb40cb170 con 0x7fadb415a470 2024-09-17T02:40:30.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadb3fff640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadb415a470 0x7fadb41581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadb9008640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb415f010 0x7fadb4158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadb3fff640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadb415a470 0x7fadb41581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.189:54612/0 (socket says 172.21.15.189:54612) 2024-09-17T02:40:30.252 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.251+0000 7fadb3fff640 1 -- 172.21.15.189:0/110132908 learned_addr learned my addr 172.21.15.189:0/110132908 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb8807640 1 --2- 172.21.15.189:0/110132908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadb40cc3b0 0x7fadb4157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb9008640 1 -- 172.21.15.189:0/110132908 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadb415a470 msgr2=0x7fadb41581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb9008640 1 --2- 172.21.15.189:0/110132908 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadb415a470 0x7fadb41581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb9008640 1 -- 172.21.15.189:0/110132908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadb40cc3b0 msgr2=0x7fadb4157c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb9008640 1 --2- 172.21.15.189:0/110132908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadb40cc3b0 0x7fadb4157c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb9008640 1 -- 172.21.15.189:0/110132908 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb40b5ff0 con 0x7fadb415f010 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb8807640 1 --2- 172.21.15.189:0/110132908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadb40cc3b0 0x7fadb4157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb3fff640 1 --2- 172.21.15.189:0/110132908 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadb415a470 0x7fadb41581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb9008640 1 --2- 172.21.15.189:0/110132908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb415f010 0x7fadb4158700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fada40ac000 tx=0x7fada4009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb17fa640 1 -- 172.21.15.189:0/110132908 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada40bb070 con 0x7fadb415f010 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb17fa640 1 -- 172.21.15.189:0/110132908 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fada4004b40 con 0x7fadb415f010 2024-09-17T02:40:30.253 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadbaa6b640 1 -- 172.21.15.189:0/110132908 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb40b62d0 con 0x7fadb415f010 2024-09-17T02:40:30.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb17fa640 1 -- 172.21.15.189:0/110132908 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada40b2040 con 0x7fadb415f010 2024-09-17T02:40:30.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadbaa6b640 1 -- 172.21.15.189:0/110132908 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb4158d00 con 0x7fadb415f010 2024-09-17T02:40:30.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb17fa640 1 -- 172.21.15.189:0/110132908 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fada40be070 con 0x7fadb415f010 2024-09-17T02:40:30.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb17fa640 1 -- 172.21.15.189:0/110132908 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fada4004790 con 0x7fadb415f010 2024-09-17T02:40:30.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadbaa6b640 1 -- 172.21.15.189:0/110132908 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb415a880 con 0x7fadb415f010 2024-09-17T02:40:30.254 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.255+0000 7fadb17fa640 1 -- 172.21.15.189:0/110132908 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fada40d8860 con 0x7fadb415f010 2024-09-17T02:40:30.327 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.331+0000 7fadbaa6b640 1 -- 172.21.15.189:0/110132908 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1246ef1d-3575-4bbd-9cad-c79513551797", "id": 5} v 0) -- 0x7fadb40b5dd0 con 0x7fadb415f010 2024-09-17T02:40:30.332 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.335+0000 7fadb17fa640 1 -- 172.21.15.189:0/110132908 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1246ef1d-3575-4bbd-9cad-c79513551797", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fada40c6030 con 0x7fadb415f010 2024-09-17T02:40:30.332 INFO:teuthology.orchestra.run.smithi189.stdout:5 2024-09-17T02:40:30.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.335+0000 7fadbaa6b640 1 -- 172.21.15.189:0/110132908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb415f010 msgr2=0x7fadb4158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.335+0000 7fadbaa6b640 1 --2- 172.21.15.189:0/110132908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb415f010 0x7fadb4158700 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fada40ac000 tx=0x7fada4009290 comp rx=0 tx=0).stop 2024-09-17T02:40:30.333 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.335+0000 7fadbaa6b640 1 -- 172.21.15.189:0/110132908 shutdown_connections 2024-09-17T02:40:30.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.335+0000 7fadbaa6b640 1 --2- 172.21.15.189:0/110132908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadb415f010 0x7fadb4158700 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.335+0000 7fadbaa6b640 1 --2- 172.21.15.189:0/110132908 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadb415a470 0x7fadb41581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.335+0000 7fadbaa6b640 1 --2- 172.21.15.189:0/110132908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadb40cc3b0 0x7fadb4157c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.335+0000 7fadbaa6b640 1 -- 172.21.15.189:0/110132908 >> 172.21.15.189:0/110132908 conn(0x7fadb40cd6a0 msgr2=0x7fadb40cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:30.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.335+0000 7fadbaa6b640 1 -- 172.21.15.189:0/110132908 shutdown_connections 2024-09-17T02:40:30.334 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.335+0000 7fadbaa6b640 1 -- 172.21.15.189:0/110132908 wait complete. 2024-09-17T02:40:30.351 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new f660a25e-255b-49d0-872a-d35c13154d63 6 2024-09-17T02:40:30.497 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.499+0000 7f182747b640 1 -- 172.21.15.189:0/3121893693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18200cae70 msgr2=0x7f18200714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.498 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.499+0000 7f182747b640 1 --2- 172.21.15.189:0/3121893693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18200cae70 0x7f18200714c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1820074900 tx=0x7f181001c7d0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.498 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.499+0000 7f182747b640 1 -- 172.21.15.189:0/3121893693 shutdown_connections 2024-09-17T02:40:30.498 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.499+0000 7f182747b640 1 --2- 172.21.15.189:0/3121893693 >> v2:172.21.15.131:3300/0 conn(0x7f18200cb280 0x7f1820071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.498 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.499+0000 7f182747b640 1 --2- 172.21.15.189:0/3121893693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18200cae70 0x7f18200714c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.498 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.499+0000 7f182747b640 1 -- 172.21.15.189:0/3121893693 >> 172.21.15.189:0/3121893693 conn(0x7f18200cd6a0 msgr2=0x7f18200cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:30.498 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.499+0000 7f182747b640 1 -- 172.21.15.189:0/3121893693 shutdown_connections 2024-09-17T02:40:30.498 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.499+0000 7f182747b640 1 -- 172.21.15.189:0/3121893693 wait complete. 2024-09-17T02:40:30.499 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f182747b640 1 Processor -- start 2024-09-17T02:40:30.499 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f182747b640 1 -- start start 2024-09-17T02:40:30.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f182747b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18200cae70 0x7f1820161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f182747b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18200cb280 0x7f1820151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f182747b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1820162270 0x7f182015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f182747b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f182016c6a0 con 0x7f18200cb280 2024-09-17T02:40:30.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f182747b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f182016c520 con 0x7f18200cae70 2024-09-17T02:40:30.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f182747b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f182016c820 con 0x7f1820162270 2024-09-17T02:40:30.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1825217640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18200cae70 0x7f1820161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1825a18640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1820162270 0x7f182015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1825a18640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1820162270 0x7f182015d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.189:54620/0 (socket says 172.21.15.189:54620) 2024-09-17T02:40:30.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1825a18640 1 -- 172.21.15.189:0/2840210703 learned_addr learned my addr 172.21.15.189:0/2840210703 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-17T02:40:30.502 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1824a16640 1 --2- 172.21.15.189:0/2840210703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18200cb280 0x7f1820151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1825a18640 1 -- 172.21.15.189:0/2840210703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18200cae70 msgr2=0x7f1820161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1825a18640 1 --2- 172.21.15.189:0/2840210703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18200cae70 0x7f1820161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1825a18640 1 -- 172.21.15.189:0/2840210703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18200cb280 msgr2=0x7f1820151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1825a18640 1 --2- 172.21.15.189:0/2840210703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18200cb280 0x7f1820151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1825a18640 1 -- 172.21.15.189:0/2840210703 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1820071400 con 0x7f1820162270 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1825217640 1 --2- 172.21.15.189:0/2840210703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18200cae70 0x7f1820161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1825a18640 1 --2- 172.21.15.189:0/2840210703 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1820162270 0x7f182015d090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f181800c3f0 tx=0x7f181800c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1815ffb640 1 -- 172.21.15.189:0/2840210703 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818017070 con 0x7f1820162270 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1815ffb640 1 -- 172.21.15.189:0/2840210703 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1818003600 con 0x7f1820162270 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f182747b640 1 -- 172.21.15.189:0/2840210703 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18201fc830 con 0x7f1820162270 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1815ffb640 1 -- 172.21.15.189:0/2840210703 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818003a50 con 0x7f1820162270 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f182747b640 1 -- 172.21.15.189:0/2840210703 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f18201fcad0 con 0x7f1820162270 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1815ffb640 1 -- 172.21.15.189:0/2840210703 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f181801a070 con 0x7f1820162270 2024-09-17T02:40:30.503 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f1815ffb640 1 -- 172.21.15.189:0/2840210703 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f18180129f0 con 0x7f1820162270 2024-09-17T02:40:30.504 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.503+0000 7f182747b640 1 -- 172.21.15.189:0/2840210703 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1820162680 con 0x7f1820162270 2024-09-17T02:40:30.504 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.507+0000 7f1815ffb640 1 -- 172.21.15.189:0/2840210703 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1818021020 con 0x7f1820162270 2024-09-17T02:40:30.570 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.571+0000 7f182747b640 1 -- 172.21.15.189:0/2840210703 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f660a25e-255b-49d0-872a-d35c13154d63", "id": 6} v 0) -- 0x7f18200ceee0 con 0x7f1820162270 2024-09-17T02:40:30.576 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.579+0000 7f1815ffb640 1 -- 172.21.15.189:0/2840210703 <== mon.2 v2:172.21.15.189:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f660a25e-255b-49d0-872a-d35c13154d63", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f1818003d20 con 0x7f1820162270 2024-09-17T02:40:30.576 INFO:teuthology.orchestra.run.smithi189.stdout:6 2024-09-17T02:40:30.577 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.579+0000 7f182747b640 1 -- 172.21.15.189:0/2840210703 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1820162270 msgr2=0x7f182015d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.578 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.579+0000 7f182747b640 1 --2- 172.21.15.189:0/2840210703 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1820162270 0x7f182015d090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f181800c3f0 tx=0x7f181800c7f0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.578 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.579+0000 7f182747b640 1 -- 172.21.15.189:0/2840210703 shutdown_connections 2024-09-17T02:40:30.578 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.579+0000 7f182747b640 1 --2- 172.21.15.189:0/2840210703 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1820162270 0x7f182015d090 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.578 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.579+0000 7f182747b640 1 --2- 172.21.15.189:0/2840210703 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18200cb280 0x7f1820151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.578 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.579+0000 7f182747b640 1 --2- 172.21.15.189:0/2840210703 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18200cae70 0x7f1820161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.578 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.579+0000 7f182747b640 1 -- 172.21.15.189:0/2840210703 >> 172.21.15.189:0/2840210703 conn(0x7f18200cd6a0 msgr2=0x7f182006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:30.578 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.579+0000 7f182747b640 1 -- 172.21.15.189:0/2840210703 shutdown_connections 2024-09-17T02:40:30.578 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.579+0000 7f182747b640 1 -- 172.21.15.189:0/2840210703 wait complete. 2024-09-17T02:40:30.594 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --cluster ceph osd new 92b097eb-7f27-4093-b098-45aebfa15244 7 2024-09-17T02:40:30.737 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.739+0000 7f05bb63e640 1 -- 172.21.15.189:0/590356619 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05b415b2e0 msgr2=0x7f05b415ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.737 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.739+0000 7f05bb63e640 1 --2- 172.21.15.189:0/590356619 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05b415b2e0 0x7f05b415ede0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009bf0 tx=0x7f05ac01cb50 comp rx=0 tx=0).stop 2024-09-17T02:40:30.738 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.739+0000 7f05bb63e640 1 -- 172.21.15.189:0/590356619 shutdown_connections 2024-09-17T02:40:30.738 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.739+0000 7f05bb63e640 1 --2- 172.21.15.189:0/590356619 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05b415b2e0 0x7f05b415ede0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.738 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.739+0000 7f05bb63e640 1 -- 172.21.15.189:0/590356619 >> 172.21.15.189:0/590356619 conn(0x7f05b40d0740 msgr2=0x7f05b40d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:30.738 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.739+0000 7f05bb63e640 1 -- 172.21.15.189:0/590356619 shutdown_connections 2024-09-17T02:40:30.738 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.739+0000 7f05bb63e640 1 -- 172.21.15.189:0/590356619 wait complete. 2024-09-17T02:40:30.739 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05bb63e640 1 Processor -- start 2024-09-17T02:40:30.739 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05bb63e640 1 -- start start 2024-09-17T02:40:30.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05bb63e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b40704a0 0x7f05b41faa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05bb63e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b40d01e0 0x7f05b414ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05bb63e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05b415b2e0 0x7f05b406ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:30.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05bb63e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05b416a1d0 con 0x7f05b40704a0 2024-09-17T02:40:30.742 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05bb63e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05b416a050 con 0x7f05b40d01e0 2024-09-17T02:40:30.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05bb63e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f05b416a350 con 0x7f05b415b2e0 2024-09-17T02:40:30.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b9bdb640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05b415b2e0 0x7f05b406ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b9bdb640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05b415b2e0 0x7f05b406ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.189:54638/0 (socket says 172.21.15.189:54638) 2024-09-17T02:40:30.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b8bd9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b40d01e0 0x7f05b414ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b9bdb640 1 -- 172.21.15.189:0/3336889279 learned_addr learned my addr 172.21.15.189:0/3336889279 (peer_addr_for_me v2:172.21.15.189:0/0) 2024-09-17T02:40:30.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b93da640 1 --2- 172.21.15.189:0/3336889279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b40704a0 0x7f05b41faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:30.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b9bdb640 1 -- 172.21.15.189:0/3336889279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b40d01e0 msgr2=0x7f05b414ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b9bdb640 1 --2- 172.21.15.189:0/3336889279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b40d01e0 0x7f05b414ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b9bdb640 1 -- 172.21.15.189:0/3336889279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b40704a0 msgr2=0x7f05b41faa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.743 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b9bdb640 1 --2- 172.21.15.189:0/3336889279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b40704a0 0x7f05b41faa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b9bdb640 1 -- 172.21.15.189:0/3336889279 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b415a980 con 0x7f05b415b2e0 2024-09-17T02:40:30.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b8bd9640 1 --2- 172.21.15.189:0/3336889279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b40d01e0 0x7f05b414ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T02:40:30.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05b9bdb640 1 --2- 172.21.15.189:0/3336889279 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05b415b2e0 0x7f05b406ce90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f05ac019760 tx=0x7f05ac009200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:30.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05a9ffb640 1 -- 172.21.15.189:0/3336889279 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0bb070 con 0x7f05b415b2e0 2024-09-17T02:40:30.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05a9ffb640 1 -- 172.21.15.189:0/3336889279 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05ac004d40 con 0x7f05b415b2e0 2024-09-17T02:40:30.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05a9ffb640 1 -- 172.21.15.189:0/3336889279 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0b2040 con 0x7f05b415b2e0 2024-09-17T02:40:30.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05bb63e640 1 -- 172.21.15.189:0/3336889279 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b4150440 con 0x7f05b415b2e0 2024-09-17T02:40:30.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05bb63e640 1 -- 172.21.15.189:0/3336889279 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b415d7c0 con 0x7f05b415b2e0 2024-09-17T02:40:30.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05a9ffb640 1 -- 172.21.15.189:0/3336889279 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f05ac0be070 con 0x7f05b415b2e0 2024-09-17T02:40:30.744 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.743+0000 7f05a9ffb640 1 -- 172.21.15.189:0/3336889279 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f05ac0056e0 con 0x7f05b415b2e0 2024-09-17T02:40:30.745 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.747+0000 7f05bb63e640 1 -- 172.21.15.189:0/3336889279 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b415ede0 con 0x7f05b415b2e0 2024-09-17T02:40:30.745 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.747+0000 7f05a9ffb640 1 -- 172.21.15.189:0/3336889279 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f05ac0d8860 con 0x7f05b415b2e0 2024-09-17T02:40:30.815 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.819+0000 7f05bb63e640 1 -- 172.21.15.189:0/3336889279 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "osd new", "uuid": "92b097eb-7f27-4093-b098-45aebfa15244", "id": 7} v 0) -- 0x7f05b4073fd0 con 0x7f05b415b2e0 2024-09-17T02:40:30.821 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.823+0000 7f05a9ffb640 1 -- 172.21.15.189:0/3336889279 <== mon.2 v2:172.21.15.189:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "92b097eb-7f27-4093-b098-45aebfa15244", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f05ac0c6030 con 0x7f05b415b2e0 2024-09-17T02:40:30.821 INFO:teuthology.orchestra.run.smithi189.stdout:7 2024-09-17T02:40:30.823 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.823+0000 7f05bb63e640 1 -- 172.21.15.189:0/3336889279 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05b415b2e0 msgr2=0x7f05b406ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:30.823 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.823+0000 7f05bb63e640 1 --2- 172.21.15.189:0/3336889279 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05b415b2e0 0x7f05b406ce90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f05ac019760 tx=0x7f05ac009200 comp rx=0 tx=0).stop 2024-09-17T02:40:30.823 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.823+0000 7f05bb63e640 1 -- 172.21.15.189:0/3336889279 shutdown_connections 2024-09-17T02:40:30.823 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.823+0000 7f05bb63e640 1 --2- 172.21.15.189:0/3336889279 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05b415b2e0 0x7f05b406ce90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.823 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.823+0000 7f05bb63e640 1 --2- 172.21.15.189:0/3336889279 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05b40d01e0 0x7f05b414ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.823 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.823+0000 7f05bb63e640 1 --2- 172.21.15.189:0/3336889279 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05b40704a0 0x7f05b41faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:30.823 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.823+0000 7f05bb63e640 1 -- 172.21.15.189:0/3336889279 >> 172.21.15.189:0/3336889279 conn(0x7f05b40d0740 msgr2=0x7f05b40cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:30.823 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.827+0000 7f05bb63e640 1 -- 172.21.15.189:0/3336889279 shutdown_connections 2024-09-17T02:40:30.823 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:40:30.827+0000 7f05bb63e640 1 -- 172.21.15.189:0/3336889279 wait complete. 2024-09-17T02:40:30.840 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T02:40:30.840 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T02:40:30.843 INFO:tasks.ceph.osd.0:Started 2024-09-17T02:40:30.843 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T02:40:30.843 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T02:40:30.846 INFO:tasks.ceph.osd.1:Started 2024-09-17T02:40:30.846 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T02:40:30.846 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T02:40:30.850 INFO:tasks.ceph.osd.2:Started 2024-09-17T02:40:30.850 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T02:40:30.850 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T02:40:30.852 INFO:tasks.ceph.osd.3:Started 2024-09-17T02:40:30.852 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T02:40:30.852 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T02:40:30.854 INFO:tasks.ceph.osd.4:Started 2024-09-17T02:40:30.854 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T02:40:30.854 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T02:40:30.856 INFO:tasks.ceph.osd.5:Started 2024-09-17T02:40:30.856 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T02:40:30.856 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T02:40:30.858 INFO:tasks.ceph.osd.6:Started 2024-09-17T02:40:30.858 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T02:40:30.858 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T02:40:30.860 INFO:tasks.ceph.osd.7:Started 2024-09-17T02:40:30.861 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:40:31.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.022+0000 7fceaf673640 1 -- 172.21.15.131:0/3203199063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea806e4e0 msgr2=0x7fcea81623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:31.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.022+0000 7fceaf673640 1 --2- 172.21.15.131:0/3203199063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea806e4e0 0x7fcea81623a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcea80b4d60 tx=0x7fcea001ca30 comp rx=0 tx=0).stop 2024-09-17T02:40:31.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.022+0000 7fceaf673640 1 -- 172.21.15.131:0/3203199063 shutdown_connections 2024-09-17T02:40:31.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.022+0000 7fceaf673640 1 --2- 172.21.15.131:0/3203199063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea806e4e0 0x7fcea81623a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:31.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.022+0000 7fceaf673640 1 --2- 172.21.15.131:0/3203199063 >> v2:172.21.15.189:3300/0 conn(0x7fcea80b4630 0x7fcea806dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:31.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.022+0000 7fceaf673640 1 --2- 172.21.15.131:0/3203199063 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcea80b2a60 0x7fcea80b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:31.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.022+0000 7fceaf673640 1 -- 172.21.15.131:0/3203199063 >> 172.21.15.131:0/3203199063 conn(0x7fcea80cf150 msgr2=0x7fcea80cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:31.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.022+0000 7fceaf673640 1 -- 172.21.15.131:0/3203199063 shutdown_connections 2024-09-17T02:40:31.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.022+0000 7fceaf673640 1 -- 172.21.15.131:0/3203199063 wait complete. 2024-09-17T02:40:31.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceaf673640 1 Processor -- start 2024-09-17T02:40:31.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceaf673640 1 -- start start 2024-09-17T02:40:31.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceaf673640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcea806e4e0 0x7fcea80713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:31.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceaf673640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80b2a60 0x7fcea8154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:31.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceaf673640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcea80b4630 0x7fcea80cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:31.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceaf673640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcea815edf0 con 0x7fcea80b2a60 2024-09-17T02:40:31.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceaf673640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcea815ec70 con 0x7fcea806e4e0 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceaf673640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcea815ef70 con 0x7fcea80b4630 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceacc0e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80b2a60 0x7fcea8154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceacc0e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80b2a60 0x7fcea8154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:49996/0 (socket says 172.21.15.131:49996) 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceacc0e640 1 -- 172.21.15.131:0/1640046751 learned_addr learned my addr 172.21.15.131:0/1640046751 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceadc10640 1 --2- 172.21.15.131:0/1640046751 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcea80b4630 0x7fcea80cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fcead40f640 1 --2- 172.21.15.131:0/1640046751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcea806e4e0 0x7fcea80713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceacc0e640 1 -- 172.21.15.131:0/1640046751 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcea80b4630 msgr2=0x7fcea80cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceacc0e640 1 --2- 172.21.15.131:0/1640046751 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcea80b4630 0x7fcea80cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceacc0e640 1 -- 172.21.15.131:0/1640046751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcea806e4e0 msgr2=0x7fcea80713c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceacc0e640 1 --2- 172.21.15.131:0/1640046751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcea806e4e0 0x7fcea80713c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceacc0e640 1 -- 172.21.15.131:0/1640046751 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea80b7000 con 0x7fcea80b2a60 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceacc0e640 1 --2- 172.21.15.131:0/1640046751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80b2a60 0x7fcea8154500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcea40058d0 tx=0x7fcea400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fce9dffb640 1 -- 172.21.15.131:0/1640046751 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4017070 con 0x7fcea80b2a60 2024-09-17T02:40:31.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fce9dffb640 1 -- 172.21.15.131:0/1640046751 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcea4009d20 con 0x7fcea80b2a60 2024-09-17T02:40:31.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fce9dffb640 1 -- 172.21.15.131:0/1640046751 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea40106c0 con 0x7fcea80b2a60 2024-09-17T02:40:31.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceaf673640 1 -- 172.21.15.131:0/1640046751 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea80b72e0 con 0x7fcea80b2a60 2024-09-17T02:40:31.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fceaf673640 1 -- 172.21.15.131:0/1640046751 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea80b78a0 con 0x7fcea80b2a60 2024-09-17T02:40:31.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fce9dffb640 1 -- 172.21.15.131:0/1640046751 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcea401a070 con 0x7fcea80b2a60 2024-09-17T02:40:31.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.026+0000 7fce9dffb640 1 -- 172.21.15.131:0/1640046751 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fcea400f3c0 con 0x7fcea80b2a60 2024-09-17T02:40:31.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.030+0000 7fceaf673640 1 -- 172.21.15.131:0/1640046751 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea815f4d0 con 0x7fcea80b2a60 2024-09-17T02:40:31.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.030+0000 7fce9dffb640 1 -- 172.21.15.131:0/1640046751 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcea4010860 con 0x7fcea80b2a60 2024-09-17T02:40:31.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.090+0000 7fceaf673640 1 -- 172.21.15.131:0/1640046751 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcea806a170 con 0x7fcea80b2a60 2024-09-17T02:40:31.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.090+0000 7fce9dffb640 1 -- 172.21.15.131:0/1640046751 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fcea4010a40 con 0x7fcea80b2a60 2024-09-17T02:40:31.089 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:40:31.089 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":10,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:30.823064+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:40:31.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.094+0000 7fceaf673640 1 -- 172.21.15.131:0/1640046751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80b2a60 msgr2=0x7fcea8154500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:31.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.094+0000 7fceaf673640 1 --2- 172.21.15.131:0/1640046751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80b2a60 0x7fcea8154500 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcea40058d0 tx=0x7fcea400dde0 comp rx=0 tx=0).stop 2024-09-17T02:40:31.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.094+0000 7fceaf673640 1 -- 172.21.15.131:0/1640046751 shutdown_connections 2024-09-17T02:40:31.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.094+0000 7fceaf673640 1 --2- 172.21.15.131:0/1640046751 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcea80b4630 0x7fcea80cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:31.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.094+0000 7fceaf673640 1 --2- 172.21.15.131:0/1640046751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcea80b2a60 0x7fcea8154500 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:31.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.094+0000 7fceaf673640 1 --2- 172.21.15.131:0/1640046751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcea806e4e0 0x7fcea80713c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:31.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.094+0000 7fceaf673640 1 -- 172.21.15.131:0/1640046751 >> 172.21.15.131:0/1640046751 conn(0x7fcea80cf150 msgr2=0x7fcea8155350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:31.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.094+0000 7fceaf673640 1 -- 172.21.15.131:0/1640046751 shutdown_connections 2024-09-17T02:40:31.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:31.094+0000 7fceaf673640 1 -- 172.21.15.131:0/1640046751 wait complete. 2024-09-17T02:40:31.109 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T02:40:31.110 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T02:40:31.915 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T02:40:32.037 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-17T02:40:32.038+0000 7f38a4e957c0 -1 Falling back to public interface 2024-09-17T02:40:32.059 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-09-17T02:40:32.061+0000 7f72fb4c97c0 -1 Falling back to public interface 2024-09-17T02:40:32.070 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.073+0000 7fc314b4f640 1 -- 172.21.15.131:0/2821136182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2f8004810 msgr2=0x7fc2f8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:32.112 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.073+0000 7fc314b4f640 1 --2- 172.21.15.131:0/2821136182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2f8004810 0x7fc2f8004c20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc300006fd0 tx=0x7fc30001f6a0 comp rx=0 tx=0).stop 2024-09-17T02:40:32.113 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.073+0000 7fc314b4f640 1 -- 172.21.15.131:0/2821136182 shutdown_connections 2024-09-17T02:40:32.113 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.073+0000 7fc314b4f640 1 --2- 172.21.15.131:0/2821136182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2f8004810 0x7fc2f8004c20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:32.113 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.073+0000 7fc314b4f640 1 -- 172.21.15.131:0/2821136182 >> 172.21.15.131:0/2821136182 conn(0x7fc3100d1b10 msgr2=0x7fc3100d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:32.113 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.073+0000 7fc314b4f640 1 -- 172.21.15.131:0/2821136182 shutdown_connections 2024-09-17T02:40:32.113 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.073+0000 7fc314b4f640 1 -- 172.21.15.131:0/2821136182 wait complete. 2024-09-17T02:40:32.116 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.073+0000 7fc314b4f640 1 Processor -- start 2024-09-17T02:40:32.116 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.073+0000 7fc314b4f640 1 -- start start 2024-09-17T02:40:32.116 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc314b4f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2f8004810 0x7fc3100ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:32.116 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc314b4f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc31015b340 0x7fc31015b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:32.116 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc314b4f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3101564b0 0x7fc3101568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:32.116 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc314b4f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc31015fbe0 con 0x7fc2f8004810 2024-09-17T02:40:32.116 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc314b4f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3100cfc30 con 0x7fc3101564b0 2024-09-17T02:40:32.116 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc314b4f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc31015fd60 con 0x7fc31015b340 2024-09-17T02:40:32.116 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc30e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2f8004810 0x7fc3100ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:32.116 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc30e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2f8004810 0x7fc3100ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51254/0 (socket says 172.21.15.131:51254) 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc30e59c640 1 -- 172.21.15.131:0/3693633795 learned_addr learned my addr 172.21.15.131:0/3693633795 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc30dd9b640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc31015b340 0x7fc31015b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc30ed9d640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3101564b0 0x7fc3101568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc30e59c640 1 -- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc31015b340 msgr2=0x7fc31015b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc30e59c640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc31015b340 0x7fc31015b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc30e59c640 1 -- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3101564b0 msgr2=0x7fc3101568c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc30e59c640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3101564b0 0x7fc3101568c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc30e59c640 1 -- 172.21.15.131:0/3693633795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc310156e00 con 0x7fc2f8004810 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc30e59c640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2f8004810 0x7fc3100ce050 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc304011ab0 tx=0x7fc304011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc2feffd640 1 -- 172.21.15.131:0/3693633795 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30401e070 con 0x7fc2f8004810 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc2feffd640 1 -- 172.21.15.131:0/3693633795 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc30400fc70 con 0x7fc2f8004810 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc314b4f640 1 -- 172.21.15.131:0/3693633795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc31015a180 con 0x7fc2f8004810 2024-09-17T02:40:32.117 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc2feffd640 1 -- 172.21.15.131:0/3693633795 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc304019410 con 0x7fc2f8004810 2024-09-17T02:40:32.118 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc314b4f640 1 -- 172.21.15.131:0/3693633795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc31015a740 con 0x7fc2f8004810 2024-09-17T02:40:32.118 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc2feffd640 1 -- 172.21.15.131:0/3693633795 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 2) ==== 48655+0+0 (secure 0 0 0) 0x7fc304021070 con 0x7fc2f8004810 2024-09-17T02:40:32.118 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc314b4f640 1 -- 172.21.15.131:0/3693633795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc31015f450 con 0x7fc2f8004810 2024-09-17T02:40:32.118 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.077+0000 7fc2feffd640 1 -- 172.21.15.131:0/3693633795 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc30404f650 con 0x7fc2f8004810 2024-09-17T02:40:32.118 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.081+0000 7fc2feffd640 1 -- 172.21.15.131:0/3693633795 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc310156e00 con 0x7fc2f8004810 2024-09-17T02:40:32.118 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.109+0000 7fc2feffd640 1 -- 172.21.15.131:0/3693633795 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mgrmap(e 3) ==== 97072+0+0 (secure 0 0 0) 0x7fc304019d10 con 0x7fc2f8004810 2024-09-17T02:40:32.118 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.109+0000 7fc2feffd640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc2f00736b0 0x7fc2f0075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:32.118 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.109+0000 7fc30dd9b640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc2f00736b0 0x7fc2f0075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:32.118 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.109+0000 7fc30dd9b640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc2f00736b0 0x7fc2f0075ad0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3000b0000 tx=0x7fc30001c4e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:32.124 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-17T02:40:32.126+0000 7f4534280640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T02:40:32.124 INFO:tasks.ceph.mgr.x.smithi189.stderr:2024-09-17T02:40:32.126+0000 7f4534280640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T02:40:32.144 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.145+0000 7fc314b4f640 1 -- 172.21.15.131:0/3693633795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3100693d0 con 0x7fc2f8004810 2024-09-17T02:40:32.145 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.145+0000 7fc2feffd640 1 -- 172.21.15.131:0/3693633795 <== mon.0 v2:172.21.15.131:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc304031ef0 con 0x7fc2f8004810 2024-09-17T02:40:32.145 INFO:teuthology.misc.health.smithi131.stdout: 2024-09-17T02:40:32.145 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":10,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:30.823064+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:40:32.146 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 -- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc2f00736b0 msgr2=0x7fc2f0075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:32.146 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc2f00736b0 0x7fc2f0075ad0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3000b0000 tx=0x7fc30001c4e0 comp rx=0 tx=0).stop 2024-09-17T02:40:32.147 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 -- 172.21.15.131:0/3693633795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2f8004810 msgr2=0x7fc3100ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:32.147 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2f8004810 0x7fc3100ce050 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc304011ab0 tx=0x7fc304011f70 comp rx=0 tx=0).stop 2024-09-17T02:40:32.147 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 -- 172.21.15.131:0/3693633795 shutdown_connections 2024-09-17T02:40:32.147 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc2f00736b0 0x7fc2f0075ad0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:32.147 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3101564b0 0x7fc3101568c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:32.147 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc31015b340 0x7fc31015b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:32.147 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 --2- 172.21.15.131:0/3693633795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2f8004810 0x7fc3100ce050 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:32.147 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 -- 172.21.15.131:0/3693633795 >> 172.21.15.131:0/3693633795 conn(0x7fc3100d1b10 msgr2=0x7fc310158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:32.147 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 -- 172.21.15.131:0/3693633795 shutdown_connections 2024-09-17T02:40:32.147 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:32.149+0000 7fc314b4f640 1 -- 172.21.15.131:0/3693633795 wait complete. 2024-09-17T02:40:32.164 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T02:40:32.267 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-17T02:40:32.269+0000 7fe6ab0fa7c0 -1 Falling back to public interface 2024-09-17T02:40:32.283 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-09-17T02:40:32.285+0000 7fa88dd727c0 -1 Falling back to public interface 2024-09-17T02:40:32.289 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-17T02:40:32.290+0000 7f0f0ba347c0 -1 Falling back to public interface 2024-09-17T02:40:32.576 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-17T02:40:32.577+0000 7f2bff5b27c0 -1 Falling back to public interface 2024-09-17T02:40:32.689 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-17T02:40:32.690+0000 7f4979c7a7c0 -1 Falling back to public interface 2024-09-17T02:40:32.752 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-17T02:40:32.754+0000 7f2fd33c47c0 -1 Falling back to public interface 2024-09-17T02:40:36.056 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-17T02:40:36.058+0000 7f38a4e957c0 -1 osd.6 0 log_to_monitors true 2024-09-17T02:40:36.146 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-09-17T02:40:36.149+0000 7f72fb4c97c0 -1 osd.2 0 log_to_monitors true 2024-09-17T02:40:36.258 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-17T02:40:36.261+0000 7fe6ab0fa7c0 -1 osd.0 0 log_to_monitors true 2024-09-17T02:40:36.267 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-17T02:40:36.270+0000 7f0f0ba347c0 -1 osd.7 0 log_to_monitors true 2024-09-17T02:40:36.570 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-09-17T02:40:36.573+0000 7fa88dd727c0 -1 osd.3 0 log_to_monitors true 2024-09-17T02:40:36.746 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-17T02:40:36.746+0000 7f2fd33c47c0 -1 osd.5 0 log_to_monitors true 2024-09-17T02:40:36.874 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-17T02:40:36.877+0000 7f2bff5b27c0 -1 osd.1 0 log_to_monitors true 2024-09-17T02:40:37.024 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-17T02:40:37.026+0000 7f4979c7a7c0 -1 osd.4 0 log_to_monitors true 2024-09-17T02:40:38.069 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-17T02:40:38.070+0000 7f38a1650640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T02:40:38.092 INFO:tasks.ceph.osd.6.smithi189.stderr:2024-09-17T02:40:38.094+0000 7f3898c38640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T02:40:38.971 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T02:40:39.157 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.157+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/2281845013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d940cd900 msgr2=0x7f8d940cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:39.157 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.157+0000 7f8d9bf0e640 1 --2- 172.21.15.131:0/2281845013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d940cd900 0x7f8d940cdd10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8d94072030 tx=0x7f8d8401c800 comp rx=0 tx=0).stop 2024-09-17T02:40:39.157 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.157+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/2281845013 shutdown_connections 2024-09-17T02:40:39.157 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.157+0000 7f8d9bf0e640 1 --2- 172.21.15.131:0/2281845013 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d940cd900 0x7f8d940cdd10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:39.157 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.157+0000 7f8d9bf0e640 1 --2- 172.21.15.131:0/2281845013 >> v2:172.21.15.189:3300/0 conn(0x7f8d940d0710 0x7f8d940d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:39.157 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.157+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/2281845013 >> 172.21.15.131:0/2281845013 conn(0x7f8d940d1130 msgr2=0x7f8d940d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:39.157 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.157+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/2281845013 shutdown_connections 2024-09-17T02:40:39.157 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.157+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/2281845013 wait complete. 2024-09-17T02:40:39.158 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9bf0e640 1 Processor -- start 2024-09-17T02:40:39.158 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9bf0e640 1 -- start start 2024-09-17T02:40:39.161 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9bf0e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8d94071210 0x7f8d9414d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:39.161 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9bf0e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d940cd900 0x7f8d9413ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:39.161 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9bf0e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d940d0710 0x7f8d9413bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:39.161 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9bf0e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d941698f0 con 0x7f8d940d0710 2024-09-17T02:40:39.161 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9bf0e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8d94169770 con 0x7f8d940cd900 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9bf0e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8d94169a70 con 0x7f8d94071210 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9a4ab640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d940d0710 0x7f8d9413bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9a4ab640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d940d0710 0x7f8d9413bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51296/0 (socket says 172.21.15.131:51296) 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9a4ab640 1 -- 172.21.15.131:0/4252401638 learned_addr learned my addr 172.21.15.131:0/4252401638 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d994a9640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d940cd900 0x7f8d9413ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d99caa640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8d94071210 0x7f8d9414d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9a4ab640 1 -- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8d94071210 msgr2=0x7f8d9414d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9a4ab640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8d94071210 0x7f8d9414d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9a4ab640 1 -- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d940cd900 msgr2=0x7f8d9413ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9a4ab640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d940cd900 0x7f8d9413ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9a4ab640 1 -- 172.21.15.131:0/4252401638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9414de20 con 0x7f8d940d0710 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d99caa640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8d94071210 0x7f8d9414d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9a4ab640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d940d0710 0x7f8d9413bfb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00cb40 tx=0x7f8d8c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:39.162 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d8a7fc640 1 -- 172.21.15.131:0/4252401638 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c016020 con 0x7f8d940d0710 2024-09-17T02:40:39.163 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d8a7fc640 1 -- 172.21.15.131:0/4252401638 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d8c00ab40 con 0x7f8d940d0710 2024-09-17T02:40:39.163 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/4252401638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d941fb4c0 con 0x7f8d940d0710 2024-09-17T02:40:39.163 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d8a7fc640 1 -- 172.21.15.131:0/4252401638 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c012420 con 0x7f8d940d0710 2024-09-17T02:40:39.163 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.161+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/4252401638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d94153de0 con 0x7f8d940d0710 2024-09-17T02:40:39.163 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.165+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/4252401638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d940cdd10 con 0x7f8d940d0710 2024-09-17T02:40:39.163 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.165+0000 7f8d8a7fc640 1 -- 172.21.15.131:0/4252401638 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 4) ==== 97178+0+0 (secure 0 0 0) 0x7f8d8c019020 con 0x7f8d940d0710 2024-09-17T02:40:39.164 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.165+0000 7f8d8a7fc640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d780731a0 0x7f8d780756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:39.164 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.165+0000 7f8d8a7fc640 1 -- 172.21.15.131:0/4252401638 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2511+0+0 (secure 0 0 0) 0x7f8d8c093650 con 0x7f8d940d0710 2024-09-17T02:40:39.164 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.165+0000 7f8d99caa640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d780731a0 0x7f8d780756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:39.164 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.165+0000 7f8d99caa640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d780731a0 0x7f8d780756b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000bbd0 tx=0x7f8d9000b5b0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:39.180 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.181+0000 7f8d8a7fc640 1 -- 172.21.15.131:0/4252401638 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8d8c020020 con 0x7f8d940d0710 2024-09-17T02:40:39.390 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.393+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/4252401638 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d9406f100 con 0x7f8d940d0710 2024-09-17T02:40:39.391 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.393+0000 7f8d8a7fc640 1 -- 172.21.15.131:0/4252401638 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7578 (secure 0 0 0) 0x7f8d9414de20 con 0x7f8d940d0710 2024-09-17T02:40:39.391 INFO:teuthology.misc.health.smithi131.stdout: 2024-09-17T02:40:39.391 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":14,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:39.197031+0000","last_up_change":"2024-09-17T02:40:38.125935+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:40:39.394 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.393+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d780731a0 msgr2=0x7f8d780756b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:39.394 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.393+0000 7f8d9bf0e640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d780731a0 0x7f8d780756b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8d9000bbd0 tx=0x7f8d9000b5b0 comp rx=0 tx=0).stop 2024-09-17T02:40:39.394 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.393+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/4252401638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d940d0710 msgr2=0x7f8d9413bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:39.394 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.393+0000 7f8d9bf0e640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d940d0710 0x7f8d9413bfb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00cb40 tx=0x7f8d8c00cc30 comp rx=0 tx=0).stop 2024-09-17T02:40:39.395 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.397+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/4252401638 shutdown_connections 2024-09-17T02:40:39.395 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.397+0000 7f8d9bf0e640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d780731a0 0x7f8d780756b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:39.395 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.397+0000 7f8d9bf0e640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d940d0710 0x7f8d9413bfb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:39.395 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.397+0000 7f8d9bf0e640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d940cd900 0x7f8d9413ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:39.395 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.397+0000 7f8d9bf0e640 1 --2- 172.21.15.131:0/4252401638 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8d94071210 0x7f8d9414d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:39.395 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.397+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/4252401638 >> 172.21.15.131:0/4252401638 conn(0x7f8d940d1130 msgr2=0x7f8d9406e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:39.395 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.397+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/4252401638 shutdown_connections 2024-09-17T02:40:39.395 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:39.397+0000 7f8d9bf0e640 1 -- 172.21.15.131:0/4252401638 wait complete. 2024-09-17T02:40:39.413 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-17T02:40:39.905 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-17T02:40:39.906+0000 7f0f079da640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T02:40:39.932 INFO:tasks.ceph.osd.7.smithi189.stderr:2024-09-17T02:40:39.934+0000 7f0eff7d7640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T02:40:39.933 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-17T02:40:39.934+0000 7f2fcfb7f640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T02:40:39.956 INFO:tasks.ceph.osd.5.smithi189.stderr:2024-09-17T02:40:39.958+0000 7f2fc7167640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T02:40:39.996 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-17T02:40:39.998+0000 7f4975c20640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T02:40:40.021 INFO:tasks.ceph.osd.4.smithi189.stderr:2024-09-17T02:40:40.022+0000 7f496da1d640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T02:40:40.356 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-09-17T02:40:40.357+0000 7fa88a52d640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T02:40:40.386 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-09-17T02:40:40.389+0000 7f72f7c84640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T02:40:40.390 INFO:tasks.ceph.osd.3.smithi131.stderr:2024-09-17T02:40:40.393+0000 7fa881b15640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T02:40:40.407 INFO:tasks.ceph.osd.2.smithi131.stderr:2024-09-17T02:40:40.409+0000 7f72ef26c640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T02:40:40.435 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-17T02:40:40.437+0000 7fe6a70a0640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T02:40:40.438 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-17T02:40:40.441+0000 7f2bfbd6d640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T02:40:40.471 INFO:tasks.ceph.osd.0.smithi131.stderr:2024-09-17T02:40:40.473+0000 7fe69ee9d640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T02:40:40.471 INFO:tasks.ceph.osd.1.smithi131.stderr:2024-09-17T02:40:40.473+0000 7f2bf3355640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T02:40:43.211 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:40:43.214 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:40:46.221 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T02:40:46.377 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.376+0000 7fc329138640 1 -- 172.21.15.131:0/1920338767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc324162d60 msgr2=0x7fc3240b70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:46.377 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.376+0000 7fc329138640 1 --2- 172.21.15.131:0/1920338767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc324162d60 0x7fc3240b70b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc3240748b0 tx=0x7fc31c01ccf0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.378 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.376+0000 7fc329138640 1 -- 172.21.15.131:0/1920338767 shutdown_connections 2024-09-17T02:40:46.378 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.376+0000 7fc329138640 1 --2- 172.21.15.131:0/1920338767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc324162d60 0x7fc3240b70b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.378 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.376+0000 7fc329138640 1 --2- 172.21.15.131:0/1920338767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc3240b44f0 0x7fc324162820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.378 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.376+0000 7fc329138640 1 --2- 172.21.15.131:0/1920338767 >> v2:172.21.15.189:3300/0 conn(0x7fc3240b5080 0x7fc3240b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.378 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.376+0000 7fc329138640 1 -- 172.21.15.131:0/1920338767 >> 172.21.15.131:0/1920338767 conn(0x7fc3240cf0b0 msgr2=0x7fc3240cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:46.378 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.376+0000 7fc329138640 1 -- 172.21.15.131:0/1920338767 shutdown_connections 2024-09-17T02:40:46.378 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.376+0000 7fc329138640 1 -- 172.21.15.131:0/1920338767 wait complete. 2024-09-17T02:40:46.379 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc329138640 1 Processor -- start 2024-09-17T02:40:46.379 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc329138640 1 -- start start 2024-09-17T02:40:46.382 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc329138640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3240b44f0 0x7fc324149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:46.382 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc329138640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3240b5080 0x7fc32406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:46.382 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc329138640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc324162d60 0x7fc32406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:46.382 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc329138640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240d05a0 con 0x7fc3240b44f0 2024-09-17T02:40:46.382 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc329138640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3240d0420 con 0x7fc3240b5080 2024-09-17T02:40:46.383 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc329138640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc3240d0720 con 0x7fc324162d60 2024-09-17T02:40:46.383 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc322d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3240b44f0 0x7fc324149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:46.383 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc322d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3240b44f0 0x7fc324149610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51880/0 (socket says 172.21.15.131:51880) 2024-09-17T02:40:46.383 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc322d9d640 1 -- 172.21.15.131:0/3421967720 learned_addr learned my addr 172.21.15.131:0/3421967720 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:46.383 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc32259c640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3240b5080 0x7fc32406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:46.383 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc32359e640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc324162d60 0x7fc32406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:46.383 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc322d9d640 1 -- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc324162d60 msgr2=0x7fc32406ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:46.383 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc322d9d640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc324162d60 0x7fc32406ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.383 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc322d9d640 1 -- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3240b5080 msgr2=0x7fc32406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:46.384 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc322d9d640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3240b5080 0x7fc32406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.384 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc322d9d640 1 -- 172.21.15.131:0/3421967720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc324149b50 con 0x7fc3240b44f0 2024-09-17T02:40:46.384 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc32259c640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3240b5080 0x7fc32406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:46.384 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc32359e640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc324162d60 0x7fc32406ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:46.384 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc322d9d640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3240b44f0 0x7fc324149610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc31400d950 tx=0x7fc31400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:46.384 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc30b7fe640 1 -- 172.21.15.131:0/3421967720 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314017070 con 0x7fc3240b44f0 2024-09-17T02:40:46.384 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc30b7fe640 1 -- 172.21.15.131:0/3421967720 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc314009d20 con 0x7fc3240b44f0 2024-09-17T02:40:46.384 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc30b7fe640 1 -- 172.21.15.131:0/3421967720 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3140106c0 con 0x7fc3240b44f0 2024-09-17T02:40:46.384 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc329138640 1 -- 172.21.15.131:0/3421967720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3240b12b0 con 0x7fc3240b44f0 2024-09-17T02:40:46.384 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.380+0000 7fc329138640 1 -- 172.21.15.131:0/3421967720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32406e9b0 con 0x7fc3240b44f0 2024-09-17T02:40:46.385 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.384+0000 7fc329138640 1 -- 172.21.15.131:0/3421967720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3240d0c60 con 0x7fc3240b44f0 2024-09-17T02:40:46.386 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.384+0000 7fc30b7fe640 1 -- 172.21.15.131:0/3421967720 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc31401a070 con 0x7fc3240b44f0 2024-09-17T02:40:46.386 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.384+0000 7fc30b7fe640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc310073230 0x7fc310075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:46.386 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.384+0000 7fc30b7fe640 1 -- 172.21.15.131:0/3421967720 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fc314094020 con 0x7fc3240b44f0 2024-09-17T02:40:46.386 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.384+0000 7fc32259c640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc310073230 0x7fc310075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:46.386 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.384+0000 7fc32259c640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc310073230 0x7fc310075740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc318005fd0 tx=0x7fc318005d60 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:46.389 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.388+0000 7fc30b7fe640 1 -- 172.21.15.131:0/3421967720 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc31405f420 con 0x7fc3240b44f0 2024-09-17T02:40:46.559 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.560+0000 7fc329138640 1 -- 172.21.15.131:0/3421967720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc32406a170 con 0x7fc3240b44f0 2024-09-17T02:40:46.561 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.560+0000 7fc30b7fe640 1 -- 172.21.15.131:0/3421967720 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12944 (secure 0 0 0) 0x7fc31405ed20 con 0x7fc3240b44f0 2024-09-17T02:40:46.561 INFO:teuthology.misc.health.smithi131.stdout: 2024-09-17T02:40:46.561 INFO:teuthology.misc.health.smithi131.stdout:{"epoch":18,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:44.003279+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:40:46.563 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 -- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc310073230 msgr2=0x7fc310075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:46.563 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc310073230 0x7fc310075740 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc318005fd0 tx=0x7fc318005d60 comp rx=0 tx=0).stop 2024-09-17T02:40:46.563 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 -- 172.21.15.131:0/3421967720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3240b44f0 msgr2=0x7fc324149610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:46.563 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3240b44f0 0x7fc324149610 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc31400d950 tx=0x7fc31400de10 comp rx=0 tx=0).stop 2024-09-17T02:40:46.564 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 -- 172.21.15.131:0/3421967720 shutdown_connections 2024-09-17T02:40:46.564 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc310073230 0x7fc310075740 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.564 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc324162d60 0x7fc32406ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.564 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3240b5080 0x7fc32406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.564 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 --2- 172.21.15.131:0/3421967720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3240b44f0 0x7fc324149610 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.564 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 -- 172.21.15.131:0/3421967720 >> 172.21.15.131:0/3421967720 conn(0x7fc3240cf0b0 msgr2=0x7fc3240c8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:46.564 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 -- 172.21.15.131:0/3421967720 shutdown_connections 2024-09-17T02:40:46.564 INFO:teuthology.misc.health.smithi131.stderr:2024-09-17T02:40:46.564+0000 7fc329138640 1 -- 172.21.15.131:0/3421967720 wait complete. 2024-09-17T02:40:46.581 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T02:40:46.581 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T02:40:46.581 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T02:40:46.581 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T02:40:46.583 INFO:tasks.ceph.mds.a:Started 2024-09-17T02:40:46.583 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T02:40:46.584 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T02:40:46.585 INFO:tasks.ceph.mds.c:Started 2024-09-17T02:40:46.585 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T02:40:46.586 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T02:40:46.588 INFO:tasks.ceph.mds.b:Started 2024-09-17T02:40:46.588 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T02:40:46.588 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T02:40:46.591 INFO:tasks.ceph.mds.d:Started 2024-09-17T02:40:46.591 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-17T02:40:46.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:40:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.752+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/3017506641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f740cdbf0 msgr2=0x7f6f740ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.752+0000 7f6f7b4a4640 1 --2- 172.21.15.131:0/3017506641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f740cdbf0 0x7f6f740ce000 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6f7406a1c0 tx=0x7f6f6c01cd10 comp rx=0 tx=0).stop 2024-09-17T02:40:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.752+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/3017506641 shutdown_connections 2024-09-17T02:40:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.752+0000 7f6f7b4a4640 1 --2- 172.21.15.131:0/3017506641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f740cdbf0 0x7f6f740ce000 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.752+0000 7f6f7b4a4640 1 --2- 172.21.15.131:0/3017506641 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6f740c4ac0 0x7f6f740c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.752+0000 7f6f7b4a4640 1 --2- 172.21.15.131:0/3017506641 >> v2:172.21.15.189:3300/0 conn(0x7f6f7406e2c0 0x7f6f740c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.752+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/3017506641 >> 172.21.15.131:0/3017506641 conn(0x7f6f7406eea0 msgr2=0x7f6f7406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.752+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/3017506641 shutdown_connections 2024-09-17T02:40:46.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.752+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/3017506641 wait complete. 2024-09-17T02:40:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.752+0000 7f6f7b4a4640 1 Processor -- start 2024-09-17T02:40:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.752+0000 7f6f7b4a4640 1 -- start start 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f7b4a4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7406e2c0 0x7f6f7414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f7b4a4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f740c4ac0 0x7f6f741547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f7b4a4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6f740cdbf0 0x7f6f74154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f7b4a4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6f740ceed0 con 0x7f6f7406e2c0 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f7b4a4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f740ced50 con 0x7f6f740c4ac0 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f7b4a4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6f740cf050 con 0x7f6f740cdbf0 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f79240640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7406e2c0 0x7f6f7414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f79240640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7406e2c0 0x7f6f7414efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51912/0 (socket says 172.21.15.131:51912) 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f79240640 1 -- 172.21.15.131:0/2929978649 learned_addr learned my addr 172.21.15.131:0/2929978649 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f78a3f640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f740c4ac0 0x7f6f741547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f79a41640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6f740cdbf0 0x7f6f74154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f79240640 1 -- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6f740cdbf0 msgr2=0x7f6f74154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f79240640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6f740cdbf0 0x7f6f74154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f79240640 1 -- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f740c4ac0 msgr2=0x7f6f741547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f79240640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f740c4ac0 0x7f6f741547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f79240640 1 -- 172.21.15.131:0/2929978649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f740ca670 con 0x7f6f7406e2c0 2024-09-17T02:40:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f79240640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7406e2c0 0x7f6f7414efc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f70005950 tx=0x7f6f7000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f69ffb640 1 -- 172.21.15.131:0/2929978649 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70007bb0 con 0x7f6f7406e2c0 2024-09-17T02:40:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f69ffb640 1 -- 172.21.15.131:0/2929978649 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f70007e40 con 0x7f6f7406e2c0 2024-09-17T02:40:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/2929978649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f741f90d0 con 0x7f6f7406e2c0 2024-09-17T02:40:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f69ffb640 1 -- 172.21.15.131:0/2929978649 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000f690 con 0x7f6f7406e2c0 2024-09-17T02:40:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/2929978649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f7415a8b0 con 0x7f6f7406e2c0 2024-09-17T02:40:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/2929978649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f740c4ed0 con 0x7f6f7406e2c0 2024-09-17T02:40:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.756+0000 7f6f69ffb640 1 -- 172.21.15.131:0/2929978649 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6f7000f830 con 0x7f6f7406e2c0 2024-09-17T02:40:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.760+0000 7f6f69ffb640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f58073230 0x7f6f58075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.760+0000 7f6f69ffb640 1 -- 172.21.15.131:0/2929978649 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f6f70094600 con 0x7f6f7406e2c0 2024-09-17T02:40:46.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.760+0000 7f6f78a3f640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f58073230 0x7f6f58075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:46.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.760+0000 7f6f78a3f640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f58073230 0x7f6f58075740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f740cc210 tx=0x7f6f64005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:46.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.764+0000 7f6f69ffb640 1 -- 172.21.15.131:0/2929978649 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6f70014030 con 0x7f6f7406e2c0 2024-09-17T02:40:46.805 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:40:46.806 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:40:46.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.976+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/2929978649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f74073310 con 0x7f6f7406e2c0 2024-09-17T02:40:46.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.976+0000 7f6f69ffb640 1 -- 172.21.15.131:0/2929978649 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1437 (secure 0 0 0) 0x7f6f7005f970 con 0x7f6f7406e2c0 2024-09-17T02:40:46.978 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:40:46.979 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":2,"btime":"2024-09-17T02:40:46:831641+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-17T02:40:46.979 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 2 2024-09-17T02:40:46.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f58073230 msgr2=0x7f6f58075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:46.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f58073230 0x7f6f58075740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f740cc210 tx=0x7f6f64005860 comp rx=0 tx=0).stop 2024-09-17T02:40:46.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/2929978649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7406e2c0 msgr2=0x7f6f7414efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:46.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7406e2c0 0x7f6f7414efc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6f70005950 tx=0x7f6f7000cdd0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/2929978649 shutdown_connections 2024-09-17T02:40:46.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f58073230 0x7f6f58075740 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6f740cdbf0 0x7f6f74154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f740c4ac0 0x7f6f741547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 --2- 172.21.15.131:0/2929978649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f7406e2c0 0x7f6f7414efc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:46.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/2929978649 >> 172.21.15.131:0/2929978649 conn(0x7f6f7406eea0 msgr2=0x7f6f740b3900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:46.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/2929978649 shutdown_connections 2024-09-17T02:40:46.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:46.980+0000 7f6f7b4a4640 1 -- 172.21.15.131:0/2929978649 wait complete. 2024-09-17T02:40:47.128 INFO:tasks.ceph.mds.b.smithi189.stdout:starting mds.b at 2024-09-17T02:40:47.135 INFO:tasks.ceph.mds.d.smithi189.stdout:starting mds.d at 2024-09-17T02:40:49.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:40:49.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.164+0000 7f127ce4a640 1 -- 172.21.15.131:0/3007894797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1278073970 msgr2=0x7f127806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.164+0000 7f127ce4a640 1 --2- 172.21.15.131:0/3007894797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1278073970 0x7f127806dea0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f127815c2d0 tx=0x7f126c01c890 comp rx=0 tx=0).stop 2024-09-17T02:40:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.164+0000 7f127ce4a640 1 -- 172.21.15.131:0/3007894797 shutdown_connections 2024-09-17T02:40:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.164+0000 7f127ce4a640 1 --2- 172.21.15.131:0/3007894797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1278073970 0x7f127806dea0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.164+0000 7f127ce4a640 1 --2- 172.21.15.131:0/3007894797 >> v2:172.21.15.189:3300/0 conn(0x7f127806e3e0 0x7f127815e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.164+0000 7f127ce4a640 1 --2- 172.21.15.131:0/3007894797 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1278155240 0x7f1278073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.164+0000 7f127ce4a640 1 -- 172.21.15.131:0/3007894797 >> 172.21.15.131:0/3007894797 conn(0x7f127806eea0 msgr2=0x7f127806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.164+0000 7f127ce4a640 1 -- 172.21.15.131:0/3007894797 shutdown_connections 2024-09-17T02:40:49.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.164+0000 7f127ce4a640 1 -- 172.21.15.131:0/3007894797 wait complete. 2024-09-17T02:40:49.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127ce4a640 1 Processor -- start 2024-09-17T02:40:49.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127ce4a640 1 -- start start 2024-09-17T02:40:49.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127ce4a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127806e3e0 0x7f12781543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127ce4a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1278073970 0x7f1278154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127ce4a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1278155240 0x7f127815c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127ce4a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f12780cace0 con 0x7f127806e3e0 2024-09-17T02:40:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127ce4a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f12780cab60 con 0x7f1278155240 2024-09-17T02:40:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127ce4a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f12780cae60 con 0x7f1278073970 2024-09-17T02:40:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127806e3e0 0x7f12781543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127806e3e0 0x7f12781543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51950/0 (socket says 172.21.15.131:51950) 2024-09-17T02:40:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127659c640 1 -- 172.21.15.131:0/367448382 learned_addr learned my addr 172.21.15.131:0/367448382 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f1276d9d640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1278155240 0x7f127815c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f1275d9b640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1278073970 0x7f1278154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127659c640 1 -- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1278073970 msgr2=0x7f1278154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127659c640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1278073970 0x7f1278154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127659c640 1 -- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1278155240 msgr2=0x7f127815c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127659c640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1278155240 0x7f127815c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127659c640 1 -- 172.21.15.131:0/367448382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127814cc90 con 0x7f127806e3e0 2024-09-17T02:40:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127659c640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127806e3e0 0x7f12781543d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f126800d720 tx=0x7f126800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f1266ffd640 1 -- 172.21.15.131:0/367448382 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268017070 con 0x7f127806e3e0 2024-09-17T02:40:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f1266ffd640 1 -- 172.21.15.131:0/367448382 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1268009d20 con 0x7f127806e3e0 2024-09-17T02:40:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127ce4a640 1 -- 172.21.15.131:0/367448382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127815e7c0 con 0x7f127806e3e0 2024-09-17T02:40:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f1266ffd640 1 -- 172.21.15.131:0/367448382 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268005280 con 0x7f127806e3e0 2024-09-17T02:40:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.168+0000 7f127ce4a640 1 -- 172.21.15.131:0/367448382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12780c6590 con 0x7f127806e3e0 2024-09-17T02:40:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.172+0000 7f127ce4a640 1 -- 172.21.15.131:0/367448382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f127806dea0 con 0x7f127806e3e0 2024-09-17T02:40:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.172+0000 7f1266ffd640 1 -- 172.21.15.131:0/367448382 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1268005420 con 0x7f127806e3e0 2024-09-17T02:40:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.172+0000 7f1266ffd640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1258073230 0x7f1258075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.172+0000 7f1266ffd640 1 -- 172.21.15.131:0/367448382 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f1268093fb0 con 0x7f127806e3e0 2024-09-17T02:40:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.172+0000 7f1275d9b640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1258073230 0x7f1258075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.172+0000 7f1275d9b640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1258073230 0x7f1258075740 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f126c0af000 tx=0x7f126c01b0d0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:49.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.176+0000 7f1266ffd640 1 -- 172.21.15.131:0/367448382 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f126805f3b0 con 0x7f127806e3e0 2024-09-17T02:40:49.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.380+0000 7f127ce4a640 1 -- 172.21.15.131:0/367448382 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f127806e9d0 con 0x7f127806e3e0 2024-09-17T02:40:49.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.380+0000 7f1266ffd640 1 -- 172.21.15.131:0/367448382 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3918 (secure 0 0 0) 0x7f126805ecb0 con 0x7f127806e3e0 2024-09-17T02:40:49.381 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:40:49.381 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":3,"btime":"2024-09-17T02:40:47:833219+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3035404898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3035404898},{"type":"v1","addr":"172.21.15.131:6835","nonce":3035404898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3147445846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3147445846},{"type":"v1","addr":"172.21.15.189:6835","nonce":3147445846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1582109479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1582109479},{"type":"v1","addr":"172.21.15.189:6837","nonce":1582109479}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-17T02:40:49.381 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 3 2024-09-17T02:40:49.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 -- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1258073230 msgr2=0x7f1258075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1258073230 0x7f1258075740 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f126c0af000 tx=0x7f126c01b0d0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 -- 172.21.15.131:0/367448382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127806e3e0 msgr2=0x7f12781543d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127806e3e0 0x7f12781543d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f126800d720 tx=0x7f126800dbe0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 -- 172.21.15.131:0/367448382 shutdown_connections 2024-09-17T02:40:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1258073230 0x7f1258075740 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1278155240 0x7f127815c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1278073970 0x7f1278154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 --2- 172.21.15.131:0/367448382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f127806e3e0 0x7f12781543d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 -- 172.21.15.131:0/367448382 >> 172.21.15.131:0/367448382 conn(0x7f127806eea0 msgr2=0x7f12780cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 -- 172.21.15.131:0/367448382 shutdown_connections 2024-09-17T02:40:49.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.384+0000 7f127ce4a640 1 -- 172.21.15.131:0/367448382 wait complete. 2024-09-17T02:40:49.401 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T02:40:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.552+0000 7fc7c6624640 1 -- 172.21.15.131:0/62945024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c0071bd0 msgr2=0x7fc7c00ae160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.552+0000 7fc7c6624640 1 --2- 172.21.15.131:0/62945024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c0071bd0 0x7fc7c00ae160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009900 tx=0x7fc7b401c8b0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.552+0000 7fc7c6624640 1 -- 172.21.15.131:0/62945024 shutdown_connections 2024-09-17T02:40:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.552+0000 7fc7c6624640 1 --2- 172.21.15.131:0/62945024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c0071bd0 0x7fc7c00ae160 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.552+0000 7fc7c6624640 1 -- 172.21.15.131:0/62945024 >> 172.21.15.131:0/62945024 conn(0x7fc7c00cd6a0 msgr2=0x7fc7c00cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.552+0000 7fc7c6624640 1 -- 172.21.15.131:0/62945024 shutdown_connections 2024-09-17T02:40:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.552+0000 7fc7c6624640 1 -- 172.21.15.131:0/62945024 wait complete. 2024-09-17T02:40:49.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7c6624640 1 Processor -- start 2024-09-17T02:40:49.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7c6624640 1 -- start start 2024-09-17T02:40:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7c6624640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c00cae70 0x7fc7c01517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7c6624640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cb280 0x7fc7c015e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7c6624640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7c00b4e20 0x7fc7c01554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7c6624640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c016cfd0 con 0x7fc7c00cb280 2024-09-17T02:40:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7c6624640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c016ce50 con 0x7fc7c00cae70 2024-09-17T02:40:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7c6624640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c016d150 con 0x7fc7c00b4e20 2024-09-17T02:40:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cb280 0x7fc7c015e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cb280 0x7fc7c015e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51962/0 (socket says 172.21.15.131:51962) 2024-09-17T02:40:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bf7fe640 1 -- 172.21.15.131:0/3401701895 learned_addr learned my addr 172.21.15.131:0/3401701895 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bffff640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c00cae70 0x7fc7c01517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7c4bc1640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7c00b4e20 0x7fc7c01554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bf7fe640 1 -- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7c00b4e20 msgr2=0x7fc7c01554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bf7fe640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7c00b4e20 0x7fc7c01554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bf7fe640 1 -- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c00cae70 msgr2=0x7fc7c01517e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bf7fe640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c00cae70 0x7fc7c01517e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bf7fe640 1 -- 172.21.15.131:0/3401701895 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c01fcff0 con 0x7fc7c00cb280 2024-09-17T02:40:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bffff640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c00cae70 0x7fc7c01517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bf7fe640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cb280 0x7fc7c015e050 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009850 tx=0x7fc7b40a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bcff9640 1 -- 172.21.15.131:0/3401701895 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40b2040 con 0x7fc7c00cb280 2024-09-17T02:40:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bcff9640 1 -- 172.21.15.131:0/3401701895 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7b4004430 con 0x7fc7c00cb280 2024-09-17T02:40:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.556+0000 7fc7bcff9640 1 -- 172.21.15.131:0/3401701895 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4003860 con 0x7fc7c00cb280 2024-09-17T02:40:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.560+0000 7fc7c6624640 1 -- 172.21.15.131:0/3401701895 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c00b5550 con 0x7fc7c00cb280 2024-09-17T02:40:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.560+0000 7fc7c6624640 1 -- 172.21.15.131:0/3401701895 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c01fc510 con 0x7fc7c00cb280 2024-09-17T02:40:49.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.560+0000 7fc7c6624640 1 -- 172.21.15.131:0/3401701895 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c00714c0 con 0x7fc7c00cb280 2024-09-17T02:40:49.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.560+0000 7fc7bcff9640 1 -- 172.21.15.131:0/3401701895 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc7b4003a00 con 0x7fc7c00cb280 2024-09-17T02:40:49.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.560+0000 7fc7bcff9640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073200 0x7fc7ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.560+0000 7fc7bcff9640 1 -- 172.21.15.131:0/3401701895 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fc7b412e2b0 con 0x7fc7c00cb280 2024-09-17T02:40:49.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.560+0000 7fc7bffff640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073200 0x7fc7ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.560+0000 7fc7bffff640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073200 0x7fc7ac075710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8007fd0 tx=0x7fc7b8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:49.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.564+0000 7fc7bcff9640 1 -- 172.21.15.131:0/3401701895 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc7b40164d0 con 0x7fc7c00cb280 2024-09-17T02:40:49.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.732+0000 7fc7c6624640 1 -- 172.21.15.131:0/3401701895 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc7c00b52e0 con 0x7fc7c00cb280 2024-09-17T02:40:49.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.732+0000 7fc7bcff9640 1 -- 172.21.15.131:0/3401701895 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7fc7b40f95a0 con 0x7fc7c00cb280 2024-09-17T02:40:49.734 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:40:49.734 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:40:49.734 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:40:49.734 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-09-17T02:40:49.735 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-09-17T02:40:49.735 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:40:49.735 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:40:49.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 -- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073200 msgr2=0x7fc7ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073200 0x7fc7ac075710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8007fd0 tx=0x7fc7b8009040 comp rx=0 tx=0).stop 2024-09-17T02:40:49.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 -- 172.21.15.131:0/3401701895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cb280 msgr2=0x7fc7c015e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cb280 0x7fc7c015e050 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4009850 tx=0x7fc7b40a5790 comp rx=0 tx=0).stop 2024-09-17T02:40:49.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 -- 172.21.15.131:0/3401701895 shutdown_connections 2024-09-17T02:40:49.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073200 0x7fc7ac075710 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7c00b4e20 0x7fc7c01554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cb280 0x7fc7c015e050 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 --2- 172.21.15.131:0/3401701895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c00cae70 0x7fc7c01517e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 -- 172.21.15.131:0/3401701895 >> 172.21.15.131:0/3401701895 conn(0x7fc7c00cd6a0 msgr2=0x7fc7c00ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:49.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 -- 172.21.15.131:0/3401701895 shutdown_connections 2024-09-17T02:40:49.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.736+0000 7fc7c6624640 1 -- 172.21.15.131:0/3401701895 wait complete. 2024-09-17T02:40:49.754 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T02:40:49.754 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T02:40:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.904+0000 7fa30847c640 1 -- 172.21.15.131:0/565737539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2ec004810 msgr2=0x7fa2ec004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.904+0000 7fa30847c640 1 --2- 172.21.15.131:0/565737539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2ec004810 0x7fa2ec004c20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa30006e950 tx=0x7fa2f001c730 comp rx=0 tx=0).stop 2024-09-17T02:40:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.904+0000 7fa30847c640 1 -- 172.21.15.131:0/565737539 shutdown_connections 2024-09-17T02:40:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.904+0000 7fa30847c640 1 --2- 172.21.15.131:0/565737539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2ec004810 0x7fa2ec004c20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.904+0000 7fa30847c640 1 -- 172.21.15.131:0/565737539 >> 172.21.15.131:0/565737539 conn(0x7fa3000d05a0 msgr2=0x7fa3000d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.904+0000 7fa30847c640 1 -- 172.21.15.131:0/565737539 shutdown_connections 2024-09-17T02:40:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.904+0000 7fa30847c640 1 -- 172.21.15.131:0/565737539 wait complete. 2024-09-17T02:40:49.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa30847c640 1 Processor -- start 2024-09-17T02:40:49.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa30847c640 1 -- start start 2024-09-17T02:40:49.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa30847c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2ec004810 0x7fa3001f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa30847c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3001441c0 0x7fa300168880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa30847c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa30013adc0 0x7fa30013b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa30847c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa3000cc260 con 0x7fa3001441c0 2024-09-17T02:40:49.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa30847c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa3000cc0e0 con 0x7fa2ec004810 2024-09-17T02:40:49.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa30847c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa3000cc3e0 con 0x7fa30013adc0 2024-09-17T02:40:49.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa305a17640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3001441c0 0x7fa300168880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa305a17640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3001441c0 0x7fa300168880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51986/0 (socket says 172.21.15.131:51986) 2024-09-17T02:40:49.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa305a17640 1 -- 172.21.15.131:0/1408520714 learned_addr learned my addr 172.21.15.131:0/1408520714 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:49.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa306a19640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa30013adc0 0x7fa30013b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa306218640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2ec004810 0x7fa3001f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa305a17640 1 -- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa30013adc0 msgr2=0x7fa30013b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa305a17640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa30013adc0 0x7fa30013b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa305a17640 1 -- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2ec004810 msgr2=0x7fa3001f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:49.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa305a17640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2ec004810 0x7fa3001f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:49.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa305a17640 1 -- 172.21.15.131:0/1408520714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa300168dc0 con 0x7fa3001441c0 2024-09-17T02:40:49.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa305a17640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3001441c0 0x7fa300168880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2f001ae30 tx=0x7fa2f0003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:49.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa2f6ffd640 1 -- 172.21.15.131:0/1408520714 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f00b9070 con 0x7fa3001441c0 2024-09-17T02:40:49.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa2f6ffd640 1 -- 172.21.15.131:0/1408520714 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2f00ae040 con 0x7fa3001441c0 2024-09-17T02:40:49.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa30847c640 1 -- 172.21.15.131:0/1408520714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3001692d0 con 0x7fa3001441c0 2024-09-17T02:40:49.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa2f6ffd640 1 -- 172.21.15.131:0/1408520714 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f00ad070 con 0x7fa3001441c0 2024-09-17T02:40:49.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.908+0000 7fa30847c640 1 -- 172.21.15.131:0/1408520714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3001f7040 con 0x7fa3001441c0 2024-09-17T02:40:49.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.912+0000 7fa30847c640 1 -- 172.21.15.131:0/1408520714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3000cc940 con 0x7fa3001441c0 2024-09-17T02:40:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.912+0000 7fa2f6ffd640 1 -- 172.21.15.131:0/1408520714 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa2f0006350 con 0x7fa3001441c0 2024-09-17T02:40:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.912+0000 7fa2f6ffd640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2e4073200 0x7fa2e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.912+0000 7fa2f6ffd640 1 -- 172.21.15.131:0/1408520714 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fa2f012d560 con 0x7fa3001441c0 2024-09-17T02:40:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.912+0000 7fa306218640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2e4073200 0x7fa2e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:49.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.912+0000 7fa306218640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2e4073200 0x7fa2e4075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8001250 tx=0x7fa2f8000f40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:49.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:49.916+0000 7fa2f6ffd640 1 -- 172.21.15.131:0/1408520714 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa2f0016490 con 0x7fa3001441c0 2024-09-17T02:40:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.072+0000 7fa30847c640 1 -- 172.21.15.131:0/1408520714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa300169020 con 0x7fa3001441c0 2024-09-17T02:40:50.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.172+0000 7fa2f6ffd640 1 -- 172.21.15.131:0/1408520714 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7fa2f00f83f0 con 0x7fa3001441c0 2024-09-17T02:40:50.174 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-09-17T02:40:50.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 -- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2e4073200 msgr2=0x7fa2e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:50.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2e4073200 0x7fa2e4075710 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8001250 tx=0x7fa2f8000f40 comp rx=0 tx=0).stop 2024-09-17T02:40:50.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 -- 172.21.15.131:0/1408520714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3001441c0 msgr2=0x7fa300168880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:50.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3001441c0 0x7fa300168880 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa2f001ae30 tx=0x7fa2f0003c60 comp rx=0 tx=0).stop 2024-09-17T02:40:50.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 -- 172.21.15.131:0/1408520714 shutdown_connections 2024-09-17T02:40:50.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2e4073200 0x7fa2e4075710 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:50.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa30013adc0 0x7fa30013b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:50.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3001441c0 0x7fa300168880 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:50.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 --2- 172.21.15.131:0/1408520714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2ec004810 0x7fa3001f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:50.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 -- 172.21.15.131:0/1408520714 >> 172.21.15.131:0/1408520714 conn(0x7fa3000d05a0 msgr2=0x7fa30015e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:50.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 -- 172.21.15.131:0/1408520714 shutdown_connections 2024-09-17T02:40:50.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.176+0000 7fa30847c640 1 -- 172.21.15.131:0/1408520714 wait complete. 2024-09-17T02:40:50.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-17T02:40:50.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.364+0000 7fe4598e8640 1 -- 172.21.15.131:0/3412991061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe43c003a90 msgr2=0x7fe43c003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:50.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.364+0000 7fe4598e8640 1 --2- 172.21.15.131:0/3412991061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe43c003a90 0x7fe43c003f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4440016f0 tx=0x7fe44401c790 comp rx=0 tx=0).stop 2024-09-17T02:40:50.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.364+0000 7fe4598e8640 1 -- 172.21.15.131:0/3412991061 shutdown_connections 2024-09-17T02:40:50.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.364+0000 7fe4598e8640 1 --2- 172.21.15.131:0/3412991061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe43c003a90 0x7fe43c003f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:50.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.364+0000 7fe4598e8640 1 -- 172.21.15.131:0/3412991061 >> 172.21.15.131:0/3412991061 conn(0x7fe45406fdb0 msgr2=0x7fe4540701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:50.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.364+0000 7fe4598e8640 1 -- 172.21.15.131:0/3412991061 shutdown_connections 2024-09-17T02:40:50.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.364+0000 7fe4598e8640 1 -- 172.21.15.131:0/3412991061 wait complete. 2024-09-17T02:40:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.364+0000 7fe4598e8640 1 Processor -- start 2024-09-17T02:40:50.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.364+0000 7fe4598e8640 1 -- start start 2024-09-17T02:40:50.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4598e8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe43c003a90 0x7fe4541f64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:50.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4598e8640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe454140410 0x7fe454140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:50.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4598e8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe454140d60 0x7fe4540cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:50.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4598e8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe4540744b0 con 0x7fe454140d60 2024-09-17T02:40:50.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4598e8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe454074330 con 0x7fe43c003a90 2024-09-17T02:40:50.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4598e8640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe454074630 con 0x7fe454140410 2024-09-17T02:40:50.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4537fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe454140d60 0x7fe4540cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:50.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4537fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe454140d60 0x7fe4540cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51996/0 (socket says 172.21.15.131:51996) 2024-09-17T02:40:50.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4537fe640 1 -- 172.21.15.131:0/2637989188 learned_addr learned my addr 172.21.15.131:0/2637989188 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4527fc640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe454140410 0x7fe454140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe452ffd640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe43c003a90 0x7fe4541f64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4537fe640 1 -- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe454140410 msgr2=0x7fe454140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4537fe640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe454140410 0x7fe454140820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4537fe640 1 -- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe43c003a90 msgr2=0x7fe4541f64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4537fe640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe43c003a90 0x7fe4541f64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4537fe640 1 -- 172.21.15.131:0/2637989188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4541468a0 con 0x7fe454140d60 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe452ffd640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe43c003a90 0x7fe4541f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4537fe640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe454140d60 0x7fe4540cc0e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe44c001340 tx=0x7fe44c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe43bfff640 1 -- 172.21.15.131:0/2637989188 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00cc20 con 0x7fe454140d60 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe43bfff640 1 -- 172.21.15.131:0/2637989188 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe44c018070 con 0x7fe454140d60 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4598e8640 1 -- 172.21.15.131:0/2637989188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe454156f80 con 0x7fe454140d60 2024-09-17T02:40:50.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe43bfff640 1 -- 172.21.15.131:0/2637989188 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44c0116c0 con 0x7fe454140d60 2024-09-17T02:40:50.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4598e8640 1 -- 172.21.15.131:0/2637989188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe454157540 con 0x7fe454140d60 2024-09-17T02:40:50.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe4598e8640 1 -- 172.21.15.131:0/2637989188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe45415f090 con 0x7fe454140d60 2024-09-17T02:40:50.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.368+0000 7fe43bfff640 1 -- 172.21.15.131:0/2637989188 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe44c011860 con 0x7fe454140d60 2024-09-17T02:40:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.372+0000 7fe43bfff640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe440073200 0x7fe440075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.372+0000 7fe43bfff640 1 -- 172.21.15.131:0/2637989188 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7fe44c095090 con 0x7fe454140d60 2024-09-17T02:40:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.372+0000 7fe452ffd640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe440073200 0x7fe440075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:50.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.372+0000 7fe452ffd640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe440073200 0x7fe440075710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4440016c0 tx=0x7fe444019580 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:50.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.376+0000 7fe43bfff640 1 -- 172.21.15.131:0/2637989188 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe44c060290 con 0x7fe454140d60 2024-09-17T02:40:50.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:50.532+0000 7fe4598e8640 1 -- 172.21.15.131:0/2637989188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fe4541594f0 con 0x7fe454140d60 2024-09-17T02:40:51.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.184+0000 7fe43bfff640 1 -- 172.21.15.131:0/2637989188 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v20) ==== 171+0+0 (secure 0 0 0) 0x7fe44c05fb90 con 0x7fe454140d60 2024-09-17T02:40:51.186 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-09-17T02:40:51.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 -- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe440073200 msgr2=0x7fe440075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:51.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe440073200 0x7fe440075710 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe4440016c0 tx=0x7fe444019580 comp rx=0 tx=0).stop 2024-09-17T02:40:51.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 -- 172.21.15.131:0/2637989188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe454140d60 msgr2=0x7fe4540cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:51.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe454140d60 0x7fe4540cc0e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe44c001340 tx=0x7fe44c00ed90 comp rx=0 tx=0).stop 2024-09-17T02:40:51.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 -- 172.21.15.131:0/2637989188 shutdown_connections 2024-09-17T02:40:51.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe440073200 0x7fe440075710 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:51.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe454140d60 0x7fe4540cc0e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:51.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe454140410 0x7fe454140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:51.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 --2- 172.21.15.131:0/2637989188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe43c003a90 0x7fe4541f64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:51.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 -- 172.21.15.131:0/2637989188 >> 172.21.15.131:0/2637989188 conn(0x7fe45406fdb0 msgr2=0x7fe4541552a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:51.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 -- 172.21.15.131:0/2637989188 shutdown_connections 2024-09-17T02:40:51.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.188+0000 7fe4598e8640 1 -- 172.21.15.131:0/2637989188 wait complete. 2024-09-17T02:40:51.227 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T02:40:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.384+0000 7f2a5a03a640 1 -- 172.21.15.131:0/194920288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a3c004d50 msgr2=0x7f2a3c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.384+0000 7f2a5a03a640 1 --2- 172.21.15.131:0/194920288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a3c004d50 0x7f2a3c005160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c006fd0 tx=0x7f2a4c01d260 comp rx=0 tx=0).stop 2024-09-17T02:40:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.384+0000 7f2a53fff640 1 -- 172.21.15.131:0/194920288 reap_dead start 2024-09-17T02:40:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.384+0000 7f2a5a03a640 1 -- 172.21.15.131:0/194920288 shutdown_connections 2024-09-17T02:40:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.384+0000 7f2a5a03a640 1 -- 172.21.15.131:0/194920288 >> 172.21.15.131:0/194920288 conn(0x7f2a540cf0b0 msgr2=0x7f2a540cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:51.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.384+0000 7f2a5a03a640 1 -- 172.21.15.131:0/194920288 shutdown_connections 2024-09-17T02:40:51.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.384+0000 7f2a5a03a640 1 -- 172.21.15.131:0/194920288 wait complete. 2024-09-17T02:40:51.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a5a03a640 1 Processor -- start 2024-09-17T02:40:51.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a5a03a640 1 -- start start 2024-09-17T02:40:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a5a03a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a541f7550 0x7f2a5414c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:51.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a5a03a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a540ca920 0x7f2a541f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a5a03a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a541f6ef0 0x7f2a5414ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a5a03a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a540711e0 con 0x7f2a541f6ef0 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a5a03a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a540b6b40 con 0x7f2a541f7550 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a5a03a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2a54071360 con 0x7f2a540ca920 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a53fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a541f6ef0 0x7f2a5414ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a53fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a541f6ef0 0x7f2a5414ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41342/0 (socket says 172.21.15.131:41342) 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a53fff640 1 -- 172.21.15.131:0/988992343 learned_addr learned my addr 172.21.15.131:0/988992343 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a52ffd640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a540ca920 0x7f2a541f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a537fe640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a541f7550 0x7f2a5414c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a53fff640 1 -- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a540ca920 msgr2=0x7f2a541f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a53fff640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a540ca920 0x7f2a541f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:51.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a53fff640 1 -- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a541f7550 msgr2=0x7f2a5414c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:51.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a53fff640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a541f7550 0x7f2a5414c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:51.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a53fff640 1 -- 172.21.15.131:0/988992343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a541f7300 con 0x7f2a541f6ef0 2024-09-17T02:40:51.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a52ffd640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a540ca920 0x7f2a541f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:40:51.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a53fff640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a541f6ef0 0x7f2a5414ef20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c001e00 tx=0x7f2a4c001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:51.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a3bfff640 1 -- 172.21.15.131:0/988992343 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c006270 con 0x7f2a541f6ef0 2024-09-17T02:40:51.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a3bfff640 1 -- 172.21.15.131:0/988992343 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a4c006500 con 0x7f2a541f6ef0 2024-09-17T02:40:51.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a5a03a640 1 -- 172.21.15.131:0/988992343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5414cba0 con 0x7f2a541f6ef0 2024-09-17T02:40:51.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a3bfff640 1 -- 172.21.15.131:0/988992343 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0068f0 con 0x7f2a541f6ef0 2024-09-17T02:40:51.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.388+0000 7f2a5a03a640 1 -- 172.21.15.131:0/988992343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a540cb150 con 0x7f2a541f6ef0 2024-09-17T02:40:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.392+0000 7f2a5a03a640 1 -- 172.21.15.131:0/988992343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a540cd010 con 0x7f2a541f6ef0 2024-09-17T02:40:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.392+0000 7f2a3bfff640 1 -- 172.21.15.131:0/988992343 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2a4c0ae3f0 con 0x7f2a541f6ef0 2024-09-17T02:40:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.392+0000 7f2a3bfff640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a40073200 0x7f2a40075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.392+0000 7f2a3bfff640 1 -- 172.21.15.131:0/988992343 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7f2a4c12e230 con 0x7f2a541f6ef0 2024-09-17T02:40:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.392+0000 7f2a537fe640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a40073200 0x7f2a40075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:51.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.392+0000 7f2a537fe640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a40073200 0x7f2a40075710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2a540ac010 tx=0x7f2a440091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:51.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.396+0000 7f2a3bfff640 1 -- 172.21.15.131:0/988992343 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2a541f7300 con 0x7f2a541f6ef0 2024-09-17T02:40:51.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:51.604+0000 7f2a5a03a640 1 -- 172.21.15.131:0/988992343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2a5406a170 con 0x7f2a541f6ef0 2024-09-17T02:40:52.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.188+0000 7f2a3bfff640 1 -- 172.21.15.131:0/988992343 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f2a540cd3e0 con 0x7f2a541f6ef0 2024-09-17T02:40:52.189 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 2 and data pool 3 2024-09-17T02:40:52.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 -- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a40073200 msgr2=0x7f2a40075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:52.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a40073200 0x7f2a40075710 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2a540ac010 tx=0x7f2a440091e0 comp rx=0 tx=0).stop 2024-09-17T02:40:52.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 -- 172.21.15.131:0/988992343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a541f6ef0 msgr2=0x7f2a5414ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:52.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a541f6ef0 0x7f2a5414ef20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c001e00 tx=0x7f2a4c001f10 comp rx=0 tx=0).stop 2024-09-17T02:40:52.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 -- 172.21.15.131:0/988992343 shutdown_connections 2024-09-17T02:40:52.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a40073200 0x7f2a40075710 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:52.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a541f6ef0 0x7f2a5414ef20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:52.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a540ca920 0x7f2a541f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:52.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 --2- 172.21.15.131:0/988992343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a541f7550 0x7f2a5414c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:52.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 -- 172.21.15.131:0/988992343 >> 172.21.15.131:0/988992343 conn(0x7f2a540cf0b0 msgr2=0x7f2a54070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:52.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 -- 172.21.15.131:0/988992343 shutdown_connections 2024-09-17T02:40:52.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.192+0000 7f2a5a03a640 1 -- 172.21.15.131:0/988992343 wait complete. 2024-09-17T02:40:52.232 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-17T02:40:52.232 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-17T02:40:52.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.400+0000 7f0de7c98640 1 -- 172.21.15.131:0/3749575145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dcc004810 msgr2=0x7f0dcc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:52.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.400+0000 7f0de7c98640 1 --2- 172.21.15.131:0/3749575145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dcc004810 0x7f0dcc004c20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0009b60 tx=0x7f0dd001d280 comp rx=0 tx=0).stop 2024-09-17T02:40:52.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.400+0000 7f0de6235640 1 -- 172.21.15.131:0/3749575145 reap_dead start 2024-09-17T02:40:52.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.400+0000 7f0de7c98640 1 -- 172.21.15.131:0/3749575145 shutdown_connections 2024-09-17T02:40:52.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.400+0000 7f0de7c98640 1 -- 172.21.15.131:0/3749575145 >> 172.21.15.131:0/3749575145 conn(0x7f0de00d05a0 msgr2=0x7f0de00d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:52.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.400+0000 7f0de7c98640 1 -- 172.21.15.131:0/3749575145 shutdown_connections 2024-09-17T02:40:52.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.400+0000 7f0de7c98640 1 -- 172.21.15.131:0/3749575145 wait complete. 2024-09-17T02:40:52.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de7c98640 1 Processor -- start 2024-09-17T02:40:52.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de7c98640 1 -- start start 2024-09-17T02:40:52.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de7c98640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dcc004810 0x7f0de01f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:52.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de7c98640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0de0155950 0x7f0de01f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:52.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de7c98640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de0157520 0x7f0de015e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:52.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de7c98640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0de0167ad0 con 0x7f0dcc004810 2024-09-17T02:40:52.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de7c98640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0de0167950 con 0x7f0de0157520 2024-09-17T02:40:52.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de7c98640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0de0167c50 con 0x7f0de0155950 2024-09-17T02:40:52.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de5a34640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dcc004810 0x7f0de01f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:52.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de5a34640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dcc004810 0x7f0de01f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41350/0 (socket says 172.21.15.131:41350) 2024-09-17T02:40:52.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de5a34640 1 -- 172.21.15.131:0/569936717 learned_addr learned my addr 172.21.15.131:0/569936717 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:52.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de5233640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0de0155950 0x7f0de01f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:52.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de6235640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de0157520 0x7f0de015e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:52.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de5a34640 1 -- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0de0155950 msgr2=0x7f0de01f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:52.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de5a34640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0de0155950 0x7f0de01f7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:52.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de5a34640 1 -- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de0157520 msgr2=0x7f0de015e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:52.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de5a34640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de0157520 0x7f0de015e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:52.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de5a34640 1 -- 172.21.15.131:0/569936717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de00cf020 con 0x7f0dcc004810 2024-09-17T02:40:52.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de5233640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0de0155950 0x7f0de01f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:52.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de6235640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de0157520 0x7f0de015e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:52.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de5a34640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dcc004810 0x7f0de01f75f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc011ab0 tx=0x7f0ddc011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:52.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0dd67fc640 1 -- 172.21.15.131:0/569936717 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc01e070 con 0x7f0dcc004810 2024-09-17T02:40:52.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0dd67fc640 1 -- 172.21.15.131:0/569936717 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ddc00fc70 con 0x7f0dcc004810 2024-09-17T02:40:52.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0dd67fc640 1 -- 172.21.15.131:0/569936717 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc019670 con 0x7f0dcc004810 2024-09-17T02:40:52.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de7c98640 1 -- 172.21.15.131:0/569936717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de00af4b0 con 0x7f0dcc004810 2024-09-17T02:40:52.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de7c98640 1 -- 172.21.15.131:0/569936717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de0156d20 con 0x7f0dcc004810 2024-09-17T02:40:52.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.404+0000 7f0de7c98640 1 -- 172.21.15.131:0/569936717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de01603d0 con 0x7f0dcc004810 2024-09-17T02:40:52.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.408+0000 7f0dd67fc640 1 -- 172.21.15.131:0/569936717 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0ddc021070 con 0x7f0dcc004810 2024-09-17T02:40:52.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.408+0000 7f0dd67fc640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0dc4073230 0x7f0dc4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:52.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.408+0000 7f0dd67fc640 1 -- 172.21.15.131:0/569936717 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0ddc094a10 con 0x7f0dcc004810 2024-09-17T02:40:52.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.408+0000 7f0de5233640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0dc4073230 0x7f0dc4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:52.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.408+0000 7f0de5233640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0dc4073230 0x7f0dc4075740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00ae000 tx=0x7f0dd001a020 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:52.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.412+0000 7f0dd67fc640 1 -- 172.21.15.131:0/569936717 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0ddc00fe60 con 0x7f0dcc004810 2024-09-17T02:40:52.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:52.591+0000 7f0de7c98640 1 -- 172.21.15.131:0/569936717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f0de00cf6d0 con 0x7f0dcc004810 2024-09-17T02:40:53.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.191+0000 7f0dd67fc640 1 -- 172.21.15.131:0/569936717 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v22) ==== 157+0+0 (secure 0 0 0) 0x7f0ddc05f360 con 0x7f0dcc004810 2024-09-17T02:40:53.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 -- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0dc4073230 msgr2=0x7f0dc4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:53.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0dc4073230 0x7f0dc4075740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0dd00ae000 tx=0x7f0dd001a020 comp rx=0 tx=0).stop 2024-09-17T02:40:53.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 -- 172.21.15.131:0/569936717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dcc004810 msgr2=0x7f0de01f75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:53.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dcc004810 0x7f0de01f75f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc011ab0 tx=0x7f0ddc011f70 comp rx=0 tx=0).stop 2024-09-17T02:40:53.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 -- 172.21.15.131:0/569936717 shutdown_connections 2024-09-17T02:40:53.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0dc4073230 0x7f0dc4075740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:53.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0de0157520 0x7f0de015e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:53.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0de0155950 0x7f0de01f7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:53.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 --2- 172.21.15.131:0/569936717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dcc004810 0x7f0de01f75f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:53.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 -- 172.21.15.131:0/569936717 >> 172.21.15.131:0/569936717 conn(0x7f0de00d05a0 msgr2=0x7f0de015c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:53.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 -- 172.21.15.131:0/569936717 shutdown_connections 2024-09-17T02:40:53.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.195+0000 7f0de7c98640 1 -- 172.21.15.131:0/569936717 wait complete. 2024-09-17T02:40:53.218 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T02:40:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.383+0000 7f8acdecc640 1 -- 172.21.15.131:0/1145147573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac80b30a0 msgr2=0x7f8ac80b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.383+0000 7f8acdecc640 1 --2- 172.21.15.131:0/1145147573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac80b30a0 0x7f8ac80b34b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009cb0 tx=0x7f8ac001cda0 comp rx=0 tx=0).stop 2024-09-17T02:40:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.383+0000 7f8acdecc640 1 -- 172.21.15.131:0/1145147573 shutdown_connections 2024-09-17T02:40:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.383+0000 7f8acdecc640 1 --2- 172.21.15.131:0/1145147573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac80b30a0 0x7f8ac80b34b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.383+0000 7f8acdecc640 1 --2- 172.21.15.131:0/1145147573 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8ac815f0a0 0x7f8ac815f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.383+0000 7f8acdecc640 1 -- 172.21.15.131:0/1145147573 >> 172.21.15.131:0/1145147573 conn(0x7f8ac806eea0 msgr2=0x7f8ac806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:53.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.383+0000 7f8acdecc640 1 -- 172.21.15.131:0/1145147573 shutdown_connections 2024-09-17T02:40:53.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.383+0000 7f8acdecc640 1 -- 172.21.15.131:0/1145147573 wait complete. 2024-09-17T02:40:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.383+0000 7f8acdecc640 1 Processor -- start 2024-09-17T02:40:53.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.383+0000 7f8acdecc640 1 -- start start 2024-09-17T02:40:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8acdecc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ac80b30a0 0x7f8ac8162280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8acdecc640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8ac8152600 0x7f8ac815e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8acdecc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac815f0a0 0x7f8ac815e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8acdecc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac8168b90 con 0x7f8ac815f0a0 2024-09-17T02:40:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8acdecc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac8168a10 con 0x7f8ac80b30a0 2024-09-17T02:40:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8acdecc640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac8168d10 con 0x7f8ac8152600 2024-09-17T02:40:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac815f0a0 0x7f8ac815e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac815f0a0 0x7f8ac815e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41366/0 (socket says 172.21.15.131:41366) 2024-09-17T02:40:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac7fff640 1 -- 172.21.15.131:0/218871561 learned_addr learned my addr 172.21.15.131:0/218871561 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac77fe640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ac80b30a0 0x7f8ac8162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:53.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac6ffd640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8ac8152600 0x7f8ac815e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac7fff640 1 -- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8ac8152600 msgr2=0x7f8ac815e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac7fff640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8ac8152600 0x7f8ac815e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac7fff640 1 -- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ac80b30a0 msgr2=0x7f8ac8162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac7fff640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ac80b30a0 0x7f8ac8162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac7fff640 1 -- 172.21.15.131:0/218871561 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac8162ab0 con 0x7f8ac815f0a0 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac6ffd640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8ac8152600 0x7f8ac815e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8ac7fff640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac815f0a0 0x7f8ac815e710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8ac001b450 tx=0x7f8ac00092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8aaffff640 1 -- 172.21.15.131:0/218871561 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac00b2040 con 0x7f8ac815f0a0 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8aaffff640 1 -- 172.21.15.131:0/218871561 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ac0004d40 con 0x7f8ac815f0a0 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8aaffff640 1 -- 172.21.15.131:0/218871561 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0004260 con 0x7f8ac815f0a0 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8acdecc640 1 -- 172.21.15.131:0/218871561 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac8162d90 con 0x7f8ac815f0a0 2024-09-17T02:40:53.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8acdecc640 1 -- 172.21.15.131:0/218871561 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac8165a70 con 0x7f8ac815f0a0 2024-09-17T02:40:53.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8acdecc640 1 -- 172.21.15.131:0/218871561 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac8165cd0 con 0x7f8ac815f0a0 2024-09-17T02:40:53.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.387+0000 7f8aaffff640 1 -- 172.21.15.131:0/218871561 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8ac00be060 con 0x7f8ac815f0a0 2024-09-17T02:40:53.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.391+0000 7f8aaffff640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ab4073200 0x7f8ab4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:53.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.391+0000 7f8aaffff640 1 -- 172.21.15.131:0/218871561 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6007+0+0 (secure 0 0 0) 0x7f8ac012e020 con 0x7f8ac815f0a0 2024-09-17T02:40:53.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.391+0000 7f8ac77fe640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ab4073200 0x7f8ab4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:53.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.391+0000 7f8ac77fe640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ab4073200 0x7f8ab4075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ac806d6f0 tx=0x7f8ab8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:53.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.395+0000 7f8aaffff640 1 -- 172.21.15.131:0/218871561 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8ac0016990 con 0x7f8ac815f0a0 2024-09-17T02:40:53.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:53.563+0000 7f8acdecc640 1 -- 172.21.15.131:0/218871561 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8ac8162840 con 0x7f8ac815f0a0 2024-09-17T02:40:55.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.263+0000 7f8aaffff640 1 -- 172.21.15.131:0/218871561 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v24) ==== 228+0+0 (secure 0 0 0) 0x7f8ac00f8fa0 con 0x7f8ac815f0a0 2024-09-17T02:40:55.266 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data_ec' created 2024-09-17T02:40:55.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 -- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ab4073200 msgr2=0x7f8ab4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:55.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ab4073200 0x7f8ab4075710 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ac806d6f0 tx=0x7f8ab8009040 comp rx=0 tx=0).stop 2024-09-17T02:40:55.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 -- 172.21.15.131:0/218871561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac815f0a0 msgr2=0x7f8ac815e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:55.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac815f0a0 0x7f8ac815e710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8ac001b450 tx=0x7f8ac00092c0 comp rx=0 tx=0).stop 2024-09-17T02:40:55.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 -- 172.21.15.131:0/218871561 shutdown_connections 2024-09-17T02:40:55.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ab4073200 0x7f8ab4075710 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:55.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8ac815f0a0 0x7f8ac815e710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:55.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8ac8152600 0x7f8ac815e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:55.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 --2- 172.21.15.131:0/218871561 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8ac80b30a0 0x7f8ac8162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:55.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 -- 172.21.15.131:0/218871561 >> 172.21.15.131:0/218871561 conn(0x7f8ac806eea0 msgr2=0x7f8ac815d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:55.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 -- 172.21.15.131:0/218871561 shutdown_connections 2024-09-17T02:40:55.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.267+0000 7f8acdecc640 1 -- 172.21.15.131:0/218871561 wait complete. 2024-09-17T02:40:55.301 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-17T02:40:55.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 -- 172.21.15.131:0/2747793503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87bc15c9d0 msgr2=0x7f87bc15cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:55.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 --2- 172.21.15.131:0/2747793503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87bc15c9d0 0x7f87bc15cde0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87bc070aa0 tx=0x7f87b401ccc0 comp rx=0 tx=0).stop 2024-09-17T02:40:55.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 -- 172.21.15.131:0/2747793503 shutdown_connections 2024-09-17T02:40:55.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 --2- 172.21.15.131:0/2747793503 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87bc15c9d0 0x7f87bc15cde0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:55.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 --2- 172.21.15.131:0/2747793503 >> v2:172.21.15.189:3300/0 conn(0x7f87bc15ffc0 0x7f87bc1603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:55.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 --2- 172.21.15.131:0/2747793503 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87bc0cc870 0x7f87bc0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:55.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 -- 172.21.15.131:0/2747793503 >> 172.21.15.131:0/2747793503 conn(0x7f87bc0d05a0 msgr2=0x7f87bc0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:55.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 -- 172.21.15.131:0/2747793503 shutdown_connections 2024-09-17T02:40:55.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 -- 172.21.15.131:0/2747793503 wait complete. 2024-09-17T02:40:55.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 Processor -- start 2024-09-17T02:40:55.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 -- start start 2024-09-17T02:40:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87bc0cc870 0x7f87bc13cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87bc15c9d0 0x7f87bc1f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87bc15ffc0 0x7f87bc1f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc070040 con 0x7f87bc15ffc0 2024-09-17T02:40:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc06fec0 con 0x7f87bc15c9d0 2024-09-17T02:40:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c41aa640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f87bc0701c0 con 0x7f87bc0cc870 2024-09-17T02:40:55.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c2747640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87bc15ffc0 0x7f87bc1f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:55.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c2747640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87bc15ffc0 0x7f87bc1f9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41380/0 (socket says 172.21.15.131:41380) 2024-09-17T02:40:55.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.459+0000 7f87c2747640 1 -- 172.21.15.131:0/1869377536 learned_addr learned my addr 172.21.15.131:0/1869377536 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:55.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c1f46640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87bc0cc870 0x7f87bc13cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:55.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c1745640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87bc15c9d0 0x7f87bc1f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:55.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c2747640 1 -- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87bc0cc870 msgr2=0x7f87bc13cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:55.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c2747640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87bc0cc870 0x7f87bc13cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:55.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c2747640 1 -- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87bc15c9d0 msgr2=0x7f87bc1f9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:55.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c2747640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87bc15c9d0 0x7f87bc1f9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:55.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c2747640 1 -- 172.21.15.131:0/1869377536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87bc1583b0 con 0x7f87bc15ffc0 2024-09-17T02:40:55.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c2747640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87bc15ffc0 0x7f87bc1f9e90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87b401b370 tx=0x7f87b40a5b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:55.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87b27fc640 1 -- 172.21.15.131:0/1869377536 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b40bb070 con 0x7f87bc15ffc0 2024-09-17T02:40:55.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87b27fc640 1 -- 172.21.15.131:0/1869377536 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87b4004e00 con 0x7f87bc15ffc0 2024-09-17T02:40:55.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c41aa640 1 -- 172.21.15.131:0/1869377536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87bc158690 con 0x7f87bc15ffc0 2024-09-17T02:40:55.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87b27fc640 1 -- 172.21.15.131:0/1869377536 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b40b2040 con 0x7f87bc15ffc0 2024-09-17T02:40:55.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c41aa640 1 -- 172.21.15.131:0/1869377536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87bc158c50 con 0x7f87bc15ffc0 2024-09-17T02:40:55.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c41aa640 1 -- 172.21.15.131:0/1869377536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87bc1fa760 con 0x7f87bc15ffc0 2024-09-17T02:40:55.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87b27fc640 1 -- 172.21.15.131:0/1869377536 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f87b40be070 con 0x7f87bc15ffc0 2024-09-17T02:40:55.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87b27fc640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87a0073200 0x7f87a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:55.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87b27fc640 1 -- 172.21.15.131:0/1869377536 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f87b412e350 con 0x7f87bc15ffc0 2024-09-17T02:40:55.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.463+0000 7f87c1f46640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87a0073200 0x7f87a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:55.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.467+0000 7f87c1f46640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87a0073200 0x7f87a0075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f87b8005fd0 tx=0x7f87b8005d10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:55.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.467+0000 7f87b27fc640 1 -- 172.21.15.131:0/1869377536 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f87b4016970 con 0x7f87bc15ffc0 2024-09-17T02:40:55.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:55.639+0000 7f87c41aa640 1 -- 172.21.15.131:0/1869377536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f87bc1f85f0 con 0x7f87bc15ffc0 2024-09-17T02:40:56.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.271+0000 7f87b27fc640 1 -- 172.21.15.131:0/1869377536 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v25) ==== 169+0+0 (secure 0 0 0) 0x7f87b40f9170 con 0x7f87bc15ffc0 2024-09-17T02:40:56.275 INFO:teuthology.orchestra.run.smithi131.stderr:set pool 4 allow_ec_overwrites to true 2024-09-17T02:40:56.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.279+0000 7f87c41aa640 1 -- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87a0073200 msgr2=0x7f87a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:56.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.279+0000 7f87c41aa640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87a0073200 0x7f87a0075710 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f87b8005fd0 tx=0x7f87b8005d10 comp rx=0 tx=0).stop 2024-09-17T02:40:56.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.279+0000 7f87c41aa640 1 -- 172.21.15.131:0/1869377536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87bc15ffc0 msgr2=0x7f87bc1f9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:56.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.279+0000 7f87c41aa640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87bc15ffc0 0x7f87bc1f9e90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87b401b370 tx=0x7f87b40a5b30 comp rx=0 tx=0).stop 2024-09-17T02:40:56.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.287+0000 7f87c41aa640 1 -- 172.21.15.131:0/1869377536 shutdown_connections 2024-09-17T02:40:56.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.287+0000 7f87c41aa640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87a0073200 0x7f87a0075710 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:56.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.287+0000 7f87c41aa640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87bc15ffc0 0x7f87bc1f9e90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:56.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.287+0000 7f87c41aa640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87bc15c9d0 0x7f87bc1f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:56.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.287+0000 7f87c41aa640 1 --2- 172.21.15.131:0/1869377536 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87bc0cc870 0x7f87bc13cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:56.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.287+0000 7f87c41aa640 1 -- 172.21.15.131:0/1869377536 >> 172.21.15.131:0/1869377536 conn(0x7f87bc0d05a0 msgr2=0x7f87bc140630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:56.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.295+0000 7f87c41aa640 1 -- 172.21.15.131:0/1869377536 shutdown_connections 2024-09-17T02:40:56.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.295+0000 7f87c41aa640 1 -- 172.21.15.131:0/1869377536 wait complete. 2024-09-17T02:40:56.333 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-17T02:40:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.487+0000 7f74a8008640 1 -- 172.21.15.131:0/4163868558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f748c004d50 msgr2=0x7f748c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:56.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.487+0000 7f74a8008640 1 --2- 172.21.15.131:0/4163868558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f748c004d50 0x7f748c005200 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7498001d60 tx=0x7f749801c940 comp rx=0 tx=0).stop 2024-09-17T02:40:56.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.487+0000 7f74a65a5640 1 -- 172.21.15.131:0/4163868558 reap_dead start 2024-09-17T02:40:56.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.487+0000 7f74a8008640 1 -- 172.21.15.131:0/4163868558 shutdown_connections 2024-09-17T02:40:56.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.487+0000 7f74a8008640 1 -- 172.21.15.131:0/4163868558 >> 172.21.15.131:0/4163868558 conn(0x7f74a006eea0 msgr2=0x7f74a006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:56.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.487+0000 7f74a8008640 1 -- 172.21.15.131:0/4163868558 shutdown_connections 2024-09-17T02:40:56.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.487+0000 7f74a8008640 1 -- 172.21.15.131:0/4163868558 wait complete. 2024-09-17T02:40:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a8008640 1 Processor -- start 2024-09-17T02:40:56.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a8008640 1 -- start start 2024-09-17T02:40:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a8008640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f748c004d50 0x7f74a0156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:56.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a8008640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a0155740 0x7f74a00cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a8008640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a00ce300 0x7f74a014b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a8008640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f74a006dea0 con 0x7f74a0155740 2024-09-17T02:40:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a8008640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f74a00cf320 con 0x7f74a00ce300 2024-09-17T02:40:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a8008640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f74a006e020 con 0x7f748c004d50 2024-09-17T02:40:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a55a3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a0155740 0x7f74a00cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a55a3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a0155740 0x7f74a00cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41392/0 (socket says 172.21.15.131:41392) 2024-09-17T02:40:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a55a3640 1 -- 172.21.15.131:0/2316152242 learned_addr learned my addr 172.21.15.131:0/2316152242 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a5da4640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f748c004d50 0x7f74a0156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a65a5640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a00ce300 0x7f74a014b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:56.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a55a3640 1 -- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f748c004d50 msgr2=0x7f74a0156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a55a3640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f748c004d50 0x7f74a0156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a55a3640 1 -- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a00ce300 msgr2=0x7f74a014b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a55a3640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a00ce300 0x7f74a014b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a55a3640 1 -- 172.21.15.131:0/2316152242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a00ae430 con 0x7f74a0155740 2024-09-17T02:40:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a55a3640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a0155740 0x7f74a00cddc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f749000d9a0 tx=0x7f749000de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:56.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74967fc640 1 -- 172.21.15.131:0/2316152242 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749000b830 con 0x7f74a0155740 2024-09-17T02:40:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74967fc640 1 -- 172.21.15.131:0/2316152242 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f749000bac0 con 0x7f74a0155740 2024-09-17T02:40:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a8008640 1 -- 172.21.15.131:0/2316152242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a01e7020 con 0x7f74a0155740 2024-09-17T02:40:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74967fc640 1 -- 172.21.15.131:0/2316152242 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490003600 con 0x7f74a0155740 2024-09-17T02:40:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.491+0000 7f74a8008640 1 -- 172.21.15.131:0/2316152242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a00ae710 con 0x7f74a0155740 2024-09-17T02:40:56.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.495+0000 7f74a8008640 1 -- 172.21.15.131:0/2316152242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a0075b80 con 0x7f74a0155740 2024-09-17T02:40:56.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.495+0000 7f74967fc640 1 -- 172.21.15.131:0/2316152242 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f74900037a0 con 0x7f74a0155740 2024-09-17T02:40:56.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.495+0000 7f74967fc640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7484073230 0x7f7484075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:56.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.495+0000 7f74967fc640 1 -- 172.21.15.131:0/2316152242 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7f7490094bd0 con 0x7f74a0155740 2024-09-17T02:40:56.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.495+0000 7f74a5da4640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7484073230 0x7f7484075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:56.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.495+0000 7f74a5da4640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7484073230 0x7f7484075740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f749c009ab0 tx=0x7f749c0091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:56.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.499+0000 7f74967fc640 1 -- 172.21.15.131:0/2316152242 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f74a006e7f0 con 0x7f74a0155740 2024-09-17T02:40:56.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:56.707+0000 7f74a8008640 1 -- 172.21.15.131:0/2316152242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f74a00af190 con 0x7f74a0155740 2024-09-17T02:40:57.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.275+0000 7f74967fc640 1 -- 172.21.15.131:0/2316152242 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f749005f8d0 con 0x7f74a0155740 2024-09-17T02:40:57.278 INFO:teuthology.orchestra.run.smithi131.stderr:added data pool 4 to fsmap 2024-09-17T02:40:57.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 -- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7484073230 msgr2=0x7f7484075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7484073230 0x7f7484075740 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f749c009ab0 tx=0x7f749c0091e0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 -- 172.21.15.131:0/2316152242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a0155740 msgr2=0x7f74a00cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a0155740 0x7f74a00cddc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f749000d9a0 tx=0x7f749000de60 comp rx=0 tx=0).stop 2024-09-17T02:40:57.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 -- 172.21.15.131:0/2316152242 shutdown_connections 2024-09-17T02:40:57.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7484073230 0x7f7484075740 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74a00ce300 0x7f74a014b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74a0155740 0x7f74a00cddc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 --2- 172.21.15.131:0/2316152242 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f748c004d50 0x7f74a0156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 -- 172.21.15.131:0/2316152242 >> 172.21.15.131:0/2316152242 conn(0x7f74a006eea0 msgr2=0x7f74a00b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 -- 172.21.15.131:0/2316152242 shutdown_connections 2024-09-17T02:40:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.279+0000 7f74a8008640 1 -- 172.21.15.131:0/2316152242 wait complete. 2024-09-17T02:40:57.304 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:40:57.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.455+0000 7f32bd6d9640 1 -- 172.21.15.131:0/3115386441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b80cbb70 msgr2=0x7f32b8165b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.455+0000 7f32bd6d9640 1 --2- 172.21.15.131:0/3115386441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b80cbb70 0x7f32b8165b60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f32b000acd0 tx=0x7f32b001def0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.455+0000 7f32bd6d9640 1 -- 172.21.15.131:0/3115386441 shutdown_connections 2024-09-17T02:40:57.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.455+0000 7f32bd6d9640 1 --2- 172.21.15.131:0/3115386441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b80cbb70 0x7f32b8165b60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.455+0000 7f32bd6d9640 1 -- 172.21.15.131:0/3115386441 >> 172.21.15.131:0/3115386441 conn(0x7f32b806eea0 msgr2=0x7f32b806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:57.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.455+0000 7f32bd6d9640 1 -- 172.21.15.131:0/3115386441 shutdown_connections 2024-09-17T02:40:57.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.455+0000 7f32bd6d9640 1 -- 172.21.15.131:0/3115386441 wait complete. 2024-09-17T02:40:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.455+0000 7f32bd6d9640 1 Processor -- start 2024-09-17T02:40:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.455+0000 7f32bd6d9640 1 -- start start 2024-09-17T02:40:57.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32bd6d9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32b806dea0 0x7f32b80c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:57.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32bd6d9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b80cbb70 0x7f32b8152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:57.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32bd6d9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32b8155240 0x7f32b80c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32bd6d9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8168b20 con 0x7f32b80cbb70 2024-09-17T02:40:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32bd6d9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81689a0 con 0x7f32b8155240 2024-09-17T02:40:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32bd6d9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f32b8168ca0 con 0x7f32b806dea0 2024-09-17T02:40:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32b67fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b80cbb70 0x7f32b8152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32b67fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b80cbb70 0x7f32b8152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41410/0 (socket says 172.21.15.131:41410) 2024-09-17T02:40:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32b67fc640 1 -- 172.21.15.131:0/910871769 learned_addr learned my addr 172.21.15.131:0/910871769 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32b77fe640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32b8155240 0x7f32b80c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32b6ffd640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32b806dea0 0x7f32b80c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32b67fc640 1 -- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32b806dea0 msgr2=0x7f32b80c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32b67fc640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32b806dea0 0x7f32b80c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32b67fc640 1 -- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32b8155240 msgr2=0x7f32b80c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32b67fc640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32b8155240 0x7f32b80c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32b67fc640 1 -- 172.21.15.131:0/910871769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b80c8810 con 0x7f32b80cbb70 2024-09-17T02:40:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32b67fc640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b80cbb70 0x7f32b8152210 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c400 tx=0x7f32ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f329f7fe640 1 -- 172.21.15.131:0/910871769 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac011790 con 0x7f32b80cbb70 2024-09-17T02:40:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f329f7fe640 1 -- 172.21.15.131:0/910871769 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32ac017070 con 0x7f32b80cbb70 2024-09-17T02:40:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32bd6d9640 1 -- 172.21.15.131:0/910871769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b80c8af0 con 0x7f32b80cbb70 2024-09-17T02:40:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f329f7fe640 1 -- 172.21.15.131:0/910871769 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac011cc0 con 0x7f32b80cbb70 2024-09-17T02:40:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32bd6d9640 1 -- 172.21.15.131:0/910871769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b8156c90 con 0x7f32b80cbb70 2024-09-17T02:40:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f32bd6d9640 1 -- 172.21.15.131:0/910871769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b8073430 con 0x7f32b80cbb70 2024-09-17T02:40:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.459+0000 7f329f7fe640 1 -- 172.21.15.131:0/910871769 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f32ac01c460 con 0x7f32b80cbb70 2024-09-17T02:40:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.463+0000 7f329f7fe640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32a4073230 0x7f32a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.463+0000 7f329f7fe640 1 -- 172.21.15.131:0/910871769 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f32ac094330 con 0x7f32b80cbb70 2024-09-17T02:40:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.463+0000 7f32b6ffd640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32a4073230 0x7f32a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:57.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.463+0000 7f32b6ffd640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32a4073230 0x7f32a4075740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32a800a750 tx=0x7f32a8008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:57.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.467+0000 7f329f7fe640 1 -- 172.21.15.131:0/910871769 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f32ac05f010 con 0x7f32b80cbb70 2024-09-17T02:40:57.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 -- 172.21.15.131:0/910871769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32b806e9d0 con 0x7f32b80cbb70 2024-09-17T02:40:57.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f329f7fe640 1 -- 172.21.15.131:0/910871769 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5231 (secure 0 0 0) 0x7f32ac05e910 con 0x7f32b80cbb70 2024-09-17T02:40:57.664 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:40:57.664 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"btime":"2024-09-17T02:40:57:278670+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3035404898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3035404898},{"type":"v1","addr":"172.21.15.131:6835","nonce":3035404898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3147445846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3147445846},{"type":"v1","addr":"172.21.15.189:6835","nonce":3147445846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:40:51.611225+0000","modified":"2024-09-17T02:40:56.713598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/1582109479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1582109479},{"type":"v1","addr":"172.21.15.189:6837","nonce":1582109479}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T02:40:57.665 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-09-17T02:40:57.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 -- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32a4073230 msgr2=0x7f32a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32a4073230 0x7f32a4075740 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32a800a750 tx=0x7f32a8008040 comp rx=0 tx=0).stop 2024-09-17T02:40:57.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 -- 172.21.15.131:0/910871769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b80cbb70 msgr2=0x7f32b8152210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b80cbb70 0x7f32b8152210 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00c400 tx=0x7f32ac00c8c0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 -- 172.21.15.131:0/910871769 shutdown_connections 2024-09-17T02:40:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32a4073230 0x7f32a4075740 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32b8155240 0x7f32b80c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32b80cbb70 0x7f32b8152210 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 --2- 172.21.15.131:0/910871769 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32b806dea0 0x7f32b80c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 -- 172.21.15.131:0/910871769 >> 172.21.15.131:0/910871769 conn(0x7f32b806eea0 msgr2=0x7f32b806d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 -- 172.21.15.131:0/910871769 shutdown_connections 2024-09-17T02:40:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.663+0000 7f32bd6d9640 1 -- 172.21.15.131:0/910871769 wait complete. 2024-09-17T02:40:57.684 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:40:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.819+0000 7ff579700640 1 -- 172.21.15.131:0/674188983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff57406e2c0 msgr2=0x7ff5740c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.819+0000 7ff579700640 1 --2- 172.21.15.131:0/674188983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff57406e2c0 0x7ff5740c4580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff564009c80 tx=0x7ff56401cd30 comp rx=0 tx=0).stop 2024-09-17T02:40:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.819+0000 7ff579700640 1 -- 172.21.15.131:0/674188983 shutdown_connections 2024-09-17T02:40:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.819+0000 7ff579700640 1 --2- 172.21.15.131:0/674188983 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff5740c4ac0 0x7ff5740c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.819+0000 7ff579700640 1 --2- 172.21.15.131:0/674188983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff57406e2c0 0x7ff5740c4580 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.819+0000 7ff579700640 1 -- 172.21.15.131:0/674188983 >> 172.21.15.131:0/674188983 conn(0x7ff57406eea0 msgr2=0x7ff57406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.819+0000 7ff579700640 1 -- 172.21.15.131:0/674188983 shutdown_connections 2024-09-17T02:40:57.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.819+0000 7ff579700640 1 -- 172.21.15.131:0/674188983 wait complete. 2024-09-17T02:40:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff579700640 1 Processor -- start 2024-09-17T02:40:57.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff579700640 1 -- start start 2024-09-17T02:40:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff579700640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff57406e2c0 0x7ff574148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff579700640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff5740c4ac0 0x7ff574157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff579700640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5740cdbf0 0x7ff5740c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff579700640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff5740cf2b0 con 0x7ff57406e2c0 2024-09-17T02:40:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff579700640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff5740cf130 con 0x7ff5740cdbf0 2024-09-17T02:40:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff579700640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff5740cf430 con 0x7ff5740c4ac0 2024-09-17T02:40:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff572ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff57406e2c0 0x7ff574148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:57.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff572ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff57406e2c0 0x7ff574148730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41432/0 (socket says 172.21.15.131:41432) 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff572ffd640 1 -- 172.21.15.131:0/4218627385 learned_addr learned my addr 172.21.15.131:0/4218627385 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff5727fc640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff5740c4ac0 0x7ff574157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff572ffd640 1 -- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff5740c4ac0 msgr2=0x7ff574157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff572ffd640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff5740c4ac0 0x7ff574157180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff572ffd640 1 -- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5740cdbf0 msgr2=0x7ff5740c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff5737fe640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5740cdbf0 0x7ff5740c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff572ffd640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5740cdbf0 0x7ff5740c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff572ffd640 1 -- 172.21.15.131:0/4218627385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5740c7610 con 0x7ff57406e2c0 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff572ffd640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff57406e2c0 0x7ff574148730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff564009bd0 tx=0x7ff5640a5b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff55b7fe640 1 -- 172.21.15.131:0/4218627385 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5640bc070 con 0x7ff57406e2c0 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff55b7fe640 1 -- 172.21.15.131:0/4218627385 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff564004430 con 0x7ff57406e2c0 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff579700640 1 -- 172.21.15.131:0/4218627385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff574158ec0 con 0x7ff57406e2c0 2024-09-17T02:40:57.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff55b7fe640 1 -- 172.21.15.131:0/4218627385 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5640b2040 con 0x7ff57406e2c0 2024-09-17T02:40:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.823+0000 7ff579700640 1 -- 172.21.15.131:0/4218627385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff574148e20 con 0x7ff57406e2c0 2024-09-17T02:40:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.827+0000 7ff579700640 1 -- 172.21.15.131:0/4218627385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5740c4580 con 0x7ff57406e2c0 2024-09-17T02:40:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.827+0000 7ff55b7fe640 1 -- 172.21.15.131:0/4218627385 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff5640bf070 con 0x7ff57406e2c0 2024-09-17T02:40:57.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.827+0000 7ff55b7fe640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff560073200 0x7ff560075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.827+0000 7ff55b7fe640 1 -- 172.21.15.131:0/4218627385 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7ff56412e580 con 0x7ff57406e2c0 2024-09-17T02:40:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.827+0000 7ff5727fc640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff560073200 0x7ff560075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:57.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.827+0000 7ff5727fc640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff560073200 0x7ff560075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff5740ce6c0 tx=0x7ff568008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:57.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.831+0000 7ff55b7fe640 1 -- 172.21.15.131:0/4218627385 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff5640169b0 con 0x7ff57406e2c0 2024-09-17T02:40:57.847 INFO:tasks.ceph.mds.d.smithi189.stderr:2024-09-17T02:40:57.844+0000 7f2504c08640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:40:57.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.991+0000 7ff579700640 1 -- 172.21.15.131:0/4218627385 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff574073310 con 0x7ff57406e2c0 2024-09-17T02:40:57.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.991+0000 7ff55b7fe640 1 -- 172.21.15.131:0/4218627385 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18650 (secure 0 0 0) 0x7ff5640f9260 con 0x7ff57406e2c0 2024-09-17T02:40:57.995 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:40:57.995 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:56.713412+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:40:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 -- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff560073200 msgr2=0x7ff560075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff560073200 0x7ff560075710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff5740ce6c0 tx=0x7ff568008040 comp rx=0 tx=0).stop 2024-09-17T02:40:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 -- 172.21.15.131:0/4218627385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff57406e2c0 msgr2=0x7ff574148730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff57406e2c0 0x7ff574148730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff564009bd0 tx=0x7ff5640a5b00 comp rx=0 tx=0).stop 2024-09-17T02:40:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 -- 172.21.15.131:0/4218627385 shutdown_connections 2024-09-17T02:40:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff560073200 0x7ff560075710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff5740cdbf0 0x7ff5740c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff5740c4ac0 0x7ff574157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 --2- 172.21.15.131:0/4218627385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff57406e2c0 0x7ff574148730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 -- 172.21.15.131:0/4218627385 >> 172.21.15.131:0/4218627385 conn(0x7ff57406eea0 msgr2=0x7ff5740b4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:57.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 -- 172.21.15.131:0/4218627385 shutdown_connections 2024-09-17T02:40:57.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:57.995+0000 7ff579700640 1 -- 172.21.15.131:0/4218627385 wait complete. 2024-09-17T02:40:58.015 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:40:58.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.151+0000 7fa444945640 1 -- 172.21.15.131:0/1174591002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa440160070 msgr2=0x7fa440160480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.151+0000 7fa444945640 1 --2- 172.21.15.131:0/1174591002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa440160070 0x7fa440160480 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa428009840 tx=0x7fa42801c790 comp rx=0 tx=0).stop 2024-09-17T02:40:58.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.151+0000 7fa444945640 1 -- 172.21.15.131:0/1174591002 shutdown_connections 2024-09-17T02:40:58.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.151+0000 7fa444945640 1 --2- 172.21.15.131:0/1174591002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa440160070 0x7fa440160480 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.151+0000 7fa444945640 1 --2- 172.21.15.131:0/1174591002 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa440156c10 0x7fa4400cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.151+0000 7fa444945640 1 -- 172.21.15.131:0/1174591002 >> 172.21.15.131:0/1174591002 conn(0x7fa4400d07a0 msgr2=0x7fa4400d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:58.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.151+0000 7fa444945640 1 -- 172.21.15.131:0/1174591002 shutdown_connections 2024-09-17T02:40:58.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.151+0000 7fa444945640 1 -- 172.21.15.131:0/1174591002 wait complete. 2024-09-17T02:40:58.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa444945640 1 Processor -- start 2024-09-17T02:40:58.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa444945640 1 -- start start 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa444945640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa440156c10 0x7fa4401561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa444945640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa44015ca80 0x7fa4401459b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa444945640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa440160070 0x7fa44014ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa444945640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa44016a2e0 con 0x7fa440160070 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa444945640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa44016a160 con 0x7fa440156c10 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa444945640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa44016a460 con 0x7fa44015ca80 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa440160070 0x7fa44014ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa440160070 0x7fa44014ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41436/0 (socket says 172.21.15.131:41436) 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43ed9d640 1 -- 172.21.15.131:0/2152608132 learned_addr learned my addr 172.21.15.131:0/2152608132 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43dd9b640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa44015ca80 0x7fa4401459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43e59c640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa440156c10 0x7fa4401561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:58.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43ed9d640 1 -- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa44015ca80 msgr2=0x7fa4401459b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43ed9d640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa44015ca80 0x7fa4401459b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43ed9d640 1 -- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa440156c10 msgr2=0x7fa4401561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43ed9d640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa440156c10 0x7fa4401561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43ed9d640 1 -- 172.21.15.131:0/2152608132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4401341c0 con 0x7fa440160070 2024-09-17T02:40:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43e59c640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa440156c10 0x7fa4401561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa43ed9d640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa440160070 0x7fa44014ed00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa43800c9f0 tx=0x7fa43800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa436ffd640 1 -- 172.21.15.131:0/2152608132 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438016020 con 0x7fa440160070 2024-09-17T02:40:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa436ffd640 1 -- 172.21.15.131:0/2152608132 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa438012920 con 0x7fa440160070 2024-09-17T02:40:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa444945640 1 -- 172.21.15.131:0/2152608132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4401344a0 con 0x7fa440160070 2024-09-17T02:40:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa436ffd640 1 -- 172.21.15.131:0/2152608132 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438012d70 con 0x7fa440160070 2024-09-17T02:40:58.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa444945640 1 -- 172.21.15.131:0/2152608132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa440134a60 con 0x7fa440160070 2024-09-17T02:40:58.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.155+0000 7fa444945640 1 -- 172.21.15.131:0/2152608132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4400cdb50 con 0x7fa440160070 2024-09-17T02:40:58.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.159+0000 7fa436ffd640 1 -- 172.21.15.131:0/2152608132 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa438019020 con 0x7fa440160070 2024-09-17T02:40:58.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.159+0000 7fa436ffd640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa420073200 0x7fa420075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.159+0000 7fa436ffd640 1 -- 172.21.15.131:0/2152608132 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa438094660 con 0x7fa440160070 2024-09-17T02:40:58.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.159+0000 7fa43e59c640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa420073200 0x7fa420075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:58.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.159+0000 7fa43e59c640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa420073200 0x7fa420075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa4400b33b0 tx=0x7fa4300091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:58.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.163+0000 7fa436ffd640 1 -- 172.21.15.131:0/2152608132 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa438020020 con 0x7fa440160070 2024-09-17T02:40:58.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.335+0000 7fa444945640 1 -- 172.21.15.131:0/2152608132 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa44006f6a0 con 0x7fa440160070 2024-09-17T02:40:58.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.335+0000 7fa436ffd640 1 -- 172.21.15.131:0/2152608132 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18650 (secure 0 0 0) 0x7fa4401609c0 con 0x7fa440160070 2024-09-17T02:40:58.338 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:40:58.338 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:56.713412+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 -- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa420073200 msgr2=0x7fa420075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa420073200 0x7fa420075710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa4400b33b0 tx=0x7fa4300091e0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 -- 172.21.15.131:0/2152608132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa440160070 msgr2=0x7fa44014ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa440160070 0x7fa44014ed00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa43800c9f0 tx=0x7fa43800cae0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 -- 172.21.15.131:0/2152608132 shutdown_connections 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa420073200 0x7fa420075710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa440160070 0x7fa44014ed00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa44015ca80 0x7fa4401459b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 --2- 172.21.15.131:0/2152608132 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa440156c10 0x7fa4401561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 -- 172.21.15.131:0/2152608132 >> 172.21.15.131:0/2152608132 conn(0x7fa4400d07a0 msgr2=0x7fa440153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 -- 172.21.15.131:0/2152608132 shutdown_connections 2024-09-17T02:40:58.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.339+0000 7fa444945640 1 -- 172.21.15.131:0/2152608132 wait complete. 2024-09-17T02:40:58.376 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:40:58.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.531+0000 7f2fe3abe640 1 -- 172.21.15.131:0/488924164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fdc15c3e0 msgr2=0x7f2fdc0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.531+0000 7f2fe3abe640 1 --2- 172.21.15.131:0/488924164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fdc15c3e0 0x7f2fdc0cbaf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4009c80 tx=0x7f2fd401cda0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.531+0000 7f2fe3abe640 1 -- 172.21.15.131:0/488924164 shutdown_connections 2024-09-17T02:40:58.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.531+0000 7f2fe3abe640 1 --2- 172.21.15.131:0/488924164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fdc15c3e0 0x7f2fdc0cbaf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.531+0000 7f2fe3abe640 1 --2- 172.21.15.131:0/488924164 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2fdc06e5b0 0x7f2fdc15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.531+0000 7f2fe3abe640 1 -- 172.21.15.131:0/488924164 >> 172.21.15.131:0/488924164 conn(0x7f2fdc06eea0 msgr2=0x7f2fdc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:58.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.531+0000 7f2fe3abe640 1 -- 172.21.15.131:0/488924164 shutdown_connections 2024-09-17T02:40:58.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.531+0000 7f2fe3abe640 1 -- 172.21.15.131:0/488924164 wait complete. 2024-09-17T02:40:58.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.531+0000 7f2fe3abe640 1 Processor -- start 2024-09-17T02:40:58.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.531+0000 7f2fe3abe640 1 -- start start 2024-09-17T02:40:58.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe3abe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2fdc0ced00 0x7f2fdc0c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe3abe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fdc15c3e0 0x7f2fdc0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe3abe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fdc155780 0x7f2fdc0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe3abe640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc168860 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe3abe640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2fdc1686e0 con 0x7f2fdc155780 2024-09-17T02:40:58.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe3abe640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2fdc1689e0 con 0x7f2fdc0ced00 2024-09-17T02:40:58.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe1059640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fdc15c3e0 0x7f2fdc0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:58.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe1059640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fdc15c3e0 0x7f2fdc0b49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41448/0 (socket says 172.21.15.131:41448) 2024-09-17T02:40:58.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe1059640 1 -- 172.21.15.131:0/3581378598 learned_addr learned my addr 172.21.15.131:0/3581378598 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe185a640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2fdc0ced00 0x7f2fdc0c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe1059640 1 -- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2fdc0ced00 msgr2=0x7f2fdc0c28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe205b640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fdc155780 0x7f2fdc0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe1059640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2fdc0ced00 0x7f2fdc0c28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe1059640 1 -- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fdc155780 msgr2=0x7f2fdc0b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe1059640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fdc155780 0x7f2fdc0b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe1059640 1 -- 172.21.15.131:0/3581378598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fdc1fbb30 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe1059640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fdc15c3e0 0x7f2fdc0b49e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00c970 tx=0x7f2fcc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fd27fc640 1 -- 172.21.15.131:0/3581378598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc004220 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fd27fc640 1 -- 172.21.15.131:0/3581378598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fcc0079b0 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fd27fc640 1 -- 172.21.15.131:0/3581378598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0044b0 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe3abe640 1 -- 172.21.15.131:0/3581378598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fdc1fbfd0 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe3abe640 1 -- 172.21.15.131:0/3581378598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fdc1fc590 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fe3abe640 1 -- 172.21.15.131:0/3581378598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fdc15bea0 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.535+0000 7f2fd27fc640 1 -- 172.21.15.131:0/3581378598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2fcc003600 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.539+0000 7f2fd27fc640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2fc0073200 0x7f2fc0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.539+0000 7f2fd27fc640 1 -- 172.21.15.131:0/3581378598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2fcc094680 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.539+0000 7f2fe185a640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2fc0073200 0x7f2fc0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:58.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.539+0000 7f2fe185a640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2fc0073200 0x7f2fc0075710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800b810 tx=0x7f2fd8008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:58.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.543+0000 7f2fd27fc640 1 -- 172.21.15.131:0/3581378598 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2fcc014030 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.727+0000 7f2fe3abe640 1 -- 172.21.15.131:0/3581378598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fdc1fbd00 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fd27fc640 1 -- 172.21.15.131:0/3581378598 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5231 (secure 0 0 0) 0x7f2fcc05f360 con 0x7f2fdc15c3e0 2024-09-17T02:40:58.732 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:40:58.732 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":7,"btime":"2024-09-17T02:40:57:278670+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3035404898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3035404898},{"type":"v1","addr":"172.21.15.131:6835","nonce":3035404898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3147445846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3147445846},{"type":"v1","addr":"172.21.15.189:6835","nonce":3147445846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:40:51.611225+0000","modified":"2024-09-17T02:40:56.713598+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/1582109479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1582109479},{"type":"v1","addr":"172.21.15.189:6837","nonce":1582109479}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T02:40:58.732 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 7 2024-09-17T02:40:58.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 -- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2fc0073200 msgr2=0x7f2fc0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2fc0073200 0x7f2fc0075710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800b810 tx=0x7f2fd8008040 comp rx=0 tx=0).stop 2024-09-17T02:40:58.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 -- 172.21.15.131:0/3581378598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fdc15c3e0 msgr2=0x7f2fdc0b49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fdc15c3e0 0x7f2fdc0b49e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc00c970 tx=0x7f2fcc00ce30 comp rx=0 tx=0).stop 2024-09-17T02:40:58.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 -- 172.21.15.131:0/3581378598 shutdown_connections 2024-09-17T02:40:58.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2fc0073200 0x7f2fc0075710 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2fdc155780 0x7f2fdc0b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2fdc15c3e0 0x7f2fdc0b49e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 --2- 172.21.15.131:0/3581378598 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2fdc0ced00 0x7f2fdc0c28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 -- 172.21.15.131:0/3581378598 >> 172.21.15.131:0/3581378598 conn(0x7f2fdc06eea0 msgr2=0x7f2fdc074610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:58.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 -- 172.21.15.131:0/3581378598 shutdown_connections 2024-09-17T02:40:58.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.731+0000 7f2fe3abe640 1 -- 172.21.15.131:0/3581378598 wait complete. 2024-09-17T02:40:58.752 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:40:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.903+0000 7efd89b3f640 1 -- 172.21.15.131:0/1296684746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd8406dea0 msgr2=0x7efd8406e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.903+0000 7efd89b3f640 1 --2- 172.21.15.131:0/1296684746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd8406dea0 0x7efd8406e2b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efd78009840 tx=0x7efd7801c7f0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.903+0000 7efd89b3f640 1 -- 172.21.15.131:0/1296684746 shutdown_connections 2024-09-17T02:40:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.903+0000 7efd89b3f640 1 --2- 172.21.15.131:0/1296684746 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7efd840cab60 0x7efd840caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.903+0000 7efd89b3f640 1 --2- 172.21.15.131:0/1296684746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd8406dea0 0x7efd8406e2b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.903+0000 7efd89b3f640 1 -- 172.21.15.131:0/1296684746 >> 172.21.15.131:0/1296684746 conn(0x7efd8406eea0 msgr2=0x7efd8406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.903+0000 7efd89b3f640 1 -- 172.21.15.131:0/1296684746 shutdown_connections 2024-09-17T02:40:58.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.903+0000 7efd89b3f640 1 -- 172.21.15.131:0/1296684746 wait complete. 2024-09-17T02:40:58.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd89b3f640 1 Processor -- start 2024-09-17T02:40:58.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd89b3f640 1 -- start start 2024-09-17T02:40:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd89b3f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd8406dea0 0x7efd84148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd89b3f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd840cab60 0x7efd84159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd89b3f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7efd84155240 0x7efd8415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd89b3f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efd840cbec0 con 0x7efd8406dea0 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd89b3f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efd840cbd40 con 0x7efd840cab60 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd89b3f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7efd84168c80 con 0x7efd84155240 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd837fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd8406dea0 0x7efd84148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd837fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd8406dea0 0x7efd84148530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41466/0 (socket says 172.21.15.131:41466) 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd837fe640 1 -- 172.21.15.131:0/4193251713 learned_addr learned my addr 172.21.15.131:0/4193251713 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd82ffd640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd840cab60 0x7efd84159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd837fe640 1 -- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7efd84155240 msgr2=0x7efd8415a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd837fe640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7efd84155240 0x7efd8415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd837fe640 1 -- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd840cab60 msgr2=0x7efd84159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd837fe640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd840cab60 0x7efd84159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd837fe640 1 -- 172.21.15.131:0/4193251713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd84148e00 con 0x7efd8406dea0 2024-09-17T02:40:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd82ffd640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd840cab60 0x7efd84159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd837fe640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd8406dea0 0x7efd84148530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd7400c3f0 tx=0x7efd7400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd6bfff640 1 -- 172.21.15.131:0/4193251713 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74011790 con 0x7efd8406dea0 2024-09-17T02:40:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd6bfff640 1 -- 172.21.15.131:0/4193251713 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd74017070 con 0x7efd8406dea0 2024-09-17T02:40:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd89b3f640 1 -- 172.21.15.131:0/4193251713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd84148b20 con 0x7efd8406dea0 2024-09-17T02:40:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd6bfff640 1 -- 172.21.15.131:0/4193251713 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74011cc0 con 0x7efd8406dea0 2024-09-17T02:40:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.907+0000 7efd89b3f640 1 -- 172.21.15.131:0/4193251713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd84149dc0 con 0x7efd8406dea0 2024-09-17T02:40:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.911+0000 7efd89b3f640 1 -- 172.21.15.131:0/4193251713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8406e2b0 con 0x7efd8406dea0 2024-09-17T02:40:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.911+0000 7efd6bfff640 1 -- 172.21.15.131:0/4193251713 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efd7401c460 con 0x7efd8406dea0 2024-09-17T02:40:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.911+0000 7efd6bfff640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efd70073230 0x7efd70075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.911+0000 7efd6bfff640 1 -- 172.21.15.131:0/4193251713 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7efd74094270 con 0x7efd8406dea0 2024-09-17T02:40:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.911+0000 7efd82ffd640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efd70073230 0x7efd70075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.911+0000 7efd82ffd640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efd70073230 0x7efd70075740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efd780af000 tx=0x7efd7801b030 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:58.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:58.915+0000 7efd6bfff640 1 -- 172.21.15.131:0/4193251713 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7efd8406e7f0 con 0x7efd8406dea0 2024-09-17T02:40:59.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.087+0000 7efd89b3f640 1 -- 172.21.15.131:0/4193251713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd8406e9d0 con 0x7efd8406dea0 2024-09-17T02:40:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.087+0000 7efd6bfff640 1 -- 172.21.15.131:0/4193251713 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18650 (secure 0 0 0) 0x7efd7405efe0 con 0x7efd8406dea0 2024-09-17T02:40:59.090 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:40:59.091 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:56.713412+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:40:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 -- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efd70073230 msgr2=0x7efd70075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efd70073230 0x7efd70075740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efd780af000 tx=0x7efd7801b030 comp rx=0 tx=0).stop 2024-09-17T02:40:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 -- 172.21.15.131:0/4193251713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd8406dea0 msgr2=0x7efd84148530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd8406dea0 0x7efd84148530 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd7400c3f0 tx=0x7efd7400c8b0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 -- 172.21.15.131:0/4193251713 shutdown_connections 2024-09-17T02:40:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efd70073230 0x7efd70075740 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7efd84155240 0x7efd8415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efd840cab60 0x7efd84159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 --2- 172.21.15.131:0/4193251713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efd8406dea0 0x7efd84148530 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 -- 172.21.15.131:0/4193251713 >> 172.21.15.131:0/4193251713 conn(0x7efd8406eea0 msgr2=0x7efd840afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 -- 172.21.15.131:0/4193251713 shutdown_connections 2024-09-17T02:40:59.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.091+0000 7efd89b3f640 1 -- 172.21.15.131:0/4193251713 wait complete. 2024-09-17T02:40:59.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:40:51.611225+0000', 'modified': '2024-09-17T02:40:56.713598+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4384}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4384': {'gid': 4384, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.189:6837/1582109479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 1582109479}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 1582109479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4384, 'qdb_cluster': [4384]} 2024-09-17T02:40:59.515 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:40:59.516 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T02:40:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.679+0000 7f2e6a22f640 1 -- 172.21.15.131:0/2052359663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e641623f0 msgr2=0x7f2e64162800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.679+0000 7f2e6a22f640 1 --2- 172.21.15.131:0/2052359663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e641623f0 0x7f2e64162800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009c80 tx=0x7f2e5c01cac0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.679+0000 7f2e6a22f640 1 -- 172.21.15.131:0/2052359663 shutdown_connections 2024-09-17T02:40:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.679+0000 7f2e6a22f640 1 --2- 172.21.15.131:0/2052359663 >> v2:172.21.15.189:3300/0 conn(0x7f2e641614f0 0x7f2e64161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.679+0000 7f2e6a22f640 1 --2- 172.21.15.131:0/2052359663 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e641623f0 0x7f2e64162800 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.679+0000 7f2e6a22f640 1 -- 172.21.15.131:0/2052359663 >> 172.21.15.131:0/2052359663 conn(0x7f2e640d44b0 msgr2=0x7f2e640d48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:59.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.679+0000 7f2e6a22f640 1 -- 172.21.15.131:0/2052359663 shutdown_connections 2024-09-17T02:40:59.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.679+0000 7f2e6a22f640 1 -- 172.21.15.131:0/2052359663 wait complete. 2024-09-17T02:40:59.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e6a22f640 1 Processor -- start 2024-09-17T02:40:59.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e6a22f640 1 -- start start 2024-09-17T02:40:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e6a22f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e64155280 0x7f2e641513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e6a22f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e641614f0 0x7f2e64158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e6a22f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e641623f0 0x7f2e64154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e6a22f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e640d3ee0 con 0x7f2e641614f0 2024-09-17T02:40:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e6a22f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e640d3d60 con 0x7f2e641623f0 2024-09-17T02:40:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e6a22f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2e6416bf70 con 0x7f2e64155280 2024-09-17T02:40:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e62ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e641614f0 0x7f2e64158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e62ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e641614f0 0x7f2e64158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41476/0 (socket says 172.21.15.131:41476) 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e62ffd640 1 -- 172.21.15.131:0/1954197602 learned_addr learned my addr 172.21.15.131:0/1954197602 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e63fff640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e641623f0 0x7f2e64154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e64155280 0x7f2e641513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e62ffd640 1 -- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e64155280 msgr2=0x7f2e641513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e64155280 0x7f2e641513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e62ffd640 1 -- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e641623f0 msgr2=0x7f2e64154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e641623f0 0x7f2e64154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e62ffd640 1 -- 172.21.15.131:0/1954197602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e6415ad70 con 0x7f2e641614f0 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e64155280 0x7f2e641513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e641614f0 0x7f2e64158d50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800b410 tx=0x7f2e5800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e4bfff640 1 -- 172.21.15.131:0/1954197602 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58015030 con 0x7f2e641614f0 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e4bfff640 1 -- 172.21.15.131:0/1954197602 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e58004020 con 0x7f2e641614f0 2024-09-17T02:40:59.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e4bfff640 1 -- 172.21.15.131:0/1954197602 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e580044b0 con 0x7f2e641614f0 2024-09-17T02:40:59.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e6a22f640 1 -- 172.21.15.131:0/1954197602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e641fbe90 con 0x7f2e641614f0 2024-09-17T02:40:59.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.683+0000 7f2e6a22f640 1 -- 172.21.15.131:0/1954197602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e641f93d0 con 0x7f2e641614f0 2024-09-17T02:40:59.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.687+0000 7f2e4bfff640 1 -- 172.21.15.131:0/1954197602 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2e58018070 con 0x7f2e641614f0 2024-09-17T02:40:59.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.687+0000 7f2e4bfff640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e50073230 0x7f2e50075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.687+0000 7f2e4bfff640 1 -- 172.21.15.131:0/1954197602 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2e580940a0 con 0x7f2e641614f0 2024-09-17T02:40:59.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.687+0000 7f2e714fa000 1 Processor -- start 2024-09-17T02:40:59.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.687+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e50073230 0x7f2e50075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.687+0000 7f2e714fa000 1 -- start start 2024-09-17T02:40:59.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e50073230 0x7f2e50075740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0af000 tx=0x7f2e5c01b300 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:59.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4335f460 0x557d438a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557d43775410 con 0x557d4374f950 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557d436b2b40 con 0x557d43738ca0 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557d436a1f70 con 0x557d4335f460 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e62ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e62ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41482/0 (socket says 172.21.15.131:41482) 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e62ffd640 1 -- 172.21.15.131:0/1498200910 learned_addr learned my addr 172.21.15.131:0/1498200910 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1498200910 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4335f460 0x557d438a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e62ffd640 1 -- 172.21.15.131:0/1498200910 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4335f460 msgr2=0x557d438a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e63fff640 1 --2- 172.21.15.131:0/1498200910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/1498200910 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4335f460 0x557d438a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e62ffd640 1 -- 172.21.15.131:0/1498200910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 msgr2=0x557d438a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:40:59.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/1498200910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438a1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e62ffd640 1 -- 172.21.15.131:0/1498200910 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d436695b0 con 0x557d4374f950 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1498200910 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4335f460 0x557d438a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/1498200910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a0be0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2e5805e690 tx=0x7f2e5805ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d683f3b587e1984 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e497fa640 1 -- 172.21.15.131:0/1498200910 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e580631e0 con 0x557d4374f950 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e497fa640 1 -- 172.21.15.131:0/1498200910 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e58068090 con 0x557d4374f950 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e497fa640 1 -- 172.21.15.131:0/1498200910 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e58078e20 con 0x557d4374f950 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 -- 172.21.15.131:0/1498200910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 msgr2=0x557d438a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1498200910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a0be0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2e5805e690 tx=0x7f2e5805ee00 comp rx=0 tx=0).stop 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 -- 172.21.15.131:0/1498200910 shutdown_connections 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1498200910 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1498200910 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a0be0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1498200910 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4335f460 0x557d438a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 -- 172.21.15.131:0/1498200910 >> 172.21.15.131:0/1498200910 conn(0x557d43825d90 msgr2=0x557d43855690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:59.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 -- 172.21.15.131:0/1498200910 shutdown_connections 2024-09-17T02:40:59.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 -- 172.21.15.131:0/1498200910 wait complete. 2024-09-17T02:40:59.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 -- start start 2024-09-17T02:40:59.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.691+0000 7f2e714fa000 1 Processor -- start 2024-09-17T02:40:59.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4335f460 0x557d438a7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557d4363c5f0 con 0x557d4374f950 2024-09-17T02:40:59.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557d436a5d80 con 0x557d43738ca0 2024-09-17T02:40:59.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557d43648520 con 0x557d4335f460 2024-09-17T02:40:59.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 10 client.0 _ll_register_callbacks cb 0x557d4364c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:40:59.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e63fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41498/0 (socket says 172.21.15.131:41498) 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e63fff640 1 -- 172.21.15.131:0/1722778677 learned_addr learned my addr 172.21.15.131:0/1722778677 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4335f460 0x557d438a7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e63fff640 1 -- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4335f460 msgr2=0x557d438a7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e63fff640 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4335f460 0x557d438a7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e63fff640 1 -- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 msgr2=0x557d438a7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e63fff640 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438a7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e63fff640 1 -- 172.21.15.131:0/1722778677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d43640190 con 0x557d4374f950 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e63fff640 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a83a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e540103f0 tx=0x7f2e540108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3069a39a0d6c5f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e427fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:40:59.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e427fc640 1 -- 172.21.15.131:0/1722778677 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54012920 con 0x557d4374f950 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e427fc640 1 -- 172.21.15.131:0/1722778677 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2e54012ac0 con 0x557d4374f950 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 1 -- 172.21.15.131:0/1722778677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557d4363a110 con 0x557d4374f950 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e427fc640 1 -- 172.21.15.131:0/1722778677 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f2e54024050 con 0x557d4374f950 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 10 client.4488 fetch_fsmap learned FSMap version 7 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 1 -- 172.21.15.131:0/1722778677 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x557d43726f10 con 0x557d4374f950 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e427fc640 1 -- 172.21.15.131:0/1722778677 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557d4363a110 con 0x557d4374f950 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e427fc640 1 -- 172.21.15.131:0/1722778677 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f2e54027020 con 0x557d4374f950 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 10 client.4488 fetch_fsmap finished waiting for FSMap version 7 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 10 client.4488 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 20 client.4488 populate_metadata read hostname 'smithi131' 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x557d433447d0 0x557d438b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 4 client.4488 mds_command: new command op to 4384 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:40:59.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e714fa000 1 -- 172.21.15.131:0/1722778677 --> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d436b4a90 con 0x557d433447d0 2024-09-17T02:40:59.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x557d433447d0 0x557d438b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x557d433447d0 0x557d438b1d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed60ebf836c7e350 server_cookie=676783e7dc6d49ba in_seq=0 out_seq=0 2024-09-17T02:40:59.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.695+0000 7f2e427fc640 10 client.4488 ms_handle_connect on v2:172.21.15.189:6836/1582109479 2024-09-17T02:40:59.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e427fc640 1 -- 172.21.15.131:0/1722778677 <== mds.0 v2:172.21.15.189:6836/1582109479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557d436b4a90 con 0x557d433447d0 2024-09-17T02:40:59.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e427fc640 10 client.4488 handle_command_reply: tid=0 2024-09-17T02:40:59.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 1 client.4488 shutdown 2024-09-17T02:40:59.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 1 -- 172.21.15.131:0/1722778677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 msgr2=0x557d438a83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a83a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2e540103f0 tx=0x7f2e540108b0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 1 -- 172.21.15.131:0/1722778677 shutdown_connections 2024-09-17T02:40:59.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x557d433447d0 0x557d438b1d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4374f950 0x557d438a83a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e427fc640 0 client.4488 ms_handle_reset on v2:172.21.15.189:6836/1582109479 2024-09-17T02:40:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1722778677 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4335f460 0x557d438a7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 1 -- 172.21.15.131:0/1722778677 >> 172.21.15.131:0/1722778677 conn(0x557d43825d90 msgr2=0x557d4385c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 1 -- 172.21.15.131:0/1722778677 shutdown_connections 2024-09-17T02:40:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 1 -- 172.21.15.131:0/1722778677 wait complete. 2024-09-17T02:40:59.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.699+0000 7f2e714fa000 20 client.4488 trim_cache size 0 max 16384 2024-09-17T02:40:59.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.715+0000 7f2e714fa000 1 Processor -- start 2024-09-17T02:40:59.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.715+0000 7f2e714fa000 1 -- start start 2024-09-17T02:40:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d43912840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d43912d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4374f950 0x557d439132c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64206880 con 0x557d4335f460 2024-09-17T02:40:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e64205860 con 0x557d43738ca0 2024-09-17T02:40:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2e64204bf0 con 0x557d4374f950 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e637fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d43912840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e637fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d43912840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41500/0 (socket says 172.21.15.131:41500) 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e637fe640 1 -- 172.21.15.131:0/1657183380 learned_addr learned my addr 172.21.15.131:0/1657183380 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e637fe640 1 -- 172.21.15.131:0/1657183380 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4374f950 msgr2=0x557d439132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1657183380 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4374f950 0x557d439132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e637fe640 1 -- 172.21.15.131:0/1657183380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 msgr2=0x557d43912d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1657183380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d43912d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e637fe640 1 -- 172.21.15.131:0/1657183380 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d436fddb0 con 0x557d4335f460 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e637fe640 1 --2- 172.21.15.131:0/1657183380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d43912840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009bd0 tx=0x7f2e5c0b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729926ef7f338d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e48ff9640 1 -- 172.21.15.131:0/1657183380 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c0be030 con 0x557d4335f460 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e48ff9640 1 -- 172.21.15.131:0/1657183380 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e5c0d0070 con 0x557d4335f460 2024-09-17T02:40:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e48ff9640 1 -- 172.21.15.131:0/1657183380 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c0b2040 con 0x557d4335f460 2024-09-17T02:40:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 -- 172.21.15.131:0/1657183380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 msgr2=0x557d43912840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1657183380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d43912840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c009bd0 tx=0x7f2e5c0b4000 comp rx=0 tx=0).stop 2024-09-17T02:40:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 -- 172.21.15.131:0/1657183380 shutdown_connections 2024-09-17T02:40:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1657183380 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4374f950 0x557d439132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1657183380 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d43912d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 --2- 172.21.15.131:0/1657183380 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d43912840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 -- 172.21.15.131:0/1657183380 >> 172.21.15.131:0/1657183380 conn(0x557d43825d90 msgr2=0x557d43854cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 -- 172.21.15.131:0/1657183380 shutdown_connections 2024-09-17T02:40:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 -- 172.21.15.131:0/1657183380 wait complete. 2024-09-17T02:40:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 -- start start 2024-09-17T02:40:59.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.719+0000 7f2e714fa000 1 Processor -- start 2024-09-17T02:40:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d438d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4374f950 0x557d438d26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557d43726a80 con 0x557d4335f460 2024-09-17T02:40:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557d436a5090 con 0x557d43738ca0 2024-09-17T02:40:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557d436b1910 con 0x557d4374f950 2024-09-17T02:40:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 10 client.0 _ll_register_callbacks cb 0x7f2e5400e2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:40:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e637fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d438d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e637fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d438d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41506/0 (socket says 172.21.15.131:41506) 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e637fe640 1 -- 172.21.15.131:0/2202349417 learned_addr learned my addr 172.21.15.131:0/2202349417 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e63fff640 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4374f950 0x557d438d26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e637fe640 1 -- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4374f950 msgr2=0x557d438d26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e637fe640 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4374f950 0x557d438d26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e637fe640 1 -- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 msgr2=0x557d438d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e637fe640 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e637fe640 1 -- 172.21.15.131:0/2202349417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d43668ab0 con 0x557d4335f460 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e637fe640 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d438d1c30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0cd040 tx=0x7f2e5c0ae450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad94249d7b91ed44 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e427fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e427fc640 1 -- 172.21.15.131:0/2202349417 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5c0be050 con 0x557d4335f460 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e427fc640 1 -- 172.21.15.131:0/2202349417 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2e5c0ae900 con 0x557d4335f460 2024-09-17T02:40:59.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 -- 172.21.15.131:0/2202349417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e5c0d0070 con 0x557d4335f460 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e427fc640 1 -- 172.21.15.131:0/2202349417 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f2e5c0e3050 con 0x557d4335f460 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 10 client.4494 fetch_fsmap learned FSMap version 7 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 -- 172.21.15.131:0/2202349417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x557d43647070 con 0x557d4335f460 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e427fc640 1 -- 172.21.15.131:0/2202349417 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e5c0d0070 con 0x557d4335f460 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e427fc640 1 -- 172.21.15.131:0/2202349417 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 7) ==== 3510+0+0 (secure 0 0 0) 0x7f2e5c0e6020 con 0x557d4335f460 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 10 client.4494 fetch_fsmap finished waiting for FSMap version 7 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 10 client.4494 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 20 client.4494 populate_metadata read hostname 'smithi131' 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x557d43332830 0x557d438d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 4 client.4494 mds_command: new command op to 4384 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 -- 172.21.15.131:0/2202349417 --> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] -- command(tid 0: {"prefix": "status"}) -- 0x557d436b0540 con 0x557d43332830 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x557d43332830 0x557d438d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e427fc640 10 client.4494 ms_handle_connect on v2:172.21.15.189:6836/1582109479 2024-09-17T02:40:59.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e62ffd640 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x557d43332830 0x557d438d2bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1f64c031a1ad950 server_cookie=20be6ac767af5768 in_seq=0 out_seq=0 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e427fc640 1 -- 172.21.15.131:0/2202349417 <== mds.0 v2:172.21.15.189:6836/1582109479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x557d436b0540 con 0x557d43332830 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e427fc640 10 client.4494 handle_command_reply: tid=0 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 client.4494 shutdown 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 -- 172.21.15.131:0/2202349417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 msgr2=0x557d438d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d438d1c30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0cd040 tx=0x7f2e5c0ae450 comp rx=0 tx=0).stop 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 -- 172.21.15.131:0/2202349417 shutdown_connections 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x557d43332830 0x557d438d2bf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557d4374f950 0x557d438d26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e427fc640 0 client.4494 ms_handle_reset on v2:172.21.15.189:6836/1582109479 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557d43738ca0 0x557d438d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 --2- 172.21.15.131:0/2202349417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557d4335f460 0x557d438d1c30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 -- 172.21.15.131:0/2202349417 >> 172.21.15.131:0/2202349417 conn(0x557d43825d90 msgr2=0x557d43855690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:59.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 -- 172.21.15.131:0/2202349417 shutdown_connections 2024-09-17T02:40:59.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 1 -- 172.21.15.131:0/2202349417 wait complete. 2024-09-17T02:40:59.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e714fa000 20 client.4494 trim_cache size 0 max 16384 2024-09-17T02:40:59.731 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:40:59.731 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:40:59.731 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:40:59.731 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4384, 2024-09-17T02:40:59.731 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:40:59.731 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:40:59.731 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:40:59.731 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 7.5347377990000002, 2024-09-17T02:40:59.731 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 7, 2024-09-17T02:40:59.732 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 26, 2024-09-17T02:40:59.732 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 25, 2024-09-17T02:40:59.732 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 12.588156510999999 2024-09-17T02:40:59.732 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 -- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e50073230 msgr2=0x7f2e50075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e50073230 0x7f2e50075740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c0af000 tx=0x7f2e5c01b300 comp rx=0 tx=0).stop 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 -- 172.21.15.131:0/1954197602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e641614f0 msgr2=0x7f2e64158d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e641614f0 0x7f2e64158d50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2e5800b410 tx=0x7f2e5800b8d0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 -- 172.21.15.131:0/1954197602 shutdown_connections 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e50073230 0x7f2e50075740 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e641623f0 0x7f2e64154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e641614f0 0x7f2e64158d50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 --2- 172.21.15.131:0/1954197602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e64155280 0x7f2e641513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 -- 172.21.15.131:0/1954197602 >> 172.21.15.131:0/1954197602 conn(0x7f2e640d44b0 msgr2=0x7f2e640cd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 -- 172.21.15.131:0/1954197602 shutdown_connections 2024-09-17T02:40:59.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.723+0000 7f2e6a22f640 1 -- 172.21.15.131:0/1954197602 wait complete. 2024-09-17T02:40:59.745 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T02:40:59.745 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-17T02:40:59.745 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:40:59.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.887+0000 7f3d1583f640 1 -- 172.21.15.131:0/3749580540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1015f7d0 msgr2=0x7f3d10162410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.887+0000 7f3d1583f640 1 --2- 172.21.15.131:0/3749580540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1015f7d0 0x7f3d10162410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d100ce990 tx=0x7f3d0001c8a0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.887+0000 7f3d1583f640 1 -- 172.21.15.131:0/3749580540 shutdown_connections 2024-09-17T02:40:59.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.887+0000 7f3d1583f640 1 --2- 172.21.15.131:0/3749580540 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1015f7d0 0x7f3d10162410 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.887+0000 7f3d1583f640 1 --2- 172.21.15.131:0/3749580540 >> v2:172.21.15.189:3300/0 conn(0x7f3d10162950 0x7f3d10162d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.887+0000 7f3d1583f640 1 --2- 172.21.15.131:0/3749580540 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d100cf710 0x7f3d1015f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.887+0000 7f3d1583f640 1 -- 172.21.15.131:0/3749580540 >> 172.21.15.131:0/3749580540 conn(0x7f3d100d1b10 msgr2=0x7f3d100d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:40:59.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.887+0000 7f3d1583f640 1 -- 172.21.15.131:0/3749580540 shutdown_connections 2024-09-17T02:40:59.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.887+0000 7f3d1583f640 1 -- 172.21.15.131:0/3749580540 wait complete. 2024-09-17T02:40:59.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d1583f640 1 Processor -- start 2024-09-17T02:40:59.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d1583f640 1 -- start start 2024-09-17T02:40:59.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d1583f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d100cf710 0x7f3d1015b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d1583f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1015f7d0 0x7f3d10152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d1583f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10162950 0x7f3d101594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d1583f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3d100d2df0 con 0x7f3d1015f7d0 2024-09-17T02:40:59.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d1583f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d100d2c70 con 0x7f3d10162950 2024-09-17T02:40:59.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d1583f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3d100d2f70 con 0x7f3d100cf710 2024-09-17T02:40:59.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d0e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1015f7d0 0x7f3d10152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d0e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1015f7d0 0x7f3d10152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41530/0 (socket says 172.21.15.131:41530) 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d0e7fc640 1 -- 172.21.15.131:0/1527436691 learned_addr learned my addr 172.21.15.131:0/1527436691 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d0f7fe640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10162950 0x7f3d101594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d0effd640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d100cf710 0x7f3d1015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d0e7fc640 1 -- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d100cf710 msgr2=0x7f3d1015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d0e7fc640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d100cf710 0x7f3d1015b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d0e7fc640 1 -- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10162950 msgr2=0x7f3d101594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d0e7fc640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10162950 0x7f3d101594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d0e7fc640 1 -- 172.21.15.131:0/1527436691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d1015bef0 con 0x7f3d1015f7d0 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d0e7fc640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1015f7d0 0x7f3d10152290 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3d000b0040 tx=0x7f3d000a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3cf7fff640 1 -- 172.21.15.131:0/1527436691 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000bc020 con 0x7f3d1015f7d0 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3cf7fff640 1 -- 172.21.15.131:0/1527436691 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d00004430 con 0x7f3d1015f7d0 2024-09-17T02:40:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d1583f640 1 -- 172.21.15.131:0/1527436691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d10154340 con 0x7f3d1015f7d0 2024-09-17T02:40:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3cf7fff640 1 -- 172.21.15.131:0/1527436691 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d000b2040 con 0x7f3d1015f7d0 2024-09-17T02:40:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d1583f640 1 -- 172.21.15.131:0/1527436691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d10154900 con 0x7f3d1015f7d0 2024-09-17T02:40:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.891+0000 7f3d1583f640 1 -- 172.21.15.131:0/1527436691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d100d3420 con 0x7f3d1015f7d0 2024-09-17T02:40:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.895+0000 7f3cf7fff640 1 -- 172.21.15.131:0/1527436691 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3d000bf070 con 0x7f3d1015f7d0 2024-09-17T02:40:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.895+0000 7f3cf7fff640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3cfc073200 0x7f3cfc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:40:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.895+0000 7f3cf7fff640 1 -- 172.21.15.131:0/1527436691 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3d0012e780 con 0x7f3d1015f7d0 2024-09-17T02:40:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.895+0000 7f3d0effd640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3cfc073200 0x7f3cfc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:40:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.895+0000 7f3d0effd640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3cfc073200 0x7f3cfc075710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d04005c90 tx=0x7f3d04005c20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:40:59.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:40:59.899+0000 7f3cf7fff640 1 -- 172.21.15.131:0/1527436691 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3d000c6080 con 0x7f3d1015f7d0 2024-09-17T02:41:00.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 -- 172.21.15.131:0/1527436691 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3d1015bc80 con 0x7f3d1015f7d0 2024-09-17T02:41:00.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3cf7fff640 1 -- 172.21.15.131:0/1527436691 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f3d00016490 con 0x7f3d1015f7d0 2024-09-17T02:41:00.092 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-17T02:41:00.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 -- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3cfc073200 msgr2=0x7f3cfc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:00.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3cfc073200 0x7f3cfc075710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d04005c90 tx=0x7f3d04005c20 comp rx=0 tx=0).stop 2024-09-17T02:41:00.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 -- 172.21.15.131:0/1527436691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1015f7d0 msgr2=0x7f3d10152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:00.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1015f7d0 0x7f3d10152290 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3d000b0040 tx=0x7f3d000a5740 comp rx=0 tx=0).stop 2024-09-17T02:41:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 -- 172.21.15.131:0/1527436691 shutdown_connections 2024-09-17T02:41:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3cfc073200 0x7f3cfc075710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d10162950 0x7f3d101594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d1015f7d0 0x7f3d10152290 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 --2- 172.21.15.131:0/1527436691 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d100cf710 0x7f3d1015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 -- 172.21.15.131:0/1527436691 >> 172.21.15.131:0/1527436691 conn(0x7f3d100d1b10 msgr2=0x7f3d100b7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 -- 172.21.15.131:0/1527436691 shutdown_connections 2024-09-17T02:41:00.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00.091+0000 7f3d1583f640 1 -- 172.21.15.131:0/1527436691 wait complete. 2024-09-17T02:41:00.112 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-17T02:41:00.112 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:41:00.112 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:41:00.112 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-17T02:41:00.112 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-17T02:41:00.112 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-17T02:41:00.113 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:41:00.113 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:41:00.113 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-17T02:41:00.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:00.113 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:41:00.119 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:41:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:41:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:41:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:41:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:00.119 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:00.119 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:41:00.119 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42446sec preferred_lft 42446sec 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:00.120 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:41:00.121 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:00.121 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T02:41:00.121 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:00.121 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:00.122 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2024-09-17T02:41:00.122 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2024-09-17T02:41:00.122 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2024-09-17T02:41:00.122 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T02:41:00.122 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:00.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:00.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:00.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:00.252 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:41:00.303 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:41:00.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:00.304 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:41:00.371 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:41:00.371 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:41:00.371 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:41:00.371 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:41:00.371 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:41:00.371 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:41:00.371 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:41:00.371 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:00.372 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:00.372 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:00.372 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:41:00.372 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:00.372 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:00.372 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:41:00.372 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:41:00.373 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:41:00.373 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:00.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:00.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:00.490 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:41:00.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:00.536 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-09-17T02:41:00.584 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:00.584 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:00.584 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T02:41:00.584 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-17T02:41:00.584 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:00.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:00.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:00.700 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-17T02:41:00.700 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:00.701 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:00.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-17T02:41:00.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T02:41:00.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-17T02:41:00.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-17T02:41:00.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-17T02:41:00.701 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:00.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:00.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:00.799 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:00.799 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:00.799 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:41:00.799 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T02:41:00.799 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:00.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:00.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:00.901 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-17T02:41:00.901 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:41:00.902 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-17T02:41:00.947 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-17T02:41:00.947 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:41:00.947 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-17T02:41:00.995 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:41:01.057 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:01.104 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:41:01.105 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:41:01.105 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:41:01.105 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:41:01.105 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:41:01.105 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:41:01.105 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:41:01.105 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:41:01.105 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:41:01.105 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:41:01.105 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:41:01.105 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:41:01.106 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:01.106 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:41:01.106 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:41:01.106 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:41:01.106 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:41:01.106 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:41:01.106 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:01.106 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:41:01.106 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:01.106 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:41:01.106 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:41:01.107 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:41:01.107 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:41:01.107 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:41:01.107 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:41:01.107 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:41:01.107 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:41:01.107 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T02:41:01.107 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T02:41:01.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:01.108 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:41:01.160 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:41:01.160 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-17T02:41:01.209 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:41:01.223 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:01.273 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:41:01.273 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:41:01.274 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:41:01.274 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:41:01.274 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:41:01.274 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:41:01.274 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:41:01.275 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:41:01.275 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:41:01.275 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:41:01.275 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:41:01.275 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:41:01.275 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:01.276 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:41:01.276 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:41:01.276 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:41:01.276 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:41:01.276 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:41:01.276 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T02:41:01.277 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T02:41:01.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:01.278 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:41:01.322 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:2024-09-17T02:41:01.319+0000 7f63716dc580 -1 init, newargv = 0x559f39715250 newargc=15 2024-09-17T02:41:01.322 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[35670]: starting ceph client 2024-09-17T02:41:01.333 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[35670]: starting fuse 2024-09-17T02:41:02.420 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:41:02.437 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:02.490 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:41:02.490 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:41:02.490 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:41:02.490 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:41:02.491 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T02:41:02.492 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-17T02:41:02.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:02.494 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:41:02.540 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:41:02.541 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:41:02.541 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:41:02.541 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:41:02.541 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:41:02.541 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:41:02.541 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:41:02.541 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:02.541 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:41:02.541 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-17T02:41:02.541 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:41:02.542 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:41:02.543 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:41:02.543 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:02.543 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.admin")) 2024-09-17T02:41:02.543 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:41:02.665 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.admin.35670.asok 2024-09-17T02:41:02.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:02.676 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.35670.asok 2024-09-17T02:41:02.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:02.677 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.35670.asok status 2024-09-17T02:41:02.859 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "admin", 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "35670", 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4503, 2024-09-17T02:41:02.860 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 4503 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 563554091 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:41:02.861 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 563554091 2024-09-17T02:41:02.862 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:02.862 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.4503 192.168.0.1:0/563554091", 2024-09-17T02:41:02.862 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/563554091", 2024-09-17T02:41:02.862 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:41:02.862 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 7, 2024-09-17T02:41:02.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 26, 2024-09-17T02:41:02.862 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:41:02.862 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:41:02.862 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:41:02.862 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:41:02.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:02.875 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:41:02.876 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-17T02:41:02.882 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:41:02.882 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-17T02:41:02.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:02.882 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-17T02:41:02.937 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-17T02:41:03.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-17T02:41:03.070 INFO:teuthology.orchestra.run.smithi131.stdout:# file: . 2024-09-17T02:41:03.075 INFO:teuthology.orchestra.run.smithi131.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-17T02:41:03.075 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:03.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:03.078 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:41:03.078 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:03.084 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:41:03.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.291+0000 7fac3f494640 1 -- 172.21.15.131:0/3147674648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac380ced00 msgr2=0x7fac3806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:03.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.291+0000 7fac3f494640 1 --2- 172.21.15.131:0/3147674648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac380ced00 0x7fac3806dea0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fac28009840 tx=0x7fac2801c740 comp rx=0 tx=0).stop 2024-09-17T02:41:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.291+0000 7fac3f494640 1 -- 172.21.15.131:0/3147674648 shutdown_connections 2024-09-17T02:41:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.291+0000 7fac3f494640 1 --2- 172.21.15.131:0/3147674648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac380ced00 0x7fac3806dea0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.291+0000 7fac3f494640 1 -- 172.21.15.131:0/3147674648 >> 172.21.15.131:0/3147674648 conn(0x7fac3806eea0 msgr2=0x7fac3806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.291+0000 7fac3f494640 1 -- 172.21.15.131:0/3147674648 shutdown_connections 2024-09-17T02:41:03.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.291+0000 7fac3f494640 1 -- 172.21.15.131:0/3147674648 wait complete. 2024-09-17T02:41:03.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3f494640 1 Processor -- start 2024-09-17T02:41:03.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3f494640 1 -- start start 2024-09-17T02:41:03.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3f494640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fac3806e3e0 0x7fac380a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3f494640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac380ced00 0x7fac3815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3f494640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac3815c5b0 0x7fac38150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3f494640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fac38169d80 con 0x7fac380ced00 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3f494640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fac38169c00 con 0x7fac3815c5b0 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3f494640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fac38169f00 con 0x7fac3806e3e0 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3ca2f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac380ced00 0x7fac3815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3ca2f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac380ced00 0x7fac3815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58580/0 (socket says 172.21.15.131:58580) 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3ca2f640 1 -- 172.21.15.131:0/548723427 learned_addr learned my addr 172.21.15.131:0/548723427 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3da31640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac3815c5b0 0x7fac38150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3d230640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fac3806e3e0 0x7fac380a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3ca2f640 1 -- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fac3806e3e0 msgr2=0x7fac380a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:03.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3ca2f640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fac3806e3e0 0x7fac380a9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3ca2f640 1 -- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac3815c5b0 msgr2=0x7fac38150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3ca2f640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac3815c5b0 0x7fac38150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3ca2f640 1 -- 172.21.15.131:0/548723427 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac380a99b0 con 0x7fac380ced00 2024-09-17T02:41:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3ca2f640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac380ced00 0x7fac3815a680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fac3400c4f0 tx=0x7fac3400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac2dffb640 1 -- 172.21.15.131:0/548723427 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34016020 con 0x7fac380ced00 2024-09-17T02:41:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac2dffb640 1 -- 172.21.15.131:0/548723427 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac34012940 con 0x7fac380ced00 2024-09-17T02:41:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3f494640 1 -- 172.21.15.131:0/548723427 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac380a9ec0 con 0x7fac380ced00 2024-09-17T02:41:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac2dffb640 1 -- 172.21.15.131:0/548723427 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac34012d90 con 0x7fac380ced00 2024-09-17T02:41:03.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.295+0000 7fac3f494640 1 -- 172.21.15.131:0/548723427 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac380aa480 con 0x7fac380ced00 2024-09-17T02:41:03.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.299+0000 7fac3f494640 1 -- 172.21.15.131:0/548723427 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac38075b80 con 0x7fac380ced00 2024-09-17T02:41:03.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.299+0000 7fac2dffb640 1 -- 172.21.15.131:0/548723427 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fac34019020 con 0x7fac380ced00 2024-09-17T02:41:03.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.299+0000 7fac2dffb640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fac1c073200 0x7fac1c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:03.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.299+0000 7fac2dffb640 1 -- 172.21.15.131:0/548723427 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fac34094a70 con 0x7fac380ced00 2024-09-17T02:41:03.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.299+0000 7fac3d230640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fac1c073200 0x7fac1c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:03.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.299+0000 7fac3d230640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fac1c073200 0x7fac1c075710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fac280af000 tx=0x7fac2801af80 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:03.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.303+0000 7fac2dffb640 1 -- 172.21.15.131:0/548723427 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fac34020020 con 0x7fac380ced00 2024-09-17T02:41:03.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.483+0000 7fac3f494640 1 -- 172.21.15.131:0/548723427 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fac3806dea0 con 0x7fac380ced00 2024-09-17T02:41:03.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac2dffb640 1 -- 172.21.15.131:0/548723427 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7fac3806e7f0 con 0x7fac380ced00 2024-09-17T02:41:03.487 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:41:03.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 -- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fac1c073200 msgr2=0x7fac1c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:03.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fac1c073200 0x7fac1c075710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fac280af000 tx=0x7fac2801af80 comp rx=0 tx=0).stop 2024-09-17T02:41:03.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 -- 172.21.15.131:0/548723427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac380ced00 msgr2=0x7fac3815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:03.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac380ced00 0x7fac3815a680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fac3400c4f0 tx=0x7fac3400c9b0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 -- 172.21.15.131:0/548723427 shutdown_connections 2024-09-17T02:41:03.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fac1c073200 0x7fac1c075710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fac3815c5b0 0x7fac38150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fac380ced00 0x7fac3815a680 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 --2- 172.21.15.131:0/548723427 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fac3806e3e0 0x7fac380a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 -- 172.21.15.131:0/548723427 >> 172.21.15.131:0/548723427 conn(0x7fac3806eea0 msgr2=0x7fac380b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:03.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 -- 172.21.15.131:0/548723427 shutdown_connections 2024-09-17T02:41:03.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.487+0000 7fac3f494640 1 -- 172.21.15.131:0/548723427 wait complete. 2024-09-17T02:41:03.509 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:41:03.509 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:03.515 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:41:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.711+0000 7f1dc4b97640 1 -- 172.21.15.131:0/660615375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc015c5b0 msgr2=0x7f1dc00cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.711+0000 7f1dc4b97640 1 --2- 172.21.15.131:0/660615375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc015c5b0 0x7f1dc00cbaf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1db0009840 tx=0x7f1db001c6b0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.711+0000 7f1dc4b97640 1 -- 172.21.15.131:0/660615375 shutdown_connections 2024-09-17T02:41:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.711+0000 7f1dc4b97640 1 --2- 172.21.15.131:0/660615375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc015c5b0 0x7f1dc00cbaf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.711+0000 7f1dc4b97640 1 -- 172.21.15.131:0/660615375 >> 172.21.15.131:0/660615375 conn(0x7f1dc006eea0 msgr2=0x7f1dc006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.711+0000 7f1dc4b97640 1 -- 172.21.15.131:0/660615375 shutdown_connections 2024-09-17T02:41:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.711+0000 7f1dc4b97640 1 -- 172.21.15.131:0/660615375 wait complete. 2024-09-17T02:41:03.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dc4b97640 1 Processor -- start 2024-09-17T02:41:03.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dc4b97640 1 -- start start 2024-09-17T02:41:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dc4b97640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc00ced00 0x7f1dc0151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:03.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dc4b97640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dc015c5b0 0x7f1dc01504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dc4b97640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1dc014ad20 0x7f1dc0150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dc4b97640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc0169a80 con 0x7f1dc00ced00 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dc4b97640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc0169900 con 0x7f1dc015c5b0 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dc4b97640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc0169c00 con 0x7f1dc014ad20 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbe59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc00ced00 0x7f1dc0151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbe59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc00ced00 0x7f1dc0151a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58596/0 (socket says 172.21.15.131:58596) 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbe59c640 1 -- 172.21.15.131:0/3640829150 learned_addr learned my addr 172.21.15.131:0/3640829150 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbdd9b640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dc015c5b0 0x7f1dc01504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbe59c640 1 -- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1dc014ad20 msgr2=0x7f1dc0150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbed9d640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1dc014ad20 0x7f1dc0150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbe59c640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1dc014ad20 0x7f1dc0150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbe59c640 1 -- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dc015c5b0 msgr2=0x7f1dc01504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbe59c640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dc015c5b0 0x7f1dc01504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbe59c640 1 -- 172.21.15.131:0/3640829150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc0158290 con 0x7f1dc00ced00 2024-09-17T02:41:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbdd9b640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dc015c5b0 0x7f1dc01504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dbe59c640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc00ced00 0x7f1dc0151a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c760 tx=0x7f1db800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1db6ffd640 1 -- 172.21.15.131:0/3640829150 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db8017070 con 0x7f1dc00ced00 2024-09-17T02:41:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1db6ffd640 1 -- 172.21.15.131:0/3640829150 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1db800a640 con 0x7f1dc00ced00 2024-09-17T02:41:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dc4b97640 1 -- 172.21.15.131:0/3640829150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc0158790 con 0x7f1dc00ced00 2024-09-17T02:41:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1db6ffd640 1 -- 172.21.15.131:0/3640829150 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db800aa90 con 0x7f1dc00ced00 2024-09-17T02:41:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.715+0000 7f1dc4b97640 1 -- 172.21.15.131:0/3640829150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc0158d50 con 0x7f1dc00ced00 2024-09-17T02:41:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.719+0000 7f1dc4b97640 1 -- 172.21.15.131:0/3640829150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc0075b80 con 0x7f1dc00ced00 2024-09-17T02:41:03.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.719+0000 7f1db6ffd640 1 -- 172.21.15.131:0/3640829150 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1db801a070 con 0x7f1dc00ced00 2024-09-17T02:41:03.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.719+0000 7f1db6ffd640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1da0073200 0x7f1da0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:03.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.719+0000 7f1db6ffd640 1 -- 172.21.15.131:0/3640829150 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f1db8093c10 con 0x7f1dc00ced00 2024-09-17T02:41:03.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.719+0000 7f1dbdd9b640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1da0073200 0x7f1da0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:03.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.719+0000 7f1dbdd9b640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1da0073200 0x7f1da0075710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1dc014dd10 tx=0x7f1da80073c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:03.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.723+0000 7f1db6ffd640 1 -- 172.21.15.131:0/3640829150 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1dc015c9c0 con 0x7f1dc00ced00 2024-09-17T02:41:03.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.907+0000 7f1dc4b97640 1 -- 172.21.15.131:0/3640829150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1dc00cbaf0 con 0x7f1dc00ced00 2024-09-17T02:41:03.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.907+0000 7f1db6ffd640 1 -- 172.21.15.131:0/3640829150 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) ==== 80+0+0 (secure 0 0 0) 0x7f1db805e8e0 con 0x7f1dc00ced00 2024-09-17T02:41:03.910 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:41:03.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 -- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1da0073200 msgr2=0x7f1da0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:03.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1da0073200 0x7f1da0075710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1dc014dd10 tx=0x7f1da80073c0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 -- 172.21.15.131:0/3640829150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc00ced00 msgr2=0x7f1dc0151a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:03.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc00ced00 0x7f1dc0151a00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1db800c760 tx=0x7f1db800cc20 comp rx=0 tx=0).stop 2024-09-17T02:41:03.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 -- 172.21.15.131:0/3640829150 shutdown_connections 2024-09-17T02:41:03.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1da0073200 0x7f1da0075710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1dc014ad20 0x7f1dc0150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1dc015c5b0 0x7f1dc01504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 --2- 172.21.15.131:0/3640829150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1dc00ced00 0x7f1dc0151a00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:03.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 -- 172.21.15.131:0/3640829150 >> 172.21.15.131:0/3640829150 conn(0x7f1dc006eea0 msgr2=0x7f1dc0159c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:03.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 -- 172.21.15.131:0/3640829150 shutdown_connections 2024-09-17T02:41:03.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:03.911+0000 7f1dc4b97640 1 -- 172.21.15.131:0/3640829150 wait complete. 2024-09-17T02:41:03.932 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:41:03.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:03.932 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-17T02:41:03.953 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi131.stderr:ceph-fuse[35670]: fuse finished with error 0 and tester_r 0 2024-09-17T02:41:03.954 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:41:10.056 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:41:10.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:10.056 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-17T02:41:10.075 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:41:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.290+0000 7fa38f227640 1 -- 172.21.15.131:0/1608652401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3880ced00 msgr2=0x7fa38806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.290+0000 7fa38f227640 1 --2- 172.21.15.131:0/1608652401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3880ced00 0x7fa38806dea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa3880b61b0 tx=0x7fa37801c6e0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.290+0000 7fa38f227640 1 -- 172.21.15.131:0/1608652401 shutdown_connections 2024-09-17T02:41:10.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.290+0000 7fa38f227640 1 --2- 172.21.15.131:0/1608652401 >> v2:172.21.15.189:3300/0 conn(0x7fa38806e3e0 0x7fa38815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.290+0000 7fa38f227640 1 --2- 172.21.15.131:0/1608652401 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3880ced00 0x7fa38806dea0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.290+0000 7fa38f227640 1 -- 172.21.15.131:0/1608652401 >> 172.21.15.131:0/1608652401 conn(0x7fa38806eea0 msgr2=0x7fa38806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:10.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.290+0000 7fa38f227640 1 -- 172.21.15.131:0/1608652401 shutdown_connections 2024-09-17T02:41:10.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.290+0000 7fa38f227640 1 -- 172.21.15.131:0/1608652401 wait complete. 2024-09-17T02:41:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.290+0000 7fa38f227640 1 Processor -- start 2024-09-17T02:41:10.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.290+0000 7fa38f227640 1 -- start start 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38f227640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa38806e3e0 0x7fa3881f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38f227640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3880ced00 0x7fa388155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38f227640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa38815c3e0 0x7fa3881557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38f227640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa388168bf0 con 0x7fa38815c3e0 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38f227640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa3880cf300 con 0x7fa3880ced00 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38f227640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa388168d70 con 0x7fa38806e3e0 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38d7c4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa38815c3e0 0x7fa3881557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38d7c4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa38815c3e0 0x7fa3881557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58610/0 (socket says 172.21.15.131:58610) 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38d7c4640 1 -- 172.21.15.131:0/480578619 learned_addr learned my addr 172.21.15.131:0/480578619 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa387fff640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3880ced00 0x7fa388155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38cfc3640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa38806e3e0 0x7fa3881f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38d7c4640 1 -- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa38806e3e0 msgr2=0x7fa3881f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38d7c4640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa38806e3e0 0x7fa3881f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38d7c4640 1 -- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3880ced00 msgr2=0x7fa388155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38d7c4640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3880ced00 0x7fa388155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38d7c4640 1 -- 172.21.15.131:0/480578619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3881f71d0 con 0x7fa38815c3e0 2024-09-17T02:41:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38d7c4640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa38815c3e0 0x7fa3881557b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa38000b9e0 tx=0x7fa38000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa3857fa640 1 -- 172.21.15.131:0/480578619 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa380003b80 con 0x7fa38815c3e0 2024-09-17T02:41:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa3857fa640 1 -- 172.21.15.131:0/480578619 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa380003e10 con 0x7fa38815c3e0 2024-09-17T02:41:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38f227640 1 -- 172.21.15.131:0/480578619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3881f74b0 con 0x7fa38815c3e0 2024-09-17T02:41:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa3857fa640 1 -- 172.21.15.131:0/480578619 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38000cac0 con 0x7fa38815c3e0 2024-09-17T02:41:10.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38f227640 1 -- 172.21.15.131:0/480578619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3881f7af0 con 0x7fa38815c3e0 2024-09-17T02:41:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.294+0000 7fa38f227640 1 -- 172.21.15.131:0/480578619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa38815bea0 con 0x7fa38815c3e0 2024-09-17T02:41:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.298+0000 7fa3857fa640 1 -- 172.21.15.131:0/480578619 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa380006d10 con 0x7fa38815c3e0 2024-09-17T02:41:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.298+0000 7fa3857fa640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa374073230 0x7fa374075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.298+0000 7fa3857fa640 1 -- 172.21.15.131:0/480578619 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa380093ff0 con 0x7fa38815c3e0 2024-09-17T02:41:10.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.298+0000 7fa38cfc3640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa374073230 0x7fa374075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.298+0000 7fa38cfc3640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa374073230 0x7fa374075740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa3780ae000 tx=0x7fa37801af20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:10.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.302+0000 7fa3857fa640 1 -- 172.21.15.131:0/480578619 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa38005ecd0 con 0x7fa38815c3e0 2024-09-17T02:41:10.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.474+0000 7fa38f227640 1 -- 172.21.15.131:0/480578619 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3880af870 con 0x7fa38815c3e0 2024-09-17T02:41:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.474+0000 7fa3857fa640 1 -- 172.21.15.131:0/480578619 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18650 (secure 0 0 0) 0x7fa38005e5d0 con 0x7fa38815c3e0 2024-09-17T02:41:10.475 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:10.475 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:56.713412+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:41:10.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 -- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa374073230 msgr2=0x7fa374075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa374073230 0x7fa374075740 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa3780ae000 tx=0x7fa37801af20 comp rx=0 tx=0).stop 2024-09-17T02:41:10.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 -- 172.21.15.131:0/480578619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa38815c3e0 msgr2=0x7fa3881557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa38815c3e0 0x7fa3881557b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa38000b9e0 tx=0x7fa38000bea0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 -- 172.21.15.131:0/480578619 shutdown_connections 2024-09-17T02:41:10.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa374073230 0x7fa374075740 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa38815c3e0 0x7fa3881557b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3880ced00 0x7fa388155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 --2- 172.21.15.131:0/480578619 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa38806e3e0 0x7fa3881f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 -- 172.21.15.131:0/480578619 >> 172.21.15.131:0/480578619 conn(0x7fa38806eea0 msgr2=0x7fa38815b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:10.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 -- 172.21.15.131:0/480578619 shutdown_connections 2024-09-17T02:41:10.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.478+0000 7fa38f227640 1 -- 172.21.15.131:0/480578619 wait complete. 2024-09-17T02:41:10.498 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:41:10.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.642+0000 7f8c51e30640 1 -- 172.21.15.131:0/3354104712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c38004810 msgr2=0x7f8c38004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.642+0000 7f8c51e30640 1 --2- 172.21.15.131:0/3354104712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c38004810 0x7f8c38004c20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009b30 tx=0x7f8c3401d1f0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.642+0000 7f8c4bfff640 1 -- 172.21.15.131:0/3354104712 reap_dead start 2024-09-17T02:41:10.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.642+0000 7f8c51e30640 1 -- 172.21.15.131:0/3354104712 shutdown_connections 2024-09-17T02:41:10.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.642+0000 7f8c51e30640 1 -- 172.21.15.131:0/3354104712 >> 172.21.15.131:0/3354104712 conn(0x7f8c4c06eea0 msgr2=0x7f8c4c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:10.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.642+0000 7f8c51e30640 1 -- 172.21.15.131:0/3354104712 shutdown_connections 2024-09-17T02:41:10.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.642+0000 7f8c51e30640 1 -- 172.21.15.131:0/3354104712 wait complete. 2024-09-17T02:41:10.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c51e30640 1 Processor -- start 2024-09-17T02:41:10.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c51e30640 1 -- start start 2024-09-17T02:41:10.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c51e30640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c38004810 0x7f8c4c15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c51e30640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c4c0c8fe0 0x7f8c4c154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c51e30640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c4c0c7720 0x7f8c4c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c51e30640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c169880 con 0x7f8c4c0c7720 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c51e30640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c169700 con 0x7f8c38004810 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c51e30640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4c169a00 con 0x7f8c4c0c8fe0 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c4c0c7720 0x7f8c4c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c4c0c7720 0x7f8c4c156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58620/0 (socket says 172.21.15.131:58620) 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4bfff640 1 -- 172.21.15.131:0/2310063410 learned_addr learned my addr 172.21.15.131:0/2310063410 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4affd640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c4c0c8fe0 0x7f8c4c154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4b7fe640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c38004810 0x7f8c4c15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4bfff640 1 -- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c4c0c8fe0 msgr2=0x7f8c4c154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4bfff640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c4c0c8fe0 0x7f8c4c154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4bfff640 1 -- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c38004810 msgr2=0x7f8c4c15bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4bfff640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c38004810 0x7f8c4c15bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4bfff640 1 -- 172.21.15.131:0/2310063410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c15b1c0 con 0x7f8c4c0c7720 2024-09-17T02:41:10.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4b7fe640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c38004810 0x7f8c4c15bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:10.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c4bfff640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c4c0c7720 0x7f8c4c156e20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c075290 tx=0x7f8c4400fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:10.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c43fff640 1 -- 172.21.15.131:0/2310063410 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44003d90 con 0x7f8c4c0c7720 2024-09-17T02:41:10.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c43fff640 1 -- 172.21.15.131:0/2310063410 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c44017070 con 0x7f8c4c0c7720 2024-09-17T02:41:10.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c51e30640 1 -- 172.21.15.131:0/2310063410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4c15b4a0 con 0x7f8c4c0c7720 2024-09-17T02:41:10.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c43fff640 1 -- 172.21.15.131:0/2310063410 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400dcb0 con 0x7f8c4c0c7720 2024-09-17T02:41:10.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c51e30640 1 -- 172.21.15.131:0/2310063410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4c1578b0 con 0x7f8c4c0c7720 2024-09-17T02:41:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.646+0000 7f8c51e30640 1 -- 172.21.15.131:0/2310063410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c4c0ce880 con 0x7f8c4c0c7720 2024-09-17T02:41:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.650+0000 7f8c43fff640 1 -- 172.21.15.131:0/2310063410 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8c44004030 con 0x7f8c4c0c7720 2024-09-17T02:41:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.650+0000 7f8c43fff640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c2c073200 0x7f8c2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.650+0000 7f8c43fff640 1 -- 172.21.15.131:0/2310063410 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8c44094d00 con 0x7f8c4c0c7720 2024-09-17T02:41:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.650+0000 7f8c4b7fe640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c2c073200 0x7f8c2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.650+0000 7f8c4b7fe640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c2c073200 0x7f8c2c075710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c001870 tx=0x7f8c3c001c90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:10.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.654+0000 7f8c43fff640 1 -- 172.21.15.131:0/2310063410 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8c4405f420 con 0x7f8c4c0c7720 2024-09-17T02:41:10.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.814+0000 7f8c51e30640 1 -- 172.21.15.131:0/2310063410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c4c070c70 con 0x7f8c4c0c7720 2024-09-17T02:41:10.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.814+0000 7f8c43fff640 1 -- 172.21.15.131:0/2310063410 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18650 (secure 0 0 0) 0x7f8c44063e50 con 0x7f8c4c0c7720 2024-09-17T02:41:10.814 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:10.814 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:56.713412+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:41:10.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 -- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c2c073200 msgr2=0x7f8c2c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c2c073200 0x7f8c2c075710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c001870 tx=0x7f8c3c001c90 comp rx=0 tx=0).stop 2024-09-17T02:41:10.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 -- 172.21.15.131:0/2310063410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c4c0c7720 msgr2=0x7f8c4c156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c4c0c7720 0x7f8c4c156e20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c075290 tx=0x7f8c4400fed0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 -- 172.21.15.131:0/2310063410 shutdown_connections 2024-09-17T02:41:10.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c2c073200 0x7f8c2c075710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c4c0c7720 0x7f8c4c156e20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c4c0c8fe0 0x7f8c4c154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 --2- 172.21.15.131:0/2310063410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c38004810 0x7f8c4c15bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 -- 172.21.15.131:0/2310063410 >> 172.21.15.131:0/2310063410 conn(0x7f8c4c06eea0 msgr2=0x7f8c4c072e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:10.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 -- 172.21.15.131:0/2310063410 shutdown_connections 2024-09-17T02:41:10.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.818+0000 7f8c51e30640 1 -- 172.21.15.131:0/2310063410 wait complete. 2024-09-17T02:41:10.836 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T02:41:10.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.986+0000 7fcd7afd5640 1 -- 172.21.15.131:0/1463725974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7415c3e0 msgr2=0x7fcd740cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.986+0000 7fcd7afd5640 1 --2- 172.21.15.131:0/1463725974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7415c3e0 0x7fcd740cbaf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcd740b61b0 tx=0x7fcd6c01cd80 comp rx=0 tx=0).stop 2024-09-17T02:41:10.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.986+0000 7fcd7afd5640 1 -- 172.21.15.131:0/1463725974 shutdown_connections 2024-09-17T02:41:10.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.986+0000 7fcd7afd5640 1 --2- 172.21.15.131:0/1463725974 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd7415c3e0 0x7fcd740cbaf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.986+0000 7fcd7afd5640 1 --2- 172.21.15.131:0/1463725974 >> v2:172.21.15.189:3300/0 conn(0x7fcd7406e5b0 0x7fcd7415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.986+0000 7fcd7afd5640 1 -- 172.21.15.131:0/1463725974 >> 172.21.15.131:0/1463725974 conn(0x7fcd7406eea0 msgr2=0x7fcd7406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:10.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.986+0000 7fcd7afd5640 1 -- 172.21.15.131:0/1463725974 shutdown_connections 2024-09-17T02:41:10.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.986+0000 7fcd7afd5640 1 -- 172.21.15.131:0/1463725974 wait complete. 2024-09-17T02:41:10.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd7afd5640 1 Processor -- start 2024-09-17T02:41:10.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd7afd5640 1 -- start start 2024-09-17T02:41:10.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd7afd5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd740ced00 0x7fcd741fb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd7afd5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd7415c3e0 0x7fcd740b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd7afd5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd740aff60 0x7fcd740b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd7afd5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd74168750 con 0x7fcd740ced00 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd7afd5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd741685d0 con 0x7fcd7415c3e0 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd7afd5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcd741688d0 con 0x7fcd740aff60 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd78d71640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd740ced00 0x7fcd741fb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd78d71640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd740ced00 0x7fcd741fb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58638/0 (socket says 172.21.15.131:58638) 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd78d71640 1 -- 172.21.15.131:0/4128951491 learned_addr learned my addr 172.21.15.131:0/4128951491 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd79572640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd740aff60 0x7fcd740b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd78d71640 1 -- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd740aff60 msgr2=0x7fcd740b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd78d71640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd740aff60 0x7fcd740b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd73fff640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd7415c3e0 0x7fcd740b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd78d71640 1 -- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd7415c3e0 msgr2=0x7fcd740b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd78d71640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd7415c3e0 0x7fcd740b3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd78d71640 1 -- 172.21.15.131:0/4128951491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd740b08b0 con 0x7fcd740ced00 2024-09-17T02:41:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd78d71640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd740ced00 0x7fcd741fb5a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400b440 tx=0x7fcd6400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd717fa640 1 -- 172.21.15.131:0/4128951491 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64015020 con 0x7fcd740ced00 2024-09-17T02:41:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd717fa640 1 -- 172.21.15.131:0/4128951491 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd64011920 con 0x7fcd740ced00 2024-09-17T02:41:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd7afd5640 1 -- 172.21.15.131:0/4128951491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd74164ae0 con 0x7fcd740ced00 2024-09-17T02:41:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd717fa640 1 -- 172.21.15.131:0/4128951491 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd64011d70 con 0x7fcd740ced00 2024-09-17T02:41:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.990+0000 7fcd7afd5640 1 -- 172.21.15.131:0/4128951491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd741650a0 con 0x7fcd740ced00 2024-09-17T02:41:10.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.994+0000 7fcd7afd5640 1 -- 172.21.15.131:0/4128951491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7415bea0 con 0x7fcd740ced00 2024-09-17T02:41:10.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.994+0000 7fcd717fa640 1 -- 172.21.15.131:0/4128951491 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcd64018020 con 0x7fcd740ced00 2024-09-17T02:41:10.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.994+0000 7fcd717fa640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd60073200 0x7fcd60075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:10.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.994+0000 7fcd717fa640 1 -- 172.21.15.131:0/4128951491 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fcd640941e0 con 0x7fcd740ced00 2024-09-17T02:41:10.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.994+0000 7fcd73fff640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd60073200 0x7fcd60075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:10.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.994+0000 7fcd73fff640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd60073200 0x7fcd60075710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd680058d0 tx=0x7fcd68005bd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:10.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:10.998+0000 7fcd717fa640 1 -- 172.21.15.131:0/4128951491 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcd6401f030 con 0x7fcd740ced00 2024-09-17T02:41:11.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:11.206+0000 7fcd7afd5640 1 -- 172.21.15.131:0/4128951491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcd74159d80 con 0x7fcd740ced00 2024-09-17T02:41:12.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.210+0000 7fcd717fa640 1 -- 172.21.15.131:0/4128951491 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fcd740b08b0 con 0x7fcd740ced00 2024-09-17T02:41:12.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 -- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd60073200 msgr2=0x7fcd60075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:12.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd60073200 0x7fcd60075710 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd680058d0 tx=0x7fcd68005bd0 comp rx=0 tx=0).stop 2024-09-17T02:41:12.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 -- 172.21.15.131:0/4128951491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd740ced00 msgr2=0x7fcd741fb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:12.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd740ced00 0x7fcd741fb5a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcd6400b440 tx=0x7fcd6400b900 comp rx=0 tx=0).stop 2024-09-17T02:41:12.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 -- 172.21.15.131:0/4128951491 shutdown_connections 2024-09-17T02:41:12.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd60073200 0x7fcd60075710 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:12.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd740aff60 0x7fcd740b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:12.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd7415c3e0 0x7fcd740b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:12.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 --2- 172.21.15.131:0/4128951491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd740ced00 0x7fcd741fb5a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:12.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 -- 172.21.15.131:0/4128951491 >> 172.21.15.131:0/4128951491 conn(0x7fcd7406eea0 msgr2=0x7fcd740cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:12.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 -- 172.21.15.131:0/4128951491 shutdown_connections 2024-09-17T02:41:12.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.214+0000 7fcd7afd5640 1 -- 172.21.15.131:0/4128951491 wait complete. 2024-09-17T02:41:12.243 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-17T02:41:12.244 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T02:41:12.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.410+0000 7f7538069640 1 -- 172.21.15.131:0/638268963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f753006e2c0 msgr2=0x7f75300c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:12.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.410+0000 7f7538069640 1 --2- 172.21.15.131:0/638268963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f753006e2c0 0x7f75300c4580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7530074bb0 tx=0x7f752c01c830 comp rx=0 tx=0).stop 2024-09-17T02:41:12.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.410+0000 7f7538069640 1 -- 172.21.15.131:0/638268963 shutdown_connections 2024-09-17T02:41:12.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.410+0000 7f7538069640 1 --2- 172.21.15.131:0/638268963 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75300cdbf0 0x7f75300ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:12.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.410+0000 7f7538069640 1 --2- 172.21.15.131:0/638268963 >> v2:172.21.15.189:3300/0 conn(0x7f75300c4ac0 0x7f75300c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:12.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.410+0000 7f7538069640 1 --2- 172.21.15.131:0/638268963 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f753006e2c0 0x7f75300c4580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:12.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.410+0000 7f7538069640 1 -- 172.21.15.131:0/638268963 >> 172.21.15.131:0/638268963 conn(0x7f753006eea0 msgr2=0x7f753006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:12.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.410+0000 7f7538069640 1 -- 172.21.15.131:0/638268963 shutdown_connections 2024-09-17T02:41:12.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.410+0000 7f7538069640 1 -- 172.21.15.131:0/638268963 wait complete. 2024-09-17T02:41:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7538069640 1 Processor -- start 2024-09-17T02:41:12.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7538069640 1 -- start start 2024-09-17T02:41:12.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7538069640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f753006e2c0 0x7f753015cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7538069640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75300c4ac0 0x7f753015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7538069640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75300cdbf0 0x7f753015d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7538069640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f75300ced50 con 0x7f753006e2c0 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7538069640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75300ceed0 con 0x7f75300c4ac0 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7538069640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f75300cf050 con 0x7f75300cdbf0 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7535e05640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f753006e2c0 0x7f753015cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7535e05640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f753006e2c0 0x7f753015cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57584/0 (socket says 172.21.15.131:57584) 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7535e05640 1 -- 172.21.15.131:0/1983188487 learned_addr learned my addr 172.21.15.131:0/1983188487 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7536606640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75300cdbf0 0x7f753015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7535604640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75300c4ac0 0x7f753015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7535e05640 1 -- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75300cdbf0 msgr2=0x7f753015d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7535e05640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75300cdbf0 0x7f753015d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:12.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7535e05640 1 -- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75300c4ac0 msgr2=0x7f753015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:12.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7535e05640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75300c4ac0 0x7f753015d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:12.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7535e05640 1 -- 172.21.15.131:0/1983188487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753015e6c0 con 0x7f753006e2c0 2024-09-17T02:41:12.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7535e05640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f753006e2c0 0x7f753015cd80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f752c0b0040 tx=0x7f752c0a6f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:12.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f75267fc640 1 -- 172.21.15.131:0/1983188487 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c0041c0 con 0x7f753006e2c0 2024-09-17T02:41:12.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f75267fc640 1 -- 172.21.15.131:0/1983188487 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f752c004450 con 0x7f753006e2c0 2024-09-17T02:41:12.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f75267fc640 1 -- 172.21.15.131:0/1983188487 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c003860 con 0x7f753006e2c0 2024-09-17T02:41:12.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7538069640 1 -- 172.21.15.131:0/1983188487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753015fa10 con 0x7f753006e2c0 2024-09-17T02:41:12.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7538069640 1 -- 172.21.15.131:0/1983188487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753015dec0 con 0x7f753006e2c0 2024-09-17T02:41:12.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.414+0000 7f7538069640 1 -- 172.21.15.131:0/1983188487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75300c4ed0 con 0x7f753006e2c0 2024-09-17T02:41:12.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.418+0000 7f75267fc640 1 -- 172.21.15.131:0/1983188487 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f752c0c0030 con 0x7f753006e2c0 2024-09-17T02:41:12.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.418+0000 7f75267fc640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7514073200 0x7f7514075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:12.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.418+0000 7f75267fc640 1 -- 172.21.15.131:0/1983188487 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f752c12f0e0 con 0x7f753006e2c0 2024-09-17T02:41:12.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.418+0000 7f7535604640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7514073200 0x7f7514075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:12.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.418+0000 7f7535604640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7514073200 0x7f7514075710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f75300b60d0 tx=0x7f752000c3c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:12.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.422+0000 7f75267fc640 1 -- 172.21.15.131:0/1983188487 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f752c0c7060 con 0x7f753006e2c0 2024-09-17T02:41:12.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:12.630+0000 7f7538069640 1 -- 172.21.15.131:0/1983188487 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7530158900 con 0x7f753006e2c0 2024-09-17T02:41:13.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.218+0000 7f75267fc640 1 -- 172.21.15.131:0/1983188487 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f752c0164b0 con 0x7f753006e2c0 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.218+0000 7f7538069640 1 -- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7514073200 msgr2=0x7f7514075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.218+0000 7f7538069640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7514073200 0x7f7514075710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f75300b60d0 tx=0x7f752000c3c0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.218+0000 7f7538069640 1 -- 172.21.15.131:0/1983188487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f753006e2c0 msgr2=0x7f753015cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.218+0000 7f7538069640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f753006e2c0 0x7f753015cd80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f752c0b0040 tx=0x7f752c0a6f60 comp rx=0 tx=0).stop 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.222+0000 7f7538069640 1 -- 172.21.15.131:0/1983188487 shutdown_connections 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.222+0000 7f7538069640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7514073200 0x7f7514075710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.222+0000 7f7538069640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75300cdbf0 0x7f753015d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.222+0000 7f7538069640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75300c4ac0 0x7f753015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.222+0000 7f7538069640 1 --2- 172.21.15.131:0/1983188487 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f753006e2c0 0x7f753015cd80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.222+0000 7f7538069640 1 -- 172.21.15.131:0/1983188487 >> 172.21.15.131:0/1983188487 conn(0x7f753006eea0 msgr2=0x7f7530153a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.222+0000 7f7538069640 1 -- 172.21.15.131:0/1983188487 shutdown_connections 2024-09-17T02:41:13.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.222+0000 7f7538069640 1 -- 172.21.15.131:0/1983188487 wait complete. 2024-09-17T02:41:13.242 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T02:41:13.242 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:41:13.253 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:41:13.293 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:41:13.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.394+0000 7fbcacd50640 1 -- 172.21.15.131:0/709318375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca80ced00 msgr2=0x7fbca806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.394+0000 7fbcacd50640 1 --2- 172.21.15.131:0/709318375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca80ced00 0x7fbca806dea0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbc98009900 tx=0x7fbc9801c700 comp rx=0 tx=0).stop 2024-09-17T02:41:13.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.394+0000 7fbcacd50640 1 -- 172.21.15.131:0/709318375 shutdown_connections 2024-09-17T02:41:13.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.394+0000 7fbcacd50640 1 --2- 172.21.15.131:0/709318375 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbca806e3e0 0x7fbca815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.394+0000 7fbcacd50640 1 --2- 172.21.15.131:0/709318375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca80ced00 0x7fbca806dea0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.394+0000 7fbcacd50640 1 -- 172.21.15.131:0/709318375 >> 172.21.15.131:0/709318375 conn(0x7fbca806eea0 msgr2=0x7fbca806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:13.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.394+0000 7fbcacd50640 1 -- 172.21.15.131:0/709318375 shutdown_connections 2024-09-17T02:41:13.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.394+0000 7fbcacd50640 1 -- 172.21.15.131:0/709318375 wait complete. 2024-09-17T02:41:13.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbcacd50640 1 Processor -- start 2024-09-17T02:41:13.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbcacd50640 1 -- start start 2024-09-17T02:41:13.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbcacd50640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbca806e3e0 0x7fbca80ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:13.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbcacd50640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbca80ced00 0x7fbca8160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:13.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbcacd50640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca815c3e0 0x7fbca815dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:13.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbcacd50640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbca8168b60 con 0x7fbca815c3e0 2024-09-17T02:41:13.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbcacd50640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbca81689e0 con 0x7fbca80ced00 2024-09-17T02:41:13.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbcacd50640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fbca8168ce0 con 0x7fbca806e3e0 2024-09-17T02:41:13.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbca6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca815c3e0 0x7fbca815dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:13.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbca6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca815c3e0 0x7fbca815dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57606/0 (socket says 172.21.15.131:57606) 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbca6d9d640 1 -- 172.21.15.131:0/2695761552 learned_addr learned my addr 172.21.15.131:0/2695761552 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbca5d9b640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbca80ced00 0x7fbca8160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbca659c640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbca806e3e0 0x7fbca80ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbca6d9d640 1 -- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbca806e3e0 msgr2=0x7fbca80ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbca6d9d640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbca806e3e0 0x7fbca80ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbca6d9d640 1 -- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbca80ced00 msgr2=0x7fbca8160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbca6d9d640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbca80ced00 0x7fbca8160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbca6d9d640 1 -- 172.21.15.131:0/2695761552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca81f9a40 con 0x7fbca815c3e0 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbca6d9d640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca815c3e0 0x7fbca815dbb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbca000b9a0 tx=0x7fbca000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbc96ffd640 1 -- 172.21.15.131:0/2695761552 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca0019070 con 0x7fbca815c3e0 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbc96ffd640 1 -- 172.21.15.131:0/2695761552 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbca000cc90 con 0x7fbca815c3e0 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbc96ffd640 1 -- 172.21.15.131:0/2695761552 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca00146a0 con 0x7fbca815c3e0 2024-09-17T02:41:13.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbcacd50640 1 -- 172.21.15.131:0/2695761552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca80cac40 con 0x7fbca815c3e0 2024-09-17T02:41:13.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.398+0000 7fbcacd50640 1 -- 172.21.15.131:0/2695761552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca8160e00 con 0x7fbca815c3e0 2024-09-17T02:41:13.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.402+0000 7fbcacd50640 1 -- 172.21.15.131:0/2695761552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca8160890 con 0x7fbca815c3e0 2024-09-17T02:41:13.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.402+0000 7fbc96ffd640 1 -- 172.21.15.131:0/2695761552 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbca001c070 con 0x7fbca815c3e0 2024-09-17T02:41:13.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.402+0000 7fbc96ffd640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbc88073230 0x7fbc88075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:13.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.402+0000 7fbc96ffd640 1 -- 172.21.15.131:0/2695761552 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbca00943e0 con 0x7fbca815c3e0 2024-09-17T02:41:13.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.402+0000 7fbca659c640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbc88073230 0x7fbc88075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:13.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.402+0000 7fbca659c640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbc88073230 0x7fbc88075740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbc980ae000 tx=0x7fbc9801af40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:13.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.406+0000 7fbc96ffd640 1 -- 172.21.15.131:0/2695761552 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbca0023080 con 0x7fbca815c3e0 2024-09-17T02:41:13.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.610+0000 7fbcacd50640 1 -- 172.21.15.131:0/2695761552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbca815bea0 con 0x7fbca815c3e0 2024-09-17T02:41:13.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.610+0000 7fbc96ffd640 1 -- 172.21.15.131:0/2695761552 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7fbca005f230 con 0x7fbca815c3e0 2024-09-17T02:41:13.619 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:13.619 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"btime":"2024-09-17T02:41:13:211310+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3035404898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3035404898},{"type":"v1","addr":"172.21.15.131:6835","nonce":3035404898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3147445846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3147445846},{"type":"v1","addr":"172.21.15.189:6835","nonce":3147445846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:40:51.611225+0000","modified":"2024-09-17T02:41:12.632560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/1582109479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1582109479},{"type":"v1","addr":"172.21.15.189:6837","nonce":1582109479}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T02:41:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-09-17T02:41:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 -- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbc88073230 msgr2=0x7fbc88075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbc88073230 0x7fbc88075740 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbc980ae000 tx=0x7fbc9801af40 comp rx=0 tx=0).stop 2024-09-17T02:41:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 -- 172.21.15.131:0/2695761552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca815c3e0 msgr2=0x7fbca815dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca815c3e0 0x7fbca815dbb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbca000b9a0 tx=0x7fbca000be60 comp rx=0 tx=0).stop 2024-09-17T02:41:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 -- 172.21.15.131:0/2695761552 shutdown_connections 2024-09-17T02:41:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbc88073230 0x7fbc88075740 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbca815c3e0 0x7fbca815dbb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbca80ced00 0x7fbca8160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 --2- 172.21.15.131:0/2695761552 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbca806e3e0 0x7fbca80ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 -- 172.21.15.131:0/2695761552 >> 172.21.15.131:0/2695761552 conn(0x7fbca806eea0 msgr2=0x7fbca8154480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 -- 172.21.15.131:0/2695761552 shutdown_connections 2024-09-17T02:41:13.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.614+0000 7fbcacd50640 1 -- 172.21.15.131:0/2695761552 wait complete. 2024-09-17T02:41:13.632 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:41:13.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.774+0000 7f46a1b83640 1 -- 172.21.15.131:0/234806052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f469c0cf2c0 msgr2=0x7f469c0cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.774+0000 7f46a1b83640 1 --2- 172.21.15.131:0/234806052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f469c0cf2c0 0x7f469c0cf6d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f469c073920 tx=0x7f468c01ccf0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.774+0000 7f46a1b83640 1 -- 172.21.15.131:0/234806052 shutdown_connections 2024-09-17T02:41:13.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.774+0000 7f46a1b83640 1 --2- 172.21.15.131:0/234806052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f469c0cf2c0 0x7f469c0cf6d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.774+0000 7f46a1b83640 1 --2- 172.21.15.131:0/234806052 >> v2:172.21.15.189:3300/0 conn(0x7f469c070f80 0x7f469c0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.774+0000 7f46a1b83640 1 -- 172.21.15.131:0/234806052 >> 172.21.15.131:0/234806052 conn(0x7f469c0cdbb0 msgr2=0x7f469c0cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:13.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.774+0000 7f46a1b83640 1 -- 172.21.15.131:0/234806052 shutdown_connections 2024-09-17T02:41:13.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.774+0000 7f46a1b83640 1 -- 172.21.15.131:0/234806052 wait complete. 2024-09-17T02:41:13.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.774+0000 7f46a1b83640 1 Processor -- start 2024-09-17T02:41:13.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.774+0000 7f46a1b83640 1 -- start start 2024-09-17T02:41:13.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f46a1b83640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c06ffb0 0x7f469c14d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:13.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f46a1b83640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f469c070f80 0x7f469c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:13.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f46a1b83640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f469c0cf2c0 0x7f469c0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:13.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f46a1b83640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f469c168d50 con 0x7f469c0cf2c0 2024-09-17T02:41:13.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f46a1b83640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f469c168bd0 con 0x7f469c06ffb0 2024-09-17T02:41:13.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f46a1b83640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f469c168ed0 con 0x7f469c070f80 2024-09-17T02:41:13.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f469c0cf2c0 0x7f469c0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:13.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f469c0cf2c0 0x7f469c0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57622/0 (socket says 172.21.15.131:57622) 2024-09-17T02:41:13.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469bfff640 1 -- 172.21.15.131:0/977308497 learned_addr learned my addr 172.21.15.131:0/977308497 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:13.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469b7fe640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c06ffb0 0x7f469c14d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:13.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469affd640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f469c070f80 0x7f469c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:13.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469bfff640 1 -- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f469c070f80 msgr2=0x7f469c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469bfff640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f469c070f80 0x7f469c15dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469bfff640 1 -- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c06ffb0 msgr2=0x7f469c14d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469bfff640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c06ffb0 0x7f469c14d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469bfff640 1 -- 172.21.15.131:0/977308497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469c14edc0 con 0x7f469c0cf2c0 2024-09-17T02:41:13.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469b7fe640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c06ffb0 0x7f469c14d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:13.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469affd640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f469c070f80 0x7f469c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:13.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f469bfff640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f469c0cf2c0 0x7f469c0c8db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f468c019900 tx=0x7f468c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:13.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f4683fff640 1 -- 172.21.15.131:0/977308497 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c0bb070 con 0x7f469c0cf2c0 2024-09-17T02:41:13.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f4683fff640 1 -- 172.21.15.131:0/977308497 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f468c004d40 con 0x7f469c0cf2c0 2024-09-17T02:41:13.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f46a1b83640 1 -- 172.21.15.131:0/977308497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469c15d1f0 con 0x7f469c0cf2c0 2024-09-17T02:41:13.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f4683fff640 1 -- 172.21.15.131:0/977308497 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c0b2040 con 0x7f469c0cf2c0 2024-09-17T02:41:13.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f46a1b83640 1 -- 172.21.15.131:0/977308497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f469c15d7b0 con 0x7f469c0cf2c0 2024-09-17T02:41:13.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f46a1b83640 1 -- 172.21.15.131:0/977308497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469c14eb30 con 0x7f469c0cf2c0 2024-09-17T02:41:13.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f4683fff640 1 -- 172.21.15.131:0/977308497 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f468c0be070 con 0x7f469c0cf2c0 2024-09-17T02:41:13.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.778+0000 7f4683fff640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4688073200 0x7f4688075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:13.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.782+0000 7f4683fff640 1 -- 172.21.15.131:0/977308497 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f468c12e180 con 0x7f469c0cf2c0 2024-09-17T02:41:13.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.782+0000 7f469b7fe640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4688073200 0x7f4688075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:13.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.782+0000 7f469b7fe640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4688073200 0x7f4688075710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4694005fd0 tx=0x7f4694005dc0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:13.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.786+0000 7f4683fff640 1 -- 172.21.15.131:0/977308497 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f468c016970 con 0x7f469c0cf2c0 2024-09-17T02:41:13.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.950+0000 7f46a1b83640 1 -- 172.21.15.131:0/977308497 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f469c14dd30 con 0x7f469c0cf2c0 2024-09-17T02:41:13.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f4683fff640 1 -- 172.21.15.131:0/977308497 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18650 (secure 0 0 0) 0x7f468c0f9000 con 0x7f469c0cf2c0 2024-09-17T02:41:13.952 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:13.952 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:56.713412+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:41:13.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 -- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4688073200 msgr2=0x7f4688075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4688073200 0x7f4688075710 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4694005fd0 tx=0x7f4694005dc0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 -- 172.21.15.131:0/977308497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f469c0cf2c0 msgr2=0x7f469c0c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:13.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f469c0cf2c0 0x7f469c0c8db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f468c019900 tx=0x7f468c005b00 comp rx=0 tx=0).stop 2024-09-17T02:41:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 -- 172.21.15.131:0/977308497 shutdown_connections 2024-09-17T02:41:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4688073200 0x7f4688075710 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f469c0cf2c0 0x7f469c0c8db0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f469c070f80 0x7f469c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 --2- 172.21.15.131:0/977308497 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f469c06ffb0 0x7f469c14d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 -- 172.21.15.131:0/977308497 >> 172.21.15.131:0/977308497 conn(0x7f469c0cdbb0 msgr2=0x7f469c158740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 -- 172.21.15.131:0/977308497 shutdown_connections 2024-09-17T02:41:13.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:13.954+0000 7f46a1b83640 1 -- 172.21.15.131:0/977308497 wait complete. 2024-09-17T02:41:13.973 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T02:41:13.973 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:41:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.114+0000 7f157b94e640 1 -- 172.21.15.131:0/3883857577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15740cf710 msgr2=0x7f157415f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.114+0000 7f157b94e640 1 --2- 172.21.15.131:0/3883857577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15740cf710 0x7f157415f290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f157406ecf0 tx=0x7f156401c8a0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.114+0000 7f157b94e640 1 -- 172.21.15.131:0/3883857577 shutdown_connections 2024-09-17T02:41:14.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.114+0000 7f157b94e640 1 --2- 172.21.15.131:0/3883857577 >> v2:172.21.15.189:3300/0 conn(0x7f157415f7d0 0x7f1574162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.114+0000 7f157b94e640 1 --2- 172.21.15.131:0/3883857577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15740cf710 0x7f157415f290 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.114+0000 7f157b94e640 1 -- 172.21.15.131:0/3883857577 >> 172.21.15.131:0/3883857577 conn(0x7f15740d1b10 msgr2=0x7f15740d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:14.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.114+0000 7f157b94e640 1 -- 172.21.15.131:0/3883857577 shutdown_connections 2024-09-17T02:41:14.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.114+0000 7f157b94e640 1 -- 172.21.15.131:0/3883857577 wait complete. 2024-09-17T02:41:14.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f157b94e640 1 Processor -- start 2024-09-17T02:41:14.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f157b94e640 1 -- start start 2024-09-17T02:41:14.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f157b94e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15740cf710 0x7f15740b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f157b94e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f157415f7d0 0x7f15741616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f157b94e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1574162950 0x7f15740cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f157b94e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f15740d3500 con 0x7f15740cf710 2024-09-17T02:41:14.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f157b94e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f15740d3380 con 0x7f157415f7d0 2024-09-17T02:41:14.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f157b94e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f15740d3680 con 0x7f1574162950 2024-09-17T02:41:14.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f15796ea640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15740cf710 0x7f15740b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f15796ea640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15740cf710 0x7f15740b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57632/0 (socket says 172.21.15.131:57632) 2024-09-17T02:41:14.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f15796ea640 1 -- 172.21.15.131:0/4195131934 learned_addr learned my addr 172.21.15.131:0/4195131934 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:14.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f1578ee9640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f157415f7d0 0x7f15741616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f15796ea640 1 -- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1574162950 msgr2=0x7f15740cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f1579eeb640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1574162950 0x7f15740cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f15796ea640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1574162950 0x7f15740cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f15796ea640 1 -- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f157415f7d0 msgr2=0x7f15741616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f15796ea640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f157415f7d0 0x7f15741616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f15796ea640 1 -- 172.21.15.131:0/4195131934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15740b4fa0 con 0x7f15740cf710 2024-09-17T02:41:14.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f1578ee9640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f157415f7d0 0x7f15741616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:14.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f15796ea640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15740cf710 0x7f15740b5190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f156401af50 tx=0x7f15640a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:14.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f1569ffb640 1 -- 172.21.15.131:0/4195131934 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15640b2040 con 0x7f15740cf710 2024-09-17T02:41:14.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f1569ffb640 1 -- 172.21.15.131:0/4195131934 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1564004430 con 0x7f15740cf710 2024-09-17T02:41:14.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f1569ffb640 1 -- 172.21.15.131:0/4195131934 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564005290 con 0x7f15740cf710 2024-09-17T02:41:14.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f157b94e640 1 -- 172.21.15.131:0/4195131934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15740b4cc0 con 0x7f15740cf710 2024-09-17T02:41:14.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.118+0000 7f157b94e640 1 -- 172.21.15.131:0/4195131934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15740cd6b0 con 0x7f15740cf710 2024-09-17T02:41:14.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.122+0000 7f157b94e640 1 -- 172.21.15.131:0/4195131934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f157415f290 con 0x7f15740cf710 2024-09-17T02:41:14.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.122+0000 7f1569ffb640 1 -- 172.21.15.131:0/4195131934 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f15640be060 con 0x7f15740cf710 2024-09-17T02:41:14.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.122+0000 7f1569ffb640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1558073200 0x7f1558075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.122+0000 7f1569ffb640 1 -- 172.21.15.131:0/4195131934 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f156412d940 con 0x7f15740cf710 2024-09-17T02:41:14.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.122+0000 7f1578ee9640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1558073200 0x7f1558075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.122+0000 7f1578ee9640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1558073200 0x7f1558075710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1570005fd0 tx=0x7f1570005de0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:14.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.126+0000 7f1569ffb640 1 -- 172.21.15.131:0/4195131934 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1564016490 con 0x7f15740cf710 2024-09-17T02:41:14.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.282+0000 7f157b94e640 1 -- 172.21.15.131:0/4195131934 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1574156610 con 0x7f1558073200 2024-09-17T02:41:14.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.302+0000 7f1569ffb640 1 -- 172.21.15.131:0/4195131934 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7f157000ea00 con 0x7f1558073200 2024-09-17T02:41:14.304 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:14.309 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:41:14.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 -- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1558073200 msgr2=0x7f1558075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1558073200 0x7f1558075710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1570005fd0 tx=0x7f1570005de0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 -- 172.21.15.131:0/4195131934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15740cf710 msgr2=0x7f15740b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15740cf710 0x7f15740b5190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f156401af50 tx=0x7f15640a5740 comp rx=0 tx=0).stop 2024-09-17T02:41:14.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 -- 172.21.15.131:0/4195131934 shutdown_connections 2024-09-17T02:41:14.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1558073200 0x7f1558075710 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1574162950 0x7f15740cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f157415f7d0 0x7f15741616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 --2- 172.21.15.131:0/4195131934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15740cf710 0x7f15740b5190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 -- 172.21.15.131:0/4195131934 >> 172.21.15.131:0/4195131934 conn(0x7f15740d1b10 msgr2=0x7f15740b99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:14.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 -- 172.21.15.131:0/4195131934 shutdown_connections 2024-09-17T02:41:14.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.306+0000 7f157b94e640 1 -- 172.21.15.131:0/4195131934 wait complete. 2024-09-17T02:41:14.324 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-17T02:41:13.122611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219564,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755156,"statfs":{"total":773094113280,"available":772869279744,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005695"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290027+0000","last_change":"2024-09-17T02:40:51.241640+0000","last_active":"2024-09-17T02:40:57.290027+0000","last_peered":"2024-09-17T02:40:57.290027+0000","last_clean":"2024-09-17T02:40:57.290027+0000","last_became_active":"2024-09-17T02:40:51.240675+0000","last_became_peered":"2024-09-17T02:40:51.240675+0000","last_unstale":"2024-09-17T02:40:57.290027+0000","last_undegraded":"2024-09-17T02:40:57.290027+0000","last_fullsized":"2024-09-17T02:40:57.290027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475107+0000","last_change":"2024-09-17T02:40:52.228237+0000","last_active":"2024-09-17T02:40:57.475107+0000","last_peered":"2024-09-17T02:40:57.475107+0000","last_clean":"2024-09-17T02:40:57.475107+0000","last_became_active":"2024-09-17T02:40:52.225329+0000","last_became_peered":"2024-09-17T02:40:52.225329+0000","last_unstale":"2024-09-17T02:40:57.475107+0000","last_undegraded":"2024-09-17T02:40:57.475107+0000","last_fullsized":"2024-09-17T02:40:57.475107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:55.015415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471940+0000","last_change":"2024-09-17T02:40:56.353097+0000","last_active":"2024-09-17T02:40:57.471940+0000","last_peered":"2024-09-17T02:40:57.471940+0000","last_clean":"2024-09-17T02:40:57.471940+0000","last_became_active":"2024-09-17T02:40:56.351847+0000","last_became_peered":"2024-09-17T02:40:56.351847+0000","last_unstale":"2024-09-17T02:40:57.471940+0000","last_undegraded":"2024-09-17T02:40:57.471940+0000","last_fullsized":"2024-09-17T02:40:57.471940+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:06.760578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475261+0000","last_change":"2024-09-17T02:40:51.243590+0000","last_active":"2024-09-17T02:40:57.475261+0000","last_peered":"2024-09-17T02:40:57.475261+0000","last_clean":"2024-09-17T02:40:57.475261+0000","last_became_active":"2024-09-17T02:40:51.242684+0000","last_became_peered":"2024-09-17T02:40:51.242684+0000","last_unstale":"2024-09-17T02:40:57.475261+0000","last_undegraded":"2024-09-17T02:40:57.475261+0000","last_fullsized":"2024-09-17T02:40:57.475261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474246+0000","last_change":"2024-09-17T02:40:52.207992+0000","last_active":"2024-09-17T02:40:57.474246+0000","last_peered":"2024-09-17T02:40:57.474246+0000","last_clean":"2024-09-17T02:40:57.474246+0000","last_became_active":"2024-09-17T02:40:52.207568+0000","last_became_peered":"2024-09-17T02:40:52.207568+0000","last_unstale":"2024-09-17T02:40:57.474246+0000","last_undegraded":"2024-09-17T02:40:57.474246+0000","last_fullsized":"2024-09-17T02:40:57.474246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:01.844374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470155+0000","last_change":"2024-09-17T02:40:56.339588+0000","last_active":"2024-09-17T02:40:57.470155+0000","last_peered":"2024-09-17T02:40:57.470155+0000","last_clean":"2024-09-17T02:40:57.470155+0000","last_became_active":"2024-09-17T02:40:56.338569+0000","last_became_peered":"2024-09-17T02:40:56.338569+0000","last_unstale":"2024-09-17T02:40:57.470155+0000","last_undegraded":"2024-09-17T02:40:57.470155+0000","last_fullsized":"2024-09-17T02:40:57.470155+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:54.995255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472080+0000","last_change":"2024-09-17T02:40:51.225527+0000","last_active":"2024-09-17T02:40:57.472080+0000","last_peered":"2024-09-17T02:40:57.472080+0000","last_clean":"2024-09-17T02:40:57.472080+0000","last_became_active":"2024-09-17T02:40:51.224640+0000","last_became_peered":"2024-09-17T02:40:51.224640+0000","last_unstale":"2024-09-17T02:40:57.472080+0000","last_undegraded":"2024-09-17T02:40:57.472080+0000","last_fullsized":"2024-09-17T02:40:57.472080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:07.028967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470812+0000","last_change":"2024-09-17T02:40:52.243749+0000","last_active":"2024-09-17T02:40:57.470812+0000","last_peered":"2024-09-17T02:40:57.470812+0000","last_clean":"2024-09-17T02:40:57.470812+0000","last_became_active":"2024-09-17T02:40:52.243361+0000","last_became_peered":"2024-09-17T02:40:52.243361+0000","last_unstale":"2024-09-17T02:40:57.470812+0000","last_undegraded":"2024-09-17T02:40:57.470812+0000","last_fullsized":"2024-09-17T02:40:57.470812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:01.471406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471331+0000","last_change":"2024-09-17T02:40:56.330627+0000","last_active":"2024-09-17T02:40:57.471331+0000","last_peered":"2024-09-17T02:40:57.471331+0000","last_clean":"2024-09-17T02:40:57.471331+0000","last_became_active":"2024-09-17T02:40:56.330123+0000","last_became_peered":"2024-09-17T02:40:56.330123+0000","last_unstale":"2024-09-17T02:40:57.471331+0000","last_undegraded":"2024-09-17T02:40:57.471331+0000","last_fullsized":"2024-09-17T02:40:57.471331+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:30.712481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472462+0000","last_change":"2024-09-17T02:40:51.220105+0000","last_active":"2024-09-17T02:40:57.472462+0000","last_peered":"2024-09-17T02:40:57.472462+0000","last_clean":"2024-09-17T02:40:57.472462+0000","last_became_active":"2024-09-17T02:40:51.216247+0000","last_became_peered":"2024-09-17T02:40:51.216247+0000","last_unstale":"2024-09-17T02:40:57.472462+0000","last_undegraded":"2024-09-17T02:40:57.472462+0000","last_fullsized":"2024-09-17T02:40:57.472462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:30.096213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288051+0000","last_change":"2024-09-17T02:40:52.255870+0000","last_active":"2024-09-17T02:40:57.288051+0000","last_peered":"2024-09-17T02:40:57.288051+0000","last_clean":"2024-09-17T02:40:57.288051+0000","last_became_active":"2024-09-17T02:40:52.255454+0000","last_became_peered":"2024-09-17T02:40:52.255454+0000","last_unstale":"2024-09-17T02:40:57.288051+0000","last_undegraded":"2024-09-17T02:40:57.288051+0000","last_fullsized":"2024-09-17T02:40:57.288051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:35.834666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470627+0000","last_change":"2024-09-17T02:40:56.339659+0000","last_active":"2024-09-17T02:40:57.470627+0000","last_peered":"2024-09-17T02:40:57.470627+0000","last_clean":"2024-09-17T02:40:57.470627+0000","last_became_active":"2024-09-17T02:40:56.339130+0000","last_became_peered":"2024-09-17T02:40:56.339130+0000","last_unstale":"2024-09-17T02:40:57.470627+0000","last_undegraded":"2024-09-17T02:40:57.470627+0000","last_fullsized":"2024-09-17T02:40:57.470627+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:38.223653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287802+0000","last_change":"2024-09-17T02:40:52.242899+0000","last_active":"2024-09-17T02:40:57.287802+0000","last_peered":"2024-09-17T02:40:57.287802+0000","last_clean":"2024-09-17T02:40:57.287802+0000","last_became_active":"2024-09-17T02:40:52.242570+0000","last_became_peered":"2024-09-17T02:40:52.242570+0000","last_unstale":"2024-09-17T02:40:57.287802+0000","last_undegraded":"2024-09-17T02:40:57.287802+0000","last_fullsized":"2024-09-17T02:40:57.287802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:45.600475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288528+0000","last_change":"2024-09-17T02:40:51.212048+0000","last_active":"2024-09-17T02:40:57.288528+0000","last_peered":"2024-09-17T02:40:57.288528+0000","last_clean":"2024-09-17T02:40:57.288528+0000","last_became_active":"2024-09-17T02:40:51.211531+0000","last_became_peered":"2024-09-17T02:40:51.211531+0000","last_unstale":"2024-09-17T02:40:57.288528+0000","last_undegraded":"2024-09-17T02:40:57.288528+0000","last_fullsized":"2024-09-17T02:40:57.288528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:10.284709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287732+0000","last_change":"2024-09-17T02:40:56.332707+0000","last_active":"2024-09-17T02:40:57.287732+0000","last_peered":"2024-09-17T02:40:57.287732+0000","last_clean":"2024-09-17T02:40:57.287732+0000","last_became_active":"2024-09-17T02:40:56.332260+0000","last_became_peered":"2024-09-17T02:40:56.332260+0000","last_unstale":"2024-09-17T02:40:57.287732+0000","last_undegraded":"2024-09-17T02:40:57.287732+0000","last_fullsized":"2024-09-17T02:40:57.287732+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:53.783346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472671+0000","last_change":"2024-09-17T02:40:52.218705+0000","last_active":"2024-09-17T02:40:57.472671+0000","last_peered":"2024-09-17T02:40:57.472671+0000","last_clean":"2024-09-17T02:40:57.472671+0000","last_became_active":"2024-09-17T02:40:52.217762+0000","last_became_peered":"2024-09-17T02:40:52.217762+0000","last_unstale":"2024-09-17T02:40:57.472671+0000","last_undegraded":"2024-09-17T02:40:57.472671+0000","last_fullsized":"2024-09-17T02:40:57.472671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:11.295603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287768+0000","last_change":"2024-09-17T02:40:51.237760+0000","last_active":"2024-09-17T02:40:57.287768+0000","last_peered":"2024-09-17T02:40:57.287768+0000","last_clean":"2024-09-17T02:40:57.287768+0000","last_became_active":"2024-09-17T02:40:51.236973+0000","last_became_peered":"2024-09-17T02:40:51.236973+0000","last_unstale":"2024-09-17T02:40:57.287768+0000","last_undegraded":"2024-09-17T02:40:57.287768+0000","last_fullsized":"2024-09-17T02:40:57.287768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:22.634046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471688+0000","last_change":"2024-09-17T02:40:56.355680+0000","last_active":"2024-09-17T02:40:57.471688+0000","last_peered":"2024-09-17T02:40:57.471688+0000","last_clean":"2024-09-17T02:40:57.471688+0000","last_became_active":"2024-09-17T02:40:56.355224+0000","last_became_peered":"2024-09-17T02:40:56.355224+0000","last_unstale":"2024-09-17T02:40:57.471688+0000","last_undegraded":"2024-09-17T02:40:57.471688+0000","last_fullsized":"2024-09-17T02:40:57.471688+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:49.679484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290367+0000","last_change":"2024-09-17T02:40:52.220376+0000","last_active":"2024-09-17T02:40:57.290367+0000","last_peered":"2024-09-17T02:40:57.290367+0000","last_clean":"2024-09-17T02:40:57.290367+0000","last_became_active":"2024-09-17T02:40:52.220031+0000","last_became_peered":"2024-09-17T02:40:52.220031+0000","last_unstale":"2024-09-17T02:40:57.290367+0000","last_undegraded":"2024-09-17T02:40:57.290367+0000","last_fullsized":"2024-09-17T02:40:57.290367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:04.162473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291288+0000","last_change":"2024-09-17T02:40:51.240925+0000","last_active":"2024-09-17T02:40:57.291288+0000","last_peered":"2024-09-17T02:40:57.291288+0000","last_clean":"2024-09-17T02:40:57.291288+0000","last_became_active":"2024-09-17T02:40:51.233638+0000","last_became_peered":"2024-09-17T02:40:51.233638+0000","last_unstale":"2024-09-17T02:40:57.291288+0000","last_undegraded":"2024-09-17T02:40:57.291288+0000","last_fullsized":"2024-09-17T02:40:57.291288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:39.387068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475651+0000","last_change":"2024-09-17T02:40:56.339397+0000","last_active":"2024-09-17T02:40:57.475651+0000","last_peered":"2024-09-17T02:40:57.475651+0000","last_clean":"2024-09-17T02:40:57.475651+0000","last_became_active":"2024-09-17T02:40:56.338591+0000","last_became_peered":"2024-09-17T02:40:56.338591+0000","last_unstale":"2024-09-17T02:40:57.475651+0000","last_undegraded":"2024-09-17T02:40:57.475651+0000","last_fullsized":"2024-09-17T02:40:57.475651+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:10.362765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287431+0000","last_change":"2024-09-17T02:40:52.245352+0000","last_active":"2024-09-17T02:40:57.287431+0000","last_peered":"2024-09-17T02:40:57.287431+0000","last_clean":"2024-09-17T02:40:57.287431+0000","last_became_active":"2024-09-17T02:40:52.244938+0000","last_became_peered":"2024-09-17T02:40:52.244938+0000","last_unstale":"2024-09-17T02:40:57.287431+0000","last_undegraded":"2024-09-17T02:40:57.287431+0000","last_fullsized":"2024-09-17T02:40:57.287431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:07.474908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.469729+0000","last_change":"2024-09-17T02:40:51.220831+0000","last_active":"2024-09-17T02:40:57.469729+0000","last_peered":"2024-09-17T02:40:57.469729+0000","last_clean":"2024-09-17T02:40:57.469729+0000","last_became_active":"2024-09-17T02:40:51.220253+0000","last_became_peered":"2024-09-17T02:40:51.220253+0000","last_unstale":"2024-09-17T02:40:57.469729+0000","last_undegraded":"2024-09-17T02:40:57.469729+0000","last_fullsized":"2024-09-17T02:40:57.469729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471876+0000","last_change":"2024-09-17T02:40:56.333001+0000","last_active":"2024-09-17T02:40:57.471876+0000","last_peered":"2024-09-17T02:40:57.471876+0000","last_clean":"2024-09-17T02:40:57.471876+0000","last_became_active":"2024-09-17T02:40:56.332421+0000","last_became_peered":"2024-09-17T02:40:56.332421+0000","last_unstale":"2024-09-17T02:40:57.471876+0000","last_undegraded":"2024-09-17T02:40:57.471876+0000","last_fullsized":"2024-09-17T02:40:57.471876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:58.168471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291218+0000","last_change":"2024-09-17T02:40:52.215438+0000","last_active":"2024-09-17T02:40:57.291218+0000","last_peered":"2024-09-17T02:40:57.291218+0000","last_clean":"2024-09-17T02:40:57.291218+0000","last_became_active":"2024-09-17T02:40:52.215073+0000","last_became_peered":"2024-09-17T02:40:52.215073+0000","last_unstale":"2024-09-17T02:40:57.291218+0000","last_undegraded":"2024-09-17T02:40:57.291218+0000","last_fullsized":"2024-09-17T02:40:57.291218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:39.361971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288442+0000","last_change":"2024-09-17T02:40:51.226075+0000","last_active":"2024-09-17T02:40:57.288442+0000","last_peered":"2024-09-17T02:40:57.288442+0000","last_clean":"2024-09-17T02:40:57.288442+0000","last_became_active":"2024-09-17T02:40:51.225620+0000","last_became_peered":"2024-09-17T02:40:51.225620+0000","last_unstale":"2024-09-17T02:40:57.288442+0000","last_undegraded":"2024-09-17T02:40:57.288442+0000","last_fullsized":"2024-09-17T02:40:57.288442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:15.238526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470178+0000","last_change":"2024-09-17T02:40:56.306280+0000","last_active":"2024-09-17T02:40:57.470178+0000","last_peered":"2024-09-17T02:40:57.470178+0000","last_clean":"2024-09-17T02:40:57.470178+0000","last_became_active":"2024-09-17T02:40:56.305348+0000","last_became_peered":"2024-09-17T02:40:56.305348+0000","last_unstale":"2024-09-17T02:40:57.470178+0000","last_undegraded":"2024-09-17T02:40:57.470178+0000","last_fullsized":"2024-09-17T02:40:57.470178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:58.295015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289748+0000","last_change":"2024-09-17T02:40:52.217580+0000","last_active":"2024-09-17T02:40:57.289748+0000","last_peered":"2024-09-17T02:40:57.289748+0000","last_clean":"2024-09-17T02:40:57.289748+0000","last_became_active":"2024-09-17T02:40:52.217246+0000","last_became_peered":"2024-09-17T02:40:52.217246+0000","last_unstale":"2024-09-17T02:40:57.289748+0000","last_undegraded":"2024-09-17T02:40:57.289748+0000","last_fullsized":"2024-09-17T02:40:57.289748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:25.696012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291979+0000","last_change":"2024-09-17T02:40:51.235278+0000","last_active":"2024-09-17T02:40:57.291979+0000","last_peered":"2024-09-17T02:40:57.291979+0000","last_clean":"2024-09-17T02:40:57.291979+0000","last_became_active":"2024-09-17T02:40:51.234012+0000","last_became_peered":"2024-09-17T02:40:51.234012+0000","last_unstale":"2024-09-17T02:40:57.291979+0000","last_undegraded":"2024-09-17T02:40:57.291979+0000","last_fullsized":"2024-09-17T02:40:57.291979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:07.028967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290548+0000","last_change":"2024-09-17T02:40:56.306563+0000","last_active":"2024-09-17T02:40:57.290548+0000","last_peered":"2024-09-17T02:40:57.290548+0000","last_clean":"2024-09-17T02:40:57.290548+0000","last_became_active":"2024-09-17T02:40:56.303819+0000","last_became_peered":"2024-09-17T02:40:56.303819+0000","last_unstale":"2024-09-17T02:40:57.290548+0000","last_undegraded":"2024-09-17T02:40:57.290548+0000","last_fullsized":"2024-09-17T02:40:57.290548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:04.387188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289492+0000","last_change":"2024-09-17T02:40:52.216405+0000","last_active":"2024-09-17T02:40:57.289492+0000","last_peered":"2024-09-17T02:40:57.289492+0000","last_clean":"2024-09-17T02:40:57.289492+0000","last_became_active":"2024-09-17T02:40:52.216037+0000","last_became_peered":"2024-09-17T02:40:52.216037+0000","last_unstale":"2024-09-17T02:40:57.289492+0000","last_undegraded":"2024-09-17T02:40:57.289492+0000","last_fullsized":"2024-09-17T02:40:57.289492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:20.565582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288509+0000","last_change":"2024-09-17T02:40:51.243903+0000","last_active":"2024-09-17T02:40:57.288509+0000","last_peered":"2024-09-17T02:40:57.288509+0000","last_clean":"2024-09-17T02:40:57.288509+0000","last_became_active":"2024-09-17T02:40:51.243568+0000","last_became_peered":"2024-09-17T02:40:51.243568+0000","last_unstale":"2024-09-17T02:40:57.288509+0000","last_undegraded":"2024-09-17T02:40:57.288509+0000","last_fullsized":"2024-09-17T02:40:57.288509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:48:49.643024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287722+0000","last_change":"2024-09-17T02:40:56.340894+0000","last_active":"2024-09-17T02:40:57.287722+0000","last_peered":"2024-09-17T02:40:57.287722+0000","last_clean":"2024-09-17T02:40:57.287722+0000","last_became_active":"2024-09-17T02:40:56.338947+0000","last_became_peered":"2024-09-17T02:40:56.338947+0000","last_unstale":"2024-09-17T02:40:57.287722+0000","last_undegraded":"2024-09-17T02:40:57.287722+0000","last_fullsized":"2024-09-17T02:40:57.287722+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:48.414928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472018+0000","last_change":"2024-09-17T02:40:52.232754+0000","last_active":"2024-09-17T02:40:57.472018+0000","last_peered":"2024-09-17T02:40:57.472018+0000","last_clean":"2024-09-17T02:40:57.472018+0000","last_became_active":"2024-09-17T02:40:52.231727+0000","last_became_peered":"2024-09-17T02:40:52.231727+0000","last_unstale":"2024-09-17T02:40:57.472018+0000","last_undegraded":"2024-09-17T02:40:57.472018+0000","last_fullsized":"2024-09-17T02:40:57.472018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:11.295603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473934+0000","last_change":"2024-09-17T02:40:51.237795+0000","last_active":"2024-09-17T02:40:57.473934+0000","last_peered":"2024-09-17T02:40:57.473934+0000","last_clean":"2024-09-17T02:40:57.473934+0000","last_became_active":"2024-09-17T02:40:51.236960+0000","last_became_peered":"2024-09-17T02:40:51.236960+0000","last_unstale":"2024-09-17T02:40:57.473934+0000","last_undegraded":"2024-09-17T02:40:57.473934+0000","last_fullsized":"2024-09-17T02:40:57.473934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:39.387068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287516+0000","last_change":"2024-09-17T02:40:56.340920+0000","last_active":"2024-09-17T02:40:57.287516+0000","last_peered":"2024-09-17T02:40:57.287516+0000","last_clean":"2024-09-17T02:40:57.287516+0000","last_became_active":"2024-09-17T02:40:56.339463+0000","last_became_peered":"2024-09-17T02:40:56.339463+0000","last_unstale":"2024-09-17T02:40:57.287516+0000","last_undegraded":"2024-09-17T02:40:57.287516+0000","last_fullsized":"2024-09-17T02:40:57.287516+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:36.386068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472660+0000","last_change":"2024-09-17T02:40:52.234607+0000","last_active":"2024-09-17T02:40:57.472660+0000","last_peered":"2024-09-17T02:40:57.472660+0000","last_clean":"2024-09-17T02:40:57.472660+0000","last_became_active":"2024-09-17T02:40:52.234175+0000","last_became_peered":"2024-09-17T02:40:52.234175+0000","last_unstale":"2024-09-17T02:40:57.472660+0000","last_undegraded":"2024-09-17T02:40:57.472660+0000","last_fullsized":"2024-09-17T02:40:57.472660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:34.144644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288850+0000","last_change":"2024-09-17T02:40:51.221611+0000","last_active":"2024-09-17T02:40:57.288850+0000","last_peered":"2024-09-17T02:40:57.288850+0000","last_clean":"2024-09-17T02:40:57.288850+0000","last_became_active":"2024-09-17T02:40:51.221259+0000","last_became_peered":"2024-09-17T02:40:51.221259+0000","last_unstale":"2024-09-17T02:40:57.288850+0000","last_undegraded":"2024-09-17T02:40:57.288850+0000","last_fullsized":"2024-09-17T02:40:57.288850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:20.572163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289392+0000","last_change":"2024-09-17T02:40:56.350014+0000","last_active":"2024-09-17T02:40:57.289392+0000","last_peered":"2024-09-17T02:40:57.289392+0000","last_clean":"2024-09-17T02:40:57.289392+0000","last_became_active":"2024-09-17T02:40:56.347244+0000","last_became_peered":"2024-09-17T02:40:56.347244+0000","last_unstale":"2024-09-17T02:40:57.289392+0000","last_undegraded":"2024-09-17T02:40:57.289392+0000","last_fullsized":"2024-09-17T02:40:57.289392+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:10.648378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471945+0000","last_change":"2024-09-17T02:40:52.221771+0000","last_active":"2024-09-17T02:40:57.471945+0000","last_peered":"2024-09-17T02:40:57.471945+0000","last_clean":"2024-09-17T02:40:57.471945+0000","last_became_active":"2024-09-17T02:40:52.221461+0000","last_became_peered":"2024-09-17T02:40:52.221461+0000","last_unstale":"2024-09-17T02:40:57.471945+0000","last_undegraded":"2024-09-17T02:40:57.471945+0000","last_fullsized":"2024-09-17T02:40:57.471945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:25.696012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472767+0000","last_change":"2024-09-17T02:40:51.247013+0000","last_active":"2024-09-17T02:40:57.472767+0000","last_peered":"2024-09-17T02:40:57.472767+0000","last_clean":"2024-09-17T02:40:57.472767+0000","last_became_active":"2024-09-17T02:40:51.246163+0000","last_became_peered":"2024-09-17T02:40:51.246163+0000","last_unstale":"2024-09-17T02:40:57.472767+0000","last_undegraded":"2024-09-17T02:40:57.472767+0000","last_fullsized":"2024-09-17T02:40:57.472767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:05.797393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288261+0000","last_change":"2024-09-17T02:40:56.340614+0000","last_active":"2024-09-17T02:40:57.288261+0000","last_peered":"2024-09-17T02:40:57.288261+0000","last_clean":"2024-09-17T02:40:57.288261+0000","last_became_active":"2024-09-17T02:40:56.338340+0000","last_became_peered":"2024-09-17T02:40:56.338340+0000","last_unstale":"2024-09-17T02:40:57.288261+0000","last_undegraded":"2024-09-17T02:40:57.288261+0000","last_fullsized":"2024-09-17T02:40:57.288261+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:03.442300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.286936+0000","last_change":"2024-09-17T02:40:52.227802+0000","last_active":"2024-09-17T02:40:57.286936+0000","last_peered":"2024-09-17T02:40:57.286936+0000","last_clean":"2024-09-17T02:40:57.286936+0000","last_became_active":"2024-09-17T02:40:52.227458+0000","last_became_peered":"2024-09-17T02:40:52.227458+0000","last_unstale":"2024-09-17T02:40:57.286936+0000","last_undegraded":"2024-09-17T02:40:57.286936+0000","last_fullsized":"2024-09-17T02:40:57.286936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:21.198509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287577+0000","last_change":"2024-09-17T02:40:51.239502+0000","last_active":"2024-09-17T02:40:57.287577+0000","last_peered":"2024-09-17T02:40:57.287577+0000","last_clean":"2024-09-17T02:40:57.287577+0000","last_became_active":"2024-09-17T02:40:51.239042+0000","last_became_peered":"2024-09-17T02:40:51.239042+0000","last_unstale":"2024-09-17T02:40:57.287577+0000","last_undegraded":"2024-09-17T02:40:57.287577+0000","last_fullsized":"2024-09-17T02:40:57.287577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:04.995836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474601+0000","last_change":"2024-09-17T02:40:56.353034+0000","last_active":"2024-09-17T02:40:57.474601+0000","last_peered":"2024-09-17T02:40:57.474601+0000","last_clean":"2024-09-17T02:40:57.474601+0000","last_became_active":"2024-09-17T02:40:56.351920+0000","last_became_peered":"2024-09-17T02:40:56.351920+0000","last_unstale":"2024-09-17T02:40:57.474601+0000","last_undegraded":"2024-09-17T02:40:57.474601+0000","last_fullsized":"2024-09-17T02:40:57.474601+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:39.913675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289211+0000","last_change":"2024-09-17T02:40:52.223050+0000","last_active":"2024-09-17T02:40:57.289211+0000","last_peered":"2024-09-17T02:40:57.289211+0000","last_clean":"2024-09-17T02:40:57.289211+0000","last_became_active":"2024-09-17T02:40:52.221680+0000","last_became_peered":"2024-09-17T02:40:52.221680+0000","last_unstale":"2024-09-17T02:40:57.289211+0000","last_undegraded":"2024-09-17T02:40:57.289211+0000","last_fullsized":"2024-09-17T02:40:57.289211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:06.283756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288675+0000","last_change":"2024-09-17T02:40:51.209280+0000","last_active":"2024-09-17T02:40:57.288675+0000","last_peered":"2024-09-17T02:40:57.288675+0000","last_clean":"2024-09-17T02:40:57.288675+0000","last_became_active":"2024-09-17T02:40:51.208951+0000","last_became_peered":"2024-09-17T02:40:51.208951+0000","last_unstale":"2024-09-17T02:40:57.288675+0000","last_undegraded":"2024-09-17T02:40:57.288675+0000","last_fullsized":"2024-09-17T02:40:57.288675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:51.093529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470279+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:40:57.470279+0000","last_peered":"2024-09-17T02:40:57.470279+0000","last_clean":"2024-09-17T02:40:57.470279+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:40:57.470279+0000","last_undegraded":"2024-09-17T02:40:57.470279+0000","last_fullsized":"2024-09-17T02:40:57.470279+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288416+0000","last_change":"2024-09-17T02:40:56.306099+0000","last_active":"2024-09-17T02:40:57.288416+0000","last_peered":"2024-09-17T02:40:57.288416+0000","last_clean":"2024-09-17T02:40:57.288416+0000","last_became_active":"2024-09-17T02:40:56.305604+0000","last_became_peered":"2024-09-17T02:40:56.305604+0000","last_unstale":"2024-09-17T02:40:57.288416+0000","last_undegraded":"2024-09-17T02:40:57.288416+0000","last_fullsized":"2024-09-17T02:40:57.288416+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:22.632347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288182+0000","last_change":"2024-09-17T02:40:52.255845+0000","last_active":"2024-09-17T02:40:57.288182+0000","last_peered":"2024-09-17T02:40:57.288182+0000","last_clean":"2024-09-17T02:40:57.288182+0000","last_became_active":"2024-09-17T02:40:52.255416+0000","last_became_peered":"2024-09-17T02:40:52.255416+0000","last_unstale":"2024-09-17T02:40:57.288182+0000","last_undegraded":"2024-09-17T02:40:57.288182+0000","last_fullsized":"2024-09-17T02:40:57.288182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:40.879599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289502+0000","last_change":"2024-09-17T02:40:51.230583+0000","last_active":"2024-09-17T02:40:57.289502+0000","last_peered":"2024-09-17T02:40:57.289502+0000","last_clean":"2024-09-17T02:40:57.289502+0000","last_became_active":"2024-09-17T02:40:51.229636+0000","last_became_peered":"2024-09-17T02:40:51.229636+0000","last_unstale":"2024-09-17T02:40:57.289502+0000","last_undegraded":"2024-09-17T02:40:57.289502+0000","last_fullsized":"2024-09-17T02:40:57.289502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:25.403066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470871+0000","last_change":"2024-09-17T02:40:56.334178+0000","last_active":"2024-09-17T02:40:57.470871+0000","last_peered":"2024-09-17T02:40:57.470871+0000","last_clean":"2024-09-17T02:40:57.470871+0000","last_became_active":"2024-09-17T02:40:56.333520+0000","last_became_peered":"2024-09-17T02:40:56.333520+0000","last_unstale":"2024-09-17T02:40:57.470871+0000","last_undegraded":"2024-09-17T02:40:57.470871+0000","last_fullsized":"2024-09-17T02:40:57.470871+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:38.037563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472395+0000","last_change":"2024-09-17T02:40:52.237704+0000","last_active":"2024-09-17T02:40:57.472395+0000","last_peered":"2024-09-17T02:40:57.472395+0000","last_clean":"2024-09-17T02:40:57.472395+0000","last_became_active":"2024-09-17T02:40:52.235768+0000","last_became_peered":"2024-09-17T02:40:52.235768+0000","last_unstale":"2024-09-17T02:40:57.472395+0000","last_undegraded":"2024-09-17T02:40:57.472395+0000","last_fullsized":"2024-09-17T02:40:57.472395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:04.549252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473130+0000","last_change":"2024-09-17T02:40:51.227571+0000","last_active":"2024-09-17T02:40:57.473130+0000","last_peered":"2024-09-17T02:40:57.473130+0000","last_clean":"2024-09-17T02:40:57.473130+0000","last_became_active":"2024-09-17T02:40:51.226748+0000","last_became_peered":"2024-09-17T02:40:51.226748+0000","last_unstale":"2024-09-17T02:40:57.473130+0000","last_undegraded":"2024-09-17T02:40:57.473130+0000","last_fullsized":"2024-09-17T02:40:57.473130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:12.330693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473168+0000","last_change":"2024-09-17T02:40:56.327712+0000","last_active":"2024-09-17T02:40:57.473168+0000","last_peered":"2024-09-17T02:40:57.473168+0000","last_clean":"2024-09-17T02:40:57.473168+0000","last_became_active":"2024-09-17T02:40:56.326836+0000","last_became_peered":"2024-09-17T02:40:56.326836+0000","last_unstale":"2024-09-17T02:40:57.473168+0000","last_undegraded":"2024-09-17T02:40:57.473168+0000","last_fullsized":"2024-09-17T02:40:57.473168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:18.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471966+0000","last_change":"2024-09-17T02:40:52.239267+0000","last_active":"2024-09-17T02:40:57.471966+0000","last_peered":"2024-09-17T02:40:57.471966+0000","last_clean":"2024-09-17T02:40:57.471966+0000","last_became_active":"2024-09-17T02:40:52.238707+0000","last_became_peered":"2024-09-17T02:40:52.238707+0000","last_unstale":"2024-09-17T02:40:57.471966+0000","last_undegraded":"2024-09-17T02:40:57.471966+0000","last_fullsized":"2024-09-17T02:40:57.471966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:07.999419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291556+0000","last_change":"2024-09-17T02:40:51.249182+0000","last_active":"2024-09-17T02:40:57.291556+0000","last_peered":"2024-09-17T02:40:57.291556+0000","last_clean":"2024-09-17T02:40:57.291556+0000","last_became_active":"2024-09-17T02:40:51.248480+0000","last_became_peered":"2024-09-17T02:40:51.248480+0000","last_unstale":"2024-09-17T02:40:57.291556+0000","last_undegraded":"2024-09-17T02:40:57.291556+0000","last_fullsized":"2024-09-17T02:40:57.291556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287153+0000","last_change":"2024-09-17T02:40:56.306150+0000","last_active":"2024-09-17T02:40:57.287153+0000","last_peered":"2024-09-17T02:40:57.287153+0000","last_clean":"2024-09-17T02:40:57.287153+0000","last_became_active":"2024-09-17T02:40:56.305729+0000","last_became_peered":"2024-09-17T02:40:56.305729+0000","last_unstale":"2024-09-17T02:40:57.287153+0000","last_undegraded":"2024-09-17T02:40:57.287153+0000","last_fullsized":"2024-09-17T02:40:57.287153+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:32.927545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472197+0000","last_change":"2024-09-17T02:40:52.229209+0000","last_active":"2024-09-17T02:40:57.472197+0000","last_peered":"2024-09-17T02:40:57.472197+0000","last_clean":"2024-09-17T02:40:57.472197+0000","last_became_active":"2024-09-17T02:40:52.228749+0000","last_became_peered":"2024-09-17T02:40:52.228749+0000","last_unstale":"2024-09-17T02:40:57.472197+0000","last_undegraded":"2024-09-17T02:40:57.472197+0000","last_fullsized":"2024-09-17T02:40:57.472197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:52.104423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471968+0000","last_change":"2024-09-17T02:40:51.244855+0000","last_active":"2024-09-17T02:40:57.471968+0000","last_peered":"2024-09-17T02:40:57.471968+0000","last_clean":"2024-09-17T02:40:57.471968+0000","last_became_active":"2024-09-17T02:40:51.244535+0000","last_became_peered":"2024-09-17T02:40:51.244535+0000","last_unstale":"2024-09-17T02:40:57.471968+0000","last_undegraded":"2024-09-17T02:40:57.471968+0000","last_fullsized":"2024-09-17T02:40:57.471968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:15.387810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471115+0000","last_change":"2024-09-17T02:40:56.316962+0000","last_active":"2024-09-17T02:40:57.471115+0000","last_peered":"2024-09-17T02:40:57.471115+0000","last_clean":"2024-09-17T02:40:57.471115+0000","last_became_active":"2024-09-17T02:40:56.316350+0000","last_became_peered":"2024-09-17T02:40:56.316350+0000","last_unstale":"2024-09-17T02:40:57.471115+0000","last_undegraded":"2024-09-17T02:40:57.471115+0000","last_fullsized":"2024-09-17T02:40:57.471115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:49.655411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287234+0000","last_change":"2024-09-17T02:40:52.237017+0000","last_active":"2024-09-17T02:40:57.287234+0000","last_peered":"2024-09-17T02:40:57.287234+0000","last_clean":"2024-09-17T02:40:57.287234+0000","last_became_active":"2024-09-17T02:40:52.230499+0000","last_became_peered":"2024-09-17T02:40:52.230499+0000","last_unstale":"2024-09-17T02:40:57.287234+0000","last_undegraded":"2024-09-17T02:40:57.287234+0000","last_fullsized":"2024-09-17T02:40:57.287234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:46.628840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290952+0000","last_change":"2024-09-17T02:40:51.241977+0000","last_active":"2024-09-17T02:40:57.290952+0000","last_peered":"2024-09-17T02:40:57.290952+0000","last_clean":"2024-09-17T02:40:57.290952+0000","last_became_active":"2024-09-17T02:40:51.241088+0000","last_became_peered":"2024-09-17T02:40:51.241088+0000","last_unstale":"2024-09-17T02:40:57.290952+0000","last_undegraded":"2024-09-17T02:40:57.290952+0000","last_fullsized":"2024-09-17T02:40:57.290952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:15.387810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475101+0000","last_change":"2024-09-17T02:40:56.349582+0000","last_active":"2024-09-17T02:40:57.475101+0000","last_peered":"2024-09-17T02:40:57.475101+0000","last_clean":"2024-09-17T02:40:57.475101+0000","last_became_active":"2024-09-17T02:40:56.348588+0000","last_became_peered":"2024-09-17T02:40:56.348588+0000","last_unstale":"2024-09-17T02:40:57.475101+0000","last_undegraded":"2024-09-17T02:40:57.475101+0000","last_fullsized":"2024-09-17T02:40:57.475101+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:38:43.776429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470147+0000","last_change":"2024-09-17T02:40:52.209126+0000","last_active":"2024-09-17T02:40:57.470147+0000","last_peered":"2024-09-17T02:40:57.470147+0000","last_clean":"2024-09-17T02:40:57.470147+0000","last_became_active":"2024-09-17T02:40:52.208689+0000","last_became_peered":"2024-09-17T02:40:52.208689+0000","last_unstale":"2024-09-17T02:40:57.470147+0000","last_undegraded":"2024-09-17T02:40:57.470147+0000","last_fullsized":"2024-09-17T02:40:57.470147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:08.039862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470654+0000","last_change":"2024-09-17T02:40:51.224834+0000","last_active":"2024-09-17T02:40:57.470654+0000","last_peered":"2024-09-17T02:40:57.470654+0000","last_clean":"2024-09-17T02:40:57.470654+0000","last_became_active":"2024-09-17T02:40:51.224133+0000","last_became_peered":"2024-09-17T02:40:51.224133+0000","last_unstale":"2024-09-17T02:40:57.470654+0000","last_undegraded":"2024-09-17T02:40:57.470654+0000","last_fullsized":"2024-09-17T02:40:57.470654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:13.491613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474143+0000","last_change":"2024-09-17T02:40:56.340807+0000","last_active":"2024-09-17T02:40:57.474143+0000","last_peered":"2024-09-17T02:40:57.474143+0000","last_clean":"2024-09-17T02:40:57.474143+0000","last_became_active":"2024-09-17T02:40:56.339879+0000","last_became_peered":"2024-09-17T02:40:56.339879+0000","last_unstale":"2024-09-17T02:40:57.474143+0000","last_undegraded":"2024-09-17T02:40:57.474143+0000","last_fullsized":"2024-09-17T02:40:57.474143+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:24.197066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288140+0000","last_change":"2024-09-17T02:40:52.244094+0000","last_active":"2024-09-17T02:40:57.288140+0000","last_peered":"2024-09-17T02:40:57.288140+0000","last_clean":"2024-09-17T02:40:57.288140+0000","last_became_active":"2024-09-17T02:40:52.242739+0000","last_became_peered":"2024-09-17T02:40:52.242739+0000","last_unstale":"2024-09-17T02:40:57.288140+0000","last_undegraded":"2024-09-17T02:40:57.288140+0000","last_fullsized":"2024-09-17T02:40:57.288140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:36.659884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288765+0000","last_change":"2024-09-17T02:40:51.222211+0000","last_active":"2024-09-17T02:40:57.288765+0000","last_peered":"2024-09-17T02:40:57.288765+0000","last_clean":"2024-09-17T02:40:57.288765+0000","last_became_active":"2024-09-17T02:40:51.221783+0000","last_became_peered":"2024-09-17T02:40:51.221783+0000","last_unstale":"2024-09-17T02:40:57.288765+0000","last_undegraded":"2024-09-17T02:40:57.288765+0000","last_fullsized":"2024-09-17T02:40:57.288765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:31.651869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288876+0000","last_change":"2024-09-17T02:40:56.348106+0000","last_active":"2024-09-17T02:40:57.288876+0000","last_peered":"2024-09-17T02:40:57.288876+0000","last_clean":"2024-09-17T02:40:57.288876+0000","last_became_active":"2024-09-17T02:40:56.339110+0000","last_became_peered":"2024-09-17T02:40:56.339110+0000","last_unstale":"2024-09-17T02:40:57.288876+0000","last_undegraded":"2024-09-17T02:40:57.288876+0000","last_fullsized":"2024-09-17T02:40:57.288876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:11.904235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287392+0000","last_change":"2024-09-17T02:40:52.230525+0000","last_active":"2024-09-17T02:40:57.287392+0000","last_peered":"2024-09-17T02:40:57.287392+0000","last_clean":"2024-09-17T02:40:57.287392+0000","last_became_active":"2024-09-17T02:40:52.230230+0000","last_became_peered":"2024-09-17T02:40:52.230230+0000","last_unstale":"2024-09-17T02:40:57.287392+0000","last_undegraded":"2024-09-17T02:40:57.287392+0000","last_fullsized":"2024-09-17T02:40:57.287392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:40.156443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473184+0000","last_change":"2024-09-17T02:40:51.232502+0000","last_active":"2024-09-17T02:40:57.473184+0000","last_peered":"2024-09-17T02:40:57.473184+0000","last_clean":"2024-09-17T02:40:57.473184+0000","last_became_active":"2024-09-17T02:40:51.230289+0000","last_became_peered":"2024-09-17T02:40:51.230289+0000","last_unstale":"2024-09-17T02:40:57.473184+0000","last_undegraded":"2024-09-17T02:40:57.473184+0000","last_fullsized":"2024-09-17T02:40:57.473184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:59.380994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288172+0000","last_change":"2024-09-17T02:40:56.325742+0000","last_active":"2024-09-17T02:40:57.288172+0000","last_peered":"2024-09-17T02:40:57.288172+0000","last_clean":"2024-09-17T02:40:57.288172+0000","last_became_active":"2024-09-17T02:40:56.325287+0000","last_became_peered":"2024-09-17T02:40:56.325287+0000","last_unstale":"2024-09-17T02:40:57.288172+0000","last_undegraded":"2024-09-17T02:40:57.288172+0000","last_fullsized":"2024-09-17T02:40:57.288172+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:28.012374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471884+0000","last_change":"2024-09-17T02:40:52.236056+0000","last_active":"2024-09-17T02:40:57.471884+0000","last_peered":"2024-09-17T02:40:57.471884+0000","last_clean":"2024-09-17T02:40:57.471884+0000","last_became_active":"2024-09-17T02:40:52.235680+0000","last_became_peered":"2024-09-17T02:40:52.235680+0000","last_unstale":"2024-09-17T02:40:57.471884+0000","last_undegraded":"2024-09-17T02:40:57.471884+0000","last_fullsized":"2024-09-17T02:40:57.471884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:27.873310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:41:03.060549+0000","last_change":"2024-09-17T02:40:51.242488+0000","last_active":"2024-09-17T02:41:03.060549+0000","last_peered":"2024-09-17T02:41:03.060549+0000","last_clean":"2024-09-17T02:41:03.060549+0000","last_became_active":"2024-09-17T02:40:51.242105+0000","last_became_peered":"2024-09-17T02:40:51.242105+0000","last_unstale":"2024-09-17T02:41:03.060549+0000","last_undegraded":"2024-09-17T02:41:03.060549+0000","last_fullsized":"2024-09-17T02:41:03.060549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:02.225605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287754+0000","last_change":"2024-09-17T02:40:56.327216+0000","last_active":"2024-09-17T02:40:57.287754+0000","last_peered":"2024-09-17T02:40:57.287754+0000","last_clean":"2024-09-17T02:40:57.287754+0000","last_became_active":"2024-09-17T02:40:56.326213+0000","last_became_peered":"2024-09-17T02:40:56.326213+0000","last_unstale":"2024-09-17T02:40:57.287754+0000","last_undegraded":"2024-09-17T02:40:57.287754+0000","last_fullsized":"2024-09-17T02:40:57.287754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:58.168471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288181+0000","last_change":"2024-09-17T02:40:52.237718+0000","last_active":"2024-09-17T02:40:57.288181+0000","last_peered":"2024-09-17T02:40:57.288181+0000","last_clean":"2024-09-17T02:40:57.288181+0000","last_became_active":"2024-09-17T02:40:52.237303+0000","last_became_peered":"2024-09-17T02:40:52.237303+0000","last_unstale":"2024-09-17T02:40:57.288181+0000","last_undegraded":"2024-09-17T02:40:57.288181+0000","last_fullsized":"2024-09-17T02:40:57.288181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:47.651165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474398+0000","last_change":"2024-09-17T02:40:51.234788+0000","last_active":"2024-09-17T02:40:57.474398+0000","last_peered":"2024-09-17T02:40:57.474398+0000","last_clean":"2024-09-17T02:40:57.474398+0000","last_became_active":"2024-09-17T02:40:51.233917+0000","last_became_peered":"2024-09-17T02:40:51.233917+0000","last_unstale":"2024-09-17T02:40:57.474398+0000","last_undegraded":"2024-09-17T02:40:57.474398+0000","last_fullsized":"2024-09-17T02:40:57.474398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:27.301395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471431+0000","last_change":"2024-09-17T02:40:56.337448+0000","last_active":"2024-09-17T02:40:57.471431+0000","last_peered":"2024-09-17T02:40:57.471431+0000","last_clean":"2024-09-17T02:40:57.471431+0000","last_became_active":"2024-09-17T02:40:56.336959+0000","last_became_peered":"2024-09-17T02:40:56.336959+0000","last_unstale":"2024-09-17T02:40:57.471431+0000","last_undegraded":"2024-09-17T02:40:57.471431+0000","last_fullsized":"2024-09-17T02:40:57.471431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:31.952318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474548+0000","last_change":"2024-09-17T02:40:52.247671+0000","last_active":"2024-09-17T02:40:57.474548+0000","last_peered":"2024-09-17T02:40:57.474548+0000","last_clean":"2024-09-17T02:40:57.474548+0000","last_became_active":"2024-09-17T02:40:52.247325+0000","last_became_peered":"2024-09-17T02:40:52.247325+0000","last_unstale":"2024-09-17T02:40:57.474548+0000","last_undegraded":"2024-09-17T02:40:57.474548+0000","last_fullsized":"2024-09-17T02:40:57.474548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:09.357623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471882+0000","last_change":"2024-09-17T02:40:51.224974+0000","last_active":"2024-09-17T02:40:57.471882+0000","last_peered":"2024-09-17T02:40:57.471882+0000","last_clean":"2024-09-17T02:40:57.471882+0000","last_became_active":"2024-09-17T02:40:51.224479+0000","last_became_peered":"2024-09-17T02:40:51.224479+0000","last_unstale":"2024-09-17T02:40:57.471882+0000","last_undegraded":"2024-09-17T02:40:57.471882+0000","last_fullsized":"2024-09-17T02:40:57.471882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471541+0000","last_change":"2024-09-17T02:40:56.333951+0000","last_active":"2024-09-17T02:40:57.471541+0000","last_peered":"2024-09-17T02:40:57.471541+0000","last_clean":"2024-09-17T02:40:57.471541+0000","last_became_active":"2024-09-17T02:40:56.332508+0000","last_became_peered":"2024-09-17T02:40:56.332508+0000","last_unstale":"2024-09-17T02:40:57.471541+0000","last_undegraded":"2024-09-17T02:40:57.471541+0000","last_fullsized":"2024-09-17T02:40:57.471541+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:08.241482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472176+0000","last_change":"2024-09-17T02:40:52.244735+0000","last_active":"2024-09-17T02:40:57.472176+0000","last_peered":"2024-09-17T02:40:57.472176+0000","last_clean":"2024-09-17T02:40:57.472176+0000","last_became_active":"2024-09-17T02:40:52.243763+0000","last_became_peered":"2024-09-17T02:40:52.243763+0000","last_unstale":"2024-09-17T02:40:57.472176+0000","last_undegraded":"2024-09-17T02:40:57.472176+0000","last_fullsized":"2024-09-17T02:40:57.472176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:32.662763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471344+0000","last_change":"2024-09-17T02:40:51.234151+0000","last_active":"2024-09-17T02:40:57.471344+0000","last_peered":"2024-09-17T02:40:57.471344+0000","last_clean":"2024-09-17T02:40:57.471344+0000","last_became_active":"2024-09-17T02:40:51.233731+0000","last_became_peered":"2024-09-17T02:40:51.233731+0000","last_unstale":"2024-09-17T02:40:57.471344+0000","last_undegraded":"2024-09-17T02:40:57.471344+0000","last_fullsized":"2024-09-17T02:40:57.471344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:15.349994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290693+0000","last_change":"2024-09-17T02:40:56.334710+0000","last_active":"2024-09-17T02:40:57.290693+0000","last_peered":"2024-09-17T02:40:57.290693+0000","last_clean":"2024-09-17T02:40:57.290693+0000","last_became_active":"2024-09-17T02:40:56.333630+0000","last_became_peered":"2024-09-17T02:40:56.333630+0000","last_unstale":"2024-09-17T02:40:57.290693+0000","last_undegraded":"2024-09-17T02:40:57.290693+0000","last_fullsized":"2024-09-17T02:40:57.290693+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:07.834564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290681+0000","last_change":"2024-09-17T02:40:56.321729+0000","last_active":"2024-09-17T02:40:57.290681+0000","last_peered":"2024-09-17T02:40:57.290681+0000","last_clean":"2024-09-17T02:40:57.290681+0000","last_became_active":"2024-09-17T02:40:56.320583+0000","last_became_peered":"2024-09-17T02:40:56.320583+0000","last_unstale":"2024-09-17T02:40:57.290681+0000","last_undegraded":"2024-09-17T02:40:57.290681+0000","last_fullsized":"2024-09-17T02:40:57.290681+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:50.707848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291114+0000","last_change":"2024-09-17T02:40:52.224867+0000","last_active":"2024-09-17T02:40:57.291114+0000","last_peered":"2024-09-17T02:40:57.291114+0000","last_clean":"2024-09-17T02:40:57.291114+0000","last_became_active":"2024-09-17T02:40:52.224558+0000","last_became_peered":"2024-09-17T02:40:52.224558+0000","last_unstale":"2024-09-17T02:40:57.291114+0000","last_undegraded":"2024-09-17T02:40:57.291114+0000","last_fullsized":"2024-09-17T02:40:57.291114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:01.098205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287555+0000","last_change":"2024-09-17T02:40:51.210128+0000","last_active":"2024-09-17T02:40:57.287555+0000","last_peered":"2024-09-17T02:40:57.287555+0000","last_clean":"2024-09-17T02:40:57.287555+0000","last_became_active":"2024-09-17T02:40:51.209655+0000","last_became_peered":"2024-09-17T02:40:51.209655+0000","last_unstale":"2024-09-17T02:40:57.287555+0000","last_undegraded":"2024-09-17T02:40:57.287555+0000","last_fullsized":"2024-09-17T02:40:57.287555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:18.053396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288450+0000","last_change":"2024-09-17T02:40:56.316501+0000","last_active":"2024-09-17T02:40:57.288450+0000","last_peered":"2024-09-17T02:40:57.288450+0000","last_clean":"2024-09-17T02:40:57.288450+0000","last_became_active":"2024-09-17T02:40:56.315839+0000","last_became_peered":"2024-09-17T02:40:56.315839+0000","last_unstale":"2024-09-17T02:40:57.288450+0000","last_undegraded":"2024-09-17T02:40:57.288450+0000","last_fullsized":"2024-09-17T02:40:57.288450+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:21.657805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472377+0000","last_change":"2024-09-17T02:40:52.228669+0000","last_active":"2024-09-17T02:40:57.472377+0000","last_peered":"2024-09-17T02:40:57.472377+0000","last_clean":"2024-09-17T02:40:57.472377+0000","last_became_active":"2024-09-17T02:40:52.226236+0000","last_became_peered":"2024-09-17T02:40:52.226236+0000","last_unstale":"2024-09-17T02:40:57.472377+0000","last_undegraded":"2024-09-17T02:40:57.472377+0000","last_fullsized":"2024-09-17T02:40:57.472377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:41.461773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287139+0000","last_change":"2024-09-17T02:40:51.226679+0000","last_active":"2024-09-17T02:40:57.287139+0000","last_peered":"2024-09-17T02:40:57.287139+0000","last_clean":"2024-09-17T02:40:57.287139+0000","last_became_active":"2024-09-17T02:40:51.225794+0000","last_became_peered":"2024-09-17T02:40:51.225794+0000","last_unstale":"2024-09-17T02:40:57.287139+0000","last_undegraded":"2024-09-17T02:40:57.287139+0000","last_fullsized":"2024-09-17T02:40:57.287139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:37.666440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288775+0000","last_change":"2024-09-17T02:40:51.226491+0000","last_active":"2024-09-17T02:40:57.288775+0000","last_peered":"2024-09-17T02:40:57.288775+0000","last_clean":"2024-09-17T02:40:57.288775+0000","last_became_active":"2024-09-17T02:40:51.225695+0000","last_became_peered":"2024-09-17T02:40:51.225695+0000","last_unstale":"2024-09-17T02:40:57.288775+0000","last_undegraded":"2024-09-17T02:40:57.288775+0000","last_fullsized":"2024-09-17T02:40:57.288775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:46.966810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288252+0000","last_change":"2024-09-17T02:40:52.222794+0000","last_active":"2024-09-17T02:40:57.288252+0000","last_peered":"2024-09-17T02:40:57.288252+0000","last_clean":"2024-09-17T02:40:57.288252+0000","last_became_active":"2024-09-17T02:40:52.221682+0000","last_became_peered":"2024-09-17T02:40:52.221682+0000","last_unstale":"2024-09-17T02:40:57.288252+0000","last_undegraded":"2024-09-17T02:40:57.288252+0000","last_fullsized":"2024-09-17T02:40:57.288252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:34.058125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.292523+0000","last_change":"2024-09-17T02:40:56.343317+0000","last_active":"2024-09-17T02:40:57.292523+0000","last_peered":"2024-09-17T02:40:57.292523+0000","last_clean":"2024-09-17T02:40:57.292523+0000","last_became_active":"2024-09-17T02:40:56.342373+0000","last_became_peered":"2024-09-17T02:40:56.342373+0000","last_unstale":"2024-09-17T02:40:57.292523+0000","last_undegraded":"2024-09-17T02:40:57.292523+0000","last_fullsized":"2024-09-17T02:40:57.292523+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:02.210967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473874+0000","last_change":"2024-09-17T02:40:51.237550+0000","last_active":"2024-09-17T02:40:57.473874+0000","last_peered":"2024-09-17T02:40:57.473874+0000","last_clean":"2024-09-17T02:40:57.473874+0000","last_became_active":"2024-09-17T02:40:51.236422+0000","last_became_peered":"2024-09-17T02:40:51.236422+0000","last_unstale":"2024-09-17T02:40:57.473874+0000","last_undegraded":"2024-09-17T02:40:57.473874+0000","last_fullsized":"2024-09-17T02:40:57.473874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470700+0000","last_change":"2024-09-17T02:40:52.226397+0000","last_active":"2024-09-17T02:40:57.470700+0000","last_peered":"2024-09-17T02:40:57.470700+0000","last_clean":"2024-09-17T02:40:57.470700+0000","last_became_active":"2024-09-17T02:40:52.225934+0000","last_became_peered":"2024-09-17T02:40:52.225934+0000","last_unstale":"2024-09-17T02:40:57.470700+0000","last_undegraded":"2024-09-17T02:40:57.470700+0000","last_fullsized":"2024-09-17T02:40:57.470700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:17:37.421483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472110+0000","last_change":"2024-09-17T02:40:56.336996+0000","last_active":"2024-09-17T02:40:57.472110+0000","last_peered":"2024-09-17T02:40:57.472110+0000","last_clean":"2024-09-17T02:40:57.472110+0000","last_became_active":"2024-09-17T02:40:56.336172+0000","last_became_peered":"2024-09-17T02:40:56.336172+0000","last_unstale":"2024-09-17T02:40:57.472110+0000","last_undegraded":"2024-09-17T02:40:57.472110+0000","last_fullsized":"2024-09-17T02:40:57.472110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:05.177214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470197+0000","last_change":"2024-09-17T02:40:52.232046+0000","last_active":"2024-09-17T02:40:57.470197+0000","last_peered":"2024-09-17T02:40:57.470197+0000","last_clean":"2024-09-17T02:40:57.470197+0000","last_became_active":"2024-09-17T02:40:52.231700+0000","last_became_peered":"2024-09-17T02:40:52.231700+0000","last_unstale":"2024-09-17T02:40:57.470197+0000","last_undegraded":"2024-09-17T02:40:57.470197+0000","last_fullsized":"2024-09-17T02:40:57.470197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:47.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472501+0000","last_change":"2024-09-17T02:40:52.239198+0000","last_active":"2024-09-17T02:40:57.472501+0000","last_peered":"2024-09-17T02:40:57.472501+0000","last_clean":"2024-09-17T02:40:57.472501+0000","last_became_active":"2024-09-17T02:40:52.238901+0000","last_became_peered":"2024-09-17T02:40:52.238901+0000","last_unstale":"2024-09-17T02:40:57.472501+0000","last_undegraded":"2024-09-17T02:40:57.472501+0000","last_fullsized":"2024-09-17T02:40:57.472501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:47.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287639+0000","last_change":"2024-09-17T02:40:52.228993+0000","last_active":"2024-09-17T02:40:57.287639+0000","last_peered":"2024-09-17T02:40:57.287639+0000","last_clean":"2024-09-17T02:40:57.287639+0000","last_became_active":"2024-09-17T02:40:52.228574+0000","last_became_peered":"2024-09-17T02:40:52.228574+0000","last_unstale":"2024-09-17T02:40:57.287639+0000","last_undegraded":"2024-09-17T02:40:57.287639+0000","last_fullsized":"2024-09-17T02:40:57.287639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:31.798195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473564+0000","last_change":"2024-09-17T02:40:52.222569+0000","last_active":"2024-09-17T02:40:57.473564+0000","last_peered":"2024-09-17T02:40:57.473564+0000","last_clean":"2024-09-17T02:40:57.473564+0000","last_became_active":"2024-09-17T02:40:52.222228+0000","last_became_peered":"2024-09-17T02:40:52.222228+0000","last_unstale":"2024-09-17T02:40:57.473564+0000","last_undegraded":"2024-09-17T02:40:57.473564+0000","last_fullsized":"2024-09-17T02:40:57.473564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:39.361971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289288+0000","last_change":"2024-09-17T02:40:52.236488+0000","last_active":"2024-09-17T02:40:57.289288+0000","last_peered":"2024-09-17T02:40:57.289288+0000","last_clean":"2024-09-17T02:40:57.289288+0000","last_became_active":"2024-09-17T02:40:52.236171+0000","last_became_peered":"2024-09-17T02:40:52.236171+0000","last_unstale":"2024-09-17T02:40:57.289288+0000","last_undegraded":"2024-09-17T02:40:57.289288+0000","last_fullsized":"2024-09-17T02:40:57.289288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:08.415078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287712+0000","last_change":"2024-09-17T02:40:52.217993+0000","last_active":"2024-09-17T02:40:57.287712+0000","last_peered":"2024-09-17T02:40:57.287712+0000","last_clean":"2024-09-17T02:40:57.287712+0000","last_became_active":"2024-09-17T02:40:52.216716+0000","last_became_peered":"2024-09-17T02:40:52.216716+0000","last_unstale":"2024-09-17T02:40:57.287712+0000","last_undegraded":"2024-09-17T02:40:57.287712+0000","last_fullsized":"2024-09-17T02:40:57.287712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:06.000439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472031+0000","last_change":"2024-09-17T02:40:52.245158+0000","last_active":"2024-09-17T02:40:57.472031+0000","last_peered":"2024-09-17T02:40:57.472031+0000","last_clean":"2024-09-17T02:40:57.472031+0000","last_became_active":"2024-09-17T02:40:52.244333+0000","last_became_peered":"2024-09-17T02:40:52.244333+0000","last_unstale":"2024-09-17T02:40:57.472031+0000","last_undegraded":"2024-09-17T02:40:57.472031+0000","last_fullsized":"2024-09-17T02:40:57.472031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:06.006731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473931+0000","last_change":"2024-09-17T02:40:52.228925+0000","last_active":"2024-09-17T02:40:57.473931+0000","last_peered":"2024-09-17T02:40:57.473931+0000","last_clean":"2024-09-17T02:40:57.473931+0000","last_became_active":"2024-09-17T02:40:52.228587+0000","last_became_peered":"2024-09-17T02:40:52.228587+0000","last_unstale":"2024-09-17T02:40:57.473931+0000","last_undegraded":"2024-09-17T02:40:57.473931+0000","last_fullsized":"2024-09-17T02:40:57.473931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:33.958554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287342+0000","last_change":"2024-09-17T02:40:52.229795+0000","last_active":"2024-09-17T02:40:57.287342+0000","last_peered":"2024-09-17T02:40:57.287342+0000","last_clean":"2024-09-17T02:40:57.287342+0000","last_became_active":"2024-09-17T02:40:52.227659+0000","last_became_peered":"2024-09-17T02:40:52.227659+0000","last_unstale":"2024-09-17T02:40:57.287342+0000","last_undegraded":"2024-09-17T02:40:57.287342+0000","last_fullsized":"2024-09-17T02:40:57.287342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:36.541471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473327+0000","last_change":"2024-09-17T02:40:52.221420+0000","last_active":"2024-09-17T02:40:57.473327+0000","last_peered":"2024-09-17T02:40:57.473327+0000","last_clean":"2024-09-17T02:40:57.473327+0000","last_became_active":"2024-09-17T02:40:52.221050+0000","last_became_peered":"2024-09-17T02:40:52.221050+0000","last_unstale":"2024-09-17T02:40:57.473327+0000","last_undegraded":"2024-09-17T02:40:57.473327+0000","last_fullsized":"2024-09-17T02:40:57.473327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:47.715501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470588+0000","last_change":"2024-09-17T02:40:52.237946+0000","last_active":"2024-09-17T02:40:57.470588+0000","last_peered":"2024-09-17T02:40:57.470588+0000","last_clean":"2024-09-17T02:40:57.470588+0000","last_became_active":"2024-09-17T02:40:52.237376+0000","last_became_peered":"2024-09-17T02:40:52.237376+0000","last_unstale":"2024-09-17T02:40:57.470588+0000","last_undegraded":"2024-09-17T02:40:57.470588+0000","last_fullsized":"2024-09-17T02:40:57.470588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:31.107108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287955+0000","last_change":"2024-09-17T02:40:52.221072+0000","last_active":"2024-09-17T02:40:57.287955+0000","last_peered":"2024-09-17T02:40:57.287955+0000","last_clean":"2024-09-17T02:40:57.287955+0000","last_became_active":"2024-09-17T02:40:52.220697+0000","last_became_peered":"2024-09-17T02:40:52.220697+0000","last_unstale":"2024-09-17T02:40:57.287955+0000","last_undegraded":"2024-09-17T02:40:57.287955+0000","last_fullsized":"2024-09-17T02:40:57.287955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:02.681569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472298+0000","last_change":"2024-09-17T02:40:52.249011+0000","last_active":"2024-09-17T02:40:57.472298+0000","last_peered":"2024-09-17T02:40:57.472298+0000","last_clean":"2024-09-17T02:40:57.472298+0000","last_became_active":"2024-09-17T02:40:52.247429+0000","last_became_peered":"2024-09-17T02:40:52.247429+0000","last_unstale":"2024-09-17T02:40:57.472298+0000","last_undegraded":"2024-09-17T02:40:57.472298+0000","last_fullsized":"2024-09-17T02:40:57.472298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:56.223383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288383+0000","last_change":"2024-09-17T02:40:52.243578+0000","last_active":"2024-09-17T02:40:57.288383+0000","last_peered":"2024-09-17T02:40:57.288383+0000","last_clean":"2024-09-17T02:40:57.288383+0000","last_became_active":"2024-09-17T02:40:52.243106+0000","last_became_peered":"2024-09-17T02:40:52.243106+0000","last_unstale":"2024-09-17T02:40:57.288383+0000","last_undegraded":"2024-09-17T02:40:57.288383+0000","last_fullsized":"2024-09-17T02:40:57.288383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:22.663615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475176+0000","last_change":"2024-09-17T02:40:52.222351+0000","last_active":"2024-09-17T02:40:57.475176+0000","last_peered":"2024-09-17T02:40:57.475176+0000","last_clean":"2024-09-17T02:40:57.475176+0000","last_became_active":"2024-09-17T02:40:52.222043+0000","last_became_peered":"2024-09-17T02:40:52.222043+0000","last_unstale":"2024-09-17T02:40:57.475176+0000","last_undegraded":"2024-09-17T02:40:57.475176+0000","last_fullsized":"2024-09-17T02:40:57.475176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:21.583058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288320+0000","last_change":"2024-09-17T02:40:52.256413+0000","last_active":"2024-09-17T02:40:57.288320+0000","last_peered":"2024-09-17T02:40:57.288320+0000","last_clean":"2024-09-17T02:40:57.288320+0000","last_became_active":"2024-09-17T02:40:52.256035+0000","last_became_peered":"2024-09-17T02:40:52.256035+0000","last_unstale":"2024-09-17T02:40:57.288320+0000","last_undegraded":"2024-09-17T02:40:57.288320+0000","last_fullsized":"2024-09-17T02:40:57.288320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:59.172988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287206+0000","last_change":"2024-09-17T02:40:52.236686+0000","last_active":"2024-09-17T02:40:57.287206+0000","last_peered":"2024-09-17T02:40:57.287206+0000","last_clean":"2024-09-17T02:40:57.287206+0000","last_became_active":"2024-09-17T02:40:52.236194+0000","last_became_peered":"2024-09-17T02:40:52.236194+0000","last_unstale":"2024-09-17T02:40:57.287206+0000","last_undegraded":"2024-09-17T02:40:57.287206+0000","last_fullsized":"2024-09-17T02:40:57.287206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:55.015415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472841+0000","last_change":"2024-09-17T02:40:52.228032+0000","last_active":"2024-09-17T02:40:57.472841+0000","last_peered":"2024-09-17T02:40:57.472841+0000","last_clean":"2024-09-17T02:40:57.472841+0000","last_became_active":"2024-09-17T02:40:52.225817+0000","last_became_peered":"2024-09-17T02:40:52.225817+0000","last_unstale":"2024-09-17T02:40:57.472841+0000","last_undegraded":"2024-09-17T02:40:57.472841+0000","last_fullsized":"2024-09-17T02:40:57.472841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:34.058125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287107+0000","last_change":"2024-09-17T02:40:52.217145+0000","last_active":"2024-09-17T02:40:57.287107+0000","last_peered":"2024-09-17T02:40:57.287107+0000","last_clean":"2024-09-17T02:40:57.287107+0000","last_became_active":"2024-09-17T02:40:52.215909+0000","last_became_peered":"2024-09-17T02:40:52.215909+0000","last_unstale":"2024-09-17T02:40:57.287107+0000","last_undegraded":"2024-09-17T02:40:57.287107+0000","last_fullsized":"2024-09-17T02:40:57.287107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:28.728656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288943+0000","last_change":"2024-09-17T02:40:52.224668+0000","last_active":"2024-09-17T02:40:57.288943+0000","last_peered":"2024-09-17T02:40:57.288943+0000","last_clean":"2024-09-17T02:40:57.288943+0000","last_became_active":"2024-09-17T02:40:52.224273+0000","last_became_peered":"2024-09-17T02:40:52.224273+0000","last_unstale":"2024-09-17T02:40:57.288943+0000","last_undegraded":"2024-09-17T02:40:57.288943+0000","last_fullsized":"2024-09-17T02:40:57.288943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:21.184527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472191+0000","last_change":"2024-09-17T02:40:52.252355+0000","last_active":"2024-09-17T02:40:57.472191+0000","last_peered":"2024-09-17T02:40:57.472191+0000","last_clean":"2024-09-17T02:40:57.472191+0000","last_became_active":"2024-09-17T02:40:52.251950+0000","last_became_peered":"2024-09-17T02:40:52.251950+0000","last_unstale":"2024-09-17T02:40:57.472191+0000","last_undegraded":"2024-09-17T02:40:57.472191+0000","last_fullsized":"2024-09-17T02:40:57.472191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:50.916246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471773+0000","last_change":"2024-09-17T02:40:52.244189+0000","last_active":"2024-09-17T02:40:57.471773+0000","last_peered":"2024-09-17T02:40:57.471773+0000","last_clean":"2024-09-17T02:40:57.471773+0000","last_became_active":"2024-09-17T02:40:52.243891+0000","last_became_peered":"2024-09-17T02:40:52.243891+0000","last_unstale":"2024-09-17T02:40:57.471773+0000","last_undegraded":"2024-09-17T02:40:57.471773+0000","last_fullsized":"2024-09-17T02:40:57.471773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:00.308180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471829+0000","last_change":"2024-09-17T02:40:52.237299+0000","last_active":"2024-09-17T02:40:57.471829+0000","last_peered":"2024-09-17T02:40:57.471829+0000","last_clean":"2024-09-17T02:40:57.471829+0000","last_became_active":"2024-09-17T02:40:52.236928+0000","last_became_peered":"2024-09-17T02:40:52.236928+0000","last_unstale":"2024-09-17T02:40:57.471829+0000","last_undegraded":"2024-09-17T02:40:57.471829+0000","last_fullsized":"2024-09-17T02:40:57.471829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:03.755555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471259+0000","last_change":"2024-09-17T02:40:52.247989+0000","last_active":"2024-09-17T02:40:57.471259+0000","last_peered":"2024-09-17T02:40:57.471259+0000","last_clean":"2024-09-17T02:40:57.471259+0000","last_became_active":"2024-09-17T02:40:52.247534+0000","last_became_peered":"2024-09-17T02:40:52.247534+0000","last_unstale":"2024-09-17T02:40:57.471259+0000","last_undegraded":"2024-09-17T02:40:57.471259+0000","last_fullsized":"2024-09-17T02:40:57.471259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:14.502507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471636+0000","last_change":"2024-09-17T02:40:52.228868+0000","last_active":"2024-09-17T02:40:57.471636+0000","last_peered":"2024-09-17T02:40:57.471636+0000","last_clean":"2024-09-17T02:40:57.471636+0000","last_became_active":"2024-09-17T02:40:52.227927+0000","last_became_peered":"2024-09-17T02:40:52.227927+0000","last_unstale":"2024-09-17T02:40:57.471636+0000","last_undegraded":"2024-09-17T02:40:57.471636+0000","last_fullsized":"2024-09-17T02:40:57.471636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:06.283756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470545+0000","last_change":"2024-09-17T02:40:52.220236+0000","last_active":"2024-09-17T02:40:57.470545+0000","last_peered":"2024-09-17T02:40:57.470545+0000","last_clean":"2024-09-17T02:40:57.470545+0000","last_became_active":"2024-09-17T02:40:52.219510+0000","last_became_peered":"2024-09-17T02:40:52.219510+0000","last_unstale":"2024-09-17T02:40:57.470545+0000","last_undegraded":"2024-09-17T02:40:57.470545+0000","last_fullsized":"2024-09-17T02:40:57.470545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:33.958554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471108+0000","last_change":"2024-09-17T02:40:52.218799+0000","last_active":"2024-09-17T02:40:57.471108+0000","last_peered":"2024-09-17T02:40:57.471108+0000","last_clean":"2024-09-17T02:40:57.471108+0000","last_became_active":"2024-09-17T02:40:52.217054+0000","last_became_peered":"2024-09-17T02:40:52.217054+0000","last_unstale":"2024-09-17T02:40:57.471108+0000","last_undegraded":"2024-09-17T02:40:57.471108+0000","last_fullsized":"2024-09-17T02:40:57.471108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:21.583058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472762+0000","last_change":"2024-09-17T02:40:52.232583+0000","last_active":"2024-09-17T02:40:57.472762+0000","last_peered":"2024-09-17T02:40:57.472762+0000","last_clean":"2024-09-17T02:40:57.472762+0000","last_became_active":"2024-09-17T02:40:52.231933+0000","last_became_peered":"2024-09-17T02:40:52.231933+0000","last_unstale":"2024-09-17T02:40:57.472762+0000","last_undegraded":"2024-09-17T02:40:57.472762+0000","last_fullsized":"2024-09-17T02:40:57.472762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:41.461773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471574+0000","last_change":"2024-09-17T02:40:52.218335+0000","last_active":"2024-09-17T02:40:57.471574+0000","last_peered":"2024-09-17T02:40:57.471574+0000","last_clean":"2024-09-17T02:40:57.471574+0000","last_became_active":"2024-09-17T02:40:52.217008+0000","last_became_peered":"2024-09-17T02:40:52.217008+0000","last_unstale":"2024-09-17T02:40:57.471574+0000","last_undegraded":"2024-09-17T02:40:57.471574+0000","last_fullsized":"2024-09-17T02:40:57.471574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:19.064290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472531+0000","last_change":"2024-09-17T02:40:52.224771+0000","last_active":"2024-09-17T02:40:57.472531+0000","last_peered":"2024-09-17T02:40:57.472531+0000","last_clean":"2024-09-17T02:40:57.472531+0000","last_became_active":"2024-09-17T02:40:52.222947+0000","last_became_peered":"2024-09-17T02:40:52.222947+0000","last_unstale":"2024-09-17T02:40:57.472531+0000","last_undegraded":"2024-09-17T02:40:57.472531+0000","last_fullsized":"2024-09-17T02:40:57.472531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:54.216006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474943+0000","last_change":"2024-09-17T02:40:52.228603+0000","last_active":"2024-09-17T02:40:57.474943+0000","last_peered":"2024-09-17T02:40:57.474943+0000","last_clean":"2024-09-17T02:40:57.474943+0000","last_became_active":"2024-09-17T02:40:52.225545+0000","last_became_peered":"2024-09-17T02:40:52.225545+0000","last_unstale":"2024-09-17T02:40:57.474943+0000","last_undegraded":"2024-09-17T02:40:57.474943+0000","last_fullsized":"2024-09-17T02:40:57.474943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:56.223383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471852+0000","last_change":"2024-09-17T02:40:52.250495+0000","last_active":"2024-09-17T02:40:57.471852+0000","last_peered":"2024-09-17T02:40:57.471852+0000","last_clean":"2024-09-17T02:40:57.471852+0000","last_became_active":"2024-09-17T02:40:52.250095+0000","last_became_peered":"2024-09-17T02:40:52.250095+0000","last_unstale":"2024-09-17T02:40:57.471852+0000","last_undegraded":"2024-09-17T02:40:57.471852+0000","last_fullsized":"2024-09-17T02:40:57.471852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:58.131959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:41:14.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:41:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 -- 172.21.15.131:0/249136024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2000ced00 msgr2=0x7fc20006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 --2- 172.21.15.131:0/249136024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2000ced00 0x7fc20006dea0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc2000b61b0 tx=0x7fc1f001c8c0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 -- 172.21.15.131:0/249136024 shutdown_connections 2024-09-17T02:41:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 --2- 172.21.15.131:0/249136024 >> v2:172.21.15.189:3300/0 conn(0x7fc20006e3e0 0x7fc20015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 --2- 172.21.15.131:0/249136024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2000ced00 0x7fc20006dea0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 -- 172.21.15.131:0/249136024 >> 172.21.15.131:0/249136024 conn(0x7fc20006eea0 msgr2=0x7fc20006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 -- 172.21.15.131:0/249136024 shutdown_connections 2024-09-17T02:41:14.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 -- 172.21.15.131:0/249136024 wait complete. 2024-09-17T02:41:14.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 Processor -- start 2024-09-17T02:41:14.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 -- start start 2024-09-17T02:41:14.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc20006e3e0 0x7fc2001fa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2000ced00 0x7fc200153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20015c3e0 0x7fc2000cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc200168b60 con 0x7fc20015c3e0 2024-09-17T02:41:14.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc2001689e0 con 0x7fc2000ced00 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc204843640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc200168ce0 con 0x7fc20006e3e0 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc1fed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20015c3e0 0x7fc2000cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc1fed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20015c3e0 0x7fc2000cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57648/0 (socket says 172.21.15.131:57648) 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc1fed9d640 1 -- 172.21.15.131:0/2424200478 learned_addr learned my addr 172.21.15.131:0/2424200478 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc1fdd9b640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2000ced00 0x7fc200153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc1fe59c640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc20006e3e0 0x7fc2001fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc1fed9d640 1 -- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc20006e3e0 msgr2=0x7fc2001fa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.526+0000 7fc1fed9d640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc20006e3e0 0x7fc2001fa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1fed9d640 1 -- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2000ced00 msgr2=0x7fc200153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1fed9d640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2000ced00 0x7fc200153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1fed9d640 1 -- 172.21.15.131:0/2424200478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2001fae40 con 0x7fc20015c3e0 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1fed9d640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20015c3e0 0x7fc2000cdbf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800c730 tx=0x7fc1f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1f6ffd640 1 -- 172.21.15.131:0/2424200478 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8003b80 con 0x7fc20015c3e0 2024-09-17T02:41:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1f6ffd640 1 -- 172.21.15.131:0/2424200478 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1f8003d20 con 0x7fc20015c3e0 2024-09-17T02:41:14.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1f6ffd640 1 -- 172.21.15.131:0/2424200478 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8005330 con 0x7fc20015c3e0 2024-09-17T02:41:14.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc204843640 1 -- 172.21.15.131:0/2424200478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2000ac670 con 0x7fc20015c3e0 2024-09-17T02:41:14.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc204843640 1 -- 172.21.15.131:0/2424200478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2000aad10 con 0x7fc20015c3e0 2024-09-17T02:41:14.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc204843640 1 -- 172.21.15.131:0/2424200478 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc20015bea0 con 0x7fc20015c3e0 2024-09-17T02:41:14.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1f6ffd640 1 -- 172.21.15.131:0/2424200478 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc1f80056e0 con 0x7fc20015c3e0 2024-09-17T02:41:14.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1f6ffd640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 0x7fc1e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1f6ffd640 1 -- 172.21.15.131:0/2424200478 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc1f8094640 con 0x7fc20015c3e0 2024-09-17T02:41:14.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1fe59c640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 0x7fc1e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.530+0000 7fc1fe59c640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 0x7fc1e0075710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1f00af000 tx=0x7fc1f001b100 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:14.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.534+0000 7fc1f6ffd640 1 -- 172.21.15.131:0/2424200478 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc2001fae40 con 0x7fc20015c3e0 2024-09-17T02:41:14.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.686+0000 7fc204843640 1 -- 172.21.15.131:0/2424200478 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc20006dec0 con 0x7fc1e0073200 2024-09-17T02:41:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.706+0000 7fc1f6ffd640 1 -- 172.21.15.131:0/2424200478 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332519 (secure 0 0 0) 0x7fc20006dec0 con 0x7fc1e0073200 2024-09-17T02:41:14.707 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:14.709 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:41:14.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 -- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 msgr2=0x7fc1e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 0x7fc1e0075710 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc1f00af000 tx=0x7fc1f001b100 comp rx=0 tx=0).stop 2024-09-17T02:41:14.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 -- 172.21.15.131:0/2424200478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20015c3e0 msgr2=0x7fc2000cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20015c3e0 0x7fc2000cdbf0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc1f800c730 tx=0x7fc1f800cbf0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 -- 172.21.15.131:0/2424200478 shutdown_connections 2024-09-17T02:41:14.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 0x7fc1e0075710 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20015c3e0 0x7fc2000cdbf0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2000ced00 0x7fc200153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 --2- 172.21.15.131:0/2424200478 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc20006e3e0 0x7fc2001fa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 -- 172.21.15.131:0/2424200478 >> 172.21.15.131:0/2424200478 conn(0x7fc20006eea0 msgr2=0x7fc2000b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:14.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 -- 172.21.15.131:0/2424200478 shutdown_connections 2024-09-17T02:41:14.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.710+0000 7fc204843640 1 -- 172.21.15.131:0/2424200478 wait complete. 2024-09-17T02:41:14.728 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-17T02:41:13.122611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219564,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755156,"statfs":{"total":773094113280,"available":772869279744,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005695"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290027+0000","last_change":"2024-09-17T02:40:51.241640+0000","last_active":"2024-09-17T02:40:57.290027+0000","last_peered":"2024-09-17T02:40:57.290027+0000","last_clean":"2024-09-17T02:40:57.290027+0000","last_became_active":"2024-09-17T02:40:51.240675+0000","last_became_peered":"2024-09-17T02:40:51.240675+0000","last_unstale":"2024-09-17T02:40:57.290027+0000","last_undegraded":"2024-09-17T02:40:57.290027+0000","last_fullsized":"2024-09-17T02:40:57.290027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475107+0000","last_change":"2024-09-17T02:40:52.228237+0000","last_active":"2024-09-17T02:40:57.475107+0000","last_peered":"2024-09-17T02:40:57.475107+0000","last_clean":"2024-09-17T02:40:57.475107+0000","last_became_active":"2024-09-17T02:40:52.225329+0000","last_became_peered":"2024-09-17T02:40:52.225329+0000","last_unstale":"2024-09-17T02:40:57.475107+0000","last_undegraded":"2024-09-17T02:40:57.475107+0000","last_fullsized":"2024-09-17T02:40:57.475107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:55.015415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471940+0000","last_change":"2024-09-17T02:40:56.353097+0000","last_active":"2024-09-17T02:40:57.471940+0000","last_peered":"2024-09-17T02:40:57.471940+0000","last_clean":"2024-09-17T02:40:57.471940+0000","last_became_active":"2024-09-17T02:40:56.351847+0000","last_became_peered":"2024-09-17T02:40:56.351847+0000","last_unstale":"2024-09-17T02:40:57.471940+0000","last_undegraded":"2024-09-17T02:40:57.471940+0000","last_fullsized":"2024-09-17T02:40:57.471940+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:06.760578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475261+0000","last_change":"2024-09-17T02:40:51.243590+0000","last_active":"2024-09-17T02:40:57.475261+0000","last_peered":"2024-09-17T02:40:57.475261+0000","last_clean":"2024-09-17T02:40:57.475261+0000","last_became_active":"2024-09-17T02:40:51.242684+0000","last_became_peered":"2024-09-17T02:40:51.242684+0000","last_unstale":"2024-09-17T02:40:57.475261+0000","last_undegraded":"2024-09-17T02:40:57.475261+0000","last_fullsized":"2024-09-17T02:40:57.475261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474246+0000","last_change":"2024-09-17T02:40:52.207992+0000","last_active":"2024-09-17T02:40:57.474246+0000","last_peered":"2024-09-17T02:40:57.474246+0000","last_clean":"2024-09-17T02:40:57.474246+0000","last_became_active":"2024-09-17T02:40:52.207568+0000","last_became_peered":"2024-09-17T02:40:52.207568+0000","last_unstale":"2024-09-17T02:40:57.474246+0000","last_undegraded":"2024-09-17T02:40:57.474246+0000","last_fullsized":"2024-09-17T02:40:57.474246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:01.844374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470155+0000","last_change":"2024-09-17T02:40:56.339588+0000","last_active":"2024-09-17T02:40:57.470155+0000","last_peered":"2024-09-17T02:40:57.470155+0000","last_clean":"2024-09-17T02:40:57.470155+0000","last_became_active":"2024-09-17T02:40:56.338569+0000","last_became_peered":"2024-09-17T02:40:56.338569+0000","last_unstale":"2024-09-17T02:40:57.470155+0000","last_undegraded":"2024-09-17T02:40:57.470155+0000","last_fullsized":"2024-09-17T02:40:57.470155+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:54.995255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472080+0000","last_change":"2024-09-17T02:40:51.225527+0000","last_active":"2024-09-17T02:40:57.472080+0000","last_peered":"2024-09-17T02:40:57.472080+0000","last_clean":"2024-09-17T02:40:57.472080+0000","last_became_active":"2024-09-17T02:40:51.224640+0000","last_became_peered":"2024-09-17T02:40:51.224640+0000","last_unstale":"2024-09-17T02:40:57.472080+0000","last_undegraded":"2024-09-17T02:40:57.472080+0000","last_fullsized":"2024-09-17T02:40:57.472080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:07.028967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470812+0000","last_change":"2024-09-17T02:40:52.243749+0000","last_active":"2024-09-17T02:40:57.470812+0000","last_peered":"2024-09-17T02:40:57.470812+0000","last_clean":"2024-09-17T02:40:57.470812+0000","last_became_active":"2024-09-17T02:40:52.243361+0000","last_became_peered":"2024-09-17T02:40:52.243361+0000","last_unstale":"2024-09-17T02:40:57.470812+0000","last_undegraded":"2024-09-17T02:40:57.470812+0000","last_fullsized":"2024-09-17T02:40:57.470812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:01.471406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471331+0000","last_change":"2024-09-17T02:40:56.330627+0000","last_active":"2024-09-17T02:40:57.471331+0000","last_peered":"2024-09-17T02:40:57.471331+0000","last_clean":"2024-09-17T02:40:57.471331+0000","last_became_active":"2024-09-17T02:40:56.330123+0000","last_became_peered":"2024-09-17T02:40:56.330123+0000","last_unstale":"2024-09-17T02:40:57.471331+0000","last_undegraded":"2024-09-17T02:40:57.471331+0000","last_fullsized":"2024-09-17T02:40:57.471331+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:30.712481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472462+0000","last_change":"2024-09-17T02:40:51.220105+0000","last_active":"2024-09-17T02:40:57.472462+0000","last_peered":"2024-09-17T02:40:57.472462+0000","last_clean":"2024-09-17T02:40:57.472462+0000","last_became_active":"2024-09-17T02:40:51.216247+0000","last_became_peered":"2024-09-17T02:40:51.216247+0000","last_unstale":"2024-09-17T02:40:57.472462+0000","last_undegraded":"2024-09-17T02:40:57.472462+0000","last_fullsized":"2024-09-17T02:40:57.472462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:30.096213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288051+0000","last_change":"2024-09-17T02:40:52.255870+0000","last_active":"2024-09-17T02:40:57.288051+0000","last_peered":"2024-09-17T02:40:57.288051+0000","last_clean":"2024-09-17T02:40:57.288051+0000","last_became_active":"2024-09-17T02:40:52.255454+0000","last_became_peered":"2024-09-17T02:40:52.255454+0000","last_unstale":"2024-09-17T02:40:57.288051+0000","last_undegraded":"2024-09-17T02:40:57.288051+0000","last_fullsized":"2024-09-17T02:40:57.288051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:35.834666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470627+0000","last_change":"2024-09-17T02:40:56.339659+0000","last_active":"2024-09-17T02:40:57.470627+0000","last_peered":"2024-09-17T02:40:57.470627+0000","last_clean":"2024-09-17T02:40:57.470627+0000","last_became_active":"2024-09-17T02:40:56.339130+0000","last_became_peered":"2024-09-17T02:40:56.339130+0000","last_unstale":"2024-09-17T02:40:57.470627+0000","last_undegraded":"2024-09-17T02:40:57.470627+0000","last_fullsized":"2024-09-17T02:40:57.470627+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:38.223653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287802+0000","last_change":"2024-09-17T02:40:52.242899+0000","last_active":"2024-09-17T02:40:57.287802+0000","last_peered":"2024-09-17T02:40:57.287802+0000","last_clean":"2024-09-17T02:40:57.287802+0000","last_became_active":"2024-09-17T02:40:52.242570+0000","last_became_peered":"2024-09-17T02:40:52.242570+0000","last_unstale":"2024-09-17T02:40:57.287802+0000","last_undegraded":"2024-09-17T02:40:57.287802+0000","last_fullsized":"2024-09-17T02:40:57.287802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:45.600475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288528+0000","last_change":"2024-09-17T02:40:51.212048+0000","last_active":"2024-09-17T02:40:57.288528+0000","last_peered":"2024-09-17T02:40:57.288528+0000","last_clean":"2024-09-17T02:40:57.288528+0000","last_became_active":"2024-09-17T02:40:51.211531+0000","last_became_peered":"2024-09-17T02:40:51.211531+0000","last_unstale":"2024-09-17T02:40:57.288528+0000","last_undegraded":"2024-09-17T02:40:57.288528+0000","last_fullsized":"2024-09-17T02:40:57.288528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:10.284709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287732+0000","last_change":"2024-09-17T02:40:56.332707+0000","last_active":"2024-09-17T02:40:57.287732+0000","last_peered":"2024-09-17T02:40:57.287732+0000","last_clean":"2024-09-17T02:40:57.287732+0000","last_became_active":"2024-09-17T02:40:56.332260+0000","last_became_peered":"2024-09-17T02:40:56.332260+0000","last_unstale":"2024-09-17T02:40:57.287732+0000","last_undegraded":"2024-09-17T02:40:57.287732+0000","last_fullsized":"2024-09-17T02:40:57.287732+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:53.783346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472671+0000","last_change":"2024-09-17T02:40:52.218705+0000","last_active":"2024-09-17T02:40:57.472671+0000","last_peered":"2024-09-17T02:40:57.472671+0000","last_clean":"2024-09-17T02:40:57.472671+0000","last_became_active":"2024-09-17T02:40:52.217762+0000","last_became_peered":"2024-09-17T02:40:52.217762+0000","last_unstale":"2024-09-17T02:40:57.472671+0000","last_undegraded":"2024-09-17T02:40:57.472671+0000","last_fullsized":"2024-09-17T02:40:57.472671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:11.295603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287768+0000","last_change":"2024-09-17T02:40:51.237760+0000","last_active":"2024-09-17T02:40:57.287768+0000","last_peered":"2024-09-17T02:40:57.287768+0000","last_clean":"2024-09-17T02:40:57.287768+0000","last_became_active":"2024-09-17T02:40:51.236973+0000","last_became_peered":"2024-09-17T02:40:51.236973+0000","last_unstale":"2024-09-17T02:40:57.287768+0000","last_undegraded":"2024-09-17T02:40:57.287768+0000","last_fullsized":"2024-09-17T02:40:57.287768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:22.634046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471688+0000","last_change":"2024-09-17T02:40:56.355680+0000","last_active":"2024-09-17T02:40:57.471688+0000","last_peered":"2024-09-17T02:40:57.471688+0000","last_clean":"2024-09-17T02:40:57.471688+0000","last_became_active":"2024-09-17T02:40:56.355224+0000","last_became_peered":"2024-09-17T02:40:56.355224+0000","last_unstale":"2024-09-17T02:40:57.471688+0000","last_undegraded":"2024-09-17T02:40:57.471688+0000","last_fullsized":"2024-09-17T02:40:57.471688+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:49.679484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290367+0000","last_change":"2024-09-17T02:40:52.220376+0000","last_active":"2024-09-17T02:40:57.290367+0000","last_peered":"2024-09-17T02:40:57.290367+0000","last_clean":"2024-09-17T02:40:57.290367+0000","last_became_active":"2024-09-17T02:40:52.220031+0000","last_became_peered":"2024-09-17T02:40:52.220031+0000","last_unstale":"2024-09-17T02:40:57.290367+0000","last_undegraded":"2024-09-17T02:40:57.290367+0000","last_fullsized":"2024-09-17T02:40:57.290367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:04.162473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291288+0000","last_change":"2024-09-17T02:40:51.240925+0000","last_active":"2024-09-17T02:40:57.291288+0000","last_peered":"2024-09-17T02:40:57.291288+0000","last_clean":"2024-09-17T02:40:57.291288+0000","last_became_active":"2024-09-17T02:40:51.233638+0000","last_became_peered":"2024-09-17T02:40:51.233638+0000","last_unstale":"2024-09-17T02:40:57.291288+0000","last_undegraded":"2024-09-17T02:40:57.291288+0000","last_fullsized":"2024-09-17T02:40:57.291288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:39.387068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475651+0000","last_change":"2024-09-17T02:40:56.339397+0000","last_active":"2024-09-17T02:40:57.475651+0000","last_peered":"2024-09-17T02:40:57.475651+0000","last_clean":"2024-09-17T02:40:57.475651+0000","last_became_active":"2024-09-17T02:40:56.338591+0000","last_became_peered":"2024-09-17T02:40:56.338591+0000","last_unstale":"2024-09-17T02:40:57.475651+0000","last_undegraded":"2024-09-17T02:40:57.475651+0000","last_fullsized":"2024-09-17T02:40:57.475651+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:10.362765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287431+0000","last_change":"2024-09-17T02:40:52.245352+0000","last_active":"2024-09-17T02:40:57.287431+0000","last_peered":"2024-09-17T02:40:57.287431+0000","last_clean":"2024-09-17T02:40:57.287431+0000","last_became_active":"2024-09-17T02:40:52.244938+0000","last_became_peered":"2024-09-17T02:40:52.244938+0000","last_unstale":"2024-09-17T02:40:57.287431+0000","last_undegraded":"2024-09-17T02:40:57.287431+0000","last_fullsized":"2024-09-17T02:40:57.287431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:07.474908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.469729+0000","last_change":"2024-09-17T02:40:51.220831+0000","last_active":"2024-09-17T02:40:57.469729+0000","last_peered":"2024-09-17T02:40:57.469729+0000","last_clean":"2024-09-17T02:40:57.469729+0000","last_became_active":"2024-09-17T02:40:51.220253+0000","last_became_peered":"2024-09-17T02:40:51.220253+0000","last_unstale":"2024-09-17T02:40:57.469729+0000","last_undegraded":"2024-09-17T02:40:57.469729+0000","last_fullsized":"2024-09-17T02:40:57.469729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471876+0000","last_change":"2024-09-17T02:40:56.333001+0000","last_active":"2024-09-17T02:40:57.471876+0000","last_peered":"2024-09-17T02:40:57.471876+0000","last_clean":"2024-09-17T02:40:57.471876+0000","last_became_active":"2024-09-17T02:40:56.332421+0000","last_became_peered":"2024-09-17T02:40:56.332421+0000","last_unstale":"2024-09-17T02:40:57.471876+0000","last_undegraded":"2024-09-17T02:40:57.471876+0000","last_fullsized":"2024-09-17T02:40:57.471876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:58.168471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291218+0000","last_change":"2024-09-17T02:40:52.215438+0000","last_active":"2024-09-17T02:40:57.291218+0000","last_peered":"2024-09-17T02:40:57.291218+0000","last_clean":"2024-09-17T02:40:57.291218+0000","last_became_active":"2024-09-17T02:40:52.215073+0000","last_became_peered":"2024-09-17T02:40:52.215073+0000","last_unstale":"2024-09-17T02:40:57.291218+0000","last_undegraded":"2024-09-17T02:40:57.291218+0000","last_fullsized":"2024-09-17T02:40:57.291218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:39.361971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288442+0000","last_change":"2024-09-17T02:40:51.226075+0000","last_active":"2024-09-17T02:40:57.288442+0000","last_peered":"2024-09-17T02:40:57.288442+0000","last_clean":"2024-09-17T02:40:57.288442+0000","last_became_active":"2024-09-17T02:40:51.225620+0000","last_became_peered":"2024-09-17T02:40:51.225620+0000","last_unstale":"2024-09-17T02:40:57.288442+0000","last_undegraded":"2024-09-17T02:40:57.288442+0000","last_fullsized":"2024-09-17T02:40:57.288442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:15.238526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470178+0000","last_change":"2024-09-17T02:40:56.306280+0000","last_active":"2024-09-17T02:40:57.470178+0000","last_peered":"2024-09-17T02:40:57.470178+0000","last_clean":"2024-09-17T02:40:57.470178+0000","last_became_active":"2024-09-17T02:40:56.305348+0000","last_became_peered":"2024-09-17T02:40:56.305348+0000","last_unstale":"2024-09-17T02:40:57.470178+0000","last_undegraded":"2024-09-17T02:40:57.470178+0000","last_fullsized":"2024-09-17T02:40:57.470178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:58.295015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289748+0000","last_change":"2024-09-17T02:40:52.217580+0000","last_active":"2024-09-17T02:40:57.289748+0000","last_peered":"2024-09-17T02:40:57.289748+0000","last_clean":"2024-09-17T02:40:57.289748+0000","last_became_active":"2024-09-17T02:40:52.217246+0000","last_became_peered":"2024-09-17T02:40:52.217246+0000","last_unstale":"2024-09-17T02:40:57.289748+0000","last_undegraded":"2024-09-17T02:40:57.289748+0000","last_fullsized":"2024-09-17T02:40:57.289748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:25.696012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291979+0000","last_change":"2024-09-17T02:40:51.235278+0000","last_active":"2024-09-17T02:40:57.291979+0000","last_peered":"2024-09-17T02:40:57.291979+0000","last_clean":"2024-09-17T02:40:57.291979+0000","last_became_active":"2024-09-17T02:40:51.234012+0000","last_became_peered":"2024-09-17T02:40:51.234012+0000","last_unstale":"2024-09-17T02:40:57.291979+0000","last_undegraded":"2024-09-17T02:40:57.291979+0000","last_fullsized":"2024-09-17T02:40:57.291979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:07.028967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290548+0000","last_change":"2024-09-17T02:40:56.306563+0000","last_active":"2024-09-17T02:40:57.290548+0000","last_peered":"2024-09-17T02:40:57.290548+0000","last_clean":"2024-09-17T02:40:57.290548+0000","last_became_active":"2024-09-17T02:40:56.303819+0000","last_became_peered":"2024-09-17T02:40:56.303819+0000","last_unstale":"2024-09-17T02:40:57.290548+0000","last_undegraded":"2024-09-17T02:40:57.290548+0000","last_fullsized":"2024-09-17T02:40:57.290548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:04.387188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289492+0000","last_change":"2024-09-17T02:40:52.216405+0000","last_active":"2024-09-17T02:40:57.289492+0000","last_peered":"2024-09-17T02:40:57.289492+0000","last_clean":"2024-09-17T02:40:57.289492+0000","last_became_active":"2024-09-17T02:40:52.216037+0000","last_became_peered":"2024-09-17T02:40:52.216037+0000","last_unstale":"2024-09-17T02:40:57.289492+0000","last_undegraded":"2024-09-17T02:40:57.289492+0000","last_fullsized":"2024-09-17T02:40:57.289492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:20.565582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288509+0000","last_change":"2024-09-17T02:40:51.243903+0000","last_active":"2024-09-17T02:40:57.288509+0000","last_peered":"2024-09-17T02:40:57.288509+0000","last_clean":"2024-09-17T02:40:57.288509+0000","last_became_active":"2024-09-17T02:40:51.243568+0000","last_became_peered":"2024-09-17T02:40:51.243568+0000","last_unstale":"2024-09-17T02:40:57.288509+0000","last_undegraded":"2024-09-17T02:40:57.288509+0000","last_fullsized":"2024-09-17T02:40:57.288509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:48:49.643024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287722+0000","last_change":"2024-09-17T02:40:56.340894+0000","last_active":"2024-09-17T02:40:57.287722+0000","last_peered":"2024-09-17T02:40:57.287722+0000","last_clean":"2024-09-17T02:40:57.287722+0000","last_became_active":"2024-09-17T02:40:56.338947+0000","last_became_peered":"2024-09-17T02:40:56.338947+0000","last_unstale":"2024-09-17T02:40:57.287722+0000","last_undegraded":"2024-09-17T02:40:57.287722+0000","last_fullsized":"2024-09-17T02:40:57.287722+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:48.414928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472018+0000","last_change":"2024-09-17T02:40:52.232754+0000","last_active":"2024-09-17T02:40:57.472018+0000","last_peered":"2024-09-17T02:40:57.472018+0000","last_clean":"2024-09-17T02:40:57.472018+0000","last_became_active":"2024-09-17T02:40:52.231727+0000","last_became_peered":"2024-09-17T02:40:52.231727+0000","last_unstale":"2024-09-17T02:40:57.472018+0000","last_undegraded":"2024-09-17T02:40:57.472018+0000","last_fullsized":"2024-09-17T02:40:57.472018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:11.295603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473934+0000","last_change":"2024-09-17T02:40:51.237795+0000","last_active":"2024-09-17T02:40:57.473934+0000","last_peered":"2024-09-17T02:40:57.473934+0000","last_clean":"2024-09-17T02:40:57.473934+0000","last_became_active":"2024-09-17T02:40:51.236960+0000","last_became_peered":"2024-09-17T02:40:51.236960+0000","last_unstale":"2024-09-17T02:40:57.473934+0000","last_undegraded":"2024-09-17T02:40:57.473934+0000","last_fullsized":"2024-09-17T02:40:57.473934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:39.387068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287516+0000","last_change":"2024-09-17T02:40:56.340920+0000","last_active":"2024-09-17T02:40:57.287516+0000","last_peered":"2024-09-17T02:40:57.287516+0000","last_clean":"2024-09-17T02:40:57.287516+0000","last_became_active":"2024-09-17T02:40:56.339463+0000","last_became_peered":"2024-09-17T02:40:56.339463+0000","last_unstale":"2024-09-17T02:40:57.287516+0000","last_undegraded":"2024-09-17T02:40:57.287516+0000","last_fullsized":"2024-09-17T02:40:57.287516+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:36.386068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472660+0000","last_change":"2024-09-17T02:40:52.234607+0000","last_active":"2024-09-17T02:40:57.472660+0000","last_peered":"2024-09-17T02:40:57.472660+0000","last_clean":"2024-09-17T02:40:57.472660+0000","last_became_active":"2024-09-17T02:40:52.234175+0000","last_became_peered":"2024-09-17T02:40:52.234175+0000","last_unstale":"2024-09-17T02:40:57.472660+0000","last_undegraded":"2024-09-17T02:40:57.472660+0000","last_fullsized":"2024-09-17T02:40:57.472660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:34.144644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288850+0000","last_change":"2024-09-17T02:40:51.221611+0000","last_active":"2024-09-17T02:40:57.288850+0000","last_peered":"2024-09-17T02:40:57.288850+0000","last_clean":"2024-09-17T02:40:57.288850+0000","last_became_active":"2024-09-17T02:40:51.221259+0000","last_became_peered":"2024-09-17T02:40:51.221259+0000","last_unstale":"2024-09-17T02:40:57.288850+0000","last_undegraded":"2024-09-17T02:40:57.288850+0000","last_fullsized":"2024-09-17T02:40:57.288850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:20.572163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289392+0000","last_change":"2024-09-17T02:40:56.350014+0000","last_active":"2024-09-17T02:40:57.289392+0000","last_peered":"2024-09-17T02:40:57.289392+0000","last_clean":"2024-09-17T02:40:57.289392+0000","last_became_active":"2024-09-17T02:40:56.347244+0000","last_became_peered":"2024-09-17T02:40:56.347244+0000","last_unstale":"2024-09-17T02:40:57.289392+0000","last_undegraded":"2024-09-17T02:40:57.289392+0000","last_fullsized":"2024-09-17T02:40:57.289392+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:10.648378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471945+0000","last_change":"2024-09-17T02:40:52.221771+0000","last_active":"2024-09-17T02:40:57.471945+0000","last_peered":"2024-09-17T02:40:57.471945+0000","last_clean":"2024-09-17T02:40:57.471945+0000","last_became_active":"2024-09-17T02:40:52.221461+0000","last_became_peered":"2024-09-17T02:40:52.221461+0000","last_unstale":"2024-09-17T02:40:57.471945+0000","last_undegraded":"2024-09-17T02:40:57.471945+0000","last_fullsized":"2024-09-17T02:40:57.471945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:25.696012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472767+0000","last_change":"2024-09-17T02:40:51.247013+0000","last_active":"2024-09-17T02:40:57.472767+0000","last_peered":"2024-09-17T02:40:57.472767+0000","last_clean":"2024-09-17T02:40:57.472767+0000","last_became_active":"2024-09-17T02:40:51.246163+0000","last_became_peered":"2024-09-17T02:40:51.246163+0000","last_unstale":"2024-09-17T02:40:57.472767+0000","last_undegraded":"2024-09-17T02:40:57.472767+0000","last_fullsized":"2024-09-17T02:40:57.472767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:05.797393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288261+0000","last_change":"2024-09-17T02:40:56.340614+0000","last_active":"2024-09-17T02:40:57.288261+0000","last_peered":"2024-09-17T02:40:57.288261+0000","last_clean":"2024-09-17T02:40:57.288261+0000","last_became_active":"2024-09-17T02:40:56.338340+0000","last_became_peered":"2024-09-17T02:40:56.338340+0000","last_unstale":"2024-09-17T02:40:57.288261+0000","last_undegraded":"2024-09-17T02:40:57.288261+0000","last_fullsized":"2024-09-17T02:40:57.288261+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:03.442300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.286936+0000","last_change":"2024-09-17T02:40:52.227802+0000","last_active":"2024-09-17T02:40:57.286936+0000","last_peered":"2024-09-17T02:40:57.286936+0000","last_clean":"2024-09-17T02:40:57.286936+0000","last_became_active":"2024-09-17T02:40:52.227458+0000","last_became_peered":"2024-09-17T02:40:52.227458+0000","last_unstale":"2024-09-17T02:40:57.286936+0000","last_undegraded":"2024-09-17T02:40:57.286936+0000","last_fullsized":"2024-09-17T02:40:57.286936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:21.198509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287577+0000","last_change":"2024-09-17T02:40:51.239502+0000","last_active":"2024-09-17T02:40:57.287577+0000","last_peered":"2024-09-17T02:40:57.287577+0000","last_clean":"2024-09-17T02:40:57.287577+0000","last_became_active":"2024-09-17T02:40:51.239042+0000","last_became_peered":"2024-09-17T02:40:51.239042+0000","last_unstale":"2024-09-17T02:40:57.287577+0000","last_undegraded":"2024-09-17T02:40:57.287577+0000","last_fullsized":"2024-09-17T02:40:57.287577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:04.995836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474601+0000","last_change":"2024-09-17T02:40:56.353034+0000","last_active":"2024-09-17T02:40:57.474601+0000","last_peered":"2024-09-17T02:40:57.474601+0000","last_clean":"2024-09-17T02:40:57.474601+0000","last_became_active":"2024-09-17T02:40:56.351920+0000","last_became_peered":"2024-09-17T02:40:56.351920+0000","last_unstale":"2024-09-17T02:40:57.474601+0000","last_undegraded":"2024-09-17T02:40:57.474601+0000","last_fullsized":"2024-09-17T02:40:57.474601+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:39.913675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289211+0000","last_change":"2024-09-17T02:40:52.223050+0000","last_active":"2024-09-17T02:40:57.289211+0000","last_peered":"2024-09-17T02:40:57.289211+0000","last_clean":"2024-09-17T02:40:57.289211+0000","last_became_active":"2024-09-17T02:40:52.221680+0000","last_became_peered":"2024-09-17T02:40:52.221680+0000","last_unstale":"2024-09-17T02:40:57.289211+0000","last_undegraded":"2024-09-17T02:40:57.289211+0000","last_fullsized":"2024-09-17T02:40:57.289211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:06.283756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288675+0000","last_change":"2024-09-17T02:40:51.209280+0000","last_active":"2024-09-17T02:40:57.288675+0000","last_peered":"2024-09-17T02:40:57.288675+0000","last_clean":"2024-09-17T02:40:57.288675+0000","last_became_active":"2024-09-17T02:40:51.208951+0000","last_became_peered":"2024-09-17T02:40:51.208951+0000","last_unstale":"2024-09-17T02:40:57.288675+0000","last_undegraded":"2024-09-17T02:40:57.288675+0000","last_fullsized":"2024-09-17T02:40:57.288675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:51.093529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470279+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:40:57.470279+0000","last_peered":"2024-09-17T02:40:57.470279+0000","last_clean":"2024-09-17T02:40:57.470279+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:40:57.470279+0000","last_undegraded":"2024-09-17T02:40:57.470279+0000","last_fullsized":"2024-09-17T02:40:57.470279+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288416+0000","last_change":"2024-09-17T02:40:56.306099+0000","last_active":"2024-09-17T02:40:57.288416+0000","last_peered":"2024-09-17T02:40:57.288416+0000","last_clean":"2024-09-17T02:40:57.288416+0000","last_became_active":"2024-09-17T02:40:56.305604+0000","last_became_peered":"2024-09-17T02:40:56.305604+0000","last_unstale":"2024-09-17T02:40:57.288416+0000","last_undegraded":"2024-09-17T02:40:57.288416+0000","last_fullsized":"2024-09-17T02:40:57.288416+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:22.632347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288182+0000","last_change":"2024-09-17T02:40:52.255845+0000","last_active":"2024-09-17T02:40:57.288182+0000","last_peered":"2024-09-17T02:40:57.288182+0000","last_clean":"2024-09-17T02:40:57.288182+0000","last_became_active":"2024-09-17T02:40:52.255416+0000","last_became_peered":"2024-09-17T02:40:52.255416+0000","last_unstale":"2024-09-17T02:40:57.288182+0000","last_undegraded":"2024-09-17T02:40:57.288182+0000","last_fullsized":"2024-09-17T02:40:57.288182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:40.879599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289502+0000","last_change":"2024-09-17T02:40:51.230583+0000","last_active":"2024-09-17T02:40:57.289502+0000","last_peered":"2024-09-17T02:40:57.289502+0000","last_clean":"2024-09-17T02:40:57.289502+0000","last_became_active":"2024-09-17T02:40:51.229636+0000","last_became_peered":"2024-09-17T02:40:51.229636+0000","last_unstale":"2024-09-17T02:40:57.289502+0000","last_undegraded":"2024-09-17T02:40:57.289502+0000","last_fullsized":"2024-09-17T02:40:57.289502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:25.403066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470871+0000","last_change":"2024-09-17T02:40:56.334178+0000","last_active":"2024-09-17T02:40:57.470871+0000","last_peered":"2024-09-17T02:40:57.470871+0000","last_clean":"2024-09-17T02:40:57.470871+0000","last_became_active":"2024-09-17T02:40:56.333520+0000","last_became_peered":"2024-09-17T02:40:56.333520+0000","last_unstale":"2024-09-17T02:40:57.470871+0000","last_undegraded":"2024-09-17T02:40:57.470871+0000","last_fullsized":"2024-09-17T02:40:57.470871+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:38.037563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472395+0000","last_change":"2024-09-17T02:40:52.237704+0000","last_active":"2024-09-17T02:40:57.472395+0000","last_peered":"2024-09-17T02:40:57.472395+0000","last_clean":"2024-09-17T02:40:57.472395+0000","last_became_active":"2024-09-17T02:40:52.235768+0000","last_became_peered":"2024-09-17T02:40:52.235768+0000","last_unstale":"2024-09-17T02:40:57.472395+0000","last_undegraded":"2024-09-17T02:40:57.472395+0000","last_fullsized":"2024-09-17T02:40:57.472395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:04.549252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473130+0000","last_change":"2024-09-17T02:40:51.227571+0000","last_active":"2024-09-17T02:40:57.473130+0000","last_peered":"2024-09-17T02:40:57.473130+0000","last_clean":"2024-09-17T02:40:57.473130+0000","last_became_active":"2024-09-17T02:40:51.226748+0000","last_became_peered":"2024-09-17T02:40:51.226748+0000","last_unstale":"2024-09-17T02:40:57.473130+0000","last_undegraded":"2024-09-17T02:40:57.473130+0000","last_fullsized":"2024-09-17T02:40:57.473130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:12.330693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473168+0000","last_change":"2024-09-17T02:40:56.327712+0000","last_active":"2024-09-17T02:40:57.473168+0000","last_peered":"2024-09-17T02:40:57.473168+0000","last_clean":"2024-09-17T02:40:57.473168+0000","last_became_active":"2024-09-17T02:40:56.326836+0000","last_became_peered":"2024-09-17T02:40:56.326836+0000","last_unstale":"2024-09-17T02:40:57.473168+0000","last_undegraded":"2024-09-17T02:40:57.473168+0000","last_fullsized":"2024-09-17T02:40:57.473168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:18.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471966+0000","last_change":"2024-09-17T02:40:52.239267+0000","last_active":"2024-09-17T02:40:57.471966+0000","last_peered":"2024-09-17T02:40:57.471966+0000","last_clean":"2024-09-17T02:40:57.471966+0000","last_became_active":"2024-09-17T02:40:52.238707+0000","last_became_peered":"2024-09-17T02:40:52.238707+0000","last_unstale":"2024-09-17T02:40:57.471966+0000","last_undegraded":"2024-09-17T02:40:57.471966+0000","last_fullsized":"2024-09-17T02:40:57.471966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:07.999419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291556+0000","last_change":"2024-09-17T02:40:51.249182+0000","last_active":"2024-09-17T02:40:57.291556+0000","last_peered":"2024-09-17T02:40:57.291556+0000","last_clean":"2024-09-17T02:40:57.291556+0000","last_became_active":"2024-09-17T02:40:51.248480+0000","last_became_peered":"2024-09-17T02:40:51.248480+0000","last_unstale":"2024-09-17T02:40:57.291556+0000","last_undegraded":"2024-09-17T02:40:57.291556+0000","last_fullsized":"2024-09-17T02:40:57.291556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287153+0000","last_change":"2024-09-17T02:40:56.306150+0000","last_active":"2024-09-17T02:40:57.287153+0000","last_peered":"2024-09-17T02:40:57.287153+0000","last_clean":"2024-09-17T02:40:57.287153+0000","last_became_active":"2024-09-17T02:40:56.305729+0000","last_became_peered":"2024-09-17T02:40:56.305729+0000","last_unstale":"2024-09-17T02:40:57.287153+0000","last_undegraded":"2024-09-17T02:40:57.287153+0000","last_fullsized":"2024-09-17T02:40:57.287153+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:32.927545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472197+0000","last_change":"2024-09-17T02:40:52.229209+0000","last_active":"2024-09-17T02:40:57.472197+0000","last_peered":"2024-09-17T02:40:57.472197+0000","last_clean":"2024-09-17T02:40:57.472197+0000","last_became_active":"2024-09-17T02:40:52.228749+0000","last_became_peered":"2024-09-17T02:40:52.228749+0000","last_unstale":"2024-09-17T02:40:57.472197+0000","last_undegraded":"2024-09-17T02:40:57.472197+0000","last_fullsized":"2024-09-17T02:40:57.472197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:52.104423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471968+0000","last_change":"2024-09-17T02:40:51.244855+0000","last_active":"2024-09-17T02:40:57.471968+0000","last_peered":"2024-09-17T02:40:57.471968+0000","last_clean":"2024-09-17T02:40:57.471968+0000","last_became_active":"2024-09-17T02:40:51.244535+0000","last_became_peered":"2024-09-17T02:40:51.244535+0000","last_unstale":"2024-09-17T02:40:57.471968+0000","last_undegraded":"2024-09-17T02:40:57.471968+0000","last_fullsized":"2024-09-17T02:40:57.471968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:15.387810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471115+0000","last_change":"2024-09-17T02:40:56.316962+0000","last_active":"2024-09-17T02:40:57.471115+0000","last_peered":"2024-09-17T02:40:57.471115+0000","last_clean":"2024-09-17T02:40:57.471115+0000","last_became_active":"2024-09-17T02:40:56.316350+0000","last_became_peered":"2024-09-17T02:40:56.316350+0000","last_unstale":"2024-09-17T02:40:57.471115+0000","last_undegraded":"2024-09-17T02:40:57.471115+0000","last_fullsized":"2024-09-17T02:40:57.471115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:49.655411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287234+0000","last_change":"2024-09-17T02:40:52.237017+0000","last_active":"2024-09-17T02:40:57.287234+0000","last_peered":"2024-09-17T02:40:57.287234+0000","last_clean":"2024-09-17T02:40:57.287234+0000","last_became_active":"2024-09-17T02:40:52.230499+0000","last_became_peered":"2024-09-17T02:40:52.230499+0000","last_unstale":"2024-09-17T02:40:57.287234+0000","last_undegraded":"2024-09-17T02:40:57.287234+0000","last_fullsized":"2024-09-17T02:40:57.287234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:46.628840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290952+0000","last_change":"2024-09-17T02:40:51.241977+0000","last_active":"2024-09-17T02:40:57.290952+0000","last_peered":"2024-09-17T02:40:57.290952+0000","last_clean":"2024-09-17T02:40:57.290952+0000","last_became_active":"2024-09-17T02:40:51.241088+0000","last_became_peered":"2024-09-17T02:40:51.241088+0000","last_unstale":"2024-09-17T02:40:57.290952+0000","last_undegraded":"2024-09-17T02:40:57.290952+0000","last_fullsized":"2024-09-17T02:40:57.290952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:15.387810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475101+0000","last_change":"2024-09-17T02:40:56.349582+0000","last_active":"2024-09-17T02:40:57.475101+0000","last_peered":"2024-09-17T02:40:57.475101+0000","last_clean":"2024-09-17T02:40:57.475101+0000","last_became_active":"2024-09-17T02:40:56.348588+0000","last_became_peered":"2024-09-17T02:40:56.348588+0000","last_unstale":"2024-09-17T02:40:57.475101+0000","last_undegraded":"2024-09-17T02:40:57.475101+0000","last_fullsized":"2024-09-17T02:40:57.475101+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:38:43.776429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470147+0000","last_change":"2024-09-17T02:40:52.209126+0000","last_active":"2024-09-17T02:40:57.470147+0000","last_peered":"2024-09-17T02:40:57.470147+0000","last_clean":"2024-09-17T02:40:57.470147+0000","last_became_active":"2024-09-17T02:40:52.208689+0000","last_became_peered":"2024-09-17T02:40:52.208689+0000","last_unstale":"2024-09-17T02:40:57.470147+0000","last_undegraded":"2024-09-17T02:40:57.470147+0000","last_fullsized":"2024-09-17T02:40:57.470147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:08.039862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470654+0000","last_change":"2024-09-17T02:40:51.224834+0000","last_active":"2024-09-17T02:40:57.470654+0000","last_peered":"2024-09-17T02:40:57.470654+0000","last_clean":"2024-09-17T02:40:57.470654+0000","last_became_active":"2024-09-17T02:40:51.224133+0000","last_became_peered":"2024-09-17T02:40:51.224133+0000","last_unstale":"2024-09-17T02:40:57.470654+0000","last_undegraded":"2024-09-17T02:40:57.470654+0000","last_fullsized":"2024-09-17T02:40:57.470654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:13.491613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474143+0000","last_change":"2024-09-17T02:40:56.340807+0000","last_active":"2024-09-17T02:40:57.474143+0000","last_peered":"2024-09-17T02:40:57.474143+0000","last_clean":"2024-09-17T02:40:57.474143+0000","last_became_active":"2024-09-17T02:40:56.339879+0000","last_became_peered":"2024-09-17T02:40:56.339879+0000","last_unstale":"2024-09-17T02:40:57.474143+0000","last_undegraded":"2024-09-17T02:40:57.474143+0000","last_fullsized":"2024-09-17T02:40:57.474143+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:24.197066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288140+0000","last_change":"2024-09-17T02:40:52.244094+0000","last_active":"2024-09-17T02:40:57.288140+0000","last_peered":"2024-09-17T02:40:57.288140+0000","last_clean":"2024-09-17T02:40:57.288140+0000","last_became_active":"2024-09-17T02:40:52.242739+0000","last_became_peered":"2024-09-17T02:40:52.242739+0000","last_unstale":"2024-09-17T02:40:57.288140+0000","last_undegraded":"2024-09-17T02:40:57.288140+0000","last_fullsized":"2024-09-17T02:40:57.288140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:36.659884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288765+0000","last_change":"2024-09-17T02:40:51.222211+0000","last_active":"2024-09-17T02:40:57.288765+0000","last_peered":"2024-09-17T02:40:57.288765+0000","last_clean":"2024-09-17T02:40:57.288765+0000","last_became_active":"2024-09-17T02:40:51.221783+0000","last_became_peered":"2024-09-17T02:40:51.221783+0000","last_unstale":"2024-09-17T02:40:57.288765+0000","last_undegraded":"2024-09-17T02:40:57.288765+0000","last_fullsized":"2024-09-17T02:40:57.288765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:31.651869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288876+0000","last_change":"2024-09-17T02:40:56.348106+0000","last_active":"2024-09-17T02:40:57.288876+0000","last_peered":"2024-09-17T02:40:57.288876+0000","last_clean":"2024-09-17T02:40:57.288876+0000","last_became_active":"2024-09-17T02:40:56.339110+0000","last_became_peered":"2024-09-17T02:40:56.339110+0000","last_unstale":"2024-09-17T02:40:57.288876+0000","last_undegraded":"2024-09-17T02:40:57.288876+0000","last_fullsized":"2024-09-17T02:40:57.288876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:11.904235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287392+0000","last_change":"2024-09-17T02:40:52.230525+0000","last_active":"2024-09-17T02:40:57.287392+0000","last_peered":"2024-09-17T02:40:57.287392+0000","last_clean":"2024-09-17T02:40:57.287392+0000","last_became_active":"2024-09-17T02:40:52.230230+0000","last_became_peered":"2024-09-17T02:40:52.230230+0000","last_unstale":"2024-09-17T02:40:57.287392+0000","last_undegraded":"2024-09-17T02:40:57.287392+0000","last_fullsized":"2024-09-17T02:40:57.287392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:40.156443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473184+0000","last_change":"2024-09-17T02:40:51.232502+0000","last_active":"2024-09-17T02:40:57.473184+0000","last_peered":"2024-09-17T02:40:57.473184+0000","last_clean":"2024-09-17T02:40:57.473184+0000","last_became_active":"2024-09-17T02:40:51.230289+0000","last_became_peered":"2024-09-17T02:40:51.230289+0000","last_unstale":"2024-09-17T02:40:57.473184+0000","last_undegraded":"2024-09-17T02:40:57.473184+0000","last_fullsized":"2024-09-17T02:40:57.473184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:59.380994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288172+0000","last_change":"2024-09-17T02:40:56.325742+0000","last_active":"2024-09-17T02:40:57.288172+0000","last_peered":"2024-09-17T02:40:57.288172+0000","last_clean":"2024-09-17T02:40:57.288172+0000","last_became_active":"2024-09-17T02:40:56.325287+0000","last_became_peered":"2024-09-17T02:40:56.325287+0000","last_unstale":"2024-09-17T02:40:57.288172+0000","last_undegraded":"2024-09-17T02:40:57.288172+0000","last_fullsized":"2024-09-17T02:40:57.288172+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:28.012374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471884+0000","last_change":"2024-09-17T02:40:52.236056+0000","last_active":"2024-09-17T02:40:57.471884+0000","last_peered":"2024-09-17T02:40:57.471884+0000","last_clean":"2024-09-17T02:40:57.471884+0000","last_became_active":"2024-09-17T02:40:52.235680+0000","last_became_peered":"2024-09-17T02:40:52.235680+0000","last_unstale":"2024-09-17T02:40:57.471884+0000","last_undegraded":"2024-09-17T02:40:57.471884+0000","last_fullsized":"2024-09-17T02:40:57.471884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:27.873310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:41:03.060549+0000","last_change":"2024-09-17T02:40:51.242488+0000","last_active":"2024-09-17T02:41:03.060549+0000","last_peered":"2024-09-17T02:41:03.060549+0000","last_clean":"2024-09-17T02:41:03.060549+0000","last_became_active":"2024-09-17T02:40:51.242105+0000","last_became_peered":"2024-09-17T02:40:51.242105+0000","last_unstale":"2024-09-17T02:41:03.060549+0000","last_undegraded":"2024-09-17T02:41:03.060549+0000","last_fullsized":"2024-09-17T02:41:03.060549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:02.225605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287754+0000","last_change":"2024-09-17T02:40:56.327216+0000","last_active":"2024-09-17T02:40:57.287754+0000","last_peered":"2024-09-17T02:40:57.287754+0000","last_clean":"2024-09-17T02:40:57.287754+0000","last_became_active":"2024-09-17T02:40:56.326213+0000","last_became_peered":"2024-09-17T02:40:56.326213+0000","last_unstale":"2024-09-17T02:40:57.287754+0000","last_undegraded":"2024-09-17T02:40:57.287754+0000","last_fullsized":"2024-09-17T02:40:57.287754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:58.168471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288181+0000","last_change":"2024-09-17T02:40:52.237718+0000","last_active":"2024-09-17T02:40:57.288181+0000","last_peered":"2024-09-17T02:40:57.288181+0000","last_clean":"2024-09-17T02:40:57.288181+0000","last_became_active":"2024-09-17T02:40:52.237303+0000","last_became_peered":"2024-09-17T02:40:52.237303+0000","last_unstale":"2024-09-17T02:40:57.288181+0000","last_undegraded":"2024-09-17T02:40:57.288181+0000","last_fullsized":"2024-09-17T02:40:57.288181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:47.651165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474398+0000","last_change":"2024-09-17T02:40:51.234788+0000","last_active":"2024-09-17T02:40:57.474398+0000","last_peered":"2024-09-17T02:40:57.474398+0000","last_clean":"2024-09-17T02:40:57.474398+0000","last_became_active":"2024-09-17T02:40:51.233917+0000","last_became_peered":"2024-09-17T02:40:51.233917+0000","last_unstale":"2024-09-17T02:40:57.474398+0000","last_undegraded":"2024-09-17T02:40:57.474398+0000","last_fullsized":"2024-09-17T02:40:57.474398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:27.301395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471431+0000","last_change":"2024-09-17T02:40:56.337448+0000","last_active":"2024-09-17T02:40:57.471431+0000","last_peered":"2024-09-17T02:40:57.471431+0000","last_clean":"2024-09-17T02:40:57.471431+0000","last_became_active":"2024-09-17T02:40:56.336959+0000","last_became_peered":"2024-09-17T02:40:56.336959+0000","last_unstale":"2024-09-17T02:40:57.471431+0000","last_undegraded":"2024-09-17T02:40:57.471431+0000","last_fullsized":"2024-09-17T02:40:57.471431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:31.952318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474548+0000","last_change":"2024-09-17T02:40:52.247671+0000","last_active":"2024-09-17T02:40:57.474548+0000","last_peered":"2024-09-17T02:40:57.474548+0000","last_clean":"2024-09-17T02:40:57.474548+0000","last_became_active":"2024-09-17T02:40:52.247325+0000","last_became_peered":"2024-09-17T02:40:52.247325+0000","last_unstale":"2024-09-17T02:40:57.474548+0000","last_undegraded":"2024-09-17T02:40:57.474548+0000","last_fullsized":"2024-09-17T02:40:57.474548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:09.357623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471882+0000","last_change":"2024-09-17T02:40:51.224974+0000","last_active":"2024-09-17T02:40:57.471882+0000","last_peered":"2024-09-17T02:40:57.471882+0000","last_clean":"2024-09-17T02:40:57.471882+0000","last_became_active":"2024-09-17T02:40:51.224479+0000","last_became_peered":"2024-09-17T02:40:51.224479+0000","last_unstale":"2024-09-17T02:40:57.471882+0000","last_undegraded":"2024-09-17T02:40:57.471882+0000","last_fullsized":"2024-09-17T02:40:57.471882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471541+0000","last_change":"2024-09-17T02:40:56.333951+0000","last_active":"2024-09-17T02:40:57.471541+0000","last_peered":"2024-09-17T02:40:57.471541+0000","last_clean":"2024-09-17T02:40:57.471541+0000","last_became_active":"2024-09-17T02:40:56.332508+0000","last_became_peered":"2024-09-17T02:40:56.332508+0000","last_unstale":"2024-09-17T02:40:57.471541+0000","last_undegraded":"2024-09-17T02:40:57.471541+0000","last_fullsized":"2024-09-17T02:40:57.471541+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:08.241482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472176+0000","last_change":"2024-09-17T02:40:52.244735+0000","last_active":"2024-09-17T02:40:57.472176+0000","last_peered":"2024-09-17T02:40:57.472176+0000","last_clean":"2024-09-17T02:40:57.472176+0000","last_became_active":"2024-09-17T02:40:52.243763+0000","last_became_peered":"2024-09-17T02:40:52.243763+0000","last_unstale":"2024-09-17T02:40:57.472176+0000","last_undegraded":"2024-09-17T02:40:57.472176+0000","last_fullsized":"2024-09-17T02:40:57.472176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:32.662763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471344+0000","last_change":"2024-09-17T02:40:51.234151+0000","last_active":"2024-09-17T02:40:57.471344+0000","last_peered":"2024-09-17T02:40:57.471344+0000","last_clean":"2024-09-17T02:40:57.471344+0000","last_became_active":"2024-09-17T02:40:51.233731+0000","last_became_peered":"2024-09-17T02:40:51.233731+0000","last_unstale":"2024-09-17T02:40:57.471344+0000","last_undegraded":"2024-09-17T02:40:57.471344+0000","last_fullsized":"2024-09-17T02:40:57.471344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:15.349994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290693+0000","last_change":"2024-09-17T02:40:56.334710+0000","last_active":"2024-09-17T02:40:57.290693+0000","last_peered":"2024-09-17T02:40:57.290693+0000","last_clean":"2024-09-17T02:40:57.290693+0000","last_became_active":"2024-09-17T02:40:56.333630+0000","last_became_peered":"2024-09-17T02:40:56.333630+0000","last_unstale":"2024-09-17T02:40:57.290693+0000","last_undegraded":"2024-09-17T02:40:57.290693+0000","last_fullsized":"2024-09-17T02:40:57.290693+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:07.834564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290681+0000","last_change":"2024-09-17T02:40:56.321729+0000","last_active":"2024-09-17T02:40:57.290681+0000","last_peered":"2024-09-17T02:40:57.290681+0000","last_clean":"2024-09-17T02:40:57.290681+0000","last_became_active":"2024-09-17T02:40:56.320583+0000","last_became_peered":"2024-09-17T02:40:56.320583+0000","last_unstale":"2024-09-17T02:40:57.290681+0000","last_undegraded":"2024-09-17T02:40:57.290681+0000","last_fullsized":"2024-09-17T02:40:57.290681+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:50.707848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291114+0000","last_change":"2024-09-17T02:40:52.224867+0000","last_active":"2024-09-17T02:40:57.291114+0000","last_peered":"2024-09-17T02:40:57.291114+0000","last_clean":"2024-09-17T02:40:57.291114+0000","last_became_active":"2024-09-17T02:40:52.224558+0000","last_became_peered":"2024-09-17T02:40:52.224558+0000","last_unstale":"2024-09-17T02:40:57.291114+0000","last_undegraded":"2024-09-17T02:40:57.291114+0000","last_fullsized":"2024-09-17T02:40:57.291114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:01.098205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287555+0000","last_change":"2024-09-17T02:40:51.210128+0000","last_active":"2024-09-17T02:40:57.287555+0000","last_peered":"2024-09-17T02:40:57.287555+0000","last_clean":"2024-09-17T02:40:57.287555+0000","last_became_active":"2024-09-17T02:40:51.209655+0000","last_became_peered":"2024-09-17T02:40:51.209655+0000","last_unstale":"2024-09-17T02:40:57.287555+0000","last_undegraded":"2024-09-17T02:40:57.287555+0000","last_fullsized":"2024-09-17T02:40:57.287555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:18.053396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288450+0000","last_change":"2024-09-17T02:40:56.316501+0000","last_active":"2024-09-17T02:40:57.288450+0000","last_peered":"2024-09-17T02:40:57.288450+0000","last_clean":"2024-09-17T02:40:57.288450+0000","last_became_active":"2024-09-17T02:40:56.315839+0000","last_became_peered":"2024-09-17T02:40:56.315839+0000","last_unstale":"2024-09-17T02:40:57.288450+0000","last_undegraded":"2024-09-17T02:40:57.288450+0000","last_fullsized":"2024-09-17T02:40:57.288450+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:21.657805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472377+0000","last_change":"2024-09-17T02:40:52.228669+0000","last_active":"2024-09-17T02:40:57.472377+0000","last_peered":"2024-09-17T02:40:57.472377+0000","last_clean":"2024-09-17T02:40:57.472377+0000","last_became_active":"2024-09-17T02:40:52.226236+0000","last_became_peered":"2024-09-17T02:40:52.226236+0000","last_unstale":"2024-09-17T02:40:57.472377+0000","last_undegraded":"2024-09-17T02:40:57.472377+0000","last_fullsized":"2024-09-17T02:40:57.472377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:41.461773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287139+0000","last_change":"2024-09-17T02:40:51.226679+0000","last_active":"2024-09-17T02:40:57.287139+0000","last_peered":"2024-09-17T02:40:57.287139+0000","last_clean":"2024-09-17T02:40:57.287139+0000","last_became_active":"2024-09-17T02:40:51.225794+0000","last_became_peered":"2024-09-17T02:40:51.225794+0000","last_unstale":"2024-09-17T02:40:57.287139+0000","last_undegraded":"2024-09-17T02:40:57.287139+0000","last_fullsized":"2024-09-17T02:40:57.287139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:37.666440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288775+0000","last_change":"2024-09-17T02:40:51.226491+0000","last_active":"2024-09-17T02:40:57.288775+0000","last_peered":"2024-09-17T02:40:57.288775+0000","last_clean":"2024-09-17T02:40:57.288775+0000","last_became_active":"2024-09-17T02:40:51.225695+0000","last_became_peered":"2024-09-17T02:40:51.225695+0000","last_unstale":"2024-09-17T02:40:57.288775+0000","last_undegraded":"2024-09-17T02:40:57.288775+0000","last_fullsized":"2024-09-17T02:40:57.288775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:46.966810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288252+0000","last_change":"2024-09-17T02:40:52.222794+0000","last_active":"2024-09-17T02:40:57.288252+0000","last_peered":"2024-09-17T02:40:57.288252+0000","last_clean":"2024-09-17T02:40:57.288252+0000","last_became_active":"2024-09-17T02:40:52.221682+0000","last_became_peered":"2024-09-17T02:40:52.221682+0000","last_unstale":"2024-09-17T02:40:57.288252+0000","last_undegraded":"2024-09-17T02:40:57.288252+0000","last_fullsized":"2024-09-17T02:40:57.288252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:34.058125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.292523+0000","last_change":"2024-09-17T02:40:56.343317+0000","last_active":"2024-09-17T02:40:57.292523+0000","last_peered":"2024-09-17T02:40:57.292523+0000","last_clean":"2024-09-17T02:40:57.292523+0000","last_became_active":"2024-09-17T02:40:56.342373+0000","last_became_peered":"2024-09-17T02:40:56.342373+0000","last_unstale":"2024-09-17T02:40:57.292523+0000","last_undegraded":"2024-09-17T02:40:57.292523+0000","last_fullsized":"2024-09-17T02:40:57.292523+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:02.210967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473874+0000","last_change":"2024-09-17T02:40:51.237550+0000","last_active":"2024-09-17T02:40:57.473874+0000","last_peered":"2024-09-17T02:40:57.473874+0000","last_clean":"2024-09-17T02:40:57.473874+0000","last_became_active":"2024-09-17T02:40:51.236422+0000","last_became_peered":"2024-09-17T02:40:51.236422+0000","last_unstale":"2024-09-17T02:40:57.473874+0000","last_undegraded":"2024-09-17T02:40:57.473874+0000","last_fullsized":"2024-09-17T02:40:57.473874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470700+0000","last_change":"2024-09-17T02:40:52.226397+0000","last_active":"2024-09-17T02:40:57.470700+0000","last_peered":"2024-09-17T02:40:57.470700+0000","last_clean":"2024-09-17T02:40:57.470700+0000","last_became_active":"2024-09-17T02:40:52.225934+0000","last_became_peered":"2024-09-17T02:40:52.225934+0000","last_unstale":"2024-09-17T02:40:57.470700+0000","last_undegraded":"2024-09-17T02:40:57.470700+0000","last_fullsized":"2024-09-17T02:40:57.470700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:17:37.421483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472110+0000","last_change":"2024-09-17T02:40:56.336996+0000","last_active":"2024-09-17T02:40:57.472110+0000","last_peered":"2024-09-17T02:40:57.472110+0000","last_clean":"2024-09-17T02:40:57.472110+0000","last_became_active":"2024-09-17T02:40:56.336172+0000","last_became_peered":"2024-09-17T02:40:56.336172+0000","last_unstale":"2024-09-17T02:40:57.472110+0000","last_undegraded":"2024-09-17T02:40:57.472110+0000","last_fullsized":"2024-09-17T02:40:57.472110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:05.177214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470197+0000","last_change":"2024-09-17T02:40:52.232046+0000","last_active":"2024-09-17T02:40:57.470197+0000","last_peered":"2024-09-17T02:40:57.470197+0000","last_clean":"2024-09-17T02:40:57.470197+0000","last_became_active":"2024-09-17T02:40:52.231700+0000","last_became_peered":"2024-09-17T02:40:52.231700+0000","last_unstale":"2024-09-17T02:40:57.470197+0000","last_undegraded":"2024-09-17T02:40:57.470197+0000","last_fullsized":"2024-09-17T02:40:57.470197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:47.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472501+0000","last_change":"2024-09-17T02:40:52.239198+0000","last_active":"2024-09-17T02:40:57.472501+0000","last_peered":"2024-09-17T02:40:57.472501+0000","last_clean":"2024-09-17T02:40:57.472501+0000","last_became_active":"2024-09-17T02:40:52.238901+0000","last_became_peered":"2024-09-17T02:40:52.238901+0000","last_unstale":"2024-09-17T02:40:57.472501+0000","last_undegraded":"2024-09-17T02:40:57.472501+0000","last_fullsized":"2024-09-17T02:40:57.472501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:47.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287639+0000","last_change":"2024-09-17T02:40:52.228993+0000","last_active":"2024-09-17T02:40:57.287639+0000","last_peered":"2024-09-17T02:40:57.287639+0000","last_clean":"2024-09-17T02:40:57.287639+0000","last_became_active":"2024-09-17T02:40:52.228574+0000","last_became_peered":"2024-09-17T02:40:52.228574+0000","last_unstale":"2024-09-17T02:40:57.287639+0000","last_undegraded":"2024-09-17T02:40:57.287639+0000","last_fullsized":"2024-09-17T02:40:57.287639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:31.798195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473564+0000","last_change":"2024-09-17T02:40:52.222569+0000","last_active":"2024-09-17T02:40:57.473564+0000","last_peered":"2024-09-17T02:40:57.473564+0000","last_clean":"2024-09-17T02:40:57.473564+0000","last_became_active":"2024-09-17T02:40:52.222228+0000","last_became_peered":"2024-09-17T02:40:52.222228+0000","last_unstale":"2024-09-17T02:40:57.473564+0000","last_undegraded":"2024-09-17T02:40:57.473564+0000","last_fullsized":"2024-09-17T02:40:57.473564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:39.361971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289288+0000","last_change":"2024-09-17T02:40:52.236488+0000","last_active":"2024-09-17T02:40:57.289288+0000","last_peered":"2024-09-17T02:40:57.289288+0000","last_clean":"2024-09-17T02:40:57.289288+0000","last_became_active":"2024-09-17T02:40:52.236171+0000","last_became_peered":"2024-09-17T02:40:52.236171+0000","last_unstale":"2024-09-17T02:40:57.289288+0000","last_undegraded":"2024-09-17T02:40:57.289288+0000","last_fullsized":"2024-09-17T02:40:57.289288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:08.415078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287712+0000","last_change":"2024-09-17T02:40:52.217993+0000","last_active":"2024-09-17T02:40:57.287712+0000","last_peered":"2024-09-17T02:40:57.287712+0000","last_clean":"2024-09-17T02:40:57.287712+0000","last_became_active":"2024-09-17T02:40:52.216716+0000","last_became_peered":"2024-09-17T02:40:52.216716+0000","last_unstale":"2024-09-17T02:40:57.287712+0000","last_undegraded":"2024-09-17T02:40:57.287712+0000","last_fullsized":"2024-09-17T02:40:57.287712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:06.000439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472031+0000","last_change":"2024-09-17T02:40:52.245158+0000","last_active":"2024-09-17T02:40:57.472031+0000","last_peered":"2024-09-17T02:40:57.472031+0000","last_clean":"2024-09-17T02:40:57.472031+0000","last_became_active":"2024-09-17T02:40:52.244333+0000","last_became_peered":"2024-09-17T02:40:52.244333+0000","last_unstale":"2024-09-17T02:40:57.472031+0000","last_undegraded":"2024-09-17T02:40:57.472031+0000","last_fullsized":"2024-09-17T02:40:57.472031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:06.006731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473931+0000","last_change":"2024-09-17T02:40:52.228925+0000","last_active":"2024-09-17T02:40:57.473931+0000","last_peered":"2024-09-17T02:40:57.473931+0000","last_clean":"2024-09-17T02:40:57.473931+0000","last_became_active":"2024-09-17T02:40:52.228587+0000","last_became_peered":"2024-09-17T02:40:52.228587+0000","last_unstale":"2024-09-17T02:40:57.473931+0000","last_undegraded":"2024-09-17T02:40:57.473931+0000","last_fullsized":"2024-09-17T02:40:57.473931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:33.958554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287342+0000","last_change":"2024-09-17T02:40:52.229795+0000","last_active":"2024-09-17T02:40:57.287342+0000","last_peered":"2024-09-17T02:40:57.287342+0000","last_clean":"2024-09-17T02:40:57.287342+0000","last_became_active":"2024-09-17T02:40:52.227659+0000","last_became_peered":"2024-09-17T02:40:52.227659+0000","last_unstale":"2024-09-17T02:40:57.287342+0000","last_undegraded":"2024-09-17T02:40:57.287342+0000","last_fullsized":"2024-09-17T02:40:57.287342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:36.541471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473327+0000","last_change":"2024-09-17T02:40:52.221420+0000","last_active":"2024-09-17T02:40:57.473327+0000","last_peered":"2024-09-17T02:40:57.473327+0000","last_clean":"2024-09-17T02:40:57.473327+0000","last_became_active":"2024-09-17T02:40:52.221050+0000","last_became_peered":"2024-09-17T02:40:52.221050+0000","last_unstale":"2024-09-17T02:40:57.473327+0000","last_undegraded":"2024-09-17T02:40:57.473327+0000","last_fullsized":"2024-09-17T02:40:57.473327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:47.715501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470588+0000","last_change":"2024-09-17T02:40:52.237946+0000","last_active":"2024-09-17T02:40:57.470588+0000","last_peered":"2024-09-17T02:40:57.470588+0000","last_clean":"2024-09-17T02:40:57.470588+0000","last_became_active":"2024-09-17T02:40:52.237376+0000","last_became_peered":"2024-09-17T02:40:52.237376+0000","last_unstale":"2024-09-17T02:40:57.470588+0000","last_undegraded":"2024-09-17T02:40:57.470588+0000","last_fullsized":"2024-09-17T02:40:57.470588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:31.107108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287955+0000","last_change":"2024-09-17T02:40:52.221072+0000","last_active":"2024-09-17T02:40:57.287955+0000","last_peered":"2024-09-17T02:40:57.287955+0000","last_clean":"2024-09-17T02:40:57.287955+0000","last_became_active":"2024-09-17T02:40:52.220697+0000","last_became_peered":"2024-09-17T02:40:52.220697+0000","last_unstale":"2024-09-17T02:40:57.287955+0000","last_undegraded":"2024-09-17T02:40:57.287955+0000","last_fullsized":"2024-09-17T02:40:57.287955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:02.681569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472298+0000","last_change":"2024-09-17T02:40:52.249011+0000","last_active":"2024-09-17T02:40:57.472298+0000","last_peered":"2024-09-17T02:40:57.472298+0000","last_clean":"2024-09-17T02:40:57.472298+0000","last_became_active":"2024-09-17T02:40:52.247429+0000","last_became_peered":"2024-09-17T02:40:52.247429+0000","last_unstale":"2024-09-17T02:40:57.472298+0000","last_undegraded":"2024-09-17T02:40:57.472298+0000","last_fullsized":"2024-09-17T02:40:57.472298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:56.223383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288383+0000","last_change":"2024-09-17T02:40:52.243578+0000","last_active":"2024-09-17T02:40:57.288383+0000","last_peered":"2024-09-17T02:40:57.288383+0000","last_clean":"2024-09-17T02:40:57.288383+0000","last_became_active":"2024-09-17T02:40:52.243106+0000","last_became_peered":"2024-09-17T02:40:52.243106+0000","last_unstale":"2024-09-17T02:40:57.288383+0000","last_undegraded":"2024-09-17T02:40:57.288383+0000","last_fullsized":"2024-09-17T02:40:57.288383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:22.663615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475176+0000","last_change":"2024-09-17T02:40:52.222351+0000","last_active":"2024-09-17T02:40:57.475176+0000","last_peered":"2024-09-17T02:40:57.475176+0000","last_clean":"2024-09-17T02:40:57.475176+0000","last_became_active":"2024-09-17T02:40:52.222043+0000","last_became_peered":"2024-09-17T02:40:52.222043+0000","last_unstale":"2024-09-17T02:40:57.475176+0000","last_undegraded":"2024-09-17T02:40:57.475176+0000","last_fullsized":"2024-09-17T02:40:57.475176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:21.583058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288320+0000","last_change":"2024-09-17T02:40:52.256413+0000","last_active":"2024-09-17T02:40:57.288320+0000","last_peered":"2024-09-17T02:40:57.288320+0000","last_clean":"2024-09-17T02:40:57.288320+0000","last_became_active":"2024-09-17T02:40:52.256035+0000","last_became_peered":"2024-09-17T02:40:52.256035+0000","last_unstale":"2024-09-17T02:40:57.288320+0000","last_undegraded":"2024-09-17T02:40:57.288320+0000","last_fullsized":"2024-09-17T02:40:57.288320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:59.172988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287206+0000","last_change":"2024-09-17T02:40:52.236686+0000","last_active":"2024-09-17T02:40:57.287206+0000","last_peered":"2024-09-17T02:40:57.287206+0000","last_clean":"2024-09-17T02:40:57.287206+0000","last_became_active":"2024-09-17T02:40:52.236194+0000","last_became_peered":"2024-09-17T02:40:52.236194+0000","last_unstale":"2024-09-17T02:40:57.287206+0000","last_undegraded":"2024-09-17T02:40:57.287206+0000","last_fullsized":"2024-09-17T02:40:57.287206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:55.015415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472841+0000","last_change":"2024-09-17T02:40:52.228032+0000","last_active":"2024-09-17T02:40:57.472841+0000","last_peered":"2024-09-17T02:40:57.472841+0000","last_clean":"2024-09-17T02:40:57.472841+0000","last_became_active":"2024-09-17T02:40:52.225817+0000","last_became_peered":"2024-09-17T02:40:52.225817+0000","last_unstale":"2024-09-17T02:40:57.472841+0000","last_undegraded":"2024-09-17T02:40:57.472841+0000","last_fullsized":"2024-09-17T02:40:57.472841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:34.058125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287107+0000","last_change":"2024-09-17T02:40:52.217145+0000","last_active":"2024-09-17T02:40:57.287107+0000","last_peered":"2024-09-17T02:40:57.287107+0000","last_clean":"2024-09-17T02:40:57.287107+0000","last_became_active":"2024-09-17T02:40:52.215909+0000","last_became_peered":"2024-09-17T02:40:52.215909+0000","last_unstale":"2024-09-17T02:40:57.287107+0000","last_undegraded":"2024-09-17T02:40:57.287107+0000","last_fullsized":"2024-09-17T02:40:57.287107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:28.728656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288943+0000","last_change":"2024-09-17T02:40:52.224668+0000","last_active":"2024-09-17T02:40:57.288943+0000","last_peered":"2024-09-17T02:40:57.288943+0000","last_clean":"2024-09-17T02:40:57.288943+0000","last_became_active":"2024-09-17T02:40:52.224273+0000","last_became_peered":"2024-09-17T02:40:52.224273+0000","last_unstale":"2024-09-17T02:40:57.288943+0000","last_undegraded":"2024-09-17T02:40:57.288943+0000","last_fullsized":"2024-09-17T02:40:57.288943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:21.184527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472191+0000","last_change":"2024-09-17T02:40:52.252355+0000","last_active":"2024-09-17T02:40:57.472191+0000","last_peered":"2024-09-17T02:40:57.472191+0000","last_clean":"2024-09-17T02:40:57.472191+0000","last_became_active":"2024-09-17T02:40:52.251950+0000","last_became_peered":"2024-09-17T02:40:52.251950+0000","last_unstale":"2024-09-17T02:40:57.472191+0000","last_undegraded":"2024-09-17T02:40:57.472191+0000","last_fullsized":"2024-09-17T02:40:57.472191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:50.916246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471773+0000","last_change":"2024-09-17T02:40:52.244189+0000","last_active":"2024-09-17T02:40:57.471773+0000","last_peered":"2024-09-17T02:40:57.471773+0000","last_clean":"2024-09-17T02:40:57.471773+0000","last_became_active":"2024-09-17T02:40:52.243891+0000","last_became_peered":"2024-09-17T02:40:52.243891+0000","last_unstale":"2024-09-17T02:40:57.471773+0000","last_undegraded":"2024-09-17T02:40:57.471773+0000","last_fullsized":"2024-09-17T02:40:57.471773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:00.308180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471829+0000","last_change":"2024-09-17T02:40:52.237299+0000","last_active":"2024-09-17T02:40:57.471829+0000","last_peered":"2024-09-17T02:40:57.471829+0000","last_clean":"2024-09-17T02:40:57.471829+0000","last_became_active":"2024-09-17T02:40:52.236928+0000","last_became_peered":"2024-09-17T02:40:52.236928+0000","last_unstale":"2024-09-17T02:40:57.471829+0000","last_undegraded":"2024-09-17T02:40:57.471829+0000","last_fullsized":"2024-09-17T02:40:57.471829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:03.755555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471259+0000","last_change":"2024-09-17T02:40:52.247989+0000","last_active":"2024-09-17T02:40:57.471259+0000","last_peered":"2024-09-17T02:40:57.471259+0000","last_clean":"2024-09-17T02:40:57.471259+0000","last_became_active":"2024-09-17T02:40:52.247534+0000","last_became_peered":"2024-09-17T02:40:52.247534+0000","last_unstale":"2024-09-17T02:40:57.471259+0000","last_undegraded":"2024-09-17T02:40:57.471259+0000","last_fullsized":"2024-09-17T02:40:57.471259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:14.502507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471636+0000","last_change":"2024-09-17T02:40:52.228868+0000","last_active":"2024-09-17T02:40:57.471636+0000","last_peered":"2024-09-17T02:40:57.471636+0000","last_clean":"2024-09-17T02:40:57.471636+0000","last_became_active":"2024-09-17T02:40:52.227927+0000","last_became_peered":"2024-09-17T02:40:52.227927+0000","last_unstale":"2024-09-17T02:40:57.471636+0000","last_undegraded":"2024-09-17T02:40:57.471636+0000","last_fullsized":"2024-09-17T02:40:57.471636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:06.283756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470545+0000","last_change":"2024-09-17T02:40:52.220236+0000","last_active":"2024-09-17T02:40:57.470545+0000","last_peered":"2024-09-17T02:40:57.470545+0000","last_clean":"2024-09-17T02:40:57.470545+0000","last_became_active":"2024-09-17T02:40:52.219510+0000","last_became_peered":"2024-09-17T02:40:52.219510+0000","last_unstale":"2024-09-17T02:40:57.470545+0000","last_undegraded":"2024-09-17T02:40:57.470545+0000","last_fullsized":"2024-09-17T02:40:57.470545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:33.958554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471108+0000","last_change":"2024-09-17T02:40:52.218799+0000","last_active":"2024-09-17T02:40:57.471108+0000","last_peered":"2024-09-17T02:40:57.471108+0000","last_clean":"2024-09-17T02:40:57.471108+0000","last_became_active":"2024-09-17T02:40:52.217054+0000","last_became_peered":"2024-09-17T02:40:52.217054+0000","last_unstale":"2024-09-17T02:40:57.471108+0000","last_undegraded":"2024-09-17T02:40:57.471108+0000","last_fullsized":"2024-09-17T02:40:57.471108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:21.583058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472762+0000","last_change":"2024-09-17T02:40:52.232583+0000","last_active":"2024-09-17T02:40:57.472762+0000","last_peered":"2024-09-17T02:40:57.472762+0000","last_clean":"2024-09-17T02:40:57.472762+0000","last_became_active":"2024-09-17T02:40:52.231933+0000","last_became_peered":"2024-09-17T02:40:52.231933+0000","last_unstale":"2024-09-17T02:40:57.472762+0000","last_undegraded":"2024-09-17T02:40:57.472762+0000","last_fullsized":"2024-09-17T02:40:57.472762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:41.461773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471574+0000","last_change":"2024-09-17T02:40:52.218335+0000","last_active":"2024-09-17T02:40:57.471574+0000","last_peered":"2024-09-17T02:40:57.471574+0000","last_clean":"2024-09-17T02:40:57.471574+0000","last_became_active":"2024-09-17T02:40:52.217008+0000","last_became_peered":"2024-09-17T02:40:52.217008+0000","last_unstale":"2024-09-17T02:40:57.471574+0000","last_undegraded":"2024-09-17T02:40:57.471574+0000","last_fullsized":"2024-09-17T02:40:57.471574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:19.064290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472531+0000","last_change":"2024-09-17T02:40:52.224771+0000","last_active":"2024-09-17T02:40:57.472531+0000","last_peered":"2024-09-17T02:40:57.472531+0000","last_clean":"2024-09-17T02:40:57.472531+0000","last_became_active":"2024-09-17T02:40:52.222947+0000","last_became_peered":"2024-09-17T02:40:52.222947+0000","last_unstale":"2024-09-17T02:40:57.472531+0000","last_undegraded":"2024-09-17T02:40:57.472531+0000","last_fullsized":"2024-09-17T02:40:57.472531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:54.216006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474943+0000","last_change":"2024-09-17T02:40:52.228603+0000","last_active":"2024-09-17T02:40:57.474943+0000","last_peered":"2024-09-17T02:40:57.474943+0000","last_clean":"2024-09-17T02:40:57.474943+0000","last_became_active":"2024-09-17T02:40:52.225545+0000","last_became_peered":"2024-09-17T02:40:52.225545+0000","last_unstale":"2024-09-17T02:40:57.474943+0000","last_undegraded":"2024-09-17T02:40:57.474943+0000","last_fullsized":"2024-09-17T02:40:57.474943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:56.223383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471852+0000","last_change":"2024-09-17T02:40:52.250495+0000","last_active":"2024-09-17T02:40:57.471852+0000","last_peered":"2024-09-17T02:40:57.471852+0000","last_clean":"2024-09-17T02:40:57.471852+0000","last_became_active":"2024-09-17T02:40:52.250095+0000","last_became_peered":"2024-09-17T02:40:52.250095+0000","last_unstale":"2024-09-17T02:40:57.471852+0000","last_undegraded":"2024-09-17T02:40:57.471852+0000","last_fullsized":"2024-09-17T02:40:57.471852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:58.131959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509450,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:41:14.734 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T02:41:14.735 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T02:41:14.735 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T02:41:14.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.926+0000 7f7466798640 1 -- 172.21.15.131:0/3274297472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74601450c0 msgr2=0x7f74600d0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.926+0000 7f7466798640 1 --2- 172.21.15.131:0/3274297472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74601450c0 0x7f74600d0710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7450009900 tx=0x7f745001c640 comp rx=0 tx=0).stop 2024-09-17T02:41:14.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.926+0000 7f7466798640 1 -- 172.21.15.131:0/3274297472 shutdown_connections 2024-09-17T02:41:14.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.926+0000 7f7466798640 1 --2- 172.21.15.131:0/3274297472 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f74600cc780 0x7f74600ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.926+0000 7f7466798640 1 --2- 172.21.15.131:0/3274297472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74601450c0 0x7f74600d0710 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.926+0000 7f7466798640 1 -- 172.21.15.131:0/3274297472 >> 172.21.15.131:0/3274297472 conn(0x7f74600d2130 msgr2=0x7f74600d2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:14.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.926+0000 7f7466798640 1 -- 172.21.15.131:0/3274297472 shutdown_connections 2024-09-17T02:41:14.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.926+0000 7f7466798640 1 -- 172.21.15.131:0/3274297472 wait complete. 2024-09-17T02:41:14.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7466798640 1 Processor -- start 2024-09-17T02:41:14.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7466798640 1 -- start start 2024-09-17T02:41:14.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7466798640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74600cc780 0x7f74601fc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7466798640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74601450c0 0x7f74601fc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7466798640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f746015ff50 0x7f746015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7466798640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7460169e40 con 0x7f74600cc780 2024-09-17T02:41:14.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7466798640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7460169cc0 con 0x7f74601450c0 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7466798640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7460169fc0 con 0x7f746015ff50 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74600cc780 0x7f74601fc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74600cc780 0x7f74601fc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57662/0 (socket says 172.21.15.131:57662) 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745ffff640 1 -- 172.21.15.131:0/4066730247 learned_addr learned my addr 172.21.15.131:0/4066730247 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745f7fe640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74601450c0 0x7f74601fc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7464d35640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f746015ff50 0x7f746015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745ffff640 1 -- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f746015ff50 msgr2=0x7f746015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745ffff640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f746015ff50 0x7f746015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745ffff640 1 -- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74601450c0 msgr2=0x7f74601fc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745ffff640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74601450c0 0x7f74601fc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745ffff640 1 -- 172.21.15.131:0/4066730247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7460155450 con 0x7f74600cc780 2024-09-17T02:41:14.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7464d35640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f746015ff50 0x7f746015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:14.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745ffff640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74600cc780 0x7f74601fc470 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f745001cb20 tx=0x7f74500a65a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:14.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745cff9640 1 -- 172.21.15.131:0/4066730247 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500ac070 con 0x7f74600cc780 2024-09-17T02:41:14.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745cff9640 1 -- 172.21.15.131:0/4066730247 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7450004340 con 0x7f74600cc780 2024-09-17T02:41:14.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7466798640 1 -- 172.21.15.131:0/4066730247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f746006ebc0 con 0x7f74600cc780 2024-09-17T02:41:14.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f745cff9640 1 -- 172.21.15.131:0/4066730247 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500053a0 con 0x7f74600cc780 2024-09-17T02:41:14.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.930+0000 7f7466798640 1 -- 172.21.15.131:0/4066730247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f746006f180 con 0x7f74600cc780 2024-09-17T02:41:14.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.934+0000 7f7466798640 1 -- 172.21.15.131:0/4066730247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74600ccb90 con 0x7f74600cc780 2024-09-17T02:41:14.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.934+0000 7f745cff9640 1 -- 172.21.15.131:0/4066730247 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f74500af070 con 0x7f74600cc780 2024-09-17T02:41:14.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.934+0000 7f745cff9640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f744c073230 0x7f744c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:14.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.934+0000 7f745cff9640 1 -- 172.21.15.131:0/4066730247 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f745012ea50 con 0x7f74600cc780 2024-09-17T02:41:14.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.934+0000 7f745f7fe640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f744c073230 0x7f744c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:14.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.934+0000 7f745f7fe640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f744c073230 0x7f744c075740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7454006fd0 tx=0x7f7454008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:14.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:14.938+0000 7f745cff9640 1 -- 172.21.15.131:0/4066730247 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f74600cd0d0 con 0x7f74600cc780 2024-09-17T02:41:15.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.106+0000 7f7466798640 1 -- 172.21.15.131:0/4066730247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f746015f430 con 0x7f74600cc780 2024-09-17T02:41:15.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.106+0000 7f745cff9640 1 -- 172.21.15.131:0/4066730247 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7f74500f9200 con 0x7f74600cc780 2024-09-17T02:41:15.106 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 -- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f744c073230 msgr2=0x7f744c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f744c073230 0x7f744c075740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7454006fd0 tx=0x7f7454008040 comp rx=0 tx=0).stop 2024-09-17T02:41:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 -- 172.21.15.131:0/4066730247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74600cc780 msgr2=0x7f74601fc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74600cc780 0x7f74601fc470 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f745001cb20 tx=0x7f74500a65a0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 -- 172.21.15.131:0/4066730247 shutdown_connections 2024-09-17T02:41:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f744c073230 0x7f744c075740 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f746015ff50 0x7f746015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f74601450c0 0x7f74601fc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 --2- 172.21.15.131:0/4066730247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f74600cc780 0x7f74601fc470 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 -- 172.21.15.131:0/4066730247 >> 172.21.15.131:0/4066730247 conn(0x7f74600d2130 msgr2=0x7f7460155960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 -- 172.21.15.131:0/4066730247 shutdown_connections 2024-09-17T02:41:15.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.110+0000 7f7466798640 1 -- 172.21.15.131:0/4066730247 wait complete. 2024-09-17T02:41:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:[{"version":31,"timestamp":"2024-09-17T02:40:40.445535+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"3598.912856"}]}] 2024-09-17T02:41:15.127 INFO:tasks.ceph_manager:config epoch is 31 2024-09-17T02:41:15.127 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T02:41:15.127 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T02:41:15.128 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T02:41:15.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 -- 172.21.15.131:0/878679549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98140ced00 msgr2=0x7f981406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 --2- 172.21.15.131:0/878679549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98140ced00 0x7f981406dea0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9804009900 tx=0x7f980401c8e0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 -- 172.21.15.131:0/878679549 shutdown_connections 2024-09-17T02:41:15.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 --2- 172.21.15.131:0/878679549 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f981415c5b0 0x7f98140cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 --2- 172.21.15.131:0/878679549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98140ced00 0x7f981406dea0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 -- 172.21.15.131:0/878679549 >> 172.21.15.131:0/878679549 conn(0x7f981406eea0 msgr2=0x7f981406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:15.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 -- 172.21.15.131:0/878679549 shutdown_connections 2024-09-17T02:41:15.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 -- 172.21.15.131:0/878679549 wait complete. 2024-09-17T02:41:15.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 Processor -- start 2024-09-17T02:41:15.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 -- start start 2024-09-17T02:41:15.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f981406e3e0 0x7f98141f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:15.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98140ced00 0x7f9814151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:15.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f981415c5b0 0x7f981415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:15.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9814168720 con 0x7f981415c5b0 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98141685a0 con 0x7f98140ced00 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9819a84640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f98141688a0 con 0x7f981406e3e0 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9813fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f981415c5b0 0x7f981415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9813fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f981415c5b0 0x7f981415a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57680/0 (socket says 172.21.15.131:57680) 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9813fff640 1 -- 172.21.15.131:0/466263458 learned_addr learned my addr 172.21.15.131:0/466263458 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9812ffd640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98140ced00 0x7f9814151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9813fff640 1 -- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f981406e3e0 msgr2=0x7f98141f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9813fff640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f981406e3e0 0x7f98141f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9813fff640 1 -- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98140ced00 msgr2=0x7f9814151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9813fff640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98140ced00 0x7f9814151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.286+0000 7f9813fff640 1 -- 172.21.15.131:0/466263458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98141600a0 con 0x7f981415c5b0 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.290+0000 7f9813fff640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f981415c5b0 0x7f981415a4b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f980c00e940 tx=0x7f980c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:15.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.290+0000 7f97fbfff640 1 -- 172.21.15.131:0/466263458 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c004d30 con 0x7f981415c5b0 2024-09-17T02:41:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.290+0000 7f97fbfff640 1 -- 172.21.15.131:0/466263458 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f980c00cb20 con 0x7f981415c5b0 2024-09-17T02:41:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.290+0000 7f97fbfff640 1 -- 172.21.15.131:0/466263458 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f980c005020 con 0x7f981415c5b0 2024-09-17T02:41:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.290+0000 7f9819a84640 1 -- 172.21.15.131:0/466263458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98141f8c80 con 0x7f981415c5b0 2024-09-17T02:41:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.290+0000 7f9819a84640 1 -- 172.21.15.131:0/466263458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98141f9240 con 0x7f981415c5b0 2024-09-17T02:41:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.290+0000 7f9819a84640 1 -- 172.21.15.131:0/466263458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98141f89f0 con 0x7f981415c5b0 2024-09-17T02:41:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.290+0000 7f97fbfff640 1 -- 172.21.15.131:0/466263458 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f980c0051c0 con 0x7f981415c5b0 2024-09-17T02:41:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.290+0000 7f97fbfff640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9800073230 0x7f9800075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.290+0000 7f97fbfff640 1 -- 172.21.15.131:0/466263458 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f980c095040 con 0x7f981415c5b0 2024-09-17T02:41:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.290+0000 7f98137fe640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9800073230 0x7f9800075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.294+0000 7f98137fe640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9800073230 0x7f9800075740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f98040af000 tx=0x7f9804019680 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:15.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.294+0000 7f97fbfff640 1 -- 172.21.15.131:0/466263458 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f980c016030 con 0x7f981415c5b0 2024-09-17T02:41:15.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.490+0000 7f9819a84640 1 -- 172.21.15.131:0/466263458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f98141f9440 con 0x7f981415c5b0 2024-09-17T02:41:15.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.498+0000 7f97fbfff640 1 -- 172.21.15.131:0/466263458 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+186461 (secure 0 0 0) 0x7f980c05fe00 con 0x7f981415c5b0 2024-09-17T02:41:15.499 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:15.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 -- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9800073230 msgr2=0x7f9800075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9800073230 0x7f9800075740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f98040af000 tx=0x7f9804019680 comp rx=0 tx=0).stop 2024-09-17T02:41:15.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 -- 172.21.15.131:0/466263458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f981415c5b0 msgr2=0x7f981415a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f981415c5b0 0x7f981415a4b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f980c00e940 tx=0x7f980c00ee00 comp rx=0 tx=0).stop 2024-09-17T02:41:15.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 -- 172.21.15.131:0/466263458 shutdown_connections 2024-09-17T02:41:15.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9800073230 0x7f9800075740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f981415c5b0 0x7f981415a4b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98140ced00 0x7f9814151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 --2- 172.21.15.131:0/466263458 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f981406e3e0 0x7f98141f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 -- 172.21.15.131:0/466263458 >> 172.21.15.131:0/466263458 conn(0x7f981406eea0 msgr2=0x7f9814155400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:15.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 -- 172.21.15.131:0/466263458 shutdown_connections 2024-09-17T02:41:15.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.502+0000 7f9819a84640 1 -- 172.21.15.131:0/466263458 wait complete. 2024-09-17T02:41:15.520 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":5,"flags":0,"active_gid":4129,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6800","nonce":674320934},{"type":"v1","addr":"172.21.15.189:6801","nonce":674320934}]},"active_addr":"172.21.15.189:6801/674320934","active_change":"2024-09-17T02:40:31.099913+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4135,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":1538954634}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":1936377939}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":3265564186}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.189:0","nonce":1684324887}]}]} 2024-09-17T02:41:15.524 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T02:41:15.524 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T02:41:15.524 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:41:15.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.682+0000 7f71f1130640 1 -- 172.21.15.131:0/550815059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec1566f0 msgr2=0x7f71ec156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.682+0000 7f71f1130640 1 --2- 172.21.15.131:0/550815059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec1566f0 0x7f71ec156b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f71ec070330 tx=0x7f71dc01c850 comp rx=0 tx=0).stop 2024-09-17T02:41:15.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.682+0000 7f71f1130640 1 -- 172.21.15.131:0/550815059 shutdown_connections 2024-09-17T02:41:15.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.682+0000 7f71f1130640 1 --2- 172.21.15.131:0/550815059 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f71ec0cc750 0x7f71ec0ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.682+0000 7f71f1130640 1 --2- 172.21.15.131:0/550815059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec1566f0 0x7f71ec156b00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.682+0000 7f71f1130640 1 -- 172.21.15.131:0/550815059 >> 172.21.15.131:0/550815059 conn(0x7f71ec0d2100 msgr2=0x7f71ec0d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:15.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.682+0000 7f71f1130640 1 -- 172.21.15.131:0/550815059 shutdown_connections 2024-09-17T02:41:15.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.682+0000 7f71f1130640 1 -- 172.21.15.131:0/550815059 wait complete. 2024-09-17T02:41:15.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.682+0000 7f71f1130640 1 Processor -- start 2024-09-17T02:41:15.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.682+0000 7f71f1130640 1 -- start start 2024-09-17T02:41:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71f1130640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec0cc750 0x7f71ec0c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71f1130640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ec1566f0 0x7f71ec0c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71f1130640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f71ec15cfa0 0x7f71ec14f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71f1130640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec071700 con 0x7f71ec0cc750 2024-09-17T02:41:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71f1130640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71ec071580 con 0x7f71ec1566f0 2024-09-17T02:41:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71f1130640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f71ec071880 con 0x7f71ec15cfa0 2024-09-17T02:41:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ead9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec0cc750 0x7f71ec0c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:15.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ead9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec0cc750 0x7f71ec0c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57692/0 (socket says 172.21.15.131:57692) 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ead9d640 1 -- 172.21.15.131:0/3370514278 learned_addr learned my addr 172.21.15.131:0/3370514278 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ea59c640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ec1566f0 0x7f71ec0c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ead9d640 1 -- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f71ec15cfa0 msgr2=0x7f71ec14f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71eb59e640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f71ec15cfa0 0x7f71ec14f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ead9d640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f71ec15cfa0 0x7f71ec14f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ead9d640 1 -- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ec1566f0 msgr2=0x7f71ec0c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ead9d640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ec1566f0 0x7f71ec0c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ead9d640 1 -- 172.21.15.131:0/3370514278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ec0c70c0 con 0x7f71ec0cc750 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ea59c640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ec1566f0 0x7f71ec0c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ead9d640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec0cc750 0x7f71ec0c5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b0040 tx=0x7f71dc0a5590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71d37fe640 1 -- 172.21.15.131:0/3370514278 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0b2040 con 0x7f71ec0cc750 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71d37fe640 1 -- 172.21.15.131:0/3370514278 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71dc0043b0 con 0x7f71ec0cc750 2024-09-17T02:41:15.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71f1130640 1 -- 172.21.15.131:0/3370514278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ec0c73a0 con 0x7f71ec0cc750 2024-09-17T02:41:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71d37fe640 1 -- 172.21.15.131:0/3370514278 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc003860 con 0x7f71ec0cc750 2024-09-17T02:41:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71f1130640 1 -- 172.21.15.131:0/3370514278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ec0c9680 con 0x7f71ec0cc750 2024-09-17T02:41:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71f1130640 1 -- 172.21.15.131:0/3370514278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ec0c9190 con 0x7f71ec0cc750 2024-09-17T02:41:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71d37fe640 1 -- 172.21.15.131:0/3370514278 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f71dc003a00 con 0x7f71ec0cc750 2024-09-17T02:41:15.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71d37fe640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71d8073200 0x7f71d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:15.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71d37fe640 1 -- 172.21.15.131:0/3370514278 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f71dc12dfd0 con 0x7f71ec0cc750 2024-09-17T02:41:15.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.686+0000 7f71ea59c640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71d8073200 0x7f71d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:15.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.690+0000 7f71ea59c640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71d8073200 0x7f71d8075710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0c7b40 tx=0x7f71e0005cc0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:15.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.690+0000 7f71d37fe640 1 -- 172.21.15.131:0/3370514278 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f71dc0164d0 con 0x7f71ec0cc750 2024-09-17T02:41:15.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.850+0000 7f71f1130640 1 -- 172.21.15.131:0/3370514278 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71ec0c7570 con 0x7f71ec0cc750 2024-09-17T02:41:15.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.850+0000 7f71d37fe640 1 -- 172.21.15.131:0/3370514278 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18650 (secure 0 0 0) 0x7f71ec0c7570 con 0x7f71ec0cc750 2024-09-17T02:41:15.850 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:15.850 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:56.713412+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:41:15.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 -- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71d8073200 msgr2=0x7f71d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71d8073200 0x7f71d8075710 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0c7b40 tx=0x7f71e0005cc0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 -- 172.21.15.131:0/3370514278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec0cc750 msgr2=0x7f71ec0c5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:15.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec0cc750 0x7f71ec0c5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b0040 tx=0x7f71dc0a5590 comp rx=0 tx=0).stop 2024-09-17T02:41:15.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 -- 172.21.15.131:0/3370514278 shutdown_connections 2024-09-17T02:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71d8073200 0x7f71d8075710 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f71ec15cfa0 0x7f71ec14f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71ec1566f0 0x7f71ec0c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 --2- 172.21.15.131:0/3370514278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71ec0cc750 0x7f71ec0c5150 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 -- 172.21.15.131:0/3370514278 >> 172.21.15.131:0/3370514278 conn(0x7f71ec0d2100 msgr2=0x7f71ec1520f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 -- 172.21.15.131:0/3370514278 shutdown_connections 2024-09-17T02:41:15.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:15.854+0000 7f71f1130640 1 -- 172.21.15.131:0/3370514278 wait complete. 2024-09-17T02:41:15.871 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T02:41:15.871 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:41:16.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.010+0000 7feb3a7ed640 1 -- 172.21.15.131:0/2670723677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb3406dea0 msgr2=0x7feb3406e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.010+0000 7feb3a7ed640 1 --2- 172.21.15.131:0/2670723677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb3406dea0 0x7feb3406e2b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feb28009900 tx=0x7feb2801c8b0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.010+0000 7feb3a7ed640 1 -- 172.21.15.131:0/2670723677 shutdown_connections 2024-09-17T02:41:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.010+0000 7feb3a7ed640 1 --2- 172.21.15.131:0/2670723677 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feb340cab60 0x7feb340caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.010+0000 7feb3a7ed640 1 --2- 172.21.15.131:0/2670723677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb3406dea0 0x7feb3406e2b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.010+0000 7feb3a7ed640 1 -- 172.21.15.131:0/2670723677 >> 172.21.15.131:0/2670723677 conn(0x7feb3406eea0 msgr2=0x7feb3406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.010+0000 7feb3a7ed640 1 -- 172.21.15.131:0/2670723677 shutdown_connections 2024-09-17T02:41:16.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.010+0000 7feb3a7ed640 1 -- 172.21.15.131:0/2670723677 wait complete. 2024-09-17T02:41:16.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb3a7ed640 1 Processor -- start 2024-09-17T02:41:16.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb3a7ed640 1 -- start start 2024-09-17T02:41:16.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb3a7ed640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb3406dea0 0x7feb3414d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb3a7ed640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb340cab60 0x7feb3415a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb3a7ed640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feb34155240 0x7feb3413edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb3a7ed640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb34168480 con 0x7feb3406dea0 2024-09-17T02:41:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb3a7ed640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb34168300 con 0x7feb340cab60 2024-09-17T02:41:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb3a7ed640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7feb34168600 con 0x7feb34155240 2024-09-17T02:41:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb33fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb3406dea0 0x7feb3414d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb33fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb3406dea0 0x7feb3414d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57724/0 (socket says 172.21.15.131:57724) 2024-09-17T02:41:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb33fff640 1 -- 172.21.15.131:0/1975377696 learned_addr learned my addr 172.21.15.131:0/1975377696 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb33fff640 1 -- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feb34155240 msgr2=0x7feb3413edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb337fe640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb340cab60 0x7feb3415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb33fff640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feb34155240 0x7feb3413edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb33fff640 1 -- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb340cab60 msgr2=0x7feb3415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb33fff640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb340cab60 0x7feb3415a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb33fff640 1 -- 172.21.15.131:0/1975377696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb341506c0 con 0x7feb3406dea0 2024-09-17T02:41:16.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb33fff640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb3406dea0 0x7feb3414d7b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb2400e690 tx=0x7feb2400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb30ff9640 1 -- 172.21.15.131:0/1975377696 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb240052d0 con 0x7feb3406dea0 2024-09-17T02:41:16.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb30ff9640 1 -- 172.21.15.131:0/1975377696 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb240016d0 con 0x7feb3406dea0 2024-09-17T02:41:16.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb3a7ed640 1 -- 172.21.15.131:0/1975377696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb341509a0 con 0x7feb3406dea0 2024-09-17T02:41:16.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb30ff9640 1 -- 172.21.15.131:0/1975377696 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24001b20 con 0x7feb3406dea0 2024-09-17T02:41:16.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.014+0000 7feb3a7ed640 1 -- 172.21.15.131:0/1975377696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb34150f60 con 0x7feb3406dea0 2024-09-17T02:41:16.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.018+0000 7feb3a7ed640 1 -- 172.21.15.131:0/1975377696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb3406e2b0 con 0x7feb3406dea0 2024-09-17T02:41:16.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.018+0000 7feb30ff9640 1 -- 172.21.15.131:0/1975377696 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feb24003620 con 0x7feb3406dea0 2024-09-17T02:41:16.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.018+0000 7feb30ff9640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb20073230 0x7feb20075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.018+0000 7feb30ff9640 1 -- 172.21.15.131:0/1975377696 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7feb240968e0 con 0x7feb3406dea0 2024-09-17T02:41:16.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.018+0000 7feb337fe640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb20073230 0x7feb20075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.018+0000 7feb337fe640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb20073230 0x7feb20075740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb280af000 tx=0x7feb2801b0f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.022+0000 7feb30ff9640 1 -- 172.21.15.131:0/1975377696 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feb24016020 con 0x7feb3406dea0 2024-09-17T02:41:16.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.190+0000 7feb3a7ed640 1 -- 172.21.15.131:0/1975377696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb3406e9d0 con 0x7feb3406dea0 2024-09-17T02:41:16.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb30ff9640 1 -- 172.21.15.131:0/1975377696 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18650 (secure 0 0 0) 0x7feb3406e7f0 con 0x7feb3406dea0 2024-09-17T02:41:16.193 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:16.193 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:56.713412+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 -- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb20073230 msgr2=0x7feb20075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb20073230 0x7feb20075740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb280af000 tx=0x7feb2801b0f0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 -- 172.21.15.131:0/1975377696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb3406dea0 msgr2=0x7feb3414d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb3406dea0 0x7feb3414d7b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7feb2400e690 tx=0x7feb2400eb50 comp rx=0 tx=0).stop 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 -- 172.21.15.131:0/1975377696 shutdown_connections 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb20073230 0x7feb20075740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feb34155240 0x7feb3413edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb340cab60 0x7feb3415a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 --2- 172.21.15.131:0/1975377696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb3406dea0 0x7feb3414d7b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 -- 172.21.15.131:0/1975377696 >> 172.21.15.131:0/1975377696 conn(0x7feb3406eea0 msgr2=0x7feb340b1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 -- 172.21.15.131:0/1975377696 shutdown_connections 2024-09-17T02:41:16.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.194+0000 7feb3a7ed640 1 -- 172.21.15.131:0/1975377696 wait complete. 2024-09-17T02:41:16.213 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T02:41:16.214 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T02:41:16.214 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T02:41:16.214 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T02:41:16.215 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T02:41:16.215 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T02:41:16.215 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T02:41:16.216 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T02:41:16.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fccbda5f640 1 -- 172.21.15.131:0/3970643083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb8074370 msgr2=0x7fccb816f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fccbda5f640 1 --2- 172.21.15.131:0/3970643083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb8074370 0x7fccb816f330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fccb80b4c80 tx=0x7fccb001c910 comp rx=0 tx=0).stop 2024-09-17T02:41:16.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fccbda5f640 1 -- 172.21.15.131:0/3970643083 shutdown_connections 2024-09-17T02:41:16.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fccbda5f640 1 --2- 172.21.15.131:0/3970643083 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb8074370 0x7fccb816f330 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fccbda5f640 1 --2- 172.21.15.131:0/3970643083 >> v2:172.21.15.189:3300/0 conn(0x7fccb806fd70 0x7fccb808b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fccbda5f640 1 -- 172.21.15.131:0/3970643083 >> 172.21.15.131:0/3970643083 conn(0x7fccb80d0ad0 msgr2=0x7fccb806ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fccbda5f640 1 -- 172.21.15.131:0/3970643083 shutdown_connections 2024-09-17T02:41:16.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fccbda5f640 1 -- 172.21.15.131:0/3970643083 wait complete. 2024-09-17T02:41:16.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fe4bdd43640 1 -- 172.21.15.131:0/4084035931 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4b80b3f70 msgr2=0x7fe4b8073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fe4bdd43640 1 --2- 172.21.15.131:0/4084035931 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4b80b3f70 0x7fe4b8073430 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009870 tx=0x7fe4a801c310 comp rx=0 tx=0).stop 2024-09-17T02:41:16.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fe4bdd43640 1 -- 172.21.15.131:0/4084035931 shutdown_connections 2024-09-17T02:41:16.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fe4bdd43640 1 --2- 172.21.15.131:0/4084035931 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4b80b3f70 0x7fe4b8073430 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fe4bdd43640 1 -- 172.21.15.131:0/4084035931 >> 172.21.15.131:0/4084035931 conn(0x7fe4b80d03f0 msgr2=0x7fe4b80d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fe4bdd43640 1 -- 172.21.15.131:0/4084035931 shutdown_connections 2024-09-17T02:41:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fc07f21b640 1 -- 172.21.15.131:0/1814855781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0780cf670 msgr2=0x7fc07806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fc07f21b640 1 --2- 172.21.15.131:0/1814855781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0780cf670 0x7fc07806d720 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00aa10 tx=0x7fc06c01dbc0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7fc07f21b640 1 -- 172.21.15.131:0/1814855781 shutdown_connections 2024-09-17T02:41:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7fc07f21b640 1 --2- 172.21.15.131:0/1814855781 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc078151750 0x7fc078072e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7fc07f21b640 1 --2- 172.21.15.131:0/1814855781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0780cf670 0x7fc07806d720 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7fc07f21b640 1 -- 172.21.15.131:0/1814855781 >> 172.21.15.131:0/1814855781 conn(0x7fc0780d03f0 msgr2=0x7fc0780d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7fccbda5f640 1 Processor -- start 2024-09-17T02:41:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7fccbda5f640 1 -- start start 2024-09-17T02:41:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7fccbda5f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb806fd70 0x7fccb820d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f96b0ecb640 1 -- 172.21.15.131:0/3196368212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96ac0cd030 msgr2=0x7f96ac06ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f96b0ecb640 1 --2- 172.21.15.131:0/3196368212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96ac0cd030 0x7f96ac06ffb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f96ac0735a0 tx=0x7f969c01cd30 comp rx=0 tx=0).stop 2024-09-17T02:41:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f96b0ecb640 1 -- 172.21.15.131:0/3196368212 shutdown_connections 2024-09-17T02:41:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f96b0ecb640 1 --2- 172.21.15.131:0/3196368212 >> v2:172.21.15.131:3300/0 conn(0x7f96ac15ea20 0x7f96ac0ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f96b0ecb640 1 --2- 172.21.15.131:0/3196368212 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ac15ee30 0x7f96ac0ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f96b0ecb640 1 --2- 172.21.15.131:0/3196368212 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96ac0cd030 0x7f96ac06ffb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f96b0ecb640 1 -- 172.21.15.131:0/3196368212 >> 172.21.15.131:0/3196368212 conn(0x7f96ac0cd8b0 msgr2=0x7f96ac0cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.446+0000 7fe4bdd43640 1 -- 172.21.15.131:0/4084035931 wait complete. 2024-09-17T02:41:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7fe4bdd43640 1 Processor -- start 2024-09-17T02:41:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7fc07f21b640 1 -- 172.21.15.131:0/1814855781 shutdown_connections 2024-09-17T02:41:16.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f93cd999640 1 -- 172.21.15.131:0/2393807750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93b8007930 msgr2=0x7f93b8009e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f93cd999640 1 --2- 172.21.15.131:0/2393807750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93b8007930 0x7f93b8009e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c700 tx=0x7f93b401c670 comp rx=0 tx=0).stop 2024-09-17T02:41:16.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f96b0ecb640 1 -- 172.21.15.131:0/3196368212 shutdown_connections 2024-09-17T02:41:16.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f96b0ecb640 1 -- 172.21.15.131:0/3196368212 wait complete. 2024-09-17T02:41:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f93cd999640 1 -- 172.21.15.131:0/2393807750 shutdown_connections 2024-09-17T02:41:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f93cd999640 1 --2- 172.21.15.131:0/2393807750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93b8007930 0x7f93b8009e10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f93cd999640 1 -- 172.21.15.131:0/2393807750 >> 172.21.15.131:0/2393807750 conn(0x7f93c806ce90 msgr2=0x7f93c806d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f93cd999640 1 -- 172.21.15.131:0/2393807750 shutdown_connections 2024-09-17T02:41:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccbda5f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccb8074370 0x7fccb82113d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccbda5f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccb808b7d0 0x7fccb8211910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccbda5f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccb817ab20 con 0x7fccb806fd70 2024-09-17T02:41:16.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccbda5f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fccb817a9a0 con 0x7fccb8074370 2024-09-17T02:41:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccbda5f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fccb817aca0 con 0x7fccb808b7d0 2024-09-17T02:41:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccb77fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccb808b7d0 0x7fccb8211910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccb77fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccb808b7d0 0x7fccb8211910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.131:36512/0 (socket says 172.21.15.131:36512) 2024-09-17T02:41:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccb77fe640 1 -- 172.21.15.131:0/1541066977 learned_addr learned my addr 172.21.15.131:0/1541066977 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccb77fe640 1 -- 172.21.15.131:0/1541066977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccb8074370 msgr2=0x7fccb82113d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T02:41:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccb77fe640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccb8074370 0x7fccb82113d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccb77fe640 1 -- 172.21.15.131:0/1541066977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb806fd70 msgr2=0x7fccb820d810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T02:41:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccb77fe640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccb806fd70 0x7fccb820d810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccb77fe640 1 -- 172.21.15.131:0/1541066977 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb816a3e0 con 0x7fccb808b7d0 2024-09-17T02:41:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccb77fe640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccb808b7d0 0x7fccb8211910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fccb001cdf0 tx=0x7fccb0008ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccaffff640 1 -- 172.21.15.131:0/1541066977 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00a5a70 con 0x7fccb808b7d0 2024-09-17T02:41:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccbda5f640 1 -- 172.21.15.131:0/1541066977 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb816a6c0 con 0x7fccb808b7d0 2024-09-17T02:41:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccbda5f640 1 -- 172.21.15.131:0/1541066977 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb816ac80 con 0x7fccb808b7d0 2024-09-17T02:41:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7f96b0ecb640 1 Processor -- start 2024-09-17T02:41:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7fe4bdd43640 1 -- start start 2024-09-17T02:41:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4bdd43640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4b80b3f70 0x7fe4b8161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4bdd43640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b80b4e60 0x7fe4b81553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b80b4e60 0x7fe4b81553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b80b4e60 0x7fe4b81553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57762/0 (socket says 172.21.15.131:57762) 2024-09-17T02:41:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b77fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4b80b3f70 0x7fe4b8161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b77fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4b80b3f70 0x7fe4b8161490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.131:36518/0 (socket says 172.21.15.131:36518) 2024-09-17T02:41:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b6ffd640 1 -- 172.21.15.131:0/1024191227 learned_addr learned my addr 172.21.15.131:0/1024191227 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4bdd43640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b815a660 0x7fe4b8155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8165790 con 0x7fe4b80b4e60 2024-09-17T02:41:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8165610 con 0x7fe4b815a660 2024-09-17T02:41:16.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b8165910 con 0x7fe4b80b3f70 2024-09-17T02:41:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7fc07f21b640 1 -- 172.21.15.131:0/1814855781 wait complete. 2024-09-17T02:41:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fc07f21b640 1 Processor -- start 2024-09-17T02:41:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fc07f21b640 1 -- start start 2024-09-17T02:41:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.450+0000 7f93cd999640 1 -- 172.21.15.131:0/2393807750 wait complete. 2024-09-17T02:41:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7f93cd999640 1 Processor -- start 2024-09-17T02:41:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccaffff640 1 -- 172.21.15.131:0/1541066977 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccb00cb030 con 0x7fccb808b7d0 2024-09-17T02:41:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccaffff640 1 -- 172.21.15.131:0/1541066977 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00a56c0 con 0x7fccb808b7d0 2024-09-17T02:41:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7f96b0ecb640 1 -- start start 2024-09-17T02:41:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b7fff640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b815a660 0x7fe4b8155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b6ffd640 1 -- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4b80b3f70 msgr2=0x7fe4b8161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b6ffd640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4b80b3f70 0x7fe4b8161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b6ffd640 1 -- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b815a660 msgr2=0x7fe4b8155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b6ffd640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b815a660 0x7fe4b8155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b6ffd640 1 -- 172.21.15.131:0/1024191227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b8161f30 con 0x7fe4b80b4e60 2024-09-17T02:41:16.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b7fff640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe4b815a660 0x7fe4b8155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:16.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fccbda5f640 1 -- 172.21.15.131:0/1541066977 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcc9c001040 con 0x7fccb808b7d0 2024-09-17T02:41:16.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96b0ecb640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ac0cd030 0x7f96ac161860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96b0ecb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ac15ea20 0x7f96ac1491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96b0ecb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96ac15ee30 0x7f96ac151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96b0ecb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac162720 con 0x7f96ac15ea20 2024-09-17T02:41:16.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96b0ecb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96ac1625a0 con 0x7f96ac15ee30 2024-09-17T02:41:16.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96b0ecb640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f96ac1628a0 con 0x7f96ac0cd030 2024-09-17T02:41:16.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fe4b6ffd640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b80b4e60 0x7fe4b81553d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8000c10 tx=0x7fe4a80a47c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fe49ffff640 1 -- 172.21.15.131:0/1024191227 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a80ba070 con 0x7fe4b80b4e60 2024-09-17T02:41:16.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07f21b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc07806dc60 0x7fc07814ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07f21b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0780cf670 0x7fc07814f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07f21b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc078151750 0x7fc078209840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07f21b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc078073780 con 0x7fc078151750 2024-09-17T02:41:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07f21b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc078073600 con 0x7fc0780cf670 2024-09-17T02:41:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07f21b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc078073900 con 0x7fc07806dc60 2024-09-17T02:41:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc077fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0780cf670 0x7fc07814f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc077fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0780cf670 0x7fc07814f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.131:55802/0 (socket says 172.21.15.131:55802) 2024-09-17T02:41:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc077fff640 1 -- 172.21.15.131:0/1254066984 learned_addr learned my addr 172.21.15.131:0/1254066984 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07cfb7640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc07806dc60 0x7fc07814ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7f93cd999640 1 -- start start 2024-09-17T02:41:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f93cd999640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93b8007930 0x7f93c80b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fd8e1f90640 1 -- 172.21.15.131:0/587277096 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd8dc0cf140 msgr2=0x7fd8dc06feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.454+0000 7fd8e1f90640 1 --2- 172.21.15.131:0/587277096 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd8dc0cf140 0x7fd8dc06feb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009930 tx=0x7fd8c801c8e0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96aa59c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ac0cd030 0x7f96ac161860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fd8e1f90640 1 -- 172.21.15.131:0/587277096 shutdown_connections 2024-09-17T02:41:16.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fd8e1f90640 1 --2- 172.21.15.131:0/587277096 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd8dc0cf140 0x7fd8dc06feb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fd8e1f90640 1 --2- 172.21.15.131:0/587277096 >> v2:172.21.15.189:3300/0 conn(0x7fd8dc15f630 0x7fd8dc0af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fd8e1f90640 1 -- 172.21.15.131:0/587277096 >> 172.21.15.131:0/587277096 conn(0x7fd8dc0cfdc0 msgr2=0x7fd8dc0d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96aa59c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ac0cd030 0x7f96ac161860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.131:36534/0 (socket says 172.21.15.131:36534) 2024-09-17T02:41:16.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96aa59c640 1 -- 172.21.15.131:0/2943263207 learned_addr learned my addr 172.21.15.131:0/2943263207 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fd8e1f90640 1 -- 172.21.15.131:0/587277096 shutdown_connections 2024-09-17T02:41:16.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fd8e1f90640 1 -- 172.21.15.131:0/587277096 wait complete. 2024-09-17T02:41:16.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b8162210 con 0x7fe4b80b4e60 2024-09-17T02:41:16.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b81627d0 con 0x7fe4b80b4e60 2024-09-17T02:41:16.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fe49ffff640 1 -- 172.21.15.131:0/1024191227 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4a80053e0 con 0x7fe4b80b4e60 2024-09-17T02:41:16.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fe49ffff640 1 -- 172.21.15.131:0/1024191227 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a80b1040 con 0x7fe4b80b4e60 2024-09-17T02:41:16.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f93cd999640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c8152840 0x7f93c80b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f93cc997640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93b8007930 0x7f93c80b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f93cc997640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93b8007930 0x7f93c80b0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.131:36544/0 (socket says 172.21.15.131:36544) 2024-09-17T02:41:16.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07cfb7640 1 -- 172.21.15.131:0/1254066984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0780cf670 msgr2=0x7fc07814f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07cfb7640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0780cf670 0x7fc07814f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07cfb7640 1 -- 172.21.15.131:0/1254066984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc078151750 msgr2=0x7fc078209840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07cfb7640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc078151750 0x7fc078209840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07cfb7640 1 -- 172.21.15.131:0/1254066984 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc078152790 con 0x7fc07806dc60 2024-09-17T02:41:16.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f93cc997640 1 -- 172.21.15.131:0/4080345660 learned_addr learned my addr 172.21.15.131:0/4080345660 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f93cd999640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c81fd670 0x7f93c80cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96aa59c640 1 -- 172.21.15.131:0/2943263207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96ac15ee30 msgr2=0x7f96ac151d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T02:41:16.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96aa59c640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96ac15ee30 0x7f96ac151d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07cfb7640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc07806dc60 0x7fc07814ed70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc078070d60 tx=0x7fc06c0a6a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f93c7fff640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c8152840 0x7f93c80b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96aa59c640 1 -- 172.21.15.131:0/2943263207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ac15ea20 msgr2=0x7f96ac1491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96aa59c640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ac15ea20 0x7f96ac1491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96aa59c640 1 -- 172.21.15.131:0/2943263207 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ac0b49d0 con 0x7f96ac0cd030 2024-09-17T02:41:16.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96aa59c640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ac0cd030 0x7f96ac161860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f969c01b3e0 tx=0x7f969c0a5c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc0757fa640 1 -- 172.21.15.131:0/1254066984 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0bc070 con 0x7fc07806dc60 2024-09-17T02:41:16.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc0757fa640 1 -- 172.21.15.131:0/1254066984 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc06c003c40 con 0x7fc07806dc60 2024-09-17T02:41:16.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc0757fa640 1 -- 172.21.15.131:0/1254066984 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0b3070 con 0x7fc07806dc60 2024-09-17T02:41:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f93cd198640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c81fd670 0x7f93c80cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f93c815e260 con 0x7f93c81fd670 2024-09-17T02:41:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fccaffff640 1 -- 172.21.15.131:0/1541066977 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fccb00090d0 con 0x7fccb808b7d0 2024-09-17T02:41:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fccaffff640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc980733d0 0x7fcc980758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fccaffff640 1 -- 172.21.15.131:0/1541066977 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fccb0140330 con 0x7fccb808b7d0 2024-09-17T02:41:16.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fccaffff640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x7fcc9807cb20 0x7fcc9807efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fccb6ffd640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc980733d0 0x7fcc980758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fccaffff640 1 -- 172.21.15.131:0/1541066977 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fccb00a8ec0 con 0x7fcc9807cb20 2024-09-17T02:41:16.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fccaffff640 1 -- 172.21.15.131:0/1541066977 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fccb0140720 con 0x7fccb808b7d0 2024-09-17T02:41:16.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07f21b640 1 -- 172.21.15.131:0/1254066984 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc078152a70 con 0x7fc07806dc60 2024-09-17T02:41:16.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07f21b640 1 -- 172.21.15.131:0/1254066984 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc078153030 con 0x7fc07806dc60 2024-09-17T02:41:16.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fccb67fc640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x7fcc9807cb20 0x7fcc9807efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe498001040 con 0x7fe4b80b4e60 2024-09-17T02:41:16.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96a2ffd640 1 -- 172.21.15.131:0/2943263207 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c0b2040 con 0x7f96ac0cd030 2024-09-17T02:41:16.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96a2ffd640 1 -- 172.21.15.131:0/2943263207 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f969c0aeac0 con 0x7f96ac0cd030 2024-09-17T02:41:16.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96a2ffd640 1 -- 172.21.15.131:0/2943263207 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969c005000 con 0x7f96ac0cd030 2024-09-17T02:41:16.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96b0ecb640 1 -- 172.21.15.131:0/2943263207 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ac0b4cb0 con 0x7f96ac0cd030 2024-09-17T02:41:16.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7f96b0ecb640 1 -- 172.21.15.131:0/2943263207 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ac0b5270 con 0x7f96ac0cd030 2024-09-17T02:41:16.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fc07f21b640 1 -- 172.21.15.131:0/1254066984 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc078153270 con 0x7fc07806dc60 2024-09-17T02:41:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fccb67fc640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x7fcc9807cb20 0x7fcc9807efb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fd8e1f90640 1 Processor -- start 2024-09-17T02:41:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.458+0000 7fd8e1f90640 1 -- start start 2024-09-17T02:41:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93c8070250 con 0x7f93c8152840 2024-09-17T02:41:16.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f93c815e3e0 con 0x7f93b8007930 2024-09-17T02:41:16.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f96b0ecb640 1 -- 172.21.15.131:0/2943263207 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9690001040 con 0x7f96ac0cd030 2024-09-17T02:41:16.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fccaffff640 1 -- 172.21.15.131:0/1541066977 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fccb00a8ec0 con 0x7fcc9807cb20 2024-09-17T02:41:16.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f51065e1640 1 -- 172.21.15.131:0/291710841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51000cfb70 msgr2=0x7f5100161980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f51065e1640 1 --2- 172.21.15.131:0/291710841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51000cfb70 0x7f5100161980 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f510008d230 tx=0x7f50f001c7f0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8e1f90640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd8dc0703f0 0x7fd8dc150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8e1f90640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8dc0cf140 0x7fd8dc0729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f51065e1640 1 -- 172.21.15.131:0/291710841 shutdown_connections 2024-09-17T02:41:16.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f51065e1640 1 --2- 172.21.15.131:0/291710841 >> v2:172.21.15.131:3300/0 conn(0x7f5100155050 0x7f5100155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f51065e1640 1 --2- 172.21.15.131:0/291710841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f51000cfb70 0x7f5100161980 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f51065e1640 1 -- 172.21.15.131:0/291710841 >> 172.21.15.131:0/291710841 conn(0x7f51000d1ba0 msgr2=0x7f51000d4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8e1f90640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8dc15f630 0x7fd8dc072ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fccb6ffd640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc980733d0 0x7fcc980758e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcca000aa40 tx=0x7fcca000a2f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93cd198640 1 -- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93b8007930 msgr2=0x7f93c80b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93cd198640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93b8007930 0x7f93c80b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93cd198640 1 -- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c8152840 msgr2=0x7f93c80b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93cd198640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c8152840 0x7f93c80b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93cd198640 1 -- 172.21.15.131:0/4080345660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c8074460 con 0x7f93c81fd670 2024-09-17T02:41:16.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8db7fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd8dc0703f0 0x7fd8dc150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8daffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8dc0cf140 0x7fd8dc0729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8db7fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd8dc0703f0 0x7fd8dc150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.131:36546/0 (socket says 172.21.15.131:36546) 2024-09-17T02:41:16.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8daffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8dc0cf140 0x7fd8dc0729b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57798/0 (socket says 172.21.15.131:57798) 2024-09-17T02:41:16.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8db7fe640 1 -- 172.21.15.131:0/1685324202 learned_addr learned my addr 172.21.15.131:0/1685324202 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8e1f90640 1 -- 172.21.15.131:0/1685324202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc1541b0 con 0x7fd8dc0cf140 2024-09-17T02:41:16.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8e1f90640 1 -- 172.21.15.131:0/1685324202 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd8dc154030 con 0x7fd8dc15f630 2024-09-17T02:41:16.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8e1f90640 1 -- 172.21.15.131:0/1685324202 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd8dc154330 con 0x7fd8dc0703f0 2024-09-17T02:41:16.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fe49ffff640 1 -- 172.21.15.131:0/1024191227 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe4a80bd070 con 0x7fe4b80b4e60 2024-09-17T02:41:16.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8dbfff640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8dc15f630 0x7fd8dc072ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f51065e1640 1 -- 172.21.15.131:0/291710841 shutdown_connections 2024-09-17T02:41:16.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fe49ffff640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe4a4073240 0x7fe4a4075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fc0757fa640 1 -- 172.21.15.131:0/1254066984 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc06c0bf070 con 0x7fc07806dc60 2024-09-17T02:41:16.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fe49ffff640 1 -- 172.21.15.131:0/1024191227 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fe4a812d5b0 con 0x7fe4b80b4e60 2024-09-17T02:41:16.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fe49ffff640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x7fe4a407ca10 0x7fe4a407eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fe49ffff640 1 -- 172.21.15.131:0/1024191227 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4a800b0b0 con 0x7fe4a407ca10 2024-09-17T02:41:16.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fe49ffff640 1 -- 172.21.15.131:0/1024191227 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe4a812d960 con 0x7fe4b80b4e60 2024-09-17T02:41:16.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8daffd640 1 -- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd8dc0703f0 msgr2=0x7fd8dc150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8daffd640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd8dc0703f0 0x7fd8dc150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8daffd640 1 -- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8dc15f630 msgr2=0x7fd8dc072ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8daffd640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8dc15f630 0x7fd8dc072ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8daffd640 1 -- 172.21.15.131:0/1685324202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8dc153e60 con 0x7fd8dc0cf140 2024-09-17T02:41:16.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fd8daffd640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8dc0cf140 0x7fd8dc0729b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80ac000 tx=0x7fd8c80a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fe4b7fff640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x7fe4a407ca10 0x7fe4a407eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fe4b77fe640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe4a4073240 0x7fe4a4075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fc0757fa640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc0640732d0 0x7fc0640757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fc0757fa640 1 -- 172.21.15.131:0/1254066984 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc06c12f210 con 0x7fc07806dc60 2024-09-17T02:41:16.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fc0757fa640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x7fc06407caa0 0x7fc06407ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fc0757fa640 1 -- 172.21.15.131:0/1254066984 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc06407f600 con 0x7fc06407caa0 2024-09-17T02:41:16.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fc0757fa640 1 -- 172.21.15.131:0/1254066984 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc06c12f600 con 0x7fc07806dc60 2024-09-17T02:41:16.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fc077fff640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc0640732d0 0x7fc0640757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fc07d7b8640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x7fc06407caa0 0x7fc06407ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7fe4b7fff640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x7fe4a407ca10 0x7fe4a407eea0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7fe4b77fe640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe4a4073240 0x7fe4a4075750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0017b0 tx=0x7fe4ac009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7fc07d7b8640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x7fc06407caa0 0x7fc06407ef30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7fc077fff640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc0640732d0 0x7fc0640757e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc078155da0 tx=0x7fc070009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93cd198640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c81fd670 0x7f93c80cb460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f93c80c6cf0 tx=0x7f93c000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93c57fa640 1 -- 172.21.15.131:0/4080345660 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c000cbc0 con 0x7f93c81fd670 2024-09-17T02:41:16.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93c57fa640 1 -- 172.21.15.131:0/4080345660 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93c0017070 con 0x7f93c81fd670 2024-09-17T02:41:16.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93c57fa640 1 -- 172.21.15.131:0/4080345660 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c00116c0 con 0x7f93c81fd670 2024-09-17T02:41:16.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c81e9f70 con 0x7f93c81fd670 2024-09-17T02:41:16.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c81fce70 con 0x7f93c81fd670 2024-09-17T02:41:16.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f93b2ffd640 1 -- 172.21.15.131:0/4080345660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f93a0001040 con 0x7f93c81fd670 2024-09-17T02:41:16.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.462+0000 7f51065e1640 1 -- 172.21.15.131:0/291710841 wait complete. 2024-09-17T02:41:16.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f51065e1640 1 Processor -- start 2024-09-17T02:41:16.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7fd8c7fff640 1 -- 172.21.15.131:0/1685324202 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c80b2040 con 0x7fd8dc0cf140 2024-09-17T02:41:16.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7fd8c7fff640 1 -- 172.21.15.131:0/1685324202 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8c80043d0 con 0x7fd8dc0cf140 2024-09-17T02:41:16.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7fd8c7fff640 1 -- 172.21.15.131:0/1685324202 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8003860 con 0x7fd8dc0cf140 2024-09-17T02:41:16.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7fd8e1f90640 1 -- 172.21.15.131:0/1685324202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8dc0a9460 con 0x7fd8dc0cf140 2024-09-17T02:41:16.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7fd8e1f90640 1 -- 172.21.15.131:0/1685324202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8dc200e20 con 0x7fd8dc0cf140 2024-09-17T02:41:16.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7fc0757fa640 1 -- 172.21.15.131:0/1254066984 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc06407f600 con 0x7fc06407caa0 2024-09-17T02:41:16.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f51065e1640 1 -- start start 2024-09-17T02:41:16.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f51065e1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f51000cfb70 0x7f51000cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f96a2ffd640 1 -- 172.21.15.131:0/2943263207 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f969c0be020 con 0x7f96ac0cd030 2024-09-17T02:41:16.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f96a2ffd640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f968c0732d0 0x7f968c0757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f96a2ffd640 1 -- 172.21.15.131:0/2943263207 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f969c12ec50 con 0x7f96ac0cd030 2024-09-17T02:41:16.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f96a2ffd640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x7f968c07ca90 0x7f968c07ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f96a9d9b640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f968c0732d0 0x7f968c0757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f96a2ffd640 1 -- 172.21.15.131:0/2943263207 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f969c0afe90 con 0x7f968c07ca90 2024-09-17T02:41:16.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f96a2ffd640 1 -- 172.21.15.131:0/2943263207 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f969c12f040 con 0x7f96ac0cd030 2024-09-17T02:41:16.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f51065e1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5100155050 0x7f51001fa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7fd8e1f90640 1 -- 172.21.15.131:0/1685324202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd8dc0a9f40 con 0x7fd8dc0cf140 2024-09-17T02:41:16.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7fd8c7fff640 1 -- 172.21.15.131:0/1685324202 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd8c8003a00 con 0x7fd8dc0cf140 2024-09-17T02:41:16.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f50ff7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5100155050 0x7f51001fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f50ff7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5100155050 0x7f51001fa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57804/0 (socket says 172.21.15.131:57804) 2024-09-17T02:41:16.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f50fffff640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f51000cfb70 0x7f51000cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f50fffff640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f51000cfb70 0x7f51000cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.131:36550/0 (socket says 172.21.15.131:36550) 2024-09-17T02:41:16.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.466+0000 7f50ff7fe640 1 -- 172.21.15.131:0/3152414036 learned_addr learned my addr 172.21.15.131:0/3152414036 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f96aad9d640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x7f968c07ca90 0x7f968c07ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f96a9d9b640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f968c0732d0 0x7f968c0757e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96ac200160 tx=0x7f9694006c00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f93c57fa640 1 -- 172.21.15.131:0/4080345660 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f93c0011860 con 0x7f93c81fd670 2024-09-17T02:41:16.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f51065e1640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5100162330 0x7f51000cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5100160e40 con 0x7f5100155050 2024-09-17T02:41:16.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5100160cc0 con 0x7f5100162330 2024-09-17T02:41:16.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5100160fc0 con 0x7f51000cfb70 2024-09-17T02:41:16.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7fe49ffff640 1 -- 172.21.15.131:0/1024191227 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4a800b0b0 con 0x7fe4a407ca10 2024-09-17T02:41:16.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f5104b7e640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5100162330 0x7f51000cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f96aad9d640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x7f968c07ca90 0x7f968c07ef20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f93c57fa640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93ac073200 0x7f93ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f93c57fa640 1 -- 172.21.15.131:0/4080345660 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f93c0095340 con 0x7f93c81fd670 2024-09-17T02:41:16.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f93cc997640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93ac073200 0x7f93ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f93c57fa640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x7f93ac07c950 0x7f93ac07ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f93c57fa640 1 -- 172.21.15.131:0/4080345660 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93c001ae80 con 0x7f93ac07c950 2024-09-17T02:41:16.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f93c57fa640 1 -- 172.21.15.131:0/4080345660 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f93c00957d0 con 0x7f93c81fd670 2024-09-17T02:41:16.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7fd8c7fff640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8c00732d0 0x7fd8c00757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f50ff7fe640 1 -- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f51000cfb70 msgr2=0x7f51000cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f50ff7fe640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f51000cfb70 0x7f51000cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f50ff7fe640 1 -- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5100162330 msgr2=0x7f51000cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f50ff7fe640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5100162330 0x7f51000cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f50ff7fe640 1 -- 172.21.15.131:0/3152414036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51000cdbb0 con 0x7f5100155050 2024-09-17T02:41:16.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7fd8c7fff640 1 -- 172.21.15.131:0/1685324202 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fd8c812df90 con 0x7fd8dc0cf140 2024-09-17T02:41:16.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f50ff7fe640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5100155050 0x7f51001fa820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f50f400c430 tx=0x7f50f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f93c7fff640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x7f93ac07c950 0x7f93ac07ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7fd8db7fe640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8c00732d0 0x7fd8c00757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f93cc997640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93ac073200 0x7f93ac075710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93bc004710 tx=0x7f93bc00d040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f50fcff9640 1 -- 172.21.15.131:0/3152414036 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4011790 con 0x7f5100155050 2024-09-17T02:41:16.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7fd8c7fff640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x7fd8c007caa0 0x7fd8c007ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7fd8c7fff640 1 -- 172.21.15.131:0/1685324202 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8c8003e90 con 0x7fd8c007caa0 2024-09-17T02:41:16.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7fd8c7fff640 1 -- 172.21.15.131:0/1685324202 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd8c81305f0 con 0x7fd8dc0cf140 2024-09-17T02:41:16.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f50fcff9640 1 -- 172.21.15.131:0/3152414036 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50f4017070 con 0x7f5100155050 2024-09-17T02:41:16.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f50fcff9640 1 -- 172.21.15.131:0/3152414036 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4011cc0 con 0x7f5100155050 2024-09-17T02:41:16.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51000cde90 con 0x7f5100155050 2024-09-17T02:41:16.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51000ce450 con 0x7f5100155050 2024-09-17T02:41:16.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7fd8dbfff640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x7fd8c007caa0 0x7fd8c007ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7fd8db7fe640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8c00732d0 0x7fd8c00757e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009c00 tx=0x7fd8d00091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7fd8dbfff640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x7fd8c007caa0 0x7fd8c007ef30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5100161980 con 0x7f5100155050 2024-09-17T02:41:16.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.470+0000 7f96a2ffd640 1 -- 172.21.15.131:0/2943263207 <== osd.1 v2:172.21.15.131:6824/1616383728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f969c0afe90 con 0x7f968c07ca90 2024-09-17T02:41:16.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f50fcff9640 1 -- 172.21.15.131:0/3152414036 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f50f401c460 con 0x7f5100155050 2024-09-17T02:41:16.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f93c7fff640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x7f93ac07c950 0x7f93ac07ede0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7fd8c7fff640 1 -- 172.21.15.131:0/1685324202 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8c8003e90 con 0x7fd8c007caa0 2024-09-17T02:41:16.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f50fcff9640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f50ec073230 0x7f50ec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f50fcff9640 1 -- 172.21.15.131:0/3152414036 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f50f4094da0 con 0x7f5100155050 2024-09-17T02:41:16.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f50fcff9640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x7f50ec07c980 0x7f50ec07ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f50fcff9640 1 -- 172.21.15.131:0/3152414036 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f50f400ee50 con 0x7f50ec07c980 2024-09-17T02:41:16.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f50fcff9640 1 -- 172.21.15.131:0/3152414036 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50f4095230 con 0x7f5100155050 2024-09-17T02:41:16.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f50fffff640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f50ec073230 0x7f50ec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f5104b7e640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x7f50ec07c980 0x7f50ec07ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f50fffff640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f50ec073230 0x7f50ec075740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50f00af000 tx=0x7f50f001b030 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f93c57fa640 1 -- 172.21.15.131:0/4080345660 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f93c001ae80 con 0x7f93ac07c950 2024-09-17T02:41:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.474+0000 7f5104b7e640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x7f50ec07c980 0x7f50ec07ee10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.478+0000 7f50fcff9640 1 -- 172.21.15.131:0/3152414036 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f50f400ee50 con 0x7f50ec07c980 2024-09-17T02:41:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.478+0000 7fc229ffb640 1 -- 172.21.15.131:0/3451480179 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240ad6a0 con 0x7fc22c0717f0 2024-09-17T02:41:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.478+0000 7fc231a2f640 1 -- 172.21.15.131:0/3451480179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc22c0717f0 msgr2=0x7fc22c06e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.478+0000 7fc231a2f640 1 --2- 172.21.15.131:0/3451480179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc22c0717f0 0x7fc22c06e190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc224009bc0 tx=0x7fc22401cbc0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.482+0000 7fc231a2f640 1 -- 172.21.15.131:0/3451480179 shutdown_connections 2024-09-17T02:41:16.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.482+0000 7fc231a2f640 1 --2- 172.21.15.131:0/3451480179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc22c0717f0 0x7fc22c06e190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.482+0000 7fc231a2f640 1 --2- 172.21.15.131:0/3451480179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc22c0722c0 0x7fc22c0ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.482+0000 7fc231a2f640 1 -- 172.21.15.131:0/3451480179 >> 172.21.15.131:0/3451480179 conn(0x7fc22c0d1130 msgr2=0x7fc22c0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.486+0000 7fc231a2f640 1 -- 172.21.15.131:0/3451480179 shutdown_connections 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.486+0000 7fc231a2f640 1 -- 172.21.15.131:0/3451480179 wait complete. 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.486+0000 7fc231a2f640 1 Processor -- start 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.486+0000 7fc231a2f640 1 -- start start 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc231a2f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc22c0713e0 0x7fc22c14acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc22affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc22c0713e0 0x7fc22c14acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc22affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc22c0713e0 0x7fc22c14acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57822/0 (socket says 172.21.15.131:57822) 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc22affd640 1 -- 172.21.15.131:0/936839720 learned_addr learned my addr 172.21.15.131:0/936839720 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc231a2f640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc22c0717f0 0x7fc22c16e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc231a2f640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc22c0722c0 0x7fc22c0b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc231a2f640 1 -- 172.21.15.131:0/936839720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c15ce20 con 0x7fc22c0713e0 2024-09-17T02:41:16.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc231a2f640 1 -- 172.21.15.131:0/936839720 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c15a920 con 0x7fc22c0717f0 2024-09-17T02:41:16.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc231a2f640 1 -- 172.21.15.131:0/936839720 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc22c15cfa0 con 0x7fc22c0722c0 2024-09-17T02:41:16.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc22a7fc640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc22c0717f0 0x7fc22c16e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc22b7fe640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc22c0722c0 0x7fc22c0b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f51000ba410 con 0x7f50ec07c980 2024-09-17T02:41:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc22b7fe640 1 -- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc22c0717f0 msgr2=0x7fc22c16e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc22b7fe640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc22c0717f0 0x7fc22c16e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc22b7fe640 1 -- 172.21.15.131:0/936839720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc22c0713e0 msgr2=0x7fc22c14acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc22b7fe640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc22c0713e0 0x7fc22c14acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.490+0000 7fc22b7fe640 1 -- 172.21.15.131:0/936839720 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c16da70 con 0x7fc22c0722c0 2024-09-17T02:41:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.494+0000 7fc22b7fe640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc22c0722c0 0x7fc22c0b1f00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc22c06fab0 tx=0x7fc224009200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.494+0000 7f50fcff9640 1 -- 172.21.15.131:0/3152414036 <== osd.7 v2:172.21.15.189:6810/3586470400 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f51000ba410 con 0x7f50ec07c980 2024-09-17T02:41:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.494+0000 7fc213fff640 1 -- 172.21.15.131:0/936839720 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240bc070 con 0x7fc22c0722c0 2024-09-17T02:41:16.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.494+0000 7fc213fff640 1 -- 172.21.15.131:0/936839720 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc224004000 con 0x7fc22c0722c0 2024-09-17T02:41:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.494+0000 7fc213fff640 1 -- 172.21.15.131:0/936839720 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2240b2040 con 0x7fc22c0722c0 2024-09-17T02:41:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.494+0000 7fc231a2f640 1 -- 172.21.15.131:0/936839720 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c155f60 con 0x7fc22c0722c0 2024-09-17T02:41:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.494+0000 7fc231a2f640 1 -- 172.21.15.131:0/936839720 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c156240 con 0x7fc22c0722c0 2024-09-17T02:41:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.494+0000 7fc07f21b640 1 -- 172.21.15.131:0/1254066984 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc078153270 con 0x7fc06407caa0 2024-09-17T02:41:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.494+0000 7fc231a2f640 1 -- 172.21.15.131:0/936839720 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc22c0ada50 con 0x7fc22c0722c0 2024-09-17T02:41:16.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.494+0000 7fc0757fa640 1 -- 172.21.15.131:0/1254066984 <== osd.3 v2:172.21.15.131:6816/4047536388 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc078153270 con 0x7fc06407caa0 2024-09-17T02:41:16.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc213fff640 1 -- 172.21.15.131:0/936839720 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc224004450 con 0x7fc22c0722c0 2024-09-17T02:41:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc07f21b640 1 -- 172.21.15.131:0/1254066984 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x7fc06407caa0 msgr2=0x7fc06407ef30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc07f21b640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x7fc06407caa0 0x7fc06407ef30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc07f21b640 1 -- 172.21.15.131:0/1254066984 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc0640732d0 msgr2=0x7fc0640757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc07f21b640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc0640732d0 0x7fc0640757e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc078155da0 tx=0x7fc070009040 comp rx=0 tx=0).stop 2024-09-17T02:41:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc07f21b640 1 -- 172.21.15.131:0/1254066984 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc07806dc60 msgr2=0x7fc07814ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc07f21b640 1 --2- 172.21.15.131:0/1254066984 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc07806dc60 0x7fc07814ed70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc078070d60 tx=0x7fc06c0a6a20 comp rx=0 tx=0).stop 2024-09-17T02:41:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc07d7b8640 1 -- 172.21.15.131:0/1254066984 reap_dead start 2024-09-17T02:41:16.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc213fff640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc218073230 0x7fc218075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc07f21b640 1 -- 172.21.15.131:0/1254066984 shutdown_connections 2024-09-17T02:41:16.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc07f21b640 1 -- 172.21.15.131:0/1254066984 >> 172.21.15.131:0/1254066984 conn(0x7fc0780d03f0 msgr2=0x7fc07815c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc213fff640 1 -- 172.21.15.131:0/936839720 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc22412e110 con 0x7fc22c0722c0 2024-09-17T02:41:16.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc07f21b640 1 -- 172.21.15.131:0/1254066984 shutdown_connections 2024-09-17T02:41:16.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc07f21b640 1 -- 172.21.15.131:0/1254066984 wait complete. 2024-09-17T02:41:16.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc213fff640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x7fc21807c900 0x7fc21807ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc213fff640 1 -- 172.21.15.131:0/936839720 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2240c3e50 con 0x7fc21807c900 2024-09-17T02:41:16.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc213fff640 1 -- 172.21.15.131:0/936839720 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc224135050 con 0x7fc22c0722c0 2024-09-17T02:41:16.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f93b2ffd640 1 -- 172.21.15.131:0/4080345660 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f93a0002de0 con 0x7f93ac07c950 2024-09-17T02:41:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc22affd640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc218073230 0x7fc218075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc22a7fc640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x7fc21807c900 0x7fc21807ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x7f50ec07c980 msgr2=0x7f50ec07ee10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f51065e1640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x7f50ec07c980 0x7f50ec07ee10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f50ec073230 msgr2=0x7f50ec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc22affd640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc218073230 0x7fc218075740 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc220009c80 tx=0x7fc220009260 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f51065e1640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f50ec073230 0x7f50ec075740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f50f00af000 tx=0x7f50f001b030 comp rx=0 tx=0).stop 2024-09-17T02:41:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5100155050 msgr2=0x7f51001fa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f51065e1640 1 --2- 172.21.15.131:0/3152414036 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5100155050 0x7f51001fa820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f50f400c430 tx=0x7f50f400c8f0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f5104b7e640 1 -- 172.21.15.131:0/3152414036 reap_dead start 2024-09-17T02:41:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96b0ecb640 1 -- 172.21.15.131:0/2943263207 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9690002d60 con 0x7f968c07ca90 2024-09-17T02:41:16.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fc22a7fc640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x7fc21807c900 0x7fc21807ed90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 shutdown_connections 2024-09-17T02:41:16.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 >> 172.21.15.131:0/3152414036 conn(0x7f51000d1ba0 msgr2=0x7f51000b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f93c57fa640 1 -- 172.21.15.131:0/4080345660 <== osd.4 v2:172.21.15.189:6818/1965940492 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f93a0002de0 con 0x7f93ac07c950 2024-09-17T02:41:16.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 shutdown_connections 2024-09-17T02:41:16.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f51065e1640 1 -- 172.21.15.131:0/3152414036 wait complete. 2024-09-17T02:41:16.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96a2ffd640 1 -- 172.21.15.131:0/2943263207 <== osd.1 v2:172.21.15.131:6824/1616383728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9690002d60 con 0x7f968c07ca90 2024-09-17T02:41:16.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96b0ecb640 1 -- 172.21.15.131:0/2943263207 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x7f968c07ca90 msgr2=0x7f968c07ef20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96b0ecb640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x7f968c07ca90 0x7f968c07ef20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96b0ecb640 1 -- 172.21.15.131:0/2943263207 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f968c0732d0 msgr2=0x7f968c0757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96b0ecb640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f968c0732d0 0x7f968c0757e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f96ac200160 tx=0x7f9694006c00 comp rx=0 tx=0).stop 2024-09-17T02:41:16.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96b0ecb640 1 -- 172.21.15.131:0/2943263207 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ac0cd030 msgr2=0x7f96ac161860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x7f93ac07c950 msgr2=0x7f93ac07ede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fccbda5f640 1 -- 172.21.15.131:0/1541066977 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcc9c002de0 con 0x7fcc9807cb20 2024-09-17T02:41:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f93cd999640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x7f93ac07c950 0x7f93ac07ede0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96b0ecb640 1 --2- 172.21.15.131:0/2943263207 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ac0cd030 0x7f96ac161860 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f969c01b3e0 tx=0x7f969c0a5c10 comp rx=0 tx=0).stop 2024-09-17T02:41:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96aad9d640 1 -- 172.21.15.131:0/2943263207 reap_dead start 2024-09-17T02:41:16.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe498002d60 con 0x7fe4a407ca10 2024-09-17T02:41:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96b0ecb640 1 -- 172.21.15.131:0/2943263207 shutdown_connections 2024-09-17T02:41:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96b0ecb640 1 -- 172.21.15.131:0/2943263207 >> 172.21.15.131:0/2943263207 conn(0x7f96ac0cd8b0 msgr2=0x7f96ac0c9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.498+0000 7f96b0ecb640 1 -- 172.21.15.131:0/2943263207 shutdown_connections 2024-09-17T02:41:16.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7f96b0ecb640 1 -- 172.21.15.131:0/2943263207 wait complete. 2024-09-17T02:41:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93ac073200 msgr2=0x7f93ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fe49ffff640 1 -- 172.21.15.131:0/1024191227 <== osd.2 v2:172.21.15.131:6800/163801004 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe498002d60 con 0x7fe4a407ca10 2024-09-17T02:41:16.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fc213fff640 1 -- 172.21.15.131:0/936839720 <== osd.6 v2:172.21.15.189:6802/952142225 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2240c3e50 con 0x7fc21807c900 2024-09-17T02:41:16.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7f93cd999640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93ac073200 0x7f93ac075710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f93bc004710 tx=0x7f93bc00d040 comp rx=0 tx=0).stop 2024-09-17T02:41:16.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fccaffff640 1 -- 172.21.15.131:0/1541066977 <== osd.0 v2:172.21.15.131:6808/1154840320 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcc9c002de0 con 0x7fcc9807cb20 2024-09-17T02:41:16.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c81fd670 msgr2=0x7f93c80cb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7f93cd999640 1 --2- 172.21.15.131:0/4080345660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c81fd670 0x7f93c80cb460 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f93c80c6cf0 tx=0x7f93c000ed90 comp rx=0 tx=0).stop 2024-09-17T02:41:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x7fe4a407ca10 msgr2=0x7fe4a407eea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7f93cd198640 1 -- 172.21.15.131:0/4080345660 reap_dead start 2024-09-17T02:41:16.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fe4bdd43640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x7fe4a407ca10 0x7fe4a407eea0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe4a4073240 msgr2=0x7fe4a4075750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fe4bdd43640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe4a4073240 0x7fe4a4075750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0017b0 tx=0x7fe4ac009040 comp rx=0 tx=0).stop 2024-09-17T02:41:16.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b80b4e60 msgr2=0x7fe4b81553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fe4bdd43640 1 --2- 172.21.15.131:0/1024191227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe4b80b4e60 0x7fe4b81553d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8000c10 tx=0x7fe4a80a47c0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fe4b7fff640 1 -- 172.21.15.131:0/1024191227 reap_dead start 2024-09-17T02:41:16.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fccbda5f640 1 -- 172.21.15.131:0/1541066977 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x7fcc9807cb20 msgr2=0x7fcc9807efb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fccbda5f640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x7fcc9807cb20 0x7fcc9807efb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fccbda5f640 1 -- 172.21.15.131:0/1541066977 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc980733d0 msgr2=0x7fcc980758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fccbda5f640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc980733d0 0x7fcc980758e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcca000aa40 tx=0x7fcca000a2f0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fccbda5f640 1 -- 172.21.15.131:0/1541066977 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccb808b7d0 msgr2=0x7fccb8211910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fccbda5f640 1 --2- 172.21.15.131:0/1541066977 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccb808b7d0 0x7fccb8211910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fccb001cdf0 tx=0x7fccb0008ef0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fccb77fe640 1 -- 172.21.15.131:0/1541066977 reap_dead start 2024-09-17T02:41:16.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fccbda5f640 1 -- 172.21.15.131:0/1541066977 shutdown_connections 2024-09-17T02:41:16.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.502+0000 7fccbda5f640 1 -- 172.21.15.131:0/1541066977 >> 172.21.15.131:0/1541066977 conn(0x7fccb80d0ad0 msgr2=0x7fccb80b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fccbda5f640 1 -- 172.21.15.131:0/1541066977 shutdown_connections 2024-09-17T02:41:16.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 shutdown_connections 2024-09-17T02:41:16.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 >> 172.21.15.131:0/1024191227 conn(0x7fe4b80d03f0 msgr2=0x7fe4b80d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fccbda5f640 1 -- 172.21.15.131:0/1541066977 wait complete. 2024-09-17T02:41:16.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 shutdown_connections 2024-09-17T02:41:16.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 shutdown_connections 2024-09-17T02:41:16.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 >> 172.21.15.131:0/4080345660 conn(0x7f93c806ce90 msgr2=0x7f93c8157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7fe4bdd43640 1 -- 172.21.15.131:0/1024191227 wait complete. 2024-09-17T02:41:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 shutdown_connections 2024-09-17T02:41:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.506+0000 7f93cd999640 1 -- 172.21.15.131:0/4080345660 wait complete. 2024-09-17T02:41:16.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.514+0000 7fd8e1f90640 1 -- 172.21.15.131:0/1685324202 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd8dc06feb0 con 0x7fd8c007caa0 2024-09-17T02:41:16.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.514+0000 7fd8c7fff640 1 -- 172.21.15.131:0/1685324202 <== osd.5 v2:172.21.15.189:6826/3175183672 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd8dc06feb0 con 0x7fd8c007caa0 2024-09-17T02:41:16.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.522+0000 7fd8c5ffb640 1 -- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x7fd8c007caa0 msgr2=0x7fd8c007ef30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.522+0000 7fd8c5ffb640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x7fd8c007caa0 0x7fd8c007ef30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.522+0000 7fd8c5ffb640 1 -- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8c00732d0 msgr2=0x7fd8c00757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.522+0000 7fd8c5ffb640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8c00732d0 0x7fd8c00757e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009c00 tx=0x7fd8d00091e0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.522+0000 7fd8c5ffb640 1 -- 172.21.15.131:0/1685324202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8dc0cf140 msgr2=0x7fd8dc0729b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.522+0000 7fd8c5ffb640 1 --2- 172.21.15.131:0/1685324202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8dc0cf140 0x7fd8dc0729b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd8c80ac000 tx=0x7fd8c80a5730 comp rx=0 tx=0).stop 2024-09-17T02:41:16.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.522+0000 7fd8dbfff640 1 -- 172.21.15.131:0/1685324202 reap_dead start 2024-09-17T02:41:16.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.522+0000 7fd8c5ffb640 1 -- 172.21.15.131:0/1685324202 shutdown_connections 2024-09-17T02:41:16.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.522+0000 7fd8c5ffb640 1 -- 172.21.15.131:0/1685324202 >> 172.21.15.131:0/1685324202 conn(0x7fd8dc0cfdc0 msgr2=0x7fd8dc0d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.522+0000 7fd8c5ffb640 1 -- 172.21.15.131:0/1685324202 shutdown_connections 2024-09-17T02:41:16.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.522+0000 7fd8c5ffb640 1 -- 172.21.15.131:0/1685324202 wait complete. 2024-09-17T02:41:16.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.526+0000 7fc231a2f640 1 -- 172.21.15.131:0/936839720 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc22c162330 con 0x7fc21807c900 2024-09-17T02:41:16.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc213fff640 1 -- 172.21.15.131:0/936839720 <== osd.6 v2:172.21.15.189:6802/952142225 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc22c162330 con 0x7fc21807c900 2024-09-17T02:41:16.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc211ffb640 1 -- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x7fc21807c900 msgr2=0x7fc21807ed90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc211ffb640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x7fc21807c900 0x7fc21807ed90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc211ffb640 1 -- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc218073230 msgr2=0x7fc218075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc211ffb640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc218073230 0x7fc218075740 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc220009c80 tx=0x7fc220009260 comp rx=0 tx=0).stop 2024-09-17T02:41:16.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc211ffb640 1 -- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc22c0722c0 msgr2=0x7fc22c0b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc211ffb640 1 --2- 172.21.15.131:0/936839720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc22c0722c0 0x7fc22c0b1f00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc22c06fab0 tx=0x7fc224009200 comp rx=0 tx=0).stop 2024-09-17T02:41:16.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc22b7fe640 1 -- 172.21.15.131:0/936839720 reap_dead start 2024-09-17T02:41:16.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc211ffb640 1 -- 172.21.15.131:0/936839720 shutdown_connections 2024-09-17T02:41:16.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc211ffb640 1 -- 172.21.15.131:0/936839720 >> 172.21.15.131:0/936839720 conn(0x7fc22c0d1130 msgr2=0x7fc22c0b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc211ffb640 1 -- 172.21.15.131:0/936839720 shutdown_connections 2024-09-17T02:41:16.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.530+0000 7fc211ffb640 1 -- 172.21.15.131:0/936839720 wait complete. 2024-09-17T02:41:16.535 INFO:teuthology.orchestra.run.smithi131.stdout:64424509451 2024-09-17T02:41:16.535 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T02:41:16.538 INFO:teuthology.orchestra.run.smithi131.stdout:64424509450 2024-09-17T02:41:16.539 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T02:41:16.541 INFO:teuthology.orchestra.run.smithi131.stdout:64424509451 2024-09-17T02:41:16.541 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T02:41:16.543 INFO:teuthology.orchestra.run.smithi131.stdout:64424509452 2024-09-17T02:41:16.544 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T02:41:16.545 INFO:teuthology.orchestra.run.smithi131.stdout:64424509451 2024-09-17T02:41:16.545 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T02:41:16.550 INFO:teuthology.orchestra.run.smithi131.stdout:64424509452 2024-09-17T02:41:16.550 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T02:41:16.560 INFO:teuthology.orchestra.run.smithi131.stdout:64424509451 2024-09-17T02:41:16.560 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T02:41:16.563 INFO:teuthology.orchestra.run.smithi131.stdout:55834574859 2024-09-17T02:41:16.564 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T02:41:16.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.742+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/108189638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e080b4df0 msgr2=0x7f3e08073790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.742+0000 7f3e0d4ff640 1 --2- 172.21.15.131:0/108189638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e080b4df0 0x7f3e08073790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc009900 tx=0x7f3dfc01c8f0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.746+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/108189638 shutdown_connections 2024-09-17T02:41:16.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.746+0000 7f3e0d4ff640 1 --2- 172.21.15.131:0/108189638 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e080707f0 0x7f3e08159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.746+0000 7f3e0d4ff640 1 --2- 172.21.15.131:0/108189638 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e080b4df0 0x7f3e08073790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.746+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/108189638 >> 172.21.15.131:0/108189638 conn(0x7f3e080d0ad0 msgr2=0x7f3e0806ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.746+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/108189638 shutdown_connections 2024-09-17T02:41:16.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.746+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/108189638 wait complete. 2024-09-17T02:41:16.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.746+0000 7f3e0d4ff640 1 Processor -- start 2024-09-17T02:41:16.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.746+0000 7f3e0d4ff640 1 -- start start 2024-09-17T02:41:16.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e0d4ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e080707f0 0x7f3e080abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e0d4ff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e08155840 0x7f3e081552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e0d4ff640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e080aaec0 0x7f3e0815e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e06ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e080707f0 0x7f3e080abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e0d4ff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0815d1f0 con 0x7f3e080707f0 2024-09-17T02:41:16.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e0d4ff640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0815d070 con 0x7f3e08155840 2024-09-17T02:41:16.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e0d4ff640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3e0815d370 con 0x7f3e080aaec0 2024-09-17T02:41:16.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e06ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e080707f0 0x7f3e080abe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57834/0 (socket says 172.21.15.131:57834) 2024-09-17T02:41:16.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e067fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e08155840 0x7f3e081552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e067fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e08155840 0x7f3e081552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.131:55868/0 (socket says 172.21.15.131:55868) 2024-09-17T02:41:16.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e067fc640 1 -- 172.21.15.131:0/3223646554 learned_addr learned my addr 172.21.15.131:0/3223646554 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e077fe640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e080aaec0 0x7f3e0815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e077fe640 1 -- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e08155840 msgr2=0x7f3e081552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e077fe640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e08155840 0x7f3e081552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e077fe640 1 -- 172.21.15.131:0/3223646554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e080707f0 msgr2=0x7f3e080abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e077fe640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e080707f0 0x7f3e080abe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e077fe640 1 -- 172.21.15.131:0/3223646554 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e080ac6a0 con 0x7f3e080aaec0 2024-09-17T02:41:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e06ffd640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e080707f0 0x7f3e080abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T02:41:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.750+0000 7f3e077fe640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e080aaec0 0x7f3e0815e450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3df800e5d0 tx=0x7f3df800ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.754+0000 7f3def7fe640 1 -- 172.21.15.131:0/3223646554 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8004220 con 0x7f3e080aaec0 2024-09-17T02:41:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.754+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/3223646554 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e080ac980 con 0x7f3e080aaec0 2024-09-17T02:41:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.754+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/3223646554 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e080aa9a0 con 0x7f3e080aaec0 2024-09-17T02:41:16.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.754+0000 7f3def7fe640 1 -- 172.21.15.131:0/3223646554 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3df8005020 con 0x7f3e080aaec0 2024-09-17T02:41:16.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.754+0000 7f3def7fe640 1 -- 172.21.15.131:0/3223646554 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df80044b0 con 0x7f3e080aaec0 2024-09-17T02:41:16.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.754+0000 7f3def7fe640 1 -- 172.21.15.131:0/3223646554 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3df8005450 con 0x7f3e080aaec0 2024-09-17T02:41:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.754+0000 7f3def7fe640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3df4073200 0x7f3df4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.754+0000 7f3def7fe640 1 -- 172.21.15.131:0/3223646554 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3df80948e0 con 0x7f3e080aaec0 2024-09-17T02:41:16.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.754+0000 7f3ded7fa640 1 -- 172.21.15.131:0/3223646554 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de00052a0 con 0x7f3e080aaec0 2024-09-17T02:41:16.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.758+0000 7f3e06ffd640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3df4073200 0x7f3df4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.758+0000 7f3e06ffd640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3df4073200 0x7f3df4075710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0af000 tx=0x7f3dfc01b130 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.774+0000 7f246aa1f640 1 -- 172.21.15.131:0/2946228723 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f245c0e1a80 msgr2=0x7f245c035270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.774+0000 7f246aa1f640 1 --2- 172.21.15.131:0/2946228723 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f245c0e1a80 0x7f245c035270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2460009840 tx=0x7f246001c790 comp rx=0 tx=0).stop 2024-09-17T02:41:16.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.774+0000 7f3def7fe640 1 -- 172.21.15.131:0/3223646554 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3df805ef60 con 0x7f3e080aaec0 2024-09-17T02:41:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f71ff59e640 1 -- 172.21.15.131:0/3864080049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8007240 msgr2=0x7f71f80df590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f71ff59e640 1 --2- 172.21.15.131:0/3864080049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8007240 0x7f71f80df590 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f71f800a340 tx=0x7f71f00203e0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f71ff59e640 1 -- 172.21.15.131:0/3864080049 shutdown_connections 2024-09-17T02:41:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f71ff59e640 1 --2- 172.21.15.131:0/3864080049 >> v2:172.21.15.189:3300/0 conn(0x7f71f8053a70 0x7f71f8053e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f71ff59e640 1 --2- 172.21.15.131:0/3864080049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f8007240 0x7f71f80df590 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f71ff59e640 1 -- 172.21.15.131:0/3864080049 >> 172.21.15.131:0/3864080049 conn(0x7f71f801e270 msgr2=0x7f71f8039ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f246aa1f640 1 -- 172.21.15.131:0/2946228723 shutdown_connections 2024-09-17T02:41:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f246aa1f640 1 --2- 172.21.15.131:0/2946228723 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f245c0e1a80 0x7f245c035270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f246aa1f640 1 --2- 172.21.15.131:0/2946228723 >> v2:172.21.15.189:3300/0 conn(0x7f245c0e1670 0x7f245c034d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f246aa1f640 1 -- 172.21.15.131:0/2946228723 >> 172.21.15.131:0/2946228723 conn(0x7f245c01eaf0 msgr2=0x7f245c03a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f246aa1f640 1 -- 172.21.15.131:0/2946228723 shutdown_connections 2024-09-17T02:41:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f71ff59e640 1 -- 172.21.15.131:0/3864080049 shutdown_connections 2024-09-17T02:41:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f71ff59e640 1 -- 172.21.15.131:0/3864080049 wait complete. 2024-09-17T02:41:16.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.778+0000 7f246aa1f640 1 -- 172.21.15.131:0/2946228723 wait complete. 2024-09-17T02:41:16.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71ff59e640 1 Processor -- start 2024-09-17T02:41:16.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71ff59e640 1 -- start start 2024-09-17T02:41:16.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f246aa1f640 1 Processor -- start 2024-09-17T02:41:16.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f246aa1f640 1 -- start start 2024-09-17T02:41:16.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71ff59e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f71f8007240 0x7f71f80d8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71ff59e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f80346b0 0x7f71f80e0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71ff59e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8053a70 0x7f71f8048a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71fdd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f80346b0 0x7f71f80e0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71fdd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f80346b0 0x7f71f80e0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57876/0 (socket says 172.21.15.131:57876) 2024-09-17T02:41:16.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71fdd9b640 1 -- 172.21.15.131:0/2314912110 learned_addr learned my addr 172.21.15.131:0/2314912110 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71ff59e640 1 -- 172.21.15.131:0/2314912110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8006330 con 0x7f71f80346b0 2024-09-17T02:41:16.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71ff59e640 1 -- 172.21.15.131:0/2314912110 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71f80061b0 con 0x7f71f8053a70 2024-09-17T02:41:16.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71fe59c640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f71f8007240 0x7f71f80d8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71ff59e640 1 -- 172.21.15.131:0/2314912110 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f71f80064b0 con 0x7f71f8007240 2024-09-17T02:41:16.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f246aa1f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c0e0890 0x7f245c0d8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f246aa1f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f245c0e1670 0x7f245c0de6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f246aa1f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f245c0e1a80 0x7f245c180e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f246aa1f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f245c0ec7c0 con 0x7f245c0e0890 2024-09-17T02:41:16.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f246aa1f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f245c0ec640 con 0x7f245c0e1a80 2024-09-17T02:41:16.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f246aa1f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f245c0ec940 con 0x7f245c0e1670 2024-09-17T02:41:16.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f246a21e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f245c0e1a80 0x7f245c180e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f2469a1d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c0e0890 0x7f245c0d8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f2469a1d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c0e0890 0x7f245c0d8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57878/0 (socket says 172.21.15.131:57878) 2024-09-17T02:41:16.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f2469a1d640 1 -- 172.21.15.131:0/2557691173 learned_addr learned my addr 172.21.15.131:0/2557691173 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.782+0000 7f71fed9d640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8053a70 0x7f71f8048a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71fdd9b640 1 -- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f71f8007240 msgr2=0x7f71f80d8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71fdd9b640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f71f8007240 0x7f71f80d8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71fdd9b640 1 -- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8053a70 msgr2=0x7f71f8048a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71fdd9b640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8053a70 0x7f71f8048a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71fdd9b640 1 -- 172.21.15.131:0/2314912110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f80e2ab0 con 0x7f71f80346b0 2024-09-17T02:41:16.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71fdd9b640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f80346b0 0x7f71f80e0540 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f71f400c3c0 tx=0x7f71f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f2469a1d640 1 -- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f245c0e1670 msgr2=0x7f245c0de6d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T02:41:16.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f2469a1d640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f245c0e1670 0x7f245c0de6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f2469a1d640 1 -- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f245c0e1a80 msgr2=0x7f245c180e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f2469a1d640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f245c0e1a80 0x7f245c180e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f2469a1d640 1 -- 172.21.15.131:0/2557691173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c0d8ec0 con 0x7f245c0e0890 2024-09-17T02:41:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f2469a1d640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c0e0890 0x7f245c0d8980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f245400ea20 tx=0x7f245400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f245a7fc640 1 -- 172.21.15.131:0/2557691173 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245400cd80 con 0x7f245c0e0890 2024-09-17T02:41:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f245a7fc640 1 -- 172.21.15.131:0/2557691173 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2454004020 con 0x7f245c0e0890 2024-09-17T02:41:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f245a7fc640 1 -- 172.21.15.131:0/2557691173 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454004450 con 0x7f245c0e0890 2024-09-17T02:41:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71eeffd640 1 -- 172.21.15.131:0/2314912110 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4017070 con 0x7f71f80346b0 2024-09-17T02:41:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71ff59e640 1 -- 172.21.15.131:0/2314912110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f80dfd80 con 0x7f71f80346b0 2024-09-17T02:41:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71ff59e640 1 -- 172.21.15.131:0/2314912110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f80e27d0 con 0x7f71f80346b0 2024-09-17T02:41:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71eeffd640 1 -- 172.21.15.131:0/2314912110 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71f40052d0 con 0x7f71f80346b0 2024-09-17T02:41:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f246aa1f640 1 -- 172.21.15.131:0/2557691173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c0df2b0 con 0x7f245c0e0890 2024-09-17T02:41:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f246aa1f640 1 -- 172.21.15.131:0/2557691173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c0df720 con 0x7f245c0e0890 2024-09-17T02:41:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71ff59e640 1 -- 172.21.15.131:0/2314912110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f8009f80 con 0x7f71f80346b0 2024-09-17T02:41:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.786+0000 7f71eeffd640 1 -- 172.21.15.131:0/2314912110 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4003860 con 0x7f71f80346b0 2024-09-17T02:41:16.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7fa3d2437640 1 -- 172.21.15.131:0/3536578087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3bc005030 msgr2=0x7fa3bc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7fa3d2437640 1 --2- 172.21.15.131:0/3536578087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3bc005030 0x7fa3bc00b9d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc06bc20 tx=0x7fa3cc098b70 comp rx=0 tx=0).stop 2024-09-17T02:41:16.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7fa3d09d4640 1 -- 172.21.15.131:0/3536578087 reap_dead start 2024-09-17T02:41:16.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7fa3d2437640 1 -- 172.21.15.131:0/3536578087 shutdown_connections 2024-09-17T02:41:16.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7fa3d2437640 1 -- 172.21.15.131:0/3536578087 >> 172.21.15.131:0/3536578087 conn(0x7fa3c401eaf0 msgr2=0x7fa3c403a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f245a7fc640 1 -- 172.21.15.131:0/2557691173 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2454003600 con 0x7f245c0e0890 2024-09-17T02:41:16.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f246aa1f640 1 -- 172.21.15.131:0/2557691173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245c0e0ca0 con 0x7f245c0e0890 2024-09-17T02:41:16.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f245a7fc640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24480732d0 0x7f24480757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f245a7fc640 1 -- 172.21.15.131:0/2557691173 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f2454094f20 con 0x7f245c0e0890 2024-09-17T02:41:16.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f71eeffd640 1 -- 172.21.15.131:0/2314912110 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f71f401a070 con 0x7f71f80346b0 2024-09-17T02:41:16.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f71eeffd640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71e0073200 0x7f71e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f71eeffd640 1 -- 172.21.15.131:0/2314912110 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f71f4093770 con 0x7f71f80346b0 2024-09-17T02:41:16.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f33770dd640 1 -- 172.21.15.131:0/1979572664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33700cc3b0 msgr2=0x7f33700cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f33770dd640 1 --2- 172.21.15.131:0/1979572664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33700cc3b0 0x7f33700cc7c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f336000a950 tx=0x7f336001db90 comp rx=0 tx=0).stop 2024-09-17T02:41:16.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f246921c640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24480732d0 0x7f24480757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f33770dd640 1 -- 172.21.15.131:0/1979572664 shutdown_connections 2024-09-17T02:41:16.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f33770dd640 1 --2- 172.21.15.131:0/1979572664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f33700cc3b0 0x7f33700cc7c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f33770dd640 1 --2- 172.21.15.131:0/1979572664 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f337015f010 0x7f337006e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f33770dd640 1 -- 172.21.15.131:0/1979572664 >> 172.21.15.131:0/1979572664 conn(0x7f33700cd6a0 msgr2=0x7f33700cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f33770dd640 1 -- 172.21.15.131:0/1979572664 shutdown_connections 2024-09-17T02:41:16.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f71fe59c640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71e0073200 0x7f71e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.790+0000 7f33770dd640 1 -- 172.21.15.131:0/1979572664 wait complete. 2024-09-17T02:41:16.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7fa3d2437640 1 -- 172.21.15.131:0/3536578087 shutdown_connections 2024-09-17T02:41:16.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f71fe59c640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71e0073200 0x7f71e0075710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f71f00b2000 tx=0x7f71f001d180 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f246921c640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24480732d0 0x7f24480757e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24600af000 tx=0x7f246001afd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f33770dd640 1 Processor -- start 2024-09-17T02:41:16.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f33770dd640 1 -- start start 2024-09-17T02:41:16.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f33770dd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33700cc3b0 0x7f33701561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7fa3d2437640 1 -- 172.21.15.131:0/3536578087 wait complete. 2024-09-17T02:41:16.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7fa3d2437640 1 Processor -- start 2024-09-17T02:41:16.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f33770dd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f337015a470 0x7f33700b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7fa3d2437640 1 -- start start 2024-09-17T02:41:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f3374e79640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33700cc3b0 0x7f33701561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f3374e79640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33700cc3b0 0x7f33701561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57914/0 (socket says 172.21.15.131:57914) 2024-09-17T02:41:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f33770dd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f337015f010 0x7f33700b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f3374e79640 1 -- 172.21.15.131:0/2793796765 learned_addr learned my addr 172.21.15.131:0/2793796765 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f33770dd640 1 -- 172.21.15.131:0/2793796765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3370070ad0 con 0x7f33700cc3b0 2024-09-17T02:41:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f33770dd640 1 -- 172.21.15.131:0/2793796765 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3370070950 con 0x7f337015f010 2024-09-17T02:41:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f33770dd640 1 -- 172.21.15.131:0/2793796765 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3370070c50 con 0x7f337015a470 2024-09-17T02:41:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f336ffff640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f337015a470 0x7f33700b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.794+0000 7f337567a640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f337015f010 0x7f33700b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3374e79640 1 -- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f337015a470 msgr2=0x7f33700b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3374e79640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f337015a470 0x7f33700b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3374e79640 1 -- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f337015f010 msgr2=0x7f33700b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3374e79640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f337015f010 0x7f33700b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3374e79640 1 -- 172.21.15.131:0/2793796765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33700cb2c0 con 0x7f33700cc3b0 2024-09-17T02:41:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3374e79640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33700cc3b0 0x7f33701561c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f336400e800 tx=0x7f336400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3ee3584640 1 -- 172.21.15.131:0/2696547196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3edc070c80 msgr2=0x7f3edc15bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3ee3584640 1 --2- 172.21.15.131:0/2696547196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3edc070c80 0x7f3edc15bc50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3edc0729e0 tx=0x7f3ed801c9b0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3d2437640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3bc005030 0x7fa3c4180b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3d2437640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa3c4055e10 0x7fa3c4053420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3cbfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3bc005030 0x7fa3c4180b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3cbfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3bc005030 0x7fa3c4180b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.131:55978/0 (socket says 172.21.15.131:55978) 2024-09-17T02:41:16.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3d2437640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c4053960 0x7fa3c418d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fdf667fc640 1 -- 172.21.15.131:0/99969275 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0036e0 con 0x7fdf50005920 2024-09-17T02:41:16.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fdf6de33640 1 -- 172.21.15.131:0/99969275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf50005920 msgr2=0x7fdf50005d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fdf6de33640 1 --2- 172.21.15.131:0/99969275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf50005920 0x7fdf50005d30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c006f50 tx=0x7fdf5c01ce00 comp rx=0 tx=0).stop 2024-09-17T02:41:16.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3cb7fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa3c4055e10 0x7fa3c4053420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3cb7fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa3c4055e10 0x7fa3c4053420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.131:36666/0 (socket says 172.21.15.131:36666) 2024-09-17T02:41:16.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3cbfff640 1 -- 172.21.15.131:0/1205703452 learned_addr learned my addr 172.21.15.131:0/1205703452 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4035440 con 0x7fa3c4053960 2024-09-17T02:41:16.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c40e12b0 con 0x7fa3bc005030 2024-09-17T02:41:16.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c40355c0 con 0x7fa3c4055e10 2024-09-17T02:41:16.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3d09d4640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c4053960 0x7fa3c418d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f71eeffd640 1 -- 172.21.15.131:0/2314912110 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f71f8007650 con 0x7f71f80346b0 2024-09-17T02:41:16.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3cbfff640 1 -- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa3c4055e10 msgr2=0x7fa3c4053420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3cbfff640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa3c4055e10 0x7fa3c4053420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3cbfff640 1 -- 172.21.15.131:0/1205703452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c4053960 msgr2=0x7fa3c418d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3cbfff640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c4053960 0x7fa3c418d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3cbfff640 1 -- 172.21.15.131:0/1205703452 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c4050a30 con 0x7fa3bc005030 2024-09-17T02:41:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3d09d4640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c4053960 0x7fa3c418d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T02:41:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3cbfff640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3bc005030 0x7fa3c4180b90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8003ed0 tx=0x7fa3b8001640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3c8ff9640 1 -- 172.21.15.131:0/1205703452 <== mon.1 v2:172.21.15.189:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8015070 con 0x7fa3bc005030 2024-09-17T02:41:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fdf6de33640 1 -- 172.21.15.131:0/99969275 shutdown_connections 2024-09-17T02:41:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fdf6de33640 1 --2- 172.21.15.131:0/99969275 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf50005920 0x7fdf50005d30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fdf6de33640 1 -- 172.21.15.131:0/99969275 >> 172.21.15.131:0/99969275 conn(0x7fdf680cd6a0 msgr2=0x7fdf680cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c4050d10 con 0x7fa3bc005030 2024-09-17T02:41:16.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fa3c8ff9640 1 -- 172.21.15.131:0/1205703452 <== mon.1 v2:172.21.15.189:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3b8011840 con 0x7fa3bc005030 2024-09-17T02:41:16.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fdf6de33640 1 -- 172.21.15.131:0/99969275 shutdown_connections 2024-09-17T02:41:16.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3ee3584640 1 -- 172.21.15.131:0/2696547196 shutdown_connections 2024-09-17T02:41:16.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3ee3584640 1 --2- 172.21.15.131:0/2696547196 >> v2:172.21.15.189:3300/0 conn(0x7f3edc151a90 0x7f3edc073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3ee3584640 1 --2- 172.21.15.131:0/2696547196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3edc070c80 0x7f3edc15bc50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7f3ee3584640 1 -- 172.21.15.131:0/2696547196 >> 172.21.15.131:0/2696547196 conn(0x7f3edc0d13d0 msgr2=0x7f3edc0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.802+0000 7fa3c8ff9640 1 -- 172.21.15.131:0/1205703452 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8011c70 con 0x7fa3bc005030 2024-09-17T02:41:16.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.802+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c4054230 con 0x7fa3bc005030 2024-09-17T02:41:16.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.798+0000 7fdf6de33640 1 -- 172.21.15.131:0/99969275 wait complete. 2024-09-17T02:41:16.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.802+0000 7fdf6de33640 1 Processor -- start 2024-09-17T02:41:16.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.802+0000 7fdf6de33640 1 -- start start 2024-09-17T02:41:16.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.802+0000 7fa3c8ff9640 1 -- 172.21.15.131:0/1205703452 <== mon.1 v2:172.21.15.189:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa3b8018070 con 0x7fa3bc005030 2024-09-17T02:41:16.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.802+0000 7f3ee3584640 1 -- 172.21.15.131:0/2696547196 shutdown_connections 2024-09-17T02:41:16.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.802+0000 7f3ee3584640 1 -- 172.21.15.131:0/2696547196 wait complete. 2024-09-17T02:41:16.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.802+0000 7fa3c8ff9640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa3b0073200 0x7fa3b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.802+0000 7fa3c8ff9640 1 -- 172.21.15.131:0/1205703452 <== mon.1 v2:172.21.15.189:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa3b8094f90 con 0x7fa3bc005030 2024-09-17T02:41:16.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c4034d30 con 0x7fa3bc005030 2024-09-17T02:41:16.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7fa3cb7fe640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa3b0073200 0x7fa3b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f336d7fa640 1 -- 172.21.15.131:0/2793796765 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336400ccc0 con 0x7f33700cc3b0 2024-09-17T02:41:16.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7fa3cb7fe640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa3b0073200 0x7fa3b0075710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40f2710 tx=0x7fa3c00091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee3584640 1 Processor -- start 2024-09-17T02:41:16.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee3584640 1 -- start start 2024-09-17T02:41:16.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f33770dd640 1 -- 172.21.15.131:0/2793796765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33700cb5a0 con 0x7f33700cc3b0 2024-09-17T02:41:16.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f245a7fc640 1 -- 172.21.15.131:0/2557691173 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f245405fc70 con 0x7f245c0e0890 2024-09-17T02:41:16.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f336d7fa640 1 -- 172.21.15.131:0/2793796765 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f336400ce60 con 0x7f33700cc3b0 2024-09-17T02:41:16.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f336d7fa640 1 -- 172.21.15.131:0/2793796765 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364010650 con 0x7f33700cc3b0 2024-09-17T02:41:16.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee3584640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3edc070c80 0x7f3edc1382d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee3584640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3edc074140 0x7f3edc1498e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee3584640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3edc151a90 0x7f3edc149e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee3584640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc069db0 con 0x7f3edc070c80 2024-09-17T02:41:16.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee3584640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3edc069c30 con 0x7f3edc074140 2024-09-17T02:41:16.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee3584640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3edc06f270 con 0x7f3edc151a90 2024-09-17T02:41:16.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f33770dd640 1 -- 172.21.15.131:0/2793796765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33700caf90 con 0x7f33700cc3b0 2024-09-17T02:41:16.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee1b21640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3edc151a90 0x7f3edc149e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee1b21640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3edc151a90 0x7f3edc149e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.131:36672/0 (socket says 172.21.15.131:36672) 2024-09-17T02:41:16.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee1b21640 1 -- 172.21.15.131:0/3250460420 learned_addr learned my addr 172.21.15.131:0/3250460420 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3ee0b1f640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3edc074140 0x7f3edc1498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.806+0000 7f3356ffd640 1 -- 172.21.15.131:0/2793796765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33700cc7c0 con 0x7f33700cc3b0 2024-09-17T02:41:16.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ee1b21640 1 -- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3edc074140 msgr2=0x7f3edc1498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ee1b21640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3edc074140 0x7f3edc1498e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ee1b21640 1 -- 172.21.15.131:0/3250460420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3edc070c80 msgr2=0x7f3edc1382d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ee1b21640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3edc070c80 0x7f3edc1382d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ee1b21640 1 -- 172.21.15.131:0/3250460420 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3edc15e000 con 0x7f3edc151a90 2024-09-17T02:41:16.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ee0b1f640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3edc074140 0x7f3edc1498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T02:41:16.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ee1b21640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3edc151a90 0x7f3edc149e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c6b0 tx=0x7f3ed400cb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf6de33640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf50005920 0x7fdf681e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf6de33640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf680b5110 0x7fdf681f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf6de33640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf681f45e0 0x7fdf681f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf6de33640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6806e630 con 0x7fdf680b5110 2024-09-17T02:41:16.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf677fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf50005920 0x7fdf681e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf677fe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf50005920 0x7fdf681e6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.131:56002/0 (socket says 172.21.15.131:56002) 2024-09-17T02:41:16.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf677fe640 1 -- 172.21.15.131:0/3104254150 learned_addr learned my addr 172.21.15.131:0/3104254150 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6806e4b0 con 0x7fdf50005920 2024-09-17T02:41:16.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f336d7fa640 1 -- 172.21.15.131:0/2793796765 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f33640107f0 con 0x7f33700cc3b0 2024-09-17T02:41:16.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f336d7fa640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f335c073230 0x7f335c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f336d7fa640 1 -- 172.21.15.131:0/2793796765 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3364094d90 con 0x7f33700cc3b0 2024-09-17T02:41:16.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f336ffff640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f335c073230 0x7f335c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf67fff640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf681f45e0 0x7fdf681f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6806e7b0 con 0x7fdf681f45e0 2024-09-17T02:41:16.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ed1ffb640 1 -- 172.21.15.131:0/3250460420 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4017070 con 0x7f3edc151a90 2024-09-17T02:41:16.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ed1ffb640 1 -- 172.21.15.131:0/3250460420 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ed4003c70 con 0x7f3edc151a90 2024-09-17T02:41:16.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ed1ffb640 1 -- 172.21.15.131:0/3250460420 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4003e60 con 0x7f3edc151a90 2024-09-17T02:41:16.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf67fff640 1 -- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf50005920 msgr2=0x7fdf681e6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf67fff640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf50005920 0x7fdf681e6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf67fff640 1 -- 172.21.15.131:0/3104254150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf680b5110 msgr2=0x7fdf681f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf67fff640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf680b5110 0x7fdf681f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf67fff640 1 -- 172.21.15.131:0/3104254150 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf681f4f30 con 0x7fdf681f45e0 2024-09-17T02:41:16.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf677fe640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf50005920 0x7fdf681e6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T02:41:16.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf67fff640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf681f45e0 0x7fdf681f49f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf68072440 tx=0x7fdf58009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ee3584640 1 -- 172.21.15.131:0/3250460420 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3edc155e00 con 0x7f3edc151a90 2024-09-17T02:41:16.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ee3584640 1 -- 172.21.15.131:0/3250460420 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3edc0ce240 con 0x7f3edc151a90 2024-09-17T02:41:16.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f3ee3584640 1 -- 172.21.15.131:0/3250460420 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3edc155b70 con 0x7f3edc151a90 2024-09-17T02:41:16.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf57fff640 1 -- 172.21.15.131:0/3104254150 <== mon.2 v2:172.21.15.189:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf580098e0 con 0x7fdf681f45e0 2024-09-17T02:41:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf57fff640 1 -- 172.21.15.131:0/3104254150 <== mon.2 v2:172.21.15.189:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf5800fda0 con 0x7fdf681f45e0 2024-09-17T02:41:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf680b58f0 con 0x7fdf681f45e0 2024-09-17T02:41:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf57fff640 1 -- 172.21.15.131:0/3104254150 <== mon.2 v2:172.21.15.189:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58004290 con 0x7fdf681f45e0 2024-09-17T02:41:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf680b5eb0 con 0x7fdf681f45e0 2024-09-17T02:41:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.810+0000 7f336ffff640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f335c073230 0x7f335c075740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f33600b0000 tx=0x7f336001c3d0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.814+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf680b5660 con 0x7fdf681f45e0 2024-09-17T02:41:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.814+0000 7f3ed1ffb640 1 -- 172.21.15.131:0/3250460420 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ed401a070 con 0x7f3edc151a90 2024-09-17T02:41:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.814+0000 7f3ed1ffb640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ec0073250 0x7f3ec0075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.814+0000 7f3ed1ffb640 1 -- 172.21.15.131:0/3250460420 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f3ed4093dc0 con 0x7f3edc151a90 2024-09-17T02:41:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.814+0000 7fdf57fff640 1 -- 172.21.15.131:0/3104254150 <== mon.2 v2:172.21.15.189:3301/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdf58020060 con 0x7fdf681f45e0 2024-09-17T02:41:16.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.814+0000 7f3ee1320640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ec0073250 0x7f3ec0075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.814+0000 7fdf57fff640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf4c073230 0x7fdf4c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.818+0000 7fdf677fe640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf4c073230 0x7fdf4c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.818+0000 7f336d7fa640 1 -- 172.21.15.131:0/2793796765 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f336409b030 con 0x7f33700cc3b0 2024-09-17T02:41:16.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.818+0000 7f3ee1320640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ec0073250 0x7f3ec0075760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80af000 tx=0x7f3ed8000bd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.818+0000 7fa3c8ff9640 1 -- 172.21.15.131:0/1205703452 <== mon.1 v2:172.21.15.189:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa3b805f740 con 0x7fa3bc005030 2024-09-17T02:41:16.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.822+0000 7fdf677fe640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf4c073230 0x7fdf4c075740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf600066a0 tx=0x7fdf60006ee0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.826+0000 7fdf57fff640 1 -- 172.21.15.131:0/3104254150 <== mon.2 v2:172.21.15.189:3301/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdf5801ae00 con 0x7fdf681f45e0 2024-09-17T02:41:16.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.826+0000 7f3ed1ffb640 1 -- 172.21.15.131:0/3250460420 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ed4021020 con 0x7f3edc151a90 2024-09-17T02:41:16.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.842+0000 7fdf57fff640 1 -- 172.21.15.131:0/3104254150 <== mon.2 v2:172.21.15.189:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdf681f4f30 con 0x7fdf681f45e0 2024-09-17T02:41:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.842+0000 7f91bd772640 1 -- 172.21.15.131:0/304892326 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91b80b48b0 msgr2=0x7f91b80cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.842+0000 7f91bd772640 1 --2- 172.21.15.131:0/304892326 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91b80b48b0 0x7f91b80cca70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f91ac009870 tx=0x7f91ac01c860 comp rx=0 tx=0).stop 2024-09-17T02:41:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.842+0000 7f91bd772640 1 -- 172.21.15.131:0/304892326 shutdown_connections 2024-09-17T02:41:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.842+0000 7f91bd772640 1 --2- 172.21.15.131:0/304892326 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91b80b48b0 0x7f91b80cca70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.842+0000 7f91bd772640 1 -- 172.21.15.131:0/304892326 >> 172.21.15.131:0/304892326 conn(0x7f91b80cf150 msgr2=0x7f91b80cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:16.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.842+0000 7f91bd772640 1 -- 172.21.15.131:0/304892326 shutdown_connections 2024-09-17T02:41:16.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.842+0000 7f91bd772640 1 -- 172.21.15.131:0/304892326 wait complete. 2024-09-17T02:41:16.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.846+0000 7f91bd772640 1 Processor -- start 2024-09-17T02:41:16.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.846+0000 7f91bd772640 1 -- start start 2024-09-17T02:41:16.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91bd772640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91b8156bd0 0x7f91b8165030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91bd772640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b8164310 0x7f91b81588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91bd772640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91b80c30b0 0x7f91b80b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91b77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91b80c30b0 0x7f91b80b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91b67fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b8164310 0x7f91b81588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91b77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91b80c30b0 0x7f91b80b74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57950/0 (socket says 172.21.15.131:57950) 2024-09-17T02:41:16.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91b67fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b8164310 0x7f91b81588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.131:56004/0 (socket says 172.21.15.131:56004) 2024-09-17T02:41:16.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91b77fe640 1 -- 172.21.15.131:0/3529493747 learned_addr learned my addr 172.21.15.131:0/3529493747 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:16.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8168be0 con 0x7f91b80c30b0 2024-09-17T02:41:16.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8168a60 con 0x7f91b8164310 2024-09-17T02:41:16.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f91b816ae70 con 0x7f91b8156bd0 2024-09-17T02:41:16.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91b77fe640 1 -- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91b8156bd0 msgr2=0x7f91b8165030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T02:41:16.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91b77fe640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91b8156bd0 0x7f91b8165030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91b77fe640 1 -- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b8164310 msgr2=0x7f91b81588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:16.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91b77fe640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b8164310 0x7f91b81588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:16.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91b77fe640 1 -- 172.21.15.131:0/3529493747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b81676e0 con 0x7f91b80c30b0 2024-09-17T02:41:16.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.850+0000 7f91b77fe640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91b80c30b0 0x7f91b80b74a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f91b000c760 tx=0x7f91b000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.862+0000 7f919f7fe640 1 -- 172.21.15.131:0/3529493747 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0003b80 con 0x7f91b80c30b0 2024-09-17T02:41:16.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.862+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b81679c0 con 0x7f91b80c30b0 2024-09-17T02:41:16.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.862+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b8157680 con 0x7f91b80c30b0 2024-09-17T02:41:16.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.862+0000 7f919f7fe640 1 -- 172.21.15.131:0/3529493747 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91b0003d20 con 0x7f91b80c30b0 2024-09-17T02:41:16.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.862+0000 7f919f7fe640 1 -- 172.21.15.131:0/3529493747 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b000aab0 con 0x7f91b80c30b0 2024-09-17T02:41:16.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.862+0000 7f919f7fe640 1 -- 172.21.15.131:0/3529493747 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f91b000ac50 con 0x7f91b80c30b0 2024-09-17T02:41:16.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.862+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91980052a0 con 0x7f91b80c30b0 2024-09-17T02:41:16.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.866+0000 7f919f7fe640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f91a4073250 0x7f91a4075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:16.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.866+0000 7f919f7fe640 1 -- 172.21.15.131:0/3529493747 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f91b0093f30 con 0x7f91b80c30b0 2024-09-17T02:41:16.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.874+0000 7f91b6ffd640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f91a4073250 0x7f91a4075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:16.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.874+0000 7f91b6ffd640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f91a4073250 0x7f91a4075760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f91a80046c0 tx=0x7f91a8007a80 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:16.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:16.874+0000 7f919f7fe640 1 -- 172.21.15.131:0/3529493747 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f91b005ed80 con 0x7f91b80c30b0 2024-09-17T02:41:17.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.118+0000 7f3ded7fa640 1 -- 172.21.15.131:0/3223646554 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f3de0005590 con 0x7f3e080aaec0 2024-09-17T02:41:17.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.122+0000 7f3def7fe640 1 -- 172.21.15.131:0/3223646554 <== mon.2 v2:172.21.15.189:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3df8063990 con 0x7f3e080aaec0 2024-09-17T02:41:17.123 INFO:teuthology.orchestra.run.smithi131.stdout:64424509449 2024-09-17T02:41:17.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.122+0000 7f246aa1f640 1 -- 172.21.15.131:0/2557691173 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f245c0dee40 con 0x7f245c0e0890 2024-09-17T02:41:17.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.122+0000 7f245a7fc640 1 -- 172.21.15.131:0/2557691173 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f245405f570 con 0x7f245c0e0890 2024-09-17T02:41:17.124 INFO:teuthology.orchestra.run.smithi131.stdout:64424509449 2024-09-17T02:41:17.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 -- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24480732d0 msgr2=0x7f24480757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24480732d0 0x7f24480757e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f24600af000 tx=0x7f246001afd0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 -- 172.21.15.131:0/2557691173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c0e0890 msgr2=0x7f245c0d8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c0e0890 0x7f245c0d8980 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f245400ea20 tx=0x7f245400ee20 comp rx=0 tx=0).stop 2024-09-17T02:41:17.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 -- 172.21.15.131:0/2557691173 shutdown_connections 2024-09-17T02:41:17.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24480732d0 0x7f24480757e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f245c0e1a80 0x7f245c180e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f245c0e1670 0x7f245c0de6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 --2- 172.21.15.131:0/2557691173 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f245c0e0890 0x7f245c0d8980 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 -- 172.21.15.131:0/2557691173 >> 172.21.15.131:0/2557691173 conn(0x7f245c01eaf0 msgr2=0x7f245c0db820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:17.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 -- 172.21.15.131:0/2557691173 shutdown_connections 2024-09-17T02:41:17.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.126+0000 7f2447fff640 1 -- 172.21.15.131:0/2557691173 wait complete. 2024-09-17T02:41:17.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.130+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3df4073200 msgr2=0x7f3df4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.130+0000 7f3e0d4ff640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3df4073200 0x7f3df4075710 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc0af000 tx=0x7f3dfc01b130 comp rx=0 tx=0).stop 2024-09-17T02:41:17.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.130+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e080aaec0 msgr2=0x7f3e0815e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.130+0000 7f3e0d4ff640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e080aaec0 0x7f3e0815e450 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3df800e5d0 tx=0x7f3df800ea90 comp rx=0 tx=0).stop 2024-09-17T02:41:17.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.134+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/3223646554 shutdown_connections 2024-09-17T02:41:17.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.134+0000 7f3e0d4ff640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3df4073200 0x7f3df4075710 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.134+0000 7f3e0d4ff640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e080aaec0 0x7f3e0815e450 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.134+0000 7f3e0d4ff640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e08155840 0x7f3e081552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.134+0000 7f3e0d4ff640 1 --2- 172.21.15.131:0/3223646554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e080707f0 0x7f3e080abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.134+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/3223646554 >> 172.21.15.131:0/3223646554 conn(0x7f3e080d0ad0 msgr2=0x7f3e08159490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:17.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.134+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/3223646554 shutdown_connections 2024-09-17T02:41:17.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.134+0000 7f3e0d4ff640 1 -- 172.21.15.131:0/3223646554 wait complete. 2024-09-17T02:41:17.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.142+0000 7f71ff59e640 1 -- 172.21.15.131:0/2314912110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f71f8022ed0 con 0x7f71f80346b0 2024-09-17T02:41:17.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.142+0000 7f71eeffd640 1 -- 172.21.15.131:0/2314912110 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f71f405ddf0 con 0x7f71f80346b0 2024-09-17T02:41:17.143 INFO:teuthology.orchestra.run.smithi131.stdout:64424509449 2024-09-17T02:41:17.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.146+0000 7f3356ffd640 1 -- 172.21.15.131:0/2793796765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f33700c7670 con 0x7f33700cc3b0 2024-09-17T02:41:17.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.146+0000 7f71ecff9640 1 -- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71e0073200 msgr2=0x7f71e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.146+0000 7f71ecff9640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71e0073200 0x7f71e0075710 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f71f00b2000 tx=0x7f71f001d180 comp rx=0 tx=0).stop 2024-09-17T02:41:17.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.146+0000 7f71ecff9640 1 -- 172.21.15.131:0/2314912110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f80346b0 msgr2=0x7f71f80e0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.146+0000 7f71ecff9640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f80346b0 0x7f71f80e0540 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f71f400c3c0 tx=0x7f71f400c880 comp rx=0 tx=0).stop 2024-09-17T02:41:17.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.146+0000 7f336d7fa640 1 -- 172.21.15.131:0/2793796765 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f336405f3a0 con 0x7f33700cc3b0 2024-09-17T02:41:17.148 INFO:teuthology.orchestra.run.smithi131.stdout:64424509450 2024-09-17T02:41:17.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f71ecff9640 1 -- 172.21.15.131:0/2314912110 shutdown_connections 2024-09-17T02:41:17.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f71ecff9640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f71e0073200 0x7f71e0075710 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f71ecff9640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71f8053a70 0x7f71f8048a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f71ecff9640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71f80346b0 0x7f71f80e0540 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f71ecff9640 1 --2- 172.21.15.131:0/2314912110 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f71f8007240 0x7f71f80d8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f71ecff9640 1 -- 172.21.15.131:0/2314912110 >> 172.21.15.131:0/2314912110 conn(0x7f71f801e270 msgr2=0x7f71f80320f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:17.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f71ecff9640 1 -- 172.21.15.131:0/2314912110 shutdown_connections 2024-09-17T02:41:17.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f71ecff9640 1 -- 172.21.15.131:0/2314912110 wait complete. 2024-09-17T02:41:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f3356ffd640 1 -- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f335c073230 msgr2=0x7f335c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f3356ffd640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f335c073230 0x7f335c075740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f33600b0000 tx=0x7f336001c3d0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f3356ffd640 1 -- 172.21.15.131:0/2793796765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33700cc3b0 msgr2=0x7f33701561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f3356ffd640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33700cc3b0 0x7f33701561c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f336400e800 tx=0x7f336400ecc0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f3356ffd640 1 -- 172.21.15.131:0/2793796765 shutdown_connections 2024-09-17T02:41:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f3356ffd640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f335c073230 0x7f335c075740 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f33600b0000 tx=0x7f336001c3d0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f3356ffd640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f337015f010 0x7f33700b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f3356ffd640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f337015a470 0x7f33700b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f3356ffd640 1 --2- 172.21.15.131:0/2793796765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f33700cc3b0 0x7f33701561c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.150+0000 7f3356ffd640 1 -- 172.21.15.131:0/2793796765 >> 172.21.15.131:0/2793796765 conn(0x7f33700cd6a0 msgr2=0x7f33700b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.154+0000 7f3356ffd640 1 -- 172.21.15.131:0/2793796765 shutdown_connections 2024-09-17T02:41:17.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.154+0000 7f3356ffd640 1 -- 172.21.15.131:0/2793796765 wait complete. 2024-09-17T02:41:17.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.158+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa3c40f2740 con 0x7fa3bc005030 2024-09-17T02:41:17.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509449 for osd.1 2024-09-17T02:41:17.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.162+0000 7fa3c8ff9640 1 -- 172.21.15.131:0/1205703452 <== mon.1 v2:172.21.15.189:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa3b8064170 con 0x7fa3bc005030 2024-09-17T02:41:17.162 INFO:teuthology.orchestra.run.smithi131.stdout:64424509448 2024-09-17T02:41:17.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.162+0000 7f3ee3584640 1 -- 172.21.15.131:0/3250460420 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3edc0cd030 con 0x7f3edc151a90 2024-09-17T02:41:17.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.162+0000 7f3ed1ffb640 1 -- 172.21.15.131:0/3250460420 <== mon.2 v2:172.21.15.189:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3edc074550 con 0x7f3edc151a90 2024-09-17T02:41:17.163 INFO:teuthology.orchestra.run.smithi131.stdout:64424509450 2024-09-17T02:41:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa3b0073200 msgr2=0x7fa3b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa3b0073200 0x7fa3b0075710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40f2710 tx=0x7fa3c00091e0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3bc005030 msgr2=0x7fa3c4180b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3bc005030 0x7fa3c4180b90 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8003ed0 tx=0x7fa3b8001640 comp rx=0 tx=0).stop 2024-09-17T02:41:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 shutdown_connections 2024-09-17T02:41:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa3b0073200 0x7fa3b0075710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa3c4053960 0x7fa3c418d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa3c4055e10 0x7fa3c4053420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 --2- 172.21.15.131:0/1205703452 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa3bc005030 0x7fa3c4180b90 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 >> 172.21.15.131:0/1205703452 conn(0x7fa3c401eaf0 msgr2=0x7fa3c403a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 shutdown_connections 2024-09-17T02:41:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.166+0000 7fa3d2437640 1 -- 172.21.15.131:0/1205703452 wait complete. 2024-09-17T02:41:17.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509449 for osd.7 2024-09-17T02:41:17.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 -- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ec0073250 msgr2=0x7f3ec0075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ec0073250 0x7f3ec0075760 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80af000 tx=0x7f3ed8000bd0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 -- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3edc151a90 msgr2=0x7f3edc149e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3edc151a90 0x7f3edc149e20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3ed400c6b0 tx=0x7f3ed400cb70 comp rx=0 tx=0).stop 2024-09-17T02:41:17.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 -- 172.21.15.131:0/3250460420 shutdown_connections 2024-09-17T02:41:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ec0073250 0x7f3ec0075760 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3edc151a90 0x7f3edc149e20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3edc074140 0x7f3edc1498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 --2- 172.21.15.131:0/3250460420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3edc070c80 0x7f3edc1382d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 -- 172.21.15.131:0/3250460420 >> 172.21.15.131:0/3250460420 conn(0x7f3edc0d13d0 msgr2=0x7f3edc15cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 -- 172.21.15.131:0/3250460420 shutdown_connections 2024-09-17T02:41:17.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.170+0000 7f3ee3584640 1 -- 172.21.15.131:0/3250460420 wait complete. 2024-09-17T02:41:17.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509449 for osd.2 2024-09-17T02:41:17.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.182+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fdf680ccaf0 con 0x7fdf681f45e0 2024-09-17T02:41:17.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf57fff640 1 -- 172.21.15.131:0/3104254150 <== mon.2 v2:172.21.15.189:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdf5805fbf0 con 0x7fdf681f45e0 2024-09-17T02:41:17.184 INFO:teuthology.orchestra.run.smithi131.stdout:64424509449 2024-09-17T02:41:17.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509450 for osd.0 2024-09-17T02:41:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf4c073230 msgr2=0x7fdf4c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf4c073230 0x7fdf4c075740 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf600066a0 tx=0x7fdf60006ee0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf681f45e0 msgr2=0x7fdf681f49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf681f45e0 0x7fdf681f49f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf68072440 tx=0x7fdf58009840 comp rx=0 tx=0).stop 2024-09-17T02:41:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 shutdown_connections 2024-09-17T02:41:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf4c073230 0x7fdf4c075740 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf681f45e0 0x7fdf681f49f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf680b5110 0x7fdf681f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 --2- 172.21.15.131:0/3104254150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf50005920 0x7fdf681e6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 >> 172.21.15.131:0/3104254150 conn(0x7fdf680cd6a0 msgr2=0x7fdf68156ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:17.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 shutdown_connections 2024-09-17T02:41:17.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.186+0000 7fdf6de33640 1 -- 172.21.15.131:0/3104254150 wait complete. 2024-09-17T02:41:17.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509450 for osd.4 2024-09-17T02:41:17.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509448 for osd.3 2024-09-17T02:41:17.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9198005860 con 0x7f91b80c30b0 2024-09-17T02:41:17.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f919f7fe640 1 -- 172.21.15.131:0/3529493747 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f91b005e680 con 0x7f91b80c30b0 2024-09-17T02:41:17.213 INFO:teuthology.orchestra.run.smithi131.stdout:55834574858 2024-09-17T02:41:17.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509449 for osd.5 2024-09-17T02:41:17.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f91a4073250 msgr2=0x7f91a4075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f91a4073250 0x7f91a4075760 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f91a80046c0 tx=0x7f91a8007a80 comp rx=0 tx=0).stop 2024-09-17T02:41:17.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91b80c30b0 msgr2=0x7f91b80b74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:17.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91b80c30b0 0x7f91b80b74a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f91b000c760 tx=0x7f91b000cc20 comp rx=0 tx=0).stop 2024-09-17T02:41:17.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 shutdown_connections 2024-09-17T02:41:17.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f91a4073250 0x7f91a4075760 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91b80c30b0 0x7f91b80b74a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91b8164310 0x7f91b81588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 --2- 172.21.15.131:0/3529493747 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91b8156bd0 0x7f91b8165030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:17.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 >> 172.21.15.131:0/3529493747 conn(0x7f91b80cf150 msgr2=0x7f91b815a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:17.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 shutdown_connections 2024-09-17T02:41:17.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:17.214+0000 7f91bd772640 1 -- 172.21.15.131:0/3529493747 wait complete. 2024-09-17T02:41:17.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2024-09-17T02:41:18.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T02:41:18.166 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T02:41:18.177 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T02:41:18.188 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T02:41:18.192 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T02:41:18.196 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T02:41:18.214 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T02:41:18.233 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T02:41:18.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.394+0000 7f32ceffd640 1 -- 172.21.15.131:0/1644049387 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c80a5e60 con 0x7f32d0073cd0 2024-09-17T02:41:18.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.394+0000 7f32d680d640 1 -- 172.21.15.131:0/1644049387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d0073cd0 msgr2=0x7f32d00700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.394+0000 7f32d680d640 1 --2- 172.21.15.131:0/1644049387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d0073cd0 0x7f32d00700e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009c80 tx=0x7f32c801cd90 comp rx=0 tx=0).stop 2024-09-17T02:41:18.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.398+0000 7f32d680d640 1 -- 172.21.15.131:0/1644049387 shutdown_connections 2024-09-17T02:41:18.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.398+0000 7f32d680d640 1 --2- 172.21.15.131:0/1644049387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d0073cd0 0x7f32d00700e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.398+0000 7f32d680d640 1 --2- 172.21.15.131:0/1644049387 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32d00b4df0 0x7f32d0073790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.398+0000 7f32d680d640 1 -- 172.21.15.131:0/1644049387 >> 172.21.15.131:0/1644049387 conn(0x7f32d00d0ad0 msgr2=0x7f32d006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.398+0000 7f32d680d640 1 -- 172.21.15.131:0/1644049387 shutdown_connections 2024-09-17T02:41:18.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.398+0000 7f32d680d640 1 -- 172.21.15.131:0/1644049387 wait complete. 2024-09-17T02:41:18.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.398+0000 7f32d680d640 1 Processor -- start 2024-09-17T02:41:18.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.398+0000 7f32d680d640 1 -- start start 2024-09-17T02:41:18.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32d680d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32d0070620 0x7f32d0154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32d680d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d00b4df0 0x7f32d01547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32d680d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32d00acde0 0x7f32d0157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32cffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32d0070620 0x7f32d0154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32d680d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f32d015d080 con 0x7f32d0070620 2024-09-17T02:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32cffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32d0070620 0x7f32d0154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57964/0 (socket says 172.21.15.131:57964) 2024-09-17T02:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32cffff640 1 -- 172.21.15.131:0/2555950854 learned_addr learned my addr 172.21.15.131:0/2555950854 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32d015cf00 con 0x7f32d00b4df0 2024-09-17T02:41:18.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f32d015d200 con 0x7f32d00acde0 2024-09-17T02:41:18.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32cf7fe640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d00b4df0 0x7f32d01547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32d4daa640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32d00acde0 0x7f32d0157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32cffff640 1 -- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32d00acde0 msgr2=0x7f32d0157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32cffff640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32d00acde0 0x7f32d0157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32cffff640 1 -- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d00b4df0 msgr2=0x7f32d01547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32cffff640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d00b4df0 0x7f32d01547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32cffff640 1 -- 172.21.15.131:0/2555950854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d00b1940 con 0x7f32d0070620 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32cffff640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32d0070620 0x7f32d0154270 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f32c000e7a0 tx=0x7f32c000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32ccff9640 1 -- 172.21.15.131:0/2555950854 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c000cc60 con 0x7f32d0070620 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32ccff9640 1 -- 172.21.15.131:0/2555950854 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32c000ee50 con 0x7f32d0070620 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32ccff9640 1 -- 172.21.15.131:0/2555950854 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c0010670 con 0x7f32d0070620 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d01564d0 con 0x7f32d0070620 2024-09-17T02:41:18.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.402+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d0156a90 con 0x7f32d0070620 2024-09-17T02:41:18.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.406+0000 7f32ccff9640 1 -- 172.21.15.131:0/2555950854 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f32c0018420 con 0x7f32d0070620 2024-09-17T02:41:18.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.406+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ac0052a0 con 0x7f32d0070620 2024-09-17T02:41:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.406+0000 7f32ccff9640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32b4073230 0x7f32b4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.406+0000 7f32ccff9640 1 -- 172.21.15.131:0/2555950854 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f32c0094ad0 con 0x7f32d0070620 2024-09-17T02:41:18.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.410+0000 7f32cf7fe640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32b4073230 0x7f32b4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.414+0000 7f32cf7fe640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32b4073230 0x7f32b4075740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f32c80af000 tx=0x7f32c801b5d0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.418+0000 7f32ccff9640 1 -- 172.21.15.131:0/2555950854 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f32c0098070 con 0x7f32d0070620 2024-09-17T02:41:18.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.426+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3740570346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb04004810 msgr2=0x7fbb04004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.426+0000 7fbb1e6e2640 1 --2- 172.21.15.131:0/3740570346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb04004810 0x7fbb04004c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbb180ce090 tx=0x7fbb1001ce30 comp rx=0 tx=0).stop 2024-09-17T02:41:18.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.426+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3740570346 shutdown_connections 2024-09-17T02:41:18.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.426+0000 7fbb1e6e2640 1 --2- 172.21.15.131:0/3740570346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb04004810 0x7fbb04004c20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.426+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3740570346 >> 172.21.15.131:0/3740570346 conn(0x7fbb180b1a30 msgr2=0x7fbb180b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.426+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3740570346 shutdown_connections 2024-09-17T02:41:18.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.426+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3740570346 wait complete. 2024-09-17T02:41:18.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.426+0000 7fbb1e6e2640 1 Processor -- start 2024-09-17T02:41:18.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.426+0000 7fbb1e6e2640 1 -- start start 2024-09-17T02:41:18.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1e6e2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb04004810 0x7fbb180b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1e6e2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb180b3f30 0x7fbb181f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1e6e2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbb18153830 0x7fbb18153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1d6e0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb04004810 0x7fbb180b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1d6e0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb04004810 0x7fbb180b58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57992/0 (socket says 172.21.15.131:57992) 2024-09-17T02:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1d6e0640 1 -- 172.21.15.131:0/3135531490 learned_addr learned my addr 172.21.15.131:0/3135531490 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1815ebb0 con 0x7fbb04004810 2024-09-17T02:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1815ea30 con 0x7fbb180b3f30 2024-09-17T02:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1815ed30 con 0x7fbb18153830 2024-09-17T02:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1dee1640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbb18153830 0x7fbb18153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1d6e0640 1 -- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbb18153830 msgr2=0x7fbb18153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1d6e0640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbb18153830 0x7fbb18153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1d6e0640 1 -- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb180b3f30 msgr2=0x7fbb181f41f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T02:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1d6e0640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb180b3f30 0x7fbb181f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1d6e0640 1 -- 172.21.15.131:0/3135531490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1815a5d0 con 0x7fbb04004810 2024-09-17T02:41:18.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1d6e0640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb04004810 0x7fbb180b58c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbb14001a40 tx=0x7fbb140039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb0dffb640 1 -- 172.21.15.131:0/3135531490 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14015070 con 0x7fbb04004810 2024-09-17T02:41:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb0dffb640 1 -- 172.21.15.131:0/3135531490 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb140119c0 con 0x7fbb04004810 2024-09-17T02:41:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb0dffb640 1 -- 172.21.15.131:0/3135531490 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14011dd0 con 0x7fbb04004810 2024-09-17T02:41:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb18154180 con 0x7fbb04004810 2024-09-17T02:41:18.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.430+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb181595e0 con 0x7fbb04004810 2024-09-17T02:41:18.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.434+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1815f190 con 0x7fbb04004810 2024-09-17T02:41:18.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.438+0000 7fbb0dffb640 1 -- 172.21.15.131:0/3135531490 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbb14018070 con 0x7fbb04004810 2024-09-17T02:41:18.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.442+0000 7fbb0dffb640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbafc073200 0x7fbafc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.442+0000 7fbb0dffb640 1 -- 172.21.15.131:0/3135531490 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fbb14095490 con 0x7fbb04004810 2024-09-17T02:41:18.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.442+0000 7fbb1cedf640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbafc073200 0x7fbafc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.442+0000 7fbb0dffb640 1 -- 172.21.15.131:0/3135531490 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbb18154180 con 0x7fbb04004810 2024-09-17T02:41:18.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.446+0000 7fbb1cedf640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbafc073200 0x7fbafc075710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb100af000 tx=0x7fbb04005840 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.446+0000 7fc0b6419640 1 -- 172.21.15.131:0/439167881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0b00cff30 msgr2=0x7fc0b0070fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.446+0000 7fc0b6419640 1 --2- 172.21.15.131:0/439167881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0b00cff30 0x7fc0b0070fc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009cb0 tx=0x7fc0a801cc80 comp rx=0 tx=0).stop 2024-09-17T02:41:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.446+0000 7fc0b6419640 1 -- 172.21.15.131:0/439167881 shutdown_connections 2024-09-17T02:41:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.446+0000 7fc0b6419640 1 --2- 172.21.15.131:0/439167881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0b00cff30 0x7fc0b0070fc0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.446+0000 7fc0b6419640 1 --2- 172.21.15.131:0/439167881 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc0b00b65d0 0x7fc0b00cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.446+0000 7fc0b6419640 1 -- 172.21.15.131:0/439167881 >> 172.21.15.131:0/439167881 conn(0x7fc0b0073b10 msgr2=0x7fc0b0073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.446+0000 7fc0b6419640 1 -- 172.21.15.131:0/439167881 shutdown_connections 2024-09-17T02:41:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.446+0000 7fafd27d7640 1 -- 172.21.15.131:0/1865446575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafcc0b4df0 msgr2=0x7fafcc0735c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.446+0000 7fafd27d7640 1 --2- 172.21.15.131:0/1865446575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafcc0b4df0 0x7fafcc0735c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fafbc009840 tx=0x7fafbc01c730 comp rx=0 tx=0).stop 2024-09-17T02:41:18.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.446+0000 7fc0b6419640 1 -- 172.21.15.131:0/439167881 wait complete. 2024-09-17T02:41:18.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.450+0000 7fc0b6419640 1 Processor -- start 2024-09-17T02:41:18.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.450+0000 7fafd27d7640 1 -- 172.21.15.131:0/1865446575 shutdown_connections 2024-09-17T02:41:18.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.450+0000 7fafd27d7640 1 --2- 172.21.15.131:0/1865446575 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fafcc073b00 0x7fafcc070320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.450+0000 7fafd27d7640 1 --2- 172.21.15.131:0/1865446575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafcc0b4df0 0x7fafcc0735c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.450+0000 7fafd27d7640 1 -- 172.21.15.131:0/1865446575 >> 172.21.15.131:0/1865446575 conn(0x7fafcc0d0ad0 msgr2=0x7fafcc06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.450+0000 7fc0b6419640 1 -- start start 2024-09-17T02:41:18.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fafd27d7640 1 -- 172.21.15.131:0/1865446575 shutdown_connections 2024-09-17T02:41:18.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fafd27d7640 1 -- 172.21.15.131:0/1865446575 wait complete. 2024-09-17T02:41:18.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b6419640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0b00b65d0 0x7fc0b00aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b6419640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc0b00cff30 0x7fc0b006ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b5417640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0b00b65d0 0x7fc0b00aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b5417640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0b00b65d0 0x7fc0b00aad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58032/0 (socket says 172.21.15.131:58032) 2024-09-17T02:41:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b5417640 1 -- 172.21.15.131:0/3001851047 learned_addr learned my addr 172.21.15.131:0/3001851047 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b6419640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0b0144980 0x7fc0b006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b6419640 1 -- 172.21.15.131:0/3001851047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0168b20 con 0x7fc0b00b65d0 2024-09-17T02:41:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b6419640 1 -- 172.21.15.131:0/3001851047 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b01689a0 con 0x7fc0b0144980 2024-09-17T02:41:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b6419640 1 -- 172.21.15.131:0/3001851047 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc0b0168ca0 con 0x7fc0b00cff30 2024-09-17T02:41:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b4c16640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc0b00cff30 0x7fc0b006ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b5c18640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0b0144980 0x7fc0b006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b5417640 1 -- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc0b00cff30 msgr2=0x7fc0b006ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b5417640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc0b00cff30 0x7fc0b006ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b5417640 1 -- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0b0144980 msgr2=0x7fc0b006d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b5417640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0b0144980 0x7fc0b006d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b5417640 1 -- 172.21.15.131:0/3001851047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b0163190 con 0x7fc0b00b65d0 2024-09-17T02:41:18.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b5c18640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0b0144980 0x7fc0b006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:18.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.454+0000 7fc0b5417640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0b00b65d0 0x7fc0b00aad50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000c420 tx=0x7fc0a000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7fafd27d7640 1 Processor -- start 2024-09-17T02:41:18.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7fc0a5ffb640 1 -- 172.21.15.131:0/3001851047 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0017070 con 0x7fc0b00b65d0 2024-09-17T02:41:18.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7fafd27d7640 1 -- start start 2024-09-17T02:41:18.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7fc0b6419640 1 -- 172.21.15.131:0/3001851047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b0163470 con 0x7fc0b00b65d0 2024-09-17T02:41:18.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7fc0b6419640 1 -- 172.21.15.131:0/3001851047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b0163a30 con 0x7fc0b00b65d0 2024-09-17T02:41:18.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7fc0b6419640 1 -- 172.21.15.131:0/3001851047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b00cf9f0 con 0x7fc0b00b65d0 2024-09-17T02:41:18.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/3220113130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc15d130 msgr2=0x7f6abc0727a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7f6ac1fa6640 1 --2- 172.21.15.131:0/3220113130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc15d130 0x7f6abc0727a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6abc074400 tx=0x7f6ab401cda0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/3220113130 shutdown_connections 2024-09-17T02:41:18.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7f6ac1fa6640 1 --2- 172.21.15.131:0/3220113130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc15d130 0x7f6abc0727a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7f6ac1fa6640 1 --2- 172.21.15.131:0/3220113130 >> v2:172.21.15.189:3300/0 conn(0x7f6abc159cf0 0x7f6abc0cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.458+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/3220113130 >> 172.21.15.131:0/3220113130 conn(0x7f6abc0d22f0 msgr2=0x7f6abc0d2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fc0a5ffb640 1 -- 172.21.15.131:0/3001851047 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0a00052d0 con 0x7fc0b00b65d0 2024-09-17T02:41:18.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fc0a5ffb640 1 -- 172.21.15.131:0/3001851047 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0021060 con 0x7fc0b00b65d0 2024-09-17T02:41:18.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd27d7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafcc073b00 0x7fafcc157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd27d7640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fafcc0b4df0 0x7fafcc14c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7f791e532640 1 -- 172.21.15.131:0/2671510651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7908003a00 msgr2=0x7f7908003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7f791e532640 1 --2- 172.21.15.131:0/2671510651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7908003a00 0x7f7908003e90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f791806bd50 tx=0x7f7918098d70 comp rx=0 tx=0).stop 2024-09-17T02:41:18.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafcbfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafcc073b00 0x7fafcc157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafcbfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafcc073b00 0x7fafcc157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.131:56082/0 (socket says 172.21.15.131:56082) 2024-09-17T02:41:18.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafcbfff640 1 -- 172.21.15.131:0/1636429153 learned_addr learned my addr 172.21.15.131:0/1636429153 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:18.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafc3fff640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fafcc0b4df0 0x7fafcc14c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd27d7640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafcc15e4f0 0x7fafcc153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd0d74640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafcc15e4f0 0x7fafcc153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fc0a5ffb640 1 -- 172.21.15.131:0/3001851047 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc0a001a070 con 0x7fc0b00b65d0 2024-09-17T02:41:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc16ba80 con 0x7fafcc15e4f0 2024-09-17T02:41:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc16b900 con 0x7fafcc073b00 2024-09-17T02:41:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fafcc16bc00 con 0x7fafcc0b4df0 2024-09-17T02:41:18.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fc0a5ffb640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc094073200 0x7fc094075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fc0a5ffb640 1 -- 172.21.15.131:0/3001851047 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fc0a0093d60 con 0x7fc0b00b65d0 2024-09-17T02:41:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fc0b4c16640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc094073200 0x7fc094075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd0d74640 1 -- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fafcc0b4df0 msgr2=0x7fafcc14c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd0d74640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fafcc0b4df0 0x7fafcc14c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd0d74640 1 -- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafcc073b00 msgr2=0x7fafcc157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd0d74640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafcc073b00 0x7fafcc157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.462+0000 7fafd0d74640 1 -- 172.21.15.131:0/1636429153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc1fcb10 con 0x7fafcc15e4f0 2024-09-17T02:41:18.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.466+0000 7fafd0d74640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafcc15e4f0 0x7fafcc153000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fafc400e990 tx=0x7fafc400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.466+0000 7fc0b4c16640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc094073200 0x7fc094075710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0058d0 tx=0x7fc0ac005d90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.466+0000 7f791e532640 1 -- 172.21.15.131:0/2671510651 shutdown_connections 2024-09-17T02:41:18.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.466+0000 7f791e532640 1 --2- 172.21.15.131:0/2671510651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7908003a00 0x7f7908003e90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.466+0000 7f791e532640 1 -- 172.21.15.131:0/2671510651 >> 172.21.15.131:0/2671510651 conn(0x7f7910051420 msgr2=0x7f7910051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.466+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/3220113130 shutdown_connections 2024-09-17T02:41:18.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.466+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/3220113130 wait complete. 2024-09-17T02:41:18.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.466+0000 7f791e532640 1 -- 172.21.15.131:0/2671510651 shutdown_connections 2024-09-17T02:41:18.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.466+0000 7f791e532640 1 -- 172.21.15.131:0/2671510651 wait complete. 2024-09-17T02:41:18.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.466+0000 7f6ac1fa6640 1 Processor -- start 2024-09-17T02:41:18.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.466+0000 7f6ac1fa6640 1 -- start start 2024-09-17T02:41:18.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f791e532640 1 Processor -- start 2024-09-17T02:41:18.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f791e532640 1 -- start start 2024-09-17T02:41:18.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7fc0a5ffb640 1 -- 172.21.15.131:0/3001851047 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc0a005e460 con 0x7fc0b00b65d0 2024-09-17T02:41:18.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f6ac1fa6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc072ce0 0x7f6abc0c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7fafc97fa640 1 -- 172.21.15.131:0/1636429153 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4004d30 con 0x7fafcc15e4f0 2024-09-17T02:41:18.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f6ac1fa6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6abc15d130 0x7f6abc0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7fafc97fa640 1 -- 172.21.15.131:0/1636429153 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafc400cb60 con 0x7fafcc15e4f0 2024-09-17T02:41:18.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc1571a0 con 0x7fafcc15e4f0 2024-09-17T02:41:18.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc1fc030 con 0x7fafcc15e4f0 2024-09-17T02:41:18.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f6ac0fa4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc072ce0 0x7f6abc0c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f6ac0fa4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc072ce0 0x7f6abc0c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58046/0 (socket says 172.21.15.131:58046) 2024-09-17T02:41:18.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f791e532640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7908003a00 0x7f79100dd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f791e532640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7910050150 0x7f79100de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f791e532640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79100de700 0x7f791017c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f7917fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7908003a00 0x7f79100dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f7917fff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7908003a00 0x7f79100dd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.131:56096/0 (socket says 172.21.15.131:56096) 2024-09-17T02:41:18.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f79177fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7910050150 0x7f79100de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f79177fe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7910050150 0x7f79100de1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.131:36790/0 (socket says 172.21.15.131:36790) 2024-09-17T02:41:18.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f6ac0fa4640 1 -- 172.21.15.131:0/1640735672 learned_addr learned my addr 172.21.15.131:0/1640735672 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:18.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7fafc97fa640 1 -- 172.21.15.131:0/1636429153 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4005020 con 0x7fafcc15e4f0 2024-09-17T02:41:18.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f791cacf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79100de700 0x7f791017c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.470+0000 7f791cacf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79100de700 0x7f791017c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58060/0 (socket says 172.21.15.131:58060) 2024-09-17T02:41:18.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6ac1fa6640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6abc0b7880 0x7f6abc0b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc169ae0 con 0x7f6abc072ce0 2024-09-17T02:41:18.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc169960 con 0x7f6abc15d130 2024-09-17T02:41:18.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6abc169c60 con 0x7f6abc0b7880 2024-09-17T02:41:18.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6abbfff640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6abc15d130 0x7f6abc0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f7917fff640 1 -- 172.21.15.131:0/2424625221 learned_addr learned my addr 172.21.15.131:0/2424625221 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:18.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f791000b190 con 0x7f79100de700 2024-09-17T02:41:18.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6ac17a5640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6abc0b7880 0x7f6abc0b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f791000b010 con 0x7f7908003a00 2024-09-17T02:41:18.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7fafc97fa640 1 -- 172.21.15.131:0/1636429153 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fafc40051c0 con 0x7fafcc15e4f0 2024-09-17T02:41:18.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f791000b310 con 0x7f7910050150 2024-09-17T02:41:18.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6ac0fa4640 1 -- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6abc0b7880 msgr2=0x7f6abc0b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6ac0fa4640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6abc0b7880 0x7f6abc0b5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6ac0fa4640 1 -- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6abc15d130 msgr2=0x7f6abc0b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6ac0fa4640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6abc15d130 0x7f6abc0b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6ac0fa4640 1 -- 172.21.15.131:0/1640735672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6abc14f370 con 0x7f6abc072ce0 2024-09-17T02:41:18.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6abbfff640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6abc15d130 0x7f6abc0b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T02:41:18.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f6ac0fa4640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc072ce0 0x7f6abc0c8c40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000b7b0 tx=0x7f6ab000bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7fafc97fa640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fafac073250 0x7fafac075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7fafc97fa640 1 -- 172.21.15.131:0/1636429153 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fafc4094fd0 con 0x7fafcc15e4f0 2024-09-17T02:41:18.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7fafcbfff640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fafac073250 0x7fafac075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f791cacf640 1 -- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7910050150 msgr2=0x7f79100de1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f791cacf640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7910050150 0x7f79100de1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f791cacf640 1 -- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7908003a00 msgr2=0x7f79100dd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f791cacf640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7908003a00 0x7f79100dd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f791cacf640 1 -- 172.21.15.131:0/2424625221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79100dd9e0 con 0x7f79100de700 2024-09-17T02:41:18.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f791cacf640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79100de700 0x7f791017c9d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f791000d630 tx=0x7f790c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f7914ff9640 1 -- 172.21.15.131:0/2424625221 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c017070 con 0x7f79100de700 2024-09-17T02:41:18.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7fafcbfff640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fafac073250 0x7fafac075760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fafb40046c0 tx=0x7fafb4007f60 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79100e0bc0 con 0x7f79100de700 2024-09-17T02:41:18.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.474+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79100e1180 con 0x7f79100de700 2024-09-17T02:41:18.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.478+0000 7f7914ff9640 1 -- 172.21.15.131:0/2424625221 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f790c004350 con 0x7f79100de700 2024-09-17T02:41:18.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.478+0000 7f7914ff9640 1 -- 172.21.15.131:0/2424625221 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790c0028f0 con 0x7f79100de700 2024-09-17T02:41:18.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.478+0000 7f6ab97fa640 1 -- 172.21.15.131:0/1640735672 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0019070 con 0x7f6abc072ce0 2024-09-17T02:41:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.478+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6abc14f610 con 0x7f6abc072ce0 2024-09-17T02:41:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.478+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6abc14fbd0 con 0x7f6abc072ce0 2024-09-17T02:41:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.478+0000 7f6ab97fa640 1 -- 172.21.15.131:0/1640735672 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ab000c830 con 0x7f6abc072ce0 2024-09-17T02:41:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.478+0000 7f6ab97fa640 1 -- 172.21.15.131:0/1640735672 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000cc80 con 0x7f6abc072ce0 2024-09-17T02:41:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.478+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f791001ca80 con 0x7f79100de700 2024-09-17T02:41:18.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.482+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb00052a0 con 0x7fafcc15e4f0 2024-09-17T02:41:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.482+0000 7f7914ff9640 1 -- 172.21.15.131:0/2424625221 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f790c01a070 con 0x7f79100de700 2024-09-17T02:41:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.482+0000 7f7914ff9640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f78fc0732d0 0x7f78fc0757e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.482+0000 7f7914ff9640 1 -- 172.21.15.131:0/2424625221 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f790c093420 con 0x7f79100de700 2024-09-17T02:41:18.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.482+0000 7f6ab97fa640 1 -- 172.21.15.131:0/1640735672 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6ab001c070 con 0x7f6abc072ce0 2024-09-17T02:41:18.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.482+0000 7f7917fff640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f78fc0732d0 0x7f78fc0757e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.486+0000 7f6ab97fa640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6aa8073280 0x7f6aa8075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.486+0000 7f6ab97fa640 1 -- 172.21.15.131:0/1640735672 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6ab0093600 con 0x7f6abc072ce0 2024-09-17T02:41:18.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.486+0000 7f7917fff640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f78fc0732d0 0x7f78fc0757e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7918052910 tx=0x7f7918130040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.486+0000 7f6abbfff640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6aa8073280 0x7f6aa8075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.486+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a9c0052a0 con 0x7f6abc072ce0 2024-09-17T02:41:18.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.486+0000 7f6abbfff640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6aa8073280 0x7f6aa8075790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40af000 tx=0x7f6ab401b5e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.486+0000 7f7914ff9640 1 -- 172.21.15.131:0/2424625221 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f790c05dbb0 con 0x7f79100de700 2024-09-17T02:41:18.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.494+0000 7f92c250f640 1 -- 172.21.15.131:0/3306877542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92bc070d50 msgr2=0x7f92bc0ceff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.494+0000 7f92c250f640 1 --2- 172.21.15.131:0/3306877542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92bc070d50 0x7f92bc0ceff0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f92bc06d2d0 tx=0x7f92b401cd80 comp rx=0 tx=0).stop 2024-09-17T02:41:18.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.494+0000 7f92c250f640 1 -- 172.21.15.131:0/3306877542 shutdown_connections 2024-09-17T02:41:18.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.494+0000 7f92c250f640 1 --2- 172.21.15.131:0/3306877542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92bc070d50 0x7f92bc0ceff0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.494+0000 7f92c250f640 1 --2- 172.21.15.131:0/3306877542 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f92bc069b80 0x7f92bc0ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.494+0000 7f92c250f640 1 --2- 172.21.15.131:0/3306877542 >> v2:172.21.15.131:3300/0 conn(0x7f92bc069770 0x7f92bc070810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.494+0000 7f92c250f640 1 -- 172.21.15.131:0/3306877542 >> 172.21.15.131:0/3306877542 conn(0x7f92bc0ccfe0 msgr2=0x7f92bc0cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.494+0000 7f92c250f640 1 -- 172.21.15.131:0/3306877542 shutdown_connections 2024-09-17T02:41:18.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.494+0000 7fafc97fa640 1 -- 172.21.15.131:0/1636429153 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fafc405f860 con 0x7fafcc15e4f0 2024-09-17T02:41:18.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.494+0000 7f92c250f640 1 -- 172.21.15.131:0/3306877542 wait complete. 2024-09-17T02:41:18.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.498+0000 7f92c250f640 1 Processor -- start 2024-09-17T02:41:18.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.498+0000 7f92c250f640 1 -- start start 2024-09-17T02:41:18.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.498+0000 7f92c250f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92bc069770 0x7f92bc2054b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.498+0000 7f92c150d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92bc069770 0x7f92bc2054b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.498+0000 7f92c150d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92bc069770 0x7f92bc2054b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58080/0 (socket says 172.21.15.131:58080) 2024-09-17T02:41:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c150d640 1 -- 172.21.15.131:0/2252275688 learned_addr learned my addr 172.21.15.131:0/2252275688 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c250f640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f92bc069b80 0x7f92bc073660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c250f640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92bc070d50 0x7f92bc073ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc15e480 con 0x7f92bc069770 2024-09-17T02:41:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c0d0c640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f92bc069b80 0x7f92bc073660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c1d0e640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92bc070d50 0x7f92bc073ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f92bc0cba10 con 0x7f92bc070d50 2024-09-17T02:41:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f92bc15e600 con 0x7f92bc069b80 2024-09-17T02:41:18.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c150d640 1 -- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f92bc069b80 msgr2=0x7f92bc073660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c150d640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f92bc069b80 0x7f92bc073660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c150d640 1 -- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92bc070d50 msgr2=0x7f92bc073ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c150d640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92bc070d50 0x7f92bc073ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c150d640 1 -- 172.21.15.131:0/2252275688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92bc2059f0 con 0x7f92bc069770 2024-09-17T02:41:18.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.502+0000 7f92c150d640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92bc069770 0x7f92bc2054b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00ea20 tx=0x7f92ac00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.506+0000 7f92b1ffb640 1 -- 172.21.15.131:0/2252275688 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00cd80 con 0x7f92bc069770 2024-09-17T02:41:18.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.506+0000 7f6ab97fa640 1 -- 172.21.15.131:0/1640735672 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6ab000ce20 con 0x7f6abc072ce0 2024-09-17T02:41:18.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.506+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92bc0b3fe0 con 0x7f92bc069770 2024-09-17T02:41:18.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.506+0000 7f92b1ffb640 1 -- 172.21.15.131:0/2252275688 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92ac004020 con 0x7f92bc069770 2024-09-17T02:41:18.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.506+0000 7f92b1ffb640 1 -- 172.21.15.131:0/2252275688 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac004470 con 0x7f92bc069770 2024-09-17T02:41:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.506+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92bc0b48f0 con 0x7f92bc069770 2024-09-17T02:41:18.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7fce69271640 1 -- 172.21.15.131:0/385876845 <== mon.1 v2:172.21.15.189:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce540a9b80 con 0x7fce5c0511b0 2024-09-17T02:41:18.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7f92b1ffb640 1 -- 172.21.15.131:0/2252275688 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f92ac003600 con 0x7f92bc069770 2024-09-17T02:41:18.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7f92b1ffb640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f92a0073230 0x7f92a0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7fce6b275640 1 -- 172.21.15.131:0/385876845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce5c0511b0 msgr2=0x7fce5c0515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7f92b1ffb640 1 -- 172.21.15.131:0/2252275688 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f92ac0945b0 con 0x7f92bc069770 2024-09-17T02:41:18.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7fce6b275640 1 --2- 172.21.15.131:0/385876845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce5c0511b0 0x7fce5c0515c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0e6af0 tx=0x7fce5401c770 comp rx=0 tx=0).stop 2024-09-17T02:41:18.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7fce6b275640 1 -- 172.21.15.131:0/385876845 shutdown_connections 2024-09-17T02:41:18.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92bc070810 con 0x7f92bc069770 2024-09-17T02:41:18.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7fce6b275640 1 --2- 172.21.15.131:0/385876845 >> v2:172.21.15.131:3300/0 conn(0x7fce5c04edf0 0x7fce5c04f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7fce6b275640 1 --2- 172.21.15.131:0/385876845 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce5c0511b0 0x7fce5c0515c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7fce6b275640 1 -- 172.21.15.131:0/385876845 >> 172.21.15.131:0/385876845 conn(0x7fce5c01d9b0 msgr2=0x7fce5c039670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7f92c0d0c640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f92a0073230 0x7f92a0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7fce6b275640 1 -- 172.21.15.131:0/385876845 shutdown_connections 2024-09-17T02:41:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7fce6b275640 1 -- 172.21.15.131:0/385876845 wait complete. 2024-09-17T02:41:18.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.510+0000 7f92c0d0c640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f92a0073230 0x7f92a0075740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0c4c70 tx=0x7f92b80091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.514+0000 7fce6b275640 1 Processor -- start 2024-09-17T02:41:18.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.514+0000 7fce6b275640 1 -- start start 2024-09-17T02:41:18.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.514+0000 7fce6b275640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce5c033bc0 0x7fce5c0d8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.514+0000 7fce6b275640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce5c04edf0 0x7fce5c0db620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6b275640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce5c0511b0 0x7fce5c0dbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6a273640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce5c033bc0 0x7fce5c0d8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6a273640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce5c033bc0 0x7fce5c0d8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3301/0 says I am v2:172.21.15.131:36816/0 (socket says 172.21.15.131:36816) 2024-09-17T02:41:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce69a72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce5c04edf0 0x7fce5c0db620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce69a72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce5c04edf0 0x7fce5c0db620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.131:56146/0 (socket says 172.21.15.131:56146) 2024-09-17T02:41:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6aa74640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce5c0511b0 0x7fce5c0dbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6aa74640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce5c0511b0 0x7fce5c0dbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58102/0 (socket says 172.21.15.131:58102) 2024-09-17T02:41:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6a273640 1 -- 172.21.15.131:0/1063799832 learned_addr learned my addr 172.21.15.131:0/1063799832 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c0f4b20 con 0x7fce5c0511b0 2024-09-17T02:41:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c0f49a0 con 0x7fce5c04edf0 2024-09-17T02:41:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c0f4ca0 con 0x7fce5c033bc0 2024-09-17T02:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6aa74640 1 -- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce5c033bc0 msgr2=0x7fce5c0d8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6aa74640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce5c033bc0 0x7fce5c0d8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6aa74640 1 -- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce5c04edf0 msgr2=0x7fce5c0db620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6aa74640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce5c04edf0 0x7fce5c0db620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6aa74640 1 -- 172.21.15.131:0/1063799832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c006610 con 0x7fce5c0511b0 2024-09-17T02:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6a273640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce5c033bc0 0x7fce5c0d8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T02:41:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6aa74640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce5c0511b0 0x7fce5c0dbb60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce64052910 tx=0x7fce64076d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7f92b1ffb640 1 -- 172.21.15.131:0/2252275688 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f92ac05ec50 con 0x7f92bc069770 2024-09-17T02:41:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce5affd640 1 -- 172.21.15.131:0/1063799832 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce64098070 con 0x7fce5c0511b0 2024-09-17T02:41:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce5affd640 1 -- 172.21.15.131:0/1063799832 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce6406abc0 con 0x7fce5c0511b0 2024-09-17T02:41:18.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.518+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c0068f0 con 0x7fce5c0511b0 2024-09-17T02:41:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.522+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c0e3190 con 0x7fce5c0511b0 2024-09-17T02:41:18.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.522+0000 7fce5affd640 1 -- 172.21.15.131:0/1063799832 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce6406af70 con 0x7fce5c0511b0 2024-09-17T02:41:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.522+0000 7fce5affd640 1 -- 172.21.15.131:0/1063799832 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fce6409b070 con 0x7fce5c0511b0 2024-09-17T02:41:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.522+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce5c04f200 con 0x7fce5c0511b0 2024-09-17T02:41:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.522+0000 7fce5affd640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce500733a0 0x7fce500758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:18.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.522+0000 7fce5affd640 1 -- 172.21.15.131:0/1063799832 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fce6410f940 con 0x7fce5c0511b0 2024-09-17T02:41:18.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.522+0000 7fce6a273640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce500733a0 0x7fce500758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:18.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.530+0000 7fce6a273640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce500733a0 0x7fce500758b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce60005480 tx=0x7fce60000bd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:18.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.534+0000 7fce5affd640 1 -- 172.21.15.131:0/1063799832 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fce64090070 con 0x7fce5c0511b0 2024-09-17T02:41:18.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.762+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f32ac005860 con 0x7f32d0070620 2024-09-17T02:41:18.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.762+0000 7f32ccff9640 1 -- 172.21.15.131:0/2555950854 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f32c005f0d0 con 0x7f32d0070620 2024-09-17T02:41:18.766 INFO:teuthology.orchestra.run.smithi131.stdout:64424509451 2024-09-17T02:41:18.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbb180ad480 con 0x7fbb04004810 2024-09-17T02:41:18.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32b4073230 msgr2=0x7f32b4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32b4073230 0x7f32b4075740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f32c80af000 tx=0x7f32c801b5d0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32d0070620 msgr2=0x7f32d0154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32d0070620 0x7f32d0154270 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f32c000e7a0 tx=0x7f32c000ec60 comp rx=0 tx=0).stop 2024-09-17T02:41:18.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7fbb0dffb640 1 -- 172.21.15.131:0/3135531490 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbb1806e7f0 con 0x7fbb04004810 2024-09-17T02:41:18.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 shutdown_connections 2024-09-17T02:41:18.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f32b4073230 0x7f32b4075740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32d00acde0 0x7f32d0157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f32d00b4df0 0x7f32d01547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 --2- 172.21.15.131:0/2555950854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f32d0070620 0x7f32d0154270 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 >> 172.21.15.131:0/2555950854 conn(0x7f32d00d0ad0 msgr2=0x7f32d00adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 shutdown_connections 2024-09-17T02:41:18.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.766+0000 7f32d680d640 1 -- 172.21.15.131:0/2555950854 wait complete. 2024-09-17T02:41:18.769 INFO:teuthology.orchestra.run.smithi131.stdout:64424509451 2024-09-17T02:41:18.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.770+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbafc073200 msgr2=0x7fbafc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.770+0000 7fbb1e6e2640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbafc073200 0x7fbafc075710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb100af000 tx=0x7fbb04005840 comp rx=0 tx=0).stop 2024-09-17T02:41:18.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.770+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb04004810 msgr2=0x7fbb180b58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.770+0000 7fbb1e6e2640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb04004810 0x7fbb180b58c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbb14001a40 tx=0x7fbb140039d0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.770+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 shutdown_connections 2024-09-17T02:41:18.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.770+0000 7fbb1e6e2640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbafc073200 0x7fbafc075710 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.770+0000 7fbb1e6e2640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbb18153830 0x7fbb18153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.770+0000 7fbb1e6e2640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb180b3f30 0x7fbb181f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.770+0000 7fbb1e6e2640 1 --2- 172.21.15.131:0/3135531490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb04004810 0x7fbb180b58c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.770+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 >> 172.21.15.131:0/3135531490 conn(0x7fbb180b1a30 msgr2=0x7fbb18073b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.774+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 shutdown_connections 2024-09-17T02:41:18.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.774+0000 7fbb1e6e2640 1 -- 172.21.15.131:0/3135531490 wait complete. 2024-09-17T02:41:18.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.794+0000 7fc0b6419640 1 -- 172.21.15.131:0/3001851047 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc0b015c2b0 con 0x7fc0b00b65d0 2024-09-17T02:41:18.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.798+0000 7fc0a5ffb640 1 -- 172.21.15.131:0/3001851047 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc0a0062e90 con 0x7fc0b00b65d0 2024-09-17T02:41:18.799 INFO:teuthology.orchestra.run.smithi131.stdout:64424509451 2024-09-17T02:41:18.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.2 2024-09-17T02:41:18.802 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:18.804 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.1 2024-09-17T02:41:18.805 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:18.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 -- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc094073200 msgr2=0x7fc094075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc094073200 0x7fc094075710 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac0058d0 tx=0x7fc0ac005d90 comp rx=0 tx=0).stop 2024-09-17T02:41:18.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 -- 172.21.15.131:0/3001851047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0b00b65d0 msgr2=0x7fc0b00aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0b00b65d0 0x7fc0b00aad50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000c420 tx=0x7fc0a000c8e0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 -- 172.21.15.131:0/3001851047 shutdown_connections 2024-09-17T02:41:18.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc094073200 0x7fc094075710 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc0b0144980 0x7fc0b006d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc0b00cff30 0x7fc0b006ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 --2- 172.21.15.131:0/3001851047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc0b00b65d0 0x7fc0b00aad50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 -- 172.21.15.131:0/3001851047 >> 172.21.15.131:0/3001851047 conn(0x7fc0b0073b10 msgr2=0x7fc0b015dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 -- 172.21.15.131:0/3001851047 shutdown_connections 2024-09-17T02:41:18.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.802+0000 7fc09b7fe640 1 -- 172.21.15.131:0/3001851047 wait complete. 2024-09-17T02:41:18.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.818+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fafb0005860 con 0x7fafcc15e4f0 2024-09-17T02:41:18.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.818+0000 7fafc97fa640 1 -- 172.21.15.131:0/1636429153 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fafc4064290 con 0x7fafcc15e4f0 2024-09-17T02:41:18.818 INFO:teuthology.orchestra.run.smithi131.stdout:64424509452 2024-09-17T02:41:18.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fafac073250 msgr2=0x7fafac075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fafac073250 0x7fafac075760 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fafb40046c0 tx=0x7fafb4007f60 comp rx=0 tx=0).stop 2024-09-17T02:41:18.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f791002a9d0 con 0x7f79100de700 2024-09-17T02:41:18.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafcc15e4f0 msgr2=0x7fafcc153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafcc15e4f0 0x7fafcc153000 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fafc400e990 tx=0x7fafc400ee50 comp rx=0 tx=0).stop 2024-09-17T02:41:18.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 shutdown_connections 2024-09-17T02:41:18.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fafac073250 0x7fafac075760 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fafcc15e4f0 0x7fafcc153000 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fafcc0b4df0 0x7fafcc14c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 --2- 172.21.15.131:0/1636429153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fafcc073b00 0x7fafcc157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 >> 172.21.15.131:0/1636429153 conn(0x7fafcc0d0ad0 msgr2=0x7fafcc0b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 shutdown_connections 2024-09-17T02:41:18.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f7914ff9640 1 -- 172.21.15.131:0/2424625221 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f790c062550 con 0x7f79100de700 2024-09-17T02:41:18.822 INFO:teuthology.orchestra.run.smithi131.stdout:64424509450 2024-09-17T02:41:18.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7fafd27d7640 1 -- 172.21.15.131:0/1636429153 wait complete. 2024-09-17T02:41:18.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f78fc0732d0 msgr2=0x7f78fc0757e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f78fc0732d0 0x7f78fc0757e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7918052910 tx=0x7f7918130040 comp rx=0 tx=0).stop 2024-09-17T02:41:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79100de700 msgr2=0x7f791017c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79100de700 0x7f791017c9d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f791000d630 tx=0x7f790c009800 comp rx=0 tx=0).stop 2024-09-17T02:41:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 shutdown_connections 2024-09-17T02:41:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f78fc0732d0 0x7f78fc0757e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f79100de700 0x7f791017c9d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7910050150 0x7f79100de1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 --2- 172.21.15.131:0/2424625221 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7908003a00 0x7f79100dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 >> 172.21.15.131:0/2424625221 conn(0x7f7910051420 msgr2=0x7f7910049ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 shutdown_connections 2024-09-17T02:41:18.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.822+0000 7f791e532640 1 -- 172.21.15.131:0/2424625221 wait complete. 2024-09-17T02:41:18.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.7 2024-09-17T02:41:18.830 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:18.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.3 2024-09-17T02:41:18.845 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:18.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.4 2024-09-17T02:41:18.846 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:18.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fce5c050e50 con 0x7fce5c0511b0 2024-09-17T02:41:18.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce5affd640 1 -- 172.21.15.131:0/1063799832 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fce640da270 con 0x7fce5c0511b0 2024-09-17T02:41:18.849 INFO:teuthology.orchestra.run.smithi131.stdout:64424509451 2024-09-17T02:41:18.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6a9c005860 con 0x7f6abc072ce0 2024-09-17T02:41:18.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7f6ab97fa640 1 -- 172.21.15.131:0/1640735672 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6ab005de90 con 0x7f6abc072ce0 2024-09-17T02:41:18.850 INFO:teuthology.orchestra.run.smithi131.stdout:64424509452 2024-09-17T02:41:18.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce500733a0 msgr2=0x7fce500758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce500733a0 0x7fce500758b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fce60005480 tx=0x7fce60000bd0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce5c0511b0 msgr2=0x7fce5c0dbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce5c0511b0 0x7fce5c0dbb60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce64052910 tx=0x7fce64076d70 comp rx=0 tx=0).stop 2024-09-17T02:41:18.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 shutdown_connections 2024-09-17T02:41:18.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce500733a0 0x7fce500758b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce5c0511b0 0x7fce5c0dbb60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce5c04edf0 0x7fce5c0db620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 --2- 172.21.15.131:0/1063799832 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce5c033bc0 0x7fce5c0d8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 >> 172.21.15.131:0/1063799832 conn(0x7fce5c01d9b0 msgr2=0x7fce5c0f1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 shutdown_connections 2024-09-17T02:41:18.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.850+0000 7fce6b275640 1 -- 172.21.15.131:0/1063799832 wait complete. 2024-09-17T02:41:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6aa8073280 msgr2=0x7f6aa8075790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6aa8073280 0x7f6aa8075790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ab40af000 tx=0x7f6ab401b5e0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc072ce0 msgr2=0x7f6abc0c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc072ce0 0x7f6abc0c8c40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000b7b0 tx=0x7f6ab000bc70 comp rx=0 tx=0).stop 2024-09-17T02:41:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 shutdown_connections 2024-09-17T02:41:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6aa8073280 0x7f6aa8075790 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6abc0b7880 0x7f6abc0b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6abc15d130 0x7f6abc0b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 --2- 172.21.15.131:0/1640735672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6abc072ce0 0x7f6abc0c8c40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 >> 172.21.15.131:0/1640735672 conn(0x7f6abc0d22f0 msgr2=0x7f6abc0b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 shutdown_connections 2024-09-17T02:41:18.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.854+0000 7f6ac1fa6640 1 -- 172.21.15.131:0/1640735672 wait complete. 2024-09-17T02:41:18.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.858+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f92bc0c6960 con 0x7f92bc069770 2024-09-17T02:41:18.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.862+0000 7f92b1ffb640 1 -- 172.21.15.131:0/2252275688 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f92ac063680 con 0x7f92bc069770 2024-09-17T02:41:18.860 INFO:teuthology.orchestra.run.smithi131.stdout:55834574859 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.862+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f92a0073230 msgr2=0x7f92a0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.862+0000 7f92c250f640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f92a0073230 0x7f92a0075740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0c4c70 tx=0x7f92b80091e0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.862+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92bc069770 msgr2=0x7f92bc2054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.862+0000 7f92c250f640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92bc069770 0x7f92bc2054b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00ea20 tx=0x7f92ac00eee0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.862+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 shutdown_connections 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.862+0000 7f92c250f640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f92a0073230 0x7f92a0075740 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.862+0000 7f92c250f640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f92bc070d50 0x7f92bc073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.862+0000 7f92c250f640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f92bc069b80 0x7f92bc073660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.862+0000 7f92c250f640 1 --2- 172.21.15.131:0/2252275688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f92bc069770 0x7f92bc2054b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.862+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 >> 172.21.15.131:0/2252275688 conn(0x7f92bc0ccfe0 msgr2=0x7f92bc06d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.866+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 shutdown_connections 2024-09-17T02:41:18.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:18.866+0000 7f92c250f640 1 -- 172.21.15.131:0/2252275688 wait complete. 2024-09-17T02:41:18.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.5 2024-09-17T02:41:18.871 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:18.879 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509452 got 64424509452 for osd.0 2024-09-17T02:41:18.879 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:18.885 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-09-17T02:41:18.886 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:18.886 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T02:41:18.886 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:41:19.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.070+0000 7f545683a640 1 -- 172.21.15.131:0/2242749246 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54500c4ac0 msgr2=0x7f54500c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.070+0000 7f545683a640 1 --2- 172.21.15.131:0/2242749246 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54500c4ac0 0x7f54500c4ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5438009840 tx=0x7f543801c790 comp rx=0 tx=0).stop 2024-09-17T02:41:19.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.070+0000 7f545683a640 1 -- 172.21.15.131:0/2242749246 shutdown_connections 2024-09-17T02:41:19.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.070+0000 7f545683a640 1 --2- 172.21.15.131:0/2242749246 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54500c4ac0 0x7f54500c4ed0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.070+0000 7f545683a640 1 --2- 172.21.15.131:0/2242749246 >> v2:172.21.15.189:3300/0 conn(0x7f545006e2c0 0x7f54500c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.070+0000 7f545683a640 1 -- 172.21.15.131:0/2242749246 >> 172.21.15.131:0/2242749246 conn(0x7f545006eea0 msgr2=0x7f545006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:19.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.070+0000 7f545683a640 1 -- 172.21.15.131:0/2242749246 shutdown_connections 2024-09-17T02:41:19.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.070+0000 7f545683a640 1 -- 172.21.15.131:0/2242749246 wait complete. 2024-09-17T02:41:19.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f545683a640 1 Processor -- start 2024-09-17T02:41:19.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f545683a640 1 -- start start 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f545683a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f545006e2c0 0x7f5450157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f545683a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54500c4ac0 0x7f54500c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f545683a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54500cdbf0 0x7f54500b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f545683a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54500cf0a0 con 0x7f54500c4ac0 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f545683a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54500cef20 con 0x7f545006e2c0 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f545683a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f54500cf220 con 0x7f54500cdbf0 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f54477fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54500c4ac0 0x7f54500c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f54477fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54500c4ac0 0x7f54500c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58112/0 (socket says 172.21.15.131:58112) 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f54477fe640 1 -- 172.21.15.131:0/772410937 learned_addr learned my addr 172.21.15.131:0/772410937 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f5454dd7640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54500cdbf0 0x7f54500b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f544ffff640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f545006e2c0 0x7f5450157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f54477fe640 1 -- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54500cdbf0 msgr2=0x7f54500b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f54477fe640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54500cdbf0 0x7f54500b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f54477fe640 1 -- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f545006e2c0 msgr2=0x7f5450157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f54477fe640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f545006e2c0 0x7f5450157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f54477fe640 1 -- 172.21.15.131:0/772410937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5450158980 con 0x7f54500c4ac0 2024-09-17T02:41:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f54477fe640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54500c4ac0 0x7f54500c90a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5438009790 tx=0x7f54380a6520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f544d7fa640 1 -- 172.21.15.131:0/772410937 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380ac030 con 0x7f54500c4ac0 2024-09-17T02:41:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f545683a640 1 -- 172.21.15.131:0/772410937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54501583c0 con 0x7f54500c4ac0 2024-09-17T02:41:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f544d7fa640 1 -- 172.21.15.131:0/772410937 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5438005430 con 0x7f54500c4ac0 2024-09-17T02:41:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f544d7fa640 1 -- 172.21.15.131:0/772410937 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380046d0 con 0x7f54500c4ac0 2024-09-17T02:41:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f545683a640 1 -- 172.21.15.131:0/772410937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f545014d890 con 0x7f54500c4ac0 2024-09-17T02:41:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.074+0000 7f545683a640 1 -- 172.21.15.131:0/772410937 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54500c89c0 con 0x7f54500c4ac0 2024-09-17T02:41:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.078+0000 7f544d7fa640 1 -- 172.21.15.131:0/772410937 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f54380af070 con 0x7f54500c4ac0 2024-09-17T02:41:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.078+0000 7f544d7fa640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5430073200 0x7f5430075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.078+0000 7f544d7fa640 1 -- 172.21.15.131:0/772410937 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f543812ebd0 con 0x7f54500c4ac0 2024-09-17T02:41:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.078+0000 7f544ffff640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5430073200 0x7f5430075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.078+0000 7f544ffff640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5430073200 0x7f5430075710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5440009a10 tx=0x7f54400091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:19.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.082+0000 7f544d7fa640 1 -- 172.21.15.131:0/772410937 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5438016410 con 0x7f54500c4ac0 2024-09-17T02:41:19.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.238+0000 7f545683a640 1 -- 172.21.15.131:0/772410937 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54500c4580 con 0x7f5430073200 2024-09-17T02:41:19.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.262+0000 7f544d7fa640 1 -- 172.21.15.131:0/772410937 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332516 (secure 0 0 0) 0x7f54500c4580 con 0x7f5430073200 2024-09-17T02:41:19.266 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:19.268 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:41:19.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 -- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5430073200 msgr2=0x7f5430075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5430073200 0x7f5430075710 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5440009a10 tx=0x7f54400091e0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 -- 172.21.15.131:0/772410937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54500c4ac0 msgr2=0x7f54500c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54500c4ac0 0x7f54500c90a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5438009790 tx=0x7f54380a6520 comp rx=0 tx=0).stop 2024-09-17T02:41:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 -- 172.21.15.131:0/772410937 shutdown_connections 2024-09-17T02:41:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5430073200 0x7f5430075710 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54500cdbf0 0x7f54500b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54500c4ac0 0x7f54500c90a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 --2- 172.21.15.131:0/772410937 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f545006e2c0 0x7f5450157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 -- 172.21.15.131:0/772410937 >> 172.21.15.131:0/772410937 conn(0x7f545006eea0 msgr2=0x7f54500b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 -- 172.21.15.131:0/772410937 shutdown_connections 2024-09-17T02:41:19.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.266+0000 7f545683a640 1 -- 172.21.15.131:0/772410937 wait complete. 2024-09-17T02:41:19.285 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-17T02:41:19.125970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219564,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755156,"statfs":{"total":773094113280,"available":772869279744,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006109"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290027+0000","last_change":"2024-09-17T02:40:51.241640+0000","last_active":"2024-09-17T02:40:57.290027+0000","last_peered":"2024-09-17T02:40:57.290027+0000","last_clean":"2024-09-17T02:40:57.290027+0000","last_became_active":"2024-09-17T02:40:51.240675+0000","last_became_peered":"2024-09-17T02:40:51.240675+0000","last_unstale":"2024-09-17T02:40:57.290027+0000","last_undegraded":"2024-09-17T02:40:57.290027+0000","last_fullsized":"2024-09-17T02:40:57.290027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475107+0000","last_change":"2024-09-17T02:40:52.228237+0000","last_active":"2024-09-17T02:40:57.475107+0000","last_peered":"2024-09-17T02:40:57.475107+0000","last_clean":"2024-09-17T02:40:57.475107+0000","last_became_active":"2024-09-17T02:40:52.225329+0000","last_became_peered":"2024-09-17T02:40:52.225329+0000","last_unstale":"2024-09-17T02:40:57.475107+0000","last_undegraded":"2024-09-17T02:40:57.475107+0000","last_fullsized":"2024-09-17T02:40:57.475107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:55.015415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471940+0000","last_change":"2024-09-17T02:40:56.353097+0000","last_active":"2024-09-17T02:40:57.471940+0000","last_peered":"2024-09-17T02:40:57.471940+0000","last_clean":"2024-09-17T02:40:57.471940+0000","last_became_active":"2024-09-17T02:40:56.351847+0000","last_became_peered":"2024-09-17T02:40:56.351847+0000","last_unstale":"2024-09-17T02:40:57.471940+0000","last_undegraded":"2024-09-17T02:40:57.471940+0000","last_fullsized":"2024-09-17T02:40:57.471940+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:06.760578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475261+0000","last_change":"2024-09-17T02:40:51.243590+0000","last_active":"2024-09-17T02:40:57.475261+0000","last_peered":"2024-09-17T02:40:57.475261+0000","last_clean":"2024-09-17T02:40:57.475261+0000","last_became_active":"2024-09-17T02:40:51.242684+0000","last_became_peered":"2024-09-17T02:40:51.242684+0000","last_unstale":"2024-09-17T02:40:57.475261+0000","last_undegraded":"2024-09-17T02:40:57.475261+0000","last_fullsized":"2024-09-17T02:40:57.475261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474246+0000","last_change":"2024-09-17T02:40:52.207992+0000","last_active":"2024-09-17T02:40:57.474246+0000","last_peered":"2024-09-17T02:40:57.474246+0000","last_clean":"2024-09-17T02:40:57.474246+0000","last_became_active":"2024-09-17T02:40:52.207568+0000","last_became_peered":"2024-09-17T02:40:52.207568+0000","last_unstale":"2024-09-17T02:40:57.474246+0000","last_undegraded":"2024-09-17T02:40:57.474246+0000","last_fullsized":"2024-09-17T02:40:57.474246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:01.844374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470155+0000","last_change":"2024-09-17T02:40:56.339588+0000","last_active":"2024-09-17T02:40:57.470155+0000","last_peered":"2024-09-17T02:40:57.470155+0000","last_clean":"2024-09-17T02:40:57.470155+0000","last_became_active":"2024-09-17T02:40:56.338569+0000","last_became_peered":"2024-09-17T02:40:56.338569+0000","last_unstale":"2024-09-17T02:40:57.470155+0000","last_undegraded":"2024-09-17T02:40:57.470155+0000","last_fullsized":"2024-09-17T02:40:57.470155+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:54.995255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472080+0000","last_change":"2024-09-17T02:40:51.225527+0000","last_active":"2024-09-17T02:40:57.472080+0000","last_peered":"2024-09-17T02:40:57.472080+0000","last_clean":"2024-09-17T02:40:57.472080+0000","last_became_active":"2024-09-17T02:40:51.224640+0000","last_became_peered":"2024-09-17T02:40:51.224640+0000","last_unstale":"2024-09-17T02:40:57.472080+0000","last_undegraded":"2024-09-17T02:40:57.472080+0000","last_fullsized":"2024-09-17T02:40:57.472080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:07.028967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470812+0000","last_change":"2024-09-17T02:40:52.243749+0000","last_active":"2024-09-17T02:40:57.470812+0000","last_peered":"2024-09-17T02:40:57.470812+0000","last_clean":"2024-09-17T02:40:57.470812+0000","last_became_active":"2024-09-17T02:40:52.243361+0000","last_became_peered":"2024-09-17T02:40:52.243361+0000","last_unstale":"2024-09-17T02:40:57.470812+0000","last_undegraded":"2024-09-17T02:40:57.470812+0000","last_fullsized":"2024-09-17T02:40:57.470812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:01.471406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471331+0000","last_change":"2024-09-17T02:40:56.330627+0000","last_active":"2024-09-17T02:40:57.471331+0000","last_peered":"2024-09-17T02:40:57.471331+0000","last_clean":"2024-09-17T02:40:57.471331+0000","last_became_active":"2024-09-17T02:40:56.330123+0000","last_became_peered":"2024-09-17T02:40:56.330123+0000","last_unstale":"2024-09-17T02:40:57.471331+0000","last_undegraded":"2024-09-17T02:40:57.471331+0000","last_fullsized":"2024-09-17T02:40:57.471331+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:30.712481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472462+0000","last_change":"2024-09-17T02:40:51.220105+0000","last_active":"2024-09-17T02:40:57.472462+0000","last_peered":"2024-09-17T02:40:57.472462+0000","last_clean":"2024-09-17T02:40:57.472462+0000","last_became_active":"2024-09-17T02:40:51.216247+0000","last_became_peered":"2024-09-17T02:40:51.216247+0000","last_unstale":"2024-09-17T02:40:57.472462+0000","last_undegraded":"2024-09-17T02:40:57.472462+0000","last_fullsized":"2024-09-17T02:40:57.472462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:30.096213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288051+0000","last_change":"2024-09-17T02:40:52.255870+0000","last_active":"2024-09-17T02:40:57.288051+0000","last_peered":"2024-09-17T02:40:57.288051+0000","last_clean":"2024-09-17T02:40:57.288051+0000","last_became_active":"2024-09-17T02:40:52.255454+0000","last_became_peered":"2024-09-17T02:40:52.255454+0000","last_unstale":"2024-09-17T02:40:57.288051+0000","last_undegraded":"2024-09-17T02:40:57.288051+0000","last_fullsized":"2024-09-17T02:40:57.288051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:35.834666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470627+0000","last_change":"2024-09-17T02:40:56.339659+0000","last_active":"2024-09-17T02:40:57.470627+0000","last_peered":"2024-09-17T02:40:57.470627+0000","last_clean":"2024-09-17T02:40:57.470627+0000","last_became_active":"2024-09-17T02:40:56.339130+0000","last_became_peered":"2024-09-17T02:40:56.339130+0000","last_unstale":"2024-09-17T02:40:57.470627+0000","last_undegraded":"2024-09-17T02:40:57.470627+0000","last_fullsized":"2024-09-17T02:40:57.470627+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:38.223653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287802+0000","last_change":"2024-09-17T02:40:52.242899+0000","last_active":"2024-09-17T02:40:57.287802+0000","last_peered":"2024-09-17T02:40:57.287802+0000","last_clean":"2024-09-17T02:40:57.287802+0000","last_became_active":"2024-09-17T02:40:52.242570+0000","last_became_peered":"2024-09-17T02:40:52.242570+0000","last_unstale":"2024-09-17T02:40:57.287802+0000","last_undegraded":"2024-09-17T02:40:57.287802+0000","last_fullsized":"2024-09-17T02:40:57.287802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:45.600475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288528+0000","last_change":"2024-09-17T02:40:51.212048+0000","last_active":"2024-09-17T02:40:57.288528+0000","last_peered":"2024-09-17T02:40:57.288528+0000","last_clean":"2024-09-17T02:40:57.288528+0000","last_became_active":"2024-09-17T02:40:51.211531+0000","last_became_peered":"2024-09-17T02:40:51.211531+0000","last_unstale":"2024-09-17T02:40:57.288528+0000","last_undegraded":"2024-09-17T02:40:57.288528+0000","last_fullsized":"2024-09-17T02:40:57.288528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:10.284709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287732+0000","last_change":"2024-09-17T02:40:56.332707+0000","last_active":"2024-09-17T02:40:57.287732+0000","last_peered":"2024-09-17T02:40:57.287732+0000","last_clean":"2024-09-17T02:40:57.287732+0000","last_became_active":"2024-09-17T02:40:56.332260+0000","last_became_peered":"2024-09-17T02:40:56.332260+0000","last_unstale":"2024-09-17T02:40:57.287732+0000","last_undegraded":"2024-09-17T02:40:57.287732+0000","last_fullsized":"2024-09-17T02:40:57.287732+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:53.783346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472671+0000","last_change":"2024-09-17T02:40:52.218705+0000","last_active":"2024-09-17T02:40:57.472671+0000","last_peered":"2024-09-17T02:40:57.472671+0000","last_clean":"2024-09-17T02:40:57.472671+0000","last_became_active":"2024-09-17T02:40:52.217762+0000","last_became_peered":"2024-09-17T02:40:52.217762+0000","last_unstale":"2024-09-17T02:40:57.472671+0000","last_undegraded":"2024-09-17T02:40:57.472671+0000","last_fullsized":"2024-09-17T02:40:57.472671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:11.295603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287768+0000","last_change":"2024-09-17T02:40:51.237760+0000","last_active":"2024-09-17T02:40:57.287768+0000","last_peered":"2024-09-17T02:40:57.287768+0000","last_clean":"2024-09-17T02:40:57.287768+0000","last_became_active":"2024-09-17T02:40:51.236973+0000","last_became_peered":"2024-09-17T02:40:51.236973+0000","last_unstale":"2024-09-17T02:40:57.287768+0000","last_undegraded":"2024-09-17T02:40:57.287768+0000","last_fullsized":"2024-09-17T02:40:57.287768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:22.634046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471688+0000","last_change":"2024-09-17T02:40:56.355680+0000","last_active":"2024-09-17T02:40:57.471688+0000","last_peered":"2024-09-17T02:40:57.471688+0000","last_clean":"2024-09-17T02:40:57.471688+0000","last_became_active":"2024-09-17T02:40:56.355224+0000","last_became_peered":"2024-09-17T02:40:56.355224+0000","last_unstale":"2024-09-17T02:40:57.471688+0000","last_undegraded":"2024-09-17T02:40:57.471688+0000","last_fullsized":"2024-09-17T02:40:57.471688+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:49.679484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290367+0000","last_change":"2024-09-17T02:40:52.220376+0000","last_active":"2024-09-17T02:40:57.290367+0000","last_peered":"2024-09-17T02:40:57.290367+0000","last_clean":"2024-09-17T02:40:57.290367+0000","last_became_active":"2024-09-17T02:40:52.220031+0000","last_became_peered":"2024-09-17T02:40:52.220031+0000","last_unstale":"2024-09-17T02:40:57.290367+0000","last_undegraded":"2024-09-17T02:40:57.290367+0000","last_fullsized":"2024-09-17T02:40:57.290367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:04.162473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291288+0000","last_change":"2024-09-17T02:40:51.240925+0000","last_active":"2024-09-17T02:40:57.291288+0000","last_peered":"2024-09-17T02:40:57.291288+0000","last_clean":"2024-09-17T02:40:57.291288+0000","last_became_active":"2024-09-17T02:40:51.233638+0000","last_became_peered":"2024-09-17T02:40:51.233638+0000","last_unstale":"2024-09-17T02:40:57.291288+0000","last_undegraded":"2024-09-17T02:40:57.291288+0000","last_fullsized":"2024-09-17T02:40:57.291288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:39.387068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475651+0000","last_change":"2024-09-17T02:40:56.339397+0000","last_active":"2024-09-17T02:40:57.475651+0000","last_peered":"2024-09-17T02:40:57.475651+0000","last_clean":"2024-09-17T02:40:57.475651+0000","last_became_active":"2024-09-17T02:40:56.338591+0000","last_became_peered":"2024-09-17T02:40:56.338591+0000","last_unstale":"2024-09-17T02:40:57.475651+0000","last_undegraded":"2024-09-17T02:40:57.475651+0000","last_fullsized":"2024-09-17T02:40:57.475651+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:10.362765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287431+0000","last_change":"2024-09-17T02:40:52.245352+0000","last_active":"2024-09-17T02:40:57.287431+0000","last_peered":"2024-09-17T02:40:57.287431+0000","last_clean":"2024-09-17T02:40:57.287431+0000","last_became_active":"2024-09-17T02:40:52.244938+0000","last_became_peered":"2024-09-17T02:40:52.244938+0000","last_unstale":"2024-09-17T02:40:57.287431+0000","last_undegraded":"2024-09-17T02:40:57.287431+0000","last_fullsized":"2024-09-17T02:40:57.287431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:07.474908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.469729+0000","last_change":"2024-09-17T02:40:51.220831+0000","last_active":"2024-09-17T02:40:57.469729+0000","last_peered":"2024-09-17T02:40:57.469729+0000","last_clean":"2024-09-17T02:40:57.469729+0000","last_became_active":"2024-09-17T02:40:51.220253+0000","last_became_peered":"2024-09-17T02:40:51.220253+0000","last_unstale":"2024-09-17T02:40:57.469729+0000","last_undegraded":"2024-09-17T02:40:57.469729+0000","last_fullsized":"2024-09-17T02:40:57.469729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471876+0000","last_change":"2024-09-17T02:40:56.333001+0000","last_active":"2024-09-17T02:40:57.471876+0000","last_peered":"2024-09-17T02:40:57.471876+0000","last_clean":"2024-09-17T02:40:57.471876+0000","last_became_active":"2024-09-17T02:40:56.332421+0000","last_became_peered":"2024-09-17T02:40:56.332421+0000","last_unstale":"2024-09-17T02:40:57.471876+0000","last_undegraded":"2024-09-17T02:40:57.471876+0000","last_fullsized":"2024-09-17T02:40:57.471876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:58.168471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291218+0000","last_change":"2024-09-17T02:40:52.215438+0000","last_active":"2024-09-17T02:40:57.291218+0000","last_peered":"2024-09-17T02:40:57.291218+0000","last_clean":"2024-09-17T02:40:57.291218+0000","last_became_active":"2024-09-17T02:40:52.215073+0000","last_became_peered":"2024-09-17T02:40:52.215073+0000","last_unstale":"2024-09-17T02:40:57.291218+0000","last_undegraded":"2024-09-17T02:40:57.291218+0000","last_fullsized":"2024-09-17T02:40:57.291218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:39.361971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288442+0000","last_change":"2024-09-17T02:40:51.226075+0000","last_active":"2024-09-17T02:40:57.288442+0000","last_peered":"2024-09-17T02:40:57.288442+0000","last_clean":"2024-09-17T02:40:57.288442+0000","last_became_active":"2024-09-17T02:40:51.225620+0000","last_became_peered":"2024-09-17T02:40:51.225620+0000","last_unstale":"2024-09-17T02:40:57.288442+0000","last_undegraded":"2024-09-17T02:40:57.288442+0000","last_fullsized":"2024-09-17T02:40:57.288442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:15.238526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470178+0000","last_change":"2024-09-17T02:40:56.306280+0000","last_active":"2024-09-17T02:40:57.470178+0000","last_peered":"2024-09-17T02:40:57.470178+0000","last_clean":"2024-09-17T02:40:57.470178+0000","last_became_active":"2024-09-17T02:40:56.305348+0000","last_became_peered":"2024-09-17T02:40:56.305348+0000","last_unstale":"2024-09-17T02:40:57.470178+0000","last_undegraded":"2024-09-17T02:40:57.470178+0000","last_fullsized":"2024-09-17T02:40:57.470178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:58.295015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289748+0000","last_change":"2024-09-17T02:40:52.217580+0000","last_active":"2024-09-17T02:40:57.289748+0000","last_peered":"2024-09-17T02:40:57.289748+0000","last_clean":"2024-09-17T02:40:57.289748+0000","last_became_active":"2024-09-17T02:40:52.217246+0000","last_became_peered":"2024-09-17T02:40:52.217246+0000","last_unstale":"2024-09-17T02:40:57.289748+0000","last_undegraded":"2024-09-17T02:40:57.289748+0000","last_fullsized":"2024-09-17T02:40:57.289748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:25.696012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291979+0000","last_change":"2024-09-17T02:40:51.235278+0000","last_active":"2024-09-17T02:40:57.291979+0000","last_peered":"2024-09-17T02:40:57.291979+0000","last_clean":"2024-09-17T02:40:57.291979+0000","last_became_active":"2024-09-17T02:40:51.234012+0000","last_became_peered":"2024-09-17T02:40:51.234012+0000","last_unstale":"2024-09-17T02:40:57.291979+0000","last_undegraded":"2024-09-17T02:40:57.291979+0000","last_fullsized":"2024-09-17T02:40:57.291979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:07.028967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290548+0000","last_change":"2024-09-17T02:40:56.306563+0000","last_active":"2024-09-17T02:40:57.290548+0000","last_peered":"2024-09-17T02:40:57.290548+0000","last_clean":"2024-09-17T02:40:57.290548+0000","last_became_active":"2024-09-17T02:40:56.303819+0000","last_became_peered":"2024-09-17T02:40:56.303819+0000","last_unstale":"2024-09-17T02:40:57.290548+0000","last_undegraded":"2024-09-17T02:40:57.290548+0000","last_fullsized":"2024-09-17T02:40:57.290548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:04.387188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289492+0000","last_change":"2024-09-17T02:40:52.216405+0000","last_active":"2024-09-17T02:40:57.289492+0000","last_peered":"2024-09-17T02:40:57.289492+0000","last_clean":"2024-09-17T02:40:57.289492+0000","last_became_active":"2024-09-17T02:40:52.216037+0000","last_became_peered":"2024-09-17T02:40:52.216037+0000","last_unstale":"2024-09-17T02:40:57.289492+0000","last_undegraded":"2024-09-17T02:40:57.289492+0000","last_fullsized":"2024-09-17T02:40:57.289492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:20.565582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288509+0000","last_change":"2024-09-17T02:40:51.243903+0000","last_active":"2024-09-17T02:40:57.288509+0000","last_peered":"2024-09-17T02:40:57.288509+0000","last_clean":"2024-09-17T02:40:57.288509+0000","last_became_active":"2024-09-17T02:40:51.243568+0000","last_became_peered":"2024-09-17T02:40:51.243568+0000","last_unstale":"2024-09-17T02:40:57.288509+0000","last_undegraded":"2024-09-17T02:40:57.288509+0000","last_fullsized":"2024-09-17T02:40:57.288509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:48:49.643024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287722+0000","last_change":"2024-09-17T02:40:56.340894+0000","last_active":"2024-09-17T02:40:57.287722+0000","last_peered":"2024-09-17T02:40:57.287722+0000","last_clean":"2024-09-17T02:40:57.287722+0000","last_became_active":"2024-09-17T02:40:56.338947+0000","last_became_peered":"2024-09-17T02:40:56.338947+0000","last_unstale":"2024-09-17T02:40:57.287722+0000","last_undegraded":"2024-09-17T02:40:57.287722+0000","last_fullsized":"2024-09-17T02:40:57.287722+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:48.414928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472018+0000","last_change":"2024-09-17T02:40:52.232754+0000","last_active":"2024-09-17T02:40:57.472018+0000","last_peered":"2024-09-17T02:40:57.472018+0000","last_clean":"2024-09-17T02:40:57.472018+0000","last_became_active":"2024-09-17T02:40:52.231727+0000","last_became_peered":"2024-09-17T02:40:52.231727+0000","last_unstale":"2024-09-17T02:40:57.472018+0000","last_undegraded":"2024-09-17T02:40:57.472018+0000","last_fullsized":"2024-09-17T02:40:57.472018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:11.295603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473934+0000","last_change":"2024-09-17T02:40:51.237795+0000","last_active":"2024-09-17T02:40:57.473934+0000","last_peered":"2024-09-17T02:40:57.473934+0000","last_clean":"2024-09-17T02:40:57.473934+0000","last_became_active":"2024-09-17T02:40:51.236960+0000","last_became_peered":"2024-09-17T02:40:51.236960+0000","last_unstale":"2024-09-17T02:40:57.473934+0000","last_undegraded":"2024-09-17T02:40:57.473934+0000","last_fullsized":"2024-09-17T02:40:57.473934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:39.387068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287516+0000","last_change":"2024-09-17T02:40:56.340920+0000","last_active":"2024-09-17T02:40:57.287516+0000","last_peered":"2024-09-17T02:40:57.287516+0000","last_clean":"2024-09-17T02:40:57.287516+0000","last_became_active":"2024-09-17T02:40:56.339463+0000","last_became_peered":"2024-09-17T02:40:56.339463+0000","last_unstale":"2024-09-17T02:40:57.287516+0000","last_undegraded":"2024-09-17T02:40:57.287516+0000","last_fullsized":"2024-09-17T02:40:57.287516+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:36.386068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472660+0000","last_change":"2024-09-17T02:40:52.234607+0000","last_active":"2024-09-17T02:40:57.472660+0000","last_peered":"2024-09-17T02:40:57.472660+0000","last_clean":"2024-09-17T02:40:57.472660+0000","last_became_active":"2024-09-17T02:40:52.234175+0000","last_became_peered":"2024-09-17T02:40:52.234175+0000","last_unstale":"2024-09-17T02:40:57.472660+0000","last_undegraded":"2024-09-17T02:40:57.472660+0000","last_fullsized":"2024-09-17T02:40:57.472660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:34.144644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288850+0000","last_change":"2024-09-17T02:40:51.221611+0000","last_active":"2024-09-17T02:40:57.288850+0000","last_peered":"2024-09-17T02:40:57.288850+0000","last_clean":"2024-09-17T02:40:57.288850+0000","last_became_active":"2024-09-17T02:40:51.221259+0000","last_became_peered":"2024-09-17T02:40:51.221259+0000","last_unstale":"2024-09-17T02:40:57.288850+0000","last_undegraded":"2024-09-17T02:40:57.288850+0000","last_fullsized":"2024-09-17T02:40:57.288850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:20.572163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289392+0000","last_change":"2024-09-17T02:40:56.350014+0000","last_active":"2024-09-17T02:40:57.289392+0000","last_peered":"2024-09-17T02:40:57.289392+0000","last_clean":"2024-09-17T02:40:57.289392+0000","last_became_active":"2024-09-17T02:40:56.347244+0000","last_became_peered":"2024-09-17T02:40:56.347244+0000","last_unstale":"2024-09-17T02:40:57.289392+0000","last_undegraded":"2024-09-17T02:40:57.289392+0000","last_fullsized":"2024-09-17T02:40:57.289392+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:10.648378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471945+0000","last_change":"2024-09-17T02:40:52.221771+0000","last_active":"2024-09-17T02:40:57.471945+0000","last_peered":"2024-09-17T02:40:57.471945+0000","last_clean":"2024-09-17T02:40:57.471945+0000","last_became_active":"2024-09-17T02:40:52.221461+0000","last_became_peered":"2024-09-17T02:40:52.221461+0000","last_unstale":"2024-09-17T02:40:57.471945+0000","last_undegraded":"2024-09-17T02:40:57.471945+0000","last_fullsized":"2024-09-17T02:40:57.471945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:25.696012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472767+0000","last_change":"2024-09-17T02:40:51.247013+0000","last_active":"2024-09-17T02:40:57.472767+0000","last_peered":"2024-09-17T02:40:57.472767+0000","last_clean":"2024-09-17T02:40:57.472767+0000","last_became_active":"2024-09-17T02:40:51.246163+0000","last_became_peered":"2024-09-17T02:40:51.246163+0000","last_unstale":"2024-09-17T02:40:57.472767+0000","last_undegraded":"2024-09-17T02:40:57.472767+0000","last_fullsized":"2024-09-17T02:40:57.472767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:05.797393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288261+0000","last_change":"2024-09-17T02:40:56.340614+0000","last_active":"2024-09-17T02:40:57.288261+0000","last_peered":"2024-09-17T02:40:57.288261+0000","last_clean":"2024-09-17T02:40:57.288261+0000","last_became_active":"2024-09-17T02:40:56.338340+0000","last_became_peered":"2024-09-17T02:40:56.338340+0000","last_unstale":"2024-09-17T02:40:57.288261+0000","last_undegraded":"2024-09-17T02:40:57.288261+0000","last_fullsized":"2024-09-17T02:40:57.288261+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:03.442300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.286936+0000","last_change":"2024-09-17T02:40:52.227802+0000","last_active":"2024-09-17T02:40:57.286936+0000","last_peered":"2024-09-17T02:40:57.286936+0000","last_clean":"2024-09-17T02:40:57.286936+0000","last_became_active":"2024-09-17T02:40:52.227458+0000","last_became_peered":"2024-09-17T02:40:52.227458+0000","last_unstale":"2024-09-17T02:40:57.286936+0000","last_undegraded":"2024-09-17T02:40:57.286936+0000","last_fullsized":"2024-09-17T02:40:57.286936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:21.198509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287577+0000","last_change":"2024-09-17T02:40:51.239502+0000","last_active":"2024-09-17T02:40:57.287577+0000","last_peered":"2024-09-17T02:40:57.287577+0000","last_clean":"2024-09-17T02:40:57.287577+0000","last_became_active":"2024-09-17T02:40:51.239042+0000","last_became_peered":"2024-09-17T02:40:51.239042+0000","last_unstale":"2024-09-17T02:40:57.287577+0000","last_undegraded":"2024-09-17T02:40:57.287577+0000","last_fullsized":"2024-09-17T02:40:57.287577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:04.995836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474601+0000","last_change":"2024-09-17T02:40:56.353034+0000","last_active":"2024-09-17T02:40:57.474601+0000","last_peered":"2024-09-17T02:40:57.474601+0000","last_clean":"2024-09-17T02:40:57.474601+0000","last_became_active":"2024-09-17T02:40:56.351920+0000","last_became_peered":"2024-09-17T02:40:56.351920+0000","last_unstale":"2024-09-17T02:40:57.474601+0000","last_undegraded":"2024-09-17T02:40:57.474601+0000","last_fullsized":"2024-09-17T02:40:57.474601+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:39.913675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289211+0000","last_change":"2024-09-17T02:40:52.223050+0000","last_active":"2024-09-17T02:40:57.289211+0000","last_peered":"2024-09-17T02:40:57.289211+0000","last_clean":"2024-09-17T02:40:57.289211+0000","last_became_active":"2024-09-17T02:40:52.221680+0000","last_became_peered":"2024-09-17T02:40:52.221680+0000","last_unstale":"2024-09-17T02:40:57.289211+0000","last_undegraded":"2024-09-17T02:40:57.289211+0000","last_fullsized":"2024-09-17T02:40:57.289211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:06.283756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288675+0000","last_change":"2024-09-17T02:40:51.209280+0000","last_active":"2024-09-17T02:40:57.288675+0000","last_peered":"2024-09-17T02:40:57.288675+0000","last_clean":"2024-09-17T02:40:57.288675+0000","last_became_active":"2024-09-17T02:40:51.208951+0000","last_became_peered":"2024-09-17T02:40:51.208951+0000","last_unstale":"2024-09-17T02:40:57.288675+0000","last_undegraded":"2024-09-17T02:40:57.288675+0000","last_fullsized":"2024-09-17T02:40:57.288675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:51.093529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470279+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:40:57.470279+0000","last_peered":"2024-09-17T02:40:57.470279+0000","last_clean":"2024-09-17T02:40:57.470279+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:40:57.470279+0000","last_undegraded":"2024-09-17T02:40:57.470279+0000","last_fullsized":"2024-09-17T02:40:57.470279+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288416+0000","last_change":"2024-09-17T02:40:56.306099+0000","last_active":"2024-09-17T02:40:57.288416+0000","last_peered":"2024-09-17T02:40:57.288416+0000","last_clean":"2024-09-17T02:40:57.288416+0000","last_became_active":"2024-09-17T02:40:56.305604+0000","last_became_peered":"2024-09-17T02:40:56.305604+0000","last_unstale":"2024-09-17T02:40:57.288416+0000","last_undegraded":"2024-09-17T02:40:57.288416+0000","last_fullsized":"2024-09-17T02:40:57.288416+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:22.632347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288182+0000","last_change":"2024-09-17T02:40:52.255845+0000","last_active":"2024-09-17T02:40:57.288182+0000","last_peered":"2024-09-17T02:40:57.288182+0000","last_clean":"2024-09-17T02:40:57.288182+0000","last_became_active":"2024-09-17T02:40:52.255416+0000","last_became_peered":"2024-09-17T02:40:52.255416+0000","last_unstale":"2024-09-17T02:40:57.288182+0000","last_undegraded":"2024-09-17T02:40:57.288182+0000","last_fullsized":"2024-09-17T02:40:57.288182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:40.879599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289502+0000","last_change":"2024-09-17T02:40:51.230583+0000","last_active":"2024-09-17T02:40:57.289502+0000","last_peered":"2024-09-17T02:40:57.289502+0000","last_clean":"2024-09-17T02:40:57.289502+0000","last_became_active":"2024-09-17T02:40:51.229636+0000","last_became_peered":"2024-09-17T02:40:51.229636+0000","last_unstale":"2024-09-17T02:40:57.289502+0000","last_undegraded":"2024-09-17T02:40:57.289502+0000","last_fullsized":"2024-09-17T02:40:57.289502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:25.403066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470871+0000","last_change":"2024-09-17T02:40:56.334178+0000","last_active":"2024-09-17T02:40:57.470871+0000","last_peered":"2024-09-17T02:40:57.470871+0000","last_clean":"2024-09-17T02:40:57.470871+0000","last_became_active":"2024-09-17T02:40:56.333520+0000","last_became_peered":"2024-09-17T02:40:56.333520+0000","last_unstale":"2024-09-17T02:40:57.470871+0000","last_undegraded":"2024-09-17T02:40:57.470871+0000","last_fullsized":"2024-09-17T02:40:57.470871+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:38.037563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472395+0000","last_change":"2024-09-17T02:40:52.237704+0000","last_active":"2024-09-17T02:40:57.472395+0000","last_peered":"2024-09-17T02:40:57.472395+0000","last_clean":"2024-09-17T02:40:57.472395+0000","last_became_active":"2024-09-17T02:40:52.235768+0000","last_became_peered":"2024-09-17T02:40:52.235768+0000","last_unstale":"2024-09-17T02:40:57.472395+0000","last_undegraded":"2024-09-17T02:40:57.472395+0000","last_fullsized":"2024-09-17T02:40:57.472395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:04.549252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473130+0000","last_change":"2024-09-17T02:40:51.227571+0000","last_active":"2024-09-17T02:40:57.473130+0000","last_peered":"2024-09-17T02:40:57.473130+0000","last_clean":"2024-09-17T02:40:57.473130+0000","last_became_active":"2024-09-17T02:40:51.226748+0000","last_became_peered":"2024-09-17T02:40:51.226748+0000","last_unstale":"2024-09-17T02:40:57.473130+0000","last_undegraded":"2024-09-17T02:40:57.473130+0000","last_fullsized":"2024-09-17T02:40:57.473130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:12.330693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473168+0000","last_change":"2024-09-17T02:40:56.327712+0000","last_active":"2024-09-17T02:40:57.473168+0000","last_peered":"2024-09-17T02:40:57.473168+0000","last_clean":"2024-09-17T02:40:57.473168+0000","last_became_active":"2024-09-17T02:40:56.326836+0000","last_became_peered":"2024-09-17T02:40:56.326836+0000","last_unstale":"2024-09-17T02:40:57.473168+0000","last_undegraded":"2024-09-17T02:40:57.473168+0000","last_fullsized":"2024-09-17T02:40:57.473168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:18.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471966+0000","last_change":"2024-09-17T02:40:52.239267+0000","last_active":"2024-09-17T02:40:57.471966+0000","last_peered":"2024-09-17T02:40:57.471966+0000","last_clean":"2024-09-17T02:40:57.471966+0000","last_became_active":"2024-09-17T02:40:52.238707+0000","last_became_peered":"2024-09-17T02:40:52.238707+0000","last_unstale":"2024-09-17T02:40:57.471966+0000","last_undegraded":"2024-09-17T02:40:57.471966+0000","last_fullsized":"2024-09-17T02:40:57.471966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:07.999419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291556+0000","last_change":"2024-09-17T02:40:51.249182+0000","last_active":"2024-09-17T02:40:57.291556+0000","last_peered":"2024-09-17T02:40:57.291556+0000","last_clean":"2024-09-17T02:40:57.291556+0000","last_became_active":"2024-09-17T02:40:51.248480+0000","last_became_peered":"2024-09-17T02:40:51.248480+0000","last_unstale":"2024-09-17T02:40:57.291556+0000","last_undegraded":"2024-09-17T02:40:57.291556+0000","last_fullsized":"2024-09-17T02:40:57.291556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287153+0000","last_change":"2024-09-17T02:40:56.306150+0000","last_active":"2024-09-17T02:40:57.287153+0000","last_peered":"2024-09-17T02:40:57.287153+0000","last_clean":"2024-09-17T02:40:57.287153+0000","last_became_active":"2024-09-17T02:40:56.305729+0000","last_became_peered":"2024-09-17T02:40:56.305729+0000","last_unstale":"2024-09-17T02:40:57.287153+0000","last_undegraded":"2024-09-17T02:40:57.287153+0000","last_fullsized":"2024-09-17T02:40:57.287153+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:32.927545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472197+0000","last_change":"2024-09-17T02:40:52.229209+0000","last_active":"2024-09-17T02:40:57.472197+0000","last_peered":"2024-09-17T02:40:57.472197+0000","last_clean":"2024-09-17T02:40:57.472197+0000","last_became_active":"2024-09-17T02:40:52.228749+0000","last_became_peered":"2024-09-17T02:40:52.228749+0000","last_unstale":"2024-09-17T02:40:57.472197+0000","last_undegraded":"2024-09-17T02:40:57.472197+0000","last_fullsized":"2024-09-17T02:40:57.472197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:52.104423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471968+0000","last_change":"2024-09-17T02:40:51.244855+0000","last_active":"2024-09-17T02:40:57.471968+0000","last_peered":"2024-09-17T02:40:57.471968+0000","last_clean":"2024-09-17T02:40:57.471968+0000","last_became_active":"2024-09-17T02:40:51.244535+0000","last_became_peered":"2024-09-17T02:40:51.244535+0000","last_unstale":"2024-09-17T02:40:57.471968+0000","last_undegraded":"2024-09-17T02:40:57.471968+0000","last_fullsized":"2024-09-17T02:40:57.471968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:15.387810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471115+0000","last_change":"2024-09-17T02:40:56.316962+0000","last_active":"2024-09-17T02:40:57.471115+0000","last_peered":"2024-09-17T02:40:57.471115+0000","last_clean":"2024-09-17T02:40:57.471115+0000","last_became_active":"2024-09-17T02:40:56.316350+0000","last_became_peered":"2024-09-17T02:40:56.316350+0000","last_unstale":"2024-09-17T02:40:57.471115+0000","last_undegraded":"2024-09-17T02:40:57.471115+0000","last_fullsized":"2024-09-17T02:40:57.471115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:49.655411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287234+0000","last_change":"2024-09-17T02:40:52.237017+0000","last_active":"2024-09-17T02:40:57.287234+0000","last_peered":"2024-09-17T02:40:57.287234+0000","last_clean":"2024-09-17T02:40:57.287234+0000","last_became_active":"2024-09-17T02:40:52.230499+0000","last_became_peered":"2024-09-17T02:40:52.230499+0000","last_unstale":"2024-09-17T02:40:57.287234+0000","last_undegraded":"2024-09-17T02:40:57.287234+0000","last_fullsized":"2024-09-17T02:40:57.287234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:46.628840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290952+0000","last_change":"2024-09-17T02:40:51.241977+0000","last_active":"2024-09-17T02:40:57.290952+0000","last_peered":"2024-09-17T02:40:57.290952+0000","last_clean":"2024-09-17T02:40:57.290952+0000","last_became_active":"2024-09-17T02:40:51.241088+0000","last_became_peered":"2024-09-17T02:40:51.241088+0000","last_unstale":"2024-09-17T02:40:57.290952+0000","last_undegraded":"2024-09-17T02:40:57.290952+0000","last_fullsized":"2024-09-17T02:40:57.290952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:15.387810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475101+0000","last_change":"2024-09-17T02:40:56.349582+0000","last_active":"2024-09-17T02:40:57.475101+0000","last_peered":"2024-09-17T02:40:57.475101+0000","last_clean":"2024-09-17T02:40:57.475101+0000","last_became_active":"2024-09-17T02:40:56.348588+0000","last_became_peered":"2024-09-17T02:40:56.348588+0000","last_unstale":"2024-09-17T02:40:57.475101+0000","last_undegraded":"2024-09-17T02:40:57.475101+0000","last_fullsized":"2024-09-17T02:40:57.475101+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:38:43.776429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470147+0000","last_change":"2024-09-17T02:40:52.209126+0000","last_active":"2024-09-17T02:40:57.470147+0000","last_peered":"2024-09-17T02:40:57.470147+0000","last_clean":"2024-09-17T02:40:57.470147+0000","last_became_active":"2024-09-17T02:40:52.208689+0000","last_became_peered":"2024-09-17T02:40:52.208689+0000","last_unstale":"2024-09-17T02:40:57.470147+0000","last_undegraded":"2024-09-17T02:40:57.470147+0000","last_fullsized":"2024-09-17T02:40:57.470147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:08.039862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470654+0000","last_change":"2024-09-17T02:40:51.224834+0000","last_active":"2024-09-17T02:40:57.470654+0000","last_peered":"2024-09-17T02:40:57.470654+0000","last_clean":"2024-09-17T02:40:57.470654+0000","last_became_active":"2024-09-17T02:40:51.224133+0000","last_became_peered":"2024-09-17T02:40:51.224133+0000","last_unstale":"2024-09-17T02:40:57.470654+0000","last_undegraded":"2024-09-17T02:40:57.470654+0000","last_fullsized":"2024-09-17T02:40:57.470654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:13.491613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474143+0000","last_change":"2024-09-17T02:40:56.340807+0000","last_active":"2024-09-17T02:40:57.474143+0000","last_peered":"2024-09-17T02:40:57.474143+0000","last_clean":"2024-09-17T02:40:57.474143+0000","last_became_active":"2024-09-17T02:40:56.339879+0000","last_became_peered":"2024-09-17T02:40:56.339879+0000","last_unstale":"2024-09-17T02:40:57.474143+0000","last_undegraded":"2024-09-17T02:40:57.474143+0000","last_fullsized":"2024-09-17T02:40:57.474143+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:24.197066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288140+0000","last_change":"2024-09-17T02:40:52.244094+0000","last_active":"2024-09-17T02:40:57.288140+0000","last_peered":"2024-09-17T02:40:57.288140+0000","last_clean":"2024-09-17T02:40:57.288140+0000","last_became_active":"2024-09-17T02:40:52.242739+0000","last_became_peered":"2024-09-17T02:40:52.242739+0000","last_unstale":"2024-09-17T02:40:57.288140+0000","last_undegraded":"2024-09-17T02:40:57.288140+0000","last_fullsized":"2024-09-17T02:40:57.288140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:36.659884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288765+0000","last_change":"2024-09-17T02:40:51.222211+0000","last_active":"2024-09-17T02:40:57.288765+0000","last_peered":"2024-09-17T02:40:57.288765+0000","last_clean":"2024-09-17T02:40:57.288765+0000","last_became_active":"2024-09-17T02:40:51.221783+0000","last_became_peered":"2024-09-17T02:40:51.221783+0000","last_unstale":"2024-09-17T02:40:57.288765+0000","last_undegraded":"2024-09-17T02:40:57.288765+0000","last_fullsized":"2024-09-17T02:40:57.288765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:31.651869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288876+0000","last_change":"2024-09-17T02:40:56.348106+0000","last_active":"2024-09-17T02:40:57.288876+0000","last_peered":"2024-09-17T02:40:57.288876+0000","last_clean":"2024-09-17T02:40:57.288876+0000","last_became_active":"2024-09-17T02:40:56.339110+0000","last_became_peered":"2024-09-17T02:40:56.339110+0000","last_unstale":"2024-09-17T02:40:57.288876+0000","last_undegraded":"2024-09-17T02:40:57.288876+0000","last_fullsized":"2024-09-17T02:40:57.288876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:11.904235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287392+0000","last_change":"2024-09-17T02:40:52.230525+0000","last_active":"2024-09-17T02:40:57.287392+0000","last_peered":"2024-09-17T02:40:57.287392+0000","last_clean":"2024-09-17T02:40:57.287392+0000","last_became_active":"2024-09-17T02:40:52.230230+0000","last_became_peered":"2024-09-17T02:40:52.230230+0000","last_unstale":"2024-09-17T02:40:57.287392+0000","last_undegraded":"2024-09-17T02:40:57.287392+0000","last_fullsized":"2024-09-17T02:40:57.287392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:40.156443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473184+0000","last_change":"2024-09-17T02:40:51.232502+0000","last_active":"2024-09-17T02:40:57.473184+0000","last_peered":"2024-09-17T02:40:57.473184+0000","last_clean":"2024-09-17T02:40:57.473184+0000","last_became_active":"2024-09-17T02:40:51.230289+0000","last_became_peered":"2024-09-17T02:40:51.230289+0000","last_unstale":"2024-09-17T02:40:57.473184+0000","last_undegraded":"2024-09-17T02:40:57.473184+0000","last_fullsized":"2024-09-17T02:40:57.473184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:59.380994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288172+0000","last_change":"2024-09-17T02:40:56.325742+0000","last_active":"2024-09-17T02:40:57.288172+0000","last_peered":"2024-09-17T02:40:57.288172+0000","last_clean":"2024-09-17T02:40:57.288172+0000","last_became_active":"2024-09-17T02:40:56.325287+0000","last_became_peered":"2024-09-17T02:40:56.325287+0000","last_unstale":"2024-09-17T02:40:57.288172+0000","last_undegraded":"2024-09-17T02:40:57.288172+0000","last_fullsized":"2024-09-17T02:40:57.288172+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:28.012374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471884+0000","last_change":"2024-09-17T02:40:52.236056+0000","last_active":"2024-09-17T02:40:57.471884+0000","last_peered":"2024-09-17T02:40:57.471884+0000","last_clean":"2024-09-17T02:40:57.471884+0000","last_became_active":"2024-09-17T02:40:52.235680+0000","last_became_peered":"2024-09-17T02:40:52.235680+0000","last_unstale":"2024-09-17T02:40:57.471884+0000","last_undegraded":"2024-09-17T02:40:57.471884+0000","last_fullsized":"2024-09-17T02:40:57.471884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:27.873310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:41:03.060549+0000","last_change":"2024-09-17T02:40:51.242488+0000","last_active":"2024-09-17T02:41:03.060549+0000","last_peered":"2024-09-17T02:41:03.060549+0000","last_clean":"2024-09-17T02:41:03.060549+0000","last_became_active":"2024-09-17T02:40:51.242105+0000","last_became_peered":"2024-09-17T02:40:51.242105+0000","last_unstale":"2024-09-17T02:41:03.060549+0000","last_undegraded":"2024-09-17T02:41:03.060549+0000","last_fullsized":"2024-09-17T02:41:03.060549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:02.225605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287754+0000","last_change":"2024-09-17T02:40:56.327216+0000","last_active":"2024-09-17T02:40:57.287754+0000","last_peered":"2024-09-17T02:40:57.287754+0000","last_clean":"2024-09-17T02:40:57.287754+0000","last_became_active":"2024-09-17T02:40:56.326213+0000","last_became_peered":"2024-09-17T02:40:56.326213+0000","last_unstale":"2024-09-17T02:40:57.287754+0000","last_undegraded":"2024-09-17T02:40:57.287754+0000","last_fullsized":"2024-09-17T02:40:57.287754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:58.168471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288181+0000","last_change":"2024-09-17T02:40:52.237718+0000","last_active":"2024-09-17T02:40:57.288181+0000","last_peered":"2024-09-17T02:40:57.288181+0000","last_clean":"2024-09-17T02:40:57.288181+0000","last_became_active":"2024-09-17T02:40:52.237303+0000","last_became_peered":"2024-09-17T02:40:52.237303+0000","last_unstale":"2024-09-17T02:40:57.288181+0000","last_undegraded":"2024-09-17T02:40:57.288181+0000","last_fullsized":"2024-09-17T02:40:57.288181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:47.651165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474398+0000","last_change":"2024-09-17T02:40:51.234788+0000","last_active":"2024-09-17T02:40:57.474398+0000","last_peered":"2024-09-17T02:40:57.474398+0000","last_clean":"2024-09-17T02:40:57.474398+0000","last_became_active":"2024-09-17T02:40:51.233917+0000","last_became_peered":"2024-09-17T02:40:51.233917+0000","last_unstale":"2024-09-17T02:40:57.474398+0000","last_undegraded":"2024-09-17T02:40:57.474398+0000","last_fullsized":"2024-09-17T02:40:57.474398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:27.301395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471431+0000","last_change":"2024-09-17T02:40:56.337448+0000","last_active":"2024-09-17T02:40:57.471431+0000","last_peered":"2024-09-17T02:40:57.471431+0000","last_clean":"2024-09-17T02:40:57.471431+0000","last_became_active":"2024-09-17T02:40:56.336959+0000","last_became_peered":"2024-09-17T02:40:56.336959+0000","last_unstale":"2024-09-17T02:40:57.471431+0000","last_undegraded":"2024-09-17T02:40:57.471431+0000","last_fullsized":"2024-09-17T02:40:57.471431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:31.952318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474548+0000","last_change":"2024-09-17T02:40:52.247671+0000","last_active":"2024-09-17T02:40:57.474548+0000","last_peered":"2024-09-17T02:40:57.474548+0000","last_clean":"2024-09-17T02:40:57.474548+0000","last_became_active":"2024-09-17T02:40:52.247325+0000","last_became_peered":"2024-09-17T02:40:52.247325+0000","last_unstale":"2024-09-17T02:40:57.474548+0000","last_undegraded":"2024-09-17T02:40:57.474548+0000","last_fullsized":"2024-09-17T02:40:57.474548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:09.357623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471882+0000","last_change":"2024-09-17T02:40:51.224974+0000","last_active":"2024-09-17T02:40:57.471882+0000","last_peered":"2024-09-17T02:40:57.471882+0000","last_clean":"2024-09-17T02:40:57.471882+0000","last_became_active":"2024-09-17T02:40:51.224479+0000","last_became_peered":"2024-09-17T02:40:51.224479+0000","last_unstale":"2024-09-17T02:40:57.471882+0000","last_undegraded":"2024-09-17T02:40:57.471882+0000","last_fullsized":"2024-09-17T02:40:57.471882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471541+0000","last_change":"2024-09-17T02:40:56.333951+0000","last_active":"2024-09-17T02:40:57.471541+0000","last_peered":"2024-09-17T02:40:57.471541+0000","last_clean":"2024-09-17T02:40:57.471541+0000","last_became_active":"2024-09-17T02:40:56.332508+0000","last_became_peered":"2024-09-17T02:40:56.332508+0000","last_unstale":"2024-09-17T02:40:57.471541+0000","last_undegraded":"2024-09-17T02:40:57.471541+0000","last_fullsized":"2024-09-17T02:40:57.471541+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:08.241482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472176+0000","last_change":"2024-09-17T02:40:52.244735+0000","last_active":"2024-09-17T02:40:57.472176+0000","last_peered":"2024-09-17T02:40:57.472176+0000","last_clean":"2024-09-17T02:40:57.472176+0000","last_became_active":"2024-09-17T02:40:52.243763+0000","last_became_peered":"2024-09-17T02:40:52.243763+0000","last_unstale":"2024-09-17T02:40:57.472176+0000","last_undegraded":"2024-09-17T02:40:57.472176+0000","last_fullsized":"2024-09-17T02:40:57.472176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:32.662763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471344+0000","last_change":"2024-09-17T02:40:51.234151+0000","last_active":"2024-09-17T02:40:57.471344+0000","last_peered":"2024-09-17T02:40:57.471344+0000","last_clean":"2024-09-17T02:40:57.471344+0000","last_became_active":"2024-09-17T02:40:51.233731+0000","last_became_peered":"2024-09-17T02:40:51.233731+0000","last_unstale":"2024-09-17T02:40:57.471344+0000","last_undegraded":"2024-09-17T02:40:57.471344+0000","last_fullsized":"2024-09-17T02:40:57.471344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:15.349994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290693+0000","last_change":"2024-09-17T02:40:56.334710+0000","last_active":"2024-09-17T02:40:57.290693+0000","last_peered":"2024-09-17T02:40:57.290693+0000","last_clean":"2024-09-17T02:40:57.290693+0000","last_became_active":"2024-09-17T02:40:56.333630+0000","last_became_peered":"2024-09-17T02:40:56.333630+0000","last_unstale":"2024-09-17T02:40:57.290693+0000","last_undegraded":"2024-09-17T02:40:57.290693+0000","last_fullsized":"2024-09-17T02:40:57.290693+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:07.834564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290681+0000","last_change":"2024-09-17T02:40:56.321729+0000","last_active":"2024-09-17T02:40:57.290681+0000","last_peered":"2024-09-17T02:40:57.290681+0000","last_clean":"2024-09-17T02:40:57.290681+0000","last_became_active":"2024-09-17T02:40:56.320583+0000","last_became_peered":"2024-09-17T02:40:56.320583+0000","last_unstale":"2024-09-17T02:40:57.290681+0000","last_undegraded":"2024-09-17T02:40:57.290681+0000","last_fullsized":"2024-09-17T02:40:57.290681+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:50.707848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291114+0000","last_change":"2024-09-17T02:40:52.224867+0000","last_active":"2024-09-17T02:40:57.291114+0000","last_peered":"2024-09-17T02:40:57.291114+0000","last_clean":"2024-09-17T02:40:57.291114+0000","last_became_active":"2024-09-17T02:40:52.224558+0000","last_became_peered":"2024-09-17T02:40:52.224558+0000","last_unstale":"2024-09-17T02:40:57.291114+0000","last_undegraded":"2024-09-17T02:40:57.291114+0000","last_fullsized":"2024-09-17T02:40:57.291114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:01.098205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287555+0000","last_change":"2024-09-17T02:40:51.210128+0000","last_active":"2024-09-17T02:40:57.287555+0000","last_peered":"2024-09-17T02:40:57.287555+0000","last_clean":"2024-09-17T02:40:57.287555+0000","last_became_active":"2024-09-17T02:40:51.209655+0000","last_became_peered":"2024-09-17T02:40:51.209655+0000","last_unstale":"2024-09-17T02:40:57.287555+0000","last_undegraded":"2024-09-17T02:40:57.287555+0000","last_fullsized":"2024-09-17T02:40:57.287555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:18.053396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288450+0000","last_change":"2024-09-17T02:40:56.316501+0000","last_active":"2024-09-17T02:40:57.288450+0000","last_peered":"2024-09-17T02:40:57.288450+0000","last_clean":"2024-09-17T02:40:57.288450+0000","last_became_active":"2024-09-17T02:40:56.315839+0000","last_became_peered":"2024-09-17T02:40:56.315839+0000","last_unstale":"2024-09-17T02:40:57.288450+0000","last_undegraded":"2024-09-17T02:40:57.288450+0000","last_fullsized":"2024-09-17T02:40:57.288450+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:21.657805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472377+0000","last_change":"2024-09-17T02:40:52.228669+0000","last_active":"2024-09-17T02:40:57.472377+0000","last_peered":"2024-09-17T02:40:57.472377+0000","last_clean":"2024-09-17T02:40:57.472377+0000","last_became_active":"2024-09-17T02:40:52.226236+0000","last_became_peered":"2024-09-17T02:40:52.226236+0000","last_unstale":"2024-09-17T02:40:57.472377+0000","last_undegraded":"2024-09-17T02:40:57.472377+0000","last_fullsized":"2024-09-17T02:40:57.472377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:41.461773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287139+0000","last_change":"2024-09-17T02:40:51.226679+0000","last_active":"2024-09-17T02:40:57.287139+0000","last_peered":"2024-09-17T02:40:57.287139+0000","last_clean":"2024-09-17T02:40:57.287139+0000","last_became_active":"2024-09-17T02:40:51.225794+0000","last_became_peered":"2024-09-17T02:40:51.225794+0000","last_unstale":"2024-09-17T02:40:57.287139+0000","last_undegraded":"2024-09-17T02:40:57.287139+0000","last_fullsized":"2024-09-17T02:40:57.287139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:37.666440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288775+0000","last_change":"2024-09-17T02:40:51.226491+0000","last_active":"2024-09-17T02:40:57.288775+0000","last_peered":"2024-09-17T02:40:57.288775+0000","last_clean":"2024-09-17T02:40:57.288775+0000","last_became_active":"2024-09-17T02:40:51.225695+0000","last_became_peered":"2024-09-17T02:40:51.225695+0000","last_unstale":"2024-09-17T02:40:57.288775+0000","last_undegraded":"2024-09-17T02:40:57.288775+0000","last_fullsized":"2024-09-17T02:40:57.288775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:46.966810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288252+0000","last_change":"2024-09-17T02:40:52.222794+0000","last_active":"2024-09-17T02:40:57.288252+0000","last_peered":"2024-09-17T02:40:57.288252+0000","last_clean":"2024-09-17T02:40:57.288252+0000","last_became_active":"2024-09-17T02:40:52.221682+0000","last_became_peered":"2024-09-17T02:40:52.221682+0000","last_unstale":"2024-09-17T02:40:57.288252+0000","last_undegraded":"2024-09-17T02:40:57.288252+0000","last_fullsized":"2024-09-17T02:40:57.288252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:34.058125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.292523+0000","last_change":"2024-09-17T02:40:56.343317+0000","last_active":"2024-09-17T02:40:57.292523+0000","last_peered":"2024-09-17T02:40:57.292523+0000","last_clean":"2024-09-17T02:40:57.292523+0000","last_became_active":"2024-09-17T02:40:56.342373+0000","last_became_peered":"2024-09-17T02:40:56.342373+0000","last_unstale":"2024-09-17T02:40:57.292523+0000","last_undegraded":"2024-09-17T02:40:57.292523+0000","last_fullsized":"2024-09-17T02:40:57.292523+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:02.210967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473874+0000","last_change":"2024-09-17T02:40:51.237550+0000","last_active":"2024-09-17T02:40:57.473874+0000","last_peered":"2024-09-17T02:40:57.473874+0000","last_clean":"2024-09-17T02:40:57.473874+0000","last_became_active":"2024-09-17T02:40:51.236422+0000","last_became_peered":"2024-09-17T02:40:51.236422+0000","last_unstale":"2024-09-17T02:40:57.473874+0000","last_undegraded":"2024-09-17T02:40:57.473874+0000","last_fullsized":"2024-09-17T02:40:57.473874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470700+0000","last_change":"2024-09-17T02:40:52.226397+0000","last_active":"2024-09-17T02:40:57.470700+0000","last_peered":"2024-09-17T02:40:57.470700+0000","last_clean":"2024-09-17T02:40:57.470700+0000","last_became_active":"2024-09-17T02:40:52.225934+0000","last_became_peered":"2024-09-17T02:40:52.225934+0000","last_unstale":"2024-09-17T02:40:57.470700+0000","last_undegraded":"2024-09-17T02:40:57.470700+0000","last_fullsized":"2024-09-17T02:40:57.470700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:17:37.421483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472110+0000","last_change":"2024-09-17T02:40:56.336996+0000","last_active":"2024-09-17T02:40:57.472110+0000","last_peered":"2024-09-17T02:40:57.472110+0000","last_clean":"2024-09-17T02:40:57.472110+0000","last_became_active":"2024-09-17T02:40:56.336172+0000","last_became_peered":"2024-09-17T02:40:56.336172+0000","last_unstale":"2024-09-17T02:40:57.472110+0000","last_undegraded":"2024-09-17T02:40:57.472110+0000","last_fullsized":"2024-09-17T02:40:57.472110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:05.177214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470197+0000","last_change":"2024-09-17T02:40:52.232046+0000","last_active":"2024-09-17T02:40:57.470197+0000","last_peered":"2024-09-17T02:40:57.470197+0000","last_clean":"2024-09-17T02:40:57.470197+0000","last_became_active":"2024-09-17T02:40:52.231700+0000","last_became_peered":"2024-09-17T02:40:52.231700+0000","last_unstale":"2024-09-17T02:40:57.470197+0000","last_undegraded":"2024-09-17T02:40:57.470197+0000","last_fullsized":"2024-09-17T02:40:57.470197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:47.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472501+0000","last_change":"2024-09-17T02:40:52.239198+0000","last_active":"2024-09-17T02:40:57.472501+0000","last_peered":"2024-09-17T02:40:57.472501+0000","last_clean":"2024-09-17T02:40:57.472501+0000","last_became_active":"2024-09-17T02:40:52.238901+0000","last_became_peered":"2024-09-17T02:40:52.238901+0000","last_unstale":"2024-09-17T02:40:57.472501+0000","last_undegraded":"2024-09-17T02:40:57.472501+0000","last_fullsized":"2024-09-17T02:40:57.472501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:47.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287639+0000","last_change":"2024-09-17T02:40:52.228993+0000","last_active":"2024-09-17T02:40:57.287639+0000","last_peered":"2024-09-17T02:40:57.287639+0000","last_clean":"2024-09-17T02:40:57.287639+0000","last_became_active":"2024-09-17T02:40:52.228574+0000","last_became_peered":"2024-09-17T02:40:52.228574+0000","last_unstale":"2024-09-17T02:40:57.287639+0000","last_undegraded":"2024-09-17T02:40:57.287639+0000","last_fullsized":"2024-09-17T02:40:57.287639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:31.798195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473564+0000","last_change":"2024-09-17T02:40:52.222569+0000","last_active":"2024-09-17T02:40:57.473564+0000","last_peered":"2024-09-17T02:40:57.473564+0000","last_clean":"2024-09-17T02:40:57.473564+0000","last_became_active":"2024-09-17T02:40:52.222228+0000","last_became_peered":"2024-09-17T02:40:52.222228+0000","last_unstale":"2024-09-17T02:40:57.473564+0000","last_undegraded":"2024-09-17T02:40:57.473564+0000","last_fullsized":"2024-09-17T02:40:57.473564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:39.361971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289288+0000","last_change":"2024-09-17T02:40:52.236488+0000","last_active":"2024-09-17T02:40:57.289288+0000","last_peered":"2024-09-17T02:40:57.289288+0000","last_clean":"2024-09-17T02:40:57.289288+0000","last_became_active":"2024-09-17T02:40:52.236171+0000","last_became_peered":"2024-09-17T02:40:52.236171+0000","last_unstale":"2024-09-17T02:40:57.289288+0000","last_undegraded":"2024-09-17T02:40:57.289288+0000","last_fullsized":"2024-09-17T02:40:57.289288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:08.415078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287712+0000","last_change":"2024-09-17T02:40:52.217993+0000","last_active":"2024-09-17T02:40:57.287712+0000","last_peered":"2024-09-17T02:40:57.287712+0000","last_clean":"2024-09-17T02:40:57.287712+0000","last_became_active":"2024-09-17T02:40:52.216716+0000","last_became_peered":"2024-09-17T02:40:52.216716+0000","last_unstale":"2024-09-17T02:40:57.287712+0000","last_undegraded":"2024-09-17T02:40:57.287712+0000","last_fullsized":"2024-09-17T02:40:57.287712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:06.000439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472031+0000","last_change":"2024-09-17T02:40:52.245158+0000","last_active":"2024-09-17T02:40:57.472031+0000","last_peered":"2024-09-17T02:40:57.472031+0000","last_clean":"2024-09-17T02:40:57.472031+0000","last_became_active":"2024-09-17T02:40:52.244333+0000","last_became_peered":"2024-09-17T02:40:52.244333+0000","last_unstale":"2024-09-17T02:40:57.472031+0000","last_undegraded":"2024-09-17T02:40:57.472031+0000","last_fullsized":"2024-09-17T02:40:57.472031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:06.006731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473931+0000","last_change":"2024-09-17T02:40:52.228925+0000","last_active":"2024-09-17T02:40:57.473931+0000","last_peered":"2024-09-17T02:40:57.473931+0000","last_clean":"2024-09-17T02:40:57.473931+0000","last_became_active":"2024-09-17T02:40:52.228587+0000","last_became_peered":"2024-09-17T02:40:52.228587+0000","last_unstale":"2024-09-17T02:40:57.473931+0000","last_undegraded":"2024-09-17T02:40:57.473931+0000","last_fullsized":"2024-09-17T02:40:57.473931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:33.958554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287342+0000","last_change":"2024-09-17T02:40:52.229795+0000","last_active":"2024-09-17T02:40:57.287342+0000","last_peered":"2024-09-17T02:40:57.287342+0000","last_clean":"2024-09-17T02:40:57.287342+0000","last_became_active":"2024-09-17T02:40:52.227659+0000","last_became_peered":"2024-09-17T02:40:52.227659+0000","last_unstale":"2024-09-17T02:40:57.287342+0000","last_undegraded":"2024-09-17T02:40:57.287342+0000","last_fullsized":"2024-09-17T02:40:57.287342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:36.541471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473327+0000","last_change":"2024-09-17T02:40:52.221420+0000","last_active":"2024-09-17T02:40:57.473327+0000","last_peered":"2024-09-17T02:40:57.473327+0000","last_clean":"2024-09-17T02:40:57.473327+0000","last_became_active":"2024-09-17T02:40:52.221050+0000","last_became_peered":"2024-09-17T02:40:52.221050+0000","last_unstale":"2024-09-17T02:40:57.473327+0000","last_undegraded":"2024-09-17T02:40:57.473327+0000","last_fullsized":"2024-09-17T02:40:57.473327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:47.715501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470588+0000","last_change":"2024-09-17T02:40:52.237946+0000","last_active":"2024-09-17T02:40:57.470588+0000","last_peered":"2024-09-17T02:40:57.470588+0000","last_clean":"2024-09-17T02:40:57.470588+0000","last_became_active":"2024-09-17T02:40:52.237376+0000","last_became_peered":"2024-09-17T02:40:52.237376+0000","last_unstale":"2024-09-17T02:40:57.470588+0000","last_undegraded":"2024-09-17T02:40:57.470588+0000","last_fullsized":"2024-09-17T02:40:57.470588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:31.107108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287955+0000","last_change":"2024-09-17T02:40:52.221072+0000","last_active":"2024-09-17T02:40:57.287955+0000","last_peered":"2024-09-17T02:40:57.287955+0000","last_clean":"2024-09-17T02:40:57.287955+0000","last_became_active":"2024-09-17T02:40:52.220697+0000","last_became_peered":"2024-09-17T02:40:52.220697+0000","last_unstale":"2024-09-17T02:40:57.287955+0000","last_undegraded":"2024-09-17T02:40:57.287955+0000","last_fullsized":"2024-09-17T02:40:57.287955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:02.681569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472298+0000","last_change":"2024-09-17T02:40:52.249011+0000","last_active":"2024-09-17T02:40:57.472298+0000","last_peered":"2024-09-17T02:40:57.472298+0000","last_clean":"2024-09-17T02:40:57.472298+0000","last_became_active":"2024-09-17T02:40:52.247429+0000","last_became_peered":"2024-09-17T02:40:52.247429+0000","last_unstale":"2024-09-17T02:40:57.472298+0000","last_undegraded":"2024-09-17T02:40:57.472298+0000","last_fullsized":"2024-09-17T02:40:57.472298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:56.223383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288383+0000","last_change":"2024-09-17T02:40:52.243578+0000","last_active":"2024-09-17T02:40:57.288383+0000","last_peered":"2024-09-17T02:40:57.288383+0000","last_clean":"2024-09-17T02:40:57.288383+0000","last_became_active":"2024-09-17T02:40:52.243106+0000","last_became_peered":"2024-09-17T02:40:52.243106+0000","last_unstale":"2024-09-17T02:40:57.288383+0000","last_undegraded":"2024-09-17T02:40:57.288383+0000","last_fullsized":"2024-09-17T02:40:57.288383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:22.663615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475176+0000","last_change":"2024-09-17T02:40:52.222351+0000","last_active":"2024-09-17T02:40:57.475176+0000","last_peered":"2024-09-17T02:40:57.475176+0000","last_clean":"2024-09-17T02:40:57.475176+0000","last_became_active":"2024-09-17T02:40:52.222043+0000","last_became_peered":"2024-09-17T02:40:52.222043+0000","last_unstale":"2024-09-17T02:40:57.475176+0000","last_undegraded":"2024-09-17T02:40:57.475176+0000","last_fullsized":"2024-09-17T02:40:57.475176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:21.583058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288320+0000","last_change":"2024-09-17T02:40:52.256413+0000","last_active":"2024-09-17T02:40:57.288320+0000","last_peered":"2024-09-17T02:40:57.288320+0000","last_clean":"2024-09-17T02:40:57.288320+0000","last_became_active":"2024-09-17T02:40:52.256035+0000","last_became_peered":"2024-09-17T02:40:52.256035+0000","last_unstale":"2024-09-17T02:40:57.288320+0000","last_undegraded":"2024-09-17T02:40:57.288320+0000","last_fullsized":"2024-09-17T02:40:57.288320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:59.172988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287206+0000","last_change":"2024-09-17T02:40:52.236686+0000","last_active":"2024-09-17T02:40:57.287206+0000","last_peered":"2024-09-17T02:40:57.287206+0000","last_clean":"2024-09-17T02:40:57.287206+0000","last_became_active":"2024-09-17T02:40:52.236194+0000","last_became_peered":"2024-09-17T02:40:52.236194+0000","last_unstale":"2024-09-17T02:40:57.287206+0000","last_undegraded":"2024-09-17T02:40:57.287206+0000","last_fullsized":"2024-09-17T02:40:57.287206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:55.015415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472841+0000","last_change":"2024-09-17T02:40:52.228032+0000","last_active":"2024-09-17T02:40:57.472841+0000","last_peered":"2024-09-17T02:40:57.472841+0000","last_clean":"2024-09-17T02:40:57.472841+0000","last_became_active":"2024-09-17T02:40:52.225817+0000","last_became_peered":"2024-09-17T02:40:52.225817+0000","last_unstale":"2024-09-17T02:40:57.472841+0000","last_undegraded":"2024-09-17T02:40:57.472841+0000","last_fullsized":"2024-09-17T02:40:57.472841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:34.058125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287107+0000","last_change":"2024-09-17T02:40:52.217145+0000","last_active":"2024-09-17T02:40:57.287107+0000","last_peered":"2024-09-17T02:40:57.287107+0000","last_clean":"2024-09-17T02:40:57.287107+0000","last_became_active":"2024-09-17T02:40:52.215909+0000","last_became_peered":"2024-09-17T02:40:52.215909+0000","last_unstale":"2024-09-17T02:40:57.287107+0000","last_undegraded":"2024-09-17T02:40:57.287107+0000","last_fullsized":"2024-09-17T02:40:57.287107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:28.728656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288943+0000","last_change":"2024-09-17T02:40:52.224668+0000","last_active":"2024-09-17T02:40:57.288943+0000","last_peered":"2024-09-17T02:40:57.288943+0000","last_clean":"2024-09-17T02:40:57.288943+0000","last_became_active":"2024-09-17T02:40:52.224273+0000","last_became_peered":"2024-09-17T02:40:52.224273+0000","last_unstale":"2024-09-17T02:40:57.288943+0000","last_undegraded":"2024-09-17T02:40:57.288943+0000","last_fullsized":"2024-09-17T02:40:57.288943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:21.184527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472191+0000","last_change":"2024-09-17T02:40:52.252355+0000","last_active":"2024-09-17T02:40:57.472191+0000","last_peered":"2024-09-17T02:40:57.472191+0000","last_clean":"2024-09-17T02:40:57.472191+0000","last_became_active":"2024-09-17T02:40:52.251950+0000","last_became_peered":"2024-09-17T02:40:52.251950+0000","last_unstale":"2024-09-17T02:40:57.472191+0000","last_undegraded":"2024-09-17T02:40:57.472191+0000","last_fullsized":"2024-09-17T02:40:57.472191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:50.916246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471773+0000","last_change":"2024-09-17T02:40:52.244189+0000","last_active":"2024-09-17T02:40:57.471773+0000","last_peered":"2024-09-17T02:40:57.471773+0000","last_clean":"2024-09-17T02:40:57.471773+0000","last_became_active":"2024-09-17T02:40:52.243891+0000","last_became_peered":"2024-09-17T02:40:52.243891+0000","last_unstale":"2024-09-17T02:40:57.471773+0000","last_undegraded":"2024-09-17T02:40:57.471773+0000","last_fullsized":"2024-09-17T02:40:57.471773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:00.308180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471829+0000","last_change":"2024-09-17T02:40:52.237299+0000","last_active":"2024-09-17T02:40:57.471829+0000","last_peered":"2024-09-17T02:40:57.471829+0000","last_clean":"2024-09-17T02:40:57.471829+0000","last_became_active":"2024-09-17T02:40:52.236928+0000","last_became_peered":"2024-09-17T02:40:52.236928+0000","last_unstale":"2024-09-17T02:40:57.471829+0000","last_undegraded":"2024-09-17T02:40:57.471829+0000","last_fullsized":"2024-09-17T02:40:57.471829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:03.755555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471259+0000","last_change":"2024-09-17T02:40:52.247989+0000","last_active":"2024-09-17T02:40:57.471259+0000","last_peered":"2024-09-17T02:40:57.471259+0000","last_clean":"2024-09-17T02:40:57.471259+0000","last_became_active":"2024-09-17T02:40:52.247534+0000","last_became_peered":"2024-09-17T02:40:52.247534+0000","last_unstale":"2024-09-17T02:40:57.471259+0000","last_undegraded":"2024-09-17T02:40:57.471259+0000","last_fullsized":"2024-09-17T02:40:57.471259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:14.502507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471636+0000","last_change":"2024-09-17T02:40:52.228868+0000","last_active":"2024-09-17T02:40:57.471636+0000","last_peered":"2024-09-17T02:40:57.471636+0000","last_clean":"2024-09-17T02:40:57.471636+0000","last_became_active":"2024-09-17T02:40:52.227927+0000","last_became_peered":"2024-09-17T02:40:52.227927+0000","last_unstale":"2024-09-17T02:40:57.471636+0000","last_undegraded":"2024-09-17T02:40:57.471636+0000","last_fullsized":"2024-09-17T02:40:57.471636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:06.283756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470545+0000","last_change":"2024-09-17T02:40:52.220236+0000","last_active":"2024-09-17T02:40:57.470545+0000","last_peered":"2024-09-17T02:40:57.470545+0000","last_clean":"2024-09-17T02:40:57.470545+0000","last_became_active":"2024-09-17T02:40:52.219510+0000","last_became_peered":"2024-09-17T02:40:52.219510+0000","last_unstale":"2024-09-17T02:40:57.470545+0000","last_undegraded":"2024-09-17T02:40:57.470545+0000","last_fullsized":"2024-09-17T02:40:57.470545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:33.958554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471108+0000","last_change":"2024-09-17T02:40:52.218799+0000","last_active":"2024-09-17T02:40:57.471108+0000","last_peered":"2024-09-17T02:40:57.471108+0000","last_clean":"2024-09-17T02:40:57.471108+0000","last_became_active":"2024-09-17T02:40:52.217054+0000","last_became_peered":"2024-09-17T02:40:52.217054+0000","last_unstale":"2024-09-17T02:40:57.471108+0000","last_undegraded":"2024-09-17T02:40:57.471108+0000","last_fullsized":"2024-09-17T02:40:57.471108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:21.583058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472762+0000","last_change":"2024-09-17T02:40:52.232583+0000","last_active":"2024-09-17T02:40:57.472762+0000","last_peered":"2024-09-17T02:40:57.472762+0000","last_clean":"2024-09-17T02:40:57.472762+0000","last_became_active":"2024-09-17T02:40:52.231933+0000","last_became_peered":"2024-09-17T02:40:52.231933+0000","last_unstale":"2024-09-17T02:40:57.472762+0000","last_undegraded":"2024-09-17T02:40:57.472762+0000","last_fullsized":"2024-09-17T02:40:57.472762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:41.461773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471574+0000","last_change":"2024-09-17T02:40:52.218335+0000","last_active":"2024-09-17T02:40:57.471574+0000","last_peered":"2024-09-17T02:40:57.471574+0000","last_clean":"2024-09-17T02:40:57.471574+0000","last_became_active":"2024-09-17T02:40:52.217008+0000","last_became_peered":"2024-09-17T02:40:52.217008+0000","last_unstale":"2024-09-17T02:40:57.471574+0000","last_undegraded":"2024-09-17T02:40:57.471574+0000","last_fullsized":"2024-09-17T02:40:57.471574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:19.064290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472531+0000","last_change":"2024-09-17T02:40:52.224771+0000","last_active":"2024-09-17T02:40:57.472531+0000","last_peered":"2024-09-17T02:40:57.472531+0000","last_clean":"2024-09-17T02:40:57.472531+0000","last_became_active":"2024-09-17T02:40:52.222947+0000","last_became_peered":"2024-09-17T02:40:52.222947+0000","last_unstale":"2024-09-17T02:40:57.472531+0000","last_undegraded":"2024-09-17T02:40:57.472531+0000","last_fullsized":"2024-09-17T02:40:57.472531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:54.216006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474943+0000","last_change":"2024-09-17T02:40:52.228603+0000","last_active":"2024-09-17T02:40:57.474943+0000","last_peered":"2024-09-17T02:40:57.474943+0000","last_clean":"2024-09-17T02:40:57.474943+0000","last_became_active":"2024-09-17T02:40:52.225545+0000","last_became_peered":"2024-09-17T02:40:52.225545+0000","last_unstale":"2024-09-17T02:40:57.474943+0000","last_undegraded":"2024-09-17T02:40:57.474943+0000","last_fullsized":"2024-09-17T02:40:57.474943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:56.223383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471852+0000","last_change":"2024-09-17T02:40:52.250495+0000","last_active":"2024-09-17T02:40:57.471852+0000","last_peered":"2024-09-17T02:40:57.471852+0000","last_clean":"2024-09-17T02:40:57.471852+0000","last_became_active":"2024-09-17T02:40:52.250095+0000","last_became_peered":"2024-09-17T02:40:52.250095+0000","last_unstale":"2024-09-17T02:40:57.471852+0000","last_undegraded":"2024-09-17T02:40:57.471852+0000","last_fullsized":"2024-09-17T02:40:57.471852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:58.131959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:41:19.291 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:41:19.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.474+0000 7f0ff8e57640 1 -- 172.21.15.131:0/1411201178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ff40cec00 msgr2=0x7f0ff40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.474+0000 7f0ff8e57640 1 --2- 172.21.15.131:0/1411201178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ff40cec00 0x7f0ff40cf010 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009900 tx=0x7f0fe801c850 comp rx=0 tx=0).stop 2024-09-17T02:41:19.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.474+0000 7f0ff8e57640 1 -- 172.21.15.131:0/1411201178 shutdown_connections 2024-09-17T02:41:19.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.474+0000 7f0ff8e57640 1 --2- 172.21.15.131:0/1411201178 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ff40c4c90 0x7f0ff40c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.474+0000 7f0ff8e57640 1 --2- 172.21.15.131:0/1411201178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ff40cec00 0x7f0ff40cf010 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.474+0000 7f0ff8e57640 1 -- 172.21.15.131:0/1411201178 >> 172.21.15.131:0/1411201178 conn(0x7f0ff406eea0 msgr2=0x7f0ff406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:19.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.474+0000 7f0ff8e57640 1 -- 172.21.15.131:0/1411201178 shutdown_connections 2024-09-17T02:41:19.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.474+0000 7f0ff8e57640 1 -- 172.21.15.131:0/1411201178 wait complete. 2024-09-17T02:41:19.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.474+0000 7f0ff8e57640 1 Processor -- start 2024-09-17T02:41:19.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.474+0000 7f0ff8e57640 1 -- start start 2024-09-17T02:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff8e57640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ff406e2c0 0x7f0ff414ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff8e57640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ff40c4c90 0x7f0ff40c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff8e57640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ff40cec00 0x7f0ff40c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff8e57640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff4169cf0 con 0x7f0ff40cec00 2024-09-17T02:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff8e57640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff415cb90 con 0x7f0ff406e2c0 2024-09-17T02:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff8e57640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff4169e70 con 0x7f0ff40c4c90 2024-09-17T02:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ff40cec00 0x7f0ff40c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ff40cec00 0x7f0ff40c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58132/0 (socket says 172.21.15.131:58132) 2024-09-17T02:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff2d9d640 1 -- 172.21.15.131:0/376710011 learned_addr learned my addr 172.21.15.131:0/376710011 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff1d9b640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ff40c4c90 0x7f0ff40c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff259c640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ff406e2c0 0x7f0ff414ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff2d9d640 1 -- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ff40c4c90 msgr2=0x7f0ff40c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff2d9d640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ff40c4c90 0x7f0ff40c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff2d9d640 1 -- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ff406e2c0 msgr2=0x7f0ff414ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff2d9d640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ff406e2c0 0x7f0ff414ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff2d9d640 1 -- 172.21.15.131:0/376710011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff414f1b0 con 0x7f0ff40cec00 2024-09-17T02:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff2d9d640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ff40cec00 0x7f0ff40c6340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00cd10 tx=0x7f0fec00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0fe2ffd640 1 -- 172.21.15.131:0/376710011 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec00f450 con 0x7f0ff40cec00 2024-09-17T02:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0fe2ffd640 1 -- 172.21.15.131:0/376710011 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fec00f700 con 0x7f0ff40cec00 2024-09-17T02:41:19.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff8e57640 1 -- 172.21.15.131:0/376710011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff40b02b0 con 0x7f0ff40cec00 2024-09-17T02:41:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0fe2ffd640 1 -- 172.21.15.131:0/376710011 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec00fb50 con 0x7f0ff40cec00 2024-09-17T02:41:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff8e57640 1 -- 172.21.15.131:0/376710011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff41521c0 con 0x7f0ff40cec00 2024-09-17T02:41:19.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0ff8e57640 1 -- 172.21.15.131:0/376710011 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff40c50a0 con 0x7f0ff40cec00 2024-09-17T02:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.478+0000 7f0fe2ffd640 1 -- 172.21.15.131:0/376710011 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0fec00fcf0 con 0x7f0ff40cec00 2024-09-17T02:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.482+0000 7f0fe2ffd640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0fd4073230 0x7f0fd4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.482+0000 7f0fe2ffd640 1 -- 172.21.15.131:0/376710011 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f0fec095000 con 0x7f0ff40cec00 2024-09-17T02:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.482+0000 7f0ff259c640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0fd4073230 0x7f0fd4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.482+0000 7f0ff259c640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0fd4073230 0x7f0fd4075740 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40c7050 tx=0x7f0fe4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:19.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.482+0000 7f0fe2ffd640 1 -- 172.21.15.131:0/376710011 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0fec014030 con 0x7f0ff40cec00 2024-09-17T02:41:19.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.638+0000 7f0ff8e57640 1 -- 172.21.15.131:0/376710011 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0ff40b0040 con 0x7f0fd4073230 2024-09-17T02:41:19.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.654+0000 7f0fe2ffd640 1 -- 172.21.15.131:0/376710011 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332516 (secure 0 0 0) 0x7f0ff40b0040 con 0x7f0fd4073230 2024-09-17T02:41:19.656 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:19.658 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:41:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 -- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0fd4073230 msgr2=0x7f0fd4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0fd4073230 0x7f0fd4075740 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40c7050 tx=0x7f0fe4009040 comp rx=0 tx=0).stop 2024-09-17T02:41:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 -- 172.21.15.131:0/376710011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ff40cec00 msgr2=0x7f0ff40c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ff40cec00 0x7f0ff40c6340 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00cd10 tx=0x7f0fec00a620 comp rx=0 tx=0).stop 2024-09-17T02:41:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 -- 172.21.15.131:0/376710011 shutdown_connections 2024-09-17T02:41:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0fd4073230 0x7f0fd4075740 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ff40cec00 0x7f0ff40c6340 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ff40c4c90 0x7f0ff40c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 --2- 172.21.15.131:0/376710011 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ff406e2c0 0x7f0ff414ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 -- 172.21.15.131:0/376710011 >> 172.21.15.131:0/376710011 conn(0x7f0ff406eea0 msgr2=0x7f0ff41533f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 -- 172.21.15.131:0/376710011 shutdown_connections 2024-09-17T02:41:19.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.658+0000 7f0ff8e57640 1 -- 172.21.15.131:0/376710011 wait complete. 2024-09-17T02:41:19.676 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-09-17T02:41:19.125970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219564,"kb_used_data":3336,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755156,"statfs":{"total":773094113280,"available":772869279744,"internally_reserved":0,"allocated":3416064,"data_stored":2037168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006109"},"pg_stats":[{"pgid":"2.1f","version":"21'4","reported_seq":27,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290027+0000","last_change":"2024-09-17T02:40:51.241640+0000","last_active":"2024-09-17T02:40:57.290027+0000","last_peered":"2024-09-17T02:40:57.290027+0000","last_clean":"2024-09-17T02:40:57.290027+0000","last_became_active":"2024-09-17T02:40:51.240675+0000","last_became_peered":"2024-09-17T02:40:51.240675+0000","last_unstale":"2024-09-17T02:40:57.290027+0000","last_undegraded":"2024-09-17T02:40:57.290027+0000","last_fullsized":"2024-09-17T02:40:57.290027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475107+0000","last_change":"2024-09-17T02:40:52.228237+0000","last_active":"2024-09-17T02:40:57.475107+0000","last_peered":"2024-09-17T02:40:57.475107+0000","last_clean":"2024-09-17T02:40:57.475107+0000","last_became_active":"2024-09-17T02:40:52.225329+0000","last_became_peered":"2024-09-17T02:40:52.225329+0000","last_unstale":"2024-09-17T02:40:57.475107+0000","last_undegraded":"2024-09-17T02:40:57.475107+0000","last_fullsized":"2024-09-17T02:40:57.475107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:55.015415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471940+0000","last_change":"2024-09-17T02:40:56.353097+0000","last_active":"2024-09-17T02:40:57.471940+0000","last_peered":"2024-09-17T02:40:57.471940+0000","last_clean":"2024-09-17T02:40:57.471940+0000","last_became_active":"2024-09-17T02:40:56.351847+0000","last_became_peered":"2024-09-17T02:40:56.351847+0000","last_unstale":"2024-09-17T02:40:57.471940+0000","last_undegraded":"2024-09-17T02:40:57.471940+0000","last_fullsized":"2024-09-17T02:40:57.471940+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:06.760578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475261+0000","last_change":"2024-09-17T02:40:51.243590+0000","last_active":"2024-09-17T02:40:57.475261+0000","last_peered":"2024-09-17T02:40:57.475261+0000","last_clean":"2024-09-17T02:40:57.475261+0000","last_became_active":"2024-09-17T02:40:51.242684+0000","last_became_peered":"2024-09-17T02:40:51.242684+0000","last_unstale":"2024-09-17T02:40:57.475261+0000","last_undegraded":"2024-09-17T02:40:57.475261+0000","last_fullsized":"2024-09-17T02:40:57.475261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474246+0000","last_change":"2024-09-17T02:40:52.207992+0000","last_active":"2024-09-17T02:40:57.474246+0000","last_peered":"2024-09-17T02:40:57.474246+0000","last_clean":"2024-09-17T02:40:57.474246+0000","last_became_active":"2024-09-17T02:40:52.207568+0000","last_became_peered":"2024-09-17T02:40:52.207568+0000","last_unstale":"2024-09-17T02:40:57.474246+0000","last_undegraded":"2024-09-17T02:40:57.474246+0000","last_fullsized":"2024-09-17T02:40:57.474246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:18:01.844374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470155+0000","last_change":"2024-09-17T02:40:56.339588+0000","last_active":"2024-09-17T02:40:57.470155+0000","last_peered":"2024-09-17T02:40:57.470155+0000","last_clean":"2024-09-17T02:40:57.470155+0000","last_became_active":"2024-09-17T02:40:56.338569+0000","last_became_peered":"2024-09-17T02:40:56.338569+0000","last_unstale":"2024-09-17T02:40:57.470155+0000","last_undegraded":"2024-09-17T02:40:57.470155+0000","last_fullsized":"2024-09-17T02:40:57.470155+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:54.995255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472080+0000","last_change":"2024-09-17T02:40:51.225527+0000","last_active":"2024-09-17T02:40:57.472080+0000","last_peered":"2024-09-17T02:40:57.472080+0000","last_clean":"2024-09-17T02:40:57.472080+0000","last_became_active":"2024-09-17T02:40:51.224640+0000","last_became_peered":"2024-09-17T02:40:51.224640+0000","last_unstale":"2024-09-17T02:40:57.472080+0000","last_undegraded":"2024-09-17T02:40:57.472080+0000","last_fullsized":"2024-09-17T02:40:57.472080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:07.028967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470812+0000","last_change":"2024-09-17T02:40:52.243749+0000","last_active":"2024-09-17T02:40:57.470812+0000","last_peered":"2024-09-17T02:40:57.470812+0000","last_clean":"2024-09-17T02:40:57.470812+0000","last_became_active":"2024-09-17T02:40:52.243361+0000","last_became_peered":"2024-09-17T02:40:52.243361+0000","last_unstale":"2024-09-17T02:40:57.470812+0000","last_undegraded":"2024-09-17T02:40:57.470812+0000","last_fullsized":"2024-09-17T02:40:57.470812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:01.471406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471331+0000","last_change":"2024-09-17T02:40:56.330627+0000","last_active":"2024-09-17T02:40:57.471331+0000","last_peered":"2024-09-17T02:40:57.471331+0000","last_clean":"2024-09-17T02:40:57.471331+0000","last_became_active":"2024-09-17T02:40:56.330123+0000","last_became_peered":"2024-09-17T02:40:56.330123+0000","last_unstale":"2024-09-17T02:40:57.471331+0000","last_undegraded":"2024-09-17T02:40:57.471331+0000","last_fullsized":"2024-09-17T02:40:57.471331+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:30.712481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472462+0000","last_change":"2024-09-17T02:40:51.220105+0000","last_active":"2024-09-17T02:40:57.472462+0000","last_peered":"2024-09-17T02:40:57.472462+0000","last_clean":"2024-09-17T02:40:57.472462+0000","last_became_active":"2024-09-17T02:40:51.216247+0000","last_became_peered":"2024-09-17T02:40:51.216247+0000","last_unstale":"2024-09-17T02:40:57.472462+0000","last_undegraded":"2024-09-17T02:40:57.472462+0000","last_fullsized":"2024-09-17T02:40:57.472462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:30.096213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288051+0000","last_change":"2024-09-17T02:40:52.255870+0000","last_active":"2024-09-17T02:40:57.288051+0000","last_peered":"2024-09-17T02:40:57.288051+0000","last_clean":"2024-09-17T02:40:57.288051+0000","last_became_active":"2024-09-17T02:40:52.255454+0000","last_became_peered":"2024-09-17T02:40:52.255454+0000","last_unstale":"2024-09-17T02:40:57.288051+0000","last_undegraded":"2024-09-17T02:40:57.288051+0000","last_fullsized":"2024-09-17T02:40:57.288051+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:35.834666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470627+0000","last_change":"2024-09-17T02:40:56.339659+0000","last_active":"2024-09-17T02:40:57.470627+0000","last_peered":"2024-09-17T02:40:57.470627+0000","last_clean":"2024-09-17T02:40:57.470627+0000","last_became_active":"2024-09-17T02:40:56.339130+0000","last_became_peered":"2024-09-17T02:40:56.339130+0000","last_unstale":"2024-09-17T02:40:57.470627+0000","last_undegraded":"2024-09-17T02:40:57.470627+0000","last_fullsized":"2024-09-17T02:40:57.470627+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:38.223653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287802+0000","last_change":"2024-09-17T02:40:52.242899+0000","last_active":"2024-09-17T02:40:57.287802+0000","last_peered":"2024-09-17T02:40:57.287802+0000","last_clean":"2024-09-17T02:40:57.287802+0000","last_became_active":"2024-09-17T02:40:52.242570+0000","last_became_peered":"2024-09-17T02:40:52.242570+0000","last_unstale":"2024-09-17T02:40:57.287802+0000","last_undegraded":"2024-09-17T02:40:57.287802+0000","last_fullsized":"2024-09-17T02:40:57.287802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:45.600475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288528+0000","last_change":"2024-09-17T02:40:51.212048+0000","last_active":"2024-09-17T02:40:57.288528+0000","last_peered":"2024-09-17T02:40:57.288528+0000","last_clean":"2024-09-17T02:40:57.288528+0000","last_became_active":"2024-09-17T02:40:51.211531+0000","last_became_peered":"2024-09-17T02:40:51.211531+0000","last_unstale":"2024-09-17T02:40:57.288528+0000","last_undegraded":"2024-09-17T02:40:57.288528+0000","last_fullsized":"2024-09-17T02:40:57.288528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:10.284709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287732+0000","last_change":"2024-09-17T02:40:56.332707+0000","last_active":"2024-09-17T02:40:57.287732+0000","last_peered":"2024-09-17T02:40:57.287732+0000","last_clean":"2024-09-17T02:40:57.287732+0000","last_became_active":"2024-09-17T02:40:56.332260+0000","last_became_peered":"2024-09-17T02:40:56.332260+0000","last_unstale":"2024-09-17T02:40:57.287732+0000","last_undegraded":"2024-09-17T02:40:57.287732+0000","last_fullsized":"2024-09-17T02:40:57.287732+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:53.783346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472671+0000","last_change":"2024-09-17T02:40:52.218705+0000","last_active":"2024-09-17T02:40:57.472671+0000","last_peered":"2024-09-17T02:40:57.472671+0000","last_clean":"2024-09-17T02:40:57.472671+0000","last_became_active":"2024-09-17T02:40:52.217762+0000","last_became_peered":"2024-09-17T02:40:52.217762+0000","last_unstale":"2024-09-17T02:40:57.472671+0000","last_undegraded":"2024-09-17T02:40:57.472671+0000","last_fullsized":"2024-09-17T02:40:57.472671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:11.295603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287768+0000","last_change":"2024-09-17T02:40:51.237760+0000","last_active":"2024-09-17T02:40:57.287768+0000","last_peered":"2024-09-17T02:40:57.287768+0000","last_clean":"2024-09-17T02:40:57.287768+0000","last_became_active":"2024-09-17T02:40:51.236973+0000","last_became_peered":"2024-09-17T02:40:51.236973+0000","last_unstale":"2024-09-17T02:40:57.287768+0000","last_undegraded":"2024-09-17T02:40:57.287768+0000","last_fullsized":"2024-09-17T02:40:57.287768+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:22.634046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471688+0000","last_change":"2024-09-17T02:40:56.355680+0000","last_active":"2024-09-17T02:40:57.471688+0000","last_peered":"2024-09-17T02:40:57.471688+0000","last_clean":"2024-09-17T02:40:57.471688+0000","last_became_active":"2024-09-17T02:40:56.355224+0000","last_became_peered":"2024-09-17T02:40:56.355224+0000","last_unstale":"2024-09-17T02:40:57.471688+0000","last_undegraded":"2024-09-17T02:40:57.471688+0000","last_fullsized":"2024-09-17T02:40:57.471688+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:49.679484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290367+0000","last_change":"2024-09-17T02:40:52.220376+0000","last_active":"2024-09-17T02:40:57.290367+0000","last_peered":"2024-09-17T02:40:57.290367+0000","last_clean":"2024-09-17T02:40:57.290367+0000","last_became_active":"2024-09-17T02:40:52.220031+0000","last_became_peered":"2024-09-17T02:40:52.220031+0000","last_unstale":"2024-09-17T02:40:57.290367+0000","last_undegraded":"2024-09-17T02:40:57.290367+0000","last_fullsized":"2024-09-17T02:40:57.290367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:04.162473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291288+0000","last_change":"2024-09-17T02:40:51.240925+0000","last_active":"2024-09-17T02:40:57.291288+0000","last_peered":"2024-09-17T02:40:57.291288+0000","last_clean":"2024-09-17T02:40:57.291288+0000","last_became_active":"2024-09-17T02:40:51.233638+0000","last_became_peered":"2024-09-17T02:40:51.233638+0000","last_unstale":"2024-09-17T02:40:57.291288+0000","last_undegraded":"2024-09-17T02:40:57.291288+0000","last_fullsized":"2024-09-17T02:40:57.291288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:39.387068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475651+0000","last_change":"2024-09-17T02:40:56.339397+0000","last_active":"2024-09-17T02:40:57.475651+0000","last_peered":"2024-09-17T02:40:57.475651+0000","last_clean":"2024-09-17T02:40:57.475651+0000","last_became_active":"2024-09-17T02:40:56.338591+0000","last_became_peered":"2024-09-17T02:40:56.338591+0000","last_unstale":"2024-09-17T02:40:57.475651+0000","last_undegraded":"2024-09-17T02:40:57.475651+0000","last_fullsized":"2024-09-17T02:40:57.475651+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:10.362765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287431+0000","last_change":"2024-09-17T02:40:52.245352+0000","last_active":"2024-09-17T02:40:57.287431+0000","last_peered":"2024-09-17T02:40:57.287431+0000","last_clean":"2024-09-17T02:40:57.287431+0000","last_became_active":"2024-09-17T02:40:52.244938+0000","last_became_peered":"2024-09-17T02:40:52.244938+0000","last_unstale":"2024-09-17T02:40:57.287431+0000","last_undegraded":"2024-09-17T02:40:57.287431+0000","last_fullsized":"2024-09-17T02:40:57.287431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:07.474908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.469729+0000","last_change":"2024-09-17T02:40:51.220831+0000","last_active":"2024-09-17T02:40:57.469729+0000","last_peered":"2024-09-17T02:40:57.469729+0000","last_clean":"2024-09-17T02:40:57.469729+0000","last_became_active":"2024-09-17T02:40:51.220253+0000","last_became_peered":"2024-09-17T02:40:51.220253+0000","last_unstale":"2024-09-17T02:40:57.469729+0000","last_undegraded":"2024-09-17T02:40:57.469729+0000","last_fullsized":"2024-09-17T02:40:57.469729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471876+0000","last_change":"2024-09-17T02:40:56.333001+0000","last_active":"2024-09-17T02:40:57.471876+0000","last_peered":"2024-09-17T02:40:57.471876+0000","last_clean":"2024-09-17T02:40:57.471876+0000","last_became_active":"2024-09-17T02:40:56.332421+0000","last_became_peered":"2024-09-17T02:40:56.332421+0000","last_unstale":"2024-09-17T02:40:57.471876+0000","last_undegraded":"2024-09-17T02:40:57.471876+0000","last_fullsized":"2024-09-17T02:40:57.471876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:58.168471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291218+0000","last_change":"2024-09-17T02:40:52.215438+0000","last_active":"2024-09-17T02:40:57.291218+0000","last_peered":"2024-09-17T02:40:57.291218+0000","last_clean":"2024-09-17T02:40:57.291218+0000","last_became_active":"2024-09-17T02:40:52.215073+0000","last_became_peered":"2024-09-17T02:40:52.215073+0000","last_unstale":"2024-09-17T02:40:57.291218+0000","last_undegraded":"2024-09-17T02:40:57.291218+0000","last_fullsized":"2024-09-17T02:40:57.291218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:39.361971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288442+0000","last_change":"2024-09-17T02:40:51.226075+0000","last_active":"2024-09-17T02:40:57.288442+0000","last_peered":"2024-09-17T02:40:57.288442+0000","last_clean":"2024-09-17T02:40:57.288442+0000","last_became_active":"2024-09-17T02:40:51.225620+0000","last_became_peered":"2024-09-17T02:40:51.225620+0000","last_unstale":"2024-09-17T02:40:57.288442+0000","last_undegraded":"2024-09-17T02:40:57.288442+0000","last_fullsized":"2024-09-17T02:40:57.288442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:15.238526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470178+0000","last_change":"2024-09-17T02:40:56.306280+0000","last_active":"2024-09-17T02:40:57.470178+0000","last_peered":"2024-09-17T02:40:57.470178+0000","last_clean":"2024-09-17T02:40:57.470178+0000","last_became_active":"2024-09-17T02:40:56.305348+0000","last_became_peered":"2024-09-17T02:40:56.305348+0000","last_unstale":"2024-09-17T02:40:57.470178+0000","last_undegraded":"2024-09-17T02:40:57.470178+0000","last_fullsized":"2024-09-17T02:40:57.470178+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:58.295015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289748+0000","last_change":"2024-09-17T02:40:52.217580+0000","last_active":"2024-09-17T02:40:57.289748+0000","last_peered":"2024-09-17T02:40:57.289748+0000","last_clean":"2024-09-17T02:40:57.289748+0000","last_became_active":"2024-09-17T02:40:52.217246+0000","last_became_peered":"2024-09-17T02:40:52.217246+0000","last_unstale":"2024-09-17T02:40:57.289748+0000","last_undegraded":"2024-09-17T02:40:57.289748+0000","last_fullsized":"2024-09-17T02:40:57.289748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:25.696012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291979+0000","last_change":"2024-09-17T02:40:51.235278+0000","last_active":"2024-09-17T02:40:57.291979+0000","last_peered":"2024-09-17T02:40:57.291979+0000","last_clean":"2024-09-17T02:40:57.291979+0000","last_became_active":"2024-09-17T02:40:51.234012+0000","last_became_peered":"2024-09-17T02:40:51.234012+0000","last_unstale":"2024-09-17T02:40:57.291979+0000","last_undegraded":"2024-09-17T02:40:57.291979+0000","last_fullsized":"2024-09-17T02:40:57.291979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:07.028967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290548+0000","last_change":"2024-09-17T02:40:56.306563+0000","last_active":"2024-09-17T02:40:57.290548+0000","last_peered":"2024-09-17T02:40:57.290548+0000","last_clean":"2024-09-17T02:40:57.290548+0000","last_became_active":"2024-09-17T02:40:56.303819+0000","last_became_peered":"2024-09-17T02:40:56.303819+0000","last_unstale":"2024-09-17T02:40:57.290548+0000","last_undegraded":"2024-09-17T02:40:57.290548+0000","last_fullsized":"2024-09-17T02:40:57.290548+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:04.387188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289492+0000","last_change":"2024-09-17T02:40:52.216405+0000","last_active":"2024-09-17T02:40:57.289492+0000","last_peered":"2024-09-17T02:40:57.289492+0000","last_clean":"2024-09-17T02:40:57.289492+0000","last_became_active":"2024-09-17T02:40:52.216037+0000","last_became_peered":"2024-09-17T02:40:52.216037+0000","last_unstale":"2024-09-17T02:40:57.289492+0000","last_undegraded":"2024-09-17T02:40:57.289492+0000","last_fullsized":"2024-09-17T02:40:57.289492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:20.565582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288509+0000","last_change":"2024-09-17T02:40:51.243903+0000","last_active":"2024-09-17T02:40:57.288509+0000","last_peered":"2024-09-17T02:40:57.288509+0000","last_clean":"2024-09-17T02:40:57.288509+0000","last_became_active":"2024-09-17T02:40:51.243568+0000","last_became_peered":"2024-09-17T02:40:51.243568+0000","last_unstale":"2024-09-17T02:40:57.288509+0000","last_undegraded":"2024-09-17T02:40:57.288509+0000","last_fullsized":"2024-09-17T02:40:57.288509+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:48:49.643024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287722+0000","last_change":"2024-09-17T02:40:56.340894+0000","last_active":"2024-09-17T02:40:57.287722+0000","last_peered":"2024-09-17T02:40:57.287722+0000","last_clean":"2024-09-17T02:40:57.287722+0000","last_became_active":"2024-09-17T02:40:56.338947+0000","last_became_peered":"2024-09-17T02:40:56.338947+0000","last_unstale":"2024-09-17T02:40:57.287722+0000","last_undegraded":"2024-09-17T02:40:57.287722+0000","last_fullsized":"2024-09-17T02:40:57.287722+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:48.414928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472018+0000","last_change":"2024-09-17T02:40:52.232754+0000","last_active":"2024-09-17T02:40:57.472018+0000","last_peered":"2024-09-17T02:40:57.472018+0000","last_clean":"2024-09-17T02:40:57.472018+0000","last_became_active":"2024-09-17T02:40:52.231727+0000","last_became_peered":"2024-09-17T02:40:52.231727+0000","last_unstale":"2024-09-17T02:40:57.472018+0000","last_undegraded":"2024-09-17T02:40:57.472018+0000","last_fullsized":"2024-09-17T02:40:57.472018+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:11.295603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473934+0000","last_change":"2024-09-17T02:40:51.237795+0000","last_active":"2024-09-17T02:40:57.473934+0000","last_peered":"2024-09-17T02:40:57.473934+0000","last_clean":"2024-09-17T02:40:57.473934+0000","last_became_active":"2024-09-17T02:40:51.236960+0000","last_became_peered":"2024-09-17T02:40:51.236960+0000","last_unstale":"2024-09-17T02:40:57.473934+0000","last_undegraded":"2024-09-17T02:40:57.473934+0000","last_fullsized":"2024-09-17T02:40:57.473934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:49:39.387068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287516+0000","last_change":"2024-09-17T02:40:56.340920+0000","last_active":"2024-09-17T02:40:57.287516+0000","last_peered":"2024-09-17T02:40:57.287516+0000","last_clean":"2024-09-17T02:40:57.287516+0000","last_became_active":"2024-09-17T02:40:56.339463+0000","last_became_peered":"2024-09-17T02:40:56.339463+0000","last_unstale":"2024-09-17T02:40:57.287516+0000","last_undegraded":"2024-09-17T02:40:57.287516+0000","last_fullsized":"2024-09-17T02:40:57.287516+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:04:36.386068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472660+0000","last_change":"2024-09-17T02:40:52.234607+0000","last_active":"2024-09-17T02:40:57.472660+0000","last_peered":"2024-09-17T02:40:57.472660+0000","last_clean":"2024-09-17T02:40:57.472660+0000","last_became_active":"2024-09-17T02:40:52.234175+0000","last_became_peered":"2024-09-17T02:40:52.234175+0000","last_unstale":"2024-09-17T02:40:57.472660+0000","last_undegraded":"2024-09-17T02:40:57.472660+0000","last_fullsized":"2024-09-17T02:40:57.472660+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:34.144644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288850+0000","last_change":"2024-09-17T02:40:51.221611+0000","last_active":"2024-09-17T02:40:57.288850+0000","last_peered":"2024-09-17T02:40:57.288850+0000","last_clean":"2024-09-17T02:40:57.288850+0000","last_became_active":"2024-09-17T02:40:51.221259+0000","last_became_peered":"2024-09-17T02:40:51.221259+0000","last_unstale":"2024-09-17T02:40:57.288850+0000","last_undegraded":"2024-09-17T02:40:57.288850+0000","last_fullsized":"2024-09-17T02:40:57.288850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:20.572163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289392+0000","last_change":"2024-09-17T02:40:56.350014+0000","last_active":"2024-09-17T02:40:57.289392+0000","last_peered":"2024-09-17T02:40:57.289392+0000","last_clean":"2024-09-17T02:40:57.289392+0000","last_became_active":"2024-09-17T02:40:56.347244+0000","last_became_peered":"2024-09-17T02:40:56.347244+0000","last_unstale":"2024-09-17T02:40:57.289392+0000","last_undegraded":"2024-09-17T02:40:57.289392+0000","last_fullsized":"2024-09-17T02:40:57.289392+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:10.648378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471945+0000","last_change":"2024-09-17T02:40:52.221771+0000","last_active":"2024-09-17T02:40:57.471945+0000","last_peered":"2024-09-17T02:40:57.471945+0000","last_clean":"2024-09-17T02:40:57.471945+0000","last_became_active":"2024-09-17T02:40:52.221461+0000","last_became_peered":"2024-09-17T02:40:52.221461+0000","last_unstale":"2024-09-17T02:40:57.471945+0000","last_undegraded":"2024-09-17T02:40:57.471945+0000","last_fullsized":"2024-09-17T02:40:57.471945+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:25.696012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472767+0000","last_change":"2024-09-17T02:40:51.247013+0000","last_active":"2024-09-17T02:40:57.472767+0000","last_peered":"2024-09-17T02:40:57.472767+0000","last_clean":"2024-09-17T02:40:57.472767+0000","last_became_active":"2024-09-17T02:40:51.246163+0000","last_became_peered":"2024-09-17T02:40:51.246163+0000","last_unstale":"2024-09-17T02:40:57.472767+0000","last_undegraded":"2024-09-17T02:40:57.472767+0000","last_fullsized":"2024-09-17T02:40:57.472767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:05.797393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288261+0000","last_change":"2024-09-17T02:40:56.340614+0000","last_active":"2024-09-17T02:40:57.288261+0000","last_peered":"2024-09-17T02:40:57.288261+0000","last_clean":"2024-09-17T02:40:57.288261+0000","last_became_active":"2024-09-17T02:40:56.338340+0000","last_became_peered":"2024-09-17T02:40:56.338340+0000","last_unstale":"2024-09-17T02:40:57.288261+0000","last_undegraded":"2024-09-17T02:40:57.288261+0000","last_fullsized":"2024-09-17T02:40:57.288261+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:50:03.442300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.286936+0000","last_change":"2024-09-17T02:40:52.227802+0000","last_active":"2024-09-17T02:40:57.286936+0000","last_peered":"2024-09-17T02:40:57.286936+0000","last_clean":"2024-09-17T02:40:57.286936+0000","last_became_active":"2024-09-17T02:40:52.227458+0000","last_became_peered":"2024-09-17T02:40:52.227458+0000","last_unstale":"2024-09-17T02:40:57.286936+0000","last_undegraded":"2024-09-17T02:40:57.286936+0000","last_fullsized":"2024-09-17T02:40:57.286936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:21.198509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287577+0000","last_change":"2024-09-17T02:40:51.239502+0000","last_active":"2024-09-17T02:40:57.287577+0000","last_peered":"2024-09-17T02:40:57.287577+0000","last_clean":"2024-09-17T02:40:57.287577+0000","last_became_active":"2024-09-17T02:40:51.239042+0000","last_became_peered":"2024-09-17T02:40:51.239042+0000","last_unstale":"2024-09-17T02:40:57.287577+0000","last_undegraded":"2024-09-17T02:40:57.287577+0000","last_fullsized":"2024-09-17T02:40:57.287577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:04.995836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474601+0000","last_change":"2024-09-17T02:40:56.353034+0000","last_active":"2024-09-17T02:40:57.474601+0000","last_peered":"2024-09-17T02:40:57.474601+0000","last_clean":"2024-09-17T02:40:57.474601+0000","last_became_active":"2024-09-17T02:40:56.351920+0000","last_became_peered":"2024-09-17T02:40:56.351920+0000","last_unstale":"2024-09-17T02:40:57.474601+0000","last_undegraded":"2024-09-17T02:40:57.474601+0000","last_fullsized":"2024-09-17T02:40:57.474601+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:39.913675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289211+0000","last_change":"2024-09-17T02:40:52.223050+0000","last_active":"2024-09-17T02:40:57.289211+0000","last_peered":"2024-09-17T02:40:57.289211+0000","last_clean":"2024-09-17T02:40:57.289211+0000","last_became_active":"2024-09-17T02:40:52.221680+0000","last_became_peered":"2024-09-17T02:40:52.221680+0000","last_unstale":"2024-09-17T02:40:57.289211+0000","last_undegraded":"2024-09-17T02:40:57.289211+0000","last_fullsized":"2024-09-17T02:40:57.289211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:06.283756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288675+0000","last_change":"2024-09-17T02:40:51.209280+0000","last_active":"2024-09-17T02:40:57.288675+0000","last_peered":"2024-09-17T02:40:57.288675+0000","last_clean":"2024-09-17T02:40:57.288675+0000","last_became_active":"2024-09-17T02:40:51.208951+0000","last_became_peered":"2024-09-17T02:40:51.208951+0000","last_unstale":"2024-09-17T02:40:57.288675+0000","last_undegraded":"2024-09-17T02:40:57.288675+0000","last_fullsized":"2024-09-17T02:40:57.288675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:51.093529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470279+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:40:57.470279+0000","last_peered":"2024-09-17T02:40:57.470279+0000","last_clean":"2024-09-17T02:40:57.470279+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:40:57.470279+0000","last_undegraded":"2024-09-17T02:40:57.470279+0000","last_fullsized":"2024-09-17T02:40:57.470279+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288416+0000","last_change":"2024-09-17T02:40:56.306099+0000","last_active":"2024-09-17T02:40:57.288416+0000","last_peered":"2024-09-17T02:40:57.288416+0000","last_clean":"2024-09-17T02:40:57.288416+0000","last_became_active":"2024-09-17T02:40:56.305604+0000","last_became_peered":"2024-09-17T02:40:56.305604+0000","last_unstale":"2024-09-17T02:40:57.288416+0000","last_undegraded":"2024-09-17T02:40:57.288416+0000","last_fullsized":"2024-09-17T02:40:57.288416+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:22.632347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288182+0000","last_change":"2024-09-17T02:40:52.255845+0000","last_active":"2024-09-17T02:40:57.288182+0000","last_peered":"2024-09-17T02:40:57.288182+0000","last_clean":"2024-09-17T02:40:57.288182+0000","last_became_active":"2024-09-17T02:40:52.255416+0000","last_became_peered":"2024-09-17T02:40:52.255416+0000","last_unstale":"2024-09-17T02:40:57.288182+0000","last_undegraded":"2024-09-17T02:40:57.288182+0000","last_fullsized":"2024-09-17T02:40:57.288182+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:40.879599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289502+0000","last_change":"2024-09-17T02:40:51.230583+0000","last_active":"2024-09-17T02:40:57.289502+0000","last_peered":"2024-09-17T02:40:57.289502+0000","last_clean":"2024-09-17T02:40:57.289502+0000","last_became_active":"2024-09-17T02:40:51.229636+0000","last_became_peered":"2024-09-17T02:40:51.229636+0000","last_unstale":"2024-09-17T02:40:57.289502+0000","last_undegraded":"2024-09-17T02:40:57.289502+0000","last_fullsized":"2024-09-17T02:40:57.289502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:25.403066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470871+0000","last_change":"2024-09-17T02:40:56.334178+0000","last_active":"2024-09-17T02:40:57.470871+0000","last_peered":"2024-09-17T02:40:57.470871+0000","last_clean":"2024-09-17T02:40:57.470871+0000","last_became_active":"2024-09-17T02:40:56.333520+0000","last_became_peered":"2024-09-17T02:40:56.333520+0000","last_unstale":"2024-09-17T02:40:57.470871+0000","last_undegraded":"2024-09-17T02:40:57.470871+0000","last_fullsized":"2024-09-17T02:40:57.470871+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:38.037563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472395+0000","last_change":"2024-09-17T02:40:52.237704+0000","last_active":"2024-09-17T02:40:57.472395+0000","last_peered":"2024-09-17T02:40:57.472395+0000","last_clean":"2024-09-17T02:40:57.472395+0000","last_became_active":"2024-09-17T02:40:52.235768+0000","last_became_peered":"2024-09-17T02:40:52.235768+0000","last_unstale":"2024-09-17T02:40:57.472395+0000","last_undegraded":"2024-09-17T02:40:57.472395+0000","last_fullsized":"2024-09-17T02:40:57.472395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:04.549252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473130+0000","last_change":"2024-09-17T02:40:51.227571+0000","last_active":"2024-09-17T02:40:57.473130+0000","last_peered":"2024-09-17T02:40:57.473130+0000","last_clean":"2024-09-17T02:40:57.473130+0000","last_became_active":"2024-09-17T02:40:51.226748+0000","last_became_peered":"2024-09-17T02:40:51.226748+0000","last_unstale":"2024-09-17T02:40:57.473130+0000","last_undegraded":"2024-09-17T02:40:57.473130+0000","last_fullsized":"2024-09-17T02:40:57.473130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:12.330693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473168+0000","last_change":"2024-09-17T02:40:56.327712+0000","last_active":"2024-09-17T02:40:57.473168+0000","last_peered":"2024-09-17T02:40:57.473168+0000","last_clean":"2024-09-17T02:40:57.473168+0000","last_became_active":"2024-09-17T02:40:56.326836+0000","last_became_peered":"2024-09-17T02:40:56.326836+0000","last_unstale":"2024-09-17T02:40:57.473168+0000","last_undegraded":"2024-09-17T02:40:57.473168+0000","last_fullsized":"2024-09-17T02:40:57.473168+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:18.756879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471966+0000","last_change":"2024-09-17T02:40:52.239267+0000","last_active":"2024-09-17T02:40:57.471966+0000","last_peered":"2024-09-17T02:40:57.471966+0000","last_clean":"2024-09-17T02:40:57.471966+0000","last_became_active":"2024-09-17T02:40:52.238707+0000","last_became_peered":"2024-09-17T02:40:52.238707+0000","last_unstale":"2024-09-17T02:40:57.471966+0000","last_undegraded":"2024-09-17T02:40:57.471966+0000","last_fullsized":"2024-09-17T02:40:57.471966+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:07.999419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291556+0000","last_change":"2024-09-17T02:40:51.249182+0000","last_active":"2024-09-17T02:40:57.291556+0000","last_peered":"2024-09-17T02:40:57.291556+0000","last_clean":"2024-09-17T02:40:57.291556+0000","last_became_active":"2024-09-17T02:40:51.248480+0000","last_became_peered":"2024-09-17T02:40:51.248480+0000","last_unstale":"2024-09-17T02:40:57.291556+0000","last_undegraded":"2024-09-17T02:40:57.291556+0000","last_fullsized":"2024-09-17T02:40:57.291556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287153+0000","last_change":"2024-09-17T02:40:56.306150+0000","last_active":"2024-09-17T02:40:57.287153+0000","last_peered":"2024-09-17T02:40:57.287153+0000","last_clean":"2024-09-17T02:40:57.287153+0000","last_became_active":"2024-09-17T02:40:56.305729+0000","last_became_peered":"2024-09-17T02:40:56.305729+0000","last_unstale":"2024-09-17T02:40:57.287153+0000","last_undegraded":"2024-09-17T02:40:57.287153+0000","last_fullsized":"2024-09-17T02:40:57.287153+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:32.927545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472197+0000","last_change":"2024-09-17T02:40:52.229209+0000","last_active":"2024-09-17T02:40:57.472197+0000","last_peered":"2024-09-17T02:40:57.472197+0000","last_clean":"2024-09-17T02:40:57.472197+0000","last_became_active":"2024-09-17T02:40:52.228749+0000","last_became_peered":"2024-09-17T02:40:52.228749+0000","last_unstale":"2024-09-17T02:40:57.472197+0000","last_undegraded":"2024-09-17T02:40:57.472197+0000","last_fullsized":"2024-09-17T02:40:57.472197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:00:52.104423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471968+0000","last_change":"2024-09-17T02:40:51.244855+0000","last_active":"2024-09-17T02:40:57.471968+0000","last_peered":"2024-09-17T02:40:57.471968+0000","last_clean":"2024-09-17T02:40:57.471968+0000","last_became_active":"2024-09-17T02:40:51.244535+0000","last_became_peered":"2024-09-17T02:40:51.244535+0000","last_unstale":"2024-09-17T02:40:57.471968+0000","last_undegraded":"2024-09-17T02:40:57.471968+0000","last_fullsized":"2024-09-17T02:40:57.471968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:15.387810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471115+0000","last_change":"2024-09-17T02:40:56.316962+0000","last_active":"2024-09-17T02:40:57.471115+0000","last_peered":"2024-09-17T02:40:57.471115+0000","last_clean":"2024-09-17T02:40:57.471115+0000","last_became_active":"2024-09-17T02:40:56.316350+0000","last_became_peered":"2024-09-17T02:40:56.316350+0000","last_unstale":"2024-09-17T02:40:57.471115+0000","last_undegraded":"2024-09-17T02:40:57.471115+0000","last_fullsized":"2024-09-17T02:40:57.471115+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:49.655411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287234+0000","last_change":"2024-09-17T02:40:52.237017+0000","last_active":"2024-09-17T02:40:57.287234+0000","last_peered":"2024-09-17T02:40:57.287234+0000","last_clean":"2024-09-17T02:40:57.287234+0000","last_became_active":"2024-09-17T02:40:52.230499+0000","last_became_peered":"2024-09-17T02:40:52.230499+0000","last_unstale":"2024-09-17T02:40:57.287234+0000","last_undegraded":"2024-09-17T02:40:57.287234+0000","last_fullsized":"2024-09-17T02:40:57.287234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:46.628840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290952+0000","last_change":"2024-09-17T02:40:51.241977+0000","last_active":"2024-09-17T02:40:57.290952+0000","last_peered":"2024-09-17T02:40:57.290952+0000","last_clean":"2024-09-17T02:40:57.290952+0000","last_became_active":"2024-09-17T02:40:51.241088+0000","last_became_peered":"2024-09-17T02:40:51.241088+0000","last_unstale":"2024-09-17T02:40:57.290952+0000","last_undegraded":"2024-09-17T02:40:57.290952+0000","last_fullsized":"2024-09-17T02:40:57.290952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:15.387810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475101+0000","last_change":"2024-09-17T02:40:56.349582+0000","last_active":"2024-09-17T02:40:57.475101+0000","last_peered":"2024-09-17T02:40:57.475101+0000","last_clean":"2024-09-17T02:40:57.475101+0000","last_became_active":"2024-09-17T02:40:56.348588+0000","last_became_peered":"2024-09-17T02:40:56.348588+0000","last_unstale":"2024-09-17T02:40:57.475101+0000","last_undegraded":"2024-09-17T02:40:57.475101+0000","last_fullsized":"2024-09-17T02:40:57.475101+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:38:43.776429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470147+0000","last_change":"2024-09-17T02:40:52.209126+0000","last_active":"2024-09-17T02:40:57.470147+0000","last_peered":"2024-09-17T02:40:57.470147+0000","last_clean":"2024-09-17T02:40:57.470147+0000","last_became_active":"2024-09-17T02:40:52.208689+0000","last_became_peered":"2024-09-17T02:40:52.208689+0000","last_unstale":"2024-09-17T02:40:57.470147+0000","last_undegraded":"2024-09-17T02:40:57.470147+0000","last_fullsized":"2024-09-17T02:40:57.470147+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:08.039862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470654+0000","last_change":"2024-09-17T02:40:51.224834+0000","last_active":"2024-09-17T02:40:57.470654+0000","last_peered":"2024-09-17T02:40:57.470654+0000","last_clean":"2024-09-17T02:40:57.470654+0000","last_became_active":"2024-09-17T02:40:51.224133+0000","last_became_peered":"2024-09-17T02:40:51.224133+0000","last_unstale":"2024-09-17T02:40:57.470654+0000","last_undegraded":"2024-09-17T02:40:57.470654+0000","last_fullsized":"2024-09-17T02:40:57.470654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:13.491613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474143+0000","last_change":"2024-09-17T02:40:56.340807+0000","last_active":"2024-09-17T02:40:57.474143+0000","last_peered":"2024-09-17T02:40:57.474143+0000","last_clean":"2024-09-17T02:40:57.474143+0000","last_became_active":"2024-09-17T02:40:56.339879+0000","last_became_peered":"2024-09-17T02:40:56.339879+0000","last_unstale":"2024-09-17T02:40:57.474143+0000","last_undegraded":"2024-09-17T02:40:57.474143+0000","last_fullsized":"2024-09-17T02:40:57.474143+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:24.197066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288140+0000","last_change":"2024-09-17T02:40:52.244094+0000","last_active":"2024-09-17T02:40:57.288140+0000","last_peered":"2024-09-17T02:40:57.288140+0000","last_clean":"2024-09-17T02:40:57.288140+0000","last_became_active":"2024-09-17T02:40:52.242739+0000","last_became_peered":"2024-09-17T02:40:52.242739+0000","last_unstale":"2024-09-17T02:40:57.288140+0000","last_undegraded":"2024-09-17T02:40:57.288140+0000","last_fullsized":"2024-09-17T02:40:57.288140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:36.659884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288765+0000","last_change":"2024-09-17T02:40:51.222211+0000","last_active":"2024-09-17T02:40:57.288765+0000","last_peered":"2024-09-17T02:40:57.288765+0000","last_clean":"2024-09-17T02:40:57.288765+0000","last_became_active":"2024-09-17T02:40:51.221783+0000","last_became_peered":"2024-09-17T02:40:51.221783+0000","last_unstale":"2024-09-17T02:40:57.288765+0000","last_undegraded":"2024-09-17T02:40:57.288765+0000","last_fullsized":"2024-09-17T02:40:57.288765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:31.651869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288876+0000","last_change":"2024-09-17T02:40:56.348106+0000","last_active":"2024-09-17T02:40:57.288876+0000","last_peered":"2024-09-17T02:40:57.288876+0000","last_clean":"2024-09-17T02:40:57.288876+0000","last_became_active":"2024-09-17T02:40:56.339110+0000","last_became_peered":"2024-09-17T02:40:56.339110+0000","last_unstale":"2024-09-17T02:40:57.288876+0000","last_undegraded":"2024-09-17T02:40:57.288876+0000","last_fullsized":"2024-09-17T02:40:57.288876+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:11.904235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287392+0000","last_change":"2024-09-17T02:40:52.230525+0000","last_active":"2024-09-17T02:40:57.287392+0000","last_peered":"2024-09-17T02:40:57.287392+0000","last_clean":"2024-09-17T02:40:57.287392+0000","last_became_active":"2024-09-17T02:40:52.230230+0000","last_became_peered":"2024-09-17T02:40:52.230230+0000","last_unstale":"2024-09-17T02:40:57.287392+0000","last_undegraded":"2024-09-17T02:40:57.287392+0000","last_fullsized":"2024-09-17T02:40:57.287392+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:40.156443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473184+0000","last_change":"2024-09-17T02:40:51.232502+0000","last_active":"2024-09-17T02:40:57.473184+0000","last_peered":"2024-09-17T02:40:57.473184+0000","last_clean":"2024-09-17T02:40:57.473184+0000","last_became_active":"2024-09-17T02:40:51.230289+0000","last_became_peered":"2024-09-17T02:40:51.230289+0000","last_unstale":"2024-09-17T02:40:57.473184+0000","last_undegraded":"2024-09-17T02:40:57.473184+0000","last_fullsized":"2024-09-17T02:40:57.473184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:59.380994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288172+0000","last_change":"2024-09-17T02:40:56.325742+0000","last_active":"2024-09-17T02:40:57.288172+0000","last_peered":"2024-09-17T02:40:57.288172+0000","last_clean":"2024-09-17T02:40:57.288172+0000","last_became_active":"2024-09-17T02:40:56.325287+0000","last_became_peered":"2024-09-17T02:40:56.325287+0000","last_unstale":"2024-09-17T02:40:57.288172+0000","last_undegraded":"2024-09-17T02:40:57.288172+0000","last_fullsized":"2024-09-17T02:40:57.288172+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:14:28.012374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471884+0000","last_change":"2024-09-17T02:40:52.236056+0000","last_active":"2024-09-17T02:40:57.471884+0000","last_peered":"2024-09-17T02:40:57.471884+0000","last_clean":"2024-09-17T02:40:57.471884+0000","last_became_active":"2024-09-17T02:40:52.235680+0000","last_became_peered":"2024-09-17T02:40:52.235680+0000","last_unstale":"2024-09-17T02:40:57.471884+0000","last_undegraded":"2024-09-17T02:40:57.471884+0000","last_fullsized":"2024-09-17T02:40:57.471884+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:27.873310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"26'9","reported_seq":31,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:41:03.060549+0000","last_change":"2024-09-17T02:40:51.242488+0000","last_active":"2024-09-17T02:41:03.060549+0000","last_peered":"2024-09-17T02:41:03.060549+0000","last_clean":"2024-09-17T02:41:03.060549+0000","last_became_active":"2024-09-17T02:40:51.242105+0000","last_became_peered":"2024-09-17T02:40:51.242105+0000","last_unstale":"2024-09-17T02:41:03.060549+0000","last_undegraded":"2024-09-17T02:41:03.060549+0000","last_fullsized":"2024-09-17T02:41:03.060549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:08:02.225605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287754+0000","last_change":"2024-09-17T02:40:56.327216+0000","last_active":"2024-09-17T02:40:57.287754+0000","last_peered":"2024-09-17T02:40:57.287754+0000","last_clean":"2024-09-17T02:40:57.287754+0000","last_became_active":"2024-09-17T02:40:56.326213+0000","last_became_peered":"2024-09-17T02:40:56.326213+0000","last_unstale":"2024-09-17T02:40:57.287754+0000","last_undegraded":"2024-09-17T02:40:57.287754+0000","last_fullsized":"2024-09-17T02:40:57.287754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:53:58.168471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288181+0000","last_change":"2024-09-17T02:40:52.237718+0000","last_active":"2024-09-17T02:40:57.288181+0000","last_peered":"2024-09-17T02:40:57.288181+0000","last_clean":"2024-09-17T02:40:57.288181+0000","last_became_active":"2024-09-17T02:40:52.237303+0000","last_became_peered":"2024-09-17T02:40:52.237303+0000","last_unstale":"2024-09-17T02:40:57.288181+0000","last_undegraded":"2024-09-17T02:40:57.288181+0000","last_fullsized":"2024-09-17T02:40:57.288181+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:47.651165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474398+0000","last_change":"2024-09-17T02:40:51.234788+0000","last_active":"2024-09-17T02:40:57.474398+0000","last_peered":"2024-09-17T02:40:57.474398+0000","last_clean":"2024-09-17T02:40:57.474398+0000","last_became_active":"2024-09-17T02:40:51.233917+0000","last_became_peered":"2024-09-17T02:40:51.233917+0000","last_unstale":"2024-09-17T02:40:57.474398+0000","last_undegraded":"2024-09-17T02:40:57.474398+0000","last_fullsized":"2024-09-17T02:40:57.474398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:33:27.301395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471431+0000","last_change":"2024-09-17T02:40:56.337448+0000","last_active":"2024-09-17T02:40:57.471431+0000","last_peered":"2024-09-17T02:40:57.471431+0000","last_clean":"2024-09-17T02:40:57.471431+0000","last_became_active":"2024-09-17T02:40:56.336959+0000","last_became_peered":"2024-09-17T02:40:56.336959+0000","last_unstale":"2024-09-17T02:40:57.471431+0000","last_undegraded":"2024-09-17T02:40:57.471431+0000","last_fullsized":"2024-09-17T02:40:57.471431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:31.952318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474548+0000","last_change":"2024-09-17T02:40:52.247671+0000","last_active":"2024-09-17T02:40:57.474548+0000","last_peered":"2024-09-17T02:40:57.474548+0000","last_clean":"2024-09-17T02:40:57.474548+0000","last_became_active":"2024-09-17T02:40:52.247325+0000","last_became_peered":"2024-09-17T02:40:52.247325+0000","last_unstale":"2024-09-17T02:40:57.474548+0000","last_undegraded":"2024-09-17T02:40:57.474548+0000","last_fullsized":"2024-09-17T02:40:57.474548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:32:09.357623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471882+0000","last_change":"2024-09-17T02:40:51.224974+0000","last_active":"2024-09-17T02:40:57.471882+0000","last_peered":"2024-09-17T02:40:57.471882+0000","last_clean":"2024-09-17T02:40:57.471882+0000","last_became_active":"2024-09-17T02:40:51.224479+0000","last_became_peered":"2024-09-17T02:40:51.224479+0000","last_unstale":"2024-09-17T02:40:57.471882+0000","last_undegraded":"2024-09-17T02:40:57.471882+0000","last_fullsized":"2024-09-17T02:40:57.471882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:29.269875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471541+0000","last_change":"2024-09-17T02:40:56.333951+0000","last_active":"2024-09-17T02:40:57.471541+0000","last_peered":"2024-09-17T02:40:57.471541+0000","last_clean":"2024-09-17T02:40:57.471541+0000","last_became_active":"2024-09-17T02:40:56.332508+0000","last_became_peered":"2024-09-17T02:40:56.332508+0000","last_unstale":"2024-09-17T02:40:57.471541+0000","last_undegraded":"2024-09-17T02:40:57.471541+0000","last_fullsized":"2024-09-17T02:40:57.471541+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:08.241482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472176+0000","last_change":"2024-09-17T02:40:52.244735+0000","last_active":"2024-09-17T02:40:57.472176+0000","last_peered":"2024-09-17T02:40:57.472176+0000","last_clean":"2024-09-17T02:40:57.472176+0000","last_became_active":"2024-09-17T02:40:52.243763+0000","last_became_peered":"2024-09-17T02:40:52.243763+0000","last_unstale":"2024-09-17T02:40:57.472176+0000","last_undegraded":"2024-09-17T02:40:57.472176+0000","last_fullsized":"2024-09-17T02:40:57.472176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:32.662763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471344+0000","last_change":"2024-09-17T02:40:51.234151+0000","last_active":"2024-09-17T02:40:57.471344+0000","last_peered":"2024-09-17T02:40:57.471344+0000","last_clean":"2024-09-17T02:40:57.471344+0000","last_became_active":"2024-09-17T02:40:51.233731+0000","last_became_peered":"2024-09-17T02:40:51.233731+0000","last_unstale":"2024-09-17T02:40:57.471344+0000","last_undegraded":"2024-09-17T02:40:57.471344+0000","last_fullsized":"2024-09-17T02:40:57.471344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:15.349994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290693+0000","last_change":"2024-09-17T02:40:56.334710+0000","last_active":"2024-09-17T02:40:57.290693+0000","last_peered":"2024-09-17T02:40:57.290693+0000","last_clean":"2024-09-17T02:40:57.290693+0000","last_became_active":"2024-09-17T02:40:56.333630+0000","last_became_peered":"2024-09-17T02:40:56.333630+0000","last_unstale":"2024-09-17T02:40:57.290693+0000","last_undegraded":"2024-09-17T02:40:57.290693+0000","last_fullsized":"2024-09-17T02:40:57.290693+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:07.834564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.290681+0000","last_change":"2024-09-17T02:40:56.321729+0000","last_active":"2024-09-17T02:40:57.290681+0000","last_peered":"2024-09-17T02:40:57.290681+0000","last_clean":"2024-09-17T02:40:57.290681+0000","last_became_active":"2024-09-17T02:40:56.320583+0000","last_became_peered":"2024-09-17T02:40:56.320583+0000","last_unstale":"2024-09-17T02:40:57.290681+0000","last_undegraded":"2024-09-17T02:40:57.290681+0000","last_fullsized":"2024-09-17T02:40:57.290681+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:50.707848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.291114+0000","last_change":"2024-09-17T02:40:52.224867+0000","last_active":"2024-09-17T02:40:57.291114+0000","last_peered":"2024-09-17T02:40:57.291114+0000","last_clean":"2024-09-17T02:40:57.291114+0000","last_became_active":"2024-09-17T02:40:52.224558+0000","last_became_peered":"2024-09-17T02:40:52.224558+0000","last_unstale":"2024-09-17T02:40:57.291114+0000","last_undegraded":"2024-09-17T02:40:57.291114+0000","last_fullsized":"2024-09-17T02:40:57.291114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:01.098205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287555+0000","last_change":"2024-09-17T02:40:51.210128+0000","last_active":"2024-09-17T02:40:57.287555+0000","last_peered":"2024-09-17T02:40:57.287555+0000","last_clean":"2024-09-17T02:40:57.287555+0000","last_became_active":"2024-09-17T02:40:51.209655+0000","last_became_peered":"2024-09-17T02:40:51.209655+0000","last_unstale":"2024-09-17T02:40:57.287555+0000","last_undegraded":"2024-09-17T02:40:57.287555+0000","last_fullsized":"2024-09-17T02:40:57.287555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:18.053396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288450+0000","last_change":"2024-09-17T02:40:56.316501+0000","last_active":"2024-09-17T02:40:57.288450+0000","last_peered":"2024-09-17T02:40:57.288450+0000","last_clean":"2024-09-17T02:40:57.288450+0000","last_became_active":"2024-09-17T02:40:56.315839+0000","last_became_peered":"2024-09-17T02:40:56.315839+0000","last_unstale":"2024-09-17T02:40:57.288450+0000","last_undegraded":"2024-09-17T02:40:57.288450+0000","last_fullsized":"2024-09-17T02:40:57.288450+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:21.657805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472377+0000","last_change":"2024-09-17T02:40:52.228669+0000","last_active":"2024-09-17T02:40:57.472377+0000","last_peered":"2024-09-17T02:40:57.472377+0000","last_clean":"2024-09-17T02:40:57.472377+0000","last_became_active":"2024-09-17T02:40:52.226236+0000","last_became_peered":"2024-09-17T02:40:52.226236+0000","last_unstale":"2024-09-17T02:40:57.472377+0000","last_undegraded":"2024-09-17T02:40:57.472377+0000","last_fullsized":"2024-09-17T02:40:57.472377+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:41.461773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287139+0000","last_change":"2024-09-17T02:40:51.226679+0000","last_active":"2024-09-17T02:40:57.287139+0000","last_peered":"2024-09-17T02:40:57.287139+0000","last_clean":"2024-09-17T02:40:57.287139+0000","last_became_active":"2024-09-17T02:40:51.225794+0000","last_became_peered":"2024-09-17T02:40:51.225794+0000","last_unstale":"2024-09-17T02:40:57.287139+0000","last_undegraded":"2024-09-17T02:40:57.287139+0000","last_fullsized":"2024-09-17T02:40:57.287139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:37.666440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":24,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288775+0000","last_change":"2024-09-17T02:40:51.226491+0000","last_active":"2024-09-17T02:40:57.288775+0000","last_peered":"2024-09-17T02:40:57.288775+0000","last_clean":"2024-09-17T02:40:57.288775+0000","last_became_active":"2024-09-17T02:40:51.225695+0000","last_became_peered":"2024-09-17T02:40:51.225695+0000","last_unstale":"2024-09-17T02:40:57.288775+0000","last_undegraded":"2024-09-17T02:40:57.288775+0000","last_fullsized":"2024-09-17T02:40:57.288775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:46.966810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288252+0000","last_change":"2024-09-17T02:40:52.222794+0000","last_active":"2024-09-17T02:40:57.288252+0000","last_peered":"2024-09-17T02:40:57.288252+0000","last_clean":"2024-09-17T02:40:57.288252+0000","last_became_active":"2024-09-17T02:40:52.221682+0000","last_became_peered":"2024-09-17T02:40:52.221682+0000","last_unstale":"2024-09-17T02:40:57.288252+0000","last_undegraded":"2024-09-17T02:40:57.288252+0000","last_fullsized":"2024-09-17T02:40:57.288252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:34.058125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.292523+0000","last_change":"2024-09-17T02:40:56.343317+0000","last_active":"2024-09-17T02:40:57.292523+0000","last_peered":"2024-09-17T02:40:57.292523+0000","last_clean":"2024-09-17T02:40:57.292523+0000","last_became_active":"2024-09-17T02:40:56.342373+0000","last_became_peered":"2024-09-17T02:40:56.342373+0000","last_unstale":"2024-09-17T02:40:57.292523+0000","last_undegraded":"2024-09-17T02:40:57.292523+0000","last_fullsized":"2024-09-17T02:40:57.292523+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:36:02.210967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473874+0000","last_change":"2024-09-17T02:40:51.237550+0000","last_active":"2024-09-17T02:40:57.473874+0000","last_peered":"2024-09-17T02:40:57.473874+0000","last_clean":"2024-09-17T02:40:57.473874+0000","last_became_active":"2024-09-17T02:40:51.236422+0000","last_became_peered":"2024-09-17T02:40:51.236422+0000","last_unstale":"2024-09-17T02:40:57.473874+0000","last_undegraded":"2024-09-17T02:40:57.473874+0000","last_fullsized":"2024-09-17T02:40:57.473874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:50.167555+0000","last_clean_scrub_stamp":"2024-09-17T02:40:50.167555+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:00.133199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470700+0000","last_change":"2024-09-17T02:40:52.226397+0000","last_active":"2024-09-17T02:40:57.470700+0000","last_peered":"2024-09-17T02:40:57.470700+0000","last_clean":"2024-09-17T02:40:57.470700+0000","last_became_active":"2024-09-17T02:40:52.225934+0000","last_became_peered":"2024-09-17T02:40:52.225934+0000","last_unstale":"2024-09-17T02:40:57.470700+0000","last_undegraded":"2024-09-17T02:40:57.470700+0000","last_fullsized":"2024-09-17T02:40:57.470700+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:17:37.421483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472110+0000","last_change":"2024-09-17T02:40:56.336996+0000","last_active":"2024-09-17T02:40:57.472110+0000","last_peered":"2024-09-17T02:40:57.472110+0000","last_clean":"2024-09-17T02:40:57.472110+0000","last_became_active":"2024-09-17T02:40:56.336172+0000","last_became_peered":"2024-09-17T02:40:56.336172+0000","last_unstale":"2024-09-17T02:40:57.472110+0000","last_undegraded":"2024-09-17T02:40:57.472110+0000","last_fullsized":"2024-09-17T02:40:57.472110+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:55.257459+0000","last_clean_scrub_stamp":"2024-09-17T02:40:55.257459+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:05.177214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470197+0000","last_change":"2024-09-17T02:40:52.232046+0000","last_active":"2024-09-17T02:40:57.470197+0000","last_peered":"2024-09-17T02:40:57.470197+0000","last_clean":"2024-09-17T02:40:57.470197+0000","last_became_active":"2024-09-17T02:40:52.231700+0000","last_became_peered":"2024-09-17T02:40:52.231700+0000","last_unstale":"2024-09-17T02:40:57.470197+0000","last_undegraded":"2024-09-17T02:40:57.470197+0000","last_fullsized":"2024-09-17T02:40:57.470197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:47.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472501+0000","last_change":"2024-09-17T02:40:52.239198+0000","last_active":"2024-09-17T02:40:57.472501+0000","last_peered":"2024-09-17T02:40:57.472501+0000","last_clean":"2024-09-17T02:40:57.472501+0000","last_became_active":"2024-09-17T02:40:52.238901+0000","last_became_peered":"2024-09-17T02:40:52.238901+0000","last_unstale":"2024-09-17T02:40:57.472501+0000","last_undegraded":"2024-09-17T02:40:57.472501+0000","last_fullsized":"2024-09-17T02:40:57.472501+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:47.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287639+0000","last_change":"2024-09-17T02:40:52.228993+0000","last_active":"2024-09-17T02:40:57.287639+0000","last_peered":"2024-09-17T02:40:57.287639+0000","last_clean":"2024-09-17T02:40:57.287639+0000","last_became_active":"2024-09-17T02:40:52.228574+0000","last_became_peered":"2024-09-17T02:40:52.228574+0000","last_unstale":"2024-09-17T02:40:57.287639+0000","last_undegraded":"2024-09-17T02:40:57.287639+0000","last_fullsized":"2024-09-17T02:40:57.287639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:31.798195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473564+0000","last_change":"2024-09-17T02:40:52.222569+0000","last_active":"2024-09-17T02:40:57.473564+0000","last_peered":"2024-09-17T02:40:57.473564+0000","last_clean":"2024-09-17T02:40:57.473564+0000","last_became_active":"2024-09-17T02:40:52.222228+0000","last_became_peered":"2024-09-17T02:40:52.222228+0000","last_unstale":"2024-09-17T02:40:57.473564+0000","last_undegraded":"2024-09-17T02:40:57.473564+0000","last_fullsized":"2024-09-17T02:40:57.473564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:39.361971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.289288+0000","last_change":"2024-09-17T02:40:52.236488+0000","last_active":"2024-09-17T02:40:57.289288+0000","last_peered":"2024-09-17T02:40:57.289288+0000","last_clean":"2024-09-17T02:40:57.289288+0000","last_became_active":"2024-09-17T02:40:52.236171+0000","last_became_peered":"2024-09-17T02:40:52.236171+0000","last_unstale":"2024-09-17T02:40:57.289288+0000","last_undegraded":"2024-09-17T02:40:57.289288+0000","last_fullsized":"2024-09-17T02:40:57.289288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:08.415078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287712+0000","last_change":"2024-09-17T02:40:52.217993+0000","last_active":"2024-09-17T02:40:57.287712+0000","last_peered":"2024-09-17T02:40:57.287712+0000","last_clean":"2024-09-17T02:40:57.287712+0000","last_became_active":"2024-09-17T02:40:52.216716+0000","last_became_peered":"2024-09-17T02:40:52.216716+0000","last_unstale":"2024-09-17T02:40:57.287712+0000","last_undegraded":"2024-09-17T02:40:57.287712+0000","last_fullsized":"2024-09-17T02:40:57.287712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:06.000439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472031+0000","last_change":"2024-09-17T02:40:52.245158+0000","last_active":"2024-09-17T02:40:57.472031+0000","last_peered":"2024-09-17T02:40:57.472031+0000","last_clean":"2024-09-17T02:40:57.472031+0000","last_became_active":"2024-09-17T02:40:52.244333+0000","last_became_peered":"2024-09-17T02:40:52.244333+0000","last_unstale":"2024-09-17T02:40:57.472031+0000","last_undegraded":"2024-09-17T02:40:57.472031+0000","last_fullsized":"2024-09-17T02:40:57.472031+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:06.006731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473931+0000","last_change":"2024-09-17T02:40:52.228925+0000","last_active":"2024-09-17T02:40:57.473931+0000","last_peered":"2024-09-17T02:40:57.473931+0000","last_clean":"2024-09-17T02:40:57.473931+0000","last_became_active":"2024-09-17T02:40:52.228587+0000","last_became_peered":"2024-09-17T02:40:52.228587+0000","last_unstale":"2024-09-17T02:40:57.473931+0000","last_undegraded":"2024-09-17T02:40:57.473931+0000","last_fullsized":"2024-09-17T02:40:57.473931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:33.958554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287342+0000","last_change":"2024-09-17T02:40:52.229795+0000","last_active":"2024-09-17T02:40:57.287342+0000","last_peered":"2024-09-17T02:40:57.287342+0000","last_clean":"2024-09-17T02:40:57.287342+0000","last_became_active":"2024-09-17T02:40:52.227659+0000","last_became_peered":"2024-09-17T02:40:52.227659+0000","last_unstale":"2024-09-17T02:40:57.287342+0000","last_undegraded":"2024-09-17T02:40:57.287342+0000","last_fullsized":"2024-09-17T02:40:57.287342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:36.541471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.473327+0000","last_change":"2024-09-17T02:40:52.221420+0000","last_active":"2024-09-17T02:40:57.473327+0000","last_peered":"2024-09-17T02:40:57.473327+0000","last_clean":"2024-09-17T02:40:57.473327+0000","last_became_active":"2024-09-17T02:40:52.221050+0000","last_became_peered":"2024-09-17T02:40:52.221050+0000","last_unstale":"2024-09-17T02:40:57.473327+0000","last_undegraded":"2024-09-17T02:40:57.473327+0000","last_fullsized":"2024-09-17T02:40:57.473327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:47.715501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470588+0000","last_change":"2024-09-17T02:40:52.237946+0000","last_active":"2024-09-17T02:40:57.470588+0000","last_peered":"2024-09-17T02:40:57.470588+0000","last_clean":"2024-09-17T02:40:57.470588+0000","last_became_active":"2024-09-17T02:40:52.237376+0000","last_became_peered":"2024-09-17T02:40:52.237376+0000","last_unstale":"2024-09-17T02:40:57.470588+0000","last_undegraded":"2024-09-17T02:40:57.470588+0000","last_fullsized":"2024-09-17T02:40:57.470588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:31.107108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287955+0000","last_change":"2024-09-17T02:40:52.221072+0000","last_active":"2024-09-17T02:40:57.287955+0000","last_peered":"2024-09-17T02:40:57.287955+0000","last_clean":"2024-09-17T02:40:57.287955+0000","last_became_active":"2024-09-17T02:40:52.220697+0000","last_became_peered":"2024-09-17T02:40:52.220697+0000","last_unstale":"2024-09-17T02:40:57.287955+0000","last_undegraded":"2024-09-17T02:40:57.287955+0000","last_fullsized":"2024-09-17T02:40:57.287955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:02.681569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472298+0000","last_change":"2024-09-17T02:40:52.249011+0000","last_active":"2024-09-17T02:40:57.472298+0000","last_peered":"2024-09-17T02:40:57.472298+0000","last_clean":"2024-09-17T02:40:57.472298+0000","last_became_active":"2024-09-17T02:40:52.247429+0000","last_became_peered":"2024-09-17T02:40:52.247429+0000","last_unstale":"2024-09-17T02:40:57.472298+0000","last_undegraded":"2024-09-17T02:40:57.472298+0000","last_fullsized":"2024-09-17T02:40:57.472298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:56.223383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288383+0000","last_change":"2024-09-17T02:40:52.243578+0000","last_active":"2024-09-17T02:40:57.288383+0000","last_peered":"2024-09-17T02:40:57.288383+0000","last_clean":"2024-09-17T02:40:57.288383+0000","last_became_active":"2024-09-17T02:40:52.243106+0000","last_became_peered":"2024-09-17T02:40:52.243106+0000","last_unstale":"2024-09-17T02:40:57.288383+0000","last_undegraded":"2024-09-17T02:40:57.288383+0000","last_fullsized":"2024-09-17T02:40:57.288383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:22.663615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.475176+0000","last_change":"2024-09-17T02:40:52.222351+0000","last_active":"2024-09-17T02:40:57.475176+0000","last_peered":"2024-09-17T02:40:57.475176+0000","last_clean":"2024-09-17T02:40:57.475176+0000","last_became_active":"2024-09-17T02:40:52.222043+0000","last_became_peered":"2024-09-17T02:40:52.222043+0000","last_unstale":"2024-09-17T02:40:57.475176+0000","last_undegraded":"2024-09-17T02:40:57.475176+0000","last_fullsized":"2024-09-17T02:40:57.475176+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:21.583058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288320+0000","last_change":"2024-09-17T02:40:52.256413+0000","last_active":"2024-09-17T02:40:57.288320+0000","last_peered":"2024-09-17T02:40:57.288320+0000","last_clean":"2024-09-17T02:40:57.288320+0000","last_became_active":"2024-09-17T02:40:52.256035+0000","last_became_peered":"2024-09-17T02:40:52.256035+0000","last_unstale":"2024-09-17T02:40:57.288320+0000","last_undegraded":"2024-09-17T02:40:57.288320+0000","last_fullsized":"2024-09-17T02:40:57.288320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:06:59.172988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287206+0000","last_change":"2024-09-17T02:40:52.236686+0000","last_active":"2024-09-17T02:40:57.287206+0000","last_peered":"2024-09-17T02:40:57.287206+0000","last_clean":"2024-09-17T02:40:57.287206+0000","last_became_active":"2024-09-17T02:40:52.236194+0000","last_became_peered":"2024-09-17T02:40:52.236194+0000","last_unstale":"2024-09-17T02:40:57.287206+0000","last_undegraded":"2024-09-17T02:40:57.287206+0000","last_fullsized":"2024-09-17T02:40:57.287206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:55.015415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472841+0000","last_change":"2024-09-17T02:40:52.228032+0000","last_active":"2024-09-17T02:40:57.472841+0000","last_peered":"2024-09-17T02:40:57.472841+0000","last_clean":"2024-09-17T02:40:57.472841+0000","last_became_active":"2024-09-17T02:40:52.225817+0000","last_became_peered":"2024-09-17T02:40:52.225817+0000","last_unstale":"2024-09-17T02:40:57.472841+0000","last_undegraded":"2024-09-17T02:40:57.472841+0000","last_fullsized":"2024-09-17T02:40:57.472841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:34.058125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.287107+0000","last_change":"2024-09-17T02:40:52.217145+0000","last_active":"2024-09-17T02:40:57.287107+0000","last_peered":"2024-09-17T02:40:57.287107+0000","last_clean":"2024-09-17T02:40:57.287107+0000","last_became_active":"2024-09-17T02:40:52.215909+0000","last_became_peered":"2024-09-17T02:40:52.215909+0000","last_unstale":"2024-09-17T02:40:57.287107+0000","last_undegraded":"2024-09-17T02:40:57.287107+0000","last_fullsized":"2024-09-17T02:40:57.287107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:28.728656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.288943+0000","last_change":"2024-09-17T02:40:52.224668+0000","last_active":"2024-09-17T02:40:57.288943+0000","last_peered":"2024-09-17T02:40:57.288943+0000","last_clean":"2024-09-17T02:40:57.288943+0000","last_became_active":"2024-09-17T02:40:52.224273+0000","last_became_peered":"2024-09-17T02:40:52.224273+0000","last_unstale":"2024-09-17T02:40:57.288943+0000","last_undegraded":"2024-09-17T02:40:57.288943+0000","last_fullsized":"2024-09-17T02:40:57.288943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:06:21.184527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472191+0000","last_change":"2024-09-17T02:40:52.252355+0000","last_active":"2024-09-17T02:40:57.472191+0000","last_peered":"2024-09-17T02:40:57.472191+0000","last_clean":"2024-09-17T02:40:57.472191+0000","last_became_active":"2024-09-17T02:40:52.251950+0000","last_became_peered":"2024-09-17T02:40:52.251950+0000","last_unstale":"2024-09-17T02:40:57.472191+0000","last_undegraded":"2024-09-17T02:40:57.472191+0000","last_fullsized":"2024-09-17T02:40:57.472191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:50.916246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471773+0000","last_change":"2024-09-17T02:40:52.244189+0000","last_active":"2024-09-17T02:40:57.471773+0000","last_peered":"2024-09-17T02:40:57.471773+0000","last_clean":"2024-09-17T02:40:57.471773+0000","last_became_active":"2024-09-17T02:40:52.243891+0000","last_became_peered":"2024-09-17T02:40:52.243891+0000","last_unstale":"2024-09-17T02:40:57.471773+0000","last_undegraded":"2024-09-17T02:40:57.471773+0000","last_fullsized":"2024-09-17T02:40:57.471773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:59:00.308180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471829+0000","last_change":"2024-09-17T02:40:52.237299+0000","last_active":"2024-09-17T02:40:57.471829+0000","last_peered":"2024-09-17T02:40:57.471829+0000","last_clean":"2024-09-17T02:40:57.471829+0000","last_became_active":"2024-09-17T02:40:52.236928+0000","last_became_peered":"2024-09-17T02:40:52.236928+0000","last_unstale":"2024-09-17T02:40:57.471829+0000","last_undegraded":"2024-09-17T02:40:57.471829+0000","last_fullsized":"2024-09-17T02:40:57.471829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:03.755555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471259+0000","last_change":"2024-09-17T02:40:52.247989+0000","last_active":"2024-09-17T02:40:57.471259+0000","last_peered":"2024-09-17T02:40:57.471259+0000","last_clean":"2024-09-17T02:40:57.471259+0000","last_became_active":"2024-09-17T02:40:52.247534+0000","last_became_peered":"2024-09-17T02:40:52.247534+0000","last_unstale":"2024-09-17T02:40:57.471259+0000","last_undegraded":"2024-09-17T02:40:57.471259+0000","last_fullsized":"2024-09-17T02:40:57.471259+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:22:14.502507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471636+0000","last_change":"2024-09-17T02:40:52.228868+0000","last_active":"2024-09-17T02:40:57.471636+0000","last_peered":"2024-09-17T02:40:57.471636+0000","last_clean":"2024-09-17T02:40:57.471636+0000","last_became_active":"2024-09-17T02:40:52.227927+0000","last_became_peered":"2024-09-17T02:40:52.227927+0000","last_unstale":"2024-09-17T02:40:57.471636+0000","last_undegraded":"2024-09-17T02:40:57.471636+0000","last_fullsized":"2024-09-17T02:40:57.471636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:12:06.283756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.470545+0000","last_change":"2024-09-17T02:40:52.220236+0000","last_active":"2024-09-17T02:40:57.470545+0000","last_peered":"2024-09-17T02:40:57.470545+0000","last_clean":"2024-09-17T02:40:57.470545+0000","last_became_active":"2024-09-17T02:40:52.219510+0000","last_became_peered":"2024-09-17T02:40:52.219510+0000","last_unstale":"2024-09-17T02:40:57.470545+0000","last_undegraded":"2024-09-17T02:40:57.470545+0000","last_fullsized":"2024-09-17T02:40:57.470545+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:33.958554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471108+0000","last_change":"2024-09-17T02:40:52.218799+0000","last_active":"2024-09-17T02:40:57.471108+0000","last_peered":"2024-09-17T02:40:57.471108+0000","last_clean":"2024-09-17T02:40:57.471108+0000","last_became_active":"2024-09-17T02:40:52.217054+0000","last_became_peered":"2024-09-17T02:40:52.217054+0000","last_unstale":"2024-09-17T02:40:57.471108+0000","last_undegraded":"2024-09-17T02:40:57.471108+0000","last_fullsized":"2024-09-17T02:40:57.471108+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:32:21.583058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472762+0000","last_change":"2024-09-17T02:40:52.232583+0000","last_active":"2024-09-17T02:40:57.472762+0000","last_peered":"2024-09-17T02:40:57.472762+0000","last_clean":"2024-09-17T02:40:57.472762+0000","last_became_active":"2024-09-17T02:40:52.231933+0000","last_became_peered":"2024-09-17T02:40:52.231933+0000","last_unstale":"2024-09-17T02:40:57.472762+0000","last_undegraded":"2024-09-17T02:40:57.472762+0000","last_fullsized":"2024-09-17T02:40:57.472762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:41.461773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471574+0000","last_change":"2024-09-17T02:40:52.218335+0000","last_active":"2024-09-17T02:40:57.471574+0000","last_peered":"2024-09-17T02:40:57.471574+0000","last_clean":"2024-09-17T02:40:57.471574+0000","last_became_active":"2024-09-17T02:40:52.217008+0000","last_became_peered":"2024-09-17T02:40:52.217008+0000","last_unstale":"2024-09-17T02:40:57.471574+0000","last_undegraded":"2024-09-17T02:40:57.471574+0000","last_fullsized":"2024-09-17T02:40:57.471574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:19.064290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.472531+0000","last_change":"2024-09-17T02:40:52.224771+0000","last_active":"2024-09-17T02:40:57.472531+0000","last_peered":"2024-09-17T02:40:57.472531+0000","last_clean":"2024-09-17T02:40:57.472531+0000","last_became_active":"2024-09-17T02:40:52.222947+0000","last_became_peered":"2024-09-17T02:40:52.222947+0000","last_unstale":"2024-09-17T02:40:57.472531+0000","last_undegraded":"2024-09-17T02:40:57.472531+0000","last_fullsized":"2024-09-17T02:40:57.472531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:39:54.216006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.474943+0000","last_change":"2024-09-17T02:40:52.228603+0000","last_active":"2024-09-17T02:40:57.474943+0000","last_peered":"2024-09-17T02:40:57.474943+0000","last_clean":"2024-09-17T02:40:57.474943+0000","last_became_active":"2024-09-17T02:40:52.225545+0000","last_became_peered":"2024-09-17T02:40:52.225545+0000","last_unstale":"2024-09-17T02:40:57.474943+0000","last_undegraded":"2024-09-17T02:40:57.474943+0000","last_fullsized":"2024-09-17T02:40:57.474943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:56.223383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-17T02:40:57.471852+0000","last_change":"2024-09-17T02:40:52.250495+0000","last_active":"2024-09-17T02:40:57.471852+0000","last_peered":"2024-09-17T02:40:57.471852+0000","last_clean":"2024-09-17T02:40:57.471852+0000","last_became_active":"2024-09-17T02:40:52.250095+0000","last_became_peered":"2024-09-17T02:40:52.250095+0000","last_unstale":"2024-09-17T02:40:57.471852+0000","last_undegraded":"2024-09-17T02:40:57.471852+0000","last_fullsized":"2024-09-17T02:40:57.471852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:51.178450+0000","last_clean_scrub_stamp":"2024-09-17T02:40:51.178450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:35:58.131959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":696354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509452,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":110408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509450,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":274432,"data_stored":106476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":109736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509452,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":105894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:41:19.682 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T02:41:19.682 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T02:41:19.682 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T02:41:19.682 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T02:41:19.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 -- 172.21.15.131:0/2433718984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c003a40 msgr2=0x7f7a6c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 --2- 172.21.15.131:0/2433718984 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c003a40 0x7f7a6c003eb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00c260 tx=0x7f7a7c01cf20 comp rx=0 tx=0).stop 2024-09-17T02:41:19.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a86b01640 1 -- 172.21.15.131:0/2433718984 reap_dead start 2024-09-17T02:41:19.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 -- 172.21.15.131:0/2433718984 shutdown_connections 2024-09-17T02:41:19.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 -- 172.21.15.131:0/2433718984 >> 172.21.15.131:0/2433718984 conn(0x7f7a8006eea0 msgr2=0x7f7a8006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:19.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 -- 172.21.15.131:0/2433718984 shutdown_connections 2024-09-17T02:41:19.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 -- 172.21.15.131:0/2433718984 wait complete. 2024-09-17T02:41:19.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 Processor -- start 2024-09-17T02:41:19.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 -- start start 2024-09-17T02:41:19.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c003a40 0x7f7a80153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a801f6d40 0x7f7a80158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7a80159ad0 0x7f7a80159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8015c230 con 0x7f7a6c003a40 2024-09-17T02:41:19.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8015c0b0 con 0x7f7a801f6d40 2024-09-17T02:41:19.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a88564640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7a800cc200 con 0x7f7a80159ad0 2024-09-17T02:41:19.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a86300640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c003a40 0x7f7a80153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a86300640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c003a40 0x7f7a80153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58146/0 (socket says 172.21.15.131:58146) 2024-09-17T02:41:19.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a86300640 1 -- 172.21.15.131:0/620491017 learned_addr learned my addr 172.21.15.131:0/620491017 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:19.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a85aff640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a801f6d40 0x7f7a80158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a86b01640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7a80159ad0 0x7f7a80159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a86300640 1 -- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7a80159ad0 msgr2=0x7f7a80159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a86300640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7a80159ad0 0x7f7a80159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a86300640 1 -- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a801f6d40 msgr2=0x7f7a80158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:19.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a86300640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a801f6d40 0x7f7a80158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:19.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.870+0000 7f7a86300640 1 -- 172.21.15.131:0/620491017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a800ceb30 con 0x7f7a6c003a40 2024-09-17T02:41:19.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a86300640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c003a40 0x7f7a80153450 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c01b620 tx=0x7f7a7c001d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:19.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a76ffd640 1 -- 172.21.15.131:0/620491017 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0014c0 con 0x7f7a6c003a40 2024-09-17T02:41:19.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a76ffd640 1 -- 172.21.15.131:0/620491017 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a7c001660 con 0x7f7a6c003a40 2024-09-17T02:41:19.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a88564640 1 -- 172.21.15.131:0/620491017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a801575f0 con 0x7f7a6c003a40 2024-09-17T02:41:19.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a76ffd640 1 -- 172.21.15.131:0/620491017 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c006330 con 0x7f7a6c003a40 2024-09-17T02:41:19.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a88564640 1 -- 172.21.15.131:0/620491017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a80157890 con 0x7f7a6c003a40 2024-09-17T02:41:19.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a88564640 1 -- 172.21.15.131:0/620491017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a80075b80 con 0x7f7a6c003a40 2024-09-17T02:41:19.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a76ffd640 1 -- 172.21.15.131:0/620491017 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7a7c0a5c70 con 0x7f7a6c003a40 2024-09-17T02:41:19.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a76ffd640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a64073230 0x7f7a64075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:19.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a76ffd640 1 -- 172.21.15.131:0/620491017 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7a7c12db90 con 0x7f7a6c003a40 2024-09-17T02:41:19.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a85aff640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a64073230 0x7f7a64075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:19.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.874+0000 7f7a85aff640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a64073230 0x7f7a64075740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a70005fd0 tx=0x7f7a70005ec0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:19.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:19.878+0000 7f7a76ffd640 1 -- 172.21.15.131:0/620491017 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7a7c016cd0 con 0x7f7a6c003a40 2024-09-17T02:41:20.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.066+0000 7f7a88564640 1 -- 172.21.15.131:0/620491017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7a8006dea0 con 0x7f7a6c003a40 2024-09-17T02:41:20.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.066+0000 7f7a76ffd640 1 -- 172.21.15.131:0/620491017 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f7a7c0f89e0 con 0x7f7a6c003a40 2024-09-17T02:41:20.068 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:20.068 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T02:41:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 -- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a64073230 msgr2=0x7f7a64075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a64073230 0x7f7a64075740 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7a70005fd0 tx=0x7f7a70005ec0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 -- 172.21.15.131:0/620491017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c003a40 msgr2=0x7f7a80153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c003a40 0x7f7a80153450 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c01b620 tx=0x7f7a7c001d70 comp rx=0 tx=0).stop 2024-09-17T02:41:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 -- 172.21.15.131:0/620491017 shutdown_connections 2024-09-17T02:41:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a64073230 0x7f7a64075740 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7a80159ad0 0x7f7a80159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7a801f6d40 0x7f7a80158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 --2- 172.21.15.131:0/620491017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7a6c003a40 0x7f7a80153450 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 -- 172.21.15.131:0/620491017 >> 172.21.15.131:0/620491017 conn(0x7f7a8006eea0 msgr2=0x7f7a800cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:20.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 -- 172.21.15.131:0/620491017 shutdown_connections 2024-09-17T02:41:20.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.070+0000 7f7a88564640 1 -- 172.21.15.131:0/620491017 wait complete. 2024-09-17T02:41:20.088 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T02:41:20.088 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:41:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.238+0000 7f19e04d2640 1 -- 172.21.15.131:0/907738214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d8162220 msgr2=0x7f19d8162630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.238+0000 7f19e04d2640 1 --2- 172.21.15.131:0/907738214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d8162220 0x7f19d8162630 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f19d80b51c0 tx=0x7f19c801c840 comp rx=0 tx=0).stop 2024-09-17T02:41:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.238+0000 7f19e04d2640 1 -- 172.21.15.131:0/907738214 shutdown_connections 2024-09-17T02:41:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.238+0000 7f19e04d2640 1 --2- 172.21.15.131:0/907738214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d8162220 0x7f19d8162630 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.238+0000 7f19e04d2640 1 --2- 172.21.15.131:0/907738214 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19d815ec30 0x7f19d815f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.238+0000 7f19e04d2640 1 --2- 172.21.15.131:0/907738214 >> v2:172.21.15.189:3300/0 conn(0x7f19d80ccdf0 0x7f19d80cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.238+0000 7f19e04d2640 1 -- 172.21.15.131:0/907738214 >> 172.21.15.131:0/907738214 conn(0x7f19d80d2100 msgr2=0x7f19d80d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.238+0000 7f19e04d2640 1 -- 172.21.15.131:0/907738214 shutdown_connections 2024-09-17T02:41:20.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.238+0000 7f19e04d2640 1 -- 172.21.15.131:0/907738214 wait complete. 2024-09-17T02:41:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19e04d2640 1 Processor -- start 2024-09-17T02:41:20.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19e04d2640 1 -- start start 2024-09-17T02:41:20.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19e04d2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d80ccdf0 0x7f19d8158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19e04d2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19d815ec30 0x7f19d81f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19e04d2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19d8162220 0x7f19d81f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19e04d2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f19d80b7cd0 con 0x7f19d80ccdf0 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19e04d2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f19d80b7b50 con 0x7f19d815ec30 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19e04d2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f19d80b7e50 con 0x7f19d8162220 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19de26e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d80ccdf0 0x7f19d8158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19de26e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d80ccdf0 0x7f19d8158170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58166/0 (socket says 172.21.15.131:58166) 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19de26e640 1 -- 172.21.15.131:0/2366872299 learned_addr learned my addr 172.21.15.131:0/2366872299 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19dda6d640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19d815ec30 0x7f19d81f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19dea6f640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19d8162220 0x7f19d81f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19de26e640 1 -- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19d8162220 msgr2=0x7f19d81f95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19de26e640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19d8162220 0x7f19d81f95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19de26e640 1 -- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19d815ec30 msgr2=0x7f19d81f90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19de26e640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19d815ec30 0x7f19d81f90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19de26e640 1 -- 172.21.15.131:0/2366872299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d80c8c90 con 0x7f19d80ccdf0 2024-09-17T02:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19de26e640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d80ccdf0 0x7f19d8158170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19d400c940 tx=0x7f19d400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19ceffd640 1 -- 172.21.15.131:0/2366872299 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d4007bb0 con 0x7f19d80ccdf0 2024-09-17T02:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19ceffd640 1 -- 172.21.15.131:0/2366872299 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d4007e40 con 0x7f19d80ccdf0 2024-09-17T02:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19e04d2640 1 -- 172.21.15.131:0/2366872299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d80c6530 con 0x7f19d80ccdf0 2024-09-17T02:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19ceffd640 1 -- 172.21.15.131:0/2366872299 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d400f690 con 0x7f19d80ccdf0 2024-09-17T02:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19e04d2640 1 -- 172.21.15.131:0/2366872299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d80c6250 con 0x7f19d80ccdf0 2024-09-17T02:41:20.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19e04d2640 1 -- 172.21.15.131:0/2366872299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d815f040 con 0x7f19d80ccdf0 2024-09-17T02:41:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.242+0000 7f19ceffd640 1 -- 172.21.15.131:0/2366872299 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f19d400f830 con 0x7f19d80ccdf0 2024-09-17T02:41:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.246+0000 7f19ceffd640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073230 0x7f19bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.246+0000 7f19ceffd640 1 -- 172.21.15.131:0/2366872299 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f19d4094cd0 con 0x7f19d80ccdf0 2024-09-17T02:41:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.246+0000 7f19dda6d640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073230 0x7f19bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:20.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.246+0000 7f19dda6d640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073230 0x7f19bc075740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19c80af000 tx=0x7f19c801b0e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:20.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.250+0000 7f19ceffd640 1 -- 172.21.15.131:0/2366872299 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f19d80c8c90 con 0x7f19d80ccdf0 2024-09-17T02:41:20.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.446+0000 7f19e04d2640 1 -- 172.21.15.131:0/2366872299 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19d80ce0b0 con 0x7f19d80ccdf0 2024-09-17T02:41:20.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.446+0000 7f19ceffd640 1 -- 172.21.15.131:0/2366872299 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f19d4014030 con 0x7f19d80ccdf0 2024-09-17T02:41:20.447 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:20.447 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"btime":"2024-09-17T02:41:13:211310+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3035404898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3035404898},{"type":"v1","addr":"172.21.15.131:6835","nonce":3035404898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3147445846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3147445846},{"type":"v1","addr":"172.21.15.189:6835","nonce":3147445846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:40:51.611225+0000","modified":"2024-09-17T02:41:12.632560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/1582109479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1582109479},{"type":"v1","addr":"172.21.15.189:6837","nonce":1582109479}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T02:41:20.447 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-09-17T02:41:20.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 -- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073230 msgr2=0x7f19bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073230 0x7f19bc075740 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19c80af000 tx=0x7f19c801b0e0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 -- 172.21.15.131:0/2366872299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d80ccdf0 msgr2=0x7f19d8158170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d80ccdf0 0x7f19d8158170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19d400c940 tx=0x7f19d400ce00 comp rx=0 tx=0).stop 2024-09-17T02:41:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 -- 172.21.15.131:0/2366872299 shutdown_connections 2024-09-17T02:41:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073230 0x7f19bc075740 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19d8162220 0x7f19d81f95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19d815ec30 0x7f19d81f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 --2- 172.21.15.131:0/2366872299 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d80ccdf0 0x7f19d8158170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 -- 172.21.15.131:0/2366872299 >> 172.21.15.131:0/2366872299 conn(0x7f19d80d2100 msgr2=0x7f19d815c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 -- 172.21.15.131:0/2366872299 shutdown_connections 2024-09-17T02:41:20.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.450+0000 7f19e04d2640 1 -- 172.21.15.131:0/2366872299 wait complete. 2024-09-17T02:41:20.467 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:41:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.618+0000 7f7b3169c640 1 -- 172.21.15.131:0/1123904941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b18003a40 msgr2=0x7f7b18003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.618+0000 7f7b3169c640 1 --2- 172.21.15.131:0/1123904941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b18003a40 0x7f7b18003eb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c009ee0 tx=0x7f7b1c01d5f0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.618+0000 7f7b2b7fe640 1 -- 172.21.15.131:0/1123904941 reap_dead start 2024-09-17T02:41:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.618+0000 7f7b3169c640 1 -- 172.21.15.131:0/1123904941 shutdown_connections 2024-09-17T02:41:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.618+0000 7f7b3169c640 1 -- 172.21.15.131:0/1123904941 >> 172.21.15.131:0/1123904941 conn(0x7f7b2c0d1130 msgr2=0x7f7b2c0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.618+0000 7f7b3169c640 1 -- 172.21.15.131:0/1123904941 shutdown_connections 2024-09-17T02:41:20.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.618+0000 7f7b3169c640 1 -- 172.21.15.131:0/1123904941 wait complete. 2024-09-17T02:41:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b3169c640 1 Processor -- start 2024-09-17T02:41:20.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b3169c640 1 -- start start 2024-09-17T02:41:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b3169c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b18003a40 0x7f7b2c144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b3169c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b2c1656f0 0x7f7b2c15ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b3169c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7b2c15b250 0x7f7b2c141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b3169c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c16a8b0 con 0x7f7b18003a40 2024-09-17T02:41:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b3169c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2c16a730 con 0x7f7b2c1656f0 2024-09-17T02:41:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b3169c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2c16aa30 con 0x7f7b2c15b250 2024-09-17T02:41:20.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b18003a40 0x7f7b2c144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b18003a40 0x7f7b2c144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58178/0 (socket says 172.21.15.131:58178) 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2affd640 1 -- 172.21.15.131:0/3279772067 learned_addr learned my addr 172.21.15.131:0/3279772067 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2a7fc640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b2c1656f0 0x7f7b2c15ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2affd640 1 -- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7b2c15b250 msgr2=0x7f7b2c141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2b7fe640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7b2c15b250 0x7f7b2c141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2affd640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7b2c15b250 0x7f7b2c141530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2affd640 1 -- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b2c1656f0 msgr2=0x7f7b2c15ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2affd640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b2c1656f0 0x7f7b2c15ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2affd640 1 -- 172.21.15.131:0/3279772067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2c1f8aa0 con 0x7f7b18003a40 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2a7fc640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b2c1656f0 0x7f7b2c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b2affd640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b18003a40 0x7f7b2c144e20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c01d420 tx=0x7f7b1c01be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:20.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b237fe640 1 -- 172.21.15.131:0/3279772067 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c0ad070 con 0x7f7b18003a40 2024-09-17T02:41:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b237fe640 1 -- 172.21.15.131:0/3279772067 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b1c003c70 con 0x7f7b18003a40 2024-09-17T02:41:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b3169c640 1 -- 172.21.15.131:0/3279772067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2c1f8d80 con 0x7f7b18003a40 2024-09-17T02:41:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b237fe640 1 -- 172.21.15.131:0/3279772067 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c003e60 con 0x7f7b18003a40 2024-09-17T02:41:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.622+0000 7f7b3169c640 1 -- 172.21.15.131:0/3279772067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b2c1f9340 con 0x7f7b18003a40 2024-09-17T02:41:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.626+0000 7f7b3169c640 1 -- 172.21.15.131:0/3279772067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b2c06df40 con 0x7f7b18003a40 2024-09-17T02:41:20.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.626+0000 7f7b237fe640 1 -- 172.21.15.131:0/3279772067 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7b1c0b0070 con 0x7f7b18003a40 2024-09-17T02:41:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.626+0000 7f7b237fe640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b0c073200 0x7f7b0c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.626+0000 7f7b237fe640 1 -- 172.21.15.131:0/3279772067 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7b1c12f680 con 0x7f7b18003a40 2024-09-17T02:41:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.626+0000 7f7b2a7fc640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b0c073200 0x7f7b0c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:20.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.626+0000 7f7b2a7fc640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b0c073200 0x7f7b0c075710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7b14001890 tx=0x7f7b14001c00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:20.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.630+0000 7f7b237fe640 1 -- 172.21.15.131:0/3279772067 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7b1c017300 con 0x7f7b18003a40 2024-09-17T02:41:20.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 -- 172.21.15.131:0/3279772067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b2c15db60 con 0x7f7b18003a40 2024-09-17T02:41:20.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b237fe640 1 -- 172.21.15.131:0/3279772067 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f7b1c0fa510 con 0x7f7b18003a40 2024-09-17T02:41:20.825 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:20.825 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"btime":"2024-09-17T02:41:13:211310+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3035404898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3035404898},{"type":"v1","addr":"172.21.15.131:6835","nonce":3035404898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3147445846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3147445846},{"type":"v1","addr":"172.21.15.189:6835","nonce":3147445846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:40:51.611225+0000","modified":"2024-09-17T02:41:12.632560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/1582109479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1582109479},{"type":"v1","addr":"172.21.15.189:6837","nonce":1582109479}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T02:41:20.825 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-09-17T02:41:20.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 -- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b0c073200 msgr2=0x7f7b0c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b0c073200 0x7f7b0c075710 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7b14001890 tx=0x7f7b14001c00 comp rx=0 tx=0).stop 2024-09-17T02:41:20.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 -- 172.21.15.131:0/3279772067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b18003a40 msgr2=0x7f7b2c144e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b18003a40 0x7f7b2c144e20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c01d420 tx=0x7f7b1c01be30 comp rx=0 tx=0).stop 2024-09-17T02:41:20.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 -- 172.21.15.131:0/3279772067 shutdown_connections 2024-09-17T02:41:20.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b0c073200 0x7f7b0c075710 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7b2c15b250 0x7f7b2c141530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b2c1656f0 0x7f7b2c15ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 --2- 172.21.15.131:0/3279772067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b18003a40 0x7f7b2c144e20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 -- 172.21.15.131:0/3279772067 >> 172.21.15.131:0/3279772067 conn(0x7f7b2c0d1130 msgr2=0x7f7b2c0af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:20.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 -- 172.21.15.131:0/3279772067 shutdown_connections 2024-09-17T02:41:20.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.826+0000 7f7b3169c640 1 -- 172.21.15.131:0/3279772067 wait complete. 2024-09-17T02:41:20.846 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:41:20.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.994+0000 7f93c4bd2640 1 -- 172.21.15.131:0/2253820673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c006e5b0 msgr2=0x7f93c00cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.994+0000 7f93c4bd2640 1 --2- 172.21.15.131:0/2253820673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c006e5b0 0x7f93c00cab60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009c80 tx=0x7f93b801cd60 comp rx=0 tx=0).stop 2024-09-17T02:41:20.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.994+0000 7f93c4bd2640 1 -- 172.21.15.131:0/2253820673 shutdown_connections 2024-09-17T02:41:20.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.994+0000 7f93c4bd2640 1 --2- 172.21.15.131:0/2253820673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c006e5b0 0x7f93c00cab60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.994+0000 7f93c4bd2640 1 --2- 172.21.15.131:0/2253820673 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93c0155240 0x7f93c0073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.994+0000 7f93c4bd2640 1 -- 172.21.15.131:0/2253820673 >> 172.21.15.131:0/2253820673 conn(0x7f93c006eea0 msgr2=0x7f93c006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:20.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.994+0000 7f93c4bd2640 1 -- 172.21.15.131:0/2253820673 shutdown_connections 2024-09-17T02:41:20.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.994+0000 7f93c4bd2640 1 -- 172.21.15.131:0/2253820673 wait complete. 2024-09-17T02:41:20.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.994+0000 7f93c4bd2640 1 Processor -- start 2024-09-17T02:41:20.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.994+0000 7f93c4bd2640 1 -- start start 2024-09-17T02:41:20.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93c4bd2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93c0073970 0x7f93c015f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93c4bd2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c0155240 0x7f93c01485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93c4bd2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c0159c70 0x7f93c015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93c4bd2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f93c00cbf10 con 0x7f93c0155240 2024-09-17T02:41:20.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93c4bd2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93c00cbd90 con 0x7f93c0159c70 2024-09-17T02:41:20.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93c4bd2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f93c0168c80 con 0x7f93c0073970 2024-09-17T02:41:20.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93bdd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c0155240 0x7f93c01485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:20.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93bdd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c0155240 0x7f93c01485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58196/0 (socket says 172.21.15.131:58196) 2024-09-17T02:41:20.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93bdd9b640 1 -- 172.21.15.131:0/602972566 learned_addr learned my addr 172.21.15.131:0/602972566 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93bdd9b640 1 -- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93c0073970 msgr2=0x7f93c015f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93bdd9b640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93c0073970 0x7f93c015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93bdd9b640 1 -- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c0159c70 msgr2=0x7f93c015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93bdd9b640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c0159c70 0x7f93c015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93bdd9b640 1 -- 172.21.15.131:0/602972566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c015a5c0 con 0x7f93c0155240 2024-09-17T02:41:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93bdd9b640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c0155240 0x7f93c01485c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c3c0 tx=0x7f93b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93aeffd640 1 -- 172.21.15.131:0/602972566 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4017070 con 0x7f93c0155240 2024-09-17T02:41:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93aeffd640 1 -- 172.21.15.131:0/602972566 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93b40053c0 con 0x7f93c0155240 2024-09-17T02:41:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93c4bd2640 1 -- 172.21.15.131:0/602972566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c01fc420 con 0x7f93c0155240 2024-09-17T02:41:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93aeffd640 1 -- 172.21.15.131:0/602972566 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b4003860 con 0x7f93c0155240 2024-09-17T02:41:20.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93c4bd2640 1 -- 172.21.15.131:0/602972566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c00c76e0 con 0x7f93c0155240 2024-09-17T02:41:20.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93c4bd2640 1 -- 172.21.15.131:0/602972566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c00cab60 con 0x7f93c0155240 2024-09-17T02:41:20.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93aeffd640 1 -- 172.21.15.131:0/602972566 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f93b401a070 con 0x7f93c0155240 2024-09-17T02:41:20.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93aeffd640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93a0073230 0x7f93a0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:20.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93aeffd640 1 -- 172.21.15.131:0/602972566 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f93b4093830 con 0x7f93c0155240 2024-09-17T02:41:20.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:20.998+0000 7f93be59c640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93a0073230 0x7f93a0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:21.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.002+0000 7f93be59c640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93a0073230 0x7f93a0075740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009690 tx=0x7f93b0006c00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:21.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.006+0000 7f93aeffd640 1 -- 172.21.15.131:0/602972566 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f93c015a5c0 con 0x7f93c0155240 2024-09-17T02:41:21.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.170+0000 7f93c4bd2640 1 -- 172.21.15.131:0/602972566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93c00736e0 con 0x7f93c0155240 2024-09-17T02:41:21.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.170+0000 7f93aeffd640 1 -- 172.21.15.131:0/602972566 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18650 (secure 0 0 0) 0x7f93b405e5a0 con 0x7f93c0155240 2024-09-17T02:41:21.171 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:21.171 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":26,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:40:56.713412+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:41:21.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 -- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93a0073230 msgr2=0x7f93a0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:21.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93a0073230 0x7f93a0075740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009690 tx=0x7f93b0006c00 comp rx=0 tx=0).stop 2024-09-17T02:41:21.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 -- 172.21.15.131:0/602972566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c0155240 msgr2=0x7f93c01485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:21.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c0155240 0x7f93c01485c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c3c0 tx=0x7f93b400c880 comp rx=0 tx=0).stop 2024-09-17T02:41:21.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 -- 172.21.15.131:0/602972566 shutdown_connections 2024-09-17T02:41:21.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93a0073230 0x7f93a0075740 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93c0159c70 0x7f93c015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93c0155240 0x7f93c01485c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 --2- 172.21.15.131:0/602972566 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93c0073970 0x7f93c015f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 -- 172.21.15.131:0/602972566 >> 172.21.15.131:0/602972566 conn(0x7f93c006eea0 msgr2=0x7f93c00b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:21.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 -- 172.21.15.131:0/602972566 shutdown_connections 2024-09-17T02:41:21.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.174+0000 7f93c4bd2640 1 -- 172.21.15.131:0/602972566 wait complete. 2024-09-17T02:41:21.191 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:41:21.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.346+0000 7f7e6f195640 1 -- 172.21.15.131:0/1925087991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e680cab60 msgr2=0x7f7e680caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:21.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.346+0000 7f7e6f195640 1 --2- 172.21.15.131:0/1925087991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e680cab60 0x7f7e680caf70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7e60009c80 tx=0x7f7e6001cd70 comp rx=0 tx=0).stop 2024-09-17T02:41:21.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.346+0000 7f7e6f195640 1 -- 172.21.15.131:0/1925087991 shutdown_connections 2024-09-17T02:41:21.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.346+0000 7f7e6f195640 1 --2- 172.21.15.131:0/1925087991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e680cab60 0x7f7e680caf70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.346+0000 7f7e6f195640 1 --2- 172.21.15.131:0/1925087991 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7e6806dea0 0x7f7e6806e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.346+0000 7f7e6f195640 1 -- 172.21.15.131:0/1925087991 >> 172.21.15.131:0/1925087991 conn(0x7f7e6806eea0 msgr2=0x7f7e6806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:21.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.346+0000 7f7e6f195640 1 -- 172.21.15.131:0/1925087991 shutdown_connections 2024-09-17T02:41:21.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.346+0000 7f7e6f195640 1 -- 172.21.15.131:0/1925087991 wait complete. 2024-09-17T02:41:21.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.346+0000 7f7e6f195640 1 Processor -- start 2024-09-17T02:41:21.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.346+0000 7f7e6f195640 1 -- start start 2024-09-17T02:41:21.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6f195640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e6806dea0 0x7f7e6815a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6f195640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7e680cab60 0x7f7e68159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6f195640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e68155240 0x7f7e68153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6f195640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7e68168430 con 0x7f7e6806dea0 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6f195640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7e681682b0 con 0x7f7e68155240 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6f195640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7e681685b0 con 0x7f7e680cab60 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6cf31640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e6806dea0 0x7f7e6815a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6cf31640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e6806dea0 0x7f7e6815a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52344/0 (socket says 172.21.15.131:52344) 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6cf31640 1 -- 172.21.15.131:0/2490463197 learned_addr learned my addr 172.21.15.131:0/2490463197 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6d732640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e68155240 0x7f7e68153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e67fff640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7e680cab60 0x7f7e68159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6cf31640 1 -- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7e680cab60 msgr2=0x7f7e68159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6cf31640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7e680cab60 0x7f7e68159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6cf31640 1 -- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e68155240 msgr2=0x7f7e68153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:21.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6cf31640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e68155240 0x7f7e68153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6cf31640 1 -- 172.21.15.131:0/2490463197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e680cb180 con 0x7f7e6806dea0 2024-09-17T02:41:21.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6cf31640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e6806dea0 0x7f7e6815a970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c3c0 tx=0x7f7e5800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:21.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e657fa640 1 -- 172.21.15.131:0/2490463197 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58017070 con 0x7f7e6806dea0 2024-09-17T02:41:21.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e657fa640 1 -- 172.21.15.131:0/2490463197 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e580053c0 con 0x7f7e6806dea0 2024-09-17T02:41:21.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e657fa640 1 -- 172.21.15.131:0/2490463197 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58003860 con 0x7f7e6806dea0 2024-09-17T02:41:21.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6f195640 1 -- 172.21.15.131:0/2490463197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6815aeb0 con 0x7f7e6806dea0 2024-09-17T02:41:21.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6f195640 1 -- 172.21.15.131:0/2490463197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e680a9ae0 con 0x7f7e6806dea0 2024-09-17T02:41:21.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e6f195640 1 -- 172.21.15.131:0/2490463197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6806e2b0 con 0x7f7e6806dea0 2024-09-17T02:41:21.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.350+0000 7f7e657fa640 1 -- 172.21.15.131:0/2490463197 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7e5801a070 con 0x7f7e6806dea0 2024-09-17T02:41:21.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.354+0000 7f7e657fa640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e54073200 0x7f7e54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:21.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.354+0000 7f7e657fa640 1 -- 172.21.15.131:0/2490463197 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f7e580939a0 con 0x7f7e6806dea0 2024-09-17T02:41:21.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.354+0000 7f7e67fff640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e54073200 0x7f7e54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:21.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.354+0000 7f7e67fff640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e54073200 0x7f7e54075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0058d0 tx=0x7f7e5c005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:21.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.358+0000 7f7e657fa640 1 -- 172.21.15.131:0/2490463197 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7e6806e7f0 con 0x7f7e6806dea0 2024-09-17T02:41:21.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.562+0000 7f7e6f195640 1 -- 172.21.15.131:0/2490463197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e680b19d0 con 0x7f7e6806dea0 2024-09-17T02:41:21.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.562+0000 7f7e657fa640 1 -- 172.21.15.131:0/2490463197 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f7e5805e600 con 0x7f7e6806dea0 2024-09-17T02:41:21.563 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:21.563 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"btime":"2024-09-17T02:41:13:211310+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3035404898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3035404898},{"type":"v1","addr":"172.21.15.131:6835","nonce":3035404898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3147445846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3147445846},{"type":"v1","addr":"172.21.15.189:6835","nonce":3147445846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:40:51.611225+0000","modified":"2024-09-17T02:41:12.632560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/1582109479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1582109479},{"type":"v1","addr":"172.21.15.189:6837","nonce":1582109479}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T02:41:21.564 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-09-17T02:41:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 -- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e54073200 msgr2=0x7f7e54075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e54073200 0x7f7e54075710 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0058d0 tx=0x7f7e5c005860 comp rx=0 tx=0).stop 2024-09-17T02:41:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 -- 172.21.15.131:0/2490463197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e6806dea0 msgr2=0x7f7e6815a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e6806dea0 0x7f7e6815a970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c3c0 tx=0x7f7e5800c880 comp rx=0 tx=0).stop 2024-09-17T02:41:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 -- 172.21.15.131:0/2490463197 shutdown_connections 2024-09-17T02:41:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e54073200 0x7f7e54075710 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7e68155240 0x7f7e68153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7e680cab60 0x7f7e68159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 --2- 172.21.15.131:0/2490463197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7e6806dea0 0x7f7e6815a970 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 -- 172.21.15.131:0/2490463197 >> 172.21.15.131:0/2490463197 conn(0x7f7e6806eea0 msgr2=0x7f7e680c90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:21.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 -- 172.21.15.131:0/2490463197 shutdown_connections 2024-09-17T02:41:21.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:21.566+0000 7f7e6f195640 1 -- 172.21.15.131:0/2490463197 wait complete. 2024-09-17T02:41:21.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:40:51.611225+0000', 'modified': '2024-09-17T02:41:12.632560+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4384}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4384': {'gid': 4384, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.189:6837/1582109479', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6836', 'nonce': 1582109479}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 1582109479}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4384, 'qdb_cluster': [4384]} 2024-09-17T02:41:21.989 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:41:21.989 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T02:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.158+0000 7fa2ee55d640 1 -- 172.21.15.131:0/3130177632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2d4005330 msgr2=0x7fa2d400b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.158+0000 7fa2ee55d640 1 --2- 172.21.15.131:0/3130177632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2d4005330 0x7fa2d400b9f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000c610 tx=0x7fa2e001d240 comp rx=0 tx=0).stop 2024-09-17T02:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.158+0000 7fa2ee55d640 1 -- 172.21.15.131:0/3130177632 shutdown_connections 2024-09-17T02:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.158+0000 7fa2ee55d640 1 --2- 172.21.15.131:0/3130177632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2d4005330 0x7fa2d400b9f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.158+0000 7fa2ee55d640 1 -- 172.21.15.131:0/3130177632 >> 172.21.15.131:0/3130177632 conn(0x7fa2e806fdb0 msgr2=0x7fa2e80701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.158+0000 7fa2ee55d640 1 -- 172.21.15.131:0/3130177632 shutdown_connections 2024-09-17T02:41:22.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.158+0000 7fa2ee55d640 1 -- 172.21.15.131:0/3130177632 wait complete. 2024-09-17T02:41:22.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ee55d640 1 Processor -- start 2024-09-17T02:41:22.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ee55d640 1 -- start start 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ee55d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa2d4005330 0x7fa2e813e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ee55d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2e815a110 0x7fa2e815a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ee55d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815aa60 0x7fa2e81f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ee55d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e8071390 con 0x7fa2e815aa60 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ee55d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e8071210 con 0x7fa2e815a110 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ee55d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e8071510 con 0x7fa2d4005330 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ecafa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815aa60 0x7fa2e81f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ecafa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815aa60 0x7fa2e81f56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52356/0 (socket says 172.21.15.131:52356) 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ecafa640 1 -- 172.21.15.131:0/2661752599 learned_addr learned my addr 172.21.15.131:0/2661752599 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa2d4005330 0x7fa2e813e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2df7fe640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2e815a110 0x7fa2e815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ecafa640 1 -- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa2d4005330 msgr2=0x7fa2e813e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ecafa640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa2d4005330 0x7fa2e813e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ecafa640 1 -- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2e815a110 msgr2=0x7fa2e815a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ecafa640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2e815a110 0x7fa2e815a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ecafa640 1 -- 172.21.15.131:0/2661752599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e81551d0 con 0x7fa2e815aa60 2024-09-17T02:41:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ecafa640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815aa60 0x7fa2e81f56e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa2e001dae0 tx=0x7fa2e0004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2e57fa640 1 -- 172.21.15.131:0/2661752599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0003b80 con 0x7fa2e815aa60 2024-09-17T02:41:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2e57fa640 1 -- 172.21.15.131:0/2661752599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2e00bc070 con 0x7fa2e815aa60 2024-09-17T02:41:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ee55d640 1 -- 172.21.15.131:0/2661752599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e8158740 con 0x7fa2e815aa60 2024-09-17T02:41:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2e57fa640 1 -- 172.21.15.131:0/2661752599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e00062f0 con 0x7fa2e815aa60 2024-09-17T02:41:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.162+0000 7fa2ee55d640 1 -- 172.21.15.131:0/2661752599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e8155730 con 0x7fa2e815aa60 2024-09-17T02:41:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e57fa640 1 -- 172.21.15.131:0/2661752599 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa2e00ac070 con 0x7fa2e815aa60 2024-09-17T02:41:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e57fa640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2c8073200 0x7fa2c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e57fa640 1 -- 172.21.15.131:0/2661752599 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa2e01309b0 con 0x7fa2e815aa60 2024-09-17T02:41:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2c8073200 0x7fa2c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2f5828000 1 Processor -- start 2024-09-17T02:41:22.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2f5828000 1 -- start start 2024-09-17T02:41:22.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2c8073200 0x7fa2c8075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009a30 tx=0x7fa2d80091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:22.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fb776460 0x5615fbcb76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fbb66950 0x5615fbcb7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb4fca0 0x5615fbcb8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2f5828000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5615fbb8c410 con 0x5615fb776460 2024-09-17T02:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2f5828000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5615fbac9b40 con 0x5615fbb4fca0 2024-09-17T02:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2f5828000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5615fbab8f70 con 0x5615fbb66950 2024-09-17T02:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fb776460 0x5615fbcb76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fb776460 0x5615fbcb76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52364/0 (socket says 172.21.15.131:52364) 2024-09-17T02:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e7fff640 1 -- 172.21.15.131:0/1198636207 learned_addr learned my addr 172.21.15.131:0/1198636207 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2ecafa640 1 --2- 172.21.15.131:0/1198636207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb4fca0 0x5615fbcb8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2df7fe640 1 --2- 172.21.15.131:0/1198636207 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fbb66950 0x5615fbcb7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e7fff640 1 -- 172.21.15.131:0/1198636207 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fbb66950 msgr2=0x5615fbcb7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/1198636207 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fbb66950 0x5615fbcb7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e7fff640 1 -- 172.21.15.131:0/1198636207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb4fca0 msgr2=0x5615fbcb8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/1198636207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb4fca0 0x5615fbcb8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.166+0000 7fa2e7fff640 1 -- 172.21.15.131:0/1198636207 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615fba805b0 con 0x5615fb776460 2024-09-17T02:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/1198636207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fb776460 0x5615fbcb76a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8013f00 tx=0x7fa2d8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af13368570b9c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2ddffb640 1 -- 172.21.15.131:0/1198636207 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8005220 con 0x5615fb776460 2024-09-17T02:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2ddffb640 1 -- 172.21.15.131:0/1198636207 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2d80053c0 con 0x5615fb776460 2024-09-17T02:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2ddffb640 1 -- 172.21.15.131:0/1198636207 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d80047b0 con 0x5615fb776460 2024-09-17T02:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 -- 172.21.15.131:0/1198636207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fb776460 msgr2=0x5615fbcb76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 --2- 172.21.15.131:0/1198636207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fb776460 0x5615fbcb76a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8013f00 tx=0x7fa2d8016000 comp rx=0 tx=0).stop 2024-09-17T02:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 -- 172.21.15.131:0/1198636207 shutdown_connections 2024-09-17T02:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 --2- 172.21.15.131:0/1198636207 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb4fca0 0x5615fbcb8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 --2- 172.21.15.131:0/1198636207 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fbb66950 0x5615fbcb7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 --2- 172.21.15.131:0/1198636207 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fb776460 0x5615fbcb76a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 -- 172.21.15.131:0/1198636207 >> 172.21.15.131:0/1198636207 conn(0x5615fbc3cd90 msgr2=0x5615fbc6c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 -- 172.21.15.131:0/1198636207 shutdown_connections 2024-09-17T02:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 -- 172.21.15.131:0/1198636207 wait complete. 2024-09-17T02:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 -- start start 2024-09-17T02:41:22.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 Processor -- start 2024-09-17T02:41:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbcbe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbcbee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbcbf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5615fba535f0 con 0x5615fbb4fca0 2024-09-17T02:41:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5615fbabcd80 con 0x5615fbb66950 2024-09-17T02:41:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5615fba5f520 con 0x5615fb776460 2024-09-17T02:41:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2f5828000 10 client.0 _ll_register_callbacks cb 0x5615fba63830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:41:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2df7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbcbee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2df7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbcbee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52374/0 (socket says 172.21.15.131:52374) 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2df7fe640 1 -- 172.21.15.131:0/2352848134 learned_addr learned my addr 172.21.15.131:0/2352848134 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2ecafa640 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbcbf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbcbe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2df7fe640 1 -- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 msgr2=0x5615fbcbe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2df7fe640 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbcbe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2df7fe640 1 -- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 msgr2=0x5615fbcbf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2df7fe640 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbcbf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2df7fe640 1 -- 172.21.15.131:0/2352848134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615fba57190 con 0x5615fbb4fca0 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2ecafa640 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbcbf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.170+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbcbe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2df7fe640 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbcbee60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00146e0 tx=0x7fa2d0014ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77c630ef0a8bcdc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:41:22.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/2352848134 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0012d70 con 0x5615fbb4fca0 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 1 -- 172.21.15.131:0/2352848134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5615fba51110 con 0x5615fbb4fca0 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/2352848134 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa2d001d440 con 0x5615fbb4fca0 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/2352848134 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fa2d0025050 con 0x5615fbb4fca0 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 10 client.4767 fetch_fsmap learned FSMap version 9 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 1 -- 172.21.15.131:0/2352848134 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5615fbb3df10 con 0x5615fbb4fca0 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/2352848134 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5615fba51110 con 0x5615fbb4fca0 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/2352848134 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7fa2d0028020 con 0x5615fbb4fca0 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 10 client.4767 fetch_fsmap finished waiting for FSMap version 9 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 10 client.4767 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 20 client.4767 populate_metadata read hostname 'smithi131' 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x5615fb75b7d0 0x5615fbcc8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 4 client.4767 mds_command: new command op to 4384 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 1 -- 172.21.15.131:0/2352848134 --> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615fbacba90 con 0x5615fb75b7d0 2024-09-17T02:41:22.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x5615fb75b7d0 0x5615fbcc8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2c6ffd640 10 client.4767 ms_handle_connect on v2:172.21.15.189:6836/1582109479 2024-09-17T02:41:22.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x5615fb75b7d0 0x5615fbcc8d00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98baadb8acde4aaf server_cookie=39d181666bc799bc in_seq=0 out_seq=0 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/2352848134 <== mds.0 v2:172.21.15.189:6836/1582109479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615fbacba90 con 0x5615fb75b7d0 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2c6ffd640 10 client.4767 handle_command_reply: tid=0 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 1 client.4767 shutdown 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 1 -- 172.21.15.131:0/2352848134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 msgr2=0x5615fbcbee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.174+0000 7fa2f5828000 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbcbee60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00146e0 tx=0x7fa2d0014ba0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.178+0000 7fa2f5828000 1 -- 172.21.15.131:0/2352848134 shutdown_connections 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.178+0000 7fa2f5828000 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x5615fb75b7d0 0x5615fbcc8d00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.178+0000 7fa2f5828000 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbcbf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.178+0000 7fa2f5828000 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbcbee60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.178+0000 7fa2f5828000 1 --2- 172.21.15.131:0/2352848134 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbcbe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.178+0000 7fa2c6ffd640 0 client.4767 ms_handle_reset on v2:172.21.15.189:6836/1582109479 2024-09-17T02:41:22.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.178+0000 7fa2f5828000 1 -- 172.21.15.131:0/2352848134 >> 172.21.15.131:0/2352848134 conn(0x5615fbc3cd90 msgr2=0x5615fbc73440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:22.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.178+0000 7fa2f5828000 1 -- 172.21.15.131:0/2352848134 shutdown_connections 2024-09-17T02:41:22.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.178+0000 7fa2f5828000 1 -- 172.21.15.131:0/2352848134 wait complete. 2024-09-17T02:41:22.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.178+0000 7fa2f5828000 20 client.4767 trim_cache size 0 max 16384 2024-09-17T02:41:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.194+0000 7fa2f5828000 1 Processor -- start 2024-09-17T02:41:22.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.194+0000 7fa2f5828000 1 -- start start 2024-09-17T02:41:22.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbd29830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd29d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbd2a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e81ff510 con 0x5615fbb4fca0 2024-09-17T02:41:22.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e81fe540 con 0x5615fbb66950 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e806ddd0 con 0x5615fb776460 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2df7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd29d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2df7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd29d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52388/0 (socket says 172.21.15.131:52388) 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2df7fe640 1 -- 172.21.15.131:0/118958072 learned_addr learned my addr 172.21.15.131:0/118958072 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2ecafa640 1 --2- 172.21.15.131:0/118958072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbd2a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2df7fe640 1 -- 172.21.15.131:0/118958072 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 msgr2=0x5615fbd29830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/118958072 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbd29830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2df7fe640 1 --2- 172.21.15.131:0/118958072 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbd29830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2df7fe640 1 -- 172.21.15.131:0/118958072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 msgr2=0x5615fbd2a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2df7fe640 1 --2- 172.21.15.131:0/118958072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbd2a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2df7fe640 1 -- 172.21.15.131:0/118958072 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615fbb14db0 con 0x5615fbb4fca0 2024-09-17T02:41:22.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2df7fe640 1 --2- 172.21.15.131:0/118958072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd29d70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa2d001d6b0 tx=0x7fa2d0000f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181329c543ea7df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:22.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2dd7fa640 1 -- 172.21.15.131:0/118958072 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0022030 con 0x5615fbb4fca0 2024-09-17T02:41:22.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2dd7fa640 1 -- 172.21.15.131:0/118958072 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2d001daa0 con 0x5615fbb4fca0 2024-09-17T02:41:22.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2dd7fa640 1 -- 172.21.15.131:0/118958072 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0012db0 con 0x5615fbb4fca0 2024-09-17T02:41:22.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 -- 172.21.15.131:0/118958072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 msgr2=0x5615fbd29d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 --2- 172.21.15.131:0/118958072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd29d70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa2d001d6b0 tx=0x7fa2d0000f70 comp rx=0 tx=0).stop 2024-09-17T02:41:22.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 -- 172.21.15.131:0/118958072 shutdown_connections 2024-09-17T02:41:22.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 --2- 172.21.15.131:0/118958072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbd2a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 --2- 172.21.15.131:0/118958072 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd29d70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 --2- 172.21.15.131:0/118958072 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbd29830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 -- 172.21.15.131:0/118958072 >> 172.21.15.131:0/118958072 conn(0x5615fbc3cd90 msgr2=0x5615fbcb5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:22.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 -- 172.21.15.131:0/118958072 shutdown_connections 2024-09-17T02:41:22.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 -- 172.21.15.131:0/118958072 wait complete. 2024-09-17T02:41:22.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 -- start start 2024-09-17T02:41:22.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.198+0000 7fa2f5828000 1 Processor -- start 2024-09-17T02:41:22.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbd2d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd2dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbd2e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5615fbb3da80 con 0x5615fbb4fca0 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5615fbabc090 con 0x5615fbb66950 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5615fbac13b0 con 0x5615fb776460 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 10 client.0 _ll_register_callbacks cb 0x7fa2d000d430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2df7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd2dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2df7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd2dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52394/0 (socket says 172.21.15.131:52394) 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2df7fe640 1 -- 172.21.15.131:0/1835133623 learned_addr learned my addr 172.21.15.131:0/1835133623 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2df7fe640 1 -- 172.21.15.131:0/1835133623 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 msgr2=0x5615fbd2d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2df7fe640 1 --2- 172.21.15.131:0/1835133623 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbd2d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2df7fe640 1 -- 172.21.15.131:0/1835133623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 msgr2=0x5615fbd2e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2df7fe640 1 --2- 172.21.15.131:0/1835133623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbd2e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2df7fe640 1 -- 172.21.15.131:0/1835133623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615fba7fab0 con 0x5615fbb4fca0 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2df7fe640 1 --2- 172.21.15.131:0/1835133623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd2dcf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa2d001d6b0 tx=0x7fa2d001dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4874b9dd2cdb21cd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:22.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/1835133623 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0039020 con 0x5615fbb4fca0 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/1835133623 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fa2d001f9f0 con 0x5615fbb4fca0 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 -- 172.21.15.131:0/1835133623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d001daa0 con 0x5615fbb4fca0 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/1835133623 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7fa2d0022070 con 0x5615fbb4fca0 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 10 client.4773 fetch_fsmap learned FSMap version 9 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 -- 172.21.15.131:0/1835133623 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x5615fba5e070 con 0x5615fbb4fca0 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/1835133623 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2d001daa0 con 0x5615fbb4fca0 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/1835133623 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 9) ==== 3510+0+0 (secure 0 0 0) 0x7fa2d0030070 con 0x5615fbb4fca0 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 10 client.4773 fetch_fsmap finished waiting for FSMap version 9 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 10 client.4773 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 20 client.4773 populate_metadata read hostname 'smithi131' 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 --2- 172.21.15.131:0/1835133623 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x5615fb749830 0x5615fbce9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 4 client.4773 mds_command: new command op to 4384 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:41:22.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 -- 172.21.15.131:0/1835133623 --> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] -- command(tid 0: {"prefix": "status"}) -- 0x5615fbac7540 con 0x5615fb749830 2024-09-17T02:41:22.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/1835133623 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x5615fb749830 0x5615fbce9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2c6ffd640 10 client.4773 ms_handle_connect on v2:172.21.15.189:6836/1582109479 2024-09-17T02:41:22.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2e7fff640 1 --2- 172.21.15.131:0/1835133623 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x5615fb749830 0x5615fbce9640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f81e96315fb68f74 server_cookie=4234d361ab714931 in_seq=0 out_seq=0 2024-09-17T02:41:22.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2c6ffd640 1 -- 172.21.15.131:0/1835133623 <== mds.0 v2:172.21.15.189:6836/1582109479 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5615fbac7540 con 0x5615fb749830 2024-09-17T02:41:22.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2c6ffd640 10 client.4773 handle_command_reply: tid=0 2024-09-17T02:41:22.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 client.4773 shutdown 2024-09-17T02:41:22.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 -- 172.21.15.131:0/1835133623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 msgr2=0x5615fbd2dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 --2- 172.21.15.131:0/1835133623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd2dcf0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa2d001d6b0 tx=0x7fa2d001dc50 comp rx=0 tx=0).stop 2024-09-17T02:41:22.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 -- 172.21.15.131:0/1835133623 shutdown_connections 2024-09-17T02:41:22.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 --2- 172.21.15.131:0/1835133623 >> [v2:172.21.15.189:6836/1582109479,v1:172.21.15.189:6837/1582109479] conn(0x5615fb749830 0x5615fbce9640 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 --2- 172.21.15.131:0/1835133623 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5615fbb66950 0x5615fbd2e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 --2- 172.21.15.131:0/1835133623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5615fbb4fca0 0x5615fbd2dcf0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2c6ffd640 0 client.4773 ms_handle_reset on v2:172.21.15.189:6836/1582109479 2024-09-17T02:41:22.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 --2- 172.21.15.131:0/1835133623 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5615fb776460 0x5615fbd2d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 -- 172.21.15.131:0/1835133623 >> 172.21.15.131:0/1835133623 conn(0x5615fbc3cd90 msgr2=0x5615fbcb5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:22.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 -- 172.21.15.131:0/1835133623 shutdown_connections 2024-09-17T02:41:22.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 1 -- 172.21.15.131:0/1835133623 wait complete. 2024-09-17T02:41:22.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.202+0000 7fa2f5828000 20 client.4773 trim_cache size 0 max 16384 2024-09-17T02:41:22.208 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:41:22.208 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:41:22.208 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:41:22.208 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4384, 2024-09-17T02:41:22.209 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:41:22.209 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:41:22.209 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:41:22.209 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 30.011355729000002, 2024-09-17T02:41:22.209 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 9, 2024-09-17T02:41:22.209 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 26, 2024-09-17T02:41:22.209 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 26, 2024-09-17T02:41:22.209 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 35.064780816999999 2024-09-17T02:41:22.209 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 -- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2c8073200 msgr2=0x7fa2c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2c8073200 0x7fa2c8075710 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009a30 tx=0x7fa2d80091e0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 -- 172.21.15.131:0/2661752599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815aa60 msgr2=0x7fa2e81f56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815aa60 0x7fa2e81f56e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa2e001dae0 tx=0x7fa2e0004e60 comp rx=0 tx=0).stop 2024-09-17T02:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 -- 172.21.15.131:0/2661752599 shutdown_connections 2024-09-17T02:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa2c8073200 0x7fa2c8075710 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2e815aa60 0x7fa2e81f56e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa2e815a110 0x7fa2e815a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 --2- 172.21.15.131:0/2661752599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa2d4005330 0x7fa2e813e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 -- 172.21.15.131:0/2661752599 >> 172.21.15.131:0/2661752599 conn(0x7fa2e806fdb0 msgr2=0x7fa2e80b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:22.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 -- 172.21.15.131:0/2661752599 shutdown_connections 2024-09-17T02:41:22.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.206+0000 7fa2ee55d640 1 -- 172.21.15.131:0/2661752599 wait complete. 2024-09-17T02:41:22.224 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-17T02:41:22.234 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-17T02:41:22.234 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-17T02:41:22.234 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-17T02:41:22.234 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T02:41:22.235 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:22.235 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-17T02:41:22.235 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T02:41:22.235 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T02:41:22.235 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-17T02:41:22.235 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T02:41:22.235 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:22.235 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-17T02:41:22.235 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T02:41:22.235 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T02:41:22.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:22.236 DEBUG:teuthology.orchestra.run.smithi189:> ip netns list 2024-09-17T02:41:22.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:22.243 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip link delete ceph-brx 2024-09-17T02:41:22.306 INFO:teuthology.orchestra.run.smithi189.stderr:Cannot find device "ceph-brx" 2024-09-17T02:41:22.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:41:22.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:22.308 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:41:22.314 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-17T02:41:22.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:22.314 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-17T02:41:22.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:22.369 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-09-17T02:41:22.553 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-17T02:41:22.554 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T02:41:22.554 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:41:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.714+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3369236318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb4004810 msgr2=0x7f6eb4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.714+0000 7f6ed1b0c640 1 --2- 172.21.15.131:0/3369236318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb4004810 0x7f6eb4004c20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ec00016f0 tx=0x7f6ec001c790 comp rx=0 tx=0).stop 2024-09-17T02:41:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3369236318 shutdown_connections 2024-09-17T02:41:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 --2- 172.21.15.131:0/3369236318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6eb4004810 0x7f6eb4004c20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3369236318 >> 172.21.15.131:0/3369236318 conn(0x7f6ecc06eea0 msgr2=0x7f6ecc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:22.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3369236318 shutdown_connections 2024-09-17T02:41:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3369236318 wait complete. 2024-09-17T02:41:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 Processor -- start 2024-09-17T02:41:22.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 -- start start 2024-09-17T02:41:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6eb4004810 0x7f6ecc145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ecc1f4e00 0x7f6ecc1426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc15d920 0x7f6ecc15dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc06fd50 con 0x7f6ecc15d920 2024-09-17T02:41:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc06fbd0 con 0x7f6ecc1f4e00 2024-09-17T02:41:22.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ed1b0c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6ecc06fed0 con 0x7f6eb4004810 2024-09-17T02:41:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc15d920 0x7f6ecc15dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc15d920 0x7f6ecc15dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52422/0 (socket says 172.21.15.131:52422) 2024-09-17T02:41:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ecbfff640 1 -- 172.21.15.131:0/3272576700 learned_addr learned my addr 172.21.15.131:0/3272576700 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ecaffd640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ecc1f4e00 0x7f6ecc1426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ecb7fe640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6eb4004810 0x7f6ecc145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ecbfff640 1 -- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6eb4004810 msgr2=0x7f6ecc145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ecbfff640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6eb4004810 0x7f6ecc145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ecbfff640 1 -- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ecc1f4e00 msgr2=0x7f6ecc1426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ecbfff640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ecc1f4e00 0x7f6ecc1426d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.718+0000 7f6ecbfff640 1 -- 172.21.15.131:0/3272576700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc15c380 con 0x7f6ecc15d920 2024-09-17T02:41:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6ecbfff640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc15d920 0x7f6ecc15dd30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc073d80 tx=0x7f6ec4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6eb3fff640 1 -- 172.21.15.131:0/3272576700 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec400f980 con 0x7f6ecc15d920 2024-09-17T02:41:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6eb3fff640 1 -- 172.21.15.131:0/3272576700 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ec400fc10 con 0x7f6ecc15d920 2024-09-17T02:41:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3272576700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc145670 con 0x7f6ecc15d920 2024-09-17T02:41:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6eb3fff640 1 -- 172.21.15.131:0/3272576700 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4013450 con 0x7f6ecc15d920 2024-09-17T02:41:22.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3272576700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc15cb60 con 0x7f6ecc15d920 2024-09-17T02:41:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3272576700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ecc15c5f0 con 0x7f6ecc15d920 2024-09-17T02:41:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6eb3fff640 1 -- 172.21.15.131:0/3272576700 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6ec40135f0 con 0x7f6ecc15d920 2024-09-17T02:41:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6eb3fff640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6eb8073200 0x7f6eb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6eb3fff640 1 -- 172.21.15.131:0/3272576700 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f6ec4098ba0 con 0x7f6ecc15d920 2024-09-17T02:41:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6ecb7fe640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6eb8073200 0x7f6eb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:22.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.722+0000 7f6ecb7fe640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6eb8073200 0x7f6eb8075710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc06bb80 tx=0x7f6ebc00a2f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:22.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.726+0000 7f6eb3fff640 1 -- 172.21.15.131:0/3272576700 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6ec4063910 con 0x7f6ecc15d920 2024-09-17T02:41:22.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.926+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3272576700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6ecc06e010 con 0x7f6ecc15d920 2024-09-17T02:41:22.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6eb3fff640 1 -- 172.21.15.131:0/3272576700 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f6ec4098070 con 0x7f6ecc15d920 2024-09-17T02:41:22.928 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-17T02:41:22.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6eb8073200 msgr2=0x7f6eb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6eb8073200 0x7f6eb8075710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc06bb80 tx=0x7f6ebc00a2f0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3272576700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc15d920 msgr2=0x7f6ecc15dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:22.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc15d920 0x7f6ecc15dd30 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc073d80 tx=0x7f6ec4011ab0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3272576700 shutdown_connections 2024-09-17T02:41:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6eb8073200 0x7f6eb8075710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ecc15d920 0x7f6ecc15dd30 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6ecc1f4e00 0x7f6ecc1426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 --2- 172.21.15.131:0/3272576700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6eb4004810 0x7f6ecc145130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3272576700 >> 172.21.15.131:0/3272576700 conn(0x7f6ecc06eea0 msgr2=0x7f6ecc0b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3272576700 shutdown_connections 2024-09-17T02:41:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:22.930+0000 7f6ed1b0c640 1 -- 172.21.15.131:0/3272576700 wait complete. 2024-09-17T02:41:22.948 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:41:22.948 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:41:22.948 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:41:22.948 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:41:22.948 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:22.948 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:41:22.948 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:41:22.948 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:41:22.948 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:41:22.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:22.949 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:41:22.954 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:41:22.954 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:41:22.954 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:41:22.955 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:22.956 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:41:22.956 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42423sec preferred_lft 42423sec 2024-09-17T02:41:22.956 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:41:22.956 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:22.956 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:41:22.956 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:22.956 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:41:22.956 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:22.956 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T02:41:22.956 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:22.957 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:22.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add name ceph-brx type bridge 2024-09-17T02:41:22.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr flush dev ceph-brx 2024-09-17T02:41:22.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set ceph-brx up 2024-09-17T02:41:22.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T02:41:22.957 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:23.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:23.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:23.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:23.079 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:41:23.128 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:41:23.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:23.129 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:41:23.180 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:41:23.180 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:41:23.180 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:41:23.180 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:41:23.180 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:41:23.180 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:41:23.180 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:41:23.180 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:23.180 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:23.180 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:23.181 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:41:23.181 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:23.181 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:23.181 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:41:23.181 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:41:23.181 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:41:23.181 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:23.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:23.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:23.301 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:41:23.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:23.308 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-09-17T02:41:23.352 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:23.352 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:23.352 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:23.352 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-17T02:41:23.352 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:23.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:23.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:23.464 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-17T02:41:23.464 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:23.464 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:23.464 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-17T02:41:23.464 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T02:41:23.464 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-17T02:41:23.464 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-17T02:41:23.464 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-17T02:41:23.465 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:23.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:23.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:23.550 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:23.551 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:23.551 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:41:23.551 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T02:41:23.551 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:23.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:23.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:23.648 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:41:23.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:41:23.648 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:23.702 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:41:23.702 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:41:23.702 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:23.748 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:41:23.804 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:23.853 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:41:23.853 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:41:23.853 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:41:23.853 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:41:23.854 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:23.855 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:23.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:23.856 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:41:23.905 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:41:23.905 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:41:23.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:41:23.968 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:24.016 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:41:24.016 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:41:24.017 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:41:24.017 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:41:24.017 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:41:24.017 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:41:24.017 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:41:24.017 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:41:24.017 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:41:24.017 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:41:24.017 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:41:24.017 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:41:24.017 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:41:24.018 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:41:24.019 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:41:24.019 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:41:24.019 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:41:24.019 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:41:24.019 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:24.019 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:24.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:24.020 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:41:24.060 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:41:24.062+0000 7f44da37c580 -1 init, newargv = 0x7f44bc0ad020 newargc=15 2024-09-17T02:41:24.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[37092]: starting ceph client 2024-09-17T02:41:24.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[37092]: starting fuse 2024-09-17T02:41:25.167 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:41:25.183 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:25.229 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:41:25.229 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:41:25.229 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:41:25.229 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:41:25.229 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:41:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:41:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:41:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:41:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:41:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:41:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:41:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:41:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:41:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:41:25.230 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:41:25.231 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:25.232 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:25.232 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:25.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:25.233 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:41:25.281 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:41:25.281 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:41:25.281 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:41:25.282 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:41:25.283 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:41:25.406 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.37092.asok 2024-09-17T02:41:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:25.418 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37092.asok 2024-09-17T02:41:25.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:25.418 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37092.asok status 2024-09-17T02:41:25.599 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:41:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:41:25.599 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "37092", 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4785, 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:41:25.600 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 4785 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2248803193 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2248803193 2024-09-17T02:41:25.601 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.4785 192.168.0.1:0/2248803193", 2024-09-17T02:41:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/2248803193", 2024-09-17T02:41:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:41:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 9, 2024-09-17T02:41:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 26, 2024-09-17T02:41:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:41:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:41:25.602 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:41:25.602 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:41:25.615 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T02:41:25.616 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:41:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.774+0000 7f08d89cd640 1 -- 172.21.15.131:0/4036101141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d40cec00 msgr2=0x7f08d40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.774+0000 7f08d89cd640 1 --2- 172.21.15.131:0/4036101141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d40cec00 0x7f08d40cf010 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009bc0 tx=0x7f08cc01cb30 comp rx=0 tx=0).stop 2024-09-17T02:41:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.774+0000 7f08d89cd640 1 -- 172.21.15.131:0/4036101141 shutdown_connections 2024-09-17T02:41:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.774+0000 7f08d89cd640 1 --2- 172.21.15.131:0/4036101141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d40cec00 0x7f08d40cf010 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.774+0000 7f08d89cd640 1 --2- 172.21.15.131:0/4036101141 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f08d406e2c0 0x7f08d40c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.774+0000 7f08d89cd640 1 -- 172.21.15.131:0/4036101141 >> 172.21.15.131:0/4036101141 conn(0x7f08d406eea0 msgr2=0x7f08d406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.774+0000 7f08d89cd640 1 -- 172.21.15.131:0/4036101141 shutdown_connections 2024-09-17T02:41:25.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.774+0000 7f08d89cd640 1 -- 172.21.15.131:0/4036101141 wait complete. 2024-09-17T02:41:25.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d89cd640 1 Processor -- start 2024-09-17T02:41:25.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d89cd640 1 -- start start 2024-09-17T02:41:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d89cd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f08d406e2c0 0x7f08d4151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d89cd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d40c4ac0 0x7f08d414d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d89cd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d40cec00 0x7f08d40b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d89cd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4169cf0 con 0x7f08d40cec00 2024-09-17T02:41:25.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d89cd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f08d415cb90 con 0x7f08d40c4ac0 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d89cd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f08d4169e70 con 0x7f08d406e2c0 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d40cec00 0x7f08d40b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d40cec00 0x7f08d40b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52448/0 (socket says 172.21.15.131:52448) 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d2d9d640 1 -- 172.21.15.131:0/3176131982 learned_addr learned my addr 172.21.15.131:0/3176131982 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d1d9b640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d40c4ac0 0x7f08d414d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d259c640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f08d406e2c0 0x7f08d4151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d2d9d640 1 -- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f08d406e2c0 msgr2=0x7f08d4151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d2d9d640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f08d406e2c0 0x7f08d4151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d2d9d640 1 -- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d40c4ac0 msgr2=0x7f08d414d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d2d9d640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d40c4ac0 0x7f08d414d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d2d9d640 1 -- 172.21.15.131:0/3176131982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d41520b0 con 0x7f08d40cec00 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d2d9d640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d40cec00 0x7f08d40b3a20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f08cc01c5a0 tx=0x7f08cc005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08caffd640 1 -- 172.21.15.131:0/3176131982 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0b2040 con 0x7f08d40cec00 2024-09-17T02:41:25.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08caffd640 1 -- 172.21.15.131:0/3176131982 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08cc0bb070 con 0x7f08d40cec00 2024-09-17T02:41:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d89cd640 1 -- 172.21.15.131:0/3176131982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d4152390 con 0x7f08d40cec00 2024-09-17T02:41:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08caffd640 1 -- 172.21.15.131:0/3176131982 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc004630 con 0x7f08d40cec00 2024-09-17T02:41:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d89cd640 1 -- 172.21.15.131:0/3176131982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d4158040 con 0x7f08d40cec00 2024-09-17T02:41:25.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.778+0000 7f08d89cd640 1 -- 172.21.15.131:0/3176131982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d40c4580 con 0x7f08d40cec00 2024-09-17T02:41:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.782+0000 7f08caffd640 1 -- 172.21.15.131:0/3176131982 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f08cc0c0460 con 0x7f08d40cec00 2024-09-17T02:41:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.782+0000 7f08caffd640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08bc073200 0x7f08bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.782+0000 7f08caffd640 1 -- 172.21.15.131:0/3176131982 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f08cc12f1d0 con 0x7f08d40cec00 2024-09-17T02:41:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.782+0000 7f08d259c640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08bc073200 0x7f08bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:25.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.782+0000 7f08d259c640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08bc073200 0x7f08bc075710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f08c400b9a0 tx=0x7f08c4005c30 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:25.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.786+0000 7f08caffd640 1 -- 172.21.15.131:0/3176131982 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f08cc0167b0 con 0x7f08d40cec00 2024-09-17T02:41:25.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.986+0000 7f08d89cd640 1 -- 172.21.15.131:0/3176131982 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f08d4151e40 con 0x7f08d40cec00 2024-09-17T02:41:25.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.986+0000 7f08caffd640 1 -- 172.21.15.131:0/3176131982 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f08cc0f9eb0 con 0x7f08d40cec00 2024-09-17T02:41:25.986 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-17T02:41:25.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.986+0000 7f08d89cd640 1 -- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08bc073200 msgr2=0x7f08bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:25.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.986+0000 7f08d89cd640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08bc073200 0x7f08bc075710 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f08c400b9a0 tx=0x7f08c4005c30 comp rx=0 tx=0).stop 2024-09-17T02:41:25.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.986+0000 7f08d89cd640 1 -- 172.21.15.131:0/3176131982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d40cec00 msgr2=0x7f08d40b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:25.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.986+0000 7f08d89cd640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d40cec00 0x7f08d40b3a20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f08cc01c5a0 tx=0x7f08cc005ae0 comp rx=0 tx=0).stop 2024-09-17T02:41:25.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.990+0000 7f08d89cd640 1 -- 172.21.15.131:0/3176131982 shutdown_connections 2024-09-17T02:41:25.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.990+0000 7f08d89cd640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08bc073200 0x7f08bc075710 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:25.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.990+0000 7f08d89cd640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08d40cec00 0x7f08d40b3a20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:25.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.990+0000 7f08d89cd640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08d40c4ac0 0x7f08d414d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:25.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.990+0000 7f08d89cd640 1 --2- 172.21.15.131:0/3176131982 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f08d406e2c0 0x7f08d4151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:25.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.990+0000 7f08d89cd640 1 -- 172.21.15.131:0/3176131982 >> 172.21.15.131:0/3176131982 conn(0x7f08d406eea0 msgr2=0x7f08d41532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:25.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.990+0000 7f08d89cd640 1 -- 172.21.15.131:0/3176131982 shutdown_connections 2024-09-17T02:41:25.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:25.990+0000 7f08d89cd640 1 -- 172.21.15.131:0/3176131982 wait complete. 2024-09-17T02:41:26.005 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:41:26.005 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:41:26.005 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:41:26.005 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-17T02:41:26.005 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T02:41:26.005 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:41:26.005 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:41:26.005 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:41:26.005 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-17T02:41:26.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:26.006 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:41:26.011 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:26.012 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42420sec preferred_lft 42420sec 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:41:26.013 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:41:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.014 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:41:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:41:26.014 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:26.014 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:41:26.070 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:41:26.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:26.071 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:41:26.124 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:41:26.124 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:41:26.124 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:41:26.124 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:41:26.124 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:41:26.124 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:41:26.124 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:41:26.124 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:26.124 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:26.124 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:26.124 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:41:26.125 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:26.125 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:26.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:41:26.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:41:26.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:41:26.125 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:26.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:26.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:26.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:26.244 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:41:26.296 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:41:26.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:26.297 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list-id 2024-09-17T02:41:26.345 INFO:teuthology.orchestra.run.smithi131.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-17T02:41:26.345 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:26.345 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:26.345 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:41:26.345 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-17T02:41:26.345 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:26.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:26.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:26.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:26.464 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T02:41:26.476 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:41:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:41:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:41:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:41:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.476 INFO:teuthology.orchestra.run.smithi131.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:41:26.476 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 86:cc:b5:ee:6c:4b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T02:41:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T02:41:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::84cc:b5ff:feee:6c4b/64 scope link 2024-09-17T02:41:26.477 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:26.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T02:41:26.532 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:41:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:41:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:41:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:41:26.533 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.533 INFO:teuthology.orchestra.run.smithi131.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:41:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 86:cc:b5:ee:6c:4b brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T02:41:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T02:41:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::84cc:b5ff:feee:6c4b/64 scope link 2024-09-17T02:41:26.534 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:26.536 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-17T02:41:26.537 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:26.537 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:26.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-17T02:41:26.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T02:41:26.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-17T02:41:26.538 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-17T02:41:26.538 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-17T02:41:26.538 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:26.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:26.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:26.677 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:26.677 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:41:26.677 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.1 up 2024-09-17T02:41:26.677 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T02:41:26.677 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:41:26.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:26.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:26.735 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-17T02:41:26.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:41:26.735 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-17T02:41:26.780 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-17T02:41:26.780 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:41:26.780 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-17T02:41:26.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:41:26.881 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:41:26.930 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:41:26.931 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:41:26.932 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:41:26.932 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:41:26.932 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:41:26.932 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:26.932 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:26.932 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:26.932 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:41:26.932 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:41:26.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:26.933 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:41:26.980 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:41:26.981 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-17T02:41:26.981 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-17T02:41:27.025 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:41:27.040 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:27.092 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:27.093 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:41:27.094 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:41:27.095 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:41:27.095 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:27.095 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:27.095 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:27.095 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:41:27.095 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:41:27.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:27.096 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:41:27.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi131.stderr:2024-09-17T02:41:27.130+0000 7f8f0bd55580 -1 init, newargv = 0x7f8ef8001280 newargc=15 2024-09-17T02:41:27.131 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi131.stderr:ceph-fuse[37237]: starting ceph client 2024-09-17T02:41:27.135 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:41:27.138 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi131.stderr:ceph-fuse[37237]: starting fuse 2024-09-17T02:41:28.237 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:41:28.253 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:28.303 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:41:28.303 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:41:28.303 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:41:28.303 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:41:28.303 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:41:28.303 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:41:28.303 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:41:28.303 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:41:28.303 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:28.304 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:41:28.305 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.2 2024-09-17T02:41:28.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:28.306 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:41:28.353 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:41:28.353 INFO:teuthology.orchestra.run.smithi131.stdout:46 2024-09-17T02:41:28.353 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-17T02:41:28.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:41:28.353 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:28.354 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:41:28.355 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:41:28.355 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:41:28.355 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:41:28.355 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:41:28.355 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:41:28.355 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:41:28.355 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:41:28.355 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:41:28.355 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.2")) 2024-09-17T02:41:28.355 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:41:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.2.37237.asok 2024-09-17T02:41:28.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:28.489 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.37237.asok 2024-09-17T02:41:28.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:28.489 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.37237.asok status 2024-09-17T02:41:28.671 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:41:28.671 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:41:28.671 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:41:28.671 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:41:28.671 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "2", 2024-09-17T02:41:28.671 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:41:28.671 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "37237", 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 4803, 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 4803 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:41:28.672 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.2:0", 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1588698179 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.2:0", 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1588698179 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.4803 192.168.0.2:0/1588698179", 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.2:0/1588698179", 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 9, 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 26, 2024-09-17T02:41:28.673 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:41:28.674 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:41:28.674 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:41:28.674 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:41:28.687 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T02:41:28.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:41:28.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.846+0000 7fdabedb4640 1 -- 172.21.15.131:0/2873412400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8155050 msgr2=0x7fdab8155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:28.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.846+0000 7fdabedb4640 1 --2- 172.21.15.131:0/2873412400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8155050 0x7fdab8155460 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdab808d230 tx=0x7fdaa801c7d0 comp rx=0 tx=0).stop 2024-09-17T02:41:28.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.846+0000 7fdabedb4640 1 -- 172.21.15.131:0/2873412400 shutdown_connections 2024-09-17T02:41:28.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.846+0000 7fdabedb4640 1 --2- 172.21.15.131:0/2873412400 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8155050 0x7fdab8155460 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:28.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.846+0000 7fdabedb4640 1 --2- 172.21.15.131:0/2873412400 >> v2:172.21.15.189:3300/0 conn(0x7fdab80cfb70 0x7fdab8161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:28.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.846+0000 7fdabedb4640 1 -- 172.21.15.131:0/2873412400 >> 172.21.15.131:0/2873412400 conn(0x7fdab80d1ba0 msgr2=0x7fdab80d4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:28.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.846+0000 7fdabedb4640 1 -- 172.21.15.131:0/2873412400 shutdown_connections 2024-09-17T02:41:28.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.846+0000 7fdabedb4640 1 -- 172.21.15.131:0/2873412400 wait complete. 2024-09-17T02:41:28.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.846+0000 7fdabedb4640 1 Processor -- start 2024-09-17T02:41:28.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.846+0000 7fdabedb4640 1 -- start start 2024-09-17T02:41:28.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdabedb4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdab80cfb70 0x7fdab814a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:28.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdabedb4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8155050 0x7fdab80c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:28.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdabedb4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdab8162330 0x7fdab81fc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:28.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdabedb4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdab8160d60 con 0x7fdab8155050 2024-09-17T02:41:28.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdabedb4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdab8160be0 con 0x7fdab80cfb70 2024-09-17T02:41:28.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdabedb4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdab8160ee0 con 0x7fdab8162330 2024-09-17T02:41:28.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8155050 0x7fdab80c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:28.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8155050 0x7fdab80c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52464/0 (socket says 172.21.15.131:52464) 2024-09-17T02:41:28.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab7fff640 1 -- 172.21.15.131:0/4033567545 learned_addr learned my addr 172.21.15.131:0/4033567545 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:28.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdabcb50640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdab80cfb70 0x7fdab814a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:28.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab7fff640 1 -- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdab8162330 msgr2=0x7fdab81fc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:28.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab7fff640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdab8162330 0x7fdab81fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:28.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab7fff640 1 -- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdab80cfb70 msgr2=0x7fdab814a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:28.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab7fff640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdab80cfb70 0x7fdab814a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:28.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab7fff640 1 -- 172.21.15.131:0/4033567545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab81539a0 con 0x7fdab8155050 2024-09-17T02:41:28.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab7fff640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8155050 0x7fdab80c6f60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000c430 tx=0x7fdaa000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:28.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab57fa640 1 -- 172.21.15.131:0/4033567545 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0011790 con 0x7fdab8155050 2024-09-17T02:41:28.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab57fa640 1 -- 172.21.15.131:0/4033567545 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdaa0017070 con 0x7fdab8155050 2024-09-17T02:41:28.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdab57fa640 1 -- 172.21.15.131:0/4033567545 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0011cc0 con 0x7fdab8155050 2024-09-17T02:41:28.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdabedb4640 1 -- 172.21.15.131:0/4033567545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab8152920 con 0x7fdab8155050 2024-09-17T02:41:28.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdabedb4640 1 -- 172.21.15.131:0/4033567545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab8153100 con 0x7fdab8155050 2024-09-17T02:41:28.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.850+0000 7fdabedb4640 1 -- 172.21.15.131:0/4033567545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab8152b70 con 0x7fdab8155050 2024-09-17T02:41:28.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.854+0000 7fdab57fa640 1 -- 172.21.15.131:0/4033567545 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdaa0005020 con 0x7fdab8155050 2024-09-17T02:41:28.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.854+0000 7fdab57fa640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda98073230 0x7fda98075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:28.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.854+0000 7fdab57fa640 1 -- 172.21.15.131:0/4033567545 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7fdaa0094930 con 0x7fdab8155050 2024-09-17T02:41:28.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.854+0000 7fdabcb50640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda98073230 0x7fda98075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:28.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.854+0000 7fdabcb50640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda98073230 0x7fda98075740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdab0005c50 tx=0x7fdab0005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:28.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:28.858+0000 7fdab57fa640 1 -- 172.21.15.131:0/4033567545 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdaa0021020 con 0x7fdab8155050 2024-09-17T02:41:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 -- 172.21.15.131:0/4033567545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdab8161980 con 0x7fdab8155050 2024-09-17T02:41:29.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdab57fa640 1 -- 172.21.15.131:0/4033567545 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fdaa005f780 con 0x7fdab8155050 2024-09-17T02:41:29.069 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-17T02:41:29.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 -- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda98073230 msgr2=0x7fda98075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:29.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda98073230 0x7fda98075740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdab0005c50 tx=0x7fdab0005be0 comp rx=0 tx=0).stop 2024-09-17T02:41:29.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 -- 172.21.15.131:0/4033567545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8155050 msgr2=0x7fdab80c6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:29.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8155050 0x7fdab80c6f60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000c430 tx=0x7fdaa000c8f0 comp rx=0 tx=0).stop 2024-09-17T02:41:29.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 -- 172.21.15.131:0/4033567545 shutdown_connections 2024-09-17T02:41:29.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda98073230 0x7fda98075740 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:29.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdab8162330 0x7fdab81fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:29.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdab8155050 0x7fdab80c6f60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:29.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 --2- 172.21.15.131:0/4033567545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdab80cfb70 0x7fdab814a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:29.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 -- 172.21.15.131:0/4033567545 >> 172.21.15.131:0/4033567545 conn(0x7fdab80d1ba0 msgr2=0x7fdab814d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:29.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 -- 172.21.15.131:0/4033567545 shutdown_connections 2024-09-17T02:41:29.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:29.070+0000 7fdabedb4640 1 -- 172.21.15.131:0/4033567545 wait complete. 2024-09-17T02:41:29.090 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:41:29.090 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:41:29.090 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi189.front.sepia.ceph.com 2024-09-17T02:41:29.090 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T02:41:29.090 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:29.090 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:41:29.091 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:41:29.091 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:41:29.091 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T02:41:29.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:29.091 DEBUG:teuthology.orchestra.run.smithi189:> ip addr 2024-09-17T02:41:29.098 INFO:teuthology.orchestra.run.smithi189.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:41:29.098 INFO:teuthology.orchestra.run.smithi189.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:41:29.098 INFO:teuthology.orchestra.run.smithi189.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:41:29.098 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:29.098 INFO:teuthology.orchestra.run.smithi189.stdout: inet6 ::1/128 scope host 2024-09-17T02:41:29.098 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:29.098 INFO:teuthology.orchestra.run.smithi189.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:29.098 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 0c:c4:7a:d9:3b:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:29.098 INFO:teuthology.orchestra.run.smithi189.stdout: altname enp7s0f0 2024-09-17T02:41:29.098 INFO:teuthology.orchestra.run.smithi189.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:29.098 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 0c:c4:7a:d9:3b:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout: altname enp7s0f1 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 0c:c4:7a:88:6d:f0 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 0c:c4:7a:88:6d:f1 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout: inet 172.21.15.189/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft 42442sec preferred_lft 42442sec 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout: inet6 fe80::ec4:7aff:fe88:6df1/64 scope link 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 02:42:76:9a:15:8b brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:41:29.099 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:29.100 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T02:41:29.100 DEBUG:teuthology.orchestra.run.smithi189:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:29.100 DEBUG:teuthology.orchestra.run.smithi189:> set -e 2024-09-17T02:41:29.100 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip link add name ceph-brx type bridge 2024-09-17T02:41:29.100 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip addr flush dev ceph-brx 2024-09-17T02:41:29.100 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip link set ceph-brx up 2024-09-17T02:41:29.100 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T02:41:29.100 DEBUG:teuthology.orchestra.run.smithi189:> ') 2024-09-17T02:41:29.189 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:29.223 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:29.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:29.230 DEBUG:teuthology.orchestra.run.smithi189:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:41:29.239 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-09-17T02:41:29.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:29.240 DEBUG:teuthology.orchestra.run.smithi189:> route 2024-09-17T02:41:29.325 INFO:teuthology.orchestra.run.smithi189.stdout:Kernel IP routing table 2024-09-17T02:41:29.326 INFO:teuthology.orchestra.run.smithi189.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:41:29.326 INFO:teuthology.orchestra.run.smithi189.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:41:29.326 INFO:teuthology.orchestra.run.smithi189.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:41:29.326 INFO:teuthology.orchestra.run.smithi189.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:41:29.326 INFO:teuthology.orchestra.run.smithi189.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:41:29.326 INFO:teuthology.orchestra.run.smithi189.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:41:29.326 INFO:teuthology.orchestra.run.smithi189.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:29.326 INFO:teuthology.orchestra.run.smithi189.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:29.326 INFO:teuthology.orchestra.run.smithi189.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:29.326 INFO:teuthology.orchestra.run.smithi189.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:41:29.327 DEBUG:teuthology.orchestra.run.smithi189:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:29.327 DEBUG:teuthology.orchestra.run.smithi189:> set -e 2024-09-17T02:41:29.327 DEBUG:teuthology.orchestra.run.smithi189:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:41:29.327 DEBUG:teuthology.orchestra.run.smithi189:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:41:29.327 DEBUG:teuthology.orchestra.run.smithi189:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:41:29.327 DEBUG:teuthology.orchestra.run.smithi189:> ') 2024-09-17T02:41:29.373 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:29.403 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:29.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:29.410 DEBUG:teuthology.orchestra.run.smithi189:> ip netns list 2024-09-17T02:41:29.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:29.416 DEBUG:teuthology.orchestra.run.smithi189:> ip netns list-id 2024-09-17T02:41:29.461 DEBUG:teuthology.orchestra.run.smithi189:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:29.462 DEBUG:teuthology.orchestra.run.smithi189:> set -e 2024-09-17T02:41:29.462 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:29.462 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-17T02:41:29.462 DEBUG:teuthology.orchestra.run.smithi189:> ') 2024-09-17T02:41:29.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:29.570 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:29.578 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-17T02:41:29.578 DEBUG:teuthology.orchestra.run.smithi189:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:29.578 DEBUG:teuthology.orchestra.run.smithi189:> set -e 2024-09-17T02:41:29.578 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-17T02:41:29.578 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T02:41:29.578 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-17T02:41:29.578 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-17T02:41:29.578 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-17T02:41:29.579 DEBUG:teuthology.orchestra.run.smithi189:> ') 2024-09-17T02:41:29.618 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:29.678 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:29.685 DEBUG:teuthology.orchestra.run.smithi189:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:29.685 DEBUG:teuthology.orchestra.run.smithi189:> set -e 2024-09-17T02:41:29.685 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip link set brx.0 up 2024-09-17T02:41:29.685 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T02:41:29.685 DEBUG:teuthology.orchestra.run.smithi189:> ') 2024-09-17T02:41:29.766 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:29.784 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:29.791 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-17T02:41:29.791 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:41:29.791 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:29.841 INFO:teuthology.orchestra.run.smithi189.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T02:41:29.841 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:41:29.841 DEBUG:teuthology.orchestra.run.smithi189:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:29.890 DEBUG:teuthology.orchestra.run.smithi189:> sudo modprobe fuse 2024-09-17T02:41:29.947 DEBUG:teuthology.orchestra.run.smithi189:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:29.994 INFO:teuthology.orchestra.run.smithi189.stdout:/sys 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/proc 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/dev 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/pts 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/run 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/ 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/security 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/shm 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/run/lock 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/cgroup 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/pstore 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/bpf 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/hugepages 2024-09-17T02:41:29.995 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/mqueue 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/debug 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/tracing 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/fs/nfsd 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/config 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/run/rpc_pipefs 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/run/user/1000 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T02:41:29.996 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T02:41:29.997 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns 2024-09-17T02:41:29.997 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:29.997 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:29.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:29.998 DEBUG:teuthology.orchestra.run.smithi189:> ls /sys/fs/fuse/connections 2024-09-17T02:41:30.049 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:41:30.049 DEBUG:teuthology.orchestra.run.smithi189:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-17T02:41:30.094 DEBUG:teuthology.orchestra.run.smithi189:> sudo modprobe fuse 2024-09-17T02:41:30.110 DEBUG:teuthology.orchestra.run.smithi189:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:30.158 INFO:teuthology.orchestra.run.smithi189.stdout:/sys 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/proc 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/dev 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/pts 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/run 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/ 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/security 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/shm 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/run/lock 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/cgroup 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/pstore 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/bpf 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:30.159 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/hugepages 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/mqueue 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/debug 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/tracing 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/fs/nfsd 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/config 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/run/rpc_pipefs 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/run/user/1000 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T02:41:30.160 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T02:41:30.161 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T02:41:30.161 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns 2024-09-17T02:41:30.161 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:30.161 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:30.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:30.161 DEBUG:teuthology.orchestra.run.smithi189:> ls /sys/fs/fuse/connections 2024-09-17T02:41:30.201 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi189.stderr:2024-09-17T02:41:30.203+0000 7f436aaff580 -1 init, newargv = 0x562e9d20e670 newargc=15 2024-09-17T02:41:30.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi189.stderr:ceph-fuse[34507]: starting ceph client 2024-09-17T02:41:30.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi189.stderr:ceph-fuse[34507]: starting fuse 2024-09-17T02:41:31.306 DEBUG:teuthology.orchestra.run.smithi189:> sudo modprobe fuse 2024-09-17T02:41:31.322 DEBUG:teuthology.orchestra.run.smithi189:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:31.370 INFO:teuthology.orchestra.run.smithi189.stdout:/sys 2024-09-17T02:41:31.371 INFO:teuthology.orchestra.run.smithi189.stdout:/proc 2024-09-17T02:41:31.371 INFO:teuthology.orchestra.run.smithi189.stdout:/dev 2024-09-17T02:41:31.371 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/pts 2024-09-17T02:41:31.371 INFO:teuthology.orchestra.run.smithi189.stdout:/run 2024-09-17T02:41:31.371 INFO:teuthology.orchestra.run.smithi189.stdout:/ 2024-09-17T02:41:31.371 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/security 2024-09-17T02:41:31.371 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/shm 2024-09-17T02:41:31.371 INFO:teuthology.orchestra.run.smithi189.stdout:/run/lock 2024-09-17T02:41:31.371 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/cgroup 2024-09-17T02:41:31.371 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/pstore 2024-09-17T02:41:31.371 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/bpf 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/hugepages 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/mqueue 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/debug 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/tracing 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/fs/nfsd 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/config 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/run/rpc_pipefs 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/run/user/1000 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T02:41:31.372 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T02:41:31.373 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T02:41:31.373 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T02:41:31.373 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns 2024-09-17T02:41:31.373 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:31.373 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:31.373 INFO:teuthology.orchestra.run.smithi189.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:31.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:31.374 DEBUG:teuthology.orchestra.run.smithi189:> ls /sys/fs/fuse/connections 2024-09-17T02:41:31.422 INFO:teuthology.orchestra.run.smithi189.stdout:45 2024-09-17T02:41:31.422 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:41:31.422 DEBUG:teuthology.orchestra.run.smithi189:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:41:31.422 DEBUG:teuthology.orchestra.run.smithi189:> import glob 2024-09-17T02:41:31.422 DEBUG:teuthology.orchestra.run.smithi189:> import re 2024-09-17T02:41:31.422 DEBUG:teuthology.orchestra.run.smithi189:> import os 2024-09-17T02:41:31.422 DEBUG:teuthology.orchestra.run.smithi189:> import subprocess 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> def _find_admin_socket(client_name): 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> files = glob.glob(asok_path) 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> # Given a non-glob path, it better be there 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> if "*" not in asok_path: 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> assert(len(files) == 1) 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> return files[0] 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> for f in files: 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:41:31.423 DEBUG:teuthology.orchestra.run.smithi189:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:41:31.424 DEBUG:teuthology.orchestra.run.smithi189:> contents = proc_f.read() 2024-09-17T02:41:31.424 DEBUG:teuthology.orchestra.run.smithi189:> if mountpoint in contents: 2024-09-17T02:41:31.424 DEBUG:teuthology.orchestra.run.smithi189:> return f 2024-09-17T02:41:31.424 DEBUG:teuthology.orchestra.run.smithi189:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:41:31.424 DEBUG:teuthology.orchestra.run.smithi189:> 2024-09-17T02:41:31.424 DEBUG:teuthology.orchestra.run.smithi189:> print(_find_admin_socket("client.1")) 2024-09-17T02:41:31.424 DEBUG:teuthology.orchestra.run.smithi189:> ' 2024-09-17T02:41:31.547 INFO:teuthology.orchestra.run.smithi189.stdout:/var/run/ceph/ceph-client.1.34507.asok 2024-09-17T02:41:31.550 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:31.558 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34507.asok 2024-09-17T02:41:31.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:31.558 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34507.asok status 2024-09-17T02:41:31.742 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-17T02:41:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "metadata": { 2024-09-17T02:41:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:41:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:41:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "entity_id": "1", 2024-09-17T02:41:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "hostname": "smithi189", 2024-09-17T02:41:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-17T02:41:31.742 INFO:teuthology.orchestra.run.smithi189.stdout: "pid": "34507", 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "root": "/" 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "dentry_count": 0, 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "dentry_pinned_count": 0, 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "id": 4520, 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "inst": { 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "name": { 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "type": "client", 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "num": 4520 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "addr": { 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "type": "v1", 2024-09-17T02:41:31.743 INFO:teuthology.orchestra.run.smithi189.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:41:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "nonce": 1130661999 2024-09-17T02:41:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-17T02:41:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-17T02:41:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "addr": { 2024-09-17T02:41:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "type": "v1", 2024-09-17T02:41:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:41:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "nonce": 1130661999 2024-09-17T02:41:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-17T02:41:31.744 INFO:teuthology.orchestra.run.smithi189.stdout: "inst_str": "client.4520 192.168.0.1:0/1130661999", 2024-09-17T02:41:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "addr_str": "192.168.0.1:0/1130661999", 2024-09-17T02:41:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "inode_count": 1, 2024-09-17T02:41:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_epoch": 9, 2024-09-17T02:41:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_epoch": 26, 2024-09-17T02:41:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:41:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "blocklisted": false, 2024-09-17T02:41:31.745 INFO:teuthology.orchestra.run.smithi189.stdout: "fs_name": "cephfs" 2024-09-17T02:41:31.745 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-17T02:41:31.758 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T02:41:31.758 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:41:31.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.922+0000 7f8c682e6640 1 -- 172.21.15.131:0/3295008117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c6006e3e0 msgr2=0x7f8c6015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:31.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.922+0000 7f8c682e6640 1 --2- 172.21.15.131:0/3295008117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c6006e3e0 0x7f8c6015bea0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009900 tx=0x7f8c5001c8e0 comp rx=0 tx=0).stop 2024-09-17T02:41:31.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.922+0000 7f8c682e6640 1 -- 172.21.15.131:0/3295008117 shutdown_connections 2024-09-17T02:41:31.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.922+0000 7f8c682e6640 1 --2- 172.21.15.131:0/3295008117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c6006e3e0 0x7f8c6015bea0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:31.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.922+0000 7f8c682e6640 1 --2- 172.21.15.131:0/3295008117 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c600ced00 0x7f8c6006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:31.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.922+0000 7f8c682e6640 1 -- 172.21.15.131:0/3295008117 >> 172.21.15.131:0/3295008117 conn(0x7f8c6006eea0 msgr2=0x7f8c6006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:31.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.922+0000 7f8c682e6640 1 -- 172.21.15.131:0/3295008117 shutdown_connections 2024-09-17T02:41:31.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.922+0000 7f8c682e6640 1 -- 172.21.15.131:0/3295008117 wait complete. 2024-09-17T02:41:31.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c682e6640 1 Processor -- start 2024-09-17T02:41:31.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c682e6640 1 -- start start 2024-09-17T02:41:31.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c682e6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c6006e3e0 0x7f8c600aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:31.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c682e6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c600ced00 0x7f8c60153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:31.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c682e6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c6015c3e0 0x7f8c60153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:31.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c682e6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8c60168b60 con 0x7f8c6006e3e0 2024-09-17T02:41:31.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c682e6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8c601689e0 con 0x7f8c600ced00 2024-09-17T02:41:31.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c682e6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8c60168ce0 con 0x7f8c6015c3e0 2024-09-17T02:41:31.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c66082640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c6006e3e0 0x7f8c600aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:31.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c66082640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c6006e3e0 0x7f8c600aeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52238/0 (socket says 172.21.15.131:52238) 2024-09-17T02:41:31.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c66082640 1 -- 172.21.15.131:0/1222726073 learned_addr learned my addr 172.21.15.131:0/1222726073 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:31.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c66883640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c6015c3e0 0x7f8c60153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:31.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c66082640 1 -- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c6015c3e0 msgr2=0x7f8c60153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:31.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c66082640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c6015c3e0 0x7f8c60153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:31.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c66082640 1 -- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c600ced00 msgr2=0x7f8c60153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:31.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c66082640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c600ced00 0x7f8c60153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:31.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c66082640 1 -- 172.21.15.131:0/1222726073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c601fc1d0 con 0x7f8c6006e3e0 2024-09-17T02:41:31.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c66883640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c6015c3e0 0x7f8c60153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:31.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c66082640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c6006e3e0 0x7f8c600aeef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0058d0 tx=0x7f8c5c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:31.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c56ffd640 1 -- 172.21.15.131:0/1222726073 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c017070 con 0x7f8c6006e3e0 2024-09-17T02:41:31.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c56ffd640 1 -- 172.21.15.131:0/1222726073 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c5c009d20 con 0x7f8c6006e3e0 2024-09-17T02:41:31.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c682e6640 1 -- 172.21.15.131:0/1222726073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c601fc4b0 con 0x7f8c6006e3e0 2024-09-17T02:41:31.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c56ffd640 1 -- 172.21.15.131:0/1222726073 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0106c0 con 0x7f8c6006e3e0 2024-09-17T02:41:31.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.926+0000 7f8c682e6640 1 -- 172.21.15.131:0/1222726073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c601fca70 con 0x7f8c6006e3e0 2024-09-17T02:41:31.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.930+0000 7f8c682e6640 1 -- 172.21.15.131:0/1222726073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c6015bea0 con 0x7f8c6006e3e0 2024-09-17T02:41:31.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.930+0000 7f8c56ffd640 1 -- 172.21.15.131:0/1222726073 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8c5c01a070 con 0x7f8c6006e3e0 2024-09-17T02:41:31.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.930+0000 7f8c56ffd640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c44073230 0x7f8c44075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:31.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.930+0000 7f8c56ffd640 1 -- 172.21.15.131:0/1222726073 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6532+0+0 (secure 0 0 0) 0x7f8c5c0946d0 con 0x7f8c6006e3e0 2024-09-17T02:41:31.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.930+0000 7f8c65881640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c44073230 0x7f8c44075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:31.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.930+0000 7f8c65881640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c44073230 0x7f8c44075740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c500af000 tx=0x7f8c50019680 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:31.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:31.934+0000 7f8c56ffd640 1 -- 172.21.15.131:0/1222726073 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8c5c05f3c0 con 0x7f8c6006e3e0 2024-09-17T02:41:32.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.146+0000 7f8c682e6640 1 -- 172.21.15.131:0/1222726073 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8c6006f4c0 con 0x7f8c6006e3e0 2024-09-17T02:41:32.150 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-17T02:41:32.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.150+0000 7f8c56ffd640 1 -- 172.21.15.131:0/1222726073 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f8c5c05ecc0 con 0x7f8c6006e3e0 2024-09-17T02:41:32.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.158+0000 7f8c682e6640 1 -- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c44073230 msgr2=0x7f8c44075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:32.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.158+0000 7f8c682e6640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c44073230 0x7f8c44075740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c500af000 tx=0x7f8c50019680 comp rx=0 tx=0).stop 2024-09-17T02:41:32.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.158+0000 7f8c682e6640 1 -- 172.21.15.131:0/1222726073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c6006e3e0 msgr2=0x7f8c600aeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:32.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.158+0000 7f8c682e6640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c6006e3e0 0x7f8c600aeef0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c0058d0 tx=0x7f8c5c00dde0 comp rx=0 tx=0).stop 2024-09-17T02:41:32.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.162+0000 7f8c682e6640 1 -- 172.21.15.131:0/1222726073 shutdown_connections 2024-09-17T02:41:32.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.162+0000 7f8c682e6640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c44073230 0x7f8c44075740 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:32.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.162+0000 7f8c682e6640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c6015c3e0 0x7f8c60153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:32.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.162+0000 7f8c682e6640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c600ced00 0x7f8c60153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:32.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.162+0000 7f8c682e6640 1 --2- 172.21.15.131:0/1222726073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c6006e3e0 0x7f8c600aeef0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:32.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.162+0000 7f8c682e6640 1 -- 172.21.15.131:0/1222726073 >> 172.21.15.131:0/1222726073 conn(0x7f8c6006eea0 msgr2=0x7f8c600afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:32.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.162+0000 7f8c682e6640 1 -- 172.21.15.131:0/1222726073 shutdown_connections 2024-09-17T02:41:32.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:32.162+0000 7f8c682e6640 1 -- 172.21.15.131:0/1222726073 wait complete. 2024-09-17T02:41:32.182 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:41:32.182 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:41:32.182 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi189.front.sepia.ceph.com 2024-09-17T02:41:32.182 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-17T02:41:32.182 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T02:41:32.182 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:41:32.183 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:41:32.183 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:41:32.183 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-17T02:41:32.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:32.183 DEBUG:teuthology.orchestra.run.smithi189:> ip addr 2024-09-17T02:41:32.190 INFO:teuthology.orchestra.run.smithi189.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:41:32.190 INFO:teuthology.orchestra.run.smithi189.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:41:32.190 INFO:teuthology.orchestra.run.smithi189.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:41:32.190 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.190 INFO:teuthology.orchestra.run.smithi189.stdout: inet6 ::1/128 scope host 2024-09-17T02:41:32.190 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.190 INFO:teuthology.orchestra.run.smithi189.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:32.190 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 0c:c4:7a:d9:3b:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:32.190 INFO:teuthology.orchestra.run.smithi189.stdout: altname enp7s0f0 2024-09-17T02:41:32.190 INFO:teuthology.orchestra.run.smithi189.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:32.190 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 0c:c4:7a:d9:3b:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout: altname enp7s0f1 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 0c:c4:7a:88:6d:f0 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 0c:c4:7a:88:6d:f1 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout: inet 172.21.15.189/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft 42439sec preferred_lft 42439sec 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout: inet6 fe80::ec4:7aff:fe88:6df1/64 scope link 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 02:42:76:9a:15:8b brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:41:32.191 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.192 INFO:teuthology.orchestra.run.smithi189.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:41:32.192 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:41:32.192 INFO:teuthology.orchestra.run.smithi189.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:41:32.192 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.192 INFO:teuthology.orchestra.run.smithi189.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:41:32.192 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.192 INFO:teuthology.orchestra.run.smithi189.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:41:32.192 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:32.192 INFO:teuthology.orchestra.run.smithi189.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:41:32.192 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:32.193 DEBUG:teuthology.orchestra.run.smithi189:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:41:32.245 INFO:teuthology.orchestra.run.smithi189.stdout:1 2024-09-17T02:41:32.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:32.246 DEBUG:teuthology.orchestra.run.smithi189:> route 2024-09-17T02:41:32.301 INFO:teuthology.orchestra.run.smithi189.stdout:Kernel IP routing table 2024-09-17T02:41:32.301 INFO:teuthology.orchestra.run.smithi189.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:41:32.301 INFO:teuthology.orchestra.run.smithi189.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:41:32.301 INFO:teuthology.orchestra.run.smithi189.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:41:32.301 INFO:teuthology.orchestra.run.smithi189.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:41:32.301 INFO:teuthology.orchestra.run.smithi189.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:41:32.301 INFO:teuthology.orchestra.run.smithi189.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:41:32.302 INFO:teuthology.orchestra.run.smithi189.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:32.302 INFO:teuthology.orchestra.run.smithi189.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:32.302 INFO:teuthology.orchestra.run.smithi189.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:41:32.302 INFO:teuthology.orchestra.run.smithi189.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:41:32.302 DEBUG:teuthology.orchestra.run.smithi189:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:32.302 DEBUG:teuthology.orchestra.run.smithi189:> set -e 2024-09-17T02:41:32.302 DEBUG:teuthology.orchestra.run.smithi189:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:41:32.302 DEBUG:teuthology.orchestra.run.smithi189:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:41:32.302 DEBUG:teuthology.orchestra.run.smithi189:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:41:32.302 DEBUG:teuthology.orchestra.run.smithi189:> ') 2024-09-17T02:41:32.395 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:32.424 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:32.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:32.431 DEBUG:teuthology.orchestra.run.smithi189:> ip netns list 2024-09-17T02:41:32.478 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T02:41:32.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:32.478 DEBUG:teuthology.orchestra.run.smithi189:> ip netns list-id 2024-09-17T02:41:32.530 INFO:teuthology.orchestra.run.smithi189.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-17T02:41:32.530 DEBUG:teuthology.orchestra.run.smithi189:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:32.530 DEBUG:teuthology.orchestra.run.smithi189:> set -e 2024-09-17T02:41:32.530 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T02:41:32.530 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-17T02:41:32.530 DEBUG:teuthology.orchestra.run.smithi189:> ') 2024-09-17T02:41:32.619 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:32.641 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:32.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:32.648 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T02:41:32.659 INFO:teuthology.orchestra.run.smithi189.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:41:32.659 INFO:teuthology.orchestra.run.smithi189.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:41:32.659 INFO:teuthology.orchestra.run.smithi189.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:41:32.659 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.660 INFO:teuthology.orchestra.run.smithi189.stdout: inet6 ::1/128 scope host 2024-09-17T02:41:32.660 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.660 INFO:teuthology.orchestra.run.smithi189.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:41:32.660 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 4e:ec:56:a8:ce:f4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T02:41:32.660 INFO:teuthology.orchestra.run.smithi189.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T02:41:32.660 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.660 INFO:teuthology.orchestra.run.smithi189.stdout: inet6 fe80::4cec:56ff:fea8:cef4/64 scope link 2024-09-17T02:41:32.660 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:32.661 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T02:41:32.716 INFO:teuthology.orchestra.run.smithi189.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:41:32.716 INFO:teuthology.orchestra.run.smithi189.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:41:32.716 INFO:teuthology.orchestra.run.smithi189.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:41:32.716 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.716 INFO:teuthology.orchestra.run.smithi189.stdout: inet6 ::1/128 scope host 2024-09-17T02:41:32.717 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.717 INFO:teuthology.orchestra.run.smithi189.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:41:32.717 INFO:teuthology.orchestra.run.smithi189.stdout: link/ether 4e:ec:56:a8:ce:f4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T02:41:32.717 INFO:teuthology.orchestra.run.smithi189.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T02:41:32.717 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.717 INFO:teuthology.orchestra.run.smithi189.stdout: inet6 fe80::4cec:56ff:fea8:cef4/64 scope link 2024-09-17T02:41:32.717 INFO:teuthology.orchestra.run.smithi189.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:41:32.718 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-17T02:41:32.718 DEBUG:teuthology.orchestra.run.smithi189:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:32.718 DEBUG:teuthology.orchestra.run.smithi189:> set -e 2024-09-17T02:41:32.718 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-17T02:41:32.718 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T02:41:32.718 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-17T02:41:32.719 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-17T02:41:32.719 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-17T02:41:32.719 DEBUG:teuthology.orchestra.run.smithi189:> ') 2024-09-17T02:41:32.806 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:32.853 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:32.860 DEBUG:teuthology.orchestra.run.smithi189:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:41:32.860 DEBUG:teuthology.orchestra.run.smithi189:> set -e 2024-09-17T02:41:32.860 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip link set brx.1 up 2024-09-17T02:41:32.860 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T02:41:32.860 DEBUG:teuthology.orchestra.run.smithi189:> ') 2024-09-17T02:41:32.899 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:41:32.914 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:32.921 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-17T02:41:32.921 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:41:32.921 DEBUG:teuthology.orchestra.run.smithi189:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-17T02:41:32.969 INFO:teuthology.orchestra.run.smithi189.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-17T02:41:32.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:41:32.970 DEBUG:teuthology.orchestra.run.smithi189:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-17T02:41:33.017 DEBUG:teuthology.orchestra.run.smithi189:> sudo modprobe fuse 2024-09-17T02:41:33.075 DEBUG:teuthology.orchestra.run.smithi189:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:33.122 INFO:teuthology.orchestra.run.smithi189.stdout:/sys 2024-09-17T02:41:33.123 INFO:teuthology.orchestra.run.smithi189.stdout:/proc 2024-09-17T02:41:33.123 INFO:teuthology.orchestra.run.smithi189.stdout:/dev 2024-09-17T02:41:33.123 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/pts 2024-09-17T02:41:33.123 INFO:teuthology.orchestra.run.smithi189.stdout:/run 2024-09-17T02:41:33.123 INFO:teuthology.orchestra.run.smithi189.stdout:/ 2024-09-17T02:41:33.123 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/security 2024-09-17T02:41:33.123 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/shm 2024-09-17T02:41:33.123 INFO:teuthology.orchestra.run.smithi189.stdout:/run/lock 2024-09-17T02:41:33.123 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/cgroup 2024-09-17T02:41:33.123 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/pstore 2024-09-17T02:41:33.123 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/bpf 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/hugepages 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/mqueue 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/debug 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/tracing 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/fs/nfsd 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/config 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/run/rpc_pipefs 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/run/user/1000 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph 2024-09-17T02:41:33.124 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T02:41:33.125 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T02:41:33.125 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T02:41:33.125 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T02:41:33.125 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns 2024-09-17T02:41:33.125 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:33.125 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:33.125 INFO:teuthology.orchestra.run.smithi189.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:33.125 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T02:41:33.125 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T02:41:33.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:33.126 DEBUG:teuthology.orchestra.run.smithi189:> ls /sys/fs/fuse/connections 2024-09-17T02:41:33.178 INFO:teuthology.orchestra.run.smithi189.stdout:45 2024-09-17T02:41:33.179 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-17T02:41:33.179 DEBUG:teuthology.orchestra.run.smithi189:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-17T02:41:33.222 DEBUG:teuthology.orchestra.run.smithi189:> sudo modprobe fuse 2024-09-17T02:41:33.236 DEBUG:teuthology.orchestra.run.smithi189:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:33.282 INFO:teuthology.orchestra.run.smithi189.stdout:/sys 2024-09-17T02:41:33.282 INFO:teuthology.orchestra.run.smithi189.stdout:/proc 2024-09-17T02:41:33.282 INFO:teuthology.orchestra.run.smithi189.stdout:/dev 2024-09-17T02:41:33.282 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/pts 2024-09-17T02:41:33.282 INFO:teuthology.orchestra.run.smithi189.stdout:/run 2024-09-17T02:41:33.282 INFO:teuthology.orchestra.run.smithi189.stdout:/ 2024-09-17T02:41:33.282 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/security 2024-09-17T02:41:33.282 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/shm 2024-09-17T02:41:33.282 INFO:teuthology.orchestra.run.smithi189.stdout:/run/lock 2024-09-17T02:41:33.282 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/cgroup 2024-09-17T02:41:33.282 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/pstore 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/bpf 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/hugepages 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/mqueue 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/debug 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/tracing 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/fs/nfsd 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/config 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/run/rpc_pipefs 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/run/user/1000 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph 2024-09-17T02:41:33.283 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T02:41:33.284 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T02:41:33.284 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T02:41:33.284 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T02:41:33.284 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns 2024-09-17T02:41:33.284 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:33.284 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:33.284 INFO:teuthology.orchestra.run.smithi189.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:33.284 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T02:41:33.284 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T02:41:33.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:33.285 DEBUG:teuthology.orchestra.run.smithi189:> ls /sys/fs/fuse/connections 2024-09-17T02:41:33.328 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi189.stderr:2024-09-17T02:41:33.327+0000 7fc664cea580 -1 init, newargv = 0x7fc6540046b0 newargc=15 2024-09-17T02:41:33.329 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi189.stderr:ceph-fuse[34626]: starting ceph client 2024-09-17T02:41:33.333 INFO:teuthology.orchestra.run.smithi189.stdout:45 2024-09-17T02:41:33.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi189.stderr:ceph-fuse[34626]: starting fuse 2024-09-17T02:41:34.434 DEBUG:teuthology.orchestra.run.smithi189:> sudo modprobe fuse 2024-09-17T02:41:34.451 DEBUG:teuthology.orchestra.run.smithi189:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:41:34.502 INFO:teuthology.orchestra.run.smithi189.stdout:/sys 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/proc 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/dev 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/pts 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/run 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/ 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/security 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/shm 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/run/lock 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/cgroup 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/pstore 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/bpf 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:41:34.503 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/hugepages 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/dev/mqueue 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/debug 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/tracing 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/proc/fs/nfsd 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/fs/fuse/connections 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/sys/kernel/config 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/run/rpc_pipefs 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/run/user/1000 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T02:41:34.504 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T02:41:34.505 INFO:teuthology.orchestra.run.smithi189.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T02:41:34.505 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns 2024-09-17T02:41:34.505 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:34.505 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:41:34.505 INFO:teuthology.orchestra.run.smithi189.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:34.505 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T02:41:34.505 INFO:teuthology.orchestra.run.smithi189.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T02:41:34.505 INFO:teuthology.orchestra.run.smithi189.stdout:/home/ubuntu/cephtest/mnt.3 2024-09-17T02:41:34.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:34.506 DEBUG:teuthology.orchestra.run.smithi189:> ls /sys/fs/fuse/connections 2024-09-17T02:41:34.554 INFO:teuthology.orchestra.run.smithi189.stdout:45 2024-09-17T02:41:34.554 INFO:teuthology.orchestra.run.smithi189.stdout:46 2024-09-17T02:41:34.555 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-17T02:41:34.555 DEBUG:teuthology.orchestra.run.smithi189:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:41:34.555 DEBUG:teuthology.orchestra.run.smithi189:> import glob 2024-09-17T02:41:34.555 DEBUG:teuthology.orchestra.run.smithi189:> import re 2024-09-17T02:41:34.555 DEBUG:teuthology.orchestra.run.smithi189:> import os 2024-09-17T02:41:34.555 DEBUG:teuthology.orchestra.run.smithi189:> import subprocess 2024-09-17T02:41:34.555 DEBUG:teuthology.orchestra.run.smithi189:> 2024-09-17T02:41:34.556 DEBUG:teuthology.orchestra.run.smithi189:> def _find_admin_socket(client_name): 2024-09-17T02:41:34.556 DEBUG:teuthology.orchestra.run.smithi189:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-17T02:41:34.556 DEBUG:teuthology.orchestra.run.smithi189:> files = glob.glob(asok_path) 2024-09-17T02:41:34.556 DEBUG:teuthology.orchestra.run.smithi189:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-17T02:41:34.556 DEBUG:teuthology.orchestra.run.smithi189:> 2024-09-17T02:41:34.556 DEBUG:teuthology.orchestra.run.smithi189:> # Given a non-glob path, it better be there 2024-09-17T02:41:34.556 DEBUG:teuthology.orchestra.run.smithi189:> if "*" not in asok_path: 2024-09-17T02:41:34.556 DEBUG:teuthology.orchestra.run.smithi189:> assert(len(files) == 1) 2024-09-17T02:41:34.556 DEBUG:teuthology.orchestra.run.smithi189:> return files[0] 2024-09-17T02:41:34.556 DEBUG:teuthology.orchestra.run.smithi189:> 2024-09-17T02:41:34.556 DEBUG:teuthology.orchestra.run.smithi189:> for f in files: 2024-09-17T02:41:34.557 DEBUG:teuthology.orchestra.run.smithi189:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:41:34.557 DEBUG:teuthology.orchestra.run.smithi189:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:41:34.557 DEBUG:teuthology.orchestra.run.smithi189:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:41:34.557 DEBUG:teuthology.orchestra.run.smithi189:> contents = proc_f.read() 2024-09-17T02:41:34.557 DEBUG:teuthology.orchestra.run.smithi189:> if mountpoint in contents: 2024-09-17T02:41:34.557 DEBUG:teuthology.orchestra.run.smithi189:> return f 2024-09-17T02:41:34.557 DEBUG:teuthology.orchestra.run.smithi189:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:41:34.557 DEBUG:teuthology.orchestra.run.smithi189:> 2024-09-17T02:41:34.557 DEBUG:teuthology.orchestra.run.smithi189:> print(_find_admin_socket("client.3")) 2024-09-17T02:41:34.557 DEBUG:teuthology.orchestra.run.smithi189:> ' 2024-09-17T02:41:34.679 INFO:teuthology.orchestra.run.smithi189.stdout:/var/run/ceph/ceph-client.3.34626.asok 2024-09-17T02:41:34.682 INFO:teuthology.orchestra.run.smithi189.stderr:2024-09-17T02:41:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:41:34.691 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.34626.asok 2024-09-17T02:41:34.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:34.691 DEBUG:teuthology.orchestra.run.smithi189:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.34626.asok status 2024-09-17T02:41:34.870 INFO:teuthology.orchestra.run.smithi189.stdout:{ 2024-09-17T02:41:34.870 INFO:teuthology.orchestra.run.smithi189.stdout: "metadata": { 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "entity_id": "3", 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "hostname": "smithi189", 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "pid": "34626", 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "root": "/" 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "dentry_count": 0, 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "dentry_pinned_count": 0, 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "id": 4529, 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "inst": { 2024-09-17T02:41:34.871 INFO:teuthology.orchestra.run.smithi189.stdout: "name": { 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: "type": "client", 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: "num": 4529 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: "addr": { 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: "type": "v1", 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: "addr": "192.168.0.2:0", 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: "nonce": 3702049340 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: } 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: "addr": { 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: "type": "v1", 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: "addr": "192.168.0.2:0", 2024-09-17T02:41:34.872 INFO:teuthology.orchestra.run.smithi189.stdout: "nonce": 3702049340 2024-09-17T02:41:34.873 INFO:teuthology.orchestra.run.smithi189.stdout: }, 2024-09-17T02:41:34.873 INFO:teuthology.orchestra.run.smithi189.stdout: "inst_str": "client.4529 192.168.0.2:0/3702049340", 2024-09-17T02:41:34.873 INFO:teuthology.orchestra.run.smithi189.stdout: "addr_str": "192.168.0.2:0/3702049340", 2024-09-17T02:41:34.873 INFO:teuthology.orchestra.run.smithi189.stdout: "inode_count": 1, 2024-09-17T02:41:34.873 INFO:teuthology.orchestra.run.smithi189.stdout: "mds_epoch": 9, 2024-09-17T02:41:34.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_epoch": 28, 2024-09-17T02:41:34.873 INFO:teuthology.orchestra.run.smithi189.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:41:34.873 INFO:teuthology.orchestra.run.smithi189.stdout: "blocklisted": false, 2024-09-17T02:41:34.873 INFO:teuthology.orchestra.run.smithi189.stdout: "fs_name": "cephfs" 2024-09-17T02:41:34.873 INFO:teuthology.orchestra.run.smithi189.stdout:} 2024-09-17T02:41:34.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:34.886 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:41:34.886 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:34.894 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:41:34.894 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:34.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:34.894 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:34.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:34.950 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:41:34.950 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-17T02:41:34.997 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:41:34.998 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-17T02:41:34.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:34.998 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-17T02:41:35.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:35.054 DEBUG:teuthology.orchestra.run.smithi189:> stat --file-system '--printf=%T 2024-09-17T02:41:35.054 DEBUG:teuthology.orchestra.run.smithi189:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:35.061 INFO:teuthology.orchestra.run.smithi189.stdout:fuseblk 2024-09-17T02:41:35.061 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:35.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:35.062 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:35.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:35.118 DEBUG:teuthology.orchestra.run.smithi189:> stat --file-system '--printf=%T 2024-09-17T02:41:35.119 DEBUG:teuthology.orchestra.run.smithi189:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-17T02:41:35.169 INFO:teuthology.orchestra.run.smithi189.stdout:fuseblk 2024-09-17T02:41:35.169 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-17T02:41:35.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:35.169 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-17T02:41:35.230 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-17T02:41:35.279 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-17T02:41:35.280 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:41:35.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.445+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/2766026320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bb40cbb70 msgr2=0x7f3bb4165b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:35.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.445+0000 7f3bb8e1a640 1 --2- 172.21.15.131:0/2766026320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bb40cbb70 0x7f3bb4165b60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009bc0 tx=0x7f3bac01cba0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.445+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/2766026320 shutdown_connections 2024-09-17T02:41:35.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.445+0000 7f3bb8e1a640 1 --2- 172.21.15.131:0/2766026320 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bb40cbb70 0x7f3bb4165b60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.445+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/2766026320 >> 172.21.15.131:0/2766026320 conn(0x7f3bb406eea0 msgr2=0x7f3bb406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:35.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/2766026320 shutdown_connections 2024-09-17T02:41:35.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/2766026320 wait complete. 2024-09-17T02:41:35.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb8e1a640 1 Processor -- start 2024-09-17T02:41:35.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb8e1a640 1 -- start start 2024-09-17T02:41:35.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb8e1a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bb406dea0 0x7f3bb40b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:35.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb8e1a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bb40cbb70 0x7f3bb40ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:35.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb8e1a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3bb4155240 0x7f3bb4152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:35.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb8e1a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4169be0 con 0x7f3bb40cbb70 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb8e1a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4070060 con 0x7f3bb406dea0 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb8e1a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb4169d60 con 0x7f3bb4155240 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb1d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bb40cbb70 0x7f3bb40ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb1d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bb40cbb70 0x7f3bb40ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52264/0 (socket says 172.21.15.131:52264) 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb1d9b640 1 -- 172.21.15.131:0/3258692714 learned_addr learned my addr 172.21.15.131:0/3258692714 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb2d9d640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3bb4155240 0x7f3bb4152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb259c640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bb406dea0 0x7f3bb40b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb1d9b640 1 -- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3bb4155240 msgr2=0x7f3bb4152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb1d9b640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3bb4155240 0x7f3bb4152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb1d9b640 1 -- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bb406dea0 msgr2=0x7f3bb40b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb1d9b640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bb406dea0 0x7f3bb40b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.449+0000 7f3bb1d9b640 1 -- 172.21.15.131:0/3258692714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb40c6ec0 con 0x7f3bb40cbb70 2024-09-17T02:41:35.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3bb1d9b640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bb40cbb70 0x7f3bb40ae870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c420 tx=0x7f3ba800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:35.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3ba2ffd640 1 -- 172.21.15.131:0/3258692714 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8011790 con 0x7f3bb40cbb70 2024-09-17T02:41:35.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3ba2ffd640 1 -- 172.21.15.131:0/3258692714 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ba8017070 con 0x7f3bb40cbb70 2024-09-17T02:41:35.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/3258692714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb40af100 con 0x7f3bb40cbb70 2024-09-17T02:41:35.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3ba2ffd640 1 -- 172.21.15.131:0/3258692714 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8011ce0 con 0x7f3bb40cbb70 2024-09-17T02:41:35.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/3258692714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb40c7180 con 0x7f3bb40cbb70 2024-09-17T02:41:35.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/3258692714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb40aee70 con 0x7f3bb40cbb70 2024-09-17T02:41:35.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3ba2ffd640 1 -- 172.21.15.131:0/3258692714 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ba801c460 con 0x7f3bb40cbb70 2024-09-17T02:41:35.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3ba2ffd640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3b94073200 0x7f3b94075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:35.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3ba2ffd640 1 -- 172.21.15.131:0/3258692714 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f3ba80943d0 con 0x7f3bb40cbb70 2024-09-17T02:41:35.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3bb259c640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3b94073200 0x7f3b94075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:35.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.453+0000 7f3bb259c640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3b94073200 0x7f3b94075710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400a750 tx=0x7f3ba4008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:35.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.457+0000 7f3ba2ffd640 1 -- 172.21.15.131:0/3258692714 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ba805f040 con 0x7f3bb40cbb70 2024-09-17T02:41:35.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.633+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/3258692714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bb4073430 con 0x7f3bb40cbb70 2024-09-17T02:41:35.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3ba2ffd640 1 -- 172.21.15.131:0/3258692714 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18839 (secure 0 0 0) 0x7f3ba805e940 con 0x7f3bb40cbb70 2024-09-17T02:41:35.637 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:35.637 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":28,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:41:33.147294+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.38","mappings":[{"from":0,"to":7}]},{"pgid":"4.e","mappings":[{"from":1,"to":0}]},{"pgid":"4.15","mappings":[{"from":3,"to":2}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:41:35.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3b94073200 msgr2=0x7f3b94075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:35.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3b94073200 0x7f3b94075710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400a750 tx=0x7f3ba4008040 comp rx=0 tx=0).stop 2024-09-17T02:41:35.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/3258692714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bb40cbb70 msgr2=0x7f3bb40ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:35.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bb40cbb70 0x7f3bb40ae870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800c420 tx=0x7f3ba800c8e0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/3258692714 shutdown_connections 2024-09-17T02:41:35.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3b94073200 0x7f3b94075710 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3bb4155240 0x7f3bb4152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bb40cbb70 0x7f3bb40ae870 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 --2- 172.21.15.131:0/3258692714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bb406dea0 0x7f3bb40b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/3258692714 >> 172.21.15.131:0/3258692714 conn(0x7f3bb406eea0 msgr2=0x7f3bb406d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:35.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/3258692714 shutdown_connections 2024-09-17T02:41:35.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.637+0000 7f3bb8e1a640 1 -- 172.21.15.131:0/3258692714 wait complete. 2024-09-17T02:41:35.660 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T02:40:41.163614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-17T02:40:50.086025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-17T02:40:50.546353+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-17T02:40:54.274506+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-17T02:41:35.660 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T02:41:35.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.817+0000 7ff42d21f640 1 -- 172.21.15.131:0/1942297673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff428155240 msgr2=0x7ff428073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:35.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.817+0000 7ff42d21f640 1 --2- 172.21.15.131:0/1942297673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff428155240 0x7ff428073430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff4280cdb80 tx=0x7ff42001cb40 comp rx=0 tx=0).stop 2024-09-17T02:41:35.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.817+0000 7ff42d21f640 1 -- 172.21.15.131:0/1942297673 shutdown_connections 2024-09-17T02:41:35.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.817+0000 7ff42d21f640 1 --2- 172.21.15.131:0/1942297673 >> v2:172.21.15.189:3300/0 conn(0x7ff42806e5b0 0x7ff4280cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.817+0000 7ff42d21f640 1 --2- 172.21.15.131:0/1942297673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff428155240 0x7ff428073430 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.817+0000 7ff42d21f640 1 -- 172.21.15.131:0/1942297673 >> 172.21.15.131:0/1942297673 conn(0x7ff42806eea0 msgr2=0x7ff42806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:35.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.817+0000 7ff42d21f640 1 -- 172.21.15.131:0/1942297673 shutdown_connections 2024-09-17T02:41:35.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.817+0000 7ff42d21f640 1 -- 172.21.15.131:0/1942297673 wait complete. 2024-09-17T02:41:35.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.817+0000 7ff42d21f640 1 Processor -- start 2024-09-17T02:41:35.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.817+0000 7ff42d21f640 1 -- start start 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42d21f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff428073970 0x7ff42815a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42d21f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4280ab170 0x7ff4280ab580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42d21f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4280a9470 0x7ff4280a9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42d21f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280cbd80 con 0x7ff428073970 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42d21f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280cbc00 con 0x7ff4280ab170 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42d21f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff4280cbf00 con 0x7ff4280a9470 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff426d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff428073970 0x7ff42815a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff426d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff428073970 0x7ff42815a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52290/0 (socket says 172.21.15.131:52290) 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff426d9d640 1 -- 172.21.15.131:0/1014033962 learned_addr learned my addr 172.21.15.131:0/1014033962 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42659c640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4280ab170 0x7ff4280ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42759e640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4280a9470 0x7ff4280a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff426d9d640 1 -- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4280a9470 msgr2=0x7ff4280a9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:35.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff426d9d640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4280a9470 0x7ff4280a9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff426d9d640 1 -- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4280ab170 msgr2=0x7ff4280ab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff426d9d640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4280ab170 0x7ff4280ab580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff426d9d640 1 -- 172.21.15.131:0/1014033962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4281539f0 con 0x7ff428073970 2024-09-17T02:41:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42759e640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4280a9470 0x7ff4280a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff426d9d640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff428073970 0x7ff42815a570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff420000f90 tx=0x7ff420003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff41f7fe640 1 -- 172.21.15.131:0/1014033962 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4200ad640 con 0x7ff428073970 2024-09-17T02:41:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff41f7fe640 1 -- 172.21.15.131:0/1014033962 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4200b8030 con 0x7ff428073970 2024-09-17T02:41:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff41f7fe640 1 -- 172.21.15.131:0/1014033962 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4200adb50 con 0x7ff428073970 2024-09-17T02:41:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42d21f640 1 -- 172.21.15.131:0/1014033962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff428152d80 con 0x7ff428073970 2024-09-17T02:41:35.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42d21f640 1 -- 172.21.15.131:0/1014033962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff428153340 con 0x7ff428073970 2024-09-17T02:41:35.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff42d21f640 1 -- 172.21.15.131:0/1014033962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4280cab60 con 0x7ff428073970 2024-09-17T02:41:35.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.821+0000 7ff41f7fe640 1 -- 172.21.15.131:0/1014033962 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff4200adcf0 con 0x7ff428073970 2024-09-17T02:41:35.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.825+0000 7ff41f7fe640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff408073200 0x7ff408075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:35.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.825+0000 7ff41f7fe640 1 -- 172.21.15.131:0/1014033962 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7ff4201309a0 con 0x7ff428073970 2024-09-17T02:41:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.825+0000 7ff42659c640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff408073200 0x7ff408075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:35.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.825+0000 7ff42659c640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff408073200 0x7ff408075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff4280b0e40 tx=0x7ff410009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:35.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:35.829+0000 7ff41f7fe640 1 -- 172.21.15.131:0/1014033962 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff420016730 con 0x7ff428073970 2024-09-17T02:41:36.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.001+0000 7ff42d21f640 1 -- 172.21.15.131:0/1014033962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4280ca130 con 0x7ff428073970 2024-09-17T02:41:36.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.001+0000 7ff41f7fe640 1 -- 172.21.15.131:0/1014033962 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) ==== 93+0+10 (secure 0 0 0) 0x7ff4200fb6a0 con 0x7ff428073970 2024-09-17T02:41:36.002 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-09-17T02:41:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 -- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff408073200 msgr2=0x7ff408075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff408073200 0x7ff408075710 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff4280b0e40 tx=0x7ff410009040 comp rx=0 tx=0).stop 2024-09-17T02:41:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 -- 172.21.15.131:0/1014033962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff428073970 msgr2=0x7ff42815a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff428073970 0x7ff42815a570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff420000f90 tx=0x7ff420003a20 comp rx=0 tx=0).stop 2024-09-17T02:41:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 -- 172.21.15.131:0/1014033962 shutdown_connections 2024-09-17T02:41:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff408073200 0x7ff408075710 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4280a9470 0x7ff4280a9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4280ab170 0x7ff4280ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 --2- 172.21.15.131:0/1014033962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff428073970 0x7ff42815a570 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 -- 172.21.15.131:0/1014033962 >> 172.21.15.131:0/1014033962 conn(0x7ff42806eea0 msgr2=0x7ff4280b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:36.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 -- 172.21.15.131:0/1014033962 shutdown_connections 2024-09-17T02:41:36.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.005+0000 7ff42d21f640 1 -- 172.21.15.131:0/1014033962 wait complete. 2024-09-17T02:41:36.024 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-17T02:41:36.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.185+0000 7f17d4c30640 1 -- 172.21.15.131:0/1340116184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d006e5b0 msgr2=0x7f17d015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.185+0000 7f17d4c30640 1 --2- 172.21.15.131:0/1340116184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d006e5b0 0x7f17d015bea0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009900 tx=0x7f17c401c8e0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.185+0000 7f17d4c30640 1 -- 172.21.15.131:0/1340116184 shutdown_connections 2024-09-17T02:41:36.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.185+0000 7f17d4c30640 1 --2- 172.21.15.131:0/1340116184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d006e5b0 0x7f17d015bea0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.185+0000 7f17d4c30640 1 -- 172.21.15.131:0/1340116184 >> 172.21.15.131:0/1340116184 conn(0x7f17d006eea0 msgr2=0x7f17d006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:36.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.185+0000 7f17d4c30640 1 -- 172.21.15.131:0/1340116184 shutdown_connections 2024-09-17T02:41:36.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.185+0000 7f17d4c30640 1 -- 172.21.15.131:0/1340116184 wait complete. 2024-09-17T02:41:36.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17d4c30640 1 Processor -- start 2024-09-17T02:41:36.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17d4c30640 1 -- start start 2024-09-17T02:41:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17d4c30640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17d00ced00 0x7f17d00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17d4c30640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d015c3e0 0x7f17d014d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17d4c30640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d01560a0 0x7f17d01f9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17d4c30640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0169e90 con 0x7f17d015c3e0 2024-09-17T02:41:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17d4c30640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0169d10 con 0x7f17d01560a0 2024-09-17T02:41:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17d4c30640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f17d016a010 con 0x7f17d00ced00 2024-09-17T02:41:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17cdd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d015c3e0 0x7f17d014d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17cdd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d015c3e0 0x7f17d014d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52320/0 (socket says 172.21.15.131:52320) 2024-09-17T02:41:36.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17cdd9b640 1 -- 172.21.15.131:0/1553237277 learned_addr learned my addr 172.21.15.131:0/1553237277 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17ce59c640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17d00ced00 0x7f17d00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17ced9d640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d01560a0 0x7f17d01f9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17cdd9b640 1 -- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17d00ced00 msgr2=0x7f17d00b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17cdd9b640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17d00ced00 0x7f17d00b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17cdd9b640 1 -- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d01560a0 msgr2=0x7f17d01f9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17cdd9b640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d01560a0 0x7f17d01f9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17cdd9b640 1 -- 172.21.15.131:0/1553237277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d01f9ac0 con 0x7f17d015c3e0 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17ce59c640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17d00ced00 0x7f17d00b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17ced9d640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d01560a0 0x7f17d01f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17cdd9b640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d015c3e0 0x7f17d014d000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009850 tx=0x7f17c40a55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17beffd640 1 -- 172.21.15.131:0/1553237277 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c40b2040 con 0x7f17d015c3e0 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17beffd640 1 -- 172.21.15.131:0/1553237277 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17c40041b0 con 0x7f17d015c3e0 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17d4c30640 1 -- 172.21.15.131:0/1553237277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d0156810 con 0x7f17d015c3e0 2024-09-17T02:41:36.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17beffd640 1 -- 172.21.15.131:0/1553237277 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4003600 con 0x7f17d015c3e0 2024-09-17T02:41:36.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17d4c30640 1 -- 172.21.15.131:0/1553237277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d0156dd0 con 0x7f17d015c3e0 2024-09-17T02:41:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.189+0000 7f17d4c30640 1 -- 172.21.15.131:0/1553237277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d0075b80 con 0x7f17d015c3e0 2024-09-17T02:41:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.193+0000 7f17beffd640 1 -- 172.21.15.131:0/1553237277 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f17c40bf020 con 0x7f17d015c3e0 2024-09-17T02:41:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.193+0000 7f17beffd640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17b0073200 0x7f17b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.193+0000 7f17beffd640 1 -- 172.21.15.131:0/1553237277 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f17c412eda0 con 0x7f17d015c3e0 2024-09-17T02:41:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.193+0000 7f17ce59c640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17b0073200 0x7f17b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.193+0000 7f17ce59c640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17b0073200 0x7f17b0075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f17c000a850 tx=0x7f17c0008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:36.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.197+0000 7f17beffd640 1 -- 172.21.15.131:0/1553237277 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f17c40c6020 con 0x7f17d015c3e0 2024-09-17T02:41:36.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.369+0000 7f17d4c30640 1 -- 172.21.15.131:0/1553237277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f17d0156fd0 con 0x7f17d015c3e0 2024-09-17T02:41:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.369+0000 7f17beffd640 1 -- 172.21.15.131:0/1553237277 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) ==== 104+0+11 (secure 0 0 0) 0x7f17c40164d0 con 0x7f17d015c3e0 2024-09-17T02:41:36.370 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 -- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17b0073200 msgr2=0x7f17b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17b0073200 0x7f17b0075710 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f17c000a850 tx=0x7f17c0008040 comp rx=0 tx=0).stop 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 -- 172.21.15.131:0/1553237277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d015c3e0 msgr2=0x7f17d014d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d015c3e0 0x7f17d014d000 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009850 tx=0x7f17c40a55a0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 -- 172.21.15.131:0/1553237277 shutdown_connections 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17b0073200 0x7f17b0075710 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17d01560a0 0x7f17d01f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d015c3e0 0x7f17d014d000 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 --2- 172.21.15.131:0/1553237277 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17d00ced00 0x7f17d00b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 -- 172.21.15.131:0/1553237277 >> 172.21.15.131:0/1553237277 conn(0x7f17d006eea0 msgr2=0x7f17d0074600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 -- 172.21.15.131:0/1553237277 shutdown_connections 2024-09-17T02:41:36.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.373+0000 7f17d4c30640 1 -- 172.21.15.131:0/1553237277 wait complete. 2024-09-17T02:41:36.390 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-17T02:41:36.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 -- 172.21.15.131:0/3263492925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b00706c0 msgr2=0x7f27b00cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 --2- 172.21.15.131:0/3263492925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b00706c0 0x7f27b00cb930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009840 tx=0x7f27a001c760 comp rx=0 tx=0).stop 2024-09-17T02:41:36.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 -- 172.21.15.131:0/3263492925 shutdown_connections 2024-09-17T02:41:36.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 --2- 172.21.15.131:0/3263492925 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27b00cbe70 0x7f27b00cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 --2- 172.21.15.131:0/3263492925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b00706c0 0x7f27b00cb930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 -- 172.21.15.131:0/3263492925 >> 172.21.15.131:0/3263492925 conn(0x7f27b00cdbb0 msgr2=0x7f27b00cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:36.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 -- 172.21.15.131:0/3263492925 shutdown_connections 2024-09-17T02:41:36.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 -- 172.21.15.131:0/3263492925 wait complete. 2024-09-17T02:41:36.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 Processor -- start 2024-09-17T02:41:36.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 -- start start 2024-09-17T02:41:36.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b0070df0 0x7f27b00c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b00cbe70 0x7f27b00b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27b00b4a50 0x7f27b0154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0169c60 con 0x7f27b00cbe70 2024-09-17T02:41:36.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0169ae0 con 0x7f27b0070df0 2024-09-17T02:41:36.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b7a66640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f27b0169de0 con 0x7f27b00b4a50 2024-09-17T02:41:36.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b5001640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b00cbe70 0x7f27b00b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b5001640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b00cbe70 0x7f27b00b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52330/0 (socket says 172.21.15.131:52330) 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.549+0000 7f27b5001640 1 -- 172.21.15.131:0/2656520808 learned_addr learned my addr 172.21.15.131:0/2656520808 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b5802640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b0070df0 0x7f27b00c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b6003640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27b00b4a50 0x7f27b0154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b5001640 1 -- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27b00b4a50 msgr2=0x7f27b0154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b5001640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27b00b4a50 0x7f27b0154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b5001640 1 -- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b0070df0 msgr2=0x7f27b00c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b5001640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b0070df0 0x7f27b00c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b5001640 1 -- 172.21.15.131:0/2656520808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b0156970 con 0x7f27b00cbe70 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b5001640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b00cbe70 0x7f27b00b52a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009790 tx=0x7f27a00a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27a67fc640 1 -- 172.21.15.131:0/2656520808 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a00bb070 con 0x7f27b00cbe70 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27a67fc640 1 -- 172.21.15.131:0/2656520808 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27a0004470 con 0x7f27b00cbe70 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b7a66640 1 -- 172.21.15.131:0/2656520808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b0156e70 con 0x7f27b00cbe70 2024-09-17T02:41:36.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27a67fc640 1 -- 172.21.15.131:0/2656520808 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a00b2040 con 0x7f27b00cbe70 2024-09-17T02:41:36.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b7a66640 1 -- 172.21.15.131:0/2656520808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b00b4250 con 0x7f27b00cbe70 2024-09-17T02:41:36.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b7a66640 1 -- 172.21.15.131:0/2656520808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b0155410 con 0x7f27b00cbe70 2024-09-17T02:41:36.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27a67fc640 1 -- 172.21.15.131:0/2656520808 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f27a00be070 con 0x7f27b00cbe70 2024-09-17T02:41:36.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27a67fc640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2794073200 0x7f2794075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27a67fc640 1 -- 172.21.15.131:0/2656520808 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f27a012dfb0 con 0x7f27b00cbe70 2024-09-17T02:41:36.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b5802640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2794073200 0x7f2794075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.553+0000 7f27b5802640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2794073200 0x7f2794075710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0015f0 tx=0x7f27ac009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:36.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.557+0000 7f27a67fc640 1 -- 172.21.15.131:0/2656520808 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f27b0155410 con 0x7f27b00cbe70 2024-09-17T02:41:36.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.725+0000 7f27b7a66640 1 -- 172.21.15.131:0/2656520808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f27b00cb930 con 0x7f27b00cbe70 2024-09-17T02:41:36.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.729+0000 7f27a67fc640 1 -- 172.21.15.131:0/2656520808 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) ==== 100+0+11 (secure 0 0 0) 0x7f27a0016410 con 0x7f27b00cbe70 2024-09-17T02:41:36.729 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 64 2024-09-17T02:41:36.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 -- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2794073200 msgr2=0x7f2794075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2794073200 0x7f2794075710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0015f0 tx=0x7f27ac009040 comp rx=0 tx=0).stop 2024-09-17T02:41:36.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 -- 172.21.15.131:0/2656520808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b00cbe70 msgr2=0x7f27b00b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b00cbe70 0x7f27b00b52a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009790 tx=0x7f27a00a56a0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 -- 172.21.15.131:0/2656520808 shutdown_connections 2024-09-17T02:41:36.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2794073200 0x7f2794075710 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27b00b4a50 0x7f27b0154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27b00cbe70 0x7f27b00b52a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 --2- 172.21.15.131:0/2656520808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27b0070df0 0x7f27b00c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 -- 172.21.15.131:0/2656520808 >> 172.21.15.131:0/2656520808 conn(0x7f27b00cdbb0 msgr2=0x7f27b00cce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:36.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 -- 172.21.15.131:0/2656520808 shutdown_connections 2024-09-17T02:41:36.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.733+0000 7f27b7a66640 1 -- 172.21.15.131:0/2656520808 wait complete. 2024-09-17T02:41:36.751 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-17T02:41:36.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.905+0000 7fe91a4ee640 1 -- 172.21.15.131:0/3457215265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8fc003a60 msgr2=0x7fe8fc003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.905+0000 7fe91a4ee640 1 --2- 172.21.15.131:0/3457215265 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8fc003a60 0x7fe8fc003ed0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe904006fc0 tx=0x7fe90401cef0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.905+0000 7fe918a8b640 1 -- 172.21.15.131:0/3457215265 reap_dead start 2024-09-17T02:41:36.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.905+0000 7fe91a4ee640 1 -- 172.21.15.131:0/3457215265 shutdown_connections 2024-09-17T02:41:36.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.905+0000 7fe91a4ee640 1 -- 172.21.15.131:0/3457215265 >> 172.21.15.131:0/3457215265 conn(0x7fe9140b42f0 msgr2=0x7fe9140d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:36.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.905+0000 7fe91a4ee640 1 -- 172.21.15.131:0/3457215265 shutdown_connections 2024-09-17T02:41:36.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.905+0000 7fe91a4ee640 1 -- 172.21.15.131:0/3457215265 wait complete. 2024-09-17T02:41:36.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe91a4ee640 1 Processor -- start 2024-09-17T02:41:36.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe91a4ee640 1 -- start start 2024-09-17T02:41:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe91a4ee640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9140c5d10 0x7fe9140c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe91a4ee640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe9140b7320 0x7fe9140b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe91a4ee640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe914166440 0x7fe914166850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe91a4ee640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe91415daa0 con 0x7fe9140c5d10 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe91a4ee640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe91415d920 con 0x7fe914166440 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe91a4ee640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe9140d1ca0 con 0x7fe9140b7320 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe913fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9140c5d10 0x7fe9140c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe913fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9140c5d10 0x7fe9140c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52352/0 (socket says 172.21.15.131:52352) 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe913fff640 1 -- 172.21.15.131:0/4024024330 learned_addr learned my addr 172.21.15.131:0/4024024330 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe9137fe640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe9140b7320 0x7fe9140b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe918a8b640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe914166440 0x7fe914166850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe913fff640 1 -- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe9140b7320 msgr2=0x7fe9140b7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe913fff640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe9140b7320 0x7fe9140b7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe913fff640 1 -- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe914166440 msgr2=0x7fe914166850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe913fff640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe914166440 0x7fe914166850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe913fff640 1 -- 172.21.15.131:0/4024024330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe914166d90 con 0x7fe9140c5d10 2024-09-17T02:41:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe913fff640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9140c5d10 0x7fe9140c6920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe904001ec0 tx=0x7fe904006be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:36.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe910ff9640 1 -- 172.21.15.131:0/4024024330 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9040036e0 con 0x7fe9140c5d10 2024-09-17T02:41:36.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe910ff9640 1 -- 172.21.15.131:0/4024024330 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe904003970 con 0x7fe9140c5d10 2024-09-17T02:41:36.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe91a4ee640 1 -- 172.21.15.131:0/4024024330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9141f6380 con 0x7fe9140c5d10 2024-09-17T02:41:36.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe910ff9640 1 -- 172.21.15.131:0/4024024330 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe904003d60 con 0x7fe9140c5d10 2024-09-17T02:41:36.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe91a4ee640 1 -- 172.21.15.131:0/4024024330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9141f6940 con 0x7fe9140c5d10 2024-09-17T02:41:36.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.909+0000 7fe91a4ee640 1 -- 172.21.15.131:0/4024024330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91415e540 con 0x7fe9140c5d10 2024-09-17T02:41:36.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.913+0000 7fe910ff9640 1 -- 172.21.15.131:0/4024024330 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe9040ba460 con 0x7fe9140c5d10 2024-09-17T02:41:36.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.913+0000 7fe910ff9640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe900073200 0x7fe900075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:36.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.913+0000 7fe910ff9640 1 -- 172.21.15.131:0/4024024330 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7fe90412f230 con 0x7fe9140c5d10 2024-09-17T02:41:36.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.913+0000 7fe9137fe640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe900073200 0x7fe900075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:36.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.913+0000 7fe9137fe640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe900073200 0x7fe900075710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe9140b7c70 tx=0x7fe90800e420 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:36.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:36.917+0000 7fe910ff9640 1 -- 172.21.15.131:0/4024024330 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe914166d90 con 0x7fe9140c5d10 2024-09-17T02:41:37.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe91a4ee640 1 -- 172.21.15.131:0/4024024330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe91415c7d0 con 0x7fe9140c5d10 2024-09-17T02:41:37.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe910ff9640 1 -- 172.21.15.131:0/4024024330 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) ==== 103+0+11 (secure 0 0 0) 0x7fe904016cd0 con 0x7fe9140c5d10 2024-09-17T02:41:37.087 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 32 2024-09-17T02:41:37.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe91a4ee640 1 -- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe900073200 msgr2=0x7fe900075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:37.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe91a4ee640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe900073200 0x7fe900075710 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe9140b7c70 tx=0x7fe90800e420 comp rx=0 tx=0).stop 2024-09-17T02:41:37.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe91a4ee640 1 -- 172.21.15.131:0/4024024330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9140c5d10 msgr2=0x7fe9140c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:37.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe91a4ee640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9140c5d10 0x7fe9140c6920 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe904001ec0 tx=0x7fe904006be0 comp rx=0 tx=0).stop 2024-09-17T02:41:37.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe91a4ee640 1 -- 172.21.15.131:0/4024024330 shutdown_connections 2024-09-17T02:41:37.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe91a4ee640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe900073200 0x7fe900075710 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:37.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe91a4ee640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe914166440 0x7fe914166850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:37.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe91a4ee640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe9140b7320 0x7fe9140b7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:37.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe91a4ee640 1 --2- 172.21.15.131:0/4024024330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9140c5d10 0x7fe9140c6920 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:37.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.085+0000 7fe91a4ee640 1 -- 172.21.15.131:0/4024024330 >> 172.21.15.131:0/4024024330 conn(0x7fe9140b42f0 msgr2=0x7fe914070900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:37.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.089+0000 7fe91a4ee640 1 -- 172.21.15.131:0/4024024330 shutdown_connections 2024-09-17T02:41:37.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.089+0000 7fe91a4ee640 1 -- 172.21.15.131:0/4024024330 wait complete. 2024-09-17T02:41:37.107 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-09-17T02:41:37.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.257+0000 7fbeda791640 1 -- 172.21.15.131:0/2934963468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc004810 msgr2=0x7fbebc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:37.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.257+0000 7fbeda791640 1 --2- 172.21.15.131:0/2934963468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc004810 0x7fbebc004c20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbec80016f0 tx=0x7fbec801c790 comp rx=0 tx=0).stop 2024-09-17T02:41:37.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.257+0000 7fbeda791640 1 -- 172.21.15.131:0/2934963468 shutdown_connections 2024-09-17T02:41:37.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.257+0000 7fbeda791640 1 --2- 172.21.15.131:0/2934963468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbebc004810 0x7fbebc004c20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:37.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.257+0000 7fbeda791640 1 -- 172.21.15.131:0/2934963468 >> 172.21.15.131:0/2934963468 conn(0x7fbed406eea0 msgr2=0x7fbed406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:37.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.257+0000 7fbeda791640 1 -- 172.21.15.131:0/2934963468 shutdown_connections 2024-09-17T02:41:37.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.257+0000 7fbeda791640 1 -- 172.21.15.131:0/2934963468 wait complete. 2024-09-17T02:41:37.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbeda791640 1 Processor -- start 2024-09-17T02:41:37.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbeda791640 1 -- start start 2024-09-17T02:41:37.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbeda791640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbebc004810 0x7fbed41f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:37.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbeda791640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbed4158a50 0x7fbed40b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:37.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbeda791640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbed41543f0 0x7fbed40b5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:37.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbeda791640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbed415c020 con 0x7fbed4158a50 2024-09-17T02:41:37.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbeda791640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbed415bea0 con 0x7fbebc004810 2024-09-17T02:41:37.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbeda791640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fbed415c1a0 con 0x7fbed41543f0 2024-09-17T02:41:37.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbed4158a50 0x7fbed40b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbed4158a50 0x7fbed40b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52366/0 (socket says 172.21.15.131:52366) 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed37fe640 1 -- 172.21.15.131:0/1292733897 learned_addr learned my addr 172.21.15.131:0/1292733897 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed3fff640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbebc004810 0x7fbed41f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed37fe640 1 -- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbed41543f0 msgr2=0x7fbed40b5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed8d2e640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbed41543f0 0x7fbed40b5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed37fe640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbed41543f0 0x7fbed40b5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed37fe640 1 -- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbebc004810 msgr2=0x7fbed41f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed37fe640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbebc004810 0x7fbed41f7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed37fe640 1 -- 172.21.15.131:0/1292733897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed41f3830 con 0x7fbed4158a50 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed3fff640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbebc004810 0x7fbed41f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed37fe640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbed4158a50 0x7fbed40b50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbec8000fe0 tx=0x7fbec8006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed0ff9640 1 -- 172.21.15.131:0/1292733897 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8003680 con 0x7fbed4158a50 2024-09-17T02:41:37.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed0ff9640 1 -- 172.21.15.131:0/1292733897 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbec8003910 con 0x7fbed4158a50 2024-09-17T02:41:37.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbeda791640 1 -- 172.21.15.131:0/1292733897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed41f3af0 con 0x7fbed4158a50 2024-09-17T02:41:37.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbed0ff9640 1 -- 172.21.15.131:0/1292733897 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8003d20 con 0x7fbed4158a50 2024-09-17T02:41:37.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.261+0000 7fbeda791640 1 -- 172.21.15.131:0/1292733897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed41f40b0 con 0x7fbed4158a50 2024-09-17T02:41:37.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.265+0000 7fbeda791640 1 -- 172.21.15.131:0/1292733897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed4075b80 con 0x7fbed4158a50 2024-09-17T02:41:37.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.265+0000 7fbed0ff9640 1 -- 172.21.15.131:0/1292733897 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbec80a55f0 con 0x7fbed4158a50 2024-09-17T02:41:37.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.265+0000 7fbed0ff9640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbec0073230 0x7fbec0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:37.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.265+0000 7fbed0ff9640 1 -- 172.21.15.131:0/1292733897 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7fbec812ed50 con 0x7fbed4158a50 2024-09-17T02:41:37.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.265+0000 7fbed3fff640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbec0073230 0x7fbec0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:37.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.265+0000 7fbed3fff640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbec0073230 0x7fbec0075740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbec40046e0 tx=0x7fbec40011a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:37.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.269+0000 7fbed0ff9640 1 -- 172.21.15.131:0/1292733897 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbec80c6080 con 0x7fbed4158a50 2024-09-17T02:41:37.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:37.485+0000 7fbeda791640 1 -- 172.21.15.131:0/1292733897 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fbed406df80 con 0x7fbed4158a50 2024-09-17T02:41:38.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.173+0000 7fbed0ff9640 1 -- 172.21.15.131:0/1292733897 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v62) ==== 145+0+0 (secure 0 0 0) 0x7fbec80164f0 con 0x7fbed4158a50 2024-09-17T02:41:38.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 -- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbec0073230 msgr2=0x7fbec0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbec0073230 0x7fbec0075740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbec40046e0 tx=0x7fbec40011a0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 -- 172.21.15.131:0/1292733897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbed4158a50 msgr2=0x7fbed40b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbed4158a50 0x7fbed40b50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbec8000fe0 tx=0x7fbec8006950 comp rx=0 tx=0).stop 2024-09-17T02:41:38.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 -- 172.21.15.131:0/1292733897 shutdown_connections 2024-09-17T02:41:38.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbec0073230 0x7fbec0075740 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbed41543f0 0x7fbed40b5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbed4158a50 0x7fbed40b50c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 --2- 172.21.15.131:0/1292733897 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbebc004810 0x7fbed41f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 -- 172.21.15.131:0/1292733897 >> 172.21.15.131:0/1292733897 conn(0x7fbed406eea0 msgr2=0x7fbed415d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:38.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 -- 172.21.15.131:0/1292733897 shutdown_connections 2024-09-17T02:41:38.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.177+0000 7fbeda791640 1 -- 172.21.15.131:0/1292733897 wait complete. 2024-09-17T02:41:38.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T02:41:38.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.361+0000 7fa11e309640 1 -- 172.21.15.131:0/2688965230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa11806dea0 msgr2=0x7fa11806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.361+0000 7fa11e309640 1 --2- 172.21.15.131:0/2688965230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa11806dea0 0x7fa11806e2b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa10800a950 tx=0x7fa10801d9d0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.361+0000 7fa11e309640 1 -- 172.21.15.131:0/2688965230 shutdown_connections 2024-09-17T02:41:38.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.361+0000 7fa11e309640 1 --2- 172.21.15.131:0/2688965230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa11806dea0 0x7fa11806e2b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.361+0000 7fa11e309640 1 -- 172.21.15.131:0/2688965230 >> 172.21.15.131:0/2688965230 conn(0x7fa11806eea0 msgr2=0x7fa11806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:38.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.361+0000 7fa11e309640 1 -- 172.21.15.131:0/2688965230 shutdown_connections 2024-09-17T02:41:38.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.361+0000 7fa11e309640 1 -- 172.21.15.131:0/2688965230 wait complete. 2024-09-17T02:41:38.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11e309640 1 Processor -- start 2024-09-17T02:41:38.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11e309640 1 -- start start 2024-09-17T02:41:38.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11e309640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa11806dea0 0x7fa1180c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:38.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11e309640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa1180cab60 0x7fa1181f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:38.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11e309640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa118155240 0x7fa1181f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11e309640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa118168b80 con 0x7fa118155240 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11e309640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa1180cbec0 con 0x7fa11806dea0 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11e309640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa118168d00 con 0x7fa1180cab60 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11c8a6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa118155240 0x7fa1181f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11c8a6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa118155240 0x7fa1181f6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52392/0 (socket says 172.21.15.131:52392) 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11c8a6640 1 -- 172.21.15.131:0/3021331506 learned_addr learned my addr 172.21.15.131:0/3021331506 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa1177fe640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa1180cab60 0x7fa1181f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa117fff640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa11806dea0 0x7fa1180c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11c8a6640 1 -- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa1180cab60 msgr2=0x7fa1181f6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11c8a6640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa1180cab60 0x7fa1181f6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11c8a6640 1 -- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa11806dea0 msgr2=0x7fa1180c8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11c8a6640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa11806dea0 0x7fa1180c8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11c8a6640 1 -- 172.21.15.131:0/3021331506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1180c7160 con 0x7fa118155240 2024-09-17T02:41:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa1177fe640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa1180cab60 0x7fa1181f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa117fff640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa11806dea0 0x7fa1180c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11c8a6640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa118155240 0x7fa1181f6990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b440 tx=0x7fa11000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa114ff9640 1 -- 172.21.15.131:0/3021331506 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110015020 con 0x7fa118155240 2024-09-17T02:41:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa114ff9640 1 -- 172.21.15.131:0/3021331506 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa110011940 con 0x7fa118155240 2024-09-17T02:41:38.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa114ff9640 1 -- 172.21.15.131:0/3021331506 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa110011d90 con 0x7fa118155240 2024-09-17T02:41:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11e309640 1 -- 172.21.15.131:0/3021331506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1180c92c0 con 0x7fa118155240 2024-09-17T02:41:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.365+0000 7fa11e309640 1 -- 172.21.15.131:0/3021331506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1181f7230 con 0x7fa118155240 2024-09-17T02:41:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.369+0000 7fa11e309640 1 -- 172.21.15.131:0/3021331506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa11806e2b0 con 0x7fa118155240 2024-09-17T02:41:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.369+0000 7fa114ff9640 1 -- 172.21.15.131:0/3021331506 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa110018020 con 0x7fa118155240 2024-09-17T02:41:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.369+0000 7fa114ff9640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa0fc073200 0x7fa0fc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:38.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.369+0000 7fa114ff9640 1 -- 172.21.15.131:0/3021331506 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7fa1100943d0 con 0x7fa118155240 2024-09-17T02:41:38.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.369+0000 7fa117fff640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa0fc073200 0x7fa0fc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:38.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.369+0000 7fa117fff640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa0fc073200 0x7fa0fc075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa100007fd0 tx=0x7fa100009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:38.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.373+0000 7fa114ff9640 1 -- 172.21.15.131:0/3021331506 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa11005f040 con 0x7fa118155240 2024-09-17T02:41:38.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.533+0000 7fa11e309640 1 -- 172.21.15.131:0/3021331506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa11806e9d0 con 0x7fa118155240 2024-09-17T02:41:38.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.533+0000 7fa114ff9640 1 -- 172.21.15.131:0/3021331506 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7fa11005e940 con 0x7fa118155240 2024-09-17T02:41:38.535 INFO:teuthology.orchestra.run.smithi131.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T02:41:38.535 INFO:teuthology.orchestra.run.smithi131.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T02:41:38.535 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced client_mount_timeout 30 2024-09-17T02:41:38.535 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_client 20/20 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_mgr 20/20 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_ms 1/1 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3855.598857 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3598.912856 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3977.024117 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3823.557728 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4643.710284 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4452.598518 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6659.946233 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4539.778076 2024-09-17T02:41:38.536 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds 20/20 2024-09-17T02:41:38.537 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T02:41:38.537 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_ms 2/2 2024-09-17T02:41:38.537 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_frag true 2024-09-17T02:41:38.537 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_scatterstat true 2024-09-17T02:41:38.537 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T02:41:38.537 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_verify_scatter true 2024-09-17T02:41:38.537 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T02:41:38.537 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T02:41:38.537 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T02:41:38.538 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced client_mount_timeout 600 2024-09-17T02:41:38.538 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_client 20/20 2024-09-17T02:41:38.538 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_ms 1/1 2024-09-17T02:41:38.538 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T02:41:38.538 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T02:41:38.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 -- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa0fc073200 msgr2=0x7fa0fc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa0fc073200 0x7fa0fc075710 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa100007fd0 tx=0x7fa100009040 comp rx=0 tx=0).stop 2024-09-17T02:41:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 -- 172.21.15.131:0/3021331506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa118155240 msgr2=0x7fa1181f6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa118155240 0x7fa1181f6990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa11000b440 tx=0x7fa11000b900 comp rx=0 tx=0).stop 2024-09-17T02:41:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 -- 172.21.15.131:0/3021331506 shutdown_connections 2024-09-17T02:41:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa0fc073200 0x7fa0fc075710 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa118155240 0x7fa1181f6990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa1180cab60 0x7fa1181f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 --2- 172.21.15.131:0/3021331506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa11806dea0 0x7fa1180c8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 -- 172.21.15.131:0/3021331506 >> 172.21.15.131:0/3021331506 conn(0x7fa11806eea0 msgr2=0x7fa1180733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 -- 172.21.15.131:0/3021331506 shutdown_connections 2024-09-17T02:41:38.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.537+0000 7fa11e309640 1 -- 172.21.15.131:0/3021331506 wait complete. 2024-09-17T02:41:38.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T02:41:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.713+0000 7f668e7da640 1 -- 172.21.15.131:0/1501766063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66880c4c90 msgr2=0x7f66880c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.713+0000 7f668e7da640 1 --2- 172.21.15.131:0/1501766063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66880c4c90 0x7f66880c50a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6680009bc0 tx=0x7f668001cb10 comp rx=0 tx=0).stop 2024-09-17T02:41:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.713+0000 7f668e7da640 1 -- 172.21.15.131:0/1501766063 shutdown_connections 2024-09-17T02:41:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.713+0000 7f668e7da640 1 --2- 172.21.15.131:0/1501766063 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66880c4c90 0x7f66880c50a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.713+0000 7f668e7da640 1 --2- 172.21.15.131:0/1501766063 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f66880cec00 0x7f66880cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.713+0000 7f668e7da640 1 -- 172.21.15.131:0/1501766063 >> 172.21.15.131:0/1501766063 conn(0x7f668806eea0 msgr2=0x7f668806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.713+0000 7f668e7da640 1 -- 172.21.15.131:0/1501766063 shutdown_connections 2024-09-17T02:41:38.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.713+0000 7f668e7da640 1 -- 172.21.15.131:0/1501766063 wait complete. 2024-09-17T02:41:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668e7da640 1 Processor -- start 2024-09-17T02:41:38.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668e7da640 1 -- start start 2024-09-17T02:41:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668e7da640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f668806e2c0 0x7f66881506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668e7da640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66880c4c90 0x7f66880b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668e7da640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f66880cec00 0x7f66880b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668e7da640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6688169e70 con 0x7f66880c4c90 2024-09-17T02:41:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668e7da640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6688169cf0 con 0x7f668806e2c0 2024-09-17T02:41:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668e7da640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6688169ff0 con 0x7f66880cec00 2024-09-17T02:41:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f66877fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66880c4c90 0x7f66880b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f66877fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66880c4c90 0x7f66880b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52410/0 (socket says 172.21.15.131:52410) 2024-09-17T02:41:38.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f66877fe640 1 -- 172.21.15.131:0/727198841 learned_addr learned my addr 172.21.15.131:0/727198841 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668cd77640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f66880cec00 0x7f66880b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f6687fff640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f668806e2c0 0x7f66881506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f66877fe640 1 -- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f66880cec00 msgr2=0x7f66880b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f66877fe640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f66880cec00 0x7f66880b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f66877fe640 1 -- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f668806e2c0 msgr2=0x7f66881506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f66877fe640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f668806e2c0 0x7f66881506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f66877fe640 1 -- 172.21.15.131:0/727198841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66880cc660 con 0x7f66880c4c90 2024-09-17T02:41:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668cd77640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f66880cec00 0x7f66880b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f66877fe640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66880c4c90 0x7f66880b4a20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f667c00d650 tx=0x7f667c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:38.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f6684ff9640 1 -- 172.21.15.131:0/727198841 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c017070 con 0x7f66880c4c90 2024-09-17T02:41:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f6684ff9640 1 -- 172.21.15.131:0/727198841 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f667c005310 con 0x7f66880c4c90 2024-09-17T02:41:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f6684ff9640 1 -- 172.21.15.131:0/727198841 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c009820 con 0x7f66880c4c90 2024-09-17T02:41:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668e7da640 1 -- 172.21.15.131:0/727198841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66880cc380 con 0x7f66880c4c90 2024-09-17T02:41:38.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668e7da640 1 -- 172.21.15.131:0/727198841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66881fc1b0 con 0x7f66880c4c90 2024-09-17T02:41:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.717+0000 7f668e7da640 1 -- 172.21.15.131:0/727198841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66880cf010 con 0x7f66880c4c90 2024-09-17T02:41:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.721+0000 7f6684ff9640 1 -- 172.21.15.131:0/727198841 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f667c0099c0 con 0x7f66880c4c90 2024-09-17T02:41:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.721+0000 7f6684ff9640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6674073200 0x7f6674075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.721+0000 7f6684ff9640 1 -- 172.21.15.131:0/727198841 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f667c0945d0 con 0x7f66880c4c90 2024-09-17T02:41:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.721+0000 7f6687fff640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6674073200 0x7f6674075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:38.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.721+0000 7f6687fff640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6674073200 0x7f6674075710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6678001640 tx=0x7f6678009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:38.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.725+0000 7f6684ff9640 1 -- 172.21.15.131:0/727198841 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f667c05f2d0 con 0x7f66880c4c90 2024-09-17T02:41:38.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.881+0000 7f668e7da640 1 -- 172.21.15.131:0/727198841 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6688073310 con 0x7f66880c4c90 2024-09-17T02:41:38.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f6684ff9640 1 -- 172.21.15.131:0/727198841 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f667c05ebd0 con 0x7f66880c4c90 2024-09-17T02:41:38.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 -- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6674073200 msgr2=0x7f6674075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6674073200 0x7f6674075710 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6678001640 tx=0x7f6678009040 comp rx=0 tx=0).stop 2024-09-17T02:41:38.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 -- 172.21.15.131:0/727198841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66880c4c90 msgr2=0x7f66880b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:38.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66880c4c90 0x7f66880b4a20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f667c00d650 tx=0x7f667c00db10 comp rx=0 tx=0).stop 2024-09-17T02:41:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 -- 172.21.15.131:0/727198841 shutdown_connections 2024-09-17T02:41:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6674073200 0x7f6674075710 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f66880cec00 0x7f66880b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f66880c4c90 0x7f66880b4a20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 --2- 172.21.15.131:0/727198841 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f668806e2c0 0x7f66881506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:38.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 -- 172.21.15.131:0/727198841 >> 172.21.15.131:0/727198841 conn(0x7f668806eea0 msgr2=0x7f66880cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:38.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 -- 172.21.15.131:0/727198841 shutdown_connections 2024-09-17T02:41:38.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:38.889+0000 7f668e7da640 1 -- 172.21.15.131:0/727198841 wait complete. 2024-09-17T02:41:38.908 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:41:38.909 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:41:38.969 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:41:39.044 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:41:39.045 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:41:39.045 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:39.050 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:41:39.050 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:39.097 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:41:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.305+0000 7f123fe9d640 1 -- 172.21.15.131:0/810719114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f123806e110 msgr2=0x7f123815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.305+0000 7f123fe9d640 1 --2- 172.21.15.131:0/810719114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f123806e110 0x7f123815e530 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1238072f20 tx=0x7f122c01c650 comp rx=0 tx=0).stop 2024-09-17T02:41:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.305+0000 7f123fe9d640 1 -- 172.21.15.131:0/810719114 shutdown_connections 2024-09-17T02:41:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.305+0000 7f123fe9d640 1 --2- 172.21.15.131:0/810719114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f123806e110 0x7f123815e530 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.305+0000 7f123fe9d640 1 --2- 172.21.15.131:0/810719114 >> v2:172.21.15.189:3300/0 conn(0x7f12380b4c10 0x7f1238070e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.305+0000 7f123fe9d640 1 -- 172.21.15.131:0/810719114 >> 172.21.15.131:0/810719114 conn(0x7f12380d0740 msgr2=0x7f12380d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.305+0000 7f123fe9d640 1 -- 172.21.15.131:0/810719114 shutdown_connections 2024-09-17T02:41:39.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.305+0000 7f123fe9d640 1 -- 172.21.15.131:0/810719114 wait complete. 2024-09-17T02:41:39.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.305+0000 7f123fe9d640 1 Processor -- start 2024-09-17T02:41:39.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.305+0000 7f123fe9d640 1 -- start start 2024-09-17T02:41:39.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123fe9d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f123806e110 0x7f12381563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:39.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123fe9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f123806e520 0x7f123815ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:39.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123fe9d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12380b4c10 0x7f12380b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:39.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123fe9d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1238168850 con 0x7f123806e520 2024-09-17T02:41:39.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123fe9d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f12381686d0 con 0x7f12380b4c10 2024-09-17T02:41:39.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123fe9d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f12381689d0 con 0x7f123806e110 2024-09-17T02:41:39.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123d438640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f123806e520 0x7f123815ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:39.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123d438640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f123806e520 0x7f123815ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52440/0 (socket says 172.21.15.131:52440) 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123d438640 1 -- 172.21.15.131:0/2188683759 learned_addr learned my addr 172.21.15.131:0/2188683759 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123e43a640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12380b4c10 0x7f12380b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123dc39640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f123806e110 0x7f12381563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123d438640 1 -- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f123806e110 msgr2=0x7f12381563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123d438640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f123806e110 0x7f12381563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123d438640 1 -- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12380b4c10 msgr2=0x7f12380b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123d438640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12380b4c10 0x7f12380b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123d438640 1 -- 172.21.15.131:0/2188683759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12381551d0 con 0x7f123806e520 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123e43a640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12380b4c10 0x7f12380b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123d438640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f123806e520 0x7f123815ba20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f122c0ac040 tx=0x7f122c0aa590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f12367fc640 1 -- 172.21.15.131:0/2188683759 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c0aab10 con 0x7f123806e520 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f12367fc640 1 -- 172.21.15.131:0/2188683759 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f122c0aada0 con 0x7f123806e520 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123fe9d640 1 -- 172.21.15.131:0/2188683759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1238158320 con 0x7f123806e520 2024-09-17T02:41:39.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f12367fc640 1 -- 172.21.15.131:0/2188683759 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c003860 con 0x7f123806e520 2024-09-17T02:41:39.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123fe9d640 1 -- 172.21.15.131:0/2188683759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1238156920 con 0x7f123806e520 2024-09-17T02:41:39.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123fe9d640 1 -- 172.21.15.131:0/2188683759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123815e530 con 0x7f123806e520 2024-09-17T02:41:39.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f12367fc640 1 -- 172.21.15.131:0/2188683759 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f122c0a8050 con 0x7f123806e520 2024-09-17T02:41:39.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f12367fc640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1224073230 0x7f1224075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:39.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f12367fc640 1 -- 172.21.15.131:0/2188683759 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f122c12f610 con 0x7f123806e520 2024-09-17T02:41:39.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.309+0000 7f123dc39640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1224073230 0x7f1224075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:39.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.313+0000 7f123dc39640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1224073230 0x7f1224075740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1230005c50 tx=0x7f1230005f50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:39.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.313+0000 7f12367fc640 1 -- 172.21.15.131:0/2188683759 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1238158320 con 0x7f123806e520 2024-09-17T02:41:39.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 -- 172.21.15.131:0/2188683759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f12381580b0 con 0x7f123806e520 2024-09-17T02:41:39.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f12367fc640 1 -- 172.21.15.131:0/2188683759 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f122c0a6040 con 0x7f123806e520 2024-09-17T02:41:39.493 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:41:39.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 -- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1224073230 msgr2=0x7f1224075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:39.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1224073230 0x7f1224075740 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1230005c50 tx=0x7f1230005f50 comp rx=0 tx=0).stop 2024-09-17T02:41:39.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 -- 172.21.15.131:0/2188683759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f123806e520 msgr2=0x7f123815ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:39.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f123806e520 0x7f123815ba20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f122c0ac040 tx=0x7f122c0aa590 comp rx=0 tx=0).stop 2024-09-17T02:41:39.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 -- 172.21.15.131:0/2188683759 shutdown_connections 2024-09-17T02:41:39.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1224073230 0x7f1224075740 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f12380b4c10 0x7f12380b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f123806e520 0x7f123815ba20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 --2- 172.21.15.131:0/2188683759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f123806e110 0x7f12381563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 -- 172.21.15.131:0/2188683759 >> 172.21.15.131:0/2188683759 conn(0x7f12380d0740 msgr2=0x7f12380b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:39.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 -- 172.21.15.131:0/2188683759 shutdown_connections 2024-09-17T02:41:39.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.493+0000 7f123fe9d640 1 -- 172.21.15.131:0/2188683759 wait complete. 2024-09-17T02:41:39.514 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:41:39.514 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:39.520 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:41:39.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.725+0000 7fa59824b640 1 -- 172.21.15.131:0/218500932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5900ced00 msgr2=0x7fa59006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:39.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.725+0000 7fa59824b640 1 --2- 172.21.15.131:0/218500932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5900ced00 0x7fa59006dea0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa5900b61b0 tx=0x7fa58c01c800 comp rx=0 tx=0).stop 2024-09-17T02:41:39.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.725+0000 7fa59824b640 1 -- 172.21.15.131:0/218500932 shutdown_connections 2024-09-17T02:41:39.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.725+0000 7fa59824b640 1 --2- 172.21.15.131:0/218500932 >> v2:172.21.15.189:3300/0 conn(0x7fa59006e3e0 0x7fa59015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.725+0000 7fa59824b640 1 --2- 172.21.15.131:0/218500932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5900ced00 0x7fa59006dea0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.725+0000 7fa59824b640 1 -- 172.21.15.131:0/218500932 >> 172.21.15.131:0/218500932 conn(0x7fa59006eea0 msgr2=0x7fa59006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:39.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.725+0000 7fa59824b640 1 -- 172.21.15.131:0/218500932 shutdown_connections 2024-09-17T02:41:39.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.725+0000 7fa59824b640 1 -- 172.21.15.131:0/218500932 wait complete. 2024-09-17T02:41:39.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.725+0000 7fa59824b640 1 Processor -- start 2024-09-17T02:41:39.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.725+0000 7fa59824b640 1 -- start start 2024-09-17T02:41:39.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa59824b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59006e3e0 0x7fa590159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:39.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa59824b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5900ced00 0x7fa590163da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:39.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa59824b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa59015c3e0 0x7fa5901642e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:39.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa59824b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa590168b60 con 0x7fa5900ced00 2024-09-17T02:41:39.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa59824b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5901689e0 con 0x7fa59006e3e0 2024-09-17T02:41:39.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa59824b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa590168ce0 con 0x7fa59015c3e0 2024-09-17T02:41:39.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5957e6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5900ced00 0x7fa590163da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:39.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5957e6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5900ced00 0x7fa590163da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52468/0 (socket says 172.21.15.131:52468) 2024-09-17T02:41:39.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5957e6640 1 -- 172.21.15.131:0/646190413 learned_addr learned my addr 172.21.15.131:0/646190413 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:39.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa595fe7640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59006e3e0 0x7fa590159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:39.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5957e6640 1 -- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa59015c3e0 msgr2=0x7fa5901642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5967e8640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa59015c3e0 0x7fa5901642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5957e6640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa59015c3e0 0x7fa5901642e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5957e6640 1 -- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59006e3e0 msgr2=0x7fa590159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5957e6640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59006e3e0 0x7fa590159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5957e6640 1 -- 172.21.15.131:0/646190413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59015b220 con 0x7fa5900ced00 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa595fe7640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59006e3e0 0x7fa590159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5957e6640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5900ced00 0x7fa590163da0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa58000d950 tx=0x7fa58000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5867fc640 1 -- 172.21.15.131:0/646190413 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa580017070 con 0x7fa5900ced00 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5867fc640 1 -- 172.21.15.131:0/646190413 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa580009d20 con 0x7fa5900ced00 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa59824b640 1 -- 172.21.15.131:0/646190413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa590165a70 con 0x7fa5900ced00 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5867fc640 1 -- 172.21.15.131:0/646190413 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5800106c0 con 0x7fa5900ced00 2024-09-17T02:41:39.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa59824b640 1 -- 172.21.15.131:0/646190413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa590161500 con 0x7fa5900ced00 2024-09-17T02:41:39.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa59824b640 1 -- 172.21.15.131:0/646190413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59015af90 con 0x7fa5900ced00 2024-09-17T02:41:39.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5867fc640 1 -- 172.21.15.131:0/646190413 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa58001a070 con 0x7fa5900ced00 2024-09-17T02:41:39.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5867fc640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa574073200 0x7fa574075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:39.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa5867fc640 1 -- 172.21.15.131:0/646190413 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7fa580094690 con 0x7fa5900ced00 2024-09-17T02:41:39.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.729+0000 7fa595fe7640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa574073200 0x7fa574075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:39.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.733+0000 7fa595fe7640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa574073200 0x7fa574075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0af000 tx=0x7fa58c01b040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:39.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.737+0000 7fa5867fc640 1 -- 172.21.15.131:0/646190413 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa58005f420 con 0x7fa5900ced00 2024-09-17T02:41:39.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.917+0000 7fa59824b640 1 -- 172.21.15.131:0/646190413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa590165f90 con 0x7fa5900ced00 2024-09-17T02:41:39.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.917+0000 7fa5867fc640 1 -- 172.21.15.131:0/646190413 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fa590165f90 con 0x7fa5900ced00 2024-09-17T02:41:39.920 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:41:39.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 -- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa574073200 msgr2=0x7fa574075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:39.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa574073200 0x7fa574075710 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0af000 tx=0x7fa58c01b040 comp rx=0 tx=0).stop 2024-09-17T02:41:39.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 -- 172.21.15.131:0/646190413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5900ced00 msgr2=0x7fa590163da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:39.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5900ced00 0x7fa590163da0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa58000d950 tx=0x7fa58000de10 comp rx=0 tx=0).stop 2024-09-17T02:41:39.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 -- 172.21.15.131:0/646190413 shutdown_connections 2024-09-17T02:41:39.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa574073200 0x7fa574075710 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa59015c3e0 0x7fa5901642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5900ced00 0x7fa590163da0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 --2- 172.21.15.131:0/646190413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa59006e3e0 0x7fa590159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:39.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 -- 172.21.15.131:0/646190413 >> 172.21.15.131:0/646190413 conn(0x7fa59006eea0 msgr2=0x7fa5900743e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:39.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 -- 172.21.15.131:0/646190413 shutdown_connections 2024-09-17T02:41:39.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:39.921+0000 7fa59824b640 1 -- 172.21.15.131:0/646190413 wait complete. 2024-09-17T02:41:39.942 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:41:39.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:39.943 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:41:39.954 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:41:39.955 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:41:39.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:39.955 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:41:40.001 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:41:40.001 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:40.008 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:41:40.008 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:40.054 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:41:40.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.209+0000 7f05d65ee640 1 -- 172.21.15.131:0/3196650387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d00cec00 msgr2=0x7f05d00cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:40.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.209+0000 7f05d65ee640 1 --2- 172.21.15.131:0/3196650387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d00cec00 0x7f05d00cf010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f05d006a1c0 tx=0x7f05c801caf0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.209+0000 7f05d65ee640 1 -- 172.21.15.131:0/3196650387 shutdown_connections 2024-09-17T02:41:40.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.209+0000 7f05d65ee640 1 --2- 172.21.15.131:0/3196650387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d00cec00 0x7f05d00cf010 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.209+0000 7f05d65ee640 1 --2- 172.21.15.131:0/3196650387 >> v2:172.21.15.189:3300/0 conn(0x7f05d006e2c0 0x7f05d00c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.209+0000 7f05d65ee640 1 -- 172.21.15.131:0/3196650387 >> 172.21.15.131:0/3196650387 conn(0x7f05d006eea0 msgr2=0x7f05d006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:40.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.209+0000 7f05d65ee640 1 -- 172.21.15.131:0/3196650387 shutdown_connections 2024-09-17T02:41:40.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.209+0000 7f05d65ee640 1 -- 172.21.15.131:0/3196650387 wait complete. 2024-09-17T02:41:40.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d65ee640 1 Processor -- start 2024-09-17T02:41:40.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d65ee640 1 -- start start 2024-09-17T02:41:40.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d65ee640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05d006e2c0 0x7f05d014efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:40.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d65ee640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05d00c4ac0 0x7f05d00c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:40.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d65ee640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d00cec00 0x7f05d00c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:40.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d65ee640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0169c60 con 0x7f05d00cec00 2024-09-17T02:41:40.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d65ee640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f05d0169ae0 con 0x7f05d006e2c0 2024-09-17T02:41:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d65ee640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f05d0169de0 con 0x7f05d00c4ac0 2024-09-17T02:41:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d4b8b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d00cec00 0x7f05d00c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d4b8b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d00cec00 0x7f05d00c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52488/0 (socket says 172.21.15.131:52488) 2024-09-17T02:41:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d4b8b640 1 -- 172.21.15.131:0/76317589 learned_addr learned my addr 172.21.15.131:0/76317589 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05c7fff640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05d00c4ac0 0x7f05d00c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05cffff640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05d006e2c0 0x7f05d014efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d4b8b640 1 -- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05d00c4ac0 msgr2=0x7f05d00c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d4b8b640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05d00c4ac0 0x7f05d00c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d4b8b640 1 -- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05d006e2c0 msgr2=0x7f05d014efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d4b8b640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05d006e2c0 0x7f05d014efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d4b8b640 1 -- 172.21.15.131:0/76317589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d00b3820 con 0x7f05d00cec00 2024-09-17T02:41:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05c7fff640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05d00c4ac0 0x7f05d00c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:41:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05cffff640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05d006e2c0 0x7f05d014efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.213+0000 7f05d4b8b640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d00cec00 0x7f05d00c93b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009b10 tx=0x7f05c8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.217+0000 7f05cd7fa640 1 -- 172.21.15.131:0/76317589 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c80bc070 con 0x7f05d00cec00 2024-09-17T02:41:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.217+0000 7f05cd7fa640 1 -- 172.21.15.131:0/76317589 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05c80042f0 con 0x7f05d00cec00 2024-09-17T02:41:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.217+0000 7f05d65ee640 1 -- 172.21.15.131:0/76317589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d01fa4b0 con 0x7f05d00cec00 2024-09-17T02:41:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.217+0000 7f05cd7fa640 1 -- 172.21.15.131:0/76317589 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c80b2040 con 0x7f05d00cec00 2024-09-17T02:41:40.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.217+0000 7f05d65ee640 1 -- 172.21.15.131:0/76317589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d01faa70 con 0x7f05d00cec00 2024-09-17T02:41:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.217+0000 7f05d65ee640 1 -- 172.21.15.131:0/76317589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d01fa220 con 0x7f05d00cec00 2024-09-17T02:41:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.217+0000 7f05cd7fa640 1 -- 172.21.15.131:0/76317589 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f05c8004700 con 0x7f05d00cec00 2024-09-17T02:41:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.217+0000 7f05cd7fa640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f05b0073200 0x7f05b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.217+0000 7f05cd7fa640 1 -- 172.21.15.131:0/76317589 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f05c812e5e0 con 0x7f05d00cec00 2024-09-17T02:41:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.217+0000 7f05cffff640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f05b0073200 0x7f05b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:40.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.217+0000 7f05cffff640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f05b0073200 0x7f05b0075710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05c0005fd0 tx=0x7f05c0005dc0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:40.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.221+0000 7f05cd7fa640 1 -- 172.21.15.131:0/76317589 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f05c8016770 con 0x7f05d00cec00 2024-09-17T02:41:40.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.405+0000 7f05d65ee640 1 -- 172.21.15.131:0/76317589 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f05d00c4580 con 0x7f05d00cec00 2024-09-17T02:41:40.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.405+0000 7f05cd7fa640 1 -- 172.21.15.131:0/76317589 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f05c80f91d0 con 0x7f05d00cec00 2024-09-17T02:41:40.407 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:41:40.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 -- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f05b0073200 msgr2=0x7f05b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:40.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f05b0073200 0x7f05b0075710 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f05c0005fd0 tx=0x7f05c0005dc0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 -- 172.21.15.131:0/76317589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d00cec00 msgr2=0x7f05d00c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:40.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d00cec00 0x7f05d00c93b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f05c8009b10 tx=0x7f05c8009310 comp rx=0 tx=0).stop 2024-09-17T02:41:40.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 -- 172.21.15.131:0/76317589 shutdown_connections 2024-09-17T02:41:40.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f05b0073200 0x7f05b0075710 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f05d00cec00 0x7f05d00c93b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f05d00c4ac0 0x7f05d00c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 --2- 172.21.15.131:0/76317589 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f05d006e2c0 0x7f05d014efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 -- 172.21.15.131:0/76317589 >> 172.21.15.131:0/76317589 conn(0x7f05d006eea0 msgr2=0x7f05d00cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:40.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 -- 172.21.15.131:0/76317589 shutdown_connections 2024-09-17T02:41:40.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.409+0000 7f05d65ee640 1 -- 172.21.15.131:0/76317589 wait complete. 2024-09-17T02:41:40.429 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:41:40.430 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:40.436 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:41:40.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.589+0000 7fe42ab5b640 1 -- 172.21.15.131:0/931960346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c003a60 msgr2=0x7fe40c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:40.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.589+0000 7fe42ab5b640 1 --2- 172.21.15.131:0/931960346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c003a60 0x7fe40c003ed0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe4140016c0 tx=0x7fe41401c740 comp rx=0 tx=0).stop 2024-09-17T02:41:40.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.589+0000 7fe42ab5b640 1 -- 172.21.15.131:0/931960346 shutdown_connections 2024-09-17T02:41:40.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.589+0000 7fe42ab5b640 1 --2- 172.21.15.131:0/931960346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c003a60 0x7fe40c003ed0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.589+0000 7fe42ab5b640 1 -- 172.21.15.131:0/931960346 >> 172.21.15.131:0/931960346 conn(0x7fe42406fdb0 msgr2=0x7fe4240701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:40.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.589+0000 7fe42ab5b640 1 -- 172.21.15.131:0/931960346 shutdown_connections 2024-09-17T02:41:40.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.589+0000 7fe42ab5b640 1 -- 172.21.15.131:0/931960346 wait complete. 2024-09-17T02:41:40.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.589+0000 7fe42ab5b640 1 Processor -- start 2024-09-17T02:41:40.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.589+0000 7fe42ab5b640 1 -- start start 2024-09-17T02:41:40.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe42ab5b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c003a60 0x7fe4241fa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:40.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe42ab5b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe42413f8b0 0x7fe42415bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:40.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe42ab5b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe424143cd0 0x7fe4241440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:40.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe42ab5b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe4240717c0 con 0x7fe40c003a60 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe42ab5b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe424071640 con 0x7fe42413f8b0 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe42ab5b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe424071940 con 0x7fe424143cd0 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4288f7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c003a60 0x7fe4241fa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4288f7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c003a60 0x7fe4241fa950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52514/0 (socket says 172.21.15.131:52514) 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4288f7640 1 -- 172.21.15.131:0/4152911545 learned_addr learned my addr 172.21.15.131:0/4152911545 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4290f8640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe424143cd0 0x7fe4241440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4288f7640 1 -- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe424143cd0 msgr2=0x7fe4241440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4288f7640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe424143cd0 0x7fe4241440e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4288f7640 1 -- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe42413f8b0 msgr2=0x7fe42415bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe423fff640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe42413f8b0 0x7fe42415bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4288f7640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe42413f8b0 0x7fe42415bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4288f7640 1 -- 172.21.15.131:0/4152911545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe424144620 con 0x7fe40c003a60 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4288f7640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c003a60 0x7fe4241fa950 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe414000fe0 tx=0x7fe414006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:40.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4217fa640 1 -- 172.21.15.131:0/4152911545 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414003680 con 0x7fe40c003a60 2024-09-17T02:41:40.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4217fa640 1 -- 172.21.15.131:0/4152911545 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe414003910 con 0x7fe40c003a60 2024-09-17T02:41:40.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe42ab5b640 1 -- 172.21.15.131:0/4152911545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4241f6a80 con 0x7fe40c003a60 2024-09-17T02:41:40.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4217fa640 1 -- 172.21.15.131:0/4152911545 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe414003d00 con 0x7fe40c003a60 2024-09-17T02:41:40.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe42ab5b640 1 -- 172.21.15.131:0/4152911545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4241f7040 con 0x7fe40c003a60 2024-09-17T02:41:40.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe42ab5b640 1 -- 172.21.15.131:0/4152911545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4240cdbb0 con 0x7fe40c003a60 2024-09-17T02:41:40.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.593+0000 7fe4217fa640 1 -- 172.21.15.131:0/4152911545 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe4140a5560 con 0x7fe40c003a60 2024-09-17T02:41:40.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.597+0000 7fe4217fa640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 0x7fe410075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:40.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.597+0000 7fe4217fa640 1 -- 172.21.15.131:0/4152911545 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7fe41412df50 con 0x7fe40c003a60 2024-09-17T02:41:40.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.597+0000 7fe423fff640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 0x7fe410075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:40.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.597+0000 7fe423fff640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 0x7fe410075710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe42406ed40 tx=0x7fe418001210 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:40.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.601+0000 7fe4217fa640 1 -- 172.21.15.131:0/4152911545 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe424144620 con 0x7fe40c003a60 2024-09-17T02:41:40.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.765+0000 7fe42ab5b640 1 -- 172.21.15.131:0/4152911545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe424069550 con 0x7fe40c003a60 2024-09-17T02:41:40.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe4217fa640 1 -- 172.21.15.131:0/4152911545 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fe4140b3070 con 0x7fe40c003a60 2024-09-17T02:41:40.769 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:41:40.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 -- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 msgr2=0x7fe410075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 0x7fe410075710 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe42406ed40 tx=0x7fe418001210 comp rx=0 tx=0).stop 2024-09-17T02:41:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 -- 172.21.15.131:0/4152911545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c003a60 msgr2=0x7fe4241fa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c003a60 0x7fe4241fa950 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe414000fe0 tx=0x7fe414006950 comp rx=0 tx=0).stop 2024-09-17T02:41:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 -- 172.21.15.131:0/4152911545 shutdown_connections 2024-09-17T02:41:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 0x7fe410075710 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe424143cd0 0x7fe4241440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe42413f8b0 0x7fe42415bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 --2- 172.21.15.131:0/4152911545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe40c003a60 0x7fe4241fa950 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 -- 172.21.15.131:0/4152911545 >> 172.21.15.131:0/4152911545 conn(0x7fe42406fdb0 msgr2=0x7fe424073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 -- 172.21.15.131:0/4152911545 shutdown_connections 2024-09-17T02:41:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:40.769+0000 7fe42ab5b640 1 -- 172.21.15.131:0/4152911545 wait complete. 2024-09-17T02:41:40.790 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi189.front.sepia.ceph.com... 2024-09-17T02:41:40.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:40.791 DEBUG:teuthology.orchestra.run.smithi189:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-17T02:41:40.816 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:41:40.816 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:41:40.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:40.817 DEBUG:teuthology.orchestra.run.smithi189:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T02:41:40.875 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:41:40.875 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:40.881 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:41:40.881 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:40.929 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:41:41.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.129+0000 7f9efe900640 1 -- 172.21.15.131:0/612092182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef8155240 msgr2=0x7f9ef8073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:41.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.129+0000 7f9efe900640 1 --2- 172.21.15.131:0/612092182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef8155240 0x7f9ef8073430 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80cdb80 tx=0x7f9ee801dbb0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.129+0000 7f9efe900640 1 -- 172.21.15.131:0/612092182 shutdown_connections 2024-09-17T02:41:41.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.129+0000 7f9efe900640 1 --2- 172.21.15.131:0/612092182 >> v2:172.21.15.189:3300/0 conn(0x7f9ef806e5b0 0x7f9ef80cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.129+0000 7f9efe900640 1 --2- 172.21.15.131:0/612092182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef8155240 0x7f9ef8073430 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.129+0000 7f9efe900640 1 -- 172.21.15.131:0/612092182 >> 172.21.15.131:0/612092182 conn(0x7f9ef806eea0 msgr2=0x7f9ef806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:41.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.129+0000 7f9efe900640 1 -- 172.21.15.131:0/612092182 shutdown_connections 2024-09-17T02:41:41.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.129+0000 7f9efe900640 1 -- 172.21.15.131:0/612092182 wait complete. 2024-09-17T02:41:41.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.129+0000 7f9efe900640 1 Processor -- start 2024-09-17T02:41:41.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.129+0000 7f9efe900640 1 -- start start 2024-09-17T02:41:41.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efe900640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9ef8073970 0x7f9ef813d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:41.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efe900640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef8155240 0x7f9ef813d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:41.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efe900640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef814f170 0x7f9ef813e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efe900640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef80cbe20 con 0x7f9ef814f170 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efe900640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef80cbca0 con 0x7f9ef8155240 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efe900640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef8167b70 con 0x7f9ef8073970 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efce9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef814f170 0x7f9ef813e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efce9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef814f170 0x7f9ef813e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39614/0 (socket says 172.21.15.131:39614) 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efce9d640 1 -- 172.21.15.131:0/1845845927 learned_addr learned my addr 172.21.15.131:0/1845845927 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9ef77fe640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef8155240 0x7f9ef813d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9ef7fff640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9ef8073970 0x7f9ef813d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efce9d640 1 -- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9ef8073970 msgr2=0x7f9ef813d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efce9d640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9ef8073970 0x7f9ef813d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efce9d640 1 -- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef8155240 msgr2=0x7f9ef813d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efce9d640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef8155240 0x7f9ef813d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efce9d640 1 -- 172.21.15.131:0/1845845927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef813ee80 con 0x7f9ef814f170 2024-09-17T02:41:41.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efce9d640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef814f170 0x7f9ef813e100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000e9a0 tx=0x7f9ef000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:41.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9ef4ff9640 1 -- 172.21.15.131:0/1845845927 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0004d30 con 0x7f9ef814f170 2024-09-17T02:41:41.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efe900640 1 -- 172.21.15.131:0/1845845927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef80a9680 con 0x7f9ef814f170 2024-09-17T02:41:41.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9ef4ff9640 1 -- 172.21.15.131:0/1845845927 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ef000cb20 con 0x7f9ef814f170 2024-09-17T02:41:41.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9ef4ff9640 1 -- 172.21.15.131:0/1845845927 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0005020 con 0x7f9ef814f170 2024-09-17T02:41:41.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efe900640 1 -- 172.21.15.131:0/1845845927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef8140ca0 con 0x7f9ef814f170 2024-09-17T02:41:41.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9efe900640 1 -- 172.21.15.131:0/1845845927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef8140f00 con 0x7f9ef814f170 2024-09-17T02:41:41.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9ef4ff9640 1 -- 172.21.15.131:0/1845845927 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9ef00051c0 con 0x7f9ef814f170 2024-09-17T02:41:41.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9ef4ff9640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ee4073200 0x7f9ee4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:41.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.133+0000 7f9ef4ff9640 1 -- 172.21.15.131:0/1845845927 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f9ef0095330 con 0x7f9ef814f170 2024-09-17T02:41:41.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.137+0000 7f9ef7fff640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ee4073200 0x7f9ee4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:41.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.137+0000 7f9ef7fff640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ee4073200 0x7f9ee4075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80b0000 tx=0x7f9ee801c3f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:41.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.137+0000 7f9ef4ff9640 1 -- 172.21.15.131:0/1845845927 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9ef0016030 con 0x7f9ef814f170 2024-09-17T02:41:41.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.321+0000 7f9efe900640 1 -- 172.21.15.131:0/1845845927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9ef80a9470 con 0x7f9ef814f170 2024-09-17T02:41:41.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9ef4ff9640 1 -- 172.21.15.131:0/1845845927 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f9ef0060030 con 0x7f9ef814f170 2024-09-17T02:41:41.325 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:41:41.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 -- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ee4073200 msgr2=0x7f9ee4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:41.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ee4073200 0x7f9ee4075710 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80b0000 tx=0x7f9ee801c3f0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 -- 172.21.15.131:0/1845845927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef814f170 msgr2=0x7f9ef813e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:41.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef814f170 0x7f9ef813e100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000e9a0 tx=0x7f9ef000ee60 comp rx=0 tx=0).stop 2024-09-17T02:41:41.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 -- 172.21.15.131:0/1845845927 shutdown_connections 2024-09-17T02:41:41.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ee4073200 0x7f9ee4075710 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ef814f170 0x7f9ef813e100 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ef8155240 0x7f9ef813d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 --2- 172.21.15.131:0/1845845927 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9ef8073970 0x7f9ef813d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 -- 172.21.15.131:0/1845845927 >> 172.21.15.131:0/1845845927 conn(0x7f9ef806eea0 msgr2=0x7f9ef80b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:41.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 -- 172.21.15.131:0/1845845927 shutdown_connections 2024-09-17T02:41:41.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.325+0000 7f9efe900640 1 -- 172.21.15.131:0/1845845927 wait complete. 2024-09-17T02:41:41.347 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:41:41.348 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:41.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:41:41.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.545+0000 7faee26f2640 1 -- 172.21.15.131:0/3294167194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faec4004e70 msgr2=0x7faec40052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:41.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.545+0000 7faee26f2640 1 --2- 172.21.15.131:0/3294167194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faec4004e70 0x7faec40052c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7faedc072b10 tx=0x7faed401c900 comp rx=0 tx=0).stop 2024-09-17T02:41:41.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.545+0000 7faee0c8f640 1 -- 172.21.15.131:0/3294167194 reap_dead start 2024-09-17T02:41:41.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.545+0000 7faee26f2640 1 -- 172.21.15.131:0/3294167194 shutdown_connections 2024-09-17T02:41:41.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.545+0000 7faee26f2640 1 -- 172.21.15.131:0/3294167194 >> 172.21.15.131:0/3294167194 conn(0x7faedc0d1130 msgr2=0x7faedc0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:41.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.545+0000 7faee26f2640 1 -- 172.21.15.131:0/3294167194 shutdown_connections 2024-09-17T02:41:41.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.545+0000 7faee26f2640 1 -- 172.21.15.131:0/3294167194 wait complete. 2024-09-17T02:41:41.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faee26f2640 1 Processor -- start 2024-09-17T02:41:41.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faee26f2640 1 -- start start 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faee26f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec4004e70 0x7faedc1f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faee26f2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc154b20 0x7faedc154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faee26f2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faedc1f6ed0 0x7faedc1f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faee26f2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faedc071210 con 0x7faedc154b20 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faee26f2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faedc0ce220 con 0x7faec4004e70 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faee26f2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7faedc071390 con 0x7faedc1f6ed0 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc154b20 0x7faedc154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faedb7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc154b20 0x7faedc154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39640/0 (socket says 172.21.15.131:39640) 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faedb7fe640 1 -- 172.21.15.131:0/876611464 learned_addr learned my addr 172.21.15.131:0/876611464 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faee0c8f640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faedc1f6ed0 0x7faedc1f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faedbfff640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec4004e70 0x7faedc1f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:41.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faedb7fe640 1 -- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faedc1f6ed0 msgr2=0x7faedc1f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faedb7fe640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faedc1f6ed0 0x7faedc1f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faedb7fe640 1 -- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec4004e70 msgr2=0x7faedc1f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faedb7fe640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec4004e70 0x7faedc1f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faedb7fe640 1 -- 172.21.15.131:0/876611464 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc1f7820 con 0x7faedc154b20 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faedb7fe640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc154b20 0x7faedc154f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faed000d720 tx=0x7faed000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faed8ff9640 1 -- 172.21.15.131:0/876611464 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0017070 con 0x7faedc154b20 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faed8ff9640 1 -- 172.21.15.131:0/876611464 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faed0009d20 con 0x7faedc154b20 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faed8ff9640 1 -- 172.21.15.131:0/876611464 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0005280 con 0x7faedc154b20 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faee26f2640 1 -- 172.21.15.131:0/876611464 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faedc155870 con 0x7faedc154b20 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.549+0000 7faee26f2640 1 -- 172.21.15.131:0/876611464 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faedc155e30 con 0x7faedc154b20 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.553+0000 7faee26f2640 1 -- 172.21.15.131:0/876611464 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faedc160d80 con 0x7faedc154b20 2024-09-17T02:41:41.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.553+0000 7faed8ff9640 1 -- 172.21.15.131:0/876611464 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faed0005420 con 0x7faedc154b20 2024-09-17T02:41:41.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.553+0000 7faed8ff9640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec8073230 0x7faec8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:41.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.553+0000 7faed8ff9640 1 -- 172.21.15.131:0/876611464 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7faed00946c0 con 0x7faedc154b20 2024-09-17T02:41:41.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.553+0000 7faedbfff640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec8073230 0x7faec8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:41.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.553+0000 7faedbfff640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec8073230 0x7faec8075740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faecc001a40 tx=0x7faecc001180 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:41.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.557+0000 7faed8ff9640 1 -- 172.21.15.131:0/876611464 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faedc1f7820 con 0x7faedc154b20 2024-09-17T02:41:41.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.737+0000 7faee26f2640 1 -- 172.21.15.131:0/876611464 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7faedc06f6d0 con 0x7faedc154b20 2024-09-17T02:41:41.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.737+0000 7faed8ff9640 1 -- 172.21.15.131:0/876611464 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7faed005f330 con 0x7faedc154b20 2024-09-17T02:41:41.739 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:41:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 -- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec8073230 msgr2=0x7faec8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec8073230 0x7faec8075740 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faecc001a40 tx=0x7faecc001180 comp rx=0 tx=0).stop 2024-09-17T02:41:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 -- 172.21.15.131:0/876611464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc154b20 msgr2=0x7faedc154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc154b20 0x7faedc154f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faed000d720 tx=0x7faed000dbe0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 -- 172.21.15.131:0/876611464 shutdown_connections 2024-09-17T02:41:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec8073230 0x7faec8075740 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faedc1f6ed0 0x7faedc1f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc154b20 0x7faedc154f30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 --2- 172.21.15.131:0/876611464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faec4004e70 0x7faedc1f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 -- 172.21.15.131:0/876611464 >> 172.21.15.131:0/876611464 conn(0x7faedc0d1130 msgr2=0x7faedc06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 -- 172.21.15.131:0/876611464 shutdown_connections 2024-09-17T02:41:41.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:41.741+0000 7faee26f2640 1 -- 172.21.15.131:0/876611464 wait complete. 2024-09-17T02:41:41.760 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:41:41.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:41.761 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-17T02:41:41.773 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:41:41.773 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:41:41.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:41.773 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-17T02:41:41.821 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:41:41.821 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:41.828 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:41:41.830 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:41.878 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:41:42.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 -- 172.21.15.131:0/2683617677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918406e5b0 msgr2=0x7f918415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 --2- 172.21.15.131:0/2683617677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918406e5b0 0x7f918415bea0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f9174009900 tx=0x7f917401c8e0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 -- 172.21.15.131:0/2683617677 shutdown_connections 2024-09-17T02:41:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 --2- 172.21.15.131:0/2683617677 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f918415c3e0 0x7f91840cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 --2- 172.21.15.131:0/2683617677 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918406e5b0 0x7f918415bea0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 -- 172.21.15.131:0/2683617677 >> 172.21.15.131:0/2683617677 conn(0x7f918406eea0 msgr2=0x7f918406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 -- 172.21.15.131:0/2683617677 shutdown_connections 2024-09-17T02:41:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 -- 172.21.15.131:0/2683617677 wait complete. 2024-09-17T02:41:42.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 Processor -- start 2024-09-17T02:41:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 -- start start 2024-09-17T02:41:42.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91840ced00 0x7f91840b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918415c3e0 0x7f918414af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91841f60f0 0x7f91840b46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9184168890 con 0x7f918415c3e0 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9184168710 con 0x7f91841f60f0 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.037+0000 7f918c3ff640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9184168a10 con 0x7f91840ced00 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918999a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918415c3e0 0x7f918414af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918999a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918415c3e0 0x7f918414af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39664/0 (socket says 172.21.15.131:39664) 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918999a640 1 -- 172.21.15.131:0/4223372274 learned_addr learned my addr 172.21.15.131:0/4223372274 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918a19b640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91840ced00 0x7f91840b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918a99c640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91841f60f0 0x7f91840b46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918999a640 1 -- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91840ced00 msgr2=0x7f91840b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918999a640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91840ced00 0x7f91840b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918999a640 1 -- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91841f60f0 msgr2=0x7f91840b46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918999a640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91841f60f0 0x7f91840b46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918999a640 1 -- 172.21.15.131:0/4223372274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918414b4a0 con 0x7f918415c3e0 2024-09-17T02:41:42.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918999a640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918415c3e0 0x7f918414af60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f91740b0040 tx=0x7f91740a5700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:42.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f917affd640 1 -- 172.21.15.131:0/4223372274 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91740ba020 con 0x7f918415c3e0 2024-09-17T02:41:42.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f917affd640 1 -- 172.21.15.131:0/4223372274 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91740044b0 con 0x7f918415c3e0 2024-09-17T02:41:42.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918c3ff640 1 -- 172.21.15.131:0/4223372274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91841f6890 con 0x7f918415c3e0 2024-09-17T02:41:42.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f917affd640 1 -- 172.21.15.131:0/4223372274 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91740b2040 con 0x7f918415c3e0 2024-09-17T02:41:42.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918c3ff640 1 -- 172.21.15.131:0/4223372274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91841f6e50 con 0x7f918415c3e0 2024-09-17T02:41:42.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918c3ff640 1 -- 172.21.15.131:0/4223372274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91841f6600 con 0x7f918415c3e0 2024-09-17T02:41:42.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f917affd640 1 -- 172.21.15.131:0/4223372274 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f91740bd070 con 0x7f918415c3e0 2024-09-17T02:41:42.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f917affd640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9168073230 0x7f9168075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f917affd640 1 -- 172.21.15.131:0/4223372274 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f917412deb0 con 0x7f918415c3e0 2024-09-17T02:41:42.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.041+0000 7f918a19b640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9168073230 0x7f9168075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:42.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.045+0000 7f918a19b640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9168073230 0x7f9168075740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9180001650 tx=0x7f9180009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:42.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.045+0000 7f917affd640 1 -- 172.21.15.131:0/4223372274 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f91740164d0 con 0x7f918415c3e0 2024-09-17T02:41:42.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.229+0000 7f918c3ff640 1 -- 172.21.15.131:0/4223372274 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f918415bea0 con 0x7f918415c3e0 2024-09-17T02:41:42.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.229+0000 7f917affd640 1 -- 172.21.15.131:0/4223372274 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7f91740f8c70 con 0x7f918415c3e0 2024-09-17T02:41:42.231 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:41:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 -- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9168073230 msgr2=0x7f9168075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9168073230 0x7f9168075740 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9180001650 tx=0x7f9180009040 comp rx=0 tx=0).stop 2024-09-17T02:41:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 -- 172.21.15.131:0/4223372274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918415c3e0 msgr2=0x7f918414af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918415c3e0 0x7f918414af60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f91740b0040 tx=0x7f91740a5700 comp rx=0 tx=0).stop 2024-09-17T02:41:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 -- 172.21.15.131:0/4223372274 shutdown_connections 2024-09-17T02:41:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9168073230 0x7f9168075740 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f91841f60f0 0x7f91840b46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f918415c3e0 0x7f918414af60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 --2- 172.21.15.131:0/4223372274 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91840ced00 0x7f91840b0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 -- 172.21.15.131:0/4223372274 >> 172.21.15.131:0/4223372274 conn(0x7f918406eea0 msgr2=0x7f9184155050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:42.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 -- 172.21.15.131:0/4223372274 shutdown_connections 2024-09-17T02:41:42.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.233+0000 7f918c3ff640 1 -- 172.21.15.131:0/4223372274 wait complete. 2024-09-17T02:41:42.253 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:41:42.254 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:41:42.260 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:41:42.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.413+0000 7fe2b691a640 1 -- 172.21.15.131:0/4259451916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2b0160280 msgr2=0x7fe2b0160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.413+0000 7fe2b691a640 1 --2- 172.21.15.131:0/4259451916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2b0160280 0x7fe2b0160690 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0074640 tx=0x7fe2a001c800 comp rx=0 tx=0).stop 2024-09-17T02:41:42.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.413+0000 7fe2b691a640 1 -- 172.21.15.131:0/4259451916 shutdown_connections 2024-09-17T02:41:42.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.413+0000 7fe2b691a640 1 --2- 172.21.15.131:0/4259451916 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2b0160280 0x7fe2b0160690 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.413+0000 7fe2b691a640 1 --2- 172.21.15.131:0/4259451916 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2b00d1370 0x7fe2b00d1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.413+0000 7fe2b691a640 1 --2- 172.21.15.131:0/4259451916 >> v2:172.21.15.189:3300/0 conn(0x7fe2b00cfdb0 0x7fe2b015d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.413+0000 7fe2b691a640 1 -- 172.21.15.131:0/4259451916 >> 172.21.15.131:0/4259451916 conn(0x7fe2b00b42f0 msgr2=0x7fe2b00d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:42.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.413+0000 7fe2b691a640 1 -- 172.21.15.131:0/4259451916 shutdown_connections 2024-09-17T02:41:42.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.413+0000 7fe2b691a640 1 -- 172.21.15.131:0/4259451916 wait complete. 2024-09-17T02:41:42.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b691a640 1 Processor -- start 2024-09-17T02:41:42.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b691a640 1 -- start start 2024-09-17T02:41:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b691a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b00cfdb0 0x7fe2b0150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b691a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2b00d1370 0x7fe2b00b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b691a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2b0160280 0x7fe2b00b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b691a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b015ee70 con 0x7fe2b00d1370 2024-09-17T02:41:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b691a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b015ecf0 con 0x7fe2b00cfdb0 2024-09-17T02:41:42.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b691a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b015eff0 con 0x7fe2b0160280 2024-09-17T02:41:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2af7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2b00d1370 0x7fe2b00b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2af7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2b00d1370 0x7fe2b00b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39678/0 (socket says 172.21.15.131:39678) 2024-09-17T02:41:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2af7fe640 1 -- 172.21.15.131:0/2258665459 learned_addr learned my addr 172.21.15.131:0/2258665459 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2affff640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b00cfdb0 0x7fe2b0150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b4eb7640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2b0160280 0x7fe2b00b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2af7fe640 1 -- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2b0160280 msgr2=0x7fe2b00b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2af7fe640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2b0160280 0x7fe2b00b7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2af7fe640 1 -- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b00cfdb0 msgr2=0x7fe2b0150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2af7fe640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b00cfdb0 0x7fe2b0150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2af7fe640 1 -- 172.21.15.131:0/2258665459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b014f290 con 0x7fe2b00d1370 2024-09-17T02:41:42.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b4eb7640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2b0160280 0x7fe2b00b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2af7fe640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2b00d1370 0x7fe2b00b52b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400d780 tx=0x7fe2a400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2acff9640 1 -- 172.21.15.131:0/2258665459 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4017070 con 0x7fe2b00d1370 2024-09-17T02:41:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2acff9640 1 -- 172.21.15.131:0/2258665459 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2a400de90 con 0x7fe2b00d1370 2024-09-17T02:41:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b691a640 1 -- 172.21.15.131:0/2258665459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b014f570 con 0x7fe2b00d1370 2024-09-17T02:41:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2acff9640 1 -- 172.21.15.131:0/2258665459 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a40099d0 con 0x7fe2b00d1370 2024-09-17T02:41:42.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.417+0000 7fe2b691a640 1 -- 172.21.15.131:0/2258665459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b014fb30 con 0x7fe2b00d1370 2024-09-17T02:41:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.421+0000 7fe2b691a640 1 -- 172.21.15.131:0/2258665459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b00d1780 con 0x7fe2b00d1370 2024-09-17T02:41:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.421+0000 7fe2acff9640 1 -- 172.21.15.131:0/2258665459 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe2a4009b70 con 0x7fe2b00d1370 2024-09-17T02:41:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.421+0000 7fe2acff9640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe29c073200 0x7fe29c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.421+0000 7fe2acff9640 1 -- 172.21.15.131:0/2258665459 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7fe2a40946f0 con 0x7fe2b00d1370 2024-09-17T02:41:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.421+0000 7fe2affff640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe29c073200 0x7fe29c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:42.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.421+0000 7fe2affff640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe29c073200 0x7fe29c075710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009d30 tx=0x7fe2a80092e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:42.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.425+0000 7fe2acff9640 1 -- 172.21.15.131:0/2258665459 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe2a405f360 con 0x7fe2b00d1370 2024-09-17T02:41:42.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.609+0000 7fe2b691a640 1 -- 172.21.15.131:0/2258665459 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe2b00d25b0 con 0x7fe2b00d1370 2024-09-17T02:41:42.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.609+0000 7fe2acff9640 1 -- 172.21.15.131:0/2258665459 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) ==== 80+0+0 (secure 0 0 0) 0x7fe2b00d25b0 con 0x7fe2b00d1370 2024-09-17T02:41:42.611 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:41:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 -- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe29c073200 msgr2=0x7fe29c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe29c073200 0x7fe29c075710 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009d30 tx=0x7fe2a80092e0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 -- 172.21.15.131:0/2258665459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2b00d1370 msgr2=0x7fe2b00b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2b00d1370 0x7fe2b00b52b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe2a400d780 tx=0x7fe2a400dc40 comp rx=0 tx=0).stop 2024-09-17T02:41:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 -- 172.21.15.131:0/2258665459 shutdown_connections 2024-09-17T02:41:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe29c073200 0x7fe29c075710 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2b0160280 0x7fe2b00b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2b00d1370 0x7fe2b00b52b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 --2- 172.21.15.131:0/2258665459 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2b00cfdb0 0x7fe2b0150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 -- 172.21.15.131:0/2258665459 >> 172.21.15.131:0/2258665459 conn(0x7fe2b00b42f0 msgr2=0x7fe2b00d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 -- 172.21.15.131:0/2258665459 shutdown_connections 2024-09-17T02:41:42.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.613+0000 7fe2b691a640 1 -- 172.21.15.131:0/2258665459 wait complete. 2024-09-17T02:41:42.632 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi189.front.sepia.ceph.com... 2024-09-17T02:41:42.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:42.633 DEBUG:teuthology.orchestra.run.smithi189:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-17T02:41:42.647 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:41:42.647 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:41:42.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:41:42.648 DEBUG:teuthology.orchestra.run.smithi189:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-17T02:41:42.694 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T02:41:42.694 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:41:42.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.849+0000 7f25eb955640 1 -- 172.21.15.131:0/2276507127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25e406e3e0 msgr2=0x7f25e415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.849+0000 7f25eb955640 1 --2- 172.21.15.131:0/2276507127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25e406e3e0 0x7f25e415bea0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f25e406a1c0 tx=0x7f25e001c6e0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.849+0000 7f25eb955640 1 -- 172.21.15.131:0/2276507127 shutdown_connections 2024-09-17T02:41:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.849+0000 7f25eb955640 1 --2- 172.21.15.131:0/2276507127 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25e406e3e0 0x7f25e415bea0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.849+0000 7f25eb955640 1 --2- 172.21.15.131:0/2276507127 >> v2:172.21.15.189:3300/0 conn(0x7f25e40ced00 0x7f25e406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.849+0000 7f25eb955640 1 -- 172.21.15.131:0/2276507127 >> 172.21.15.131:0/2276507127 conn(0x7f25e406eea0 msgr2=0x7f25e406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.849+0000 7f25eb955640 1 -- 172.21.15.131:0/2276507127 shutdown_connections 2024-09-17T02:41:42.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.849+0000 7f25eb955640 1 -- 172.21.15.131:0/2276507127 wait complete. 2024-09-17T02:41:42.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25eb955640 1 Processor -- start 2024-09-17T02:41:42.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25eb955640 1 -- start start 2024-09-17T02:41:42.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25eb955640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25e406e3e0 0x7f25e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25eb955640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25e40ced00 0x7f25e4159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25eb955640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25e415c3e0 0x7f25e4159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25eb955640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25e4168bf0 con 0x7f25e415c3e0 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25eb955640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f25e40cf300 con 0x7f25e406e3e0 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25eb955640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f25e4168d70 con 0x7f25e40ced00 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25e9ef2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25e415c3e0 0x7f25e4159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25e9ef2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25e415c3e0 0x7f25e4159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39704/0 (socket says 172.21.15.131:39704) 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25e9ef2640 1 -- 172.21.15.131:0/1952725052 learned_addr learned my addr 172.21.15.131:0/1952725052 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25e96f1640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25e406e3e0 0x7f25e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25e9ef2640 1 -- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25e40ced00 msgr2=0x7f25e4159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25e9ef2640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25e40ced00 0x7f25e4159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25e9ef2640 1 -- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25e406e3e0 msgr2=0x7f25e415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:42.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25e9ef2640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25e406e3e0 0x7f25e415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25e9ef2640 1 -- 172.21.15.131:0/1952725052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e4157080 con 0x7f25e415c3e0 2024-09-17T02:41:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25e96f1640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25e406e3e0 0x7f25e415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25e9ef2640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25e415c3e0 0x7f25e4159b40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00b9e0 tx=0x7f25dc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25d9ffb640 1 -- 172.21.15.131:0/1952725052 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc019070 con 0x7f25e415c3e0 2024-09-17T02:41:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25d9ffb640 1 -- 172.21.15.131:0/1952725052 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25dc00cc10 con 0x7f25e415c3e0 2024-09-17T02:41:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25eb955640 1 -- 172.21.15.131:0/1952725052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e4157320 con 0x7f25e415c3e0 2024-09-17T02:41:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25d9ffb640 1 -- 172.21.15.131:0/1952725052 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc014440 con 0x7f25e415c3e0 2024-09-17T02:41:42.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.853+0000 7f25eb955640 1 -- 172.21.15.131:0/1952725052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e41578e0 con 0x7f25e415c3e0 2024-09-17T02:41:42.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.857+0000 7f25eb955640 1 -- 172.21.15.131:0/1952725052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e415bea0 con 0x7f25e415c3e0 2024-09-17T02:41:42.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.857+0000 7f25d9ffb640 1 -- 172.21.15.131:0/1952725052 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f25dc01c070 con 0x7f25e415c3e0 2024-09-17T02:41:42.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.857+0000 7f25d9ffb640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25c8073200 0x7f25c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:42.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.857+0000 7f25d9ffb640 1 -- 172.21.15.131:0/1952725052 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f25dc094430 con 0x7f25e415c3e0 2024-09-17T02:41:42.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.857+0000 7f25e96f1640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25c8073200 0x7f25c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:42.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.857+0000 7f25e96f1640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25c8073200 0x7f25c8075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f25d40058d0 tx=0x7f25d4005ca0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:42.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:42.861+0000 7f25d9ffb640 1 -- 172.21.15.131:0/1952725052 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f25dc05ef80 con 0x7f25e415c3e0 2024-09-17T02:41:43.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.049+0000 7f25eb955640 1 -- 172.21.15.131:0/1952725052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25e40cbe70 con 0x7f25e415c3e0 2024-09-17T02:41:43.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.049+0000 7f25d9ffb640 1 -- 172.21.15.131:0/1952725052 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f25dc05e880 con 0x7f25e415c3e0 2024-09-17T02:41:43.051 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:43.051 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"btime":"2024-09-17T02:41:13:211310+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3035404898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3035404898},{"type":"v1","addr":"172.21.15.131:6835","nonce":3035404898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3147445846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3147445846},{"type":"v1","addr":"172.21.15.189:6835","nonce":3147445846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:40:51.611225+0000","modified":"2024-09-17T02:41:12.632560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/1582109479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1582109479},{"type":"v1","addr":"172.21.15.189:6837","nonce":1582109479}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T02:41:43.052 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-09-17T02:41:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 -- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25c8073200 msgr2=0x7f25c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25c8073200 0x7f25c8075710 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f25d40058d0 tx=0x7f25d4005ca0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 -- 172.21.15.131:0/1952725052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25e415c3e0 msgr2=0x7f25e4159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25e415c3e0 0x7f25e4159b40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00b9e0 tx=0x7f25dc00bea0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 -- 172.21.15.131:0/1952725052 shutdown_connections 2024-09-17T02:41:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25c8073200 0x7f25c8075710 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25e415c3e0 0x7f25e4159b40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25e40ced00 0x7f25e4159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 --2- 172.21.15.131:0/1952725052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25e406e3e0 0x7f25e415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 -- 172.21.15.131:0/1952725052 >> 172.21.15.131:0/1952725052 conn(0x7f25e406eea0 msgr2=0x7f25e41549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:43.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 -- 172.21.15.131:0/1952725052 shutdown_connections 2024-09-17T02:41:43.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.053+0000 7f25eb955640 1 -- 172.21.15.131:0/1952725052 wait complete. 2024-09-17T02:41:43.073 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:41:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.221+0000 7f9018fb3640 1 -- 172.21.15.131:0/31639228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f901415c5b0 msgr2=0x7f90140cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.221+0000 7f9018fb3640 1 --2- 172.21.15.131:0/31639228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f901415c5b0 0x7f90140cbaf0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f901406a1c0 tx=0x7f900c01cba0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.221+0000 7f9018fb3640 1 -- 172.21.15.131:0/31639228 shutdown_connections 2024-09-17T02:41:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.221+0000 7f9018fb3640 1 --2- 172.21.15.131:0/31639228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f901415c5b0 0x7f90140cbaf0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.221+0000 7f9018fb3640 1 --2- 172.21.15.131:0/31639228 >> v2:172.21.15.189:3300/0 conn(0x7f90140ced00 0x7f901406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.221+0000 7f9018fb3640 1 -- 172.21.15.131:0/31639228 >> 172.21.15.131:0/31639228 conn(0x7f901406eea0 msgr2=0x7f901406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.221+0000 7f9018fb3640 1 -- 172.21.15.131:0/31639228 shutdown_connections 2024-09-17T02:41:43.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.221+0000 7f9018fb3640 1 -- 172.21.15.131:0/31639228 wait complete. 2024-09-17T02:41:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.221+0000 7f9018fb3640 1 Processor -- start 2024-09-17T02:41:43.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.221+0000 7f9018fb3640 1 -- start start 2024-09-17T02:41:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9018fb3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f901406e3e0 0x7f90141f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9018fb3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f90140ced00 0x7f90140b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9018fb3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f901415c5b0 0x7f90140b5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9018fb3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9014168720 con 0x7f901415c5b0 2024-09-17T02:41:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9018fb3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90141685a0 con 0x7f901406e3e0 2024-09-17T02:41:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9018fb3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f90141688a0 con 0x7f90140ced00 2024-09-17T02:41:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f901259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f901406e3e0 0x7f90141f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9012d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f901415c5b0 0x7f90140b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9009d9b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f90140ced00 0x7f90140b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f901259c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f901406e3e0 0x7f90141f84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.131:39768/0 (socket says 172.21.15.131:39768) 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f901259c640 1 -- 172.21.15.131:0/3074289422 learned_addr learned my addr 172.21.15.131:0/3074289422 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9012d9d640 1 -- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f90140ced00 msgr2=0x7f90140b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9012d9d640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f90140ced00 0x7f90140b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9012d9d640 1 -- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f901406e3e0 msgr2=0x7f90141f84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9012d9d640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f901406e3e0 0x7f90141f84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9012d9d640 1 -- 172.21.15.131:0/3074289422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90140b5d90 con 0x7f901415c5b0 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9012d9d640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f901415c5b0 0x7f90140b5850 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f900c0ac000 tx=0x7f900c0a5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f900b7fe640 1 -- 172.21.15.131:0/3074289422 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c0bb070 con 0x7f901415c5b0 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f900b7fe640 1 -- 172.21.15.131:0/3074289422 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f900c004e20 con 0x7f901415c5b0 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f900b7fe640 1 -- 172.21.15.131:0/3074289422 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f900c0b2040 con 0x7f901415c5b0 2024-09-17T02:41:43.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9018fb3640 1 -- 172.21.15.131:0/3074289422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90141fc610 con 0x7f901415c5b0 2024-09-17T02:41:43.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9018fb3640 1 -- 172.21.15.131:0/3074289422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90141fcbd0 con 0x7f901415c5b0 2024-09-17T02:41:43.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f9018fb3640 1 -- 172.21.15.131:0/3074289422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90141fc380 con 0x7f901415c5b0 2024-09-17T02:41:43.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f900b7fe640 1 -- 172.21.15.131:0/3074289422 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f900c0be070 con 0x7f901415c5b0 2024-09-17T02:41:43.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.225+0000 7f900b7fe640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ff4073200 0x7f8ff4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.229+0000 7f900b7fe640 1 -- 172.21.15.131:0/3074289422 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f900c12e240 con 0x7f901415c5b0 2024-09-17T02:41:43.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.229+0000 7f901259c640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ff4073200 0x7f8ff4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.229+0000 7f901259c640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ff4073200 0x7f8ff4075710 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f90040099b0 tx=0x7f9004009190 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:43.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.233+0000 7f900b7fe640 1 -- 172.21.15.131:0/3074289422 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f901406e7f0 con 0x7f901415c5b0 2024-09-17T02:41:43.329 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:41:43.332 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:41:43.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.421+0000 7f9018fb3640 1 -- 172.21.15.131:0/3074289422 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9014075b80 con 0x7f901415c5b0 2024-09-17T02:41:43.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.421+0000 7f900b7fe640 1 -- 172.21.15.131:0/3074289422 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f900c0f8980 con 0x7f901415c5b0 2024-09-17T02:41:43.424 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:43.425 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"btime":"2024-09-17T02:41:13:211310+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3035404898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3035404898},{"type":"v1","addr":"172.21.15.131:6835","nonce":3035404898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3147445846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3147445846},{"type":"v1","addr":"172.21.15.189:6835","nonce":3147445846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:40:51.611225+0000","modified":"2024-09-17T02:41:12.632560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/1582109479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1582109479},{"type":"v1","addr":"172.21.15.189:6837","nonce":1582109479}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T02:41:43.425 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-09-17T02:41:43.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 -- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ff4073200 msgr2=0x7f8ff4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ff4073200 0x7f8ff4075710 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f90040099b0 tx=0x7f9004009190 comp rx=0 tx=0).stop 2024-09-17T02:41:43.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 -- 172.21.15.131:0/3074289422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f901415c5b0 msgr2=0x7f90140b5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f901415c5b0 0x7f90140b5850 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f900c0ac000 tx=0x7f900c0a5920 comp rx=0 tx=0).stop 2024-09-17T02:41:43.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 -- 172.21.15.131:0/3074289422 shutdown_connections 2024-09-17T02:41:43.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8ff4073200 0x7f8ff4075710 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f901415c5b0 0x7f90140b5850 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f90140ced00 0x7f90140b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 --2- 172.21.15.131:0/3074289422 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f901406e3e0 0x7f90141f84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 -- 172.21.15.131:0/3074289422 >> 172.21.15.131:0/3074289422 conn(0x7f901406eea0 msgr2=0x7f9014155550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:43.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 -- 172.21.15.131:0/3074289422 shutdown_connections 2024-09-17T02:41:43.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.425+0000 7f9018fb3640 1 -- 172.21.15.131:0/3074289422 wait complete. 2024-09-17T02:41:43.445 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:41:43.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 -- 172.21.15.131:0/1841555066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228c0cf710 msgr2=0x7f228c15f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 --2- 172.21.15.131:0/1841555066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228c0cf710 0x7f228c15f290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f228c0ce990 tx=0x7f227c01c8a0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 -- 172.21.15.131:0/1841555066 shutdown_connections 2024-09-17T02:41:43.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 --2- 172.21.15.131:0/1841555066 >> v2:172.21.15.189:3300/0 conn(0x7f228c162950 0x7f228c162d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 --2- 172.21.15.131:0/1841555066 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f228c15f7d0 0x7f228c162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 --2- 172.21.15.131:0/1841555066 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228c0cf710 0x7f228c15f290 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 -- 172.21.15.131:0/1841555066 >> 172.21.15.131:0/1841555066 conn(0x7f228c0d1b10 msgr2=0x7f228c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:43.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 -- 172.21.15.131:0/1841555066 shutdown_connections 2024-09-17T02:41:43.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 -- 172.21.15.131:0/1841555066 wait complete. 2024-09-17T02:41:43.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 Processor -- start 2024-09-17T02:41:43.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 -- start start 2024-09-17T02:41:43.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f228c0cf710 0x7f228c151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228c15f7d0 0x7f228c15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f228c162950 0x7f228c15af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f228c0d2df0 con 0x7f228c15f7d0 2024-09-17T02:41:43.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f228c0d2c70 con 0x7f228c162950 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f229257a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f228c0d2f70 con 0x7f228c0cf710 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f228b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228c15f7d0 0x7f228c15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f228b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228c15f7d0 0x7f228c15aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39750/0 (socket says 172.21.15.131:39750) 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f228b7fe640 1 -- 172.21.15.131:0/2591581276 learned_addr learned my addr 172.21.15.131:0/2591581276 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f2290b17640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f228c162950 0x7f228c15af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f228bfff640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f228c0cf710 0x7f228c151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f228b7fe640 1 -- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f228c0cf710 msgr2=0x7f228c151330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f228b7fe640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f228c0cf710 0x7f228c151330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f228b7fe640 1 -- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f228c162950 msgr2=0x7f228c15af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f228b7fe640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f228c162950 0x7f228c15af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.601+0000 7f228b7fe640 1 -- 172.21.15.131:0/2591581276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c153200 con 0x7f228c15f7d0 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f228b7fe640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228c15f7d0 0x7f228c15aa00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f228000d690 tx=0x7f228000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:43.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f2288ff9640 1 -- 172.21.15.131:0/2591581276 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280017070 con 0x7f228c15f7d0 2024-09-17T02:41:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f2288ff9640 1 -- 172.21.15.131:0/2591581276 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2280005310 con 0x7f228c15f7d0 2024-09-17T02:41:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f2288ff9640 1 -- 172.21.15.131:0/2591581276 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280009820 con 0x7f228c15f7d0 2024-09-17T02:41:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f229257a640 1 -- 172.21.15.131:0/2591581276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228c15c1c0 con 0x7f228c15f7d0 2024-09-17T02:41:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f229257a640 1 -- 172.21.15.131:0/2591581276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228c15a200 con 0x7f228c15f7d0 2024-09-17T02:41:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f229257a640 1 -- 172.21.15.131:0/2591581276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f228c15bf30 con 0x7f228c15f7d0 2024-09-17T02:41:43.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f2288ff9640 1 -- 172.21.15.131:0/2591581276 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f22800099c0 con 0x7f228c15f7d0 2024-09-17T02:41:43.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f2288ff9640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2278073230 0x7f2278075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f2288ff9640 1 -- 172.21.15.131:0/2591581276 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f2280094940 con 0x7f228c15f7d0 2024-09-17T02:41:43.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f228bfff640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2278073230 0x7f2278075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.605+0000 7f228bfff640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2278073230 0x7f2278075740 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f227c0af000 tx=0x7f227c01b0e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:43.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.609+0000 7f2288ff9640 1 -- 172.21.15.131:0/2591581276 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f228005f720 con 0x7f228c15f7d0 2024-09-17T02:41:43.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.777+0000 7f229257a640 1 -- 172.21.15.131:0/2591581276 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f228c0d3460 con 0x7f228c15f7d0 2024-09-17T02:41:43.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.781+0000 7f2288ff9640 1 -- 172.21.15.131:0/2591581276 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18839 (secure 0 0 0) 0x7f228005f020 con 0x7f228c15f7d0 2024-09-17T02:41:43.782 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:43.783 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":28,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:41:33.147294+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.38","mappings":[{"from":0,"to":7}]},{"pgid":"4.e","mappings":[{"from":1,"to":0}]},{"pgid":"4.15","mappings":[{"from":3,"to":2}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:41:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 -- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2278073230 msgr2=0x7f2278075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2278073230 0x7f2278075740 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f227c0af000 tx=0x7f227c01b0e0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 -- 172.21.15.131:0/2591581276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228c15f7d0 msgr2=0x7f228c15aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228c15f7d0 0x7f228c15aa00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f228000d690 tx=0x7f228000db50 comp rx=0 tx=0).stop 2024-09-17T02:41:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 -- 172.21.15.131:0/2591581276 shutdown_connections 2024-09-17T02:41:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2278073230 0x7f2278075740 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f228c162950 0x7f228c15af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f228c15f7d0 0x7f228c15aa00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 --2- 172.21.15.131:0/2591581276 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f228c0cf710 0x7f228c151330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 -- 172.21.15.131:0/2591581276 >> 172.21.15.131:0/2591581276 conn(0x7f228c0d1b10 msgr2=0x7f228c0b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:43.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 -- 172.21.15.131:0/2591581276 shutdown_connections 2024-09-17T02:41:43.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.785+0000 7f229257a640 1 -- 172.21.15.131:0/2591581276 wait complete. 2024-09-17T02:41:43.806 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T02:41:43.806 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:41:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.961+0000 7f89de034640 1 -- 172.21.15.131:0/253998178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89c4003a40 msgr2=0x7f89c4003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.961+0000 7f89de034640 1 --2- 172.21.15.131:0/253998178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89c4003a40 0x7f89c4003eb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f89d80b7610 tx=0x7f89d001caf0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.961+0000 7f89d7fff640 1 -- 172.21.15.131:0/253998178 reap_dead start 2024-09-17T02:41:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.961+0000 7f89de034640 1 -- 172.21.15.131:0/253998178 shutdown_connections 2024-09-17T02:41:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.961+0000 7f89de034640 1 -- 172.21.15.131:0/253998178 >> 172.21.15.131:0/253998178 conn(0x7f89d80d1b10 msgr2=0x7f89d80d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.961+0000 7f89de034640 1 -- 172.21.15.131:0/253998178 shutdown_connections 2024-09-17T02:41:43.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.961+0000 7f89de034640 1 -- 172.21.15.131:0/253998178 wait complete. 2024-09-17T02:41:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89de034640 1 Processor -- start 2024-09-17T02:41:43.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89de034640 1 -- start start 2024-09-17T02:41:43.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89de034640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c4003a40 0x7f89d814ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89de034640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f89d81f8fe0 0x7f89d80cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89de034640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d80cda30 0x7f89d814d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89de034640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f89d816ab50 con 0x7f89d80cda30 2024-09-17T02:41:43.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89de034640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89d816a9d0 con 0x7f89c4003a40 2024-09-17T02:41:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89de034640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f89d816acd0 con 0x7f89d81f8fe0 2024-09-17T02:41:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89d7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d80cda30 0x7f89d814d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89d7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d80cda30 0x7f89d814d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39778/0 (socket says 172.21.15.131:39778) 2024-09-17T02:41:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89d7fff640 1 -- 172.21.15.131:0/1317332519 learned_addr learned my addr 172.21.15.131:0/1317332519 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89d6ffd640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f89d81f8fe0 0x7f89d80cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89d77fe640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c4003a40 0x7f89d814ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89d7fff640 1 -- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f89d81f8fe0 msgr2=0x7f89d80cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89d7fff640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f89d81f8fe0 0x7f89d80cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89d7fff640 1 -- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c4003a40 msgr2=0x7f89d814ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:43.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89d7fff640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c4003a40 0x7f89d814ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89d7fff640 1 -- 172.21.15.131:0/1317332519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d81fc560 con 0x7f89d80cda30 2024-09-17T02:41:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89d7fff640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d80cda30 0x7f89d814d7c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f89d80b72b0 tx=0x7f89c8010ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89cffff640 1 -- 172.21.15.131:0/1317332519 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c8004240 con 0x7f89d80cda30 2024-09-17T02:41:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89cffff640 1 -- 172.21.15.131:0/1317332519 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89c8005020 con 0x7f89d80cda30 2024-09-17T02:41:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89de034640 1 -- 172.21.15.131:0/1317332519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d81f8500 con 0x7f89d80cda30 2024-09-17T02:41:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89cffff640 1 -- 172.21.15.131:0/1317332519 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c80044f0 con 0x7f89d80cda30 2024-09-17T02:41:43.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.965+0000 7f89de034640 1 -- 172.21.15.131:0/1317332519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d81f8ac0 con 0x7f89d80cda30 2024-09-17T02:41:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.969+0000 7f89de034640 1 -- 172.21.15.131:0/1317332519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89d81f8310 con 0x7f89d80cda30 2024-09-17T02:41:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.969+0000 7f89cffff640 1 -- 172.21.15.131:0/1317332519 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f89c801c020 con 0x7f89d80cda30 2024-09-17T02:41:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.969+0000 7f89cffff640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89b8073200 0x7f89b8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.969+0000 7f89cffff640 1 -- 172.21.15.131:0/1317332519 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f89c80944b0 con 0x7f89d80cda30 2024-09-17T02:41:43.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.969+0000 7f89d77fe640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89b8073200 0x7f89b8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:43.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.969+0000 7f89d77fe640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89b8073200 0x7f89b8075710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f89d00aa000 tx=0x7f89d00198e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:43.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:43.973+0000 7f89cffff640 1 -- 172.21.15.131:0/1317332519 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f89c805f080 con 0x7f89d80cda30 2024-09-17T02:41:44.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 -- 172.21.15.131:0/1317332519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89d8154520 con 0x7f89d80cda30 2024-09-17T02:41:44.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89cffff640 1 -- 172.21.15.131:0/1317332519 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f89c805e980 con 0x7f89d80cda30 2024-09-17T02:41:44.185 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:44.185 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"btime":"2024-09-17T02:41:13:211310+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3035404898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3035404898},{"type":"v1","addr":"172.21.15.131:6835","nonce":3035404898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3147445846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3147445846},{"type":"v1","addr":"172.21.15.189:6835","nonce":3147445846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:40:51.611225+0000","modified":"2024-09-17T02:41:12.632560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/1582109479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1582109479},{"type":"v1","addr":"172.21.15.189:6837","nonce":1582109479}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T02:41:44.186 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-09-17T02:41:44.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 -- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89b8073200 msgr2=0x7f89b8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89b8073200 0x7f89b8075710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f89d00aa000 tx=0x7f89d00198e0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 -- 172.21.15.131:0/1317332519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d80cda30 msgr2=0x7f89d814d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d80cda30 0x7f89d814d7c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f89d80b72b0 tx=0x7f89c8010ac0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 -- 172.21.15.131:0/1317332519 shutdown_connections 2024-09-17T02:41:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89b8073200 0x7f89b8075710 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89d80cda30 0x7f89d814d7c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f89d81f8fe0 0x7f89d80cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 --2- 172.21.15.131:0/1317332519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89c4003a40 0x7f89d814ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 -- 172.21.15.131:0/1317332519 >> 172.21.15.131:0/1317332519 conn(0x7f89d80d1b10 msgr2=0x7f89d81537d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:44.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 -- 172.21.15.131:0/1317332519 shutdown_connections 2024-09-17T02:41:44.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.185+0000 7f89de034640 1 -- 172.21.15.131:0/1317332519 wait complete. 2024-09-17T02:41:44.208 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:41:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 -- 172.21.15.131:0/2619244677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21e80b48b0 msgr2=0x7f21e80cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 --2- 172.21.15.131:0/2619244677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21e80b48b0 0x7f21e80cca70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f21d8009840 tx=0x7f21d801c7f0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 -- 172.21.15.131:0/2619244677 shutdown_connections 2024-09-17T02:41:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 --2- 172.21.15.131:0/2619244677 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21e8164310 0x7f21e8166730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 --2- 172.21.15.131:0/2619244677 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21e80b48b0 0x7f21e80cca70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 -- 172.21.15.131:0/2619244677 >> 172.21.15.131:0/2619244677 conn(0x7f21e80cf150 msgr2=0x7f21e80cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 -- 172.21.15.131:0/2619244677 shutdown_connections 2024-09-17T02:41:44.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 -- 172.21.15.131:0/2619244677 wait complete. 2024-09-17T02:41:44.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 Processor -- start 2024-09-17T02:41:44.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 -- start start 2024-09-17T02:41:44.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21e8156bd0 0x7f21e80b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:44.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21e8164310 0x7f21e80c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:44.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e80b5100 0x7f21e80b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:44.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f21e8169820 con 0x7f21e80b5100 2024-09-17T02:41:44.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21e81696a0 con 0x7f21e8164310 2024-09-17T02:41:44.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21efcab640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f21e81699a0 con 0x7f21e8156bd0 2024-09-17T02:41:44.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21ee248640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e80b5100 0x7f21e80b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:44.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21ee248640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e80b5100 0x7f21e80b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39780/0 (socket says 172.21.15.131:39780) 2024-09-17T02:41:44.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21ee248640 1 -- 172.21.15.131:0/1108439835 learned_addr learned my addr 172.21.15.131:0/1108439835 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21ed246640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21e8164310 0x7f21e80c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.353+0000 7f21eda47640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21e8156bd0 0x7f21e80b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21ee248640 1 -- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21e8156bd0 msgr2=0x7f21e80b7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21ee248640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21e8156bd0 0x7f21e80b7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21ee248640 1 -- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21e8164310 msgr2=0x7f21e80c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21ee248640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21e8164310 0x7f21e80c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21ee248640 1 -- 172.21.15.131:0/1108439835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e80b5a50 con 0x7f21e80b5100 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21ed246640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21e8164310 0x7f21e80c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21ee248640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e80b5100 0x7f21e80b5510 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f21e000ea80 tx=0x7f21e000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21de7fc640 1 -- 172.21.15.131:0/1108439835 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e000ce40 con 0x7f21e80b5100 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21de7fc640 1 -- 172.21.15.131:0/1108439835 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21e0004b30 con 0x7f21e80b5100 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21efcab640 1 -- 172.21.15.131:0/1108439835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e80c8460 con 0x7f21e80b5100 2024-09-17T02:41:44.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21de7fc640 1 -- 172.21.15.131:0/1108439835 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0005020 con 0x7f21e80b5100 2024-09-17T02:41:44.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21efcab640 1 -- 172.21.15.131:0/1108439835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e815c080 con 0x7f21e80b5100 2024-09-17T02:41:44.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21efcab640 1 -- 172.21.15.131:0/1108439835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e80cca70 con 0x7f21e80b5100 2024-09-17T02:41:44.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21de7fc640 1 -- 172.21.15.131:0/1108439835 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f21e00051c0 con 0x7f21e80b5100 2024-09-17T02:41:44.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21de7fc640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21cc073200 0x7f21cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:44.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21de7fc640 1 -- 172.21.15.131:0/1108439835 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f21e00952b0 con 0x7f21e80b5100 2024-09-17T02:41:44.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21eda47640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21cc073200 0x7f21cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:44.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.357+0000 7f21eda47640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21cc073200 0x7f21cc075710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f21e80c5dd0 tx=0x7f21e4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:44.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.361+0000 7f21de7fc640 1 -- 172.21.15.131:0/1108439835 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f21e80b5a50 con 0x7f21e80b5100 2024-09-17T02:41:44.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.561+0000 7f21efcab640 1 -- 172.21.15.131:0/1108439835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21e806a170 con 0x7f21e80b5100 2024-09-17T02:41:44.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.561+0000 7f21de7fc640 1 -- 172.21.15.131:0/1108439835 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5231 (secure 0 0 0) 0x7f21e005fea0 con 0x7f21e80b5100 2024-09-17T02:41:44.563 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:44.563 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":9,"btime":"2024-09-17T02:41:13:211310+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4374,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2489001713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2489001713},{"type":"v1","addr":"172.21.15.131:6833","nonce":2489001713}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4377,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3035404898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3035404898},{"type":"v1","addr":"172.21.15.131:6835","nonce":3035404898}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4378,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3147445846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3147445846},{"type":"v1","addr":"172.21.15.189:6835","nonce":3147445846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:40:51.611225+0000","modified":"2024-09-17T02:41:12.632560+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4384},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4384":{"gid":4384,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/1582109479","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1582109479},{"type":"v1","addr":"172.21.15.189:6837","nonce":1582109479}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4384,"qdb_cluster":[4384]},"id":1}]} 2024-09-17T02:41:44.563 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 9 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 -- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21cc073200 msgr2=0x7f21cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21cc073200 0x7f21cc075710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f21e80c5dd0 tx=0x7f21e4009040 comp rx=0 tx=0).stop 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 -- 172.21.15.131:0/1108439835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e80b5100 msgr2=0x7f21e80b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e80b5100 0x7f21e80b5510 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f21e000ea80 tx=0x7f21e000ef40 comp rx=0 tx=0).stop 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 -- 172.21.15.131:0/1108439835 shutdown_connections 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21cc073200 0x7f21cc075710 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21e80b5100 0x7f21e80b5510 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21e8164310 0x7f21e80c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 --2- 172.21.15.131:0/1108439835 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21e8156bd0 0x7f21e80b7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 -- 172.21.15.131:0/1108439835 >> 172.21.15.131:0/1108439835 conn(0x7f21e80cf150 msgr2=0x7f21e815ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 -- 172.21.15.131:0/1108439835 shutdown_connections 2024-09-17T02:41:44.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.565+0000 7f21efcab640 1 -- 172.21.15.131:0/1108439835 wait complete. 2024-09-17T02:41:44.584 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:41:44.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.721+0000 7f4f384c1640 1 -- 172.21.15.131:0/4291070626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f300cec00 msgr2=0x7f4f300cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.721+0000 7f4f384c1640 1 --2- 172.21.15.131:0/4291070626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f300cec00 0x7f4f300cf010 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009bc0 tx=0x7f4f2801cb30 comp rx=0 tx=0).stop 2024-09-17T02:41:44.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.721+0000 7f4f384c1640 1 -- 172.21.15.131:0/4291070626 shutdown_connections 2024-09-17T02:41:44.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.721+0000 7f4f384c1640 1 --2- 172.21.15.131:0/4291070626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f300cec00 0x7f4f300cf010 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.721+0000 7f4f384c1640 1 --2- 172.21.15.131:0/4291070626 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f3006e2c0 0x7f4f300c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.721+0000 7f4f384c1640 1 -- 172.21.15.131:0/4291070626 >> 172.21.15.131:0/4291070626 conn(0x7f4f3006eea0 msgr2=0x7f4f3006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:44.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.721+0000 7f4f384c1640 1 -- 172.21.15.131:0/4291070626 shutdown_connections 2024-09-17T02:41:44.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.721+0000 7f4f384c1640 1 -- 172.21.15.131:0/4291070626 wait complete. 2024-09-17T02:41:44.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f384c1640 1 Processor -- start 2024-09-17T02:41:44.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f384c1640 1 -- start start 2024-09-17T02:41:44.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f384c1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f3006e2c0 0x7f4f301fcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f384c1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f300c4ac0 0x7f4f30152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f384c1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f300cec00 0x7f4f300b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f384c1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30169cf0 con 0x7f4f3006e2c0 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f384c1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3015cb90 con 0x7f4f300cec00 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f384c1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4f30169e70 con 0x7f4f300c4ac0 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f3625d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f3006e2c0 0x7f4f301fcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f3625d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f3006e2c0 0x7f4f301fcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39784/0 (socket says 172.21.15.131:39784) 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f3625d640 1 -- 172.21.15.131:0/2349536064 learned_addr learned my addr 172.21.15.131:0/2349536064 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f36a5e640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f300cec00 0x7f4f300b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f3625d640 1 -- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f300c4ac0 msgr2=0x7f4f30152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f35a5c640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f300c4ac0 0x7f4f30152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f3625d640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f300c4ac0 0x7f4f30152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f3625d640 1 -- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f300cec00 msgr2=0x7f4f300b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f3625d640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f300cec00 0x7f4f300b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f3625d640 1 -- 172.21.15.131:0/2349536064 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f30153140 con 0x7f4f3006e2c0 2024-09-17T02:41:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f36a5e640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f300cec00 0x7f4f300b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f3625d640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f3006e2c0 0x7f4f301fcaf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00e7a0 tx=0x7f4f2c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f26ffd640 1 -- 172.21.15.131:0/2349536064 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c00ccc0 con 0x7f4f3006e2c0 2024-09-17T02:41:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f26ffd640 1 -- 172.21.15.131:0/2349536064 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f2c010430 con 0x7f4f3006e2c0 2024-09-17T02:41:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f26ffd640 1 -- 172.21.15.131:0/2349536064 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c010860 con 0x7f4f3006e2c0 2024-09-17T02:41:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f384c1640 1 -- 172.21.15.131:0/2349536064 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f30165800 con 0x7f4f3006e2c0 2024-09-17T02:41:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f384c1640 1 -- 172.21.15.131:0/2349536064 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f300cae70 con 0x7f4f3006e2c0 2024-09-17T02:41:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.725+0000 7f4f384c1640 1 -- 172.21.15.131:0/2349536064 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f30165d20 con 0x7f4f3006e2c0 2024-09-17T02:41:44.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.729+0000 7f4f26ffd640 1 -- 172.21.15.131:0/2349536064 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4f2c010a00 con 0x7f4f3006e2c0 2024-09-17T02:41:44.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.729+0000 7f4f26ffd640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4f14073230 0x7f4f14075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:44.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.729+0000 7f4f26ffd640 1 -- 172.21.15.131:0/2349536064 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f4f2c094f70 con 0x7f4f3006e2c0 2024-09-17T02:41:44.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.729+0000 7f4f35a5c640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4f14073230 0x7f4f14075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:44.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.729+0000 7f4f35a5c640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4f14073230 0x7f4f14075740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4f200046c0 tx=0x7f4f20009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:44.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.733+0000 7f4f26ffd640 1 -- 172.21.15.131:0/2349536064 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4f2c05fc70 con 0x7f4f3006e2c0 2024-09-17T02:41:44.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.889+0000 7f4f384c1640 1 -- 172.21.15.131:0/2349536064 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f300cac60 con 0x7f4f3006e2c0 2024-09-17T02:41:44.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f26ffd640 1 -- 172.21.15.131:0/2349536064 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) ==== 74+0+18839 (secure 0 0 0) 0x7f4f2c05f570 con 0x7f4f3006e2c0 2024-09-17T02:41:44.894 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:44.894 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":28,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:41:33.147294+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:40:50.086025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T02:40:50.546353+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-17T02:40:54.274506+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":27,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.38","mappings":[{"from":0,"to":7}]},{"pgid":"4.e","mappings":[{"from":1,"to":0}]},{"pgid":"4.15","mappings":[{"from":3,"to":2}]},{"pgid":"4.1d","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:41:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 -- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4f14073230 msgr2=0x7f4f14075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4f14073230 0x7f4f14075740 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4f200046c0 tx=0x7f4f20009040 comp rx=0 tx=0).stop 2024-09-17T02:41:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 -- 172.21.15.131:0/2349536064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f3006e2c0 msgr2=0x7f4f301fcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f3006e2c0 0x7f4f301fcaf0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00e7a0 tx=0x7f4f2c00ec60 comp rx=0 tx=0).stop 2024-09-17T02:41:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 -- 172.21.15.131:0/2349536064 shutdown_connections 2024-09-17T02:41:44.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4f14073230 0x7f4f14075740 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4f300cec00 0x7f4f300b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4f300c4ac0 0x7f4f30152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 --2- 172.21.15.131:0/2349536064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4f3006e2c0 0x7f4f301fcaf0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:44.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 -- 172.21.15.131:0/2349536064 >> 172.21.15.131:0/2349536064 conn(0x7f4f3006eea0 msgr2=0x7f4f301533d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:44.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 -- 172.21.15.131:0/2349536064 shutdown_connections 2024-09-17T02:41:44.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:44.893+0000 7f4f384c1640 1 -- 172.21.15.131:0/2349536064 wait complete. 2024-09-17T02:41:44.915 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:41:45.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.053+0000 7f84292ae640 1 -- 172.21.15.131:0/3600153874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f842406e5b0 msgr2=0x7f84240cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:45.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.053+0000 7f84292ae640 1 --2- 172.21.15.131:0/3600153874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f842406e5b0 0x7f84240cab60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f842406a1c0 tx=0x7f841c01cd40 comp rx=0 tx=0).stop 2024-09-17T02:41:45.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.053+0000 7f84292ae640 1 -- 172.21.15.131:0/3600153874 shutdown_connections 2024-09-17T02:41:45.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.053+0000 7f84292ae640 1 --2- 172.21.15.131:0/3600153874 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f842406e5b0 0x7f84240cab60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.053+0000 7f84292ae640 1 --2- 172.21.15.131:0/3600153874 >> v2:172.21.15.189:3300/0 conn(0x7f8424155240 0x7f8424073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.053+0000 7f84292ae640 1 -- 172.21.15.131:0/3600153874 >> 172.21.15.131:0/3600153874 conn(0x7f842406eea0 msgr2=0x7f842406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:45.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.053+0000 7f84292ae640 1 -- 172.21.15.131:0/3600153874 shutdown_connections 2024-09-17T02:41:45.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.053+0000 7f84292ae640 1 -- 172.21.15.131:0/3600153874 wait complete. 2024-09-17T02:41:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84292ae640 1 Processor -- start 2024-09-17T02:41:45.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84292ae640 1 -- start start 2024-09-17T02:41:45.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84292ae640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8424073970 0x7f8424159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:45.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84292ae640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8424155240 0x7f8424145450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:45.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84292ae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f842415d500 0x7f842415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:45.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84292ae640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f84240cbe20 con 0x7f8424155240 2024-09-17T02:41:45.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84292ae640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84240cbca0 con 0x7f842415d500 2024-09-17T02:41:45.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84292ae640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8424168c80 con 0x7f8424073970 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84227fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8424155240 0x7f8424145450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84227fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8424155240 0x7f8424145450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39798/0 (socket says 172.21.15.131:39798) 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84227fc640 1 -- 172.21.15.131:0/1821502489 learned_addr learned my addr 172.21.15.131:0/1821502489 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f8422ffd640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8424073970 0x7f8424159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84237fe640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f842415d500 0x7f842415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84227fc640 1 -- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8424073970 msgr2=0x7f8424159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84227fc640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8424073970 0x7f8424159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84227fc640 1 -- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f842415d500 msgr2=0x7f842415d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84227fc640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f842415d500 0x7f842415d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84227fc640 1 -- 172.21.15.131:0/1821502489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842415de50 con 0x7f8424155240 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84227fc640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8424155240 0x7f8424145450 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f840c001ec0 tx=0x7f840c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f841b7fe640 1 -- 172.21.15.131:0/1821502489 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c017070 con 0x7f8424155240 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f841b7fe640 1 -- 172.21.15.131:0/1821502489 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f840c0053c0 con 0x7f8424155240 2024-09-17T02:41:45.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f84292ae640 1 -- 172.21.15.131:0/1821502489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8424159e10 con 0x7f8424155240 2024-09-17T02:41:45.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.057+0000 7f841b7fe640 1 -- 172.21.15.131:0/1821502489 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c003860 con 0x7f8424155240 2024-09-17T02:41:45.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.061+0000 7f84292ae640 1 -- 172.21.15.131:0/1821502489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842415a3d0 con 0x7f8424155240 2024-09-17T02:41:45.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.061+0000 7f84292ae640 1 -- 172.21.15.131:0/1821502489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84240cab60 con 0x7f8424155240 2024-09-17T02:41:45.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.061+0000 7f841b7fe640 1 -- 172.21.15.131:0/1821502489 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f840c01a070 con 0x7f8424155240 2024-09-17T02:41:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.061+0000 7f841b7fe640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8404073200 0x7f8404075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.061+0000 7f841b7fe640 1 -- 172.21.15.131:0/1821502489 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f840c0939b0 con 0x7f8424155240 2024-09-17T02:41:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.061+0000 7f8422ffd640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8404073200 0x7f8404075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:45.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.061+0000 7f8422ffd640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8404073200 0x7f8404075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8414005fd0 tx=0x7f841400c460 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:45.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.065+0000 7f841b7fe640 1 -- 172.21.15.131:0/1821502489 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f842415de50 con 0x7f8424155240 2024-09-17T02:41:45.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.257+0000 7f84292ae640 1 -- 172.21.15.131:0/1821502489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f842415ab40 con 0x7f8424155240 2024-09-17T02:41:45.260 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:41:45.257+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:41:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.261+0000 7f841b7fe640 1 -- 172.21.15.131:0/1821502489 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f840c05e620 con 0x7f8424155240 2024-09-17T02:41:45.263 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:41:45.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 -- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8404073200 msgr2=0x7f8404075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:45.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8404073200 0x7f8404075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8414005fd0 tx=0x7f841400c460 comp rx=0 tx=0).stop 2024-09-17T02:41:45.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 -- 172.21.15.131:0/1821502489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8424155240 msgr2=0x7f8424145450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:45.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8424155240 0x7f8424145450 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f840c001ec0 tx=0x7f840c00c890 comp rx=0 tx=0).stop 2024-09-17T02:41:45.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 -- 172.21.15.131:0/1821502489 shutdown_connections 2024-09-17T02:41:45.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8404073200 0x7f8404075710 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f842415d500 0x7f842415d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8424155240 0x7f8424145450 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 --2- 172.21.15.131:0/1821502489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8424073970 0x7f8424159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 -- 172.21.15.131:0/1821502489 >> 172.21.15.131:0/1821502489 conn(0x7f842406eea0 msgr2=0x7f84240cf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 -- 172.21.15.131:0/1821502489 shutdown_connections 2024-09-17T02:41:45.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.265+0000 7f84292ae640 1 -- 172.21.15.131:0/1821502489 wait complete. 2024-09-17T02:41:45.281 INFO:tasks.ceph.mds.d.smithi189.stderr: -3822> 2024-09-17T02:40:57.844+0000 7f2504c08640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:41:45.285 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T02:41:45.386 INFO:tasks.ceph.mds.d.smithi189.stdout:starting mds.d at 2024-09-17T02:41:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 -- 172.21.15.131:0/2923723282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb1015be80 msgr2=0x7ffb10071750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 --2- 172.21.15.131:0/2923723282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb1015be80 0x7ffb10071750 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffb1006e530 tx=0x7ffb0001c850 comp rx=0 tx=0).stop 2024-09-17T02:41:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 -- 172.21.15.131:0/2923723282 shutdown_connections 2024-09-17T02:41:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 --2- 172.21.15.131:0/2923723282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb1015be80 0x7ffb10071750 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 --2- 172.21.15.131:0/2923723282 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb1015ba70 0x7ffb10071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 -- 172.21.15.131:0/2923723282 >> 172.21.15.131:0/2923723282 conn(0x7ffb100d1130 msgr2=0x7ffb100d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 -- 172.21.15.131:0/2923723282 shutdown_connections 2024-09-17T02:41:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 -- 172.21.15.131:0/2923723282 wait complete. 2024-09-17T02:41:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 Processor -- start 2024-09-17T02:41:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 -- start start 2024-09-17T02:41:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb10152550 0x7ffb100c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb1015ba70 0x7ffb100c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1015be80 0x7ffb10158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10168a00 con 0x7ffb10152550 2024-09-17T02:41:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10168880 con 0x7ffb1015be80 2024-09-17T02:41:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb14add640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ffb10168b80 con 0x7ffb1015ba70 2024-09-17T02:41:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb0e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb10152550 0x7ffb100c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb0e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb10152550 0x7ffb100c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39810/0 (socket says 172.21.15.131:39810) 2024-09-17T02:41:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb0e59c640 1 -- 172.21.15.131:0/2532763519 learned_addr learned my addr 172.21.15.131:0/2532763519 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb0dd9b640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb1015ba70 0x7ffb100c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.425+0000 7ffb0ed9d640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1015be80 0x7ffb10158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb0e59c640 1 -- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb1015ba70 msgr2=0x7ffb100c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb0e59c640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb1015ba70 0x7ffb100c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb0e59c640 1 -- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1015be80 msgr2=0x7ffb10158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb0e59c640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1015be80 0x7ffb10158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb0e59c640 1 -- 172.21.15.131:0/2532763519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb101504d0 con 0x7ffb10152550 2024-09-17T02:41:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb0ed9d640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1015be80 0x7ffb10158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:41:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb0e59c640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb10152550 0x7ffb100c67d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c730 tx=0x7ffb0800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb06ffd640 1 -- 172.21.15.131:0/2532763519 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08003b80 con 0x7ffb10152550 2024-09-17T02:41:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb06ffd640 1 -- 172.21.15.131:0/2532763519 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb08003d40 con 0x7ffb10152550 2024-09-17T02:41:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb06ffd640 1 -- 172.21.15.131:0/2532763519 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0800aab0 con 0x7ffb10152550 2024-09-17T02:41:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb14add640 1 -- 172.21.15.131:0/2532763519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb10157340 con 0x7ffb10152550 2024-09-17T02:41:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb14add640 1 -- 172.21.15.131:0/2532763519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb10157900 con 0x7ffb10152550 2024-09-17T02:41:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb14add640 1 -- 172.21.15.131:0/2532763519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1006e840 con 0x7ffb10152550 2024-09-17T02:41:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb06ffd640 1 -- 172.21.15.131:0/2532763519 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffb0800ac50 con 0x7ffb10152550 2024-09-17T02:41:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb06ffd640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaf0073200 0x7ffaf0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb06ffd640 1 -- 172.21.15.131:0/2532763519 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6734+0+0 (secure 0 0 0) 0x7ffb080942d0 con 0x7ffb10152550 2024-09-17T02:41:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb0dd9b640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaf0073200 0x7ffaf0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:45.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.429+0000 7ffb0dd9b640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaf0073200 0x7ffaf0075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffb100b2ab0 tx=0x7ffaf80091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:45.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.433+0000 7ffb06ffd640 1 -- 172.21.15.131:0/2532763519 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffb0805ef80 con 0x7ffb10152550 2024-09-17T02:41:45.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:45.621+0000 7ffb14add640 1 -- 172.21.15.131:0/2532763519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ffb10071ff0 con 0x7ffb10152550 2024-09-17T02:41:46.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.265+0000 7ffb06ffd640 1 -- 172.21.15.131:0/2532763519 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7ffb0805e880 con 0x7ffb10152550 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 -- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaf0073200 msgr2=0x7ffaf0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaf0073200 0x7ffaf0075710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffb100b2ab0 tx=0x7ffaf80091e0 comp rx=0 tx=0).stop 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 -- 172.21.15.131:0/2532763519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb10152550 msgr2=0x7ffb100c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb10152550 0x7ffb100c67d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800c730 tx=0x7ffb0800cbf0 comp rx=0 tx=0).stop 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 -- 172.21.15.131:0/2532763519 shutdown_connections 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaf0073200 0x7ffaf0075710 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb1015be80 0x7ffb10158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb1015ba70 0x7ffb100c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 --2- 172.21.15.131:0/2532763519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb10152550 0x7ffb100c67d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 -- 172.21.15.131:0/2532763519 >> 172.21.15.131:0/2532763519 conn(0x7ffb100d1130 msgr2=0x7ffb101516c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 -- 172.21.15.131:0/2532763519 shutdown_connections 2024-09-17T02:41:46.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.269+0000 7ffb14add640 1 -- 172.21.15.131:0/2532763519 wait complete. 2024-09-17T02:41:46.292 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T02:41:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.449+0000 7fea60f87640 1 -- 172.21.15.131:0/2157974366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea5c1511f0 msgr2=0x7fea5c151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.449+0000 7fea60f87640 1 --2- 172.21.15.131:0/2157974366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea5c1511f0 0x7fea5c151600 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fea5c070f10 tx=0x7fea5401ccc0 comp rx=0 tx=0).stop 2024-09-17T02:41:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.449+0000 7fea60f87640 1 -- 172.21.15.131:0/2157974366 shutdown_connections 2024-09-17T02:41:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.449+0000 7fea60f87640 1 --2- 172.21.15.131:0/2157974366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea5c1511f0 0x7fea5c151600 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.449+0000 7fea60f87640 1 --2- 172.21.15.131:0/2157974366 >> v2:172.21.15.189:3300/0 conn(0x7fea5c15bb70 0x7fea5c0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.449+0000 7fea60f87640 1 -- 172.21.15.131:0/2157974366 >> 172.21.15.131:0/2157974366 conn(0x7fea5c0d05a0 msgr2=0x7fea5c0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.449+0000 7fea60f87640 1 -- 172.21.15.131:0/2157974366 shutdown_connections 2024-09-17T02:41:46.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.449+0000 7fea60f87640 1 -- 172.21.15.131:0/2157974366 wait complete. 2024-09-17T02:41:46.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.449+0000 7fea60f87640 1 Processor -- start 2024-09-17T02:41:46.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.449+0000 7fea60f87640 1 -- start start 2024-09-17T02:41:46.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea60f87640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea5c06f700 0x7fea5c159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:46.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea60f87640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c1511f0 0x7fea5c15a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:46.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea60f87640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea5c15bb70 0x7fea5c13a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:46.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea60f87640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c0b3a20 con 0x7fea5c06f700 2024-09-17T02:41:46.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea60f87640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea5c0b38a0 con 0x7fea5c1511f0 2024-09-17T02:41:46.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea60f87640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fea5c0b3ba0 con 0x7fea5c15bb70 2024-09-17T02:41:46.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea5a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea5c06f700 0x7fea5c159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:46.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea5a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea5c06f700 0x7fea5c159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39828/0 (socket says 172.21.15.131:39828) 2024-09-17T02:41:46.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea5a59c640 1 -- 172.21.15.131:0/1189704518 learned_addr learned my addr 172.21.15.131:0/1189704518 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:46.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea53fff640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c1511f0 0x7fea5c15a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:46.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea5ad9d640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea5c15bb70 0x7fea5c13a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:46.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea5a59c640 1 -- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea5c15bb70 msgr2=0x7fea5c13a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:46.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea5a59c640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea5c15bb70 0x7fea5c13a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:46.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea5a59c640 1 -- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c1511f0 msgr2=0x7fea5c15a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:46.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea5a59c640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c1511f0 0x7fea5c15a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:46.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea5a59c640 1 -- 172.21.15.131:0/1189704518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5c14cf30 con 0x7fea5c06f700 2024-09-17T02:41:46.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea53fff640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c1511f0 0x7fea5c15a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:46.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea5a59c640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea5c06f700 0x7fea5c159db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fea4c007920 tx=0x7fea4c007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:46.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea52ffd640 1 -- 172.21.15.131:0/1189704518 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c019070 con 0x7fea5c06f700 2024-09-17T02:41:46.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea52ffd640 1 -- 172.21.15.131:0/1189704518 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea4c004020 con 0x7fea5c06f700 2024-09-17T02:41:46.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea60f87640 1 -- 172.21.15.131:0/1189704518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5c14d210 con 0x7fea5c06f700 2024-09-17T02:41:46.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea52ffd640 1 -- 172.21.15.131:0/1189704518 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea4c004470 con 0x7fea5c06f700 2024-09-17T02:41:46.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea60f87640 1 -- 172.21.15.131:0/1189704518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea5c14d7d0 con 0x7fea5c06f700 2024-09-17T02:41:46.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea60f87640 1 -- 172.21.15.131:0/1189704518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5c151600 con 0x7fea5c06f700 2024-09-17T02:41:46.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea52ffd640 1 -- 172.21.15.131:0/1189704518 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fea4c01c070 con 0x7fea5c06f700 2024-09-17T02:41:46.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea52ffd640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea3c073200 0x7fea3c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:46.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea52ffd640 1 -- 172.21.15.131:0/1189704518 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6734+0+0 (secure 0 0 0) 0x7fea4c095de0 con 0x7fea5c06f700 2024-09-17T02:41:46.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.453+0000 7fea53fff640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea3c073200 0x7fea3c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:46.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.457+0000 7fea53fff640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea3c073200 0x7fea3c075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0ce490 tx=0x7fea44008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:46.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.457+0000 7fea52ffd640 1 -- 172.21.15.131:0/1189704518 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fea4c023080 con 0x7fea5c06f700 2024-09-17T02:41:46.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:46.629+0000 7fea60f87640 1 -- 172.21.15.131:0/1189704518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fea5c0cecb0 con 0x7fea5c06f700 2024-09-17T02:41:47.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.273+0000 7fea52ffd640 1 -- 172.21.15.131:0/1189704518 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v30) ==== 181+0+0 (secure 0 0 0) 0x7fea4c060b70 con 0x7fea5c06f700 2024-09-17T02:41:47.277 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-09-17T02:41:47.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 -- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea3c073200 msgr2=0x7fea3c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:47.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea3c073200 0x7fea3c075710 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0ce490 tx=0x7fea44008040 comp rx=0 tx=0).stop 2024-09-17T02:41:47.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 -- 172.21.15.131:0/1189704518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea5c06f700 msgr2=0x7fea5c159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:47.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea5c06f700 0x7fea5c159db0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fea4c007920 tx=0x7fea4c007de0 comp rx=0 tx=0).stop 2024-09-17T02:41:47.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 -- 172.21.15.131:0/1189704518 shutdown_connections 2024-09-17T02:41:47.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea3c073200 0x7fea3c075710 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:47.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea5c15bb70 0x7fea5c13a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:47.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea5c1511f0 0x7fea5c15a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:47.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 --2- 172.21.15.131:0/1189704518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea5c06f700 0x7fea5c159db0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:47.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 -- 172.21.15.131:0/1189704518 >> 172.21.15.131:0/1189704518 conn(0x7fea5c0d05a0 msgr2=0x7fea5c0b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:47.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 -- 172.21.15.131:0/1189704518 shutdown_connections 2024-09-17T02:41:47.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.281+0000 7fea60f87640 1 -- 172.21.15.131:0/1189704518 wait complete. 2024-09-17T02:41:47.307 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T02:41:47.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.469+0000 7f4b31b4f640 1 -- 172.21.15.131:0/2292181928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b2c0d0d10 msgr2=0x7f4b2c0d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:47.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.469+0000 7f4b31b4f640 1 --2- 172.21.15.131:0/2292181928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b2c0d0d10 0x7f4b2c0d1120 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c073860 tx=0x7f4b2401cd50 comp rx=0 tx=0).stop 2024-09-17T02:41:47.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.469+0000 7f4b31b4f640 1 -- 172.21.15.131:0/2292181928 shutdown_connections 2024-09-17T02:41:47.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.469+0000 7f4b31b4f640 1 --2- 172.21.15.131:0/2292181928 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b2c0d0d10 0x7f4b2c0d1120 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:47.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.469+0000 7f4b31b4f640 1 --2- 172.21.15.131:0/2292181928 >> v2:172.21.15.189:3300/0 conn(0x7f4b2c0b6890 0x7f4b2c0b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:47.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.469+0000 7f4b31b4f640 1 --2- 172.21.15.131:0/2292181928 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b2c0701b0 0x7f4b2c0705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:47.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.469+0000 7f4b31b4f640 1 -- 172.21.15.131:0/2292181928 >> 172.21.15.131:0/2292181928 conn(0x7f4b2c0d2060 msgr2=0x7f4b2c0d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:47.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.469+0000 7f4b31b4f640 1 -- 172.21.15.131:0/2292181928 shutdown_connections 2024-09-17T02:41:47.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.469+0000 7f4b31b4f640 1 -- 172.21.15.131:0/2292181928 wait complete. 2024-09-17T02:41:47.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.469+0000 7f4b31b4f640 1 Processor -- start 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.469+0000 7f4b31b4f640 1 -- start start 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b31b4f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b2c0701b0 0x7f4b2c15a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b31b4f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c0b6890 0x7f4b2c1436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b31b4f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b2c0d0d10 0x7f4b2c1656f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b31b4f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c1608a0 con 0x7f4b2c0d0d10 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b31b4f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4b2c160720 con 0x7f4b2c0b6890 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b31b4f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4b2c160a20 con 0x7f4b2c0701b0 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b2bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b2c0d0d10 0x7f4b2c1656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b2bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b2c0d0d10 0x7f4b2c1656f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39846/0 (socket says 172.21.15.131:39846) 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b2bfff640 1 -- 172.21.15.131:0/1937014411 learned_addr learned my addr 172.21.15.131:0/1937014411 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b2b7fe640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b2c0701b0 0x7f4b2c15a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:47.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b2affd640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c0b6890 0x7f4b2c1436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:47.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b2bfff640 1 -- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b2c0701b0 msgr2=0x7f4b2c15a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:47.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b2bfff640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b2c0701b0 0x7f4b2c15a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:47.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b2bfff640 1 -- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c0b6890 msgr2=0x7f4b2c1436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:47.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b2bfff640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c0b6890 0x7f4b2c1436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:47.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b2bfff640 1 -- 172.21.15.131:0/1937014411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2c163210 con 0x7f4b2c0d0d10 2024-09-17T02:41:47.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b2bfff640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b2c0d0d10 0x7f4b2c1656f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009bd0 tx=0x7f4b24003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:47.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b23fff640 1 -- 172.21.15.131:0/1937014411 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b240bc070 con 0x7f4b2c0d0d10 2024-09-17T02:41:47.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b23fff640 1 -- 172.21.15.131:0/1937014411 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b240ad8e0 con 0x7f4b2c0d0d10 2024-09-17T02:41:47.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b31b4f640 1 -- 172.21.15.131:0/1937014411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2c1634f0 con 0x7f4b2c0d0d10 2024-09-17T02:41:47.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b23fff640 1 -- 172.21.15.131:0/1937014411 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b240b2040 con 0x7f4b2c0d0d10 2024-09-17T02:41:47.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b31b4f640 1 -- 172.21.15.131:0/1937014411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2c165050 con 0x7f4b2c0d0d10 2024-09-17T02:41:47.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b31b4f640 1 -- 172.21.15.131:0/1937014411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b2c144190 con 0x7f4b2c0d0d10 2024-09-17T02:41:47.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.473+0000 7f4b23fff640 1 -- 172.21.15.131:0/1937014411 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4b240bf070 con 0x7f4b2c0d0d10 2024-09-17T02:41:47.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.477+0000 7f4b23fff640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b0c073200 0x7f4b0c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:47.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.477+0000 7f4b23fff640 1 -- 172.21.15.131:0/1937014411 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6296+0+0 (secure 0 0 0) 0x7f4b2412e350 con 0x7f4b2c0d0d10 2024-09-17T02:41:47.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.477+0000 7f4b2b7fe640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b0c073200 0x7f4b0c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:47.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.477+0000 7f4b2b7fe640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b0c073200 0x7f4b0c075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0058d0 tx=0x7f4b1c005d90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:47.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.477+0000 7f4b23fff640 1 -- 172.21.15.131:0/1937014411 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4b24016970 con 0x7f4b2c0d0d10 2024-09-17T02:41:47.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:47.649+0000 7f4b31b4f640 1 -- 172.21.15.131:0/1937014411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4b2c1652b0 con 0x7f4b2c0d0d10 2024-09-17T02:41:48.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.281+0000 7f4b23fff640 1 -- 172.21.15.131:0/1937014411 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v31) ==== 169+0+0 (secure 0 0 0) 0x7f4b240f9320 con 0x7f4b2c0d0d10 2024-09-17T02:41:48.283 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-09-17T02:41:48.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 -- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b0c073200 msgr2=0x7f4b0c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:48.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b0c073200 0x7f4b0c075710 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0058d0 tx=0x7f4b1c005d90 comp rx=0 tx=0).stop 2024-09-17T02:41:48.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 -- 172.21.15.131:0/1937014411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b2c0d0d10 msgr2=0x7f4b2c1656f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:48.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b2c0d0d10 0x7f4b2c1656f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009bd0 tx=0x7f4b24003f80 comp rx=0 tx=0).stop 2024-09-17T02:41:48.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 -- 172.21.15.131:0/1937014411 shutdown_connections 2024-09-17T02:41:48.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b0c073200 0x7f4b0c075710 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:48.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b2c0d0d10 0x7f4b2c1656f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:48.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b2c0b6890 0x7f4b2c1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:48.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 --2- 172.21.15.131:0/1937014411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b2c0701b0 0x7f4b2c15a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:48.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 -- 172.21.15.131:0/1937014411 >> 172.21.15.131:0/1937014411 conn(0x7f4b2c0d2060 msgr2=0x7f4b2c0d2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:48.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 -- 172.21.15.131:0/1937014411 shutdown_connections 2024-09-17T02:41:48.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.285+0000 7f4b31b4f640 1 -- 172.21.15.131:0/1937014411 wait complete. 2024-09-17T02:41:48.313 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-17T02:41:48.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.481+0000 7f19df286640 1 -- 172.21.15.131:0/1622629196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19c4003a10 msgr2=0x7f19c4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:48.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.481+0000 7f19df286640 1 --2- 172.21.15.131:0/1622629196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19c4003a10 0x7f19c4003e80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f19d400ac70 tx=0x7f19d401e0d0 comp rx=0 tx=0).stop 2024-09-17T02:41:48.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.481+0000 7f19dd823640 1 -- 172.21.15.131:0/1622629196 reap_dead start 2024-09-17T02:41:48.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.481+0000 7f19df286640 1 -- 172.21.15.131:0/1622629196 shutdown_connections 2024-09-17T02:41:48.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.481+0000 7f19df286640 1 -- 172.21.15.131:0/1622629196 >> 172.21.15.131:0/1622629196 conn(0x7f19d80d05a0 msgr2=0x7f19d80d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:48.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.481+0000 7f19df286640 1 -- 172.21.15.131:0/1622629196 shutdown_connections 2024-09-17T02:41:48.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.481+0000 7f19df286640 1 -- 172.21.15.131:0/1622629196 wait complete. 2024-09-17T02:41:48.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.481+0000 7f19df286640 1 Processor -- start 2024-09-17T02:41:48.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.481+0000 7f19df286640 1 -- start start 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19df286640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19d8142590 0x7f19d81f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19df286640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19d8143720 0x7f19d8159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19df286640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d813ae30 0x7f19d815a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19df286640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f19d80731c0 con 0x7f19d813ae30 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19df286640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f19d80ccb40 con 0x7f19d8143720 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19df286640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f19d8073340 con 0x7f19d8142590 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dd823640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d813ae30 0x7f19d815a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dd823640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d813ae30 0x7f19d815a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39864/0 (socket says 172.21.15.131:39864) 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dd823640 1 -- 172.21.15.131:0/3487037848 learned_addr learned my addr 172.21.15.131:0/3487037848 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dd022640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19d8142590 0x7f19d81f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dc821640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19d8143720 0x7f19d8159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:48.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dd823640 1 -- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19d8142590 msgr2=0x7f19d81f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dd823640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19d8142590 0x7f19d81f84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dd823640 1 -- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19d8143720 msgr2=0x7f19d8159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dd823640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19d8143720 0x7f19d8159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dd823640 1 -- 172.21.15.131:0/3487037848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d8151830 con 0x7f19d813ae30 2024-09-17T02:41:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dc821640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19d8143720 0x7f19d8159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:41:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19dd823640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d813ae30 0x7f19d815a060 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f19d80b5280 tx=0x7f19d000dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19cdffb640 1 -- 172.21.15.131:0/3487037848 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d000dea0 con 0x7f19d813ae30 2024-09-17T02:41:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19cdffb640 1 -- 172.21.15.131:0/3487037848 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19d0003a90 con 0x7f19d813ae30 2024-09-17T02:41:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19cdffb640 1 -- 172.21.15.131:0/3487037848 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d000b920 con 0x7f19d813ae30 2024-09-17T02:41:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19df286640 1 -- 172.21.15.131:0/3487037848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d81f8a10 con 0x7f19d813ae30 2024-09-17T02:41:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.485+0000 7f19df286640 1 -- 172.21.15.131:0/3487037848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d8155190 con 0x7f19d813ae30 2024-09-17T02:41:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.489+0000 7f19df286640 1 -- 172.21.15.131:0/3487037848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d80cf6d0 con 0x7f19d813ae30 2024-09-17T02:41:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.489+0000 7f19cdffb640 1 -- 172.21.15.131:0/3487037848 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f19d001b030 con 0x7f19d813ae30 2024-09-17T02:41:48.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.489+0000 7f19cdffb640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073200 0x7f19bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:48.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.489+0000 7f19cdffb640 1 -- 172.21.15.131:0/3487037848 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5853+0+0 (secure 0 0 0) 0x7f19d00939a0 con 0x7f19d813ae30 2024-09-17T02:41:48.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.489+0000 7f19dd022640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073200 0x7f19bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:48.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.489+0000 7f19dd022640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073200 0x7f19bc075710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f19d40b0000 tx=0x7f19d401ae70 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:48.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.493+0000 7f19cdffb640 1 -- 172.21.15.131:0/3487037848 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f19d0022060 con 0x7f19d813ae30 2024-09-17T02:41:48.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:48.665+0000 7f19df286640 1 -- 172.21.15.131:0/3487037848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f19d8154a20 con 0x7f19d813ae30 2024-09-17T02:41:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.285+0000 7f19cdffb640 1 -- 172.21.15.131:0/3487037848 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v32) ==== 178+0+0 (secure 0 0 0) 0x7f19d81f8a10 con 0x7f19d813ae30 2024-09-17T02:41:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data_ec' removed 2024-09-17T02:41:49.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.289+0000 7f19df286640 1 -- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073200 msgr2=0x7f19bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:49.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.289+0000 7f19df286640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073200 0x7f19bc075710 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f19d40b0000 tx=0x7f19d401ae70 comp rx=0 tx=0).stop 2024-09-17T02:41:49.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.289+0000 7f19df286640 1 -- 172.21.15.131:0/3487037848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d813ae30 msgr2=0x7f19d815a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:49.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.289+0000 7f19df286640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d813ae30 0x7f19d815a060 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f19d80b5280 tx=0x7f19d000dc50 comp rx=0 tx=0).stop 2024-09-17T02:41:49.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.289+0000 7f19df286640 1 -- 172.21.15.131:0/3487037848 shutdown_connections 2024-09-17T02:41:49.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.289+0000 7f19df286640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f19bc073200 0x7f19bc075710 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:49.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.289+0000 7f19df286640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19d813ae30 0x7f19d815a060 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:49.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.289+0000 7f19df286640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f19d8143720 0x7f19d8159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:49.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.289+0000 7f19df286640 1 --2- 172.21.15.131:0/3487037848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19d8142590 0x7f19d81f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:49.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.289+0000 7f19df286640 1 -- 172.21.15.131:0/3487037848 >> 172.21.15.131:0/3487037848 conn(0x7f19d80d05a0 msgr2=0x7f19d815b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:49.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.293+0000 7f19df286640 1 -- 172.21.15.131:0/3487037848 shutdown_connections 2024-09-17T02:41:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.293+0000 7f19df286640 1 -- 172.21.15.131:0/3487037848 wait complete. 2024-09-17T02:41:49.323 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T02:41:49.323 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T02:41:49.324 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T02:41:49.324 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:41:49.324 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T02:41:49.324 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T02:41:49.324 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T02:41:49.324 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:41:49.324 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T02:41:49.324 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T02:41:49.325 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T02:41:49.325 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:41:49.325 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T02:41:49.325 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T02:41:49.325 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T02:41:49.325 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:41:49.338 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T02:41:49.338 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T02:41:49.339 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T02:41:49.339 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T02:41:49.347 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T02:41:49.347 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T02:41:49.348 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T02:41:49.348 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T02:41:49.385 INFO:tasks.ceph.mds.c:Started 2024-09-17T02:41:49.386 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:49.386 INFO:tasks.ceph.mds.a:Started 2024-09-17T02:41:49.386 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:49.389 INFO:tasks.ceph.mds.b:Started 2024-09-17T02:41:49.390 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:49.391 INFO:tasks.ceph.mds.d:Started 2024-09-17T02:41:49.391 DEBUG:teuthology.parallel:result is None 2024-09-17T02:41:49.391 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T02:41:49.479 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:41:49.484 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:41:49.489 INFO:tasks.ceph.mds.d.smithi189.stdout:starting mds.d at 2024-09-17T02:41:49.489 INFO:tasks.ceph.mds.b.smithi189.stdout:starting mds.b at 2024-09-17T02:41:49.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.553+0000 7f8de8844640 1 -- 172.21.15.131:0/1916999979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8de415b310 msgr2=0x7f8de40cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.553+0000 7f8de8844640 1 --2- 172.21.15.131:0/1916999979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8de415b310 0x7f8de40cebe0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8de40cb5f0 tx=0x7f8ddc01c890 comp rx=0 tx=0).stop 2024-09-17T02:41:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.553+0000 7f8de8844640 1 -- 172.21.15.131:0/1916999979 shutdown_connections 2024-09-17T02:41:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.553+0000 7f8de8844640 1 --2- 172.21.15.131:0/1916999979 >> v2:172.21.15.189:3300/0 conn(0x7f8de40d0630 0x7f8de40cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.553+0000 7f8de8844640 1 --2- 172.21.15.131:0/1916999979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8de415b310 0x7f8de40cebe0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.553+0000 7f8de8844640 1 -- 172.21.15.131:0/1916999979 >> 172.21.15.131:0/1916999979 conn(0x7f8de406fbe0 msgr2=0x7f8de406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.553+0000 7f8de8844640 1 -- 172.21.15.131:0/1916999979 shutdown_connections 2024-09-17T02:41:49.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.553+0000 7f8de8844640 1 -- 172.21.15.131:0/1916999979 wait complete. 2024-09-17T02:41:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.553+0000 7f8de8844640 1 Processor -- start 2024-09-17T02:41:49.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.553+0000 7f8de8844640 1 -- start start 2024-09-17T02:41:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de8844640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8de415b310 0x7f8de40b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de8844640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8de415f750 0x7f8de41645c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de8844640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8de4164b00 0x7f8de41f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de8844640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8de4168d90 con 0x7f8de4164b00 2024-09-17T02:41:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de8844640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8de4168c10 con 0x7f8de415f750 2024-09-17T02:41:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de8844640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8de4168f10 con 0x7f8de415b310 2024-09-17T02:41:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8de4164b00 0x7f8de41f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8de4164b00 0x7f8de41f8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39920/0 (socket says 172.21.15.131:39920) 2024-09-17T02:41:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de2d9d640 1 -- 172.21.15.131:0/3153597748 learned_addr learned my addr 172.21.15.131:0/3153597748 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de259c640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8de415b310 0x7f8de40b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:49.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de1d9b640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8de415f750 0x7f8de41645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de2d9d640 1 -- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8de415b310 msgr2=0x7f8de40b5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de2d9d640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8de415b310 0x7f8de40b5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de2d9d640 1 -- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8de415f750 msgr2=0x7f8de41645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de2d9d640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8de415f750 0x7f8de41645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de2d9d640 1 -- 172.21.15.131:0/3153597748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de414faf0 con 0x7f8de4164b00 2024-09-17T02:41:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de2d9d640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8de4164b00 0x7f8de41f8290 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400b970 tx=0x7f8dd400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8ddaffd640 1 -- 172.21.15.131:0/3153597748 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4019070 con 0x7f8de4164b00 2024-09-17T02:41:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8ddaffd640 1 -- 172.21.15.131:0/3153597748 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dd400cc50 con 0x7f8de4164b00 2024-09-17T02:41:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8ddaffd640 1 -- 172.21.15.131:0/3153597748 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd40146a0 con 0x7f8de4164b00 2024-09-17T02:41:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de8844640 1 -- 172.21.15.131:0/3153597748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de41fc9b0 con 0x7f8de4164b00 2024-09-17T02:41:49.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de8844640 1 -- 172.21.15.131:0/3153597748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de41fcf70 con 0x7f8de4164b00 2024-09-17T02:41:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de8844640 1 -- 172.21.15.131:0/3153597748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de41fc720 con 0x7f8de4164b00 2024-09-17T02:41:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8ddaffd640 1 -- 172.21.15.131:0/3153597748 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8dd401c070 con 0x7f8de4164b00 2024-09-17T02:41:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8ddaffd640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dc4073200 0x7f8dc4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8ddaffd640 1 -- 172.21.15.131:0/3153597748 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5335+0+0 (secure 0 0 0) 0x7f8dd4093dd0 con 0x7f8de4164b00 2024-09-17T02:41:49.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.557+0000 7f8de259c640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dc4073200 0x7f8dc4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:49.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.561+0000 7f8de259c640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dc4073200 0x7f8dc4075710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0af000 tx=0x7f8ddc01b0d0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:49.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.565+0000 7f8ddaffd640 1 -- 172.21.15.131:0/3153597748 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8dd405e910 con 0x7f8de4164b00 2024-09-17T02:41:49.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:49.745+0000 7f8de8844640 1 -- 172.21.15.131:0/3153597748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8de40cc480 con 0x7f8de4164b00 2024-09-17T02:41:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.329+0000 7f8ddaffd640 1 -- 172.21.15.131:0/3153597748 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v34) ==== 97+0+0 (secure 0 0 0) 0x7f8dd4095020 con 0x7f8de4164b00 2024-09-17T02:41:50.331 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-09-17T02:41:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 -- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dc4073200 msgr2=0x7f8dc4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dc4073200 0x7f8dc4075710 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc0af000 tx=0x7f8ddc01b0d0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 -- 172.21.15.131:0/3153597748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8de4164b00 msgr2=0x7f8de41f8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8de4164b00 0x7f8de41f8290 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400b970 tx=0x7f8dd400be30 comp rx=0 tx=0).stop 2024-09-17T02:41:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 -- 172.21.15.131:0/3153597748 shutdown_connections 2024-09-17T02:41:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dc4073200 0x7f8dc4075710 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8de4164b00 0x7f8de41f8290 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8de415f750 0x7f8de41645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 --2- 172.21.15.131:0/3153597748 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8de415b310 0x7f8de40b5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 -- 172.21.15.131:0/3153597748 >> 172.21.15.131:0/3153597748 conn(0x7f8de406fbe0 msgr2=0x7f8de40b0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 -- 172.21.15.131:0/3153597748 shutdown_connections 2024-09-17T02:41:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.333+0000 7f8de8844640 1 -- 172.21.15.131:0/3153597748 wait complete. 2024-09-17T02:41:50.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T02:41:50.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.505+0000 7f153e7e9640 1 -- 172.21.15.131:0/1204689558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1524004810 msgr2=0x7f1524004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:50.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.505+0000 7f153e7e9640 1 --2- 172.21.15.131:0/1204689558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1524004810 0x7f1524004c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f152800c2e0 tx=0x7f152801f700 comp rx=0 tx=0).stop 2024-09-17T02:41:50.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.505+0000 7f153cd86640 1 -- 172.21.15.131:0/1204689558 reap_dead start 2024-09-17T02:41:50.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.505+0000 7f153e7e9640 1 -- 172.21.15.131:0/1204689558 shutdown_connections 2024-09-17T02:41:50.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.505+0000 7f153e7e9640 1 -- 172.21.15.131:0/1204689558 >> 172.21.15.131:0/1204689558 conn(0x7f153806fbe0 msgr2=0x7f153806fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:50.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.505+0000 7f153e7e9640 1 -- 172.21.15.131:0/1204689558 shutdown_connections 2024-09-17T02:41:50.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.505+0000 7f153e7e9640 1 -- 172.21.15.131:0/1204689558 wait complete. 2024-09-17T02:41:50.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f153e7e9640 1 Processor -- start 2024-09-17T02:41:50.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f153e7e9640 1 -- start start 2024-09-17T02:41:50.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f153e7e9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1524004810 0x7f15380b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:50.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f153e7e9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1538159520 0x7f15380b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:50.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f153e7e9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1538159930 0x7f15380b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:50.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f153e7e9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f153815fce0 con 0x7f1538159520 2024-09-17T02:41:50.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f153e7e9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f153815fb60 con 0x7f1524004810 2024-09-17T02:41:50.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f153e7e9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f153815fe60 con 0x7f1538159930 2024-09-17T02:41:50.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f15377fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1538159520 0x7f15380b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:50.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f15377fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1538159520 0x7f15380b6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39934/0 (socket says 172.21.15.131:39934) 2024-09-17T02:41:50.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f15377fe640 1 -- 172.21.15.131:0/180281758 learned_addr learned my addr 172.21.15.131:0/180281758 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:50.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f15377fe640 1 -- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1538159930 msgr2=0x7f15380b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:50.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f15377fe640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1538159930 0x7f15380b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f15377fe640 1 -- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1524004810 msgr2=0x7f15380b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f15377fe640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1524004810 0x7f15380b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f15377fe640 1 -- 172.21.15.131:0/180281758 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15380b53b0 con 0x7f1538159520 2024-09-17T02:41:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f15377fe640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1538159520 0x7f15380b6160 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1528000c10 tx=0x7f1528006d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f1534ff9640 1 -- 172.21.15.131:0/180281758 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15280bf070 con 0x7f1538159520 2024-09-17T02:41:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f1534ff9640 1 -- 172.21.15.131:0/180281758 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1528003710 con 0x7f1538159520 2024-09-17T02:41:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f1534ff9640 1 -- 172.21.15.131:0/180281758 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15280b3070 con 0x7f1538159520 2024-09-17T02:41:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f153e7e9640 1 -- 172.21.15.131:0/180281758 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15380b5690 con 0x7f1538159520 2024-09-17T02:41:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.509+0000 7f153e7e9640 1 -- 172.21.15.131:0/180281758 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15381f62a0 con 0x7f1538159520 2024-09-17T02:41:50.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.513+0000 7f153e7e9640 1 -- 172.21.15.131:0/180281758 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15380cebc0 con 0x7f1538159520 2024-09-17T02:41:50.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.513+0000 7f1534ff9640 1 -- 172.21.15.131:0/180281758 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1528003b20 con 0x7f1538159520 2024-09-17T02:41:50.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.513+0000 7f1534ff9640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1518073200 0x7f1518075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:50.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.513+0000 7f1534ff9640 1 -- 172.21.15.131:0/180281758 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f1528131920 con 0x7f1538159520 2024-09-17T02:41:50.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.513+0000 7f1537fff640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1518073200 0x7f1518075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:50.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.513+0000 7f1537fff640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1518073200 0x7f1518075710 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f152000da90 tx=0x7f152000d470 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:50.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.517+0000 7f1534ff9640 1 -- 172.21.15.131:0/180281758 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f15280194b0 con 0x7f1538159520 2024-09-17T02:41:50.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f153e7e9640 1 -- 172.21.15.131:0/180281758 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f15380cd4f0 con 0x7f1538159520 2024-09-17T02:41:50.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f1534ff9640 1 -- 172.21.15.131:0/180281758 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+6645 (secure 0 0 0) 0x7f15280fc4c0 con 0x7f1538159520 2024-09-17T02:41:50.750 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:50.750 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:41:50.750 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-09-17T02:41:50.750 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.750 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-09-17T02:41:50.751 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm4U1cHhAATu86VrwT2vMjL/cW2SICMw==", 2024-09-17T02:41:50.751 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.751 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:41:50.751 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:41:50.751 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:41:50.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.751 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.751 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.751 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.751 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-09-17T02:41:50.752 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmVpUSKxAA4l5zGHXg0VRasiyBI/j4ig==", 2024-09-17T02:41:50.752 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.752 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:41:50.752 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:41:50.752 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:41:50.752 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.752 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.752 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmwHuCJRAAlOj+za8ZmqMI55PsShjKDw==", 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm9f+9MhAAS8z9jZFYon3nrmu9/4JMPg==", 2024-09-17T02:41:50.753 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD16+hmV9xkBxAAT+IDf2J4awJfqYcx2xDoNA==", 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:41:50.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:41:50.755 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.755 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.755 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.755 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.755 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-09-17T02:41:50.755 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD46+hmOhJsGBAAdlA4BSdOPk8BySUTfo1ghg==", 2024-09-17T02:41:50.755 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.755 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:41:50.755 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:41:50.755 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.756 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.756 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.756 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.756 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-09-17T02:41:50.756 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD76+hm3mSoKBAAQPoGFy816yWZQW6vPIK6bA==", 2024-09-17T02:41:50.756 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.756 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:41:50.756 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:41:50.756 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.756 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.756 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.757 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.757 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-09-17T02:41:50.757 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD+6+hmpZaPNxAAp0EV6bn5SugDMwA4RnnJwg==", 2024-09-17T02:41:50.757 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.757 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:41:50.757 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:41:50.757 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.757 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAF7OhmgDfVAxAALACK3bk+PK3UhG20b498HQ==", 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.758 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAI7Ohm6w44FBAA90GkelUAbiNjAwn+6IJWcg==", 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAL7Ohm6S6TIxAArISJW//rmEONG2bC/sfkXA==", 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:41:50.759 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:41:50.760 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.760 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.760 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.760 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.760 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-09-17T02:41:50.760 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAO7OhmICwTMxAArAr7DhcO8A0FDSu7VEBehw==", 2024-09-17T02:41:50.760 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.760 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:41:50.760 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA==", 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.761 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmduVoBRAAwr50jSXw6sCyo+5t+3iT7A==", 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-09-17T02:41:50.762 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmBBCDABAAMw9581QyaOxJk5fs35o+ng==", 2024-09-17T02:41:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:41:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:41:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:41:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:41:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-09-17T02:41:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmgznRCRAA1Z8d32ADoSG+7RHtdPGmTQ==", 2024-09-17T02:41:50.763 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.764 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:41:50.764 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:41:50.764 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:41:50.764 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:41:50.764 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.764 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.764 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.764 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-09-17T02:41:50.764 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmpUfvABAAzclxwjR4E+UxvtkbwVqDXw==", 2024-09-17T02:41:50.764 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.764 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-mds", 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAa7OhmZzZADBAAHyE8W6tRWXGSJuOxPfPwKw==", 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-mds" 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.765 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.766 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.766 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-mgr", 2024-09-17T02:41:50.766 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAa7Ohm6VBADBAAplL+VX9TC2bbr5L4WSeWMw==", 2024-09-17T02:41:50.766 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.766 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-17T02:41:50.766 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.766 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.767 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.767 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-osd", 2024-09-17T02:41:50.767 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAa7OhmH2hADBAAJyNeZaypi7QovWbEyBz4sA==", 2024-09-17T02:41:50.767 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.767 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-osd" 2024-09-17T02:41:50.767 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.768 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.768 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.768 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rbd", 2024-09-17T02:41:50.768 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAa7Ohm5H1ADBAAEwVdFu+/YLwNnPADljzBsQ==", 2024-09-17T02:41:50.768 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.769 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-17T02:41:50.769 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.769 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.769 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.769 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-17T02:41:50.769 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAa7Ohm6JRADBAA7ndGKpDjYeujP6pj2caRDQ==", 2024-09-17T02:41:50.769 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.769 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-17T02:41:50.770 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.770 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.770 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.770 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.bootstrap-rgw", 2024-09-17T02:41:50.770 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAa7OhmHK1ADBAAVXg1JcCdb97ELtxzoLyv/Q==", 2024-09-17T02:41:50.770 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.770 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-17T02:41:50.770 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.770 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmnWmJGBAA6fzewrm8ELqUAz/MczTOGA==", 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmrTq3ExAAuR8+2bItlpHqqFzlvPxASA==", 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:41:50.771 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:41:50.772 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:41:50.772 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.772 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:50.772 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-09-17T02:41:50.772 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f153e7e9640 1 -- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1518073200 msgr2=0x7f1518075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f153e7e9640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1518073200 0x7f1518075710 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f152000da90 tx=0x7f152000d470 comp rx=0 tx=0).stop 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f153e7e9640 1 -- 172.21.15.131:0/180281758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1538159520 msgr2=0x7f15380b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f153e7e9640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1538159520 0x7f15380b6160 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1528000c10 tx=0x7f1528006d90 comp rx=0 tx=0).stop 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f153e7e9640 1 -- 172.21.15.131:0/180281758 shutdown_connections 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f153e7e9640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1518073200 0x7f1518075710 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f153e7e9640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1538159930 0x7f15380b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f153e7e9640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1538159520 0x7f15380b6160 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f153e7e9640 1 --2- 172.21.15.131:0/180281758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1524004810 0x7f15380b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.749+0000 7f153e7e9640 1 -- 172.21.15.131:0/180281758 >> 172.21.15.131:0/180281758 conn(0x7f153806fbe0 msgr2=0x7f15380cff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.753+0000 7f153e7e9640 1 -- 172.21.15.131:0/180281758 shutdown_connections 2024-09-17T02:41:50.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.753+0000 7f153e7e9640 1 -- 172.21.15.131:0/180281758 wait complete. 2024-09-17T02:41:50.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-17T02:41:50.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.933+0000 7f0bacf5a640 1 -- 172.21.15.131:0/4026792229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba806e3e0 msgr2=0x7f0ba815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:50.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.933+0000 7f0bacf5a640 1 --2- 172.21.15.131:0/4026792229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba806e3e0 0x7f0ba815bea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80cca40 tx=0x7f0b9c01c8c0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.933+0000 7f0bacf5a640 1 -- 172.21.15.131:0/4026792229 shutdown_connections 2024-09-17T02:41:50.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.933+0000 7f0bacf5a640 1 --2- 172.21.15.131:0/4026792229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba806e3e0 0x7f0ba815bea0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.933+0000 7f0bacf5a640 1 --2- 172.21.15.131:0/4026792229 >> v2:172.21.15.189:3300/0 conn(0x7f0ba815c3e0 0x7f0ba80cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.933+0000 7f0bacf5a640 1 --2- 172.21.15.131:0/4026792229 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ba80ced00 0x7f0ba806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.933+0000 7f0bacf5a640 1 -- 172.21.15.131:0/4026792229 >> 172.21.15.131:0/4026792229 conn(0x7f0ba806eea0 msgr2=0x7f0ba806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:50.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.933+0000 7f0bacf5a640 1 -- 172.21.15.131:0/4026792229 shutdown_connections 2024-09-17T02:41:50.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.933+0000 7f0bacf5a640 1 -- 172.21.15.131:0/4026792229 wait complete. 2024-09-17T02:41:50.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0bacf5a640 1 Processor -- start 2024-09-17T02:41:50.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0bacf5a640 1 -- start start 2024-09-17T02:41:50.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0bacf5a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba80ced00 0x7f0ba814d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0bacf5a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ba815c3e0 0x7f0ba814b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0bacf5a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba814cca0 0x7f0ba80cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0bacf5a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba80caa00 con 0x7f0ba814cca0 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0bacf5a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba80ca880 con 0x7f0ba80ced00 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0bacf5a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba80cab80 con 0x7f0ba815c3e0 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba814cca0 0x7f0ba80cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba814cca0 0x7f0ba80cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39956/0 (socket says 172.21.15.131:39956) 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba6d9d640 1 -- 172.21.15.131:0/1205078592 learned_addr learned my addr 172.21.15.131:0/1205078592 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba5d9b640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ba815c3e0 0x7f0ba814b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba659c640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba80ced00 0x7f0ba814d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba6d9d640 1 -- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ba815c3e0 msgr2=0x7f0ba814b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:50.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba6d9d640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ba815c3e0 0x7f0ba814b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba6d9d640 1 -- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba80ced00 msgr2=0x7f0ba814d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba6d9d640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba80ced00 0x7f0ba814d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba6d9d640 1 -- 172.21.15.131:0/1205078592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba80b3aa0 con 0x7f0ba814cca0 2024-09-17T02:41:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba5d9b640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ba815c3e0 0x7f0ba814b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0ba6d9d640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba814cca0 0x7f0ba80cdbf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000e9a0 tx=0x7f0ba000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0b96ffd640 1 -- 172.21.15.131:0/1205078592 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0004d30 con 0x7f0ba814cca0 2024-09-17T02:41:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0b96ffd640 1 -- 172.21.15.131:0/1205078592 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ba000cb20 con 0x7f0ba814cca0 2024-09-17T02:41:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0b96ffd640 1 -- 172.21.15.131:0/1205078592 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0005020 con 0x7f0ba814cca0 2024-09-17T02:41:50.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0bacf5a640 1 -- 172.21.15.131:0/1205078592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba80b3d80 con 0x7f0ba814cca0 2024-09-17T02:41:50.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0bacf5a640 1 -- 172.21.15.131:0/1205078592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba8157500 con 0x7f0ba814cca0 2024-09-17T02:41:50.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.937+0000 7f0bacf5a640 1 -- 172.21.15.131:0/1205078592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba8157760 con 0x7f0ba814cca0 2024-09-17T02:41:50.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.941+0000 7f0b96ffd640 1 -- 172.21.15.131:0/1205078592 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0ba00051c0 con 0x7f0ba814cca0 2024-09-17T02:41:50.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.941+0000 7f0b96ffd640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0b88073200 0x7f0b88075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:50.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.941+0000 7f0b96ffd640 1 -- 172.21.15.131:0/1205078592 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f0ba0094a30 con 0x7f0ba814cca0 2024-09-17T02:41:50.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.941+0000 7f0ba659c640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0b88073200 0x7f0b88075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:50.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.941+0000 7f0ba659c640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0b88073200 0x7f0b88075710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009980 tx=0x7f0b98005ca0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:50.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:50.945+0000 7f0b96ffd640 1 -- 172.21.15.131:0/1205078592 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0ba0016030 con 0x7f0ba814cca0 2024-09-17T02:41:51.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.177+0000 7f0bacf5a640 1 -- 172.21.15.131:0/1205078592 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f0ba806dea0 con 0x7f0ba814cca0 2024-09-17T02:41:51.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0b96ffd640 1 -- 172.21.15.131:0/1205078592 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v23) ==== 90+0+0 (secure 0 0 0) 0x7f0ba005fc20 con 0x7f0ba814cca0 2024-09-17T02:41:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 -- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0b88073200 msgr2=0x7f0b88075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0b88073200 0x7f0b88075710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0b98009980 tx=0x7f0b98005ca0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 -- 172.21.15.131:0/1205078592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba814cca0 msgr2=0x7f0ba80cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba814cca0 0x7f0ba80cdbf0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000e9a0 tx=0x7f0ba000ee60 comp rx=0 tx=0).stop 2024-09-17T02:41:51.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 -- 172.21.15.131:0/1205078592 shutdown_connections 2024-09-17T02:41:51.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0b88073200 0x7f0b88075710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0ba814cca0 0x7f0ba80cdbf0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0ba815c3e0 0x7f0ba814b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 --2- 172.21.15.131:0/1205078592 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0ba80ced00 0x7f0ba814d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 -- 172.21.15.131:0/1205078592 >> 172.21.15.131:0/1205078592 conn(0x7f0ba806eea0 msgr2=0x7f0ba806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:51.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 -- 172.21.15.131:0/1205078592 shutdown_connections 2024-09-17T02:41:51.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.181+0000 7f0bacf5a640 1 -- 172.21.15.131:0/1205078592 wait complete. 2024-09-17T02:41:51.201 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-17T02:41:51.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.345+0000 7f45993a1640 1 -- 172.21.15.131:0/1398840093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f459406e3b0 msgr2=0x7f4594071250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.345+0000 7f45993a1640 1 --2- 172.21.15.131:0/1398840093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f459406e3b0 0x7f4594071250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f458c009c80 tx=0x7f458c01cc80 comp rx=0 tx=0).stop 2024-09-17T02:41:51.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.345+0000 7f45993a1640 1 -- 172.21.15.131:0/1398840093 shutdown_connections 2024-09-17T02:41:51.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.345+0000 7f45993a1640 1 --2- 172.21.15.131:0/1398840093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f459406e3b0 0x7f4594071250 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.345+0000 7f45993a1640 1 -- 172.21.15.131:0/1398840093 >> 172.21.15.131:0/1398840093 conn(0x7f45940b25e0 msgr2=0x7f45940b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:51.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.345+0000 7f45993a1640 1 -- 172.21.15.131:0/1398840093 shutdown_connections 2024-09-17T02:41:51.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.345+0000 7f45993a1640 1 -- 172.21.15.131:0/1398840093 wait complete. 2024-09-17T02:41:51.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45993a1640 1 Processor -- start 2024-09-17T02:41:51.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45993a1640 1 -- start start 2024-09-17T02:41:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45993a1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f459406dfa0 0x7f459414bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45993a1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f459406e3b0 0x7f4594166170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45993a1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45940c95b0 0x7f45940c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45993a1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4594169720 con 0x7f459406dfa0 2024-09-17T02:41:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45993a1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45941695a0 con 0x7f45940c95b0 2024-09-17T02:41:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45993a1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f45941698a0 con 0x7f459406e3b0 2024-09-17T02:41:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f4592ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f459406dfa0 0x7f459414bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f4592ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f459406dfa0 0x7f459414bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38022/0 (socket says 172.21.15.131:38022) 2024-09-17T02:41:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f4592ffd640 1 -- 172.21.15.131:0/1271569605 learned_addr learned my addr 172.21.15.131:0/1271569605 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45937fe640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45940c95b0 0x7f45940c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45927fc640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f459406e3b0 0x7f4594166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f4592ffd640 1 -- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f459406e3b0 msgr2=0x7f4594166170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f4592ffd640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f459406e3b0 0x7f4594166170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f4592ffd640 1 -- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45940c95b0 msgr2=0x7f45940c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f4592ffd640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45940c95b0 0x7f45940c99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f4592ffd640 1 -- 172.21.15.131:0/1271569605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45940c9f00 con 0x7f459406dfa0 2024-09-17T02:41:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45927fc640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f459406e3b0 0x7f4594166170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f4592ffd640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f459406dfa0 0x7f459414bba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f458400c3c0 tx=0x7f458400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f458b7fe640 1 -- 172.21.15.131:0/1271569605 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584017070 con 0x7f459406dfa0 2024-09-17T02:41:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f458b7fe640 1 -- 172.21.15.131:0/1271569605 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45840053c0 con 0x7f459406dfa0 2024-09-17T02:41:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45993a1640 1 -- 172.21.15.131:0/1271569605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45941fbc30 con 0x7f459406dfa0 2024-09-17T02:41:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f458b7fe640 1 -- 172.21.15.131:0/1271569605 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584003860 con 0x7f459406dfa0 2024-09-17T02:41:51.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45993a1640 1 -- 172.21.15.131:0/1271569605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45941fc1f0 con 0x7f459406dfa0 2024-09-17T02:41:51.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.349+0000 7f45993a1640 1 -- 172.21.15.131:0/1271569605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45940cc5d0 con 0x7f459406dfa0 2024-09-17T02:41:51.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.353+0000 7f458b7fe640 1 -- 172.21.15.131:0/1271569605 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f458401a070 con 0x7f459406dfa0 2024-09-17T02:41:51.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.353+0000 7f458b7fe640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4574073230 0x7f4574075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:51.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.353+0000 7f458b7fe640 1 -- 172.21.15.131:0/1271569605 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f4584093370 con 0x7f459406dfa0 2024-09-17T02:41:51.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.353+0000 7f45927fc640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4574073230 0x7f4574075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:51.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.353+0000 7f45927fc640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4574073230 0x7f4574075740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f45940ca6c0 tx=0x7f457c008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:51.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.357+0000 7f458b7fe640 1 -- 172.21.15.131:0/1271569605 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f45940c9f00 con 0x7f459406dfa0 2024-09-17T02:41:51.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.569+0000 7f45993a1640 1 -- 172.21.15.131:0/1271569605 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f45940ce4d0 con 0x7f459406dfa0 2024-09-17T02:41:51.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f458b7fe640 1 -- 172.21.15.131:0/1271569605 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v24) ==== 90+0+0 (secure 0 0 0) 0x7f458405e550 con 0x7f459406dfa0 2024-09-17T02:41:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 -- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4574073230 msgr2=0x7f4574075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4574073230 0x7f4574075740 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f45940ca6c0 tx=0x7f457c008040 comp rx=0 tx=0).stop 2024-09-17T02:41:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 -- 172.21.15.131:0/1271569605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f459406dfa0 msgr2=0x7f459414bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f459406dfa0 0x7f459414bba0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f458400c3c0 tx=0x7f458400c880 comp rx=0 tx=0).stop 2024-09-17T02:41:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 -- 172.21.15.131:0/1271569605 shutdown_connections 2024-09-17T02:41:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4574073230 0x7f4574075740 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45940c95b0 0x7f45940c99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f459406e3b0 0x7f4594166170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 --2- 172.21.15.131:0/1271569605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f459406dfa0 0x7f459414bba0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 -- 172.21.15.131:0/1271569605 >> 172.21.15.131:0/1271569605 conn(0x7f45940b25e0 msgr2=0x7f45940cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:51.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 -- 172.21.15.131:0/1271569605 shutdown_connections 2024-09-17T02:41:51.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.577+0000 7f45993a1640 1 -- 172.21.15.131:0/1271569605 wait complete. 2024-09-17T02:41:51.599 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-17T02:41:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.745+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3292151471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9c0cec00 msgr2=0x7f9c9c0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.745+0000 7f9ca3f15640 1 --2- 172.21.15.131:0/3292151471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9c0cec00 0x7f9c9c0cf010 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c074bb0 tx=0x7f9c8c01c830 comp rx=0 tx=0).stop 2024-09-17T02:41:51.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.745+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3292151471 shutdown_connections 2024-09-17T02:41:51.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.745+0000 7f9ca3f15640 1 --2- 172.21.15.131:0/3292151471 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c9c0c4c90 0x7f9c9c0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.745+0000 7f9ca3f15640 1 --2- 172.21.15.131:0/3292151471 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9c0cec00 0x7f9c9c0cf010 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.745+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3292151471 >> 172.21.15.131:0/3292151471 conn(0x7f9c9c06eea0 msgr2=0x7f9c9c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:51.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.745+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3292151471 shutdown_connections 2024-09-17T02:41:51.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.745+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3292151471 wait complete. 2024-09-17T02:41:51.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca3f15640 1 Processor -- start 2024-09-17T02:41:51.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca3f15640 1 -- start start 2024-09-17T02:41:51.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca3f15640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c9c06e2c0 0x7f9c9c1505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:51.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca3f15640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c9c0c4c90 0x7f9c9c0c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:51.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca3f15640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9c0cec00 0x7f9c9c0c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:51.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca3f15640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c169c60 con 0x7f9c9c0cec00 2024-09-17T02:41:51.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca3f15640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9c169ae0 con 0x7f9c9c0c4c90 2024-09-17T02:41:51.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca3f15640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9c169de0 con 0x7f9c9c06e2c0 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca24b2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9c0cec00 0x7f9c9c0c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca24b2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9c0cec00 0x7f9c9c0c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38032/0 (socket says 172.21.15.131:38032) 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca24b2640 1 -- 172.21.15.131:0/3539373129 learned_addr learned my addr 172.21.15.131:0/3539373129 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca14b0640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c9c0c4c90 0x7f9c9c0c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca24b2640 1 -- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c9c06e2c0 msgr2=0x7f9c9c1505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca1cb1640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c9c06e2c0 0x7f9c9c1505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca24b2640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c9c06e2c0 0x7f9c9c1505b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca24b2640 1 -- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c9c0c4c90 msgr2=0x7f9c9c0c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca24b2640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c9c0c4c90 0x7f9c9c0c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca24b2640 1 -- 172.21.15.131:0/3539373129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9c1510a0 con 0x7f9c9c0cec00 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca24b2640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9c0cec00 0x7f9c9c0c7590 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400b3c0 tx=0x7f9c9400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9c927fc640 1 -- 172.21.15.131:0/3539373129 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c94011830 con 0x7f9c9c0cec00 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9c927fc640 1 -- 172.21.15.131:0/3539373129 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c94016070 con 0x7f9c9c0cec00 2024-09-17T02:41:51.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9c927fc640 1 -- 172.21.15.131:0/3539373129 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c94011d60 con 0x7f9c9c0cec00 2024-09-17T02:41:51.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3539373129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9c151380 con 0x7f9c9c0cec00 2024-09-17T02:41:51.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.749+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3539373129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9c151940 con 0x7f9c9c0cec00 2024-09-17T02:41:51.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.753+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3539373129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c9c0b1590 con 0x7f9c9c0cec00 2024-09-17T02:41:51.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.753+0000 7f9c927fc640 1 -- 172.21.15.131:0/3539373129 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9c9401b460 con 0x7f9c9c0cec00 2024-09-17T02:41:51.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.753+0000 7f9c927fc640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c80073230 0x7f9c80075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:51.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.753+0000 7f9c927fc640 1 -- 172.21.15.131:0/3539373129 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f9c94093cd0 con 0x7f9c9c0cec00 2024-09-17T02:41:51.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.753+0000 7f9ca1cb1640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c80073230 0x7f9c80075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:51.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.753+0000 7f9ca1cb1640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c80073230 0x7f9c80075740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c98007bf0 tx=0x7f9c98007240 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:51.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.757+0000 7f9c927fc640 1 -- 172.21.15.131:0/3539373129 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9c9405ef40 con 0x7f9c9c0cec00 2024-09-17T02:41:51.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.977+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3539373129 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f9c9c0c50a0 con 0x7f9c9c0cec00 2024-09-17T02:41:51.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.981+0000 7f9c927fc640 1 -- 172.21.15.131:0/3539373129 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v25) ==== 90+0+0 (secure 0 0 0) 0x7f9c9405e840 con 0x7f9c9c0cec00 2024-09-17T02:41:51.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c80073230 msgr2=0x7f9c80075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c80073230 0x7f9c80075740 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c98007bf0 tx=0x7f9c98007240 comp rx=0 tx=0).stop 2024-09-17T02:41:51.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3539373129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9c0cec00 msgr2=0x7f9c9c0c7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:51.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9c0cec00 0x7f9c9c0c7590 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400b3c0 tx=0x7f9c9400b880 comp rx=0 tx=0).stop 2024-09-17T02:41:51.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3539373129 shutdown_connections 2024-09-17T02:41:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c80073230 0x7f9c80075740 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c9c0cec00 0x7f9c9c0c7590 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c9c0c4c90 0x7f9c9c0c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 --2- 172.21.15.131:0/3539373129 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c9c06e2c0 0x7f9c9c1505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3539373129 >> 172.21.15.131:0/3539373129 conn(0x7f9c9c06eea0 msgr2=0x7f9c9c1534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3539373129 shutdown_connections 2024-09-17T02:41:51.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:51.985+0000 7f9ca3f15640 1 -- 172.21.15.131:0/3539373129 wait complete. 2024-09-17T02:41:52.004 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-17T02:41:52.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 -- 172.21.15.131:0/3496391803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe128004b70 msgr2=0x7fe128004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 --2- 172.21.15.131:0/3496391803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe128004b70 0x7fe128004fe0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe130001dc0 tx=0x7fe13001cb00 comp rx=0 tx=0).stop 2024-09-17T02:41:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 -- 172.21.15.131:0/3496391803 shutdown_connections 2024-09-17T02:41:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 --2- 172.21.15.131:0/3496391803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe128004b70 0x7fe128004fe0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 -- 172.21.15.131:0/3496391803 >> 172.21.15.131:0/3496391803 conn(0x7fe14006eea0 msgr2=0x7fe14006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 -- 172.21.15.131:0/3496391803 shutdown_connections 2024-09-17T02:41:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 -- 172.21.15.131:0/3496391803 wait complete. 2024-09-17T02:41:52.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 Processor -- start 2024-09-17T02:41:52.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 -- start start 2024-09-17T02:41:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe128004b70 0x7fe1401525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1400ae9c0 0x7fe1400ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1400ced30 0x7fe1401fa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe14015e7d0 con 0x7fe1400ae9c0 2024-09-17T02:41:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe140073c60 con 0x7fe1400ced30 2024-09-17T02:41:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.161+0000 7fe146dfc640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe14015e950 con 0x7fe128004b70 2024-09-17T02:41:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1400ae9c0 0x7fe1400ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1400ae9c0 0x7fe1400ce7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38046/0 (socket says 172.21.15.131:38046) 2024-09-17T02:41:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13ffff640 1 -- 172.21.15.131:0/675764796 learned_addr learned my addr 172.21.15.131:0/675764796 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe144b98640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe128004b70 0x7fe1401525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe145399640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1400ced30 0x7fe1401fa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13ffff640 1 -- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe128004b70 msgr2=0x7fe1401525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13ffff640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe128004b70 0x7fe1401525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13ffff640 1 -- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1400ced30 msgr2=0x7fe1401fa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13ffff640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1400ced30 0x7fe1401fa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13ffff640 1 -- 172.21.15.131:0/675764796 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1400cf140 con 0x7fe1400ae9c0 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe145399640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1400ced30 0x7fe1401fa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13ffff640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1400ae9c0 0x7fe1400ce7f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe134011a50 tx=0x7fe134011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13d7fa640 1 -- 172.21.15.131:0/675764796 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13401f070 con 0x7fe1400ae9c0 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13d7fa640 1 -- 172.21.15.131:0/675764796 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe13400fe20 con 0x7fe1400ae9c0 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe146dfc640 1 -- 172.21.15.131:0/675764796 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe140157fe0 con 0x7fe1400ae9c0 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13d7fa640 1 -- 172.21.15.131:0/675764796 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134012690 con 0x7fe1400ae9c0 2024-09-17T02:41:52.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe146dfc640 1 -- 172.21.15.131:0/675764796 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1401585a0 con 0x7fe1400ae9c0 2024-09-17T02:41:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe146dfc640 1 -- 172.21.15.131:0/675764796 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe14015eee0 con 0x7fe1400ae9c0 2024-09-17T02:41:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13d7fa640 1 -- 172.21.15.131:0/675764796 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe134022070 con 0x7fe1400ae9c0 2024-09-17T02:41:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13d7fa640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe12c073230 0x7fe12c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe13d7fa640 1 -- 172.21.15.131:0/675764796 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fe134094920 con 0x7fe1400ae9c0 2024-09-17T02:41:52.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.165+0000 7fe144b98640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe12c073230 0x7fe12c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.169+0000 7fe144b98640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe12c073230 0x7fe12c075740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe1300aa000 tx=0x7fe1300a7040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.169+0000 7fe13d7fa640 1 -- 172.21.15.131:0/675764796 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe1400cf140 con 0x7fe1400ae9c0 2024-09-17T02:41:52.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.389+0000 7fe146dfc640 1 -- 172.21.15.131:0/675764796 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fe14006e9d0 con 0x7fe1400ae9c0 2024-09-17T02:41:52.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe13d7fa640 1 -- 172.21.15.131:0/675764796 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v26) ==== 90+0+0 (secure 0 0 0) 0x7fe13405f540 con 0x7fe1400ae9c0 2024-09-17T02:41:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 -- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe12c073230 msgr2=0x7fe12c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe12c073230 0x7fe12c075740 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe1300aa000 tx=0x7fe1300a7040 comp rx=0 tx=0).stop 2024-09-17T02:41:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 -- 172.21.15.131:0/675764796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1400ae9c0 msgr2=0x7fe1400ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1400ae9c0 0x7fe1400ce7f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe134011a50 tx=0x7fe134011f10 comp rx=0 tx=0).stop 2024-09-17T02:41:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 -- 172.21.15.131:0/675764796 shutdown_connections 2024-09-17T02:41:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe12c073230 0x7fe12c075740 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1400ced30 0x7fe1401fa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1400ae9c0 0x7fe1400ce7f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 --2- 172.21.15.131:0/675764796 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe128004b70 0x7fe1401525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 -- 172.21.15.131:0/675764796 >> 172.21.15.131:0/675764796 conn(0x7fe14006eea0 msgr2=0x7fe1400b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:52.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 -- 172.21.15.131:0/675764796 shutdown_connections 2024-09-17T02:41:52.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.393+0000 7fe146dfc640 1 -- 172.21.15.131:0/675764796 wait complete. 2024-09-17T02:41:52.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-17T02:41:52.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.553+0000 7fd6eba35640 1 -- 172.21.15.131:0/3125939000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6e40ced00 msgr2=0x7fd6e406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.553+0000 7fd6eba35640 1 --2- 172.21.15.131:0/3125939000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6e40ced00 0x7fd6e406dea0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40cfe00 tx=0x7fd6e001c6e0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.553+0000 7fd6eba35640 1 -- 172.21.15.131:0/3125939000 shutdown_connections 2024-09-17T02:41:52.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.553+0000 7fd6eba35640 1 --2- 172.21.15.131:0/3125939000 >> v2:172.21.15.189:3300/0 conn(0x7fd6e415c5b0 0x7fd6e40cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.553+0000 7fd6eba35640 1 --2- 172.21.15.131:0/3125939000 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6e40ced00 0x7fd6e406dea0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.553+0000 7fd6eba35640 1 -- 172.21.15.131:0/3125939000 >> 172.21.15.131:0/3125939000 conn(0x7fd6e406eea0 msgr2=0x7fd6e406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:52.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.553+0000 7fd6eba35640 1 -- 172.21.15.131:0/3125939000 shutdown_connections 2024-09-17T02:41:52.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.553+0000 7fd6eba35640 1 -- 172.21.15.131:0/3125939000 wait complete. 2024-09-17T02:41:52.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6eba35640 1 Processor -- start 2024-09-17T02:41:52.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6eba35640 1 -- start start 2024-09-17T02:41:52.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6eba35640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd6e406e3e0 0x7fd6e41f9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6eba35640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6e40ced00 0x7fd6e41f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6eba35640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6e415c5b0 0x7fd6e41563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6eba35640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e41686c0 con 0x7fd6e415c5b0 2024-09-17T02:41:52.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6eba35640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4168540 con 0x7fd6e40ced00 2024-09-17T02:41:52.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6eba35640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e4168840 con 0x7fd6e406e3e0 2024-09-17T02:41:52.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e9fd2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6e415c5b0 0x7fd6e41563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e9fd2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6e415c5b0 0x7fd6e41563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38070/0 (socket says 172.21.15.131:38070) 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e9fd2640 1 -- 172.21.15.131:0/3700947632 learned_addr learned my addr 172.21.15.131:0/3700947632 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e97d1640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd6e406e3e0 0x7fd6e41f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e8fd0640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6e40ced00 0x7fd6e41f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e9fd2640 1 -- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd6e406e3e0 msgr2=0x7fd6e41f9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e9fd2640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd6e406e3e0 0x7fd6e41f9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e9fd2640 1 -- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6e40ced00 msgr2=0x7fd6e41f9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e9fd2640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6e40ced00 0x7fd6e41f9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e9fd2640 1 -- 172.21.15.131:0/3700947632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e4158c80 con 0x7fd6e415c5b0 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e8fd0640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6e40ced00 0x7fd6e41f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6e9fd2640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6e415c5b0 0x7fd6e41563d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00e970 tx=0x7fd6dc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6d9ffb640 1 -- 172.21.15.131:0/3700947632 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc004d30 con 0x7fd6e415c5b0 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6d9ffb640 1 -- 172.21.15.131:0/3700947632 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6dc00cb20 con 0x7fd6e415c5b0 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6eba35640 1 -- 172.21.15.131:0/3700947632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e4158f60 con 0x7fd6e415c5b0 2024-09-17T02:41:52.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6d9ffb640 1 -- 172.21.15.131:0/3700947632 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc005020 con 0x7fd6e415c5b0 2024-09-17T02:41:52.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.557+0000 7fd6eba35640 1 -- 172.21.15.131:0/3700947632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e4159520 con 0x7fd6e415c5b0 2024-09-17T02:41:52.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.561+0000 7fd6eba35640 1 -- 172.21.15.131:0/3700947632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e4159720 con 0x7fd6e415c5b0 2024-09-17T02:41:52.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.561+0000 7fd6d9ffb640 1 -- 172.21.15.131:0/3700947632 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd6dc0051c0 con 0x7fd6e415c5b0 2024-09-17T02:41:52.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.561+0000 7fd6d9ffb640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6c8073230 0x7fd6c8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.561+0000 7fd6d9ffb640 1 -- 172.21.15.131:0/3700947632 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fd6dc094ce0 con 0x7fd6e415c5b0 2024-09-17T02:41:52.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.561+0000 7fd6e97d1640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6c8073230 0x7fd6c8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.561+0000 7fd6e97d1640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6c8073230 0x7fd6c8075740 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00ae000 tx=0x7fd6e001af20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:52.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.565+0000 7fd6d9ffb640 1 -- 172.21.15.131:0/3700947632 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd6dc016030 con 0x7fd6e415c5b0 2024-09-17T02:41:52.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.789+0000 7fd6eba35640 1 -- 172.21.15.131:0/3700947632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fd6e4156090 con 0x7fd6e415c5b0 2024-09-17T02:41:52.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.793+0000 7fd6d9ffb640 1 -- 172.21.15.131:0/3700947632 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v27) ==== 97+0+0 (secure 0 0 0) 0x7fd6dc05ffa0 con 0x7fd6e415c5b0 2024-09-17T02:41:52.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 -- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6c8073230 msgr2=0x7fd6c8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6c8073230 0x7fd6c8075740 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00ae000 tx=0x7fd6e001af20 comp rx=0 tx=0).stop 2024-09-17T02:41:52.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 -- 172.21.15.131:0/3700947632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6e415c5b0 msgr2=0x7fd6e41563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6e415c5b0 0x7fd6e41563d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00e970 tx=0x7fd6dc00ee30 comp rx=0 tx=0).stop 2024-09-17T02:41:52.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 -- 172.21.15.131:0/3700947632 shutdown_connections 2024-09-17T02:41:52.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6c8073230 0x7fd6c8075740 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6e415c5b0 0x7fd6e41563d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6e40ced00 0x7fd6e41f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 --2- 172.21.15.131:0/3700947632 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd6e406e3e0 0x7fd6e41f9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 -- 172.21.15.131:0/3700947632 >> 172.21.15.131:0/3700947632 conn(0x7fd6e406eea0 msgr2=0x7fd6e40b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:52.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 -- 172.21.15.131:0/3700947632 shutdown_connections 2024-09-17T02:41:52.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.797+0000 7fd6eba35640 1 -- 172.21.15.131:0/3700947632 wait complete. 2024-09-17T02:41:52.814 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-17T02:41:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.969+0000 7fda0dcd2640 1 -- 172.21.15.131:0/451219075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda0806e2c0 msgr2=0x7fda080c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.969+0000 7fda0dcd2640 1 --2- 172.21.15.131:0/451219075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda0806e2c0 0x7fda080c4580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda08074bb0 tx=0x7fd9f801c690 comp rx=0 tx=0).stop 2024-09-17T02:41:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.969+0000 7fda0dcd2640 1 -- 172.21.15.131:0/451219075 shutdown_connections 2024-09-17T02:41:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.969+0000 7fda0dcd2640 1 --2- 172.21.15.131:0/451219075 >> v2:172.21.15.189:3300/0 conn(0x7fda080c4ac0 0x7fda080c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.969+0000 7fda0dcd2640 1 --2- 172.21.15.131:0/451219075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda0806e2c0 0x7fda080c4580 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.969+0000 7fda0dcd2640 1 -- 172.21.15.131:0/451219075 >> 172.21.15.131:0/451219075 conn(0x7fda0806eea0 msgr2=0x7fda0806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.969+0000 7fda0dcd2640 1 -- 172.21.15.131:0/451219075 shutdown_connections 2024-09-17T02:41:52.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.969+0000 7fda0dcd2640 1 -- 172.21.15.131:0/451219075 wait complete. 2024-09-17T02:41:52.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.969+0000 7fda0dcd2640 1 Processor -- start 2024-09-17T02:41:52.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.969+0000 7fda0dcd2640 1 -- start start 2024-09-17T02:41:52.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda0dcd2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda0806e2c0 0x7fda0815bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda0dcd2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda080c4ac0 0x7fda08149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda0dcd2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda080cdbf0 0x7fda08152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda0dcd2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda080cef20 con 0x7fda080cdbf0 2024-09-17T02:41:52.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda0dcd2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda080cf0a0 con 0x7fda080c4ac0 2024-09-17T02:41:52.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda0dcd2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fda080cf220 con 0x7fda0806e2c0 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda07fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda080cdbf0 0x7fda08152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda07fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda080cdbf0 0x7fda08152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38098/0 (socket says 172.21.15.131:38098) 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda07fff640 1 -- 172.21.15.131:0/2077573074 learned_addr learned my addr 172.21.15.131:0/2077573074 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda077fe640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda0806e2c0 0x7fda0815bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fd9feffd640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda080c4ac0 0x7fda08149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda07fff640 1 -- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda0806e2c0 msgr2=0x7fda0815bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda07fff640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda0806e2c0 0x7fda0815bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda07fff640 1 -- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda080c4ac0 msgr2=0x7fda08149b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda07fff640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda080c4ac0 0x7fda08149b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda07fff640 1 -- 172.21.15.131:0/2077573074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0815c410 con 0x7fda080cdbf0 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda07fff640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda080cdbf0 0x7fda08152a40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fda0000cbc0 tx=0x7fda0000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda04ff9640 1 -- 172.21.15.131:0/2077573074 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00016020 con 0x7fda080cdbf0 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda04ff9640 1 -- 172.21.15.131:0/2077573074 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda0000ab60 con 0x7fda080cdbf0 2024-09-17T02:41:52.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda0dcd2640 1 -- 172.21.15.131:0/2077573074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda080ca710 con 0x7fda080cdbf0 2024-09-17T02:41:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda04ff9640 1 -- 172.21.15.131:0/2077573074 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00012420 con 0x7fda080cdbf0 2024-09-17T02:41:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda0dcd2640 1 -- 172.21.15.131:0/2077573074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda080cacd0 con 0x7fda080cdbf0 2024-09-17T02:41:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda0dcd2640 1 -- 172.21.15.131:0/2077573074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0815c180 con 0x7fda080cdbf0 2024-09-17T02:41:52.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda04ff9640 1 -- 172.21.15.131:0/2077573074 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fda00019020 con 0x7fda080cdbf0 2024-09-17T02:41:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda04ff9640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd9e8073200 0x7fd9e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda04ff9640 1 -- 172.21.15.131:0/2077573074 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fda00094330 con 0x7fda080cdbf0 2024-09-17T02:41:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.973+0000 7fda077fe640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd9e8073200 0x7fd9e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:52.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.977+0000 7fda077fe640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd9e8073200 0x7fd9e8075710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80af000 tx=0x7fd9f801aed0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:52.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:52.977+0000 7fda04ff9640 1 -- 172.21.15.131:0/2077573074 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda00020020 con 0x7fda080cdbf0 2024-09-17T02:41:53.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.209+0000 7fda0dcd2640 1 -- 172.21.15.131:0/2077573074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7fda080c4580 con 0x7fda080cdbf0 2024-09-17T02:41:53.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.213+0000 7fda04ff9640 1 -- 172.21.15.131:0/2077573074 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v28) ==== 90+0+0 (secure 0 0 0) 0x7fda0005f5a0 con 0x7fda080cdbf0 2024-09-17T02:41:53.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 -- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd9e8073200 msgr2=0x7fd9e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:53.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd9e8073200 0x7fd9e8075710 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80af000 tx=0x7fd9f801aed0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 -- 172.21.15.131:0/2077573074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda080cdbf0 msgr2=0x7fda08152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:53.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda080cdbf0 0x7fda08152a40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fda0000cbc0 tx=0x7fda0000ccb0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 -- 172.21.15.131:0/2077573074 shutdown_connections 2024-09-17T02:41:53.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd9e8073200 0x7fd9e8075710 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda080cdbf0 0x7fda08152a40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda080c4ac0 0x7fda08149b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 --2- 172.21.15.131:0/2077573074 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda0806e2c0 0x7fda0815bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 -- 172.21.15.131:0/2077573074 >> 172.21.15.131:0/2077573074 conn(0x7fda0806eea0 msgr2=0x7fda080b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:53.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 -- 172.21.15.131:0/2077573074 shutdown_connections 2024-09-17T02:41:53.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.217+0000 7fda0dcd2640 1 -- 172.21.15.131:0/2077573074 wait complete. 2024-09-17T02:41:53.238 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T02:41:53.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 -- 172.21.15.131:0/4216799905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399c06e2c0 msgr2=0x7f399c0c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:53.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 --2- 172.21.15.131:0/4216799905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399c06e2c0 0x7f399c0c4580 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f398c009840 tx=0x7f398c01c790 comp rx=0 tx=0).stop 2024-09-17T02:41:53.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 -- 172.21.15.131:0/4216799905 shutdown_connections 2024-09-17T02:41:53.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 --2- 172.21.15.131:0/4216799905 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399c06e2c0 0x7f399c0c4580 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 -- 172.21.15.131:0/4216799905 >> 172.21.15.131:0/4216799905 conn(0x7f399c06eea0 msgr2=0x7f399c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:53.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 -- 172.21.15.131:0/4216799905 shutdown_connections 2024-09-17T02:41:53.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 -- 172.21.15.131:0/4216799905 wait complete. 2024-09-17T02:41:53.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 Processor -- start 2024-09-17T02:41:53.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 -- start start 2024-09-17T02:41:53.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f399c06e2c0 0x7f399c0b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399c0c4ac0 0x7f399c0c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f399c0cec00 0x7f399c1521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f399c16b790 con 0x7f399c0c4ac0 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f399c16b610 con 0x7f399c06e2c0 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f39a2561640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f399c16b910 con 0x7f399c0cec00 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.397+0000 7f399b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399c0c4ac0 0x7f399c0c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f399b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399c0c4ac0 0x7f399c0c3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38110/0 (socket says 172.21.15.131:38110) 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f399b7fe640 1 -- 172.21.15.131:0/92068447 learned_addr learned my addr 172.21.15.131:0/92068447 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f399bfff640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f399c06e2c0 0x7f399c0b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f39a0afe640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f399c0cec00 0x7f399c1521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f399b7fe640 1 -- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f399c0cec00 msgr2=0x7f399c1521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f399b7fe640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f399c0cec00 0x7f399c1521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f399b7fe640 1 -- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f399c06e2c0 msgr2=0x7f399c0b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:53.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f399b7fe640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f399c06e2c0 0x7f399c0b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f399b7fe640 1 -- 172.21.15.131:0/92068447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f399c14dc60 con 0x7f399c0c4ac0 2024-09-17T02:41:53.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f399b7fe640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399c0c4ac0 0x7f399c0c3d10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f399000c3c0 tx=0x7f399000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:53.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f3998ff9640 1 -- 172.21.15.131:0/92068447 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990017070 con 0x7f399c0c4ac0 2024-09-17T02:41:53.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f3998ff9640 1 -- 172.21.15.131:0/92068447 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3990003600 con 0x7f399c0c4ac0 2024-09-17T02:41:53.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f3998ff9640 1 -- 172.21.15.131:0/92068447 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3990003a50 con 0x7f399c0c4ac0 2024-09-17T02:41:53.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f39a2561640 1 -- 172.21.15.131:0/92068447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f399c14df40 con 0x7f399c0c4ac0 2024-09-17T02:41:53.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f39a2561640 1 -- 172.21.15.131:0/92068447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f399c14e500 con 0x7f399c0c4ac0 2024-09-17T02:41:53.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f39a2561640 1 -- 172.21.15.131:0/92068447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f399c15c600 con 0x7f399c0c4ac0 2024-09-17T02:41:53.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f3998ff9640 1 -- 172.21.15.131:0/92068447 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f399001a070 con 0x7f399c0c4ac0 2024-09-17T02:41:53.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f3998ff9640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3988073200 0x7f3988075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:53.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f3998ff9640 1 -- 172.21.15.131:0/92068447 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7f39900932a0 con 0x7f399c0c4ac0 2024-09-17T02:41:53.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.401+0000 7f399bfff640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3988073200 0x7f3988075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:53.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.405+0000 7f399bfff640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3988073200 0x7f3988075710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f398c0af000 tx=0x7f398c01afd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:53.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.409+0000 7f3998ff9640 1 -- 172.21.15.131:0/92068447 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f399005e510 con 0x7f399c0c4ac0 2024-09-17T02:41:53.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.564+0000 7f39a2561640 1 -- 172.21.15.131:0/92068447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f399c073310 con 0x7f399c0c4ac0 2024-09-17T02:41:53.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.564+0000 7f3998ff9640 1 -- 172.21.15.131:0/92068447 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) ==== 84+0+65 (secure 0 0 0) 0x7f399005de10 con 0x7f399c0c4ac0 2024-09-17T02:41:53.568 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:53.568 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:41:53.569 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:41:53.569 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-09-17T02:41:53.569 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-09-17T02:41:53.569 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:41:53.569 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:41:53.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 -- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3988073200 msgr2=0x7f3988075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:53.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3988073200 0x7f3988075710 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f398c0af000 tx=0x7f398c01afd0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 -- 172.21.15.131:0/92068447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399c0c4ac0 msgr2=0x7f399c0c3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:53.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399c0c4ac0 0x7f399c0c3d10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f399000c3c0 tx=0x7f399000c880 comp rx=0 tx=0).stop 2024-09-17T02:41:53.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 -- 172.21.15.131:0/92068447 shutdown_connections 2024-09-17T02:41:53.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3988073200 0x7f3988075710 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f399c0cec00 0x7f399c1521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399c0c4ac0 0x7f399c0c3d10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 --2- 172.21.15.131:0/92068447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f399c06e2c0 0x7f399c0b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 -- 172.21.15.131:0/92068447 >> 172.21.15.131:0/92068447 conn(0x7f399c06eea0 msgr2=0x7f399c15bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:53.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 -- 172.21.15.131:0/92068447 shutdown_connections 2024-09-17T02:41:53.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.568+0000 7f39a2561640 1 -- 172.21.15.131:0/92068447 wait complete. 2024-09-17T02:41:53.589 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T02:41:53.589 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T02:41:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 -- 172.21.15.131:0/2800253326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecb41606d0 msgr2=0x7fecb40cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 --2- 172.21.15.131:0/2800253326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecb41606d0 0x7fecb40cf8f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7feca4009840 tx=0x7feca401c790 comp rx=0 tx=0).stop 2024-09-17T02:41:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 -- 172.21.15.131:0/2800253326 shutdown_connections 2024-09-17T02:41:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 --2- 172.21.15.131:0/2800253326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecb41606d0 0x7fecb40cf8f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 -- 172.21.15.131:0/2800253326 >> 172.21.15.131:0/2800253326 conn(0x7fecb40d05a0 msgr2=0x7fecb40d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 -- 172.21.15.131:0/2800253326 shutdown_connections 2024-09-17T02:41:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 -- 172.21.15.131:0/2800253326 wait complete. 2024-09-17T02:41:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 Processor -- start 2024-09-17T02:41:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 -- start start 2024-09-17T02:41:53.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb406f460 0x7fecb4159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:53.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fecb40cc870 0x7fecb4153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecb413a290 0x7fecb4140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fecb416ac90 con 0x7fecb413a290 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fecb416ab10 con 0x7fecb406f460 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecbb12e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fecb416ae10 con 0x7fecb40cc870 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecb96cb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecb413a290 0x7fecb4140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecb96cb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecb413a290 0x7fecb4140480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38134/0 (socket says 172.21.15.131:38134) 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.732+0000 7fecb96cb640 1 -- 172.21.15.131:0/2380985509 learned_addr learned my addr 172.21.15.131:0/2380985509 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecb8eca640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb406f460 0x7fecb4159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecabfff640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fecb40cc870 0x7fecb4153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecb96cb640 1 -- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fecb40cc870 msgr2=0x7fecb4153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecb96cb640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fecb40cc870 0x7fecb4153c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecb96cb640 1 -- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb406f460 msgr2=0x7fecb4159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecb96cb640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb406f460 0x7fecb4159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecb96cb640 1 -- 172.21.15.131:0/2380985509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb4159210 con 0x7fecb413a290 2024-09-17T02:41:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecabfff640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fecb40cc870 0x7fecb4153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecb8eca640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb406f460 0x7fecb4159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecb96cb640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecb413a290 0x7fecb4140480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fecac00b9e0 tx=0x7fecac00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7feca97fa640 1 -- 172.21.15.131:0/2380985509 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac003b80 con 0x7fecb413a290 2024-09-17T02:41:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7feca97fa640 1 -- 172.21.15.131:0/2380985509 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecac003e10 con 0x7fecb413a290 2024-09-17T02:41:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecbb12e640 1 -- 172.21.15.131:0/2380985509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb41594f0 con 0x7fecb413a290 2024-09-17T02:41:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7feca97fa640 1 -- 172.21.15.131:0/2380985509 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac00cb80 con 0x7fecb413a290 2024-09-17T02:41:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecbb12e640 1 -- 172.21.15.131:0/2380985509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb4154180 con 0x7fecb413a290 2024-09-17T02:41:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecbb12e640 1 -- 172.21.15.131:0/2380985509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb40cf8f0 con 0x7fecb413a290 2024-09-17T02:41:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7feca97fa640 1 -- 172.21.15.131:0/2380985509 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fecac006d10 con 0x7fecb413a290 2024-09-17T02:41:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7feca97fa640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feca0073200 0x7feca0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7feca97fa640 1 -- 172.21.15.131:0/2380985509 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5249+0+0 (secure 0 0 0) 0x7fecac093bf0 con 0x7fecb413a290 2024-09-17T02:41:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecb8eca640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feca0073200 0x7feca0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.736+0000 7fecb8eca640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feca0073200 0x7feca0075710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fecb0001630 tx=0x7fecb0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:53.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.740+0000 7feca97fa640 1 -- 172.21.15.131:0/2380985509 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fecac05edd0 con 0x7fecb413a290 2024-09-17T02:41:53.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:53.900+0000 7fecbb12e640 1 -- 172.21.15.131:0/2380985509 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fecb415d450 con 0x7fecb413a290 2024-09-17T02:41:54.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.796+0000 7feca97fa640 1 -- 172.21.15.131:0/2380985509 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) ==== 138+0+0 (secure 0 0 0) 0x7fecac05e6d0 con 0x7fecb413a290 2024-09-17T02:41:54.801 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-09-17T02:41:54.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 -- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feca0073200 msgr2=0x7feca0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:54.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feca0073200 0x7feca0075710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fecb0001630 tx=0x7fecb0009040 comp rx=0 tx=0).stop 2024-09-17T02:41:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 -- 172.21.15.131:0/2380985509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecb413a290 msgr2=0x7fecb4140480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecb413a290 0x7fecb4140480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fecac00b9e0 tx=0x7fecac00bea0 comp rx=0 tx=0).stop 2024-09-17T02:41:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 -- 172.21.15.131:0/2380985509 shutdown_connections 2024-09-17T02:41:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feca0073200 0x7feca0075710 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fecb413a290 0x7fecb4140480 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fecb40cc870 0x7fecb4153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 --2- 172.21.15.131:0/2380985509 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fecb406f460 0x7fecb4159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 -- 172.21.15.131:0/2380985509 >> 172.21.15.131:0/2380985509 conn(0x7fecb40d05a0 msgr2=0x7fecb415a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 -- 172.21.15.131:0/2380985509 shutdown_connections 2024-09-17T02:41:54.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:54.800+0000 7fecbb12e640 1 -- 172.21.15.131:0/2380985509 wait complete. 2024-09-17T02:41:54.834 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T02:41:55.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.000+0000 7ff4dc30b640 1 -- 172.21.15.131:0/23201978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d415eb50 msgr2=0x7ff4d4070210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:55.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.000+0000 7ff4dc30b640 1 --2- 172.21.15.131:0/23201978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d415eb50 0x7ff4d4070210 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4009900 tx=0x7ff4c401c790 comp rx=0 tx=0).stop 2024-09-17T02:41:55.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.000+0000 7ff4dc30b640 1 -- 172.21.15.131:0/23201978 shutdown_connections 2024-09-17T02:41:55.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.000+0000 7ff4dc30b640 1 --2- 172.21.15.131:0/23201978 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4d40af4c0 0x7ff4d40af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:55.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.000+0000 7ff4dc30b640 1 --2- 172.21.15.131:0/23201978 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d415eb50 0x7ff4d4070210 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:55.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.000+0000 7ff4dc30b640 1 -- 172.21.15.131:0/23201978 >> 172.21.15.131:0/23201978 conn(0x7ff4d40d0de0 msgr2=0x7ff4d40cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:55.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.000+0000 7ff4dc30b640 1 -- 172.21.15.131:0/23201978 shutdown_connections 2024-09-17T02:41:55.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.000+0000 7ff4dc30b640 1 -- 172.21.15.131:0/23201978 wait complete. 2024-09-17T02:41:55.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.000+0000 7ff4dc30b640 1 Processor -- start 2024-09-17T02:41:55.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.000+0000 7ff4dc30b640 1 -- start start 2024-09-17T02:41:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4dc30b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d406fe00 0x7ff4d41f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4dc30b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4d40af4c0 0x7ff4d4140f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4dc30b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4d40b5820 0x7ff4d40b5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4dc30b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d4168a20 con 0x7ff4d406fe00 2024-09-17T02:41:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4dc30b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d41688a0 con 0x7ff4d40b5820 2024-09-17T02:41:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4dc30b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d4168ba0 con 0x7ff4d40af4c0 2024-09-17T02:41:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4da0a7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d406fe00 0x7ff4d41f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4da0a7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d406fe00 0x7ff4d41f9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38148/0 (socket says 172.21.15.131:38148) 2024-09-17T02:41:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4da0a7640 1 -- 172.21.15.131:0/1038904417 learned_addr learned my addr 172.21.15.131:0/1038904417 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:55.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4da8a8640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4d40b5820 0x7ff4d40b5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:55.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4d98a6640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4d40af4c0 0x7ff4d4140f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:55.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4da0a7640 1 -- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4d40af4c0 msgr2=0x7ff4d4140f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:55.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4da0a7640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4d40af4c0 0x7ff4d4140f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:55.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4da0a7640 1 -- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4d40b5820 msgr2=0x7ff4d40b5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:55.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4da0a7640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4d40b5820 0x7ff4d40b5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:55.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4da0a7640 1 -- 172.21.15.131:0/1038904417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d40b6170 con 0x7ff4d406fe00 2024-09-17T02:41:55.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4da0a7640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d406fe00 0x7ff4d41f9a70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000c3c0 tx=0x7ff4d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:55.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4caffd640 1 -- 172.21.15.131:0/1038904417 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0017070 con 0x7ff4d406fe00 2024-09-17T02:41:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4caffd640 1 -- 172.21.15.131:0/1038904417 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4d00053c0 con 0x7ff4d406fe00 2024-09-17T02:41:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4dc30b640 1 -- 172.21.15.131:0/1038904417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d41fa530 con 0x7ff4d406fe00 2024-09-17T02:41:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4caffd640 1 -- 172.21.15.131:0/1038904417 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0003860 con 0x7ff4d406fe00 2024-09-17T02:41:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4dc30b640 1 -- 172.21.15.131:0/1038904417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d4155840 con 0x7ff4d406fe00 2024-09-17T02:41:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4dc30b640 1 -- 172.21.15.131:0/1038904417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d4155630 con 0x7ff4d406fe00 2024-09-17T02:41:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.004+0000 7ff4caffd640 1 -- 172.21.15.131:0/1038904417 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff4d001a070 con 0x7ff4d406fe00 2024-09-17T02:41:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.008+0000 7ff4caffd640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff4b8073230 0x7ff4b8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.008+0000 7ff4caffd640 1 -- 172.21.15.131:0/1038904417 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5619+0+0 (secure 0 0 0) 0x7ff4d00938f0 con 0x7ff4d406fe00 2024-09-17T02:41:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.008+0000 7ff4d98a6640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff4b8073230 0x7ff4b8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:55.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.008+0000 7ff4d98a6640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff4b8073230 0x7ff4b8075740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40af000 tx=0x7ff4c401afd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:55.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.012+0000 7ff4caffd640 1 -- 172.21.15.131:0/1038904417 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff4d0021020 con 0x7ff4d406fe00 2024-09-17T02:41:55.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.168+0000 7ff4dc30b640 1 -- 172.21.15.131:0/1038904417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff4d40af8d0 con 0x7ff4d406fe00 2024-09-17T02:41:55.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.804+0000 7ff4caffd640 1 -- 172.21.15.131:0/1038904417 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) ==== 170+0+0 (secure 0 0 0) 0x7ff4d40b6170 con 0x7ff4d406fe00 2024-09-17T02:41:55.806 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-09-17T02:41:55.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 -- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff4b8073230 msgr2=0x7ff4b8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:55.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff4b8073230 0x7ff4b8075740 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40af000 tx=0x7ff4c401afd0 comp rx=0 tx=0).stop 2024-09-17T02:41:55.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 -- 172.21.15.131:0/1038904417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d406fe00 msgr2=0x7ff4d41f9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:55.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d406fe00 0x7ff4d41f9a70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000c3c0 tx=0x7ff4d000c880 comp rx=0 tx=0).stop 2024-09-17T02:41:55.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 -- 172.21.15.131:0/1038904417 shutdown_connections 2024-09-17T02:41:55.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff4b8073230 0x7ff4b8075740 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:55.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4d40b5820 0x7ff4d40b5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:55.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4d40af4c0 0x7ff4d4140f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:55.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 --2- 172.21.15.131:0/1038904417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4d406fe00 0x7ff4d41f9a70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:55.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 -- 172.21.15.131:0/1038904417 >> 172.21.15.131:0/1038904417 conn(0x7ff4d40d0de0 msgr2=0x7ff4d415c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:55.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 -- 172.21.15.131:0/1038904417 shutdown_connections 2024-09-17T02:41:55.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:55.808+0000 7ff4dc30b640 1 -- 172.21.15.131:0/1038904417 wait complete. 2024-09-17T02:41:55.844 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T02:41:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.000+0000 7faac3600640 1 -- 172.21.15.131:0/3309248731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa8004810 msgr2=0x7faaa8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.000+0000 7faac3600640 1 --2- 172.21.15.131:0/3309248731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faaa8004810 0x7faaa8004c20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7faaac0016f0 tx=0x7faaac01c7b0 comp rx=0 tx=0).stop 2024-09-17T02:41:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.000+0000 7faac1b9d640 1 -- 172.21.15.131:0/3309248731 reap_dead start 2024-09-17T02:41:56.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.000+0000 7faac3600640 1 -- 172.21.15.131:0/3309248731 shutdown_connections 2024-09-17T02:41:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.000+0000 7faac3600640 1 -- 172.21.15.131:0/3309248731 >> 172.21.15.131:0/3309248731 conn(0x7faabc0d1b10 msgr2=0x7faabc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.000+0000 7faac3600640 1 -- 172.21.15.131:0/3309248731 shutdown_connections 2024-09-17T02:41:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.000+0000 7faac3600640 1 -- 172.21.15.131:0/3309248731 wait complete. 2024-09-17T02:41:56.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.000+0000 7faac3600640 1 Processor -- start 2024-09-17T02:41:56.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.000+0000 7faac3600640 1 -- start start 2024-09-17T02:41:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac3600640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faaa8004810 0x7faabc1f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac3600640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faabc1514e0 0x7faabc14ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac3600640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc14f3d0 0x7faabc14f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac3600640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faabc15fbe0 con 0x7faabc14f3d0 2024-09-17T02:41:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac3600640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faabc0cfc60 con 0x7faabc1514e0 2024-09-17T02:41:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac3600640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7faabc15fd60 con 0x7faaa8004810 2024-09-17T02:41:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac1b9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc14f3d0 0x7faabc14f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac1b9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc14f3d0 0x7faabc14f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38170/0 (socket says 172.21.15.131:38170) 2024-09-17T02:41:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac1b9d640 1 -- 172.21.15.131:0/3410931075 learned_addr learned my addr 172.21.15.131:0/3410931075 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac139c640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faaa8004810 0x7faabc1f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:56.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac0b9b640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faabc1514e0 0x7faabc14ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac1b9d640 1 -- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faaa8004810 msgr2=0x7faabc1f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac1b9d640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faaa8004810 0x7faabc1f6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac1b9d640 1 -- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faabc1514e0 msgr2=0x7faabc14ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac1b9d640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faabc1514e0 0x7faabc14ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac1b9d640 1 -- 172.21.15.131:0/3410931075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faabc14fd20 con 0x7faabc14f3d0 2024-09-17T02:41:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac1b9d640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc14f3d0 0x7faabc14f7e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7faab40013e0 tx=0x7faab400e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faab1ffb640 1 -- 172.21.15.131:0/3410931075 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4013480 con 0x7faabc14f3d0 2024-09-17T02:41:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faab1ffb640 1 -- 172.21.15.131:0/3410931075 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faab4013710 con 0x7faabc14f3d0 2024-09-17T02:41:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac3600640 1 -- 172.21.15.131:0/3410931075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faabc151c10 con 0x7faabc14f3d0 2024-09-17T02:41:56.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faab1ffb640 1 -- 172.21.15.131:0/3410931075 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4013b20 con 0x7faabc14f3d0 2024-09-17T02:41:56.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac3600640 1 -- 172.21.15.131:0/3410931075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faabc1521d0 con 0x7faabc14f3d0 2024-09-17T02:41:56.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faac3600640 1 -- 172.21.15.131:0/3410931075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faabc151980 con 0x7faabc14f3d0 2024-09-17T02:41:56.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.004+0000 7faab1ffb640 1 -- 172.21.15.131:0/3410931075 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faab4013cc0 con 0x7faabc14f3d0 2024-09-17T02:41:56.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.008+0000 7faab1ffb640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faaa0073200 0x7faaa0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.008+0000 7faab1ffb640 1 -- 172.21.15.131:0/3410931075 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6001+0+0 (secure 0 0 0) 0x7faab4098ef0 con 0x7faabc14f3d0 2024-09-17T02:41:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.008+0000 7faac139c640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faaa0073200 0x7faaa0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:56.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.008+0000 7faac139c640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faaa0073200 0x7faaa0075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faabc0b7610 tx=0x7faab80091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:56.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.012+0000 7faab1ffb640 1 -- 172.21.15.131:0/3410931075 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faabc14fd20 con 0x7faabc14f3d0 2024-09-17T02:41:56.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.224+0000 7faac3600640 1 -- 172.21.15.131:0/3410931075 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7faabc15f510 con 0x7faabc14f3d0 2024-09-17T02:41:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.824+0000 7faab1ffb640 1 -- 172.21.15.131:0/3410931075 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7faab4063ec0 con 0x7faabc14f3d0 2024-09-17T02:41:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 5 and data pool 6 2024-09-17T02:41:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.828+0000 7faac3600640 1 -- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faaa0073200 msgr2=0x7faaa0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.828+0000 7faac3600640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faaa0073200 0x7faaa0075710 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faabc0b7610 tx=0x7faab80091e0 comp rx=0 tx=0).stop 2024-09-17T02:41:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.828+0000 7faac3600640 1 -- 172.21.15.131:0/3410931075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc14f3d0 msgr2=0x7faabc14f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.828+0000 7faac3600640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc14f3d0 0x7faabc14f7e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7faab40013e0 tx=0x7faab400e650 comp rx=0 tx=0).stop 2024-09-17T02:41:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.828+0000 7faac3600640 1 -- 172.21.15.131:0/3410931075 shutdown_connections 2024-09-17T02:41:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.828+0000 7faac3600640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faaa0073200 0x7faaa0075710 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.828+0000 7faac3600640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faabc14f3d0 0x7faabc14f7e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.828+0000 7faac3600640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faabc1514e0 0x7faabc14ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.828+0000 7faac3600640 1 --2- 172.21.15.131:0/3410931075 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faaa8004810 0x7faabc1f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.828+0000 7faac3600640 1 -- 172.21.15.131:0/3410931075 >> 172.21.15.131:0/3410931075 conn(0x7faabc0d1b10 msgr2=0x7faabc161760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.832+0000 7faac3600640 1 -- 172.21.15.131:0/3410931075 shutdown_connections 2024-09-17T02:41:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:56.832+0000 7faac3600640 1 -- 172.21.15.131:0/3410931075 wait complete. 2024-09-17T02:41:56.880 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:41:57.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.028+0000 7fa86895d640 1 -- 172.21.15.131:0/1629401191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa86406e5b0 msgr2=0x7fa8640cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.028+0000 7fa86895d640 1 --2- 172.21.15.131:0/1629401191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa86406e5b0 0x7fa8640cab60 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fa8640cdb80 tx=0x7fa85c01cd00 comp rx=0 tx=0).stop 2024-09-17T02:41:57.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.028+0000 7fa86895d640 1 -- 172.21.15.131:0/1629401191 shutdown_connections 2024-09-17T02:41:57.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.028+0000 7fa86895d640 1 --2- 172.21.15.131:0/1629401191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa86406e5b0 0x7fa8640cab60 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.028+0000 7fa86895d640 1 --2- 172.21.15.131:0/1629401191 >> v2:172.21.15.189:3300/0 conn(0x7fa864155240 0x7fa864073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.028+0000 7fa86895d640 1 -- 172.21.15.131:0/1629401191 >> 172.21.15.131:0/1629401191 conn(0x7fa86406eea0 msgr2=0x7fa86406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:57.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.028+0000 7fa86895d640 1 -- 172.21.15.131:0/1629401191 shutdown_connections 2024-09-17T02:41:57.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.028+0000 7fa86895d640 1 -- 172.21.15.131:0/1629401191 wait complete. 2024-09-17T02:41:57.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86895d640 1 Processor -- start 2024-09-17T02:41:57.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86895d640 1 -- start start 2024-09-17T02:41:57.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86895d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa864073970 0x7fa8641f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86895d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa864155240 0x7fa86414de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86895d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa86414eb30 0x7fa864150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86895d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8640cbe20 con 0x7fa86414eb30 2024-09-17T02:41:57.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86895d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8640cbca0 con 0x7fa864073970 2024-09-17T02:41:57.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86895d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa864168c80 con 0x7fa864155240 2024-09-17T02:41:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa862d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa86414eb30 0x7fa864150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa862d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa86414eb30 0x7fa864150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38184/0 (socket says 172.21.15.131:38184) 2024-09-17T02:41:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa862d9d640 1 -- 172.21.15.131:0/761224234 learned_addr learned my addr 172.21.15.131:0/761224234 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86259c640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa864073970 0x7fa8641f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa862d9d640 1 -- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa864155240 msgr2=0x7fa86414de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa862d9d640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa864155240 0x7fa86414de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa862d9d640 1 -- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa864073970 msgr2=0x7fa8641f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa862d9d640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa864073970 0x7fa8641f5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa862d9d640 1 -- 172.21.15.131:0/761224234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86414e3d0 con 0x7fa86414eb30 2024-09-17T02:41:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86259c640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa864073970 0x7fa8641f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:57.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa862d9d640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa86414eb30 0x7fa864150960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009bd0 tx=0x7fa85c009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa852ffd640 1 -- 172.21.15.131:0/761224234 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0bc070 con 0x7fa86414eb30 2024-09-17T02:41:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa852ffd640 1 -- 172.21.15.131:0/761224234 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa85c0ad8e0 con 0x7fa86414eb30 2024-09-17T02:41:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86895d640 1 -- 172.21.15.131:0/761224234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8640c95e0 con 0x7fa86414eb30 2024-09-17T02:41:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa852ffd640 1 -- 172.21.15.131:0/761224234 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c0b2040 con 0x7fa86414eb30 2024-09-17T02:41:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86895d640 1 -- 172.21.15.131:0/761224234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa864150160 con 0x7fa86414eb30 2024-09-17T02:41:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.032+0000 7fa86895d640 1 -- 172.21.15.131:0/761224234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8640cab60 con 0x7fa86414eb30 2024-09-17T02:41:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.036+0000 7fa852ffd640 1 -- 172.21.15.131:0/761224234 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa85c0bf070 con 0x7fa86414eb30 2024-09-17T02:41:57.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.036+0000 7fa852ffd640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa844073200 0x7fa844075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.036+0000 7fa852ffd640 1 -- 172.21.15.131:0/761224234 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa85c12e2f0 con 0x7fa86414eb30 2024-09-17T02:41:57.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.036+0000 7fa86259c640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa844073200 0x7fa844075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:57.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.040+0000 7fa86259c640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa844073200 0x7fa844075710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa854005fd0 tx=0x7fa854005d00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.040+0000 7fa852ffd640 1 -- 172.21.15.131:0/761224234 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa85c0168f0 con 0x7fa86414eb30 2024-09-17T02:41:57.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.200+0000 7fa86895d640 1 -- 172.21.15.131:0/761224234 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa86414e920 con 0x7fa86414eb30 2024-09-17T02:41:57.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa852ffd640 1 -- 172.21.15.131:0/761224234 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17063 (secure 0 0 0) 0x7fa85c0f92f0 con 0x7fa86414eb30 2024-09-17T02:41:57.205 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:57.205 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":39,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:41:56.228032+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:41:53.913442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-17T02:41:55.184999+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:41:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 -- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa844073200 msgr2=0x7fa844075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa844073200 0x7fa844075710 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa854005fd0 tx=0x7fa854005d00 comp rx=0 tx=0).stop 2024-09-17T02:41:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 -- 172.21.15.131:0/761224234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa86414eb30 msgr2=0x7fa864150960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa86414eb30 0x7fa864150960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009bd0 tx=0x7fa85c009340 comp rx=0 tx=0).stop 2024-09-17T02:41:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 -- 172.21.15.131:0/761224234 shutdown_connections 2024-09-17T02:41:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa844073200 0x7fa844075710 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa86414eb30 0x7fa864150960 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa864155240 0x7fa86414de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 --2- 172.21.15.131:0/761224234 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa864073970 0x7fa8641f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 -- 172.21.15.131:0/761224234 >> 172.21.15.131:0/761224234 conn(0x7fa86406eea0 msgr2=0x7fa8640b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 -- 172.21.15.131:0/761224234 shutdown_connections 2024-09-17T02:41:57.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.204+0000 7fa86895d640 1 -- 172.21.15.131:0/761224234 wait complete. 2024-09-17T02:41:57.225 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:41:57.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.364+0000 7f41d7c26640 1 -- 172.21.15.131:0/3981926598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41d00cbb70 msgr2=0x7f41d0165b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.364+0000 7f41d7c26640 1 --2- 172.21.15.131:0/3981926598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41d00cbb70 0x7f41d0165b60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f41c8009bc0 tx=0x7f41c801ca30 comp rx=0 tx=0).stop 2024-09-17T02:41:57.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.364+0000 7f41d7c26640 1 -- 172.21.15.131:0/3981926598 shutdown_connections 2024-09-17T02:41:57.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.364+0000 7f41d7c26640 1 --2- 172.21.15.131:0/3981926598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41d00cbb70 0x7f41d0165b60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.364+0000 7f41d7c26640 1 -- 172.21.15.131:0/3981926598 >> 172.21.15.131:0/3981926598 conn(0x7f41d006eea0 msgr2=0x7f41d006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:57.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.364+0000 7f41d7c26640 1 -- 172.21.15.131:0/3981926598 shutdown_connections 2024-09-17T02:41:57.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.364+0000 7f41d7c26640 1 -- 172.21.15.131:0/3981926598 wait complete. 2024-09-17T02:41:57.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.364+0000 7f41d7c26640 1 Processor -- start 2024-09-17T02:41:57.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.364+0000 7f41d7c26640 1 -- start start 2024-09-17T02:41:57.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d7c26640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f41d006dea0 0x7f41d00c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d7c26640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41d00cbb70 0x7f41d0149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d7c26640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41d0155240 0x7f41d0166380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d7c26640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f41d006ff40 con 0x7f41d00cbb70 2024-09-17T02:41:57.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d7c26640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f41d006fdc0 con 0x7f41d0155240 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d7c26640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f41d0166f20 con 0x7f41d006dea0 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d51c1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41d00cbb70 0x7f41d0149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d51c1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41d00cbb70 0x7f41d0149f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38208/0 (socket says 172.21.15.131:38208) 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d51c1640 1 -- 172.21.15.131:0/2086277544 learned_addr learned my addr 172.21.15.131:0/2086277544 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d59c2640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f41d006dea0 0x7f41d00c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d51c1640 1 -- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f41d006dea0 msgr2=0x7f41d00c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d51c1640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f41d006dea0 0x7f41d00c9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d61c3640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41d0155240 0x7f41d0166380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d51c1640 1 -- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41d0155240 msgr2=0x7f41d0166380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d51c1640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41d0155240 0x7f41d0166380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d51c1640 1 -- 172.21.15.131:0/2086277544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41d0168fe0 con 0x7f41d00cbb70 2024-09-17T02:41:57.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d51c1640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41d00cbb70 0x7f41d0149f70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00fb20 tx=0x7f41cc004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:57.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41c67fc640 1 -- 172.21.15.131:0/2086277544 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc01a070 con 0x7f41d00cbb70 2024-09-17T02:41:57.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41c67fc640 1 -- 172.21.15.131:0/2086277544 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41cc008590 con 0x7f41d00cbb70 2024-09-17T02:41:57.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d7c26640 1 -- 172.21.15.131:0/2086277544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41d00c9090 con 0x7f41d00cbb70 2024-09-17T02:41:57.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41c67fc640 1 -- 172.21.15.131:0/2086277544 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41cc006980 con 0x7f41d00cbb70 2024-09-17T02:41:57.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d7c26640 1 -- 172.21.15.131:0/2086277544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41d0168d00 con 0x7f41d00cbb70 2024-09-17T02:41:57.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.368+0000 7f41d7c26640 1 -- 172.21.15.131:0/2086277544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41d0073430 con 0x7f41d00cbb70 2024-09-17T02:41:57.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.372+0000 7f41c67fc640 1 -- 172.21.15.131:0/2086277544 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f41cc01d070 con 0x7f41d00cbb70 2024-09-17T02:41:57.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.372+0000 7f41c67fc640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f41b4073230 0x7f41b4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.372+0000 7f41c67fc640 1 -- 172.21.15.131:0/2086277544 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f41cc096c90 con 0x7f41d00cbb70 2024-09-17T02:41:57.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.372+0000 7f41d59c2640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f41b4073230 0x7f41b4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:57.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.372+0000 7f41d59c2640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f41b4073230 0x7f41b4075740 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41c000a750 tx=0x7f41c0008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:57.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.376+0000 7f41c67fc640 1 -- 172.21.15.131:0/2086277544 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f41cc061bb0 con 0x7f41d00cbb70 2024-09-17T02:41:57.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.536+0000 7f41d7c26640 1 -- 172.21.15.131:0/2086277544 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41d006f490 con 0x7f41d00cbb70 2024-09-17T02:41:57.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41c67fc640 1 -- 172.21.15.131:0/2086277544 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17063 (secure 0 0 0) 0x7f41cc0614b0 con 0x7f41d00cbb70 2024-09-17T02:41:57.541 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:41:57.541 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":39,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:41:56.228032+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:41:53.913442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-17T02:41:55.184999+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:41:57.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 -- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f41b4073230 msgr2=0x7f41b4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f41b4073230 0x7f41b4075740 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f41c000a750 tx=0x7f41c0008040 comp rx=0 tx=0).stop 2024-09-17T02:41:57.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 -- 172.21.15.131:0/2086277544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41d00cbb70 msgr2=0x7f41d0149f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41d00cbb70 0x7f41d0149f70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00fb20 tx=0x7f41cc004560 comp rx=0 tx=0).stop 2024-09-17T02:41:57.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 -- 172.21.15.131:0/2086277544 shutdown_connections 2024-09-17T02:41:57.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f41b4073230 0x7f41b4075740 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f41d0155240 0x7f41d0166380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f41d00cbb70 0x7f41d0149f70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 --2- 172.21.15.131:0/2086277544 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f41d006dea0 0x7f41d00c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 -- 172.21.15.131:0/2086277544 >> 172.21.15.131:0/2086277544 conn(0x7f41d006eea0 msgr2=0x7f41d00ce7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:57.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 -- 172.21.15.131:0/2086277544 shutdown_connections 2024-09-17T02:41:57.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.540+0000 7f41d7c26640 1 -- 172.21.15.131:0/2086277544 wait complete. 2024-09-17T02:41:57.561 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T02:41:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.704+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/1999317885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb6815c5b0 msgr2=0x7ffb680cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.704+0000 7ffb6d3c3640 1 --2- 172.21.15.131:0/1999317885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb6815c5b0 0x7ffb680cbaf0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009bc0 tx=0x7ffb6001cba0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.704+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/1999317885 shutdown_connections 2024-09-17T02:41:57.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.704+0000 7ffb6d3c3640 1 --2- 172.21.15.131:0/1999317885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb6815c5b0 0x7ffb680cbaf0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.704+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/1999317885 >> 172.21.15.131:0/1999317885 conn(0x7ffb6806eea0 msgr2=0x7ffb6806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.704+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/1999317885 shutdown_connections 2024-09-17T02:41:57.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.704+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/1999317885 wait complete. 2024-09-17T02:41:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb6d3c3640 1 Processor -- start 2024-09-17T02:41:57.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb6d3c3640 1 -- start start 2024-09-17T02:41:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb6d3c3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb680ced00 0x7ffb68147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb6d3c3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb6815c5b0 0x7ffb680b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb6d3c3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb6815ccc0 0x7ffb680b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb6d3c3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffb68169ad0 con 0x7ffb680ced00 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb6d3c3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffb68169950 con 0x7ffb6815c5b0 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb6d3c3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ffb68169c50 con 0x7ffb6815ccc0 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb66ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb680ced00 0x7ffb68147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb66ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb680ced00 0x7ffb68147d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38232/0 (socket says 172.21.15.131:38232) 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb66ffd640 1 -- 172.21.15.131:0/235502015 learned_addr learned my addr 172.21.15.131:0/235502015 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb667fc640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb6815c5b0 0x7ffb680b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb677fe640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb6815ccc0 0x7ffb680b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb66ffd640 1 -- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb6815ccc0 msgr2=0x7ffb680b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb66ffd640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb6815ccc0 0x7ffb680b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb66ffd640 1 -- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb6815c5b0 msgr2=0x7ffb680b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb66ffd640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb6815c5b0 0x7ffb680b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb66ffd640 1 -- 172.21.15.131:0/235502015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb680b3b90 con 0x7ffb680ced00 2024-09-17T02:41:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb677fe640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb6815ccc0 0x7ffb680b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb66ffd640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb680ced00 0x7ffb68147d20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800c430 tx=0x7ffb5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb4f7fe640 1 -- 172.21.15.131:0/235502015 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58011790 con 0x7ffb680ced00 2024-09-17T02:41:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb4f7fe640 1 -- 172.21.15.131:0/235502015 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb58017070 con 0x7ffb680ced00 2024-09-17T02:41:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/235502015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb681f6ad0 con 0x7ffb680ced00 2024-09-17T02:41:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb4f7fe640 1 -- 172.21.15.131:0/235502015 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb58011cc0 con 0x7ffb680ced00 2024-09-17T02:41:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/235502015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb681f7090 con 0x7ffb680ced00 2024-09-17T02:41:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.708+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/235502015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb68075b80 con 0x7ffb680ced00 2024-09-17T02:41:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.712+0000 7ffb4f7fe640 1 -- 172.21.15.131:0/235502015 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffb5801c460 con 0x7ffb680ced00 2024-09-17T02:41:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.712+0000 7ffb4f7fe640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffb54073200 0x7ffb54075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.712+0000 7ffb4f7fe640 1 -- 172.21.15.131:0/235502015 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ffb58094b50 con 0x7ffb680ced00 2024-09-17T02:41:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.712+0000 7ffb667fc640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffb54073200 0x7ffb54075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:57.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.712+0000 7ffb667fc640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffb54073200 0x7ffb54075710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffb68152290 tx=0x7ffb5c008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:57.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.716+0000 7ffb4f7fe640 1 -- 172.21.15.131:0/235502015 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffb6815c9c0 con 0x7ffb680ced00 2024-09-17T02:41:57.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:57.916+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/235502015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ffb680cbaf0 con 0x7ffb680ced00 2024-09-17T02:41:58.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.868+0000 7ffb4f7fe640 1 -- 172.21.15.131:0/235502015 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7ffb5805fb50 con 0x7ffb680ced00 2024-09-17T02:41:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffb54073200 msgr2=0x7ffb54075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffb54073200 0x7ffb54075710 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffb68152290 tx=0x7ffb5c008040 comp rx=0 tx=0).stop 2024-09-17T02:41:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/235502015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb680ced00 msgr2=0x7ffb68147d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb680ced00 0x7ffb68147d20 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800c430 tx=0x7ffb5800c8f0 comp rx=0 tx=0).stop 2024-09-17T02:41:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/235502015 shutdown_connections 2024-09-17T02:41:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffb54073200 0x7ffb54075710 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffb6815ccc0 0x7ffb680b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffb6815c5b0 0x7ffb680b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 --2- 172.21.15.131:0/235502015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffb680ced00 0x7ffb68147d20 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/235502015 >> 172.21.15.131:0/235502015 conn(0x7ffb6806eea0 msgr2=0x7ffb68151230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/235502015 shutdown_connections 2024-09-17T02:41:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:58.872+0000 7ffb6d3c3640 1 -- 172.21.15.131:0/235502015 wait complete. 2024-09-17T02:41:58.899 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T02:41:58.899 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T02:41:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.056+0000 7fe469b60640 1 -- 172.21.15.131:0/352487693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe46406e3e0 msgr2=0x7fe46415e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:59.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.056+0000 7fe469b60640 1 --2- 172.21.15.131:0/352487693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe46406e3e0 0x7fe46415e7d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe4640757f0 tx=0x7fe45401cd50 comp rx=0 tx=0).stop 2024-09-17T02:41:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.056+0000 7fe469b60640 1 -- 172.21.15.131:0/352487693 shutdown_connections 2024-09-17T02:41:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.056+0000 7fe469b60640 1 --2- 172.21.15.131:0/352487693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe46406e3e0 0x7fe46415e7d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.056+0000 7fe469b60640 1 --2- 172.21.15.131:0/352487693 >> v2:172.21.15.189:3300/0 conn(0x7fe464073970 0x7fe46406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.056+0000 7fe469b60640 1 --2- 172.21.15.131:0/352487693 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe464155240 0x7fe464073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.056+0000 7fe469b60640 1 -- 172.21.15.131:0/352487693 >> 172.21.15.131:0/352487693 conn(0x7fe46406eea0 msgr2=0x7fe46406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.056+0000 7fe469b60640 1 -- 172.21.15.131:0/352487693 shutdown_connections 2024-09-17T02:41:59.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.056+0000 7fe469b60640 1 -- 172.21.15.131:0/352487693 wait complete. 2024-09-17T02:41:59.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe469b60640 1 Processor -- start 2024-09-17T02:41:59.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe469b60640 1 -- start start 2024-09-17T02:41:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe469b60640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe46406e3e0 0x7fe4641f9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe469b60640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe464073970 0x7fe4641544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe469b60640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe464155240 0x7fe464162f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:59.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe469b60640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe4640cace0 con 0x7fe464155240 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe469b60640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe4640cab60 con 0x7fe464073970 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe469b60640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe4640cae60 con 0x7fe46406e3e0 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe463fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe464155240 0x7fe464162f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe463fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe464155240 0x7fe464162f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38264/0 (socket says 172.21.15.131:38264) 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe463fff640 1 -- 172.21.15.131:0/655878080 learned_addr learned my addr 172.21.15.131:0/655878080 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe462ffd640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe464073970 0x7fe4641544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe4637fe640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe46406e3e0 0x7fe4641f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe463fff640 1 -- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe46406e3e0 msgr2=0x7fe4641f9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe463fff640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe46406e3e0 0x7fe4641f9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe463fff640 1 -- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe464073970 msgr2=0x7fe4641544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe463fff640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe464073970 0x7fe4641544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe463fff640 1 -- 172.21.15.131:0/655878080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4641fa060 con 0x7fe464155240 2024-09-17T02:41:59.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe462ffd640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe464073970 0x7fe4641544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:41:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe463fff640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe464155240 0x7fe464162f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe45401b400 tx=0x7fe454005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe45bfff640 1 -- 172.21.15.131:0/655878080 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4540bb070 con 0x7fe464155240 2024-09-17T02:41:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe45bfff640 1 -- 172.21.15.131:0/655878080 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe454004d40 con 0x7fe464155240 2024-09-17T02:41:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe45bfff640 1 -- 172.21.15.131:0/655878080 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4540b2040 con 0x7fe464155240 2024-09-17T02:41:59.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe469b60640 1 -- 172.21.15.131:0/655878080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4641fa340 con 0x7fe464155240 2024-09-17T02:41:59.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.060+0000 7fe469b60640 1 -- 172.21.15.131:0/655878080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4640b3370 con 0x7fe464155240 2024-09-17T02:41:59.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.064+0000 7fe469b60640 1 -- 172.21.15.131:0/655878080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4641f9dd0 con 0x7fe464155240 2024-09-17T02:41:59.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.064+0000 7fe45bfff640 1 -- 172.21.15.131:0/655878080 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe454004240 con 0x7fe464155240 2024-09-17T02:41:59.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.064+0000 7fe45bfff640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe444073230 0x7fe444075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:41:59.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.064+0000 7fe45bfff640 1 -- 172.21.15.131:0/655878080 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fe45412e2d0 con 0x7fe464155240 2024-09-17T02:41:59.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.064+0000 7fe4637fe640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe444073230 0x7fe444075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:41:59.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.064+0000 7fe4637fe640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe444073230 0x7fe444075740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0058d0 tx=0x7fe45c005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:41:59.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.068+0000 7fe45bfff640 1 -- 172.21.15.131:0/655878080 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe454016970 con 0x7fe464155240 2024-09-17T02:41:59.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.272+0000 7fe469b60640 1 -- 172.21.15.131:0/655878080 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe46406dea0 con 0x7fe464155240 2024-09-17T02:41:59.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe45bfff640 1 -- 172.21.15.131:0/655878080 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7fe4540f90e0 con 0x7fe464155240 2024-09-17T02:41:59.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 -- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe444073230 msgr2=0x7fe444075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:59.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe444073230 0x7fe444075740 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe45c0058d0 tx=0x7fe45c005860 comp rx=0 tx=0).stop 2024-09-17T02:41:59.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 -- 172.21.15.131:0/655878080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe464155240 msgr2=0x7fe464162f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:41:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe464155240 0x7fe464162f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe45401b400 tx=0x7fe454005ae0 comp rx=0 tx=0).stop 2024-09-17T02:41:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 -- 172.21.15.131:0/655878080 shutdown_connections 2024-09-17T02:41:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe444073230 0x7fe444075740 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe464155240 0x7fe464162f60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe464073970 0x7fe4641544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 --2- 172.21.15.131:0/655878080 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe46406e3e0 0x7fe4641f9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:41:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 -- 172.21.15.131:0/655878080 >> 172.21.15.131:0/655878080 conn(0x7fe46406eea0 msgr2=0x7fe4640cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:41:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 -- 172.21.15.131:0/655878080 shutdown_connections 2024-09-17T02:41:59.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:41:59.880+0000 7fe469b60640 1 -- 172.21.15.131:0/655878080 wait complete. 2024-09-17T02:41:59.906 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T02:41:59.907 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.060+0000 7f697e2e8640 1 -- 172.21.15.131:0/3971757675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697815c5b0 msgr2=0x7f69780cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.060+0000 7f697e2e8640 1 --2- 172.21.15.131:0/3971757675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697815c5b0 0x7f69780cbaf0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f697806a1c0 tx=0x7f697001cba0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.060+0000 7f697e2e8640 1 -- 172.21.15.131:0/3971757675 shutdown_connections 2024-09-17T02:42:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.060+0000 7f697e2e8640 1 --2- 172.21.15.131:0/3971757675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f697815c5b0 0x7f69780cbaf0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.060+0000 7f697e2e8640 1 --2- 172.21.15.131:0/3971757675 >> v2:172.21.15.189:3300/0 conn(0x7f69780ced00 0x7f697806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.060+0000 7f697e2e8640 1 -- 172.21.15.131:0/3971757675 >> 172.21.15.131:0/3971757675 conn(0x7f697806eea0 msgr2=0x7f697806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.060+0000 7f697e2e8640 1 -- 172.21.15.131:0/3971757675 shutdown_connections 2024-09-17T02:42:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.060+0000 7f697e2e8640 1 -- 172.21.15.131:0/3971757675 wait complete. 2024-09-17T02:42:00.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.060+0000 7f697e2e8640 1 Processor -- start 2024-09-17T02:42:00.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.060+0000 7f697e2e8640 1 -- start start 2024-09-17T02:42:00.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f697e2e8640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f697806e3e0 0x7f697814c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f697e2e8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69780ced00 0x7f69780cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f697e2e8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f697815c5b0 0x7f69780ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f697e2e8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6978168720 con 0x7f69780ced00 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f697e2e8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f69781685a0 con 0x7f697815c5b0 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f697e2e8640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f69781688a0 con 0x7f697806e3e0 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f696ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69780ced00 0x7f69780cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f696ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69780ced00 0x7f69780cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38280/0 (socket says 172.21.15.131:38280) 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f696ffff640 1 -- 172.21.15.131:0/2454397388 learned_addr learned my addr 172.21.15.131:0/2454397388 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f697c885640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f697815c5b0 0x7f69780ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f6977fff640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f697806e3e0 0x7f697814c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f696ffff640 1 -- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f697806e3e0 msgr2=0x7f697814c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f696ffff640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f697806e3e0 0x7f697814c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f696ffff640 1 -- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f697815c5b0 msgr2=0x7f69780ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f696ffff640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f697815c5b0 0x7f69780ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f696ffff640 1 -- 172.21.15.131:0/2454397388 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69781fba90 con 0x7f69780ced00 2024-09-17T02:42:00.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f696ffff640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69780ced00 0x7f69780cdbf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f696000b470 tx=0x7f696000b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:00.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f69757fa640 1 -- 172.21.15.131:0/2454397388 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960015030 con 0x7f69780ced00 2024-09-17T02:42:00.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f69757fa640 1 -- 172.21.15.131:0/2454397388 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6960004020 con 0x7f69780ced00 2024-09-17T02:42:00.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f697e2e8640 1 -- 172.21.15.131:0/2454397388 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69781fbd70 con 0x7f69780ced00 2024-09-17T02:42:00.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f69757fa640 1 -- 172.21.15.131:0/2454397388 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960004470 con 0x7f69780ced00 2024-09-17T02:42:00.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f697e2e8640 1 -- 172.21.15.131:0/2454397388 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69781fc330 con 0x7f69780ced00 2024-09-17T02:42:00.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f697e2e8640 1 -- 172.21.15.131:0/2454397388 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697806dea0 con 0x7f69780ced00 2024-09-17T02:42:00.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.064+0000 7f69757fa640 1 -- 172.21.15.131:0/2454397388 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6960018070 con 0x7f69780ced00 2024-09-17T02:42:00.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.068+0000 7f69757fa640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6958073230 0x7f6958075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.068+0000 7f69757fa640 1 -- 172.21.15.131:0/2454397388 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f6960094140 con 0x7f69780ced00 2024-09-17T02:42:00.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.068+0000 7f6977fff640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6958073230 0x7f6958075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:00.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.068+0000 7f6977fff640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6958073230 0x7f6958075740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f69680058d0 tx=0x7f6968005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:00.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.072+0000 7f69757fa640 1 -- 172.21.15.131:0/2454397388 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f696000d040 con 0x7f69780ced00 2024-09-17T02:42:00.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 -- 172.21.15.131:0/2454397388 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f697806f5f0 con 0x7f69780ced00 2024-09-17T02:42:00.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f69757fa640 1 -- 172.21.15.131:0/2454397388 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7f696005efd0 con 0x7f69780ced00 2024-09-17T02:42:00.278 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:00.278 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":18,"btime":"2024-09-17T02:41:59:873356+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3342719482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3342719482},{"type":"v1","addr":"172.21.15.131:6835","nonce":3342719482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/3070378350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3070378350},{"type":"v1","addr":"172.21.15.189:6836","nonce":3070378350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/575920659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":575920659},{"type":"v1","addr":"172.21.15.189:6837","nonce":575920659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:41:56.227947+0000","modified":"2024-09-17T02:41:59.279613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6833/1415478569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1415478569},{"type":"v1","addr":"172.21.15.131:6833","nonce":1415478569}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-09-17T02:42:00.278 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 18 2024-09-17T02:42:00.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 -- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6958073230 msgr2=0x7f6958075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6958073230 0x7f6958075740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f69680058d0 tx=0x7f6968005860 comp rx=0 tx=0).stop 2024-09-17T02:42:00.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 -- 172.21.15.131:0/2454397388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69780ced00 msgr2=0x7f69780cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69780ced00 0x7f69780cdbf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f696000b470 tx=0x7f696000b930 comp rx=0 tx=0).stop 2024-09-17T02:42:00.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 -- 172.21.15.131:0/2454397388 shutdown_connections 2024-09-17T02:42:00.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6958073230 0x7f6958075740 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f697815c5b0 0x7f69780ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f69780ced00 0x7f69780cdbf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 --2- 172.21.15.131:0/2454397388 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f697806e3e0 0x7f697814c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 -- 172.21.15.131:0/2454397388 >> 172.21.15.131:0/2454397388 conn(0x7f697806eea0 msgr2=0x7f69780b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:00.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 -- 172.21.15.131:0/2454397388 shutdown_connections 2024-09-17T02:42:00.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.276+0000 7f697e2e8640 1 -- 172.21.15.131:0/2454397388 wait complete. 2024-09-17T02:42:00.300 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:42:00.329 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-09-17T02:42:00.328+0000 7fcb6c222640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:42:00.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.452+0000 7fdcabedd640 1 -- 172.21.15.131:0/1758406561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cf710 msgr2=0x7fdca415f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.452+0000 7fdcabedd640 1 --2- 172.21.15.131:0/1758406561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cf710 0x7fdca415f290 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdca0009840 tx=0x7fdca001c820 comp rx=0 tx=0).stop 2024-09-17T02:42:00.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.452+0000 7fdcabedd640 1 -- 172.21.15.131:0/1758406561 shutdown_connections 2024-09-17T02:42:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.452+0000 7fdcabedd640 1 --2- 172.21.15.131:0/1758406561 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdca415f7d0 0x7fdca4162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.452+0000 7fdcabedd640 1 --2- 172.21.15.131:0/1758406561 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca40cf710 0x7fdca415f290 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.452+0000 7fdcabedd640 1 -- 172.21.15.131:0/1758406561 >> 172.21.15.131:0/1758406561 conn(0x7fdca40d1b10 msgr2=0x7fdca40d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.452+0000 7fdcabedd640 1 -- 172.21.15.131:0/1758406561 shutdown_connections 2024-09-17T02:42:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.452+0000 7fdcabedd640 1 -- 172.21.15.131:0/1758406561 wait complete. 2024-09-17T02:42:00.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdcabedd640 1 Processor -- start 2024-09-17T02:42:00.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdcabedd640 1 -- start start 2024-09-17T02:42:00.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdcabedd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdca40cf710 0x7fdca414ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdcabedd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca415f7d0 0x7fdca4149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdcabedd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdca4162950 0x7fdca414a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdcabedd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdca40d3380 con 0x7fdca415f7d0 2024-09-17T02:42:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdcabedd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdca415fdf0 con 0x7fdca40cf710 2024-09-17T02:42:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdcabedd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdca40d3500 con 0x7fdca4162950 2024-09-17T02:42:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdca9478640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca415f7d0 0x7fdca4149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:00.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdca9478640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca415f7d0 0x7fdca4149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38300/0 (socket says 172.21.15.131:38300) 2024-09-17T02:42:00.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdca9478640 1 -- 172.21.15.131:0/3260335717 learned_addr learned my addr 172.21.15.131:0/3260335717 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:00.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdcaa47a640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdca4162950 0x7fdca414a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:00.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdca9c79640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdca40cf710 0x7fdca414ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:00.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdca9478640 1 -- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdca4162950 msgr2=0x7fdca414a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdca9478640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdca4162950 0x7fdca414a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdca9478640 1 -- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdca40cf710 msgr2=0x7fdca414ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdca9478640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdca40cf710 0x7fdca414ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdca9478640 1 -- 172.21.15.131:0/3260335717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca4152210 con 0x7fdca415f7d0 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdca9c79640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdca40cf710 0x7fdca414ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdca9478640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca415f7d0 0x7fdca4149780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdc940058d0 tx=0x7fdc9400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdc9a7fc640 1 -- 172.21.15.131:0/3260335717 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94017070 con 0x7fdca415f7d0 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdc9a7fc640 1 -- 172.21.15.131:0/3260335717 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc94009d20 con 0x7fdca415f7d0 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdcabedd640 1 -- 172.21.15.131:0/3260335717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca4151160 con 0x7fdca415f7d0 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdc9a7fc640 1 -- 172.21.15.131:0/3260335717 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc940106c0 con 0x7fdca415f7d0 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.456+0000 7fdcabedd640 1 -- 172.21.15.131:0/3260335717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca4151720 con 0x7fdca415f7d0 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.460+0000 7fdcabedd640 1 -- 172.21.15.131:0/3260335717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca4151f80 con 0x7fdca415f7d0 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.460+0000 7fdc9a7fc640 1 -- 172.21.15.131:0/3260335717 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdc9401a070 con 0x7fdca415f7d0 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.460+0000 7fdc9a7fc640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdc88073230 0x7fdc88075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.460+0000 7fdc9a7fc640 1 -- 172.21.15.131:0/3260335717 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fdc94094560 con 0x7fdca415f7d0 2024-09-17T02:42:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.460+0000 7fdca9c79640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdc88073230 0x7fdc88075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:00.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.460+0000 7fdca9c79640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdc88073230 0x7fdc88075740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdca00af000 tx=0x7fdca001b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:00.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.464+0000 7fdc9a7fc640 1 -- 172.21.15.131:0/3260335717 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdc9405f370 con 0x7fdca415f7d0 2024-09-17T02:42:00.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.636+0000 7fdcabedd640 1 -- 172.21.15.131:0/3260335717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdca415f290 con 0x7fdca415f7d0 2024-09-17T02:42:00.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.640+0000 7fdc9a7fc640 1 -- 172.21.15.131:0/3260335717 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17050 (secure 0 0 0) 0x7fdc9405ec70 con 0x7fdca415f7d0 2024-09-17T02:42:00.643 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:00.643 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:41:57.843697+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:41:53.913442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-17T02:41:55.184999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:42:00.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 -- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdc88073230 msgr2=0x7fdc88075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdc88073230 0x7fdc88075740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdca00af000 tx=0x7fdca001b060 comp rx=0 tx=0).stop 2024-09-17T02:42:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 -- 172.21.15.131:0/3260335717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca415f7d0 msgr2=0x7fdca4149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca415f7d0 0x7fdca4149780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdc940058d0 tx=0x7fdc9400dde0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 -- 172.21.15.131:0/3260335717 shutdown_connections 2024-09-17T02:42:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdc88073230 0x7fdc88075740 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdca4162950 0x7fdca414a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdca415f7d0 0x7fdca4149780 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 --2- 172.21.15.131:0/3260335717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdca40cf710 0x7fdca414ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 -- 172.21.15.131:0/3260335717 >> 172.21.15.131:0/3260335717 conn(0x7fdca40d1b10 msgr2=0x7fdca40b7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 -- 172.21.15.131:0/3260335717 shutdown_connections 2024-09-17T02:42:00.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.644+0000 7fdcabedd640 1 -- 172.21.15.131:0/3260335717 wait complete. 2024-09-17T02:42:00.666 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T02:42:00.666 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:42:00.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.856+0000 7f0c1efae640 1 -- 172.21.15.131:0/1672270624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c18073970 msgr2=0x7f0c1806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.856+0000 7f0c1efae640 1 --2- 172.21.15.131:0/1672270624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c18073970 0x7f0c1806dea0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009900 tx=0x7f0c0401c8b0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.856+0000 7f0c1efae640 1 -- 172.21.15.131:0/1672270624 shutdown_connections 2024-09-17T02:42:00.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.856+0000 7f0c1efae640 1 --2- 172.21.15.131:0/1672270624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c18073970 0x7f0c1806dea0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.856+0000 7f0c1efae640 1 --2- 172.21.15.131:0/1672270624 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0c18155240 0x7f0c18073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.856+0000 7f0c1efae640 1 -- 172.21.15.131:0/1672270624 >> 172.21.15.131:0/1672270624 conn(0x7f0c1806eea0 msgr2=0x7f0c1806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:00.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.856+0000 7f0c1efae640 1 -- 172.21.15.131:0/1672270624 shutdown_connections 2024-09-17T02:42:00.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.856+0000 7f0c1efae640 1 -- 172.21.15.131:0/1672270624 wait complete. 2024-09-17T02:42:00.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1efae640 1 Processor -- start 2024-09-17T02:42:00.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1efae640 1 -- start start 2024-09-17T02:42:00.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1efae640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1806e3e0 0x7f0c180c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1efae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c18073970 0x7f0c181f9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1efae640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0c18155240 0x7f0c181f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1efae640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18165ef0 con 0x7f0c1806e3e0 2024-09-17T02:42:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1efae640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18165d70 con 0x7f0c18073970 2024-09-17T02:42:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1efae640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0c18166070 con 0x7f0c18155240 2024-09-17T02:42:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1cd4a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1806e3e0 0x7f0c180c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1cd4a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1806e3e0 0x7f0c180c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38320/0 (socket says 172.21.15.131:38320) 2024-09-17T02:42:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1cd4a640 1 -- 172.21.15.131:0/2814770022 learned_addr learned my addr 172.21.15.131:0/2814770022 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c17fff640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c18073970 0x7f0c181f9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1cd4a640 1 -- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0c18155240 msgr2=0x7f0c181f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1cd4a640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0c18155240 0x7f0c181f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1cd4a640 1 -- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c18073970 msgr2=0x7f0c181f9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1cd4a640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c18073970 0x7f0c181f9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1cd4a640 1 -- 172.21.15.131:0/2814770022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c18150490 con 0x7f0c1806e3e0 2024-09-17T02:42:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c17fff640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c18073970 0x7f0c181f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1cd4a640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1806e3e0 0x7f0c180c47d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c940 tx=0x7f0c0c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c157fa640 1 -- 172.21.15.131:0/2814770022 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c004220 con 0x7f0c1806e3e0 2024-09-17T02:42:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c157fa640 1 -- 172.21.15.131:0/2814770022 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c0c0079b0 con 0x7f0c1806e3e0 2024-09-17T02:42:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c157fa640 1 -- 172.21.15.131:0/2814770022 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0044b0 con 0x7f0c1806e3e0 2024-09-17T02:42:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1efae640 1 -- 172.21.15.131:0/2814770022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c181517a0 con 0x7f0c1806e3e0 2024-09-17T02:42:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.860+0000 7f0c1efae640 1 -- 172.21.15.131:0/2814770022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c180b5390 con 0x7f0c1806e3e0 2024-09-17T02:42:00.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.864+0000 7f0c1efae640 1 -- 172.21.15.131:0/2814770022 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c180b3900 con 0x7f0c1806e3e0 2024-09-17T02:42:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.864+0000 7f0c157fa640 1 -- 172.21.15.131:0/2814770022 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0c0c003600 con 0x7f0c1806e3e0 2024-09-17T02:42:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.864+0000 7f0c157fa640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073200 0x7f0bfc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.864+0000 7f0c157fa640 1 -- 172.21.15.131:0/2814770022 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f0c0c094430 con 0x7f0c1806e3e0 2024-09-17T02:42:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.864+0000 7f0c17fff640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073200 0x7f0bfc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:00.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.864+0000 7f0c17fff640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073200 0x7f0bfc075710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0c040af000 tx=0x7f0c0401b0f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:00.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:00.868+0000 7f0c157fa640 1 -- 172.21.15.131:0/2814770022 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0c0c014030 con 0x7f0c1806e3e0 2024-09-17T02:42:01.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.028+0000 7f0c1efae640 1 -- 172.21.15.131:0/2814770022 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c18151550 con 0x7f0bfc073200 2024-09-17T02:42:01.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.040+0000 7f0c157fa640 1 -- 172.21.15.131:0/2814770022 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245474 (secure 0 0 0) 0x7f0c18151550 con 0x7f0bfc073200 2024-09-17T02:42:01.045 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:01.047 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:42:01.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 -- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073200 msgr2=0x7f0bfc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073200 0x7f0bfc075710 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0c040af000 tx=0x7f0c0401b0f0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 -- 172.21.15.131:0/2814770022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1806e3e0 msgr2=0x7f0c180c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1806e3e0 0x7f0c180c47d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c940 tx=0x7f0c0c00ce00 comp rx=0 tx=0).stop 2024-09-17T02:42:01.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 -- 172.21.15.131:0/2814770022 shutdown_connections 2024-09-17T02:42:01.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073200 0x7f0bfc075710 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0c18155240 0x7f0c181f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c18073970 0x7f0c181f9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 --2- 172.21.15.131:0/2814770022 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1806e3e0 0x7f0c180c47d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 -- 172.21.15.131:0/2814770022 >> 172.21.15.131:0/2814770022 conn(0x7f0c1806eea0 msgr2=0x7f0c180b0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:01.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 -- 172.21.15.131:0/2814770022 shutdown_connections 2024-09-17T02:42:01.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.044+0000 7f0c1efae640 1 -- 172.21.15.131:0/2814770022 wait complete. 2024-09-17T02:42:01.063 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-09-17T02:41:59.143105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220664,"kb_used_data":4424,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754056,"statfs":{"total":773094113280,"available":772868153344,"internally_reserved":0,"allocated":4530176,"data_stored":2578298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.334709"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.850774+0000","last_change":"2024-09-17T02:41:55.850774+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.850774+0000","last_undegraded":"2024-09-17T02:41:55.850774+0000","last_fullsized":"2024-09-17T02:41:55.850774+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:55.802457+0000","last_change":"2024-09-17T02:41:55.802457+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.802457+0000","last_undegraded":"2024-09-17T02:41:55.802457+0000","last_fullsized":"2024-09-17T02:41:55.802457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.836800+0000","last_change":"2024-09-17T02:41:55.836800+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.836800+0000","last_undegraded":"2024-09-17T02:41:55.836800+0000","last_fullsized":"2024-09-17T02:41:55.836800+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.838796+0000","last_change":"2024-09-17T02:41:55.838796+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.838796+0000","last_undegraded":"2024-09-17T02:41:55.838796+0000","last_fullsized":"2024-09-17T02:41:55.838796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822660+0000","last_change":"2024-09-17T02:41:55.822660+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822660+0000","last_undegraded":"2024-09-17T02:41:55.822660+0000","last_fullsized":"2024-09-17T02:41:55.822660+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:55.802457+0000","last_change":"2024-09-17T02:41:55.802457+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.802457+0000","last_undegraded":"2024-09-17T02:41:55.802457+0000","last_fullsized":"2024-09-17T02:41:55.802457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825753+0000","last_change":"2024-09-17T02:41:55.825753+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825753+0000","last_undegraded":"2024-09-17T02:41:55.825753+0000","last_fullsized":"2024-09-17T02:41:55.825753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.855791+0000","last_change":"2024-09-17T02:41:55.855791+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.855791+0000","last_undegraded":"2024-09-17T02:41:55.855791+0000","last_fullsized":"2024-09-17T02:41:55.855791+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815860+0000","last_change":"2024-09-17T02:41:55.815860+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815860+0000","last_undegraded":"2024-09-17T02:41:55.815860+0000","last_fullsized":"2024-09-17T02:41:55.815860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820570+0000","last_change":"2024-09-17T02:41:55.820570+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820570+0000","last_undegraded":"2024-09-17T02:41:55.820570+0000","last_fullsized":"2024-09-17T02:41:55.820570+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830051+0000","last_change":"2024-09-17T02:41:55.830051+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830051+0000","last_undegraded":"2024-09-17T02:41:55.830051+0000","last_fullsized":"2024-09-17T02:41:55.830051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.817345+0000","last_change":"2024-09-17T02:41:55.817345+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.817345+0000","last_undegraded":"2024-09-17T02:41:55.817345+0000","last_fullsized":"2024-09-17T02:41:55.817345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820760+0000","last_change":"2024-09-17T02:41:55.820760+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820760+0000","last_undegraded":"2024-09-17T02:41:55.820760+0000","last_fullsized":"2024-09-17T02:41:55.820760+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822279+0000","last_change":"2024-09-17T02:41:55.822279+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822279+0000","last_undegraded":"2024-09-17T02:41:55.822279+0000","last_fullsized":"2024-09-17T02:41:55.822279+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.832996+0000","last_change":"2024-09-17T02:41:55.832996+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.832996+0000","last_undegraded":"2024-09-17T02:41:55.832996+0000","last_fullsized":"2024-09-17T02:41:55.832996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821113+0000","last_change":"2024-09-17T02:41:55.821113+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821113+0000","last_undegraded":"2024-09-17T02:41:55.821113+0000","last_fullsized":"2024-09-17T02:41:55.821113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.833655+0000","last_change":"2024-09-17T02:41:55.833655+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.833655+0000","last_undegraded":"2024-09-17T02:41:55.833655+0000","last_fullsized":"2024-09-17T02:41:55.833655+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830989+0000","last_change":"2024-09-17T02:41:55.830989+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830989+0000","last_undegraded":"2024-09-17T02:41:55.830989+0000","last_fullsized":"2024-09-17T02:41:55.830989+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830539+0000","last_change":"2024-09-17T02:41:55.830539+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830539+0000","last_undegraded":"2024-09-17T02:41:55.830539+0000","last_fullsized":"2024-09-17T02:41:55.830539+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.828730+0000","last_change":"2024-09-17T02:41:55.828730+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.828730+0000","last_undegraded":"2024-09-17T02:41:55.828730+0000","last_fullsized":"2024-09-17T02:41:55.828730+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.829387+0000","last_change":"2024-09-17T02:41:55.829387+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.829387+0000","last_undegraded":"2024-09-17T02:41:55.829387+0000","last_fullsized":"2024-09-17T02:41:55.829387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826821+0000","last_change":"2024-09-17T02:41:55.826821+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826821+0000","last_undegraded":"2024-09-17T02:41:55.826821+0000","last_fullsized":"2024-09-17T02:41:55.826821+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.827563+0000","last_change":"2024-09-17T02:41:55.827563+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.827563+0000","last_undegraded":"2024-09-17T02:41:55.827563+0000","last_fullsized":"2024-09-17T02:41:55.827563+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.841871+0000","last_change":"2024-09-17T02:41:55.841871+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.841871+0000","last_undegraded":"2024-09-17T02:41:55.841871+0000","last_fullsized":"2024-09-17T02:41:55.841871+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823236+0000","last_change":"2024-09-17T02:41:55.823236+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823236+0000","last_undegraded":"2024-09-17T02:41:55.823236+0000","last_fullsized":"2024-09-17T02:41:55.823236+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.839845+0000","last_change":"2024-09-17T02:41:55.839845+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.839845+0000","last_undegraded":"2024-09-17T02:41:55.839845+0000","last_fullsized":"2024-09-17T02:41:55.839845+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.824021+0000","last_change":"2024-09-17T02:41:55.824021+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.824021+0000","last_undegraded":"2024-09-17T02:41:55.824021+0000","last_fullsized":"2024-09-17T02:41:55.824021+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.824324+0000","last_change":"2024-09-17T02:41:55.824324+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.824324+0000","last_undegraded":"2024-09-17T02:41:55.824324+0000","last_fullsized":"2024-09-17T02:41:55.824324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.829096+0000","last_change":"2024-09-17T02:41:55.829096+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.829096+0000","last_undegraded":"2024-09-17T02:41:55.829096+0000","last_fullsized":"2024-09-17T02:41:55.829096+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826857+0000","last_change":"2024-09-17T02:41:55.826857+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826857+0000","last_undegraded":"2024-09-17T02:41:55.826857+0000","last_fullsized":"2024-09-17T02:41:55.826857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821650+0000","last_change":"2024-09-17T02:41:55.821650+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821650+0000","last_undegraded":"2024-09-17T02:41:55.821650+0000","last_fullsized":"2024-09-17T02:41:55.821650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.840670+0000","last_change":"2024-09-17T02:41:55.840670+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.840670+0000","last_undegraded":"2024-09-17T02:41:55.840670+0000","last_fullsized":"2024-09-17T02:41:55.840670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819919+0000","last_change":"2024-09-17T02:41:55.819919+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819919+0000","last_undegraded":"2024-09-17T02:41:55.819919+0000","last_fullsized":"2024-09-17T02:41:55.819919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848148+0000","last_change":"2024-09-17T02:41:55.843782+0000","last_active":"2024-09-17T02:41:55.848148+0000","last_peered":"2024-09-17T02:41:55.848148+0000","last_clean":"2024-09-17T02:41:55.848148+0000","last_became_active":"2024-09-17T02:41:55.843362+0000","last_became_peered":"2024-09-17T02:41:55.843362+0000","last_unstale":"2024-09-17T02:41:55.848148+0000","last_undegraded":"2024-09-17T02:41:55.848148+0000","last_fullsized":"2024-09-17T02:41:55.848148+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:39.655039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:55.802457+0000","last_change":"2024-09-17T02:41:55.802457+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.802457+0000","last_undegraded":"2024-09-17T02:41:55.802457+0000","last_fullsized":"2024-09-17T02:41:55.802457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:54.799092+0000","last_change":"2024-09-17T02:41:54.799092+0000","last_active":"2024-09-17T02:41:54.799092+0000","last_peered":"2024-09-17T02:41:54.799092+0000","last_clean":"2024-09-17T02:41:54.799092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:54.799092+0000","last_undegraded":"2024-09-17T02:41:54.799092+0000","last_fullsized":"2024-09-17T02:41:54.799092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830088+0000","last_change":"2024-09-17T02:41:55.830088+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830088+0000","last_undegraded":"2024-09-17T02:41:55.830088+0000","last_fullsized":"2024-09-17T02:41:55.830088+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869994+0000","last_change":"2024-09-17T02:41:55.865708+0000","last_active":"2024-09-17T02:41:55.869994+0000","last_peered":"2024-09-17T02:41:55.869994+0000","last_clean":"2024-09-17T02:41:55.869994+0000","last_became_active":"2024-09-17T02:41:55.861337+0000","last_became_peered":"2024-09-17T02:41:55.861337+0000","last_unstale":"2024-09-17T02:41:55.869994+0000","last_undegraded":"2024-09-17T02:41:55.869994+0000","last_fullsized":"2024-09-17T02:41:55.869994+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:28:47.106157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:55.802457+0000","last_change":"2024-09-17T02:41:55.802457+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.802457+0000","last_undegraded":"2024-09-17T02:41:55.802457+0000","last_fullsized":"2024-09-17T02:41:55.802457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.862703+0000","last_change":"2024-09-17T02:41:55.859306+0000","last_active":"2024-09-17T02:41:55.862703+0000","last_peered":"2024-09-17T02:41:55.862703+0000","last_clean":"2024-09-17T02:41:55.862703+0000","last_became_active":"2024-09-17T02:41:55.858980+0000","last_became_peered":"2024-09-17T02:41:55.858980+0000","last_unstale":"2024-09-17T02:41:55.862703+0000","last_undegraded":"2024-09-17T02:41:55.862703+0000","last_fullsized":"2024-09-17T02:41:55.862703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:13.459554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.839286+0000","last_change":"2024-09-17T02:41:55.839286+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.839286+0000","last_undegraded":"2024-09-17T02:41:55.839286+0000","last_fullsized":"2024-09-17T02:41:55.839286+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.836960+0000","last_change":"2024-09-17T02:41:55.832205+0000","last_active":"2024-09-17T02:41:55.836960+0000","last_peered":"2024-09-17T02:41:55.836960+0000","last_clean":"2024-09-17T02:41:55.836960+0000","last_became_active":"2024-09-17T02:41:55.831773+0000","last_became_peered":"2024-09-17T02:41:55.831773+0000","last_unstale":"2024-09-17T02:41:55.836960+0000","last_undegraded":"2024-09-17T02:41:55.836960+0000","last_fullsized":"2024-09-17T02:41:55.836960+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:09.662328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822131+0000","last_change":"2024-09-17T02:41:55.822131+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822131+0000","last_undegraded":"2024-09-17T02:41:55.822131+0000","last_fullsized":"2024-09-17T02:41:55.822131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.871586+0000","last_change":"2024-09-17T02:41:55.867371+0000","last_active":"2024-09-17T02:41:55.871586+0000","last_peered":"2024-09-17T02:41:55.871586+0000","last_clean":"2024-09-17T02:41:55.871586+0000","last_became_active":"2024-09-17T02:41:55.866967+0000","last_became_peered":"2024-09-17T02:41:55.866967+0000","last_unstale":"2024-09-17T02:41:55.871586+0000","last_undegraded":"2024-09-17T02:41:55.871586+0000","last_fullsized":"2024-09-17T02:41:55.871586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:08.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821328+0000","last_change":"2024-09-17T02:41:55.821328+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821328+0000","last_undegraded":"2024-09-17T02:41:55.821328+0000","last_fullsized":"2024-09-17T02:41:55.821328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869429+0000","last_change":"2024-09-17T02:41:55.862221+0000","last_active":"2024-09-17T02:41:55.869429+0000","last_peered":"2024-09-17T02:41:55.869429+0000","last_clean":"2024-09-17T02:41:55.869429+0000","last_became_active":"2024-09-17T02:41:55.861013+0000","last_became_peered":"2024-09-17T02:41:55.861013+0000","last_unstale":"2024-09-17T02:41:55.869429+0000","last_undegraded":"2024-09-17T02:41:55.869429+0000","last_fullsized":"2024-09-17T02:41:55.869429+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:45.117350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:55.802457+0000","last_change":"2024-09-17T02:41:55.802457+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.802457+0000","last_undegraded":"2024-09-17T02:41:55.802457+0000","last_fullsized":"2024-09-17T02:41:55.802457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.835588+0000","last_change":"2024-09-17T02:41:55.833725+0000","last_active":"2024-09-17T02:41:55.835588+0000","last_peered":"2024-09-17T02:41:55.835588+0000","last_clean":"2024-09-17T02:41:55.835588+0000","last_became_active":"2024-09-17T02:41:55.833225+0000","last_became_peered":"2024-09-17T02:41:55.833225+0000","last_unstale":"2024-09-17T02:41:55.835588+0000","last_undegraded":"2024-09-17T02:41:55.835588+0000","last_fullsized":"2024-09-17T02:41:55.835588+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:19.961178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.837456+0000","last_change":"2024-09-17T02:41:55.837456+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.837456+0000","last_undegraded":"2024-09-17T02:41:55.837456+0000","last_fullsized":"2024-09-17T02:41:55.837456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870175+0000","last_change":"2024-09-17T02:41:55.865849+0000","last_active":"2024-09-17T02:41:55.870175+0000","last_peered":"2024-09-17T02:41:55.870175+0000","last_clean":"2024-09-17T02:41:55.870175+0000","last_became_active":"2024-09-17T02:41:55.863121+0000","last_became_peered":"2024-09-17T02:41:55.863121+0000","last_unstale":"2024-09-17T02:41:55.870175+0000","last_undegraded":"2024-09-17T02:41:55.870175+0000","last_fullsized":"2024-09-17T02:41:55.870175+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:37.028521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822346+0000","last_change":"2024-09-17T02:41:55.822346+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822346+0000","last_undegraded":"2024-09-17T02:41:55.822346+0000","last_fullsized":"2024-09-17T02:41:55.822346+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848532+0000","last_change":"2024-09-17T02:41:55.846533+0000","last_active":"2024-09-17T02:41:55.848532+0000","last_peered":"2024-09-17T02:41:55.848532+0000","last_clean":"2024-09-17T02:41:55.848532+0000","last_became_active":"2024-09-17T02:41:55.846130+0000","last_became_peered":"2024-09-17T02:41:55.846130+0000","last_unstale":"2024-09-17T02:41:55.848532+0000","last_undegraded":"2024-09-17T02:41:55.848532+0000","last_fullsized":"2024-09-17T02:41:55.848532+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:31.071112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825732+0000","last_change":"2024-09-17T02:41:55.825732+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825732+0000","last_undegraded":"2024-09-17T02:41:55.825732+0000","last_fullsized":"2024-09-17T02:41:55.825732+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.871635+0000","last_change":"2024-09-17T02:41:55.866610+0000","last_active":"2024-09-17T02:41:55.871635+0000","last_peered":"2024-09-17T02:41:55.871635+0000","last_clean":"2024-09-17T02:41:55.871635+0000","last_became_active":"2024-09-17T02:41:55.866246+0000","last_became_peered":"2024-09-17T02:41:55.866246+0000","last_unstale":"2024-09-17T02:41:55.871635+0000","last_undegraded":"2024-09-17T02:41:55.871635+0000","last_fullsized":"2024-09-17T02:41:55.871635+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:45.605666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820352+0000","last_change":"2024-09-17T02:41:55.820352+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820352+0000","last_undegraded":"2024-09-17T02:41:55.820352+0000","last_fullsized":"2024-09-17T02:41:55.820352+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869861+0000","last_change":"2024-09-17T02:41:55.863340+0000","last_active":"2024-09-17T02:41:55.869861+0000","last_peered":"2024-09-17T02:41:55.869861+0000","last_clean":"2024-09-17T02:41:55.869861+0000","last_became_active":"2024-09-17T02:41:55.861822+0000","last_became_peered":"2024-09-17T02:41:55.861822+0000","last_unstale":"2024-09-17T02:41:55.869861+0000","last_undegraded":"2024-09-17T02:41:55.869861+0000","last_fullsized":"2024-09-17T02:41:55.869861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:44.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.838002+0000","last_change":"2024-09-17T02:41:55.838002+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.838002+0000","last_undegraded":"2024-09-17T02:41:55.838002+0000","last_fullsized":"2024-09-17T02:41:55.838002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870109+0000","last_change":"2024-09-17T02:41:55.866575+0000","last_active":"2024-09-17T02:41:55.870109+0000","last_peered":"2024-09-17T02:41:55.870109+0000","last_clean":"2024-09-17T02:41:55.870109+0000","last_became_active":"2024-09-17T02:41:55.862842+0000","last_became_peered":"2024-09-17T02:41:55.862842+0000","last_unstale":"2024-09-17T02:41:55.870109+0000","last_undegraded":"2024-09-17T02:41:55.870109+0000","last_fullsized":"2024-09-17T02:41:55.870109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:46.934262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:55.802457+0000","last_change":"2024-09-17T02:41:55.802457+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.802457+0000","last_undegraded":"2024-09-17T02:41:55.802457+0000","last_fullsized":"2024-09-17T02:41:55.802457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:54.799092+0000","last_change":"2024-09-17T02:41:54.799092+0000","last_active":"2024-09-17T02:41:54.799092+0000","last_peered":"2024-09-17T02:41:54.799092+0000","last_clean":"2024-09-17T02:41:54.799092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:54.799092+0000","last_undegraded":"2024-09-17T02:41:54.799092+0000","last_fullsized":"2024-09-17T02:41:54.799092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823496+0000","last_change":"2024-09-17T02:41:55.823496+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823496+0000","last_undegraded":"2024-09-17T02:41:55.823496+0000","last_fullsized":"2024-09-17T02:41:55.823496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.840993+0000","last_change":"2024-09-17T02:41:55.830025+0000","last_active":"2024-09-17T02:41:55.840993+0000","last_peered":"2024-09-17T02:41:55.840993+0000","last_clean":"2024-09-17T02:41:55.840993+0000","last_became_active":"2024-09-17T02:41:55.829641+0000","last_became_peered":"2024-09-17T02:41:55.829641+0000","last_unstale":"2024-09-17T02:41:55.840993+0000","last_undegraded":"2024-09-17T02:41:55.840993+0000","last_fullsized":"2024-09-17T02:41:55.840993+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:33.231862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815645+0000","last_change":"2024-09-17T02:41:55.815645+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815645+0000","last_undegraded":"2024-09-17T02:41:55.815645+0000","last_fullsized":"2024-09-17T02:41:55.815645+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.830119+0000","last_change":"2024-09-17T02:41:55.824740+0000","last_active":"2024-09-17T02:41:55.830119+0000","last_peered":"2024-09-17T02:41:55.830119+0000","last_clean":"2024-09-17T02:41:55.830119+0000","last_became_active":"2024-09-17T02:41:55.824315+0000","last_became_peered":"2024-09-17T02:41:55.824315+0000","last_unstale":"2024-09-17T02:41:55.830119+0000","last_undegraded":"2024-09-17T02:41:55.830119+0000","last_fullsized":"2024-09-17T02:41:55.830119+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:20.013546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:55.802457+0000","last_change":"2024-09-17T02:41:55.802457+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.802457+0000","last_undegraded":"2024-09-17T02:41:55.802457+0000","last_fullsized":"2024-09-17T02:41:55.802457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.835545+0000","last_change":"2024-09-17T02:41:55.833534+0000","last_active":"2024-09-17T02:41:55.835545+0000","last_peered":"2024-09-17T02:41:55.835545+0000","last_clean":"2024-09-17T02:41:55.835545+0000","last_became_active":"2024-09-17T02:41:55.833218+0000","last_became_peered":"2024-09-17T02:41:55.833218+0000","last_unstale":"2024-09-17T02:41:55.835545+0000","last_undegraded":"2024-09-17T02:41:55.835545+0000","last_fullsized":"2024-09-17T02:41:55.835545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:57.836648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821387+0000","last_change":"2024-09-17T02:41:55.821387+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821387+0000","last_undegraded":"2024-09-17T02:41:55.821387+0000","last_fullsized":"2024-09-17T02:41:55.821387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.863199+0000","last_change":"2024-09-17T02:41:55.861548+0000","last_active":"2024-09-17T02:41:55.863199+0000","last_peered":"2024-09-17T02:41:55.863199+0000","last_clean":"2024-09-17T02:41:55.863199+0000","last_became_active":"2024-09-17T02:41:55.861200+0000","last_became_peered":"2024-09-17T02:41:55.861200+0000","last_unstale":"2024-09-17T02:41:55.863199+0000","last_undegraded":"2024-09-17T02:41:55.863199+0000","last_fullsized":"2024-09-17T02:41:55.863199+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:08.387963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819800+0000","last_change":"2024-09-17T02:41:55.819800+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819800+0000","last_undegraded":"2024-09-17T02:41:55.819800+0000","last_fullsized":"2024-09-17T02:41:55.819800+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850416+0000","last_change":"2024-09-17T02:41:55.844139+0000","last_active":"2024-09-17T02:41:55.850416+0000","last_peered":"2024-09-17T02:41:55.850416+0000","last_clean":"2024-09-17T02:41:55.850416+0000","last_became_active":"2024-09-17T02:41:55.842953+0000","last_became_peered":"2024-09-17T02:41:55.842953+0000","last_unstale":"2024-09-17T02:41:55.850416+0000","last_undegraded":"2024-09-17T02:41:55.850416+0000","last_fullsized":"2024-09-17T02:41:55.850416+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:57.710104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.833247+0000","last_change":"2024-09-17T02:41:55.833247+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.833247+0000","last_undegraded":"2024-09-17T02:41:55.833247+0000","last_fullsized":"2024-09-17T02:41:55.833247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.845102+0000","last_change":"2024-09-17T02:41:55.840256+0000","last_active":"2024-09-17T02:41:55.845102+0000","last_peered":"2024-09-17T02:41:55.845102+0000","last_clean":"2024-09-17T02:41:55.845102+0000","last_became_active":"2024-09-17T02:41:55.839277+0000","last_became_peered":"2024-09-17T02:41:55.839277+0000","last_unstale":"2024-09-17T02:41:55.845102+0000","last_undegraded":"2024-09-17T02:41:55.845102+0000","last_fullsized":"2024-09-17T02:41:55.845102+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:06:51.053427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819364+0000","last_change":"2024-09-17T02:41:55.819364+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819364+0000","last_undegraded":"2024-09-17T02:41:55.819364+0000","last_fullsized":"2024-09-17T02:41:55.819364+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848889+0000","last_change":"2024-09-17T02:41:55.846024+0000","last_active":"2024-09-17T02:41:55.848889+0000","last_peered":"2024-09-17T02:41:55.848889+0000","last_clean":"2024-09-17T02:41:55.848889+0000","last_became_active":"2024-09-17T02:41:55.845095+0000","last_became_peered":"2024-09-17T02:41:55.845095+0000","last_unstale":"2024-09-17T02:41:55.848889+0000","last_undegraded":"2024-09-17T02:41:55.848889+0000","last_fullsized":"2024-09-17T02:41:55.848889+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:51.113568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825036+0000","last_change":"2024-09-17T02:41:55.825036+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825036+0000","last_undegraded":"2024-09-17T02:41:55.825036+0000","last_fullsized":"2024-09-17T02:41:55.825036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:54.799092+0000","last_change":"2024-09-17T02:41:54.799092+0000","last_active":"2024-09-17T02:41:54.799092+0000","last_peered":"2024-09-17T02:41:54.799092+0000","last_clean":"2024-09-17T02:41:54.799092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:54.799092+0000","last_undegraded":"2024-09-17T02:41:54.799092+0000","last_fullsized":"2024-09-17T02:41:54.799092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826200+0000","last_change":"2024-09-17T02:41:55.826200+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826200+0000","last_undegraded":"2024-09-17T02:41:55.826200+0000","last_fullsized":"2024-09-17T02:41:55.826200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.836148+0000","last_change":"2024-09-17T02:41:55.833701+0000","last_active":"2024-09-17T02:41:55.836148+0000","last_peered":"2024-09-17T02:41:55.836148+0000","last_clean":"2024-09-17T02:41:55.836148+0000","last_became_active":"2024-09-17T02:41:55.833254+0000","last_became_peered":"2024-09-17T02:41:55.833254+0000","last_unstale":"2024-09-17T02:41:55.836148+0000","last_undegraded":"2024-09-17T02:41:55.836148+0000","last_fullsized":"2024-09-17T02:41:55.836148+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:51.336142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826130+0000","last_change":"2024-09-17T02:41:55.826130+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826130+0000","last_undegraded":"2024-09-17T02:41:55.826130+0000","last_fullsized":"2024-09-17T02:41:55.826130+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:54.799092+0000","last_change":"2024-09-17T02:41:54.799092+0000","last_active":"2024-09-17T02:41:54.799092+0000","last_peered":"2024-09-17T02:41:54.799092+0000","last_clean":"2024-09-17T02:41:54.799092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:54.799092+0000","last_undegraded":"2024-09-17T02:41:54.799092+0000","last_fullsized":"2024-09-17T02:41:54.799092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815903+0000","last_change":"2024-09-17T02:41:55.815903+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815903+0000","last_undegraded":"2024-09-17T02:41:55.815903+0000","last_fullsized":"2024-09-17T02:41:55.815903+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850489+0000","last_change":"2024-09-17T02:41:55.849654+0000","last_active":"2024-09-17T02:41:55.850489+0000","last_peered":"2024-09-17T02:41:55.850489+0000","last_clean":"2024-09-17T02:41:55.850489+0000","last_became_active":"2024-09-17T02:41:55.849235+0000","last_became_peered":"2024-09-17T02:41:55.849235+0000","last_unstale":"2024-09-17T02:41:55.850489+0000","last_undegraded":"2024-09-17T02:41:55.850489+0000","last_fullsized":"2024-09-17T02:41:55.850489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:34.982736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823333+0000","last_change":"2024-09-17T02:41:55.823333+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823333+0000","last_undegraded":"2024-09-17T02:41:55.823333+0000","last_fullsized":"2024-09-17T02:41:55.823333+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.852875+0000","last_change":"2024-09-17T02:41:55.843817+0000","last_active":"2024-09-17T02:41:55.852875+0000","last_peered":"2024-09-17T02:41:55.852875+0000","last_clean":"2024-09-17T02:41:55.852875+0000","last_became_active":"2024-09-17T02:41:55.841737+0000","last_became_peered":"2024-09-17T02:41:55.841737+0000","last_unstale":"2024-09-17T02:41:55.852875+0000","last_undegraded":"2024-09-17T02:41:55.852875+0000","last_fullsized":"2024-09-17T02:41:55.852875+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:31.493951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.828622+0000","last_change":"2024-09-17T02:41:55.828622+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.828622+0000","last_undegraded":"2024-09-17T02:41:55.828622+0000","last_fullsized":"2024-09-17T02:41:55.828622+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870027+0000","last_change":"2024-09-17T02:41:55.868286+0000","last_active":"2024-09-17T02:41:55.870027+0000","last_peered":"2024-09-17T02:41:55.870027+0000","last_clean":"2024-09-17T02:41:55.870027+0000","last_became_active":"2024-09-17T02:41:55.867258+0000","last_became_peered":"2024-09-17T02:41:55.867258+0000","last_unstale":"2024-09-17T02:41:55.870027+0000","last_undegraded":"2024-09-17T02:41:55.870027+0000","last_fullsized":"2024-09-17T02:41:55.870027+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:03.928821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815674+0000","last_change":"2024-09-17T02:41:55.815674+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815674+0000","last_undegraded":"2024-09-17T02:41:55.815674+0000","last_fullsized":"2024-09-17T02:41:55.815674+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869829+0000","last_change":"2024-09-17T02:41:55.863138+0000","last_active":"2024-09-17T02:41:55.869829+0000","last_peered":"2024-09-17T02:41:55.869829+0000","last_clean":"2024-09-17T02:41:55.869829+0000","last_became_active":"2024-09-17T02:41:55.862501+0000","last_became_peered":"2024-09-17T02:41:55.862501+0000","last_unstale":"2024-09-17T02:41:55.869829+0000","last_undegraded":"2024-09-17T02:41:55.869829+0000","last_fullsized":"2024-09-17T02:41:55.869829+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:16.370567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:41:55.802457+0000","last_change":"2024-09-17T02:41:55.802457+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.802457+0000","last_undegraded":"2024-09-17T02:41:55.802457+0000","last_fullsized":"2024-09-17T02:41:55.802457+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.853308+0000","last_change":"2024-09-17T02:41:55.850201+0000","last_active":"2024-09-17T02:41:55.853308+0000","last_peered":"2024-09-17T02:41:55.853308+0000","last_clean":"2024-09-17T02:41:55.853308+0000","last_became_active":"2024-09-17T02:41:55.849734+0000","last_became_peered":"2024-09-17T02:41:55.849734+0000","last_unstale":"2024-09-17T02:41:55.853308+0000","last_undegraded":"2024-09-17T02:41:55.853308+0000","last_fullsized":"2024-09-17T02:41:55.853308+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:44.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.818667+0000","last_change":"2024-09-17T02:41:55.818667+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.818667+0000","last_undegraded":"2024-09-17T02:41:55.818667+0000","last_fullsized":"2024-09-17T02:41:55.818667+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850540+0000","last_change":"2024-09-17T02:41:55.849649+0000","last_active":"2024-09-17T02:41:55.850540+0000","last_peered":"2024-09-17T02:41:55.850540+0000","last_clean":"2024-09-17T02:41:55.850540+0000","last_became_active":"2024-09-17T02:41:55.849237+0000","last_became_peered":"2024-09-17T02:41:55.849237+0000","last_unstale":"2024-09-17T02:41:55.850540+0000","last_undegraded":"2024-09-17T02:41:55.850540+0000","last_fullsized":"2024-09-17T02:41:55.850540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:40.869732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822074+0000","last_change":"2024-09-17T02:41:55.822074+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822074+0000","last_undegraded":"2024-09-17T02:41:55.822074+0000","last_fullsized":"2024-09-17T02:41:55.822074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869389+0000","last_change":"2024-09-17T02:41:55.862706+0000","last_active":"2024-09-17T02:41:55.869389+0000","last_peered":"2024-09-17T02:41:55.869389+0000","last_clean":"2024-09-17T02:41:55.869389+0000","last_became_active":"2024-09-17T02:41:55.862091+0000","last_became_peered":"2024-09-17T02:41:55.862091+0000","last_unstale":"2024-09-17T02:41:55.869389+0000","last_undegraded":"2024-09-17T02:41:55.869389+0000","last_fullsized":"2024-09-17T02:41:55.869389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:16.349899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820910+0000","last_change":"2024-09-17T02:41:55.820910+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820910+0000","last_undegraded":"2024-09-17T02:41:55.820910+0000","last_fullsized":"2024-09-17T02:41:55.820910+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.852959+0000","last_change":"2024-09-17T02:41:55.850095+0000","last_active":"2024-09-17T02:41:55.852959+0000","last_peered":"2024-09-17T02:41:55.852959+0000","last_clean":"2024-09-17T02:41:55.852959+0000","last_became_active":"2024-09-17T02:41:55.849684+0000","last_became_peered":"2024-09-17T02:41:55.849684+0000","last_unstale":"2024-09-17T02:41:55.852959+0000","last_undegraded":"2024-09-17T02:41:55.852959+0000","last_fullsized":"2024-09-17T02:41:55.852959+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:16.370567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":141,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.825082+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:41:55.825082+0000","last_peered":"2024-09-17T02:41:55.825082+0000","last_clean":"2024-09-17T02:41:55.825082+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:41:55.825082+0000","last_undegraded":"2024-09-17T02:41:55.825082+0000","last_fullsized":"2024-09-17T02:41:55.825082+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":112,"acting":112,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":56,"acting":56,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":765006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":765006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":430080,"data_stored":181305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27436,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":405504,"data_stored":168429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:42:01.068 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:42:01.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.256+0000 7f8e7b233640 1 -- 172.21.15.131:0/246600860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e740c4ac0 msgr2=0x7f8e740c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.256+0000 7f8e7b233640 1 --2- 172.21.15.131:0/246600860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e740c4ac0 0x7f8e740c4ed0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8e68009900 tx=0x7f8e6801c850 comp rx=0 tx=0).stop 2024-09-17T02:42:01.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.256+0000 7f8e7b233640 1 -- 172.21.15.131:0/246600860 shutdown_connections 2024-09-17T02:42:01.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.256+0000 7f8e7b233640 1 --2- 172.21.15.131:0/246600860 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e740c4ac0 0x7f8e740c4ed0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.256+0000 7f8e7b233640 1 --2- 172.21.15.131:0/246600860 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e7406e2c0 0x7f8e740c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.256+0000 7f8e7b233640 1 -- 172.21.15.131:0/246600860 >> 172.21.15.131:0/246600860 conn(0x7f8e7406eea0 msgr2=0x7f8e7406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:01.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.256+0000 7f8e7b233640 1 -- 172.21.15.131:0/246600860 shutdown_connections 2024-09-17T02:42:01.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.256+0000 7f8e7b233640 1 -- 172.21.15.131:0/246600860 wait complete. 2024-09-17T02:42:01.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e7b233640 1 Processor -- start 2024-09-17T02:42:01.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e7b233640 1 -- start start 2024-09-17T02:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e7b233640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7406e2c0 0x7f8e74149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e7b233640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e740c4ac0 0x7f8e740cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e7b233640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e740cdbf0 0x7f8e74159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e7b233640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e740cf2b0 con 0x7f8e740cdbf0 2024-09-17T02:42:01.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e7b233640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8e740cf130 con 0x7f8e7406e2c0 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e7b233640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8e740cf430 con 0x7f8e740c4ac0 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e797d0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e740cdbf0 0x7f8e74159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e797d0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e740cdbf0 0x7f8e74159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57058/0 (socket says 172.21.15.131:57058) 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e797d0640 1 -- 172.21.15.131:0/3116005417 learned_addr learned my addr 172.21.15.131:0/3116005417 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e78fcf640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7406e2c0 0x7f8e74149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e73fff640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e740c4ac0 0x7f8e740cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e797d0640 1 -- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e740c4ac0 msgr2=0x7f8e740cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e797d0640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e740c4ac0 0x7f8e740cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e797d0640 1 -- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7406e2c0 msgr2=0x7f8e74149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e797d0640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7406e2c0 0x7f8e74149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e797d0640 1 -- 172.21.15.131:0/3116005417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e740cc2d0 con 0x7f8e740cdbf0 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e797d0640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e740cdbf0 0x7f8e74159020 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00c3c0 tx=0x7f8e6c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e717fa640 1 -- 172.21.15.131:0/3116005417 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c017070 con 0x7f8e740cdbf0 2024-09-17T02:42:01.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e717fa640 1 -- 172.21.15.131:0/3116005417 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e6c0053c0 con 0x7f8e740cdbf0 2024-09-17T02:42:01.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e7b233640 1 -- 172.21.15.131:0/3116005417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e740c71b0 con 0x7f8e740cdbf0 2024-09-17T02:42:01.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e717fa640 1 -- 172.21.15.131:0/3116005417 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c003860 con 0x7f8e740cdbf0 2024-09-17T02:42:01.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e7b233640 1 -- 172.21.15.131:0/3116005417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e740c7770 con 0x7f8e740cdbf0 2024-09-17T02:42:01.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.260+0000 7f8e7b233640 1 -- 172.21.15.131:0/3116005417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e740c4580 con 0x7f8e740cdbf0 2024-09-17T02:42:01.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.264+0000 7f8e717fa640 1 -- 172.21.15.131:0/3116005417 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8e6c01a070 con 0x7f8e740cdbf0 2024-09-17T02:42:01.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.264+0000 7f8e717fa640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e60073200 0x7f8e60075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:01.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.264+0000 7f8e717fa640 1 -- 172.21.15.131:0/3116005417 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f8e6c0936f0 con 0x7f8e740cdbf0 2024-09-17T02:42:01.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.264+0000 7f8e78fcf640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e60073200 0x7f8e60075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:01.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.264+0000 7f8e78fcf640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e60073200 0x7f8e60075710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005c50 tx=0x7f8e64005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:01.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.268+0000 7f8e717fa640 1 -- 172.21.15.131:0/3116005417 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8e6c05e580 con 0x7f8e740cdbf0 2024-09-17T02:42:01.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.436+0000 7f8e7b233640 1 -- 172.21.15.131:0/3116005417 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e740cc500 con 0x7f8e60073200 2024-09-17T02:42:01.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.448+0000 7f8e717fa640 1 -- 172.21.15.131:0/3116005417 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247319 (secure 0 0 0) 0x7f8e740cc500 con 0x7f8e60073200 2024-09-17T02:42:01.451 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:01.453 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:42:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 -- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e60073200 msgr2=0x7f8e60075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e60073200 0x7f8e60075710 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e64005c50 tx=0x7f8e64005be0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 -- 172.21.15.131:0/3116005417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e740cdbf0 msgr2=0x7f8e74159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e740cdbf0 0x7f8e74159020 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c00c3c0 tx=0x7f8e6c00c880 comp rx=0 tx=0).stop 2024-09-17T02:42:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 -- 172.21.15.131:0/3116005417 shutdown_connections 2024-09-17T02:42:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e60073200 0x7f8e60075710 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e740cdbf0 0x7f8e74159020 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e740c4ac0 0x7f8e740cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 --2- 172.21.15.131:0/3116005417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e7406e2c0 0x7f8e74149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 -- 172.21.15.131:0/3116005417 >> 172.21.15.131:0/3116005417 conn(0x7f8e7406eea0 msgr2=0x7f8e740b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 -- 172.21.15.131:0/3116005417 shutdown_connections 2024-09-17T02:42:01.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.452+0000 7f8e7b233640 1 -- 172.21.15.131:0/3116005417 wait complete. 2024-09-17T02:42:01.473 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-17T02:42:01.143735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224868,"kb_used_data":4476,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749852,"statfs":{"total":773094113280,"available":772863848448,"internally_reserved":0,"allocated":4583424,"data_stored":2605067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001879"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.850774+0000","last_change":"2024-09-17T02:41:55.850774+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.850774+0000","last_undegraded":"2024-09-17T02:41:55.850774+0000","last_fullsized":"2024-09-17T02:41:55.850774+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858600+0000","last_change":"2024-09-17T02:41:56.867810+0000","last_active":"2024-09-17T02:41:57.858600+0000","last_peered":"2024-09-17T02:41:57.858600+0000","last_clean":"2024-09-17T02:41:57.858600+0000","last_became_active":"2024-09-17T02:41:56.866944+0000","last_became_peered":"2024-09-17T02:41:56.866944+0000","last_unstale":"2024-09-17T02:41:57.858600+0000","last_undegraded":"2024-09-17T02:41:57.858600+0000","last_fullsized":"2024-09-17T02:41:57.858600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:59.549330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.836800+0000","last_change":"2024-09-17T02:41:55.836800+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.836800+0000","last_undegraded":"2024-09-17T02:41:55.836800+0000","last_fullsized":"2024-09-17T02:41:55.836800+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.838796+0000","last_change":"2024-09-17T02:41:55.838796+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.838796+0000","last_undegraded":"2024-09-17T02:41:55.838796+0000","last_fullsized":"2024-09-17T02:41:55.838796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822660+0000","last_change":"2024-09-17T02:41:55.822660+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822660+0000","last_undegraded":"2024-09-17T02:41:55.822660+0000","last_fullsized":"2024-09-17T02:41:55.822660+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857974+0000","last_change":"2024-09-17T02:41:56.880704+0000","last_active":"2024-09-17T02:41:57.857974+0000","last_peered":"2024-09-17T02:41:57.857974+0000","last_clean":"2024-09-17T02:41:57.857974+0000","last_became_active":"2024-09-17T02:41:56.879945+0000","last_became_peered":"2024-09-17T02:41:56.879945+0000","last_unstale":"2024-09-17T02:41:57.857974+0000","last_undegraded":"2024-09-17T02:41:57.857974+0000","last_fullsized":"2024-09-17T02:41:57.857974+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:36.026817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825753+0000","last_change":"2024-09-17T02:41:55.825753+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825753+0000","last_undegraded":"2024-09-17T02:41:55.825753+0000","last_fullsized":"2024-09-17T02:41:55.825753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.855791+0000","last_change":"2024-09-17T02:41:55.855791+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.855791+0000","last_undegraded":"2024-09-17T02:41:55.855791+0000","last_fullsized":"2024-09-17T02:41:55.855791+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815860+0000","last_change":"2024-09-17T02:41:55.815860+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815860+0000","last_undegraded":"2024-09-17T02:41:55.815860+0000","last_fullsized":"2024-09-17T02:41:55.815860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820570+0000","last_change":"2024-09-17T02:41:55.820570+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820570+0000","last_undegraded":"2024-09-17T02:41:55.820570+0000","last_fullsized":"2024-09-17T02:41:55.820570+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830051+0000","last_change":"2024-09-17T02:41:55.830051+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830051+0000","last_undegraded":"2024-09-17T02:41:55.830051+0000","last_fullsized":"2024-09-17T02:41:55.830051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.817345+0000","last_change":"2024-09-17T02:41:55.817345+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.817345+0000","last_undegraded":"2024-09-17T02:41:55.817345+0000","last_fullsized":"2024-09-17T02:41:55.817345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820760+0000","last_change":"2024-09-17T02:41:55.820760+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820760+0000","last_undegraded":"2024-09-17T02:41:55.820760+0000","last_fullsized":"2024-09-17T02:41:55.820760+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822279+0000","last_change":"2024-09-17T02:41:55.822279+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822279+0000","last_undegraded":"2024-09-17T02:41:55.822279+0000","last_fullsized":"2024-09-17T02:41:55.822279+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.832996+0000","last_change":"2024-09-17T02:41:55.832996+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.832996+0000","last_undegraded":"2024-09-17T02:41:55.832996+0000","last_fullsized":"2024-09-17T02:41:55.832996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821113+0000","last_change":"2024-09-17T02:41:55.821113+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821113+0000","last_undegraded":"2024-09-17T02:41:55.821113+0000","last_fullsized":"2024-09-17T02:41:55.821113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.833655+0000","last_change":"2024-09-17T02:41:55.833655+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.833655+0000","last_undegraded":"2024-09-17T02:41:55.833655+0000","last_fullsized":"2024-09-17T02:41:55.833655+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830989+0000","last_change":"2024-09-17T02:41:55.830989+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830989+0000","last_undegraded":"2024-09-17T02:41:55.830989+0000","last_fullsized":"2024-09-17T02:41:55.830989+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830539+0000","last_change":"2024-09-17T02:41:55.830539+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830539+0000","last_undegraded":"2024-09-17T02:41:55.830539+0000","last_fullsized":"2024-09-17T02:41:55.830539+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.828730+0000","last_change":"2024-09-17T02:41:55.828730+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.828730+0000","last_undegraded":"2024-09-17T02:41:55.828730+0000","last_fullsized":"2024-09-17T02:41:55.828730+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.829387+0000","last_change":"2024-09-17T02:41:55.829387+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.829387+0000","last_undegraded":"2024-09-17T02:41:55.829387+0000","last_fullsized":"2024-09-17T02:41:55.829387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826821+0000","last_change":"2024-09-17T02:41:55.826821+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826821+0000","last_undegraded":"2024-09-17T02:41:55.826821+0000","last_fullsized":"2024-09-17T02:41:55.826821+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.827563+0000","last_change":"2024-09-17T02:41:55.827563+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.827563+0000","last_undegraded":"2024-09-17T02:41:55.827563+0000","last_fullsized":"2024-09-17T02:41:55.827563+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.841871+0000","last_change":"2024-09-17T02:41:55.841871+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.841871+0000","last_undegraded":"2024-09-17T02:41:55.841871+0000","last_fullsized":"2024-09-17T02:41:55.841871+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823236+0000","last_change":"2024-09-17T02:41:55.823236+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823236+0000","last_undegraded":"2024-09-17T02:41:55.823236+0000","last_fullsized":"2024-09-17T02:41:55.823236+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.839845+0000","last_change":"2024-09-17T02:41:55.839845+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.839845+0000","last_undegraded":"2024-09-17T02:41:55.839845+0000","last_fullsized":"2024-09-17T02:41:55.839845+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.824021+0000","last_change":"2024-09-17T02:41:55.824021+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.824021+0000","last_undegraded":"2024-09-17T02:41:55.824021+0000","last_fullsized":"2024-09-17T02:41:55.824021+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.824324+0000","last_change":"2024-09-17T02:41:55.824324+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.824324+0000","last_undegraded":"2024-09-17T02:41:55.824324+0000","last_fullsized":"2024-09-17T02:41:55.824324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.829096+0000","last_change":"2024-09-17T02:41:55.829096+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.829096+0000","last_undegraded":"2024-09-17T02:41:55.829096+0000","last_fullsized":"2024-09-17T02:41:55.829096+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826857+0000","last_change":"2024-09-17T02:41:55.826857+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826857+0000","last_undegraded":"2024-09-17T02:41:55.826857+0000","last_fullsized":"2024-09-17T02:41:55.826857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821650+0000","last_change":"2024-09-17T02:41:55.821650+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821650+0000","last_undegraded":"2024-09-17T02:41:55.821650+0000","last_fullsized":"2024-09-17T02:41:55.821650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.840670+0000","last_change":"2024-09-17T02:41:55.840670+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.840670+0000","last_undegraded":"2024-09-17T02:41:55.840670+0000","last_fullsized":"2024-09-17T02:41:55.840670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819919+0000","last_change":"2024-09-17T02:41:55.819919+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819919+0000","last_undegraded":"2024-09-17T02:41:55.819919+0000","last_fullsized":"2024-09-17T02:41:55.819919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848148+0000","last_change":"2024-09-17T02:41:55.843782+0000","last_active":"2024-09-17T02:41:55.848148+0000","last_peered":"2024-09-17T02:41:55.848148+0000","last_clean":"2024-09-17T02:41:55.848148+0000","last_became_active":"2024-09-17T02:41:55.843362+0000","last_became_peered":"2024-09-17T02:41:55.843362+0000","last_unstale":"2024-09-17T02:41:55.848148+0000","last_undegraded":"2024-09-17T02:41:55.848148+0000","last_fullsized":"2024-09-17T02:41:55.848148+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:39.655039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858515+0000","last_change":"2024-09-17T02:41:56.882518+0000","last_active":"2024-09-17T02:41:57.858515+0000","last_peered":"2024-09-17T02:41:57.858515+0000","last_clean":"2024-09-17T02:41:57.858515+0000","last_became_active":"2024-09-17T02:41:56.879984+0000","last_became_peered":"2024-09-17T02:41:56.879984+0000","last_unstale":"2024-09-17T02:41:57.858515+0000","last_undegraded":"2024-09-17T02:41:57.858515+0000","last_fullsized":"2024-09-17T02:41:57.858515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:35.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858745+0000","last_change":"2024-09-17T02:41:55.844353+0000","last_active":"2024-09-17T02:41:57.858745+0000","last_peered":"2024-09-17T02:41:57.858745+0000","last_clean":"2024-09-17T02:41:57.858745+0000","last_became_active":"2024-09-17T02:41:55.843505+0000","last_became_peered":"2024-09-17T02:41:55.843505+0000","last_unstale":"2024-09-17T02:41:57.858745+0000","last_undegraded":"2024-09-17T02:41:57.858745+0000","last_fullsized":"2024-09-17T02:41:57.858745+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:43.510214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830088+0000","last_change":"2024-09-17T02:41:55.830088+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830088+0000","last_undegraded":"2024-09-17T02:41:55.830088+0000","last_fullsized":"2024-09-17T02:41:55.830088+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869994+0000","last_change":"2024-09-17T02:41:55.865708+0000","last_active":"2024-09-17T02:41:55.869994+0000","last_peered":"2024-09-17T02:41:55.869994+0000","last_clean":"2024-09-17T02:41:55.869994+0000","last_became_active":"2024-09-17T02:41:55.861337+0000","last_became_peered":"2024-09-17T02:41:55.861337+0000","last_unstale":"2024-09-17T02:41:55.869994+0000","last_undegraded":"2024-09-17T02:41:55.869994+0000","last_fullsized":"2024-09-17T02:41:55.869994+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:28:47.106157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.859238+0000","last_change":"2024-09-17T02:41:56.867038+0000","last_active":"2024-09-17T02:41:57.859238+0000","last_peered":"2024-09-17T02:41:57.859238+0000","last_clean":"2024-09-17T02:41:57.859238+0000","last_became_active":"2024-09-17T02:41:56.865986+0000","last_became_peered":"2024-09-17T02:41:56.865986+0000","last_unstale":"2024-09-17T02:41:57.859238+0000","last_undegraded":"2024-09-17T02:41:57.859238+0000","last_fullsized":"2024-09-17T02:41:57.859238+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:31.091078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.862703+0000","last_change":"2024-09-17T02:41:55.859306+0000","last_active":"2024-09-17T02:41:55.862703+0000","last_peered":"2024-09-17T02:41:55.862703+0000","last_clean":"2024-09-17T02:41:55.862703+0000","last_became_active":"2024-09-17T02:41:55.858980+0000","last_became_peered":"2024-09-17T02:41:55.858980+0000","last_unstale":"2024-09-17T02:41:55.862703+0000","last_undegraded":"2024-09-17T02:41:55.862703+0000","last_fullsized":"2024-09-17T02:41:55.862703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:13.459554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.839286+0000","last_change":"2024-09-17T02:41:55.839286+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.839286+0000","last_undegraded":"2024-09-17T02:41:55.839286+0000","last_fullsized":"2024-09-17T02:41:55.839286+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.836960+0000","last_change":"2024-09-17T02:41:55.832205+0000","last_active":"2024-09-17T02:41:55.836960+0000","last_peered":"2024-09-17T02:41:55.836960+0000","last_clean":"2024-09-17T02:41:55.836960+0000","last_became_active":"2024-09-17T02:41:55.831773+0000","last_became_peered":"2024-09-17T02:41:55.831773+0000","last_unstale":"2024-09-17T02:41:55.836960+0000","last_undegraded":"2024-09-17T02:41:55.836960+0000","last_fullsized":"2024-09-17T02:41:55.836960+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:09.662328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822131+0000","last_change":"2024-09-17T02:41:55.822131+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822131+0000","last_undegraded":"2024-09-17T02:41:55.822131+0000","last_fullsized":"2024-09-17T02:41:55.822131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.871586+0000","last_change":"2024-09-17T02:41:55.867371+0000","last_active":"2024-09-17T02:41:55.871586+0000","last_peered":"2024-09-17T02:41:55.871586+0000","last_clean":"2024-09-17T02:41:55.871586+0000","last_became_active":"2024-09-17T02:41:55.866967+0000","last_became_peered":"2024-09-17T02:41:55.866967+0000","last_unstale":"2024-09-17T02:41:55.871586+0000","last_undegraded":"2024-09-17T02:41:55.871586+0000","last_fullsized":"2024-09-17T02:41:55.871586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:08.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821328+0000","last_change":"2024-09-17T02:41:55.821328+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821328+0000","last_undegraded":"2024-09-17T02:41:55.821328+0000","last_fullsized":"2024-09-17T02:41:55.821328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869429+0000","last_change":"2024-09-17T02:41:55.862221+0000","last_active":"2024-09-17T02:41:55.869429+0000","last_peered":"2024-09-17T02:41:55.869429+0000","last_clean":"2024-09-17T02:41:55.869429+0000","last_became_active":"2024-09-17T02:41:55.861013+0000","last_became_peered":"2024-09-17T02:41:55.861013+0000","last_unstale":"2024-09-17T02:41:55.869429+0000","last_undegraded":"2024-09-17T02:41:55.869429+0000","last_fullsized":"2024-09-17T02:41:55.869429+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:45.117350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.859026+0000","last_change":"2024-09-17T02:41:56.875750+0000","last_active":"2024-09-17T02:41:57.859026+0000","last_peered":"2024-09-17T02:41:57.859026+0000","last_clean":"2024-09-17T02:41:57.859026+0000","last_became_active":"2024-09-17T02:41:56.874828+0000","last_became_peered":"2024-09-17T02:41:56.874828+0000","last_unstale":"2024-09-17T02:41:57.859026+0000","last_undegraded":"2024-09-17T02:41:57.859026+0000","last_fullsized":"2024-09-17T02:41:57.859026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:39.693566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.835588+0000","last_change":"2024-09-17T02:41:55.833725+0000","last_active":"2024-09-17T02:41:55.835588+0000","last_peered":"2024-09-17T02:41:55.835588+0000","last_clean":"2024-09-17T02:41:55.835588+0000","last_became_active":"2024-09-17T02:41:55.833225+0000","last_became_peered":"2024-09-17T02:41:55.833225+0000","last_unstale":"2024-09-17T02:41:55.835588+0000","last_undegraded":"2024-09-17T02:41:55.835588+0000","last_fullsized":"2024-09-17T02:41:55.835588+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:19.961178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.837456+0000","last_change":"2024-09-17T02:41:55.837456+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.837456+0000","last_undegraded":"2024-09-17T02:41:55.837456+0000","last_fullsized":"2024-09-17T02:41:55.837456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870175+0000","last_change":"2024-09-17T02:41:55.865849+0000","last_active":"2024-09-17T02:41:55.870175+0000","last_peered":"2024-09-17T02:41:55.870175+0000","last_clean":"2024-09-17T02:41:55.870175+0000","last_became_active":"2024-09-17T02:41:55.863121+0000","last_became_peered":"2024-09-17T02:41:55.863121+0000","last_unstale":"2024-09-17T02:41:55.870175+0000","last_undegraded":"2024-09-17T02:41:55.870175+0000","last_fullsized":"2024-09-17T02:41:55.870175+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:37.028521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822346+0000","last_change":"2024-09-17T02:41:55.822346+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822346+0000","last_undegraded":"2024-09-17T02:41:55.822346+0000","last_fullsized":"2024-09-17T02:41:55.822346+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848532+0000","last_change":"2024-09-17T02:41:55.846533+0000","last_active":"2024-09-17T02:41:55.848532+0000","last_peered":"2024-09-17T02:41:55.848532+0000","last_clean":"2024-09-17T02:41:55.848532+0000","last_became_active":"2024-09-17T02:41:55.846130+0000","last_became_peered":"2024-09-17T02:41:55.846130+0000","last_unstale":"2024-09-17T02:41:55.848532+0000","last_undegraded":"2024-09-17T02:41:55.848532+0000","last_fullsized":"2024-09-17T02:41:55.848532+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:31.071112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825732+0000","last_change":"2024-09-17T02:41:55.825732+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825732+0000","last_undegraded":"2024-09-17T02:41:55.825732+0000","last_fullsized":"2024-09-17T02:41:55.825732+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.871635+0000","last_change":"2024-09-17T02:41:55.866610+0000","last_active":"2024-09-17T02:41:55.871635+0000","last_peered":"2024-09-17T02:41:55.871635+0000","last_clean":"2024-09-17T02:41:55.871635+0000","last_became_active":"2024-09-17T02:41:55.866246+0000","last_became_peered":"2024-09-17T02:41:55.866246+0000","last_unstale":"2024-09-17T02:41:55.871635+0000","last_undegraded":"2024-09-17T02:41:55.871635+0000","last_fullsized":"2024-09-17T02:41:55.871635+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:45.605666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820352+0000","last_change":"2024-09-17T02:41:55.820352+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820352+0000","last_undegraded":"2024-09-17T02:41:55.820352+0000","last_fullsized":"2024-09-17T02:41:55.820352+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869861+0000","last_change":"2024-09-17T02:41:55.863340+0000","last_active":"2024-09-17T02:41:55.869861+0000","last_peered":"2024-09-17T02:41:55.869861+0000","last_clean":"2024-09-17T02:41:55.869861+0000","last_became_active":"2024-09-17T02:41:55.861822+0000","last_became_peered":"2024-09-17T02:41:55.861822+0000","last_unstale":"2024-09-17T02:41:55.869861+0000","last_undegraded":"2024-09-17T02:41:55.869861+0000","last_fullsized":"2024-09-17T02:41:55.869861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:44.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.838002+0000","last_change":"2024-09-17T02:41:55.838002+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.838002+0000","last_undegraded":"2024-09-17T02:41:55.838002+0000","last_fullsized":"2024-09-17T02:41:55.838002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870109+0000","last_change":"2024-09-17T02:41:55.866575+0000","last_active":"2024-09-17T02:41:55.870109+0000","last_peered":"2024-09-17T02:41:55.870109+0000","last_clean":"2024-09-17T02:41:55.870109+0000","last_became_active":"2024-09-17T02:41:55.862842+0000","last_became_peered":"2024-09-17T02:41:55.862842+0000","last_unstale":"2024-09-17T02:41:55.870109+0000","last_undegraded":"2024-09-17T02:41:55.870109+0000","last_fullsized":"2024-09-17T02:41:55.870109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:46.934262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857825+0000","last_change":"2024-09-17T02:41:56.881999+0000","last_active":"2024-09-17T02:41:57.857825+0000","last_peered":"2024-09-17T02:41:57.857825+0000","last_clean":"2024-09-17T02:41:57.857825+0000","last_became_active":"2024-09-17T02:41:56.881115+0000","last_became_peered":"2024-09-17T02:41:56.881115+0000","last_unstale":"2024-09-17T02:41:57.857825+0000","last_undegraded":"2024-09-17T02:41:57.857825+0000","last_fullsized":"2024-09-17T02:41:57.857825+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:35.710278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857830+0000","last_change":"2024-09-17T02:41:55.839604+0000","last_active":"2024-09-17T02:41:57.857830+0000","last_peered":"2024-09-17T02:41:57.857830+0000","last_clean":"2024-09-17T02:41:57.857830+0000","last_became_active":"2024-09-17T02:41:55.835167+0000","last_became_peered":"2024-09-17T02:41:55.835167+0000","last_unstale":"2024-09-17T02:41:57.857830+0000","last_undegraded":"2024-09-17T02:41:57.857830+0000","last_fullsized":"2024-09-17T02:41:57.857830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:44.487692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823496+0000","last_change":"2024-09-17T02:41:55.823496+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823496+0000","last_undegraded":"2024-09-17T02:41:55.823496+0000","last_fullsized":"2024-09-17T02:41:55.823496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.840993+0000","last_change":"2024-09-17T02:41:55.830025+0000","last_active":"2024-09-17T02:41:55.840993+0000","last_peered":"2024-09-17T02:41:55.840993+0000","last_clean":"2024-09-17T02:41:55.840993+0000","last_became_active":"2024-09-17T02:41:55.829641+0000","last_became_peered":"2024-09-17T02:41:55.829641+0000","last_unstale":"2024-09-17T02:41:55.840993+0000","last_undegraded":"2024-09-17T02:41:55.840993+0000","last_fullsized":"2024-09-17T02:41:55.840993+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:33.231862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815645+0000","last_change":"2024-09-17T02:41:55.815645+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815645+0000","last_undegraded":"2024-09-17T02:41:55.815645+0000","last_fullsized":"2024-09-17T02:41:55.815645+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.830119+0000","last_change":"2024-09-17T02:41:55.824740+0000","last_active":"2024-09-17T02:41:55.830119+0000","last_peered":"2024-09-17T02:41:55.830119+0000","last_clean":"2024-09-17T02:41:55.830119+0000","last_became_active":"2024-09-17T02:41:55.824315+0000","last_became_peered":"2024-09-17T02:41:55.824315+0000","last_unstale":"2024-09-17T02:41:55.830119+0000","last_undegraded":"2024-09-17T02:41:55.830119+0000","last_fullsized":"2024-09-17T02:41:55.830119+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:20.013546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858509+0000","last_change":"2024-09-17T02:41:56.881490+0000","last_active":"2024-09-17T02:41:57.858509+0000","last_peered":"2024-09-17T02:41:57.858509+0000","last_clean":"2024-09-17T02:41:57.858509+0000","last_became_active":"2024-09-17T02:41:56.880621+0000","last_became_peered":"2024-09-17T02:41:56.880621+0000","last_unstale":"2024-09-17T02:41:57.858509+0000","last_undegraded":"2024-09-17T02:41:57.858509+0000","last_fullsized":"2024-09-17T02:41:57.858509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:27.688857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.835545+0000","last_change":"2024-09-17T02:41:55.833534+0000","last_active":"2024-09-17T02:41:55.835545+0000","last_peered":"2024-09-17T02:41:55.835545+0000","last_clean":"2024-09-17T02:41:55.835545+0000","last_became_active":"2024-09-17T02:41:55.833218+0000","last_became_peered":"2024-09-17T02:41:55.833218+0000","last_unstale":"2024-09-17T02:41:55.835545+0000","last_undegraded":"2024-09-17T02:41:55.835545+0000","last_fullsized":"2024-09-17T02:41:55.835545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:57.836648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821387+0000","last_change":"2024-09-17T02:41:55.821387+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821387+0000","last_undegraded":"2024-09-17T02:41:55.821387+0000","last_fullsized":"2024-09-17T02:41:55.821387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.863199+0000","last_change":"2024-09-17T02:41:55.861548+0000","last_active":"2024-09-17T02:41:55.863199+0000","last_peered":"2024-09-17T02:41:55.863199+0000","last_clean":"2024-09-17T02:41:55.863199+0000","last_became_active":"2024-09-17T02:41:55.861200+0000","last_became_peered":"2024-09-17T02:41:55.861200+0000","last_unstale":"2024-09-17T02:41:55.863199+0000","last_undegraded":"2024-09-17T02:41:55.863199+0000","last_fullsized":"2024-09-17T02:41:55.863199+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:08.387963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819800+0000","last_change":"2024-09-17T02:41:55.819800+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819800+0000","last_undegraded":"2024-09-17T02:41:55.819800+0000","last_fullsized":"2024-09-17T02:41:55.819800+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850416+0000","last_change":"2024-09-17T02:41:55.844139+0000","last_active":"2024-09-17T02:41:55.850416+0000","last_peered":"2024-09-17T02:41:55.850416+0000","last_clean":"2024-09-17T02:41:55.850416+0000","last_became_active":"2024-09-17T02:41:55.842953+0000","last_became_peered":"2024-09-17T02:41:55.842953+0000","last_unstale":"2024-09-17T02:41:55.850416+0000","last_undegraded":"2024-09-17T02:41:55.850416+0000","last_fullsized":"2024-09-17T02:41:55.850416+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:57.710104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.833247+0000","last_change":"2024-09-17T02:41:55.833247+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.833247+0000","last_undegraded":"2024-09-17T02:41:55.833247+0000","last_fullsized":"2024-09-17T02:41:55.833247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.845102+0000","last_change":"2024-09-17T02:41:55.840256+0000","last_active":"2024-09-17T02:41:55.845102+0000","last_peered":"2024-09-17T02:41:55.845102+0000","last_clean":"2024-09-17T02:41:55.845102+0000","last_became_active":"2024-09-17T02:41:55.839277+0000","last_became_peered":"2024-09-17T02:41:55.839277+0000","last_unstale":"2024-09-17T02:41:55.845102+0000","last_undegraded":"2024-09-17T02:41:55.845102+0000","last_fullsized":"2024-09-17T02:41:55.845102+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:06:51.053427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819364+0000","last_change":"2024-09-17T02:41:55.819364+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819364+0000","last_undegraded":"2024-09-17T02:41:55.819364+0000","last_fullsized":"2024-09-17T02:41:55.819364+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848889+0000","last_change":"2024-09-17T02:41:55.846024+0000","last_active":"2024-09-17T02:41:55.848889+0000","last_peered":"2024-09-17T02:41:55.848889+0000","last_clean":"2024-09-17T02:41:55.848889+0000","last_became_active":"2024-09-17T02:41:55.845095+0000","last_became_peered":"2024-09-17T02:41:55.845095+0000","last_unstale":"2024-09-17T02:41:55.848889+0000","last_undegraded":"2024-09-17T02:41:55.848889+0000","last_fullsized":"2024-09-17T02:41:55.848889+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:51.113568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825036+0000","last_change":"2024-09-17T02:41:55.825036+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825036+0000","last_undegraded":"2024-09-17T02:41:55.825036+0000","last_fullsized":"2024-09-17T02:41:55.825036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858145+0000","last_change":"2024-09-17T02:41:55.838665+0000","last_active":"2024-09-17T02:41:57.858145+0000","last_peered":"2024-09-17T02:41:57.858145+0000","last_clean":"2024-09-17T02:41:57.858145+0000","last_became_active":"2024-09-17T02:41:55.837817+0000","last_became_peered":"2024-09-17T02:41:55.837817+0000","last_unstale":"2024-09-17T02:41:57.858145+0000","last_undegraded":"2024-09-17T02:41:57.858145+0000","last_fullsized":"2024-09-17T02:41:57.858145+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:05.524969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826200+0000","last_change":"2024-09-17T02:41:55.826200+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826200+0000","last_undegraded":"2024-09-17T02:41:55.826200+0000","last_fullsized":"2024-09-17T02:41:55.826200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.836148+0000","last_change":"2024-09-17T02:41:55.833701+0000","last_active":"2024-09-17T02:41:55.836148+0000","last_peered":"2024-09-17T02:41:55.836148+0000","last_clean":"2024-09-17T02:41:55.836148+0000","last_became_active":"2024-09-17T02:41:55.833254+0000","last_became_peered":"2024-09-17T02:41:55.833254+0000","last_unstale":"2024-09-17T02:41:55.836148+0000","last_undegraded":"2024-09-17T02:41:55.836148+0000","last_fullsized":"2024-09-17T02:41:55.836148+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:51.336142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826130+0000","last_change":"2024-09-17T02:41:55.826130+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826130+0000","last_undegraded":"2024-09-17T02:41:55.826130+0000","last_fullsized":"2024-09-17T02:41:55.826130+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857048+0000","last_change":"2024-09-17T02:41:55.863410+0000","last_active":"2024-09-17T02:41:57.857048+0000","last_peered":"2024-09-17T02:41:57.857048+0000","last_clean":"2024-09-17T02:41:57.857048+0000","last_became_active":"2024-09-17T02:41:55.862647+0000","last_became_peered":"2024-09-17T02:41:55.862647+0000","last_unstale":"2024-09-17T02:41:57.857048+0000","last_undegraded":"2024-09-17T02:41:57.857048+0000","last_fullsized":"2024-09-17T02:41:57.857048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:25.110212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815903+0000","last_change":"2024-09-17T02:41:55.815903+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815903+0000","last_undegraded":"2024-09-17T02:41:55.815903+0000","last_fullsized":"2024-09-17T02:41:55.815903+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850489+0000","last_change":"2024-09-17T02:41:55.849654+0000","last_active":"2024-09-17T02:41:55.850489+0000","last_peered":"2024-09-17T02:41:55.850489+0000","last_clean":"2024-09-17T02:41:55.850489+0000","last_became_active":"2024-09-17T02:41:55.849235+0000","last_became_peered":"2024-09-17T02:41:55.849235+0000","last_unstale":"2024-09-17T02:41:55.850489+0000","last_undegraded":"2024-09-17T02:41:55.850489+0000","last_fullsized":"2024-09-17T02:41:55.850489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:34.982736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823333+0000","last_change":"2024-09-17T02:41:55.823333+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823333+0000","last_undegraded":"2024-09-17T02:41:55.823333+0000","last_fullsized":"2024-09-17T02:41:55.823333+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.852875+0000","last_change":"2024-09-17T02:41:55.843817+0000","last_active":"2024-09-17T02:41:55.852875+0000","last_peered":"2024-09-17T02:41:55.852875+0000","last_clean":"2024-09-17T02:41:55.852875+0000","last_became_active":"2024-09-17T02:41:55.841737+0000","last_became_peered":"2024-09-17T02:41:55.841737+0000","last_unstale":"2024-09-17T02:41:55.852875+0000","last_undegraded":"2024-09-17T02:41:55.852875+0000","last_fullsized":"2024-09-17T02:41:55.852875+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:31.493951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.828622+0000","last_change":"2024-09-17T02:41:55.828622+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.828622+0000","last_undegraded":"2024-09-17T02:41:55.828622+0000","last_fullsized":"2024-09-17T02:41:55.828622+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870027+0000","last_change":"2024-09-17T02:41:55.868286+0000","last_active":"2024-09-17T02:41:55.870027+0000","last_peered":"2024-09-17T02:41:55.870027+0000","last_clean":"2024-09-17T02:41:55.870027+0000","last_became_active":"2024-09-17T02:41:55.867258+0000","last_became_peered":"2024-09-17T02:41:55.867258+0000","last_unstale":"2024-09-17T02:41:55.870027+0000","last_undegraded":"2024-09-17T02:41:55.870027+0000","last_fullsized":"2024-09-17T02:41:55.870027+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:03.928821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815674+0000","last_change":"2024-09-17T02:41:55.815674+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815674+0000","last_undegraded":"2024-09-17T02:41:55.815674+0000","last_fullsized":"2024-09-17T02:41:55.815674+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869829+0000","last_change":"2024-09-17T02:41:55.863138+0000","last_active":"2024-09-17T02:41:55.869829+0000","last_peered":"2024-09-17T02:41:55.869829+0000","last_clean":"2024-09-17T02:41:55.869829+0000","last_became_active":"2024-09-17T02:41:55.862501+0000","last_became_peered":"2024-09-17T02:41:55.862501+0000","last_unstale":"2024-09-17T02:41:55.869829+0000","last_undegraded":"2024-09-17T02:41:55.869829+0000","last_fullsized":"2024-09-17T02:41:55.869829+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:16.370567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858000+0000","last_change":"2024-09-17T02:41:56.876227+0000","last_active":"2024-09-17T02:41:57.858000+0000","last_peered":"2024-09-17T02:41:57.858000+0000","last_clean":"2024-09-17T02:41:57.858000+0000","last_became_active":"2024-09-17T02:41:56.875330+0000","last_became_peered":"2024-09-17T02:41:56.875330+0000","last_unstale":"2024-09-17T02:41:57.858000+0000","last_undegraded":"2024-09-17T02:41:57.858000+0000","last_fullsized":"2024-09-17T02:41:57.858000+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:53.391175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.853308+0000","last_change":"2024-09-17T02:41:55.850201+0000","last_active":"2024-09-17T02:41:55.853308+0000","last_peered":"2024-09-17T02:41:55.853308+0000","last_clean":"2024-09-17T02:41:55.853308+0000","last_became_active":"2024-09-17T02:41:55.849734+0000","last_became_peered":"2024-09-17T02:41:55.849734+0000","last_unstale":"2024-09-17T02:41:55.853308+0000","last_undegraded":"2024-09-17T02:41:55.853308+0000","last_fullsized":"2024-09-17T02:41:55.853308+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:44.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.818667+0000","last_change":"2024-09-17T02:41:55.818667+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.818667+0000","last_undegraded":"2024-09-17T02:41:55.818667+0000","last_fullsized":"2024-09-17T02:41:55.818667+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850540+0000","last_change":"2024-09-17T02:41:55.849649+0000","last_active":"2024-09-17T02:41:55.850540+0000","last_peered":"2024-09-17T02:41:55.850540+0000","last_clean":"2024-09-17T02:41:55.850540+0000","last_became_active":"2024-09-17T02:41:55.849237+0000","last_became_peered":"2024-09-17T02:41:55.849237+0000","last_unstale":"2024-09-17T02:41:55.850540+0000","last_undegraded":"2024-09-17T02:41:55.850540+0000","last_fullsized":"2024-09-17T02:41:55.850540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:40.869732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822074+0000","last_change":"2024-09-17T02:41:55.822074+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822074+0000","last_undegraded":"2024-09-17T02:41:55.822074+0000","last_fullsized":"2024-09-17T02:41:55.822074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869389+0000","last_change":"2024-09-17T02:41:55.862706+0000","last_active":"2024-09-17T02:41:55.869389+0000","last_peered":"2024-09-17T02:41:55.869389+0000","last_clean":"2024-09-17T02:41:55.869389+0000","last_became_active":"2024-09-17T02:41:55.862091+0000","last_became_peered":"2024-09-17T02:41:55.862091+0000","last_unstale":"2024-09-17T02:41:55.869389+0000","last_undegraded":"2024-09-17T02:41:55.869389+0000","last_fullsized":"2024-09-17T02:41:55.869389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:16.349899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820910+0000","last_change":"2024-09-17T02:41:55.820910+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820910+0000","last_undegraded":"2024-09-17T02:41:55.820910+0000","last_fullsized":"2024-09-17T02:41:55.820910+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.852959+0000","last_change":"2024-09-17T02:41:55.850095+0000","last_active":"2024-09-17T02:41:55.852959+0000","last_peered":"2024-09-17T02:41:55.852959+0000","last_clean":"2024-09-17T02:41:55.852959+0000","last_became_active":"2024-09-17T02:41:55.849684+0000","last_became_peered":"2024-09-17T02:41:55.849684+0000","last_unstale":"2024-09-17T02:41:55.852959+0000","last_undegraded":"2024-09-17T02:41:55.852959+0000","last_fullsized":"2024-09-17T02:41:55.852959+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:16.370567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":141,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.825082+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:41:55.825082+0000","last_peered":"2024-09-17T02:41:55.825082+0000","last_clean":"2024-09-17T02:41:55.825082+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:41:55.825082+0000","last_undegraded":"2024-09-17T02:41:55.825082+0000","last_fullsized":"2024-09-17T02:41:55.825082+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":765006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":765006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":430080,"data_stored":181305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":195198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:42:01.477 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:42:01.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.672+0000 7fdef5ef2640 1 -- 172.21.15.131:0/1202515587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdef015c5b0 msgr2=0x7fdef00cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.672+0000 7fdef5ef2640 1 --2- 172.21.15.131:0/1202515587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdef015c5b0 0x7fdef00cbaf0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009bc0 tx=0x7fdee801cba0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.672+0000 7fdef5ef2640 1 -- 172.21.15.131:0/1202515587 shutdown_connections 2024-09-17T02:42:01.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.672+0000 7fdef5ef2640 1 --2- 172.21.15.131:0/1202515587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdef015c5b0 0x7fdef00cbaf0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.672+0000 7fdef5ef2640 1 --2- 172.21.15.131:0/1202515587 >> v2:172.21.15.189:3300/0 conn(0x7fdef00ced00 0x7fdef006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.672+0000 7fdef5ef2640 1 -- 172.21.15.131:0/1202515587 >> 172.21.15.131:0/1202515587 conn(0x7fdef006eea0 msgr2=0x7fdef006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:01.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.672+0000 7fdef5ef2640 1 -- 172.21.15.131:0/1202515587 shutdown_connections 2024-09-17T02:42:01.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.672+0000 7fdef5ef2640 1 -- 172.21.15.131:0/1202515587 wait complete. 2024-09-17T02:42:01.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.672+0000 7fdef5ef2640 1 Processor -- start 2024-09-17T02:42:01.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdef5ef2640 1 -- start start 2024-09-17T02:42:01.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdef5ef2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef006e3e0 0x7fdef014c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:01.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdef5ef2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdef00ced00 0x7fdef014e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:01.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdef5ef2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdef015c5b0 0x7fdef00b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:01.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdef5ef2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdef01686c0 con 0x7fdef006e3e0 2024-09-17T02:42:01.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdef5ef2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdef0168540 con 0x7fdef015c5b0 2024-09-17T02:42:01.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdef5ef2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdef0168840 con 0x7fdef00ced00 2024-09-17T02:42:01.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeef7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef006e3e0 0x7fdef014c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:01.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeef7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef006e3e0 0x7fdef014c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57062/0 (socket says 172.21.15.131:57062) 2024-09-17T02:42:01.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeef7fe640 1 -- 172.21.15.131:0/2107132433 learned_addr learned my addr 172.21.15.131:0/2107132433 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeeffff640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdef015c5b0 0x7fdef00b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeeeffd640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdef00ced00 0x7fdef014e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeef7fe640 1 -- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdef00ced00 msgr2=0x7fdef014e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeef7fe640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdef00ced00 0x7fdef014e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeef7fe640 1 -- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdef015c5b0 msgr2=0x7fdef00b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeef7fe640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdef015c5b0 0x7fdef00b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeef7fe640 1 -- 172.21.15.131:0/2107132433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef0153960 con 0x7fdef006e3e0 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeeeffd640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdef00ced00 0x7fdef014e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdeef7fe640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef006e3e0 0x7fdef014c5e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdee000e9a0 tx=0x7fdee000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdee7fff640 1 -- 172.21.15.131:0/2107132433 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee000cd80 con 0x7fdef006e3e0 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdee7fff640 1 -- 172.21.15.131:0/2107132433 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdee0004020 con 0x7fdef006e3e0 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdef5ef2640 1 -- 172.21.15.131:0/2107132433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef0153c00 con 0x7fdef006e3e0 2024-09-17T02:42:01.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdee7fff640 1 -- 172.21.15.131:0/2107132433 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee0004450 con 0x7fdef006e3e0 2024-09-17T02:42:01.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdef5ef2640 1 -- 172.21.15.131:0/2107132433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef01541c0 con 0x7fdef006e3e0 2024-09-17T02:42:01.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdef5ef2640 1 -- 172.21.15.131:0/2107132433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef006dea0 con 0x7fdef006e3e0 2024-09-17T02:42:01.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.676+0000 7fdee7fff640 1 -- 172.21.15.131:0/2107132433 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdee0003600 con 0x7fdef006e3e0 2024-09-17T02:42:01.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.680+0000 7fdee7fff640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fded0073230 0x7fded0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:01.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.680+0000 7fdee7fff640 1 -- 172.21.15.131:0/2107132433 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fdee0094cb0 con 0x7fdef006e3e0 2024-09-17T02:42:01.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.680+0000 7fdeeeffd640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fded0073230 0x7fded0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:01.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.680+0000 7fdeeeffd640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fded0073230 0x7fded0075740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdef00b4fe0 tx=0x7fded8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:01.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.684+0000 7fdee7fff640 1 -- 172.21.15.131:0/2107132433 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdee005fbd0 con 0x7fdef006e3e0 2024-09-17T02:42:01.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.844+0000 7fdef5ef2640 1 -- 172.21.15.131:0/2107132433 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdef00af5f0 con 0x7fded0073230 2024-09-17T02:42:01.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.856+0000 7fdee7fff640 1 -- 172.21.15.131:0/2107132433 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247319 (secure 0 0 0) 0x7fdef00af5f0 con 0x7fded0073230 2024-09-17T02:42:01.859 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:01.863 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:42:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 -- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fded0073230 msgr2=0x7fded0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fded0073230 0x7fded0075740 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdef00b4fe0 tx=0x7fded8009040 comp rx=0 tx=0).stop 2024-09-17T02:42:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 -- 172.21.15.131:0/2107132433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef006e3e0 msgr2=0x7fdef014c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef006e3e0 0x7fdef014c5e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdee000e9a0 tx=0x7fdee000ee60 comp rx=0 tx=0).stop 2024-09-17T02:42:01.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 -- 172.21.15.131:0/2107132433 shutdown_connections 2024-09-17T02:42:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fded0073230 0x7fded0075740 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdef015c5b0 0x7fdef00b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdef00ced00 0x7fdef014e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 --2- 172.21.15.131:0/2107132433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef006e3e0 0x7fdef014c5e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 -- 172.21.15.131:0/2107132433 >> 172.21.15.131:0/2107132433 conn(0x7fdef006eea0 msgr2=0x7fdef015e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 -- 172.21.15.131:0/2107132433 shutdown_connections 2024-09-17T02:42:01.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:01.860+0000 7fdef5ef2640 1 -- 172.21.15.131:0/2107132433 wait complete. 2024-09-17T02:42:01.879 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-17T02:42:01.143735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224868,"kb_used_data":4476,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749852,"statfs":{"total":773094113280,"available":772863848448,"internally_reserved":0,"allocated":4583424,"data_stored":2605067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001879"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.850774+0000","last_change":"2024-09-17T02:41:55.850774+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.850774+0000","last_undegraded":"2024-09-17T02:41:55.850774+0000","last_fullsized":"2024-09-17T02:41:55.850774+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858600+0000","last_change":"2024-09-17T02:41:56.867810+0000","last_active":"2024-09-17T02:41:57.858600+0000","last_peered":"2024-09-17T02:41:57.858600+0000","last_clean":"2024-09-17T02:41:57.858600+0000","last_became_active":"2024-09-17T02:41:56.866944+0000","last_became_peered":"2024-09-17T02:41:56.866944+0000","last_unstale":"2024-09-17T02:41:57.858600+0000","last_undegraded":"2024-09-17T02:41:57.858600+0000","last_fullsized":"2024-09-17T02:41:57.858600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:59.549330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.836800+0000","last_change":"2024-09-17T02:41:55.836800+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.836800+0000","last_undegraded":"2024-09-17T02:41:55.836800+0000","last_fullsized":"2024-09-17T02:41:55.836800+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.838796+0000","last_change":"2024-09-17T02:41:55.838796+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.838796+0000","last_undegraded":"2024-09-17T02:41:55.838796+0000","last_fullsized":"2024-09-17T02:41:55.838796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822660+0000","last_change":"2024-09-17T02:41:55.822660+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822660+0000","last_undegraded":"2024-09-17T02:41:55.822660+0000","last_fullsized":"2024-09-17T02:41:55.822660+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857974+0000","last_change":"2024-09-17T02:41:56.880704+0000","last_active":"2024-09-17T02:41:57.857974+0000","last_peered":"2024-09-17T02:41:57.857974+0000","last_clean":"2024-09-17T02:41:57.857974+0000","last_became_active":"2024-09-17T02:41:56.879945+0000","last_became_peered":"2024-09-17T02:41:56.879945+0000","last_unstale":"2024-09-17T02:41:57.857974+0000","last_undegraded":"2024-09-17T02:41:57.857974+0000","last_fullsized":"2024-09-17T02:41:57.857974+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:36.026817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825753+0000","last_change":"2024-09-17T02:41:55.825753+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825753+0000","last_undegraded":"2024-09-17T02:41:55.825753+0000","last_fullsized":"2024-09-17T02:41:55.825753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.855791+0000","last_change":"2024-09-17T02:41:55.855791+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.855791+0000","last_undegraded":"2024-09-17T02:41:55.855791+0000","last_fullsized":"2024-09-17T02:41:55.855791+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815860+0000","last_change":"2024-09-17T02:41:55.815860+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815860+0000","last_undegraded":"2024-09-17T02:41:55.815860+0000","last_fullsized":"2024-09-17T02:41:55.815860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820570+0000","last_change":"2024-09-17T02:41:55.820570+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820570+0000","last_undegraded":"2024-09-17T02:41:55.820570+0000","last_fullsized":"2024-09-17T02:41:55.820570+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830051+0000","last_change":"2024-09-17T02:41:55.830051+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830051+0000","last_undegraded":"2024-09-17T02:41:55.830051+0000","last_fullsized":"2024-09-17T02:41:55.830051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.817345+0000","last_change":"2024-09-17T02:41:55.817345+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.817345+0000","last_undegraded":"2024-09-17T02:41:55.817345+0000","last_fullsized":"2024-09-17T02:41:55.817345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820760+0000","last_change":"2024-09-17T02:41:55.820760+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820760+0000","last_undegraded":"2024-09-17T02:41:55.820760+0000","last_fullsized":"2024-09-17T02:41:55.820760+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822279+0000","last_change":"2024-09-17T02:41:55.822279+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822279+0000","last_undegraded":"2024-09-17T02:41:55.822279+0000","last_fullsized":"2024-09-17T02:41:55.822279+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.832996+0000","last_change":"2024-09-17T02:41:55.832996+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.832996+0000","last_undegraded":"2024-09-17T02:41:55.832996+0000","last_fullsized":"2024-09-17T02:41:55.832996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821113+0000","last_change":"2024-09-17T02:41:55.821113+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821113+0000","last_undegraded":"2024-09-17T02:41:55.821113+0000","last_fullsized":"2024-09-17T02:41:55.821113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.833655+0000","last_change":"2024-09-17T02:41:55.833655+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.833655+0000","last_undegraded":"2024-09-17T02:41:55.833655+0000","last_fullsized":"2024-09-17T02:41:55.833655+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830989+0000","last_change":"2024-09-17T02:41:55.830989+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830989+0000","last_undegraded":"2024-09-17T02:41:55.830989+0000","last_fullsized":"2024-09-17T02:41:55.830989+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830539+0000","last_change":"2024-09-17T02:41:55.830539+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830539+0000","last_undegraded":"2024-09-17T02:41:55.830539+0000","last_fullsized":"2024-09-17T02:41:55.830539+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.828730+0000","last_change":"2024-09-17T02:41:55.828730+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.828730+0000","last_undegraded":"2024-09-17T02:41:55.828730+0000","last_fullsized":"2024-09-17T02:41:55.828730+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.829387+0000","last_change":"2024-09-17T02:41:55.829387+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.829387+0000","last_undegraded":"2024-09-17T02:41:55.829387+0000","last_fullsized":"2024-09-17T02:41:55.829387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826821+0000","last_change":"2024-09-17T02:41:55.826821+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826821+0000","last_undegraded":"2024-09-17T02:41:55.826821+0000","last_fullsized":"2024-09-17T02:41:55.826821+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.827563+0000","last_change":"2024-09-17T02:41:55.827563+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.827563+0000","last_undegraded":"2024-09-17T02:41:55.827563+0000","last_fullsized":"2024-09-17T02:41:55.827563+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.841871+0000","last_change":"2024-09-17T02:41:55.841871+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.841871+0000","last_undegraded":"2024-09-17T02:41:55.841871+0000","last_fullsized":"2024-09-17T02:41:55.841871+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823236+0000","last_change":"2024-09-17T02:41:55.823236+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823236+0000","last_undegraded":"2024-09-17T02:41:55.823236+0000","last_fullsized":"2024-09-17T02:41:55.823236+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.839845+0000","last_change":"2024-09-17T02:41:55.839845+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.839845+0000","last_undegraded":"2024-09-17T02:41:55.839845+0000","last_fullsized":"2024-09-17T02:41:55.839845+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.824021+0000","last_change":"2024-09-17T02:41:55.824021+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.824021+0000","last_undegraded":"2024-09-17T02:41:55.824021+0000","last_fullsized":"2024-09-17T02:41:55.824021+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.824324+0000","last_change":"2024-09-17T02:41:55.824324+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.824324+0000","last_undegraded":"2024-09-17T02:41:55.824324+0000","last_fullsized":"2024-09-17T02:41:55.824324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.829096+0000","last_change":"2024-09-17T02:41:55.829096+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.829096+0000","last_undegraded":"2024-09-17T02:41:55.829096+0000","last_fullsized":"2024-09-17T02:41:55.829096+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826857+0000","last_change":"2024-09-17T02:41:55.826857+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826857+0000","last_undegraded":"2024-09-17T02:41:55.826857+0000","last_fullsized":"2024-09-17T02:41:55.826857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821650+0000","last_change":"2024-09-17T02:41:55.821650+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821650+0000","last_undegraded":"2024-09-17T02:41:55.821650+0000","last_fullsized":"2024-09-17T02:41:55.821650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.840670+0000","last_change":"2024-09-17T02:41:55.840670+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.840670+0000","last_undegraded":"2024-09-17T02:41:55.840670+0000","last_fullsized":"2024-09-17T02:41:55.840670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819919+0000","last_change":"2024-09-17T02:41:55.819919+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819919+0000","last_undegraded":"2024-09-17T02:41:55.819919+0000","last_fullsized":"2024-09-17T02:41:55.819919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848148+0000","last_change":"2024-09-17T02:41:55.843782+0000","last_active":"2024-09-17T02:41:55.848148+0000","last_peered":"2024-09-17T02:41:55.848148+0000","last_clean":"2024-09-17T02:41:55.848148+0000","last_became_active":"2024-09-17T02:41:55.843362+0000","last_became_peered":"2024-09-17T02:41:55.843362+0000","last_unstale":"2024-09-17T02:41:55.848148+0000","last_undegraded":"2024-09-17T02:41:55.848148+0000","last_fullsized":"2024-09-17T02:41:55.848148+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:39.655039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858515+0000","last_change":"2024-09-17T02:41:56.882518+0000","last_active":"2024-09-17T02:41:57.858515+0000","last_peered":"2024-09-17T02:41:57.858515+0000","last_clean":"2024-09-17T02:41:57.858515+0000","last_became_active":"2024-09-17T02:41:56.879984+0000","last_became_peered":"2024-09-17T02:41:56.879984+0000","last_unstale":"2024-09-17T02:41:57.858515+0000","last_undegraded":"2024-09-17T02:41:57.858515+0000","last_fullsized":"2024-09-17T02:41:57.858515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:35.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858745+0000","last_change":"2024-09-17T02:41:55.844353+0000","last_active":"2024-09-17T02:41:57.858745+0000","last_peered":"2024-09-17T02:41:57.858745+0000","last_clean":"2024-09-17T02:41:57.858745+0000","last_became_active":"2024-09-17T02:41:55.843505+0000","last_became_peered":"2024-09-17T02:41:55.843505+0000","last_unstale":"2024-09-17T02:41:57.858745+0000","last_undegraded":"2024-09-17T02:41:57.858745+0000","last_fullsized":"2024-09-17T02:41:57.858745+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:43.510214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830088+0000","last_change":"2024-09-17T02:41:55.830088+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830088+0000","last_undegraded":"2024-09-17T02:41:55.830088+0000","last_fullsized":"2024-09-17T02:41:55.830088+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869994+0000","last_change":"2024-09-17T02:41:55.865708+0000","last_active":"2024-09-17T02:41:55.869994+0000","last_peered":"2024-09-17T02:41:55.869994+0000","last_clean":"2024-09-17T02:41:55.869994+0000","last_became_active":"2024-09-17T02:41:55.861337+0000","last_became_peered":"2024-09-17T02:41:55.861337+0000","last_unstale":"2024-09-17T02:41:55.869994+0000","last_undegraded":"2024-09-17T02:41:55.869994+0000","last_fullsized":"2024-09-17T02:41:55.869994+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:28:47.106157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.859238+0000","last_change":"2024-09-17T02:41:56.867038+0000","last_active":"2024-09-17T02:41:57.859238+0000","last_peered":"2024-09-17T02:41:57.859238+0000","last_clean":"2024-09-17T02:41:57.859238+0000","last_became_active":"2024-09-17T02:41:56.865986+0000","last_became_peered":"2024-09-17T02:41:56.865986+0000","last_unstale":"2024-09-17T02:41:57.859238+0000","last_undegraded":"2024-09-17T02:41:57.859238+0000","last_fullsized":"2024-09-17T02:41:57.859238+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:31.091078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.862703+0000","last_change":"2024-09-17T02:41:55.859306+0000","last_active":"2024-09-17T02:41:55.862703+0000","last_peered":"2024-09-17T02:41:55.862703+0000","last_clean":"2024-09-17T02:41:55.862703+0000","last_became_active":"2024-09-17T02:41:55.858980+0000","last_became_peered":"2024-09-17T02:41:55.858980+0000","last_unstale":"2024-09-17T02:41:55.862703+0000","last_undegraded":"2024-09-17T02:41:55.862703+0000","last_fullsized":"2024-09-17T02:41:55.862703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:13.459554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.839286+0000","last_change":"2024-09-17T02:41:55.839286+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.839286+0000","last_undegraded":"2024-09-17T02:41:55.839286+0000","last_fullsized":"2024-09-17T02:41:55.839286+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.836960+0000","last_change":"2024-09-17T02:41:55.832205+0000","last_active":"2024-09-17T02:41:55.836960+0000","last_peered":"2024-09-17T02:41:55.836960+0000","last_clean":"2024-09-17T02:41:55.836960+0000","last_became_active":"2024-09-17T02:41:55.831773+0000","last_became_peered":"2024-09-17T02:41:55.831773+0000","last_unstale":"2024-09-17T02:41:55.836960+0000","last_undegraded":"2024-09-17T02:41:55.836960+0000","last_fullsized":"2024-09-17T02:41:55.836960+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:09.662328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822131+0000","last_change":"2024-09-17T02:41:55.822131+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822131+0000","last_undegraded":"2024-09-17T02:41:55.822131+0000","last_fullsized":"2024-09-17T02:41:55.822131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.871586+0000","last_change":"2024-09-17T02:41:55.867371+0000","last_active":"2024-09-17T02:41:55.871586+0000","last_peered":"2024-09-17T02:41:55.871586+0000","last_clean":"2024-09-17T02:41:55.871586+0000","last_became_active":"2024-09-17T02:41:55.866967+0000","last_became_peered":"2024-09-17T02:41:55.866967+0000","last_unstale":"2024-09-17T02:41:55.871586+0000","last_undegraded":"2024-09-17T02:41:55.871586+0000","last_fullsized":"2024-09-17T02:41:55.871586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:08.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821328+0000","last_change":"2024-09-17T02:41:55.821328+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821328+0000","last_undegraded":"2024-09-17T02:41:55.821328+0000","last_fullsized":"2024-09-17T02:41:55.821328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869429+0000","last_change":"2024-09-17T02:41:55.862221+0000","last_active":"2024-09-17T02:41:55.869429+0000","last_peered":"2024-09-17T02:41:55.869429+0000","last_clean":"2024-09-17T02:41:55.869429+0000","last_became_active":"2024-09-17T02:41:55.861013+0000","last_became_peered":"2024-09-17T02:41:55.861013+0000","last_unstale":"2024-09-17T02:41:55.869429+0000","last_undegraded":"2024-09-17T02:41:55.869429+0000","last_fullsized":"2024-09-17T02:41:55.869429+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:45.117350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.859026+0000","last_change":"2024-09-17T02:41:56.875750+0000","last_active":"2024-09-17T02:41:57.859026+0000","last_peered":"2024-09-17T02:41:57.859026+0000","last_clean":"2024-09-17T02:41:57.859026+0000","last_became_active":"2024-09-17T02:41:56.874828+0000","last_became_peered":"2024-09-17T02:41:56.874828+0000","last_unstale":"2024-09-17T02:41:57.859026+0000","last_undegraded":"2024-09-17T02:41:57.859026+0000","last_fullsized":"2024-09-17T02:41:57.859026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:39.693566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.835588+0000","last_change":"2024-09-17T02:41:55.833725+0000","last_active":"2024-09-17T02:41:55.835588+0000","last_peered":"2024-09-17T02:41:55.835588+0000","last_clean":"2024-09-17T02:41:55.835588+0000","last_became_active":"2024-09-17T02:41:55.833225+0000","last_became_peered":"2024-09-17T02:41:55.833225+0000","last_unstale":"2024-09-17T02:41:55.835588+0000","last_undegraded":"2024-09-17T02:41:55.835588+0000","last_fullsized":"2024-09-17T02:41:55.835588+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:19.961178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.837456+0000","last_change":"2024-09-17T02:41:55.837456+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.837456+0000","last_undegraded":"2024-09-17T02:41:55.837456+0000","last_fullsized":"2024-09-17T02:41:55.837456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870175+0000","last_change":"2024-09-17T02:41:55.865849+0000","last_active":"2024-09-17T02:41:55.870175+0000","last_peered":"2024-09-17T02:41:55.870175+0000","last_clean":"2024-09-17T02:41:55.870175+0000","last_became_active":"2024-09-17T02:41:55.863121+0000","last_became_peered":"2024-09-17T02:41:55.863121+0000","last_unstale":"2024-09-17T02:41:55.870175+0000","last_undegraded":"2024-09-17T02:41:55.870175+0000","last_fullsized":"2024-09-17T02:41:55.870175+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:37.028521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822346+0000","last_change":"2024-09-17T02:41:55.822346+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822346+0000","last_undegraded":"2024-09-17T02:41:55.822346+0000","last_fullsized":"2024-09-17T02:41:55.822346+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848532+0000","last_change":"2024-09-17T02:41:55.846533+0000","last_active":"2024-09-17T02:41:55.848532+0000","last_peered":"2024-09-17T02:41:55.848532+0000","last_clean":"2024-09-17T02:41:55.848532+0000","last_became_active":"2024-09-17T02:41:55.846130+0000","last_became_peered":"2024-09-17T02:41:55.846130+0000","last_unstale":"2024-09-17T02:41:55.848532+0000","last_undegraded":"2024-09-17T02:41:55.848532+0000","last_fullsized":"2024-09-17T02:41:55.848532+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:31.071112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825732+0000","last_change":"2024-09-17T02:41:55.825732+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825732+0000","last_undegraded":"2024-09-17T02:41:55.825732+0000","last_fullsized":"2024-09-17T02:41:55.825732+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.871635+0000","last_change":"2024-09-17T02:41:55.866610+0000","last_active":"2024-09-17T02:41:55.871635+0000","last_peered":"2024-09-17T02:41:55.871635+0000","last_clean":"2024-09-17T02:41:55.871635+0000","last_became_active":"2024-09-17T02:41:55.866246+0000","last_became_peered":"2024-09-17T02:41:55.866246+0000","last_unstale":"2024-09-17T02:41:55.871635+0000","last_undegraded":"2024-09-17T02:41:55.871635+0000","last_fullsized":"2024-09-17T02:41:55.871635+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:45.605666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820352+0000","last_change":"2024-09-17T02:41:55.820352+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820352+0000","last_undegraded":"2024-09-17T02:41:55.820352+0000","last_fullsized":"2024-09-17T02:41:55.820352+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869861+0000","last_change":"2024-09-17T02:41:55.863340+0000","last_active":"2024-09-17T02:41:55.869861+0000","last_peered":"2024-09-17T02:41:55.869861+0000","last_clean":"2024-09-17T02:41:55.869861+0000","last_became_active":"2024-09-17T02:41:55.861822+0000","last_became_peered":"2024-09-17T02:41:55.861822+0000","last_unstale":"2024-09-17T02:41:55.869861+0000","last_undegraded":"2024-09-17T02:41:55.869861+0000","last_fullsized":"2024-09-17T02:41:55.869861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:44.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.838002+0000","last_change":"2024-09-17T02:41:55.838002+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.838002+0000","last_undegraded":"2024-09-17T02:41:55.838002+0000","last_fullsized":"2024-09-17T02:41:55.838002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870109+0000","last_change":"2024-09-17T02:41:55.866575+0000","last_active":"2024-09-17T02:41:55.870109+0000","last_peered":"2024-09-17T02:41:55.870109+0000","last_clean":"2024-09-17T02:41:55.870109+0000","last_became_active":"2024-09-17T02:41:55.862842+0000","last_became_peered":"2024-09-17T02:41:55.862842+0000","last_unstale":"2024-09-17T02:41:55.870109+0000","last_undegraded":"2024-09-17T02:41:55.870109+0000","last_fullsized":"2024-09-17T02:41:55.870109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:46.934262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857825+0000","last_change":"2024-09-17T02:41:56.881999+0000","last_active":"2024-09-17T02:41:57.857825+0000","last_peered":"2024-09-17T02:41:57.857825+0000","last_clean":"2024-09-17T02:41:57.857825+0000","last_became_active":"2024-09-17T02:41:56.881115+0000","last_became_peered":"2024-09-17T02:41:56.881115+0000","last_unstale":"2024-09-17T02:41:57.857825+0000","last_undegraded":"2024-09-17T02:41:57.857825+0000","last_fullsized":"2024-09-17T02:41:57.857825+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:35.710278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857830+0000","last_change":"2024-09-17T02:41:55.839604+0000","last_active":"2024-09-17T02:41:57.857830+0000","last_peered":"2024-09-17T02:41:57.857830+0000","last_clean":"2024-09-17T02:41:57.857830+0000","last_became_active":"2024-09-17T02:41:55.835167+0000","last_became_peered":"2024-09-17T02:41:55.835167+0000","last_unstale":"2024-09-17T02:41:57.857830+0000","last_undegraded":"2024-09-17T02:41:57.857830+0000","last_fullsized":"2024-09-17T02:41:57.857830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:44.487692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823496+0000","last_change":"2024-09-17T02:41:55.823496+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823496+0000","last_undegraded":"2024-09-17T02:41:55.823496+0000","last_fullsized":"2024-09-17T02:41:55.823496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.840993+0000","last_change":"2024-09-17T02:41:55.830025+0000","last_active":"2024-09-17T02:41:55.840993+0000","last_peered":"2024-09-17T02:41:55.840993+0000","last_clean":"2024-09-17T02:41:55.840993+0000","last_became_active":"2024-09-17T02:41:55.829641+0000","last_became_peered":"2024-09-17T02:41:55.829641+0000","last_unstale":"2024-09-17T02:41:55.840993+0000","last_undegraded":"2024-09-17T02:41:55.840993+0000","last_fullsized":"2024-09-17T02:41:55.840993+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:33.231862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815645+0000","last_change":"2024-09-17T02:41:55.815645+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815645+0000","last_undegraded":"2024-09-17T02:41:55.815645+0000","last_fullsized":"2024-09-17T02:41:55.815645+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.830119+0000","last_change":"2024-09-17T02:41:55.824740+0000","last_active":"2024-09-17T02:41:55.830119+0000","last_peered":"2024-09-17T02:41:55.830119+0000","last_clean":"2024-09-17T02:41:55.830119+0000","last_became_active":"2024-09-17T02:41:55.824315+0000","last_became_peered":"2024-09-17T02:41:55.824315+0000","last_unstale":"2024-09-17T02:41:55.830119+0000","last_undegraded":"2024-09-17T02:41:55.830119+0000","last_fullsized":"2024-09-17T02:41:55.830119+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:20.013546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858509+0000","last_change":"2024-09-17T02:41:56.881490+0000","last_active":"2024-09-17T02:41:57.858509+0000","last_peered":"2024-09-17T02:41:57.858509+0000","last_clean":"2024-09-17T02:41:57.858509+0000","last_became_active":"2024-09-17T02:41:56.880621+0000","last_became_peered":"2024-09-17T02:41:56.880621+0000","last_unstale":"2024-09-17T02:41:57.858509+0000","last_undegraded":"2024-09-17T02:41:57.858509+0000","last_fullsized":"2024-09-17T02:41:57.858509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:27.688857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.835545+0000","last_change":"2024-09-17T02:41:55.833534+0000","last_active":"2024-09-17T02:41:55.835545+0000","last_peered":"2024-09-17T02:41:55.835545+0000","last_clean":"2024-09-17T02:41:55.835545+0000","last_became_active":"2024-09-17T02:41:55.833218+0000","last_became_peered":"2024-09-17T02:41:55.833218+0000","last_unstale":"2024-09-17T02:41:55.835545+0000","last_undegraded":"2024-09-17T02:41:55.835545+0000","last_fullsized":"2024-09-17T02:41:55.835545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:57.836648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821387+0000","last_change":"2024-09-17T02:41:55.821387+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821387+0000","last_undegraded":"2024-09-17T02:41:55.821387+0000","last_fullsized":"2024-09-17T02:41:55.821387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.863199+0000","last_change":"2024-09-17T02:41:55.861548+0000","last_active":"2024-09-17T02:41:55.863199+0000","last_peered":"2024-09-17T02:41:55.863199+0000","last_clean":"2024-09-17T02:41:55.863199+0000","last_became_active":"2024-09-17T02:41:55.861200+0000","last_became_peered":"2024-09-17T02:41:55.861200+0000","last_unstale":"2024-09-17T02:41:55.863199+0000","last_undegraded":"2024-09-17T02:41:55.863199+0000","last_fullsized":"2024-09-17T02:41:55.863199+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:08.387963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819800+0000","last_change":"2024-09-17T02:41:55.819800+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819800+0000","last_undegraded":"2024-09-17T02:41:55.819800+0000","last_fullsized":"2024-09-17T02:41:55.819800+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850416+0000","last_change":"2024-09-17T02:41:55.844139+0000","last_active":"2024-09-17T02:41:55.850416+0000","last_peered":"2024-09-17T02:41:55.850416+0000","last_clean":"2024-09-17T02:41:55.850416+0000","last_became_active":"2024-09-17T02:41:55.842953+0000","last_became_peered":"2024-09-17T02:41:55.842953+0000","last_unstale":"2024-09-17T02:41:55.850416+0000","last_undegraded":"2024-09-17T02:41:55.850416+0000","last_fullsized":"2024-09-17T02:41:55.850416+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:57.710104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.833247+0000","last_change":"2024-09-17T02:41:55.833247+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.833247+0000","last_undegraded":"2024-09-17T02:41:55.833247+0000","last_fullsized":"2024-09-17T02:41:55.833247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.845102+0000","last_change":"2024-09-17T02:41:55.840256+0000","last_active":"2024-09-17T02:41:55.845102+0000","last_peered":"2024-09-17T02:41:55.845102+0000","last_clean":"2024-09-17T02:41:55.845102+0000","last_became_active":"2024-09-17T02:41:55.839277+0000","last_became_peered":"2024-09-17T02:41:55.839277+0000","last_unstale":"2024-09-17T02:41:55.845102+0000","last_undegraded":"2024-09-17T02:41:55.845102+0000","last_fullsized":"2024-09-17T02:41:55.845102+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:06:51.053427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819364+0000","last_change":"2024-09-17T02:41:55.819364+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819364+0000","last_undegraded":"2024-09-17T02:41:55.819364+0000","last_fullsized":"2024-09-17T02:41:55.819364+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848889+0000","last_change":"2024-09-17T02:41:55.846024+0000","last_active":"2024-09-17T02:41:55.848889+0000","last_peered":"2024-09-17T02:41:55.848889+0000","last_clean":"2024-09-17T02:41:55.848889+0000","last_became_active":"2024-09-17T02:41:55.845095+0000","last_became_peered":"2024-09-17T02:41:55.845095+0000","last_unstale":"2024-09-17T02:41:55.848889+0000","last_undegraded":"2024-09-17T02:41:55.848889+0000","last_fullsized":"2024-09-17T02:41:55.848889+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:51.113568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825036+0000","last_change":"2024-09-17T02:41:55.825036+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825036+0000","last_undegraded":"2024-09-17T02:41:55.825036+0000","last_fullsized":"2024-09-17T02:41:55.825036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858145+0000","last_change":"2024-09-17T02:41:55.838665+0000","last_active":"2024-09-17T02:41:57.858145+0000","last_peered":"2024-09-17T02:41:57.858145+0000","last_clean":"2024-09-17T02:41:57.858145+0000","last_became_active":"2024-09-17T02:41:55.837817+0000","last_became_peered":"2024-09-17T02:41:55.837817+0000","last_unstale":"2024-09-17T02:41:57.858145+0000","last_undegraded":"2024-09-17T02:41:57.858145+0000","last_fullsized":"2024-09-17T02:41:57.858145+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:05.524969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826200+0000","last_change":"2024-09-17T02:41:55.826200+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826200+0000","last_undegraded":"2024-09-17T02:41:55.826200+0000","last_fullsized":"2024-09-17T02:41:55.826200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.836148+0000","last_change":"2024-09-17T02:41:55.833701+0000","last_active":"2024-09-17T02:41:55.836148+0000","last_peered":"2024-09-17T02:41:55.836148+0000","last_clean":"2024-09-17T02:41:55.836148+0000","last_became_active":"2024-09-17T02:41:55.833254+0000","last_became_peered":"2024-09-17T02:41:55.833254+0000","last_unstale":"2024-09-17T02:41:55.836148+0000","last_undegraded":"2024-09-17T02:41:55.836148+0000","last_fullsized":"2024-09-17T02:41:55.836148+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:51.336142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826130+0000","last_change":"2024-09-17T02:41:55.826130+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826130+0000","last_undegraded":"2024-09-17T02:41:55.826130+0000","last_fullsized":"2024-09-17T02:41:55.826130+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857048+0000","last_change":"2024-09-17T02:41:55.863410+0000","last_active":"2024-09-17T02:41:57.857048+0000","last_peered":"2024-09-17T02:41:57.857048+0000","last_clean":"2024-09-17T02:41:57.857048+0000","last_became_active":"2024-09-17T02:41:55.862647+0000","last_became_peered":"2024-09-17T02:41:55.862647+0000","last_unstale":"2024-09-17T02:41:57.857048+0000","last_undegraded":"2024-09-17T02:41:57.857048+0000","last_fullsized":"2024-09-17T02:41:57.857048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:25.110212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815903+0000","last_change":"2024-09-17T02:41:55.815903+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815903+0000","last_undegraded":"2024-09-17T02:41:55.815903+0000","last_fullsized":"2024-09-17T02:41:55.815903+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850489+0000","last_change":"2024-09-17T02:41:55.849654+0000","last_active":"2024-09-17T02:41:55.850489+0000","last_peered":"2024-09-17T02:41:55.850489+0000","last_clean":"2024-09-17T02:41:55.850489+0000","last_became_active":"2024-09-17T02:41:55.849235+0000","last_became_peered":"2024-09-17T02:41:55.849235+0000","last_unstale":"2024-09-17T02:41:55.850489+0000","last_undegraded":"2024-09-17T02:41:55.850489+0000","last_fullsized":"2024-09-17T02:41:55.850489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:34.982736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823333+0000","last_change":"2024-09-17T02:41:55.823333+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823333+0000","last_undegraded":"2024-09-17T02:41:55.823333+0000","last_fullsized":"2024-09-17T02:41:55.823333+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.852875+0000","last_change":"2024-09-17T02:41:55.843817+0000","last_active":"2024-09-17T02:41:55.852875+0000","last_peered":"2024-09-17T02:41:55.852875+0000","last_clean":"2024-09-17T02:41:55.852875+0000","last_became_active":"2024-09-17T02:41:55.841737+0000","last_became_peered":"2024-09-17T02:41:55.841737+0000","last_unstale":"2024-09-17T02:41:55.852875+0000","last_undegraded":"2024-09-17T02:41:55.852875+0000","last_fullsized":"2024-09-17T02:41:55.852875+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:31.493951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.828622+0000","last_change":"2024-09-17T02:41:55.828622+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.828622+0000","last_undegraded":"2024-09-17T02:41:55.828622+0000","last_fullsized":"2024-09-17T02:41:55.828622+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870027+0000","last_change":"2024-09-17T02:41:55.868286+0000","last_active":"2024-09-17T02:41:55.870027+0000","last_peered":"2024-09-17T02:41:55.870027+0000","last_clean":"2024-09-17T02:41:55.870027+0000","last_became_active":"2024-09-17T02:41:55.867258+0000","last_became_peered":"2024-09-17T02:41:55.867258+0000","last_unstale":"2024-09-17T02:41:55.870027+0000","last_undegraded":"2024-09-17T02:41:55.870027+0000","last_fullsized":"2024-09-17T02:41:55.870027+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:03.928821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815674+0000","last_change":"2024-09-17T02:41:55.815674+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815674+0000","last_undegraded":"2024-09-17T02:41:55.815674+0000","last_fullsized":"2024-09-17T02:41:55.815674+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869829+0000","last_change":"2024-09-17T02:41:55.863138+0000","last_active":"2024-09-17T02:41:55.869829+0000","last_peered":"2024-09-17T02:41:55.869829+0000","last_clean":"2024-09-17T02:41:55.869829+0000","last_became_active":"2024-09-17T02:41:55.862501+0000","last_became_peered":"2024-09-17T02:41:55.862501+0000","last_unstale":"2024-09-17T02:41:55.869829+0000","last_undegraded":"2024-09-17T02:41:55.869829+0000","last_fullsized":"2024-09-17T02:41:55.869829+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:16.370567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858000+0000","last_change":"2024-09-17T02:41:56.876227+0000","last_active":"2024-09-17T02:41:57.858000+0000","last_peered":"2024-09-17T02:41:57.858000+0000","last_clean":"2024-09-17T02:41:57.858000+0000","last_became_active":"2024-09-17T02:41:56.875330+0000","last_became_peered":"2024-09-17T02:41:56.875330+0000","last_unstale":"2024-09-17T02:41:57.858000+0000","last_undegraded":"2024-09-17T02:41:57.858000+0000","last_fullsized":"2024-09-17T02:41:57.858000+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:53.391175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.853308+0000","last_change":"2024-09-17T02:41:55.850201+0000","last_active":"2024-09-17T02:41:55.853308+0000","last_peered":"2024-09-17T02:41:55.853308+0000","last_clean":"2024-09-17T02:41:55.853308+0000","last_became_active":"2024-09-17T02:41:55.849734+0000","last_became_peered":"2024-09-17T02:41:55.849734+0000","last_unstale":"2024-09-17T02:41:55.853308+0000","last_undegraded":"2024-09-17T02:41:55.853308+0000","last_fullsized":"2024-09-17T02:41:55.853308+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:44.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.818667+0000","last_change":"2024-09-17T02:41:55.818667+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.818667+0000","last_undegraded":"2024-09-17T02:41:55.818667+0000","last_fullsized":"2024-09-17T02:41:55.818667+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850540+0000","last_change":"2024-09-17T02:41:55.849649+0000","last_active":"2024-09-17T02:41:55.850540+0000","last_peered":"2024-09-17T02:41:55.850540+0000","last_clean":"2024-09-17T02:41:55.850540+0000","last_became_active":"2024-09-17T02:41:55.849237+0000","last_became_peered":"2024-09-17T02:41:55.849237+0000","last_unstale":"2024-09-17T02:41:55.850540+0000","last_undegraded":"2024-09-17T02:41:55.850540+0000","last_fullsized":"2024-09-17T02:41:55.850540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:40.869732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822074+0000","last_change":"2024-09-17T02:41:55.822074+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822074+0000","last_undegraded":"2024-09-17T02:41:55.822074+0000","last_fullsized":"2024-09-17T02:41:55.822074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869389+0000","last_change":"2024-09-17T02:41:55.862706+0000","last_active":"2024-09-17T02:41:55.869389+0000","last_peered":"2024-09-17T02:41:55.869389+0000","last_clean":"2024-09-17T02:41:55.869389+0000","last_became_active":"2024-09-17T02:41:55.862091+0000","last_became_peered":"2024-09-17T02:41:55.862091+0000","last_unstale":"2024-09-17T02:41:55.869389+0000","last_undegraded":"2024-09-17T02:41:55.869389+0000","last_fullsized":"2024-09-17T02:41:55.869389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:16.349899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820910+0000","last_change":"2024-09-17T02:41:55.820910+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820910+0000","last_undegraded":"2024-09-17T02:41:55.820910+0000","last_fullsized":"2024-09-17T02:41:55.820910+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.852959+0000","last_change":"2024-09-17T02:41:55.850095+0000","last_active":"2024-09-17T02:41:55.852959+0000","last_peered":"2024-09-17T02:41:55.852959+0000","last_clean":"2024-09-17T02:41:55.852959+0000","last_became_active":"2024-09-17T02:41:55.849684+0000","last_became_peered":"2024-09-17T02:41:55.849684+0000","last_unstale":"2024-09-17T02:41:55.852959+0000","last_undegraded":"2024-09-17T02:41:55.852959+0000","last_fullsized":"2024-09-17T02:41:55.852959+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:16.370567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":141,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.825082+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:41:55.825082+0000","last_peered":"2024-09-17T02:41:55.825082+0000","last_clean":"2024-09-17T02:41:55.825082+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:41:55.825082+0000","last_undegraded":"2024-09-17T02:41:55.825082+0000","last_fullsized":"2024-09-17T02:41:55.825082+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":765006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":765006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":430080,"data_stored":181305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":195198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:42:01.883 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3f is not active+clean 2024-09-17T02:42:01.883 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.850774+0000', 'last_change': '2024-09-17T02:41:55.850774+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.850774+0000', 'last_undegraded': '2024-09-17T02:41:55.850774+0000', 'last_fullsized': '2024-09-17T02:41:55.850774+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.883 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3d is not active+clean 2024-09-17T02:42:01.884 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.836800+0000', 'last_change': '2024-09-17T02:41:55.836800+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.836800+0000', 'last_undegraded': '2024-09-17T02:41:55.836800+0000', 'last_fullsized': '2024-09-17T02:41:55.836800+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.884 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3c is not active+clean 2024-09-17T02:42:01.884 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.838796+0000', 'last_change': '2024-09-17T02:41:55.838796+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.838796+0000', 'last_undegraded': '2024-09-17T02:41:55.838796+0000', 'last_fullsized': '2024-09-17T02:41:55.838796+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.884 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3b is not active+clean 2024-09-17T02:42:01.884 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.822660+0000', 'last_change': '2024-09-17T02:41:55.822660+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.822660+0000', 'last_undegraded': '2024-09-17T02:41:55.822660+0000', 'last_fullsized': '2024-09-17T02:41:55.822660+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T02:42:01.884 INFO:tasks.ceph.ceph_manager.ceph:PG 6.39 is not active+clean 2024-09-17T02:42:01.884 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.825753+0000', 'last_change': '2024-09-17T02:41:55.825753+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.825753+0000', 'last_undegraded': '2024-09-17T02:41:55.825753+0000', 'last_fullsized': '2024-09-17T02:41:55.825753+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:42:01.884 INFO:tasks.ceph.ceph_manager.ceph:PG 6.38 is not active+clean 2024-09-17T02:42:01.884 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.855791+0000', 'last_change': '2024-09-17T02:41:55.855791+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.855791+0000', 'last_undegraded': '2024-09-17T02:41:55.855791+0000', 'last_fullsized': '2024-09-17T02:41:55.855791+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.884 INFO:tasks.ceph.ceph_manager.ceph:PG 6.37 is not active+clean 2024-09-17T02:42:01.885 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.815860+0000', 'last_change': '2024-09-17T02:41:55.815860+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.815860+0000', 'last_undegraded': '2024-09-17T02:41:55.815860+0000', 'last_fullsized': '2024-09-17T02:41:55.815860+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T02:42:01.885 INFO:tasks.ceph.ceph_manager.ceph:PG 6.36 is not active+clean 2024-09-17T02:42:01.885 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.820570+0000', 'last_change': '2024-09-17T02:41:55.820570+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.820570+0000', 'last_undegraded': '2024-09-17T02:41:55.820570+0000', 'last_fullsized': '2024-09-17T02:41:55.820570+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:42:01.885 INFO:tasks.ceph.ceph_manager.ceph:PG 6.35 is not active+clean 2024-09-17T02:42:01.885 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.830051+0000', 'last_change': '2024-09-17T02:41:55.830051+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.830051+0000', 'last_undegraded': '2024-09-17T02:41:55.830051+0000', 'last_fullsized': '2024-09-17T02:41:55.830051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:42:01.885 INFO:tasks.ceph.ceph_manager.ceph:PG 6.34 is not active+clean 2024-09-17T02:42:01.885 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.817345+0000', 'last_change': '2024-09-17T02:41:55.817345+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.817345+0000', 'last_undegraded': '2024-09-17T02:41:55.817345+0000', 'last_fullsized': '2024-09-17T02:41:55.817345+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T02:42:01.885 INFO:tasks.ceph.ceph_manager.ceph:PG 6.33 is not active+clean 2024-09-17T02:42:01.885 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.820760+0000', 'last_change': '2024-09-17T02:41:55.820760+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.820760+0000', 'last_undegraded': '2024-09-17T02:41:55.820760+0000', 'last_fullsized': '2024-09-17T02:41:55.820760+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:42:01.885 INFO:tasks.ceph.ceph_manager.ceph:PG 6.32 is not active+clean 2024-09-17T02:42:01.885 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.822279+0000', 'last_change': '2024-09-17T02:41:55.822279+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.822279+0000', 'last_undegraded': '2024-09-17T02:41:55.822279+0000', 'last_fullsized': '2024-09-17T02:41:55.822279+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T02:42:01.886 INFO:tasks.ceph.ceph_manager.ceph:PG 6.31 is not active+clean 2024-09-17T02:42:01.886 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.832996+0000', 'last_change': '2024-09-17T02:41:55.832996+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.832996+0000', 'last_undegraded': '2024-09-17T02:41:55.832996+0000', 'last_fullsized': '2024-09-17T02:41:55.832996+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.886 INFO:tasks.ceph.ceph_manager.ceph:PG 6.30 is not active+clean 2024-09-17T02:42:01.886 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.821113+0000', 'last_change': '2024-09-17T02:41:55.821113+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.821113+0000', 'last_undegraded': '2024-09-17T02:41:55.821113+0000', 'last_fullsized': '2024-09-17T02:41:55.821113+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:42:01.886 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2f is not active+clean 2024-09-17T02:42:01.886 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.833655+0000', 'last_change': '2024-09-17T02:41:55.833655+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.833655+0000', 'last_undegraded': '2024-09-17T02:41:55.833655+0000', 'last_fullsized': '2024-09-17T02:41:55.833655+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.886 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2e is not active+clean 2024-09-17T02:42:01.886 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.830989+0000', 'last_change': '2024-09-17T02:41:55.830989+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.830989+0000', 'last_undegraded': '2024-09-17T02:41:55.830989+0000', 'last_fullsized': '2024-09-17T02:41:55.830989+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.886 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2d is not active+clean 2024-09-17T02:42:01.886 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.830539+0000', 'last_change': '2024-09-17T02:41:55.830539+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.830539+0000', 'last_undegraded': '2024-09-17T02:41:55.830539+0000', 'last_fullsized': '2024-09-17T02:41:55.830539+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T02:42:01.887 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2c is not active+clean 2024-09-17T02:42:01.887 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.828730+0000', 'last_change': '2024-09-17T02:41:55.828730+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.828730+0000', 'last_undegraded': '2024-09-17T02:41:55.828730+0000', 'last_fullsized': '2024-09-17T02:41:55.828730+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:42:01.887 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2b is not active+clean 2024-09-17T02:42:01.887 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.829387+0000', 'last_change': '2024-09-17T02:41:55.829387+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.829387+0000', 'last_undegraded': '2024-09-17T02:41:55.829387+0000', 'last_fullsized': '2024-09-17T02:41:55.829387+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:42:01.887 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2a is not active+clean 2024-09-17T02:42:01.887 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.826821+0000', 'last_change': '2024-09-17T02:41:55.826821+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.826821+0000', 'last_undegraded': '2024-09-17T02:41:55.826821+0000', 'last_fullsized': '2024-09-17T02:41:55.826821+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:42:01.887 INFO:tasks.ceph.ceph_manager.ceph:PG 6.29 is not active+clean 2024-09-17T02:42:01.887 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.827563+0000', 'last_change': '2024-09-17T02:41:55.827563+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.827563+0000', 'last_undegraded': '2024-09-17T02:41:55.827563+0000', 'last_fullsized': '2024-09-17T02:41:55.827563+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.887 INFO:tasks.ceph.ceph_manager.ceph:PG 6.28 is not active+clean 2024-09-17T02:42:01.887 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.841871+0000', 'last_change': '2024-09-17T02:41:55.841871+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.841871+0000', 'last_undegraded': '2024-09-17T02:41:55.841871+0000', 'last_fullsized': '2024-09-17T02:41:55.841871+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.888 INFO:tasks.ceph.ceph_manager.ceph:PG 6.27 is not active+clean 2024-09-17T02:42:01.888 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.823236+0000', 'last_change': '2024-09-17T02:41:55.823236+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.823236+0000', 'last_undegraded': '2024-09-17T02:41:55.823236+0000', 'last_fullsized': '2024-09-17T02:41:55.823236+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T02:42:01.888 INFO:tasks.ceph.ceph_manager.ceph:PG 6.26 is not active+clean 2024-09-17T02:42:01.888 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.839845+0000', 'last_change': '2024-09-17T02:41:55.839845+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.839845+0000', 'last_undegraded': '2024-09-17T02:41:55.839845+0000', 'last_fullsized': '2024-09-17T02:41:55.839845+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.888 INFO:tasks.ceph.ceph_manager.ceph:PG 6.25 is not active+clean 2024-09-17T02:42:01.888 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.824021+0000', 'last_change': '2024-09-17T02:41:55.824021+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.824021+0000', 'last_undegraded': '2024-09-17T02:41:55.824021+0000', 'last_fullsized': '2024-09-17T02:41:55.824021+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:42:01.888 INFO:tasks.ceph.ceph_manager.ceph:PG 6.24 is not active+clean 2024-09-17T02:42:01.888 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.824324+0000', 'last_change': '2024-09-17T02:41:55.824324+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.824324+0000', 'last_undegraded': '2024-09-17T02:41:55.824324+0000', 'last_fullsized': '2024-09-17T02:41:55.824324+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:42:01.888 INFO:tasks.ceph.ceph_manager.ceph:PG 6.23 is not active+clean 2024-09-17T02:42:01.889 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.829096+0000', 'last_change': '2024-09-17T02:41:55.829096+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.829096+0000', 'last_undegraded': '2024-09-17T02:41:55.829096+0000', 'last_fullsized': '2024-09-17T02:41:55.829096+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.889 INFO:tasks.ceph.ceph_manager.ceph:PG 6.22 is not active+clean 2024-09-17T02:42:01.889 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.826857+0000', 'last_change': '2024-09-17T02:41:55.826857+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.826857+0000', 'last_undegraded': '2024-09-17T02:41:55.826857+0000', 'last_fullsized': '2024-09-17T02:41:55.826857+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.889 INFO:tasks.ceph.ceph_manager.ceph:PG 6.21 is not active+clean 2024-09-17T02:42:01.889 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.821650+0000', 'last_change': '2024-09-17T02:41:55.821650+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.821650+0000', 'last_undegraded': '2024-09-17T02:41:55.821650+0000', 'last_fullsized': '2024-09-17T02:41:55.821650+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:42:01.889 INFO:tasks.ceph.ceph_manager.ceph:PG 6.20 is not active+clean 2024-09-17T02:42:01.889 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.840670+0000', 'last_change': '2024-09-17T02:41:55.840670+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.840670+0000', 'last_undegraded': '2024-09-17T02:41:55.840670+0000', 'last_fullsized': '2024-09-17T02:41:55.840670+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.889 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-09-17T02:42:01.889 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.819919+0000', 'last_change': '2024-09-17T02:41:55.819919+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.819919+0000', 'last_undegraded': '2024-09-17T02:41:55.819919+0000', 'last_fullsized': '2024-09-17T02:41:55.819919+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.889 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1e is not active+clean 2024-09-17T02:42:01.890 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.830088+0000', 'last_change': '2024-09-17T02:41:55.830088+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.830088+0000', 'last_undegraded': '2024-09-17T02:41:55.830088+0000', 'last_fullsized': '2024-09-17T02:41:55.830088+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.890 INFO:tasks.ceph.ceph_manager.ceph:PG 6.18 is not active+clean 2024-09-17T02:42:01.890 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.839286+0000', 'last_change': '2024-09-17T02:41:55.839286+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.839286+0000', 'last_undegraded': '2024-09-17T02:41:55.839286+0000', 'last_fullsized': '2024-09-17T02:41:55.839286+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.890 INFO:tasks.ceph.ceph_manager.ceph:PG 6.19 is not active+clean 2024-09-17T02:42:01.890 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.822131+0000', 'last_change': '2024-09-17T02:41:55.822131+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.822131+0000', 'last_undegraded': '2024-09-17T02:41:55.822131+0000', 'last_fullsized': '2024-09-17T02:41:55.822131+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:42:01.890 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1a is not active+clean 2024-09-17T02:42:01.890 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.821328+0000', 'last_change': '2024-09-17T02:41:55.821328+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.821328+0000', 'last_undegraded': '2024-09-17T02:41:55.821328+0000', 'last_fullsized': '2024-09-17T02:41:55.821328+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.890 INFO:tasks.ceph.ceph_manager.ceph:PG 6.14 is not active+clean 2024-09-17T02:42:01.890 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.837456+0000', 'last_change': '2024-09-17T02:41:55.837456+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.837456+0000', 'last_undegraded': '2024-09-17T02:41:55.837456+0000', 'last_fullsized': '2024-09-17T02:41:55.837456+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.890 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-09-17T02:42:01.890 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.822346+0000', 'last_change': '2024-09-17T02:41:55.822346+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.822346+0000', 'last_undegraded': '2024-09-17T02:41:55.822346+0000', 'last_fullsized': '2024-09-17T02:41:55.822346+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T02:42:01.891 INFO:tasks.ceph.ceph_manager.ceph:PG 6.16 is not active+clean 2024-09-17T02:42:01.891 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.825732+0000', 'last_change': '2024-09-17T02:41:55.825732+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.825732+0000', 'last_undegraded': '2024-09-17T02:41:55.825732+0000', 'last_fullsized': '2024-09-17T02:41:55.825732+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.891 INFO:tasks.ceph.ceph_manager.ceph:PG 6.17 is not active+clean 2024-09-17T02:42:01.891 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.820352+0000', 'last_change': '2024-09-17T02:41:55.820352+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.820352+0000', 'last_undegraded': '2024-09-17T02:41:55.820352+0000', 'last_fullsized': '2024-09-17T02:41:55.820352+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:42:01.891 INFO:tasks.ceph.ceph_manager.ceph:PG 6.10 is not active+clean 2024-09-17T02:42:01.891 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.838002+0000', 'last_change': '2024-09-17T02:41:55.838002+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.838002+0000', 'last_undegraded': '2024-09-17T02:41:55.838002+0000', 'last_fullsized': '2024-09-17T02:41:55.838002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.891 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-09-17T02:42:01.891 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.823496+0000', 'last_change': '2024-09-17T02:41:55.823496+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.823496+0000', 'last_undegraded': '2024-09-17T02:41:55.823496+0000', 'last_fullsized': '2024-09-17T02:41:55.823496+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:42:01.891 INFO:tasks.ceph.ceph_manager.ceph:PG 6.13 is not active+clean 2024-09-17T02:42:01.891 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.815645+0000', 'last_change': '2024-09-17T02:41:55.815645+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.815645+0000', 'last_undegraded': '2024-09-17T02:41:55.815645+0000', 'last_fullsized': '2024-09-17T02:41:55.815645+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T02:42:01.892 INFO:tasks.ceph.ceph_manager.ceph:PG 6.d is not active+clean 2024-09-17T02:42:01.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.821387+0000', 'last_change': '2024-09-17T02:41:55.821387+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.821387+0000', 'last_undegraded': '2024-09-17T02:41:55.821387+0000', 'last_fullsized': '2024-09-17T02:41:55.821387+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:42:01.892 INFO:tasks.ceph.ceph_manager.ceph:PG 6.e is not active+clean 2024-09-17T02:42:01.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.819800+0000', 'last_change': '2024-09-17T02:41:55.819800+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.819800+0000', 'last_undegraded': '2024-09-17T02:41:55.819800+0000', 'last_fullsized': '2024-09-17T02:41:55.819800+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:42:01.892 INFO:tasks.ceph.ceph_manager.ceph:PG 6.f is not active+clean 2024-09-17T02:42:01.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.833247+0000', 'last_change': '2024-09-17T02:41:55.833247+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.833247+0000', 'last_undegraded': '2024-09-17T02:41:55.833247+0000', 'last_fullsized': '2024-09-17T02:41:55.833247+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T02:42:01.892 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-09-17T02:42:01.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.819364+0000', 'last_change': '2024-09-17T02:41:55.819364+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.819364+0000', 'last_undegraded': '2024-09-17T02:41:55.819364+0000', 'last_fullsized': '2024-09-17T02:41:55.819364+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.892 INFO:tasks.ceph.ceph_manager.ceph:PG 6.9 is not active+clean 2024-09-17T02:42:01.892 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.825036+0000', 'last_change': '2024-09-17T02:41:55.825036+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.825036+0000', 'last_undegraded': '2024-09-17T02:41:55.825036+0000', 'last_fullsized': '2024-09-17T02:41:55.825036+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.893 INFO:tasks.ceph.ceph_manager.ceph:PG 6.a is not active+clean 2024-09-17T02:42:01.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.826200+0000', 'last_change': '2024-09-17T02:41:55.826200+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.826200+0000', 'last_undegraded': '2024-09-17T02:41:55.826200+0000', 'last_fullsized': '2024-09-17T02:41:55.826200+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:42:01.893 INFO:tasks.ceph.ceph_manager.ceph:PG 6.b is not active+clean 2024-09-17T02:42:01.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.826130+0000', 'last_change': '2024-09-17T02:41:55.826130+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.826130+0000', 'last_undegraded': '2024-09-17T02:41:55.826130+0000', 'last_fullsized': '2024-09-17T02:41:55.826130+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T02:42:01.893 INFO:tasks.ceph.ceph_manager.ceph:PG 6.4 is not active+clean 2024-09-17T02:42:01.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.815903+0000', 'last_change': '2024-09-17T02:41:55.815903+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.815903+0000', 'last_undegraded': '2024-09-17T02:41:55.815903+0000', 'last_fullsized': '2024-09-17T02:41:55.815903+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T02:42:01.893 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-09-17T02:42:01.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.823333+0000', 'last_change': '2024-09-17T02:41:55.823333+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.823333+0000', 'last_undegraded': '2024-09-17T02:41:55.823333+0000', 'last_fullsized': '2024-09-17T02:41:55.823333+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:42:01.893 INFO:tasks.ceph.ceph_manager.ceph:PG 6.6 is not active+clean 2024-09-17T02:42:01.893 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.828622+0000', 'last_change': '2024-09-17T02:41:55.828622+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.828622+0000', 'last_undegraded': '2024-09-17T02:41:55.828622+0000', 'last_fullsized': '2024-09-17T02:41:55.828622+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T02:42:01.894 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-09-17T02:42:01.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.815674+0000', 'last_change': '2024-09-17T02:41:55.815674+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.815674+0000', 'last_undegraded': '2024-09-17T02:41:55.815674+0000', 'last_fullsized': '2024-09-17T02:41:55.815674+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T02:42:01.894 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2 is not active+clean 2024-09-17T02:42:01.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.818667+0000', 'last_change': '2024-09-17T02:41:55.818667+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.818667+0000', 'last_undegraded': '2024-09-17T02:41:55.818667+0000', 'last_fullsized': '2024-09-17T02:41:55.818667+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.894 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-09-17T02:42:01.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.822074+0000', 'last_change': '2024-09-17T02:41:55.822074+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.822074+0000', 'last_undegraded': '2024-09-17T02:41:55.822074+0000', 'last_fullsized': '2024-09-17T02:41:55.822074+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:42:01.894 INFO:tasks.ceph.ceph_manager.ceph:PG 6.7 is not active+clean 2024-09-17T02:42:01.894 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 38, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:41:55.820910+0000', 'last_change': '2024-09-17T02:41:55.820910+0000', 'last_active': '2024-09-17T02:41:55.802457+0000', 'last_peered': '2024-09-17T02:41:55.802457+0000', 'last_clean': '2024-09-17T02:41:55.802457+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:41:55.820910+0000', 'last_undegraded': '2024-09-17T02:41:55.820910+0000', 'last_fullsized': '2024-09-17T02:41:55.820910+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 38, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'last_clean_scrub_stamp': '2024-09-17T02:41:55.802457+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:42:01.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T02:42:02.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.068+0000 7fc79ff34640 1 -- 172.21.15.131:0/3760850005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc79815c5b0 msgr2=0x7fc7980cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:02.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.068+0000 7fc79ff34640 1 --2- 172.21.15.131:0/3760850005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc79815c5b0 0x7fc7980cbaf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc790009bc0 tx=0x7fc79001cba0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.068+0000 7fc79ff34640 1 -- 172.21.15.131:0/3760850005 shutdown_connections 2024-09-17T02:42:02.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.068+0000 7fc79ff34640 1 --2- 172.21.15.131:0/3760850005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc79815c5b0 0x7fc7980cbaf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.068+0000 7fc79ff34640 1 -- 172.21.15.131:0/3760850005 >> 172.21.15.131:0/3760850005 conn(0x7fc79806eea0 msgr2=0x7fc79806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:02.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.068+0000 7fc79ff34640 1 -- 172.21.15.131:0/3760850005 shutdown_connections 2024-09-17T02:42:02.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.068+0000 7fc79ff34640 1 -- 172.21.15.131:0/3760850005 wait complete. 2024-09-17T02:42:02.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79ff34640 1 Processor -- start 2024-09-17T02:42:02.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79ff34640 1 -- start start 2024-09-17T02:42:02.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79ff34640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7980ced00 0x7fc798151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:02.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79ff34640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc79815c5b0 0x7fc7981504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:02.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79ff34640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc79814ad20 0x7fc798150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:02.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79ff34640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc798169ad0 con 0x7fc79815c5b0 2024-09-17T02:42:02.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79ff34640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc798169950 con 0x7fc7980ced00 2024-09-17T02:42:02.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79ff34640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc798169c50 con 0x7fc79814ad20 2024-09-17T02:42:02.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79d4cf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc79815c5b0 0x7fc7981504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:02.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79d4cf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc79815c5b0 0x7fc7981504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57080/0 (socket says 172.21.15.131:57080) 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79d4cf640 1 -- 172.21.15.131:0/3100018884 learned_addr learned my addr 172.21.15.131:0/3100018884 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79d4cf640 1 -- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc79814ad20 msgr2=0x7fc798150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79d4cf640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc79814ad20 0x7fc798150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79d4cf640 1 -- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7980ced00 msgr2=0x7fc798151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79dcd0640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7980ced00 0x7fc798151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79d4cf640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7980ced00 0x7fc798151a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79d4cf640 1 -- 172.21.15.131:0/3100018884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc798158290 con 0x7fc79815c5b0 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79d4cf640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc79815c5b0 0x7fc7981504d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc78800b410 tx=0x7fc78800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc78e7fc640 1 -- 172.21.15.131:0/3100018884 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788015020 con 0x7fc79815c5b0 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc78e7fc640 1 -- 172.21.15.131:0/3100018884 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc788011920 con 0x7fc79815c5b0 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79ff34640 1 -- 172.21.15.131:0/3100018884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc798158790 con 0x7fc79815c5b0 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc78e7fc640 1 -- 172.21.15.131:0/3100018884 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788011d70 con 0x7fc79815c5b0 2024-09-17T02:42:02.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79ff34640 1 -- 172.21.15.131:0/3100018884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc798158d50 con 0x7fc79815c5b0 2024-09-17T02:42:02.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc79ff34640 1 -- 172.21.15.131:0/3100018884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc798158500 con 0x7fc79815c5b0 2024-09-17T02:42:02.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.072+0000 7fc78e7fc640 1 -- 172.21.15.131:0/3100018884 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc788018020 con 0x7fc79815c5b0 2024-09-17T02:42:02.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.076+0000 7fc78e7fc640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc77c073200 0x7fc77c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:02.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.076+0000 7fc78e7fc640 1 -- 172.21.15.131:0/3100018884 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fc7880942c0 con 0x7fc79815c5b0 2024-09-17T02:42:02.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.076+0000 7fc79dcd0640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc77c073200 0x7fc77c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:02.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.076+0000 7fc79dcd0640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc77c073200 0x7fc77c075710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc794001570 tx=0x7fc794009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:02.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.080+0000 7fc78e7fc640 1 -- 172.21.15.131:0/3100018884 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc78801f020 con 0x7fc79815c5b0 2024-09-17T02:42:02.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.296+0000 7fc79ff34640 1 -- 172.21.15.131:0/3100018884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fc798075b80 con 0x7fc79815c5b0 2024-09-17T02:42:02.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.296+0000 7fc78e7fc640 1 -- 172.21.15.131:0/3100018884 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1236 (secure 0 0 0) 0x7fc78805f1e0 con 0x7fc79815c5b0 2024-09-17T02:42:02.299 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:02.299 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":96,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":40,"num_osds":8,"num_up_osds":8,"osd_up_since":1726540840,"num_in_osds":8,"osd_in_since":1726540830,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":56},{"state_name":"active+clean","count":41}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":230264832,"bytes_avail":772863848448,"bytes_total":773094113280,"inactive_pgs_ratio":0.57731956243515015},"fsmap":{"epoch":18,"btime":"2024-09-17T02:41:59:873356+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5022}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-17T02:41:49.140000+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T02:42:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 -- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc77c073200 msgr2=0x7fc77c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc77c073200 0x7fc77c075710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc794001570 tx=0x7fc794009040 comp rx=0 tx=0).stop 2024-09-17T02:42:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 -- 172.21.15.131:0/3100018884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc79815c5b0 msgr2=0x7fc7981504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc79815c5b0 0x7fc7981504d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc78800b410 tx=0x7fc78800b8d0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 -- 172.21.15.131:0/3100018884 shutdown_connections 2024-09-17T02:42:02.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc77c073200 0x7fc77c075710 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc79814ad20 0x7fc798150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc79815c5b0 0x7fc7981504d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 --2- 172.21.15.131:0/3100018884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7980ced00 0x7fc798151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 -- 172.21.15.131:0/3100018884 >> 172.21.15.131:0/3100018884 conn(0x7fc79806eea0 msgr2=0x7fc798159c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:02.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 -- 172.21.15.131:0/3100018884 shutdown_connections 2024-09-17T02:42:02.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.300+0000 7fc79ff34640 1 -- 172.21.15.131:0/3100018884 wait complete. 2024-09-17T02:42:02.319 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T02:42:02.319 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:42:02.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 -- 172.21.15.131:0/3328605068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefc4003a60 msgr2=0x7fefc4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:02.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 --2- 172.21.15.131:0/3328605068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefc4003a60 0x7fefc4003ed0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c610 tx=0x7fefd001d210 comp rx=0 tx=0).stop 2024-09-17T02:42:02.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 -- 172.21.15.131:0/3328605068 shutdown_connections 2024-09-17T02:42:02.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 --2- 172.21.15.131:0/3328605068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefc4003a60 0x7fefc4003ed0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 -- 172.21.15.131:0/3328605068 >> 172.21.15.131:0/3328605068 conn(0x7fefd80cf150 msgr2=0x7fefd80cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:02.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 -- 172.21.15.131:0/3328605068 shutdown_connections 2024-09-17T02:42:02.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 -- 172.21.15.131:0/3328605068 wait complete. 2024-09-17T02:42:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 Processor -- start 2024-09-17T02:42:02.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 -- start start 2024-09-17T02:42:02.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefc4003a60 0x7fefd806f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:02.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd80cb7f0 0x7fefd80cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:02.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fefd81fc150 0x7fefd81fc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fefd8160530 con 0x7fefc4003a60 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fefd81603b0 con 0x7fefd80cb7f0 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.476+0000 7fefdd2ce640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fefd81606b0 con 0x7fefd81fc150 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefc4003a60 0x7fefd806f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefc4003a60 0x7fefd806f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57096/0 (socket says 172.21.15.131:57096) 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd6ffd640 1 -- 172.21.15.131:0/936652300 learned_addr learned my addr 172.21.15.131:0/936652300 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd77fe640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fefd81fc150 0x7fefd81fc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd67fc640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd80cb7f0 0x7fefd80cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd6ffd640 1 -- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fefd81fc150 msgr2=0x7fefd81fc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd6ffd640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fefd81fc150 0x7fefd81fc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd6ffd640 1 -- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd80cb7f0 msgr2=0x7fefd80cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd6ffd640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd80cb7f0 0x7fefd80cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd6ffd640 1 -- 172.21.15.131:0/936652300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd80cc140 con 0x7fefc4003a60 2024-09-17T02:42:02.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd77fe640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fefd81fc150 0x7fefd81fc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:42:02.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd67fc640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd80cb7f0 0x7fefd80cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:42:02.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd6ffd640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefc4003a60 0x7fefd806f780 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fefd001ccd0 tx=0x7fefd0001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:02.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefcf7fe640 1 -- 172.21.15.131:0/936652300 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0004e60 con 0x7fefc4003a60 2024-09-17T02:42:02.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefcf7fe640 1 -- 172.21.15.131:0/936652300 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fefd0003c90 con 0x7fefc4003a60 2024-09-17T02:42:02.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefdd2ce640 1 -- 172.21.15.131:0/936652300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd81f8920 con 0x7fefc4003a60 2024-09-17T02:42:02.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefcf7fe640 1 -- 172.21.15.131:0/936652300 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0006290 con 0x7fefc4003a60 2024-09-17T02:42:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefdd2ce640 1 -- 172.21.15.131:0/936652300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd81f8ee0 con 0x7fefc4003a60 2024-09-17T02:42:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefdd2ce640 1 -- 172.21.15.131:0/936652300 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd8160c10 con 0x7fefc4003a60 2024-09-17T02:42:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefcf7fe640 1 -- 172.21.15.131:0/936652300 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fefd0006430 con 0x7fefc4003a60 2024-09-17T02:42:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefcf7fe640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fefb8073200 0x7fefb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefcf7fe640 1 -- 172.21.15.131:0/936652300 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fefd012eb60 con 0x7fefc4003a60 2024-09-17T02:42:02.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.480+0000 7fefd67fc640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fefb8073200 0x7fefb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:02.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.484+0000 7fefd67fc640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fefb8073200 0x7fefb8075710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fefc0006fd0 tx=0x7fefc0006b50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:02.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.488+0000 7fefcf7fe640 1 -- 172.21.15.131:0/936652300 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fefd80cc140 con 0x7fefc4003a60 2024-09-17T02:42:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.652+0000 7fefdd2ce640 1 -- 172.21.15.131:0/936652300 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefd815c780 con 0x7fefb8073200 2024-09-17T02:42:02.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.664+0000 7fefcf7fe640 1 -- 172.21.15.131:0/936652300 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247319 (secure 0 0 0) 0x7fefd815c780 con 0x7fefb8073200 2024-09-17T02:42:02.668 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:02.670 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:42:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 -- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fefb8073200 msgr2=0x7fefb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:02.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fefb8073200 0x7fefb8075710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fefc0006fd0 tx=0x7fefc0006b50 comp rx=0 tx=0).stop 2024-09-17T02:42:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 -- 172.21.15.131:0/936652300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefc4003a60 msgr2=0x7fefd806f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefc4003a60 0x7fefd806f780 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fefd001ccd0 tx=0x7fefd0001260 comp rx=0 tx=0).stop 2024-09-17T02:42:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 -- 172.21.15.131:0/936652300 shutdown_connections 2024-09-17T02:42:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fefb8073200 0x7fefb8075710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fefd81fc150 0x7fefd81fc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fefd80cb7f0 0x7fefd80cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 --2- 172.21.15.131:0/936652300 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fefc4003a60 0x7fefd806f780 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 -- 172.21.15.131:0/936652300 >> 172.21.15.131:0/936652300 conn(0x7fefd80cf150 msgr2=0x7fefd815bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 -- 172.21.15.131:0/936652300 shutdown_connections 2024-09-17T02:42:02.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:02.668+0000 7fefdd2ce640 1 -- 172.21.15.131:0/936652300 wait complete. 2024-09-17T02:42:02.690 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-17T02:42:01.143735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224868,"kb_used_data":4476,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749852,"statfs":{"total":773094113280,"available":772863848448,"internally_reserved":0,"allocated":4583424,"data_stored":2605067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001879"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.850774+0000","last_change":"2024-09-17T02:41:55.850774+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.850774+0000","last_undegraded":"2024-09-17T02:41:55.850774+0000","last_fullsized":"2024-09-17T02:41:55.850774+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858600+0000","last_change":"2024-09-17T02:41:56.867810+0000","last_active":"2024-09-17T02:41:57.858600+0000","last_peered":"2024-09-17T02:41:57.858600+0000","last_clean":"2024-09-17T02:41:57.858600+0000","last_became_active":"2024-09-17T02:41:56.866944+0000","last_became_peered":"2024-09-17T02:41:56.866944+0000","last_unstale":"2024-09-17T02:41:57.858600+0000","last_undegraded":"2024-09-17T02:41:57.858600+0000","last_fullsized":"2024-09-17T02:41:57.858600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:59.549330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.836800+0000","last_change":"2024-09-17T02:41:55.836800+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.836800+0000","last_undegraded":"2024-09-17T02:41:55.836800+0000","last_fullsized":"2024-09-17T02:41:55.836800+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.838796+0000","last_change":"2024-09-17T02:41:55.838796+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.838796+0000","last_undegraded":"2024-09-17T02:41:55.838796+0000","last_fullsized":"2024-09-17T02:41:55.838796+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822660+0000","last_change":"2024-09-17T02:41:55.822660+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822660+0000","last_undegraded":"2024-09-17T02:41:55.822660+0000","last_fullsized":"2024-09-17T02:41:55.822660+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857974+0000","last_change":"2024-09-17T02:41:56.880704+0000","last_active":"2024-09-17T02:41:57.857974+0000","last_peered":"2024-09-17T02:41:57.857974+0000","last_clean":"2024-09-17T02:41:57.857974+0000","last_became_active":"2024-09-17T02:41:56.879945+0000","last_became_peered":"2024-09-17T02:41:56.879945+0000","last_unstale":"2024-09-17T02:41:57.857974+0000","last_undegraded":"2024-09-17T02:41:57.857974+0000","last_fullsized":"2024-09-17T02:41:57.857974+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:36.026817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825753+0000","last_change":"2024-09-17T02:41:55.825753+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825753+0000","last_undegraded":"2024-09-17T02:41:55.825753+0000","last_fullsized":"2024-09-17T02:41:55.825753+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.855791+0000","last_change":"2024-09-17T02:41:55.855791+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.855791+0000","last_undegraded":"2024-09-17T02:41:55.855791+0000","last_fullsized":"2024-09-17T02:41:55.855791+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815860+0000","last_change":"2024-09-17T02:41:55.815860+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815860+0000","last_undegraded":"2024-09-17T02:41:55.815860+0000","last_fullsized":"2024-09-17T02:41:55.815860+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820570+0000","last_change":"2024-09-17T02:41:55.820570+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820570+0000","last_undegraded":"2024-09-17T02:41:55.820570+0000","last_fullsized":"2024-09-17T02:41:55.820570+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830051+0000","last_change":"2024-09-17T02:41:55.830051+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830051+0000","last_undegraded":"2024-09-17T02:41:55.830051+0000","last_fullsized":"2024-09-17T02:41:55.830051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.817345+0000","last_change":"2024-09-17T02:41:55.817345+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.817345+0000","last_undegraded":"2024-09-17T02:41:55.817345+0000","last_fullsized":"2024-09-17T02:41:55.817345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820760+0000","last_change":"2024-09-17T02:41:55.820760+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820760+0000","last_undegraded":"2024-09-17T02:41:55.820760+0000","last_fullsized":"2024-09-17T02:41:55.820760+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822279+0000","last_change":"2024-09-17T02:41:55.822279+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822279+0000","last_undegraded":"2024-09-17T02:41:55.822279+0000","last_fullsized":"2024-09-17T02:41:55.822279+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.832996+0000","last_change":"2024-09-17T02:41:55.832996+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.832996+0000","last_undegraded":"2024-09-17T02:41:55.832996+0000","last_fullsized":"2024-09-17T02:41:55.832996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821113+0000","last_change":"2024-09-17T02:41:55.821113+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821113+0000","last_undegraded":"2024-09-17T02:41:55.821113+0000","last_fullsized":"2024-09-17T02:41:55.821113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.833655+0000","last_change":"2024-09-17T02:41:55.833655+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.833655+0000","last_undegraded":"2024-09-17T02:41:55.833655+0000","last_fullsized":"2024-09-17T02:41:55.833655+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830989+0000","last_change":"2024-09-17T02:41:55.830989+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830989+0000","last_undegraded":"2024-09-17T02:41:55.830989+0000","last_fullsized":"2024-09-17T02:41:55.830989+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830539+0000","last_change":"2024-09-17T02:41:55.830539+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830539+0000","last_undegraded":"2024-09-17T02:41:55.830539+0000","last_fullsized":"2024-09-17T02:41:55.830539+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.828730+0000","last_change":"2024-09-17T02:41:55.828730+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.828730+0000","last_undegraded":"2024-09-17T02:41:55.828730+0000","last_fullsized":"2024-09-17T02:41:55.828730+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.829387+0000","last_change":"2024-09-17T02:41:55.829387+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.829387+0000","last_undegraded":"2024-09-17T02:41:55.829387+0000","last_fullsized":"2024-09-17T02:41:55.829387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826821+0000","last_change":"2024-09-17T02:41:55.826821+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826821+0000","last_undegraded":"2024-09-17T02:41:55.826821+0000","last_fullsized":"2024-09-17T02:41:55.826821+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.827563+0000","last_change":"2024-09-17T02:41:55.827563+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.827563+0000","last_undegraded":"2024-09-17T02:41:55.827563+0000","last_fullsized":"2024-09-17T02:41:55.827563+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.841871+0000","last_change":"2024-09-17T02:41:55.841871+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.841871+0000","last_undegraded":"2024-09-17T02:41:55.841871+0000","last_fullsized":"2024-09-17T02:41:55.841871+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823236+0000","last_change":"2024-09-17T02:41:55.823236+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823236+0000","last_undegraded":"2024-09-17T02:41:55.823236+0000","last_fullsized":"2024-09-17T02:41:55.823236+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.839845+0000","last_change":"2024-09-17T02:41:55.839845+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.839845+0000","last_undegraded":"2024-09-17T02:41:55.839845+0000","last_fullsized":"2024-09-17T02:41:55.839845+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.824021+0000","last_change":"2024-09-17T02:41:55.824021+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.824021+0000","last_undegraded":"2024-09-17T02:41:55.824021+0000","last_fullsized":"2024-09-17T02:41:55.824021+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.824324+0000","last_change":"2024-09-17T02:41:55.824324+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.824324+0000","last_undegraded":"2024-09-17T02:41:55.824324+0000","last_fullsized":"2024-09-17T02:41:55.824324+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.829096+0000","last_change":"2024-09-17T02:41:55.829096+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.829096+0000","last_undegraded":"2024-09-17T02:41:55.829096+0000","last_fullsized":"2024-09-17T02:41:55.829096+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826857+0000","last_change":"2024-09-17T02:41:55.826857+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826857+0000","last_undegraded":"2024-09-17T02:41:55.826857+0000","last_fullsized":"2024-09-17T02:41:55.826857+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821650+0000","last_change":"2024-09-17T02:41:55.821650+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821650+0000","last_undegraded":"2024-09-17T02:41:55.821650+0000","last_fullsized":"2024-09-17T02:41:55.821650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.840670+0000","last_change":"2024-09-17T02:41:55.840670+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.840670+0000","last_undegraded":"2024-09-17T02:41:55.840670+0000","last_fullsized":"2024-09-17T02:41:55.840670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819919+0000","last_change":"2024-09-17T02:41:55.819919+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819919+0000","last_undegraded":"2024-09-17T02:41:55.819919+0000","last_fullsized":"2024-09-17T02:41:55.819919+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848148+0000","last_change":"2024-09-17T02:41:55.843782+0000","last_active":"2024-09-17T02:41:55.848148+0000","last_peered":"2024-09-17T02:41:55.848148+0000","last_clean":"2024-09-17T02:41:55.848148+0000","last_became_active":"2024-09-17T02:41:55.843362+0000","last_became_peered":"2024-09-17T02:41:55.843362+0000","last_unstale":"2024-09-17T02:41:55.848148+0000","last_undegraded":"2024-09-17T02:41:55.848148+0000","last_fullsized":"2024-09-17T02:41:55.848148+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:39.655039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858515+0000","last_change":"2024-09-17T02:41:56.882518+0000","last_active":"2024-09-17T02:41:57.858515+0000","last_peered":"2024-09-17T02:41:57.858515+0000","last_clean":"2024-09-17T02:41:57.858515+0000","last_became_active":"2024-09-17T02:41:56.879984+0000","last_became_peered":"2024-09-17T02:41:56.879984+0000","last_unstale":"2024-09-17T02:41:57.858515+0000","last_undegraded":"2024-09-17T02:41:57.858515+0000","last_fullsized":"2024-09-17T02:41:57.858515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:35.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858745+0000","last_change":"2024-09-17T02:41:55.844353+0000","last_active":"2024-09-17T02:41:57.858745+0000","last_peered":"2024-09-17T02:41:57.858745+0000","last_clean":"2024-09-17T02:41:57.858745+0000","last_became_active":"2024-09-17T02:41:55.843505+0000","last_became_peered":"2024-09-17T02:41:55.843505+0000","last_unstale":"2024-09-17T02:41:57.858745+0000","last_undegraded":"2024-09-17T02:41:57.858745+0000","last_fullsized":"2024-09-17T02:41:57.858745+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:43.510214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.830088+0000","last_change":"2024-09-17T02:41:55.830088+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.830088+0000","last_undegraded":"2024-09-17T02:41:55.830088+0000","last_fullsized":"2024-09-17T02:41:55.830088+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869994+0000","last_change":"2024-09-17T02:41:55.865708+0000","last_active":"2024-09-17T02:41:55.869994+0000","last_peered":"2024-09-17T02:41:55.869994+0000","last_clean":"2024-09-17T02:41:55.869994+0000","last_became_active":"2024-09-17T02:41:55.861337+0000","last_became_peered":"2024-09-17T02:41:55.861337+0000","last_unstale":"2024-09-17T02:41:55.869994+0000","last_undegraded":"2024-09-17T02:41:55.869994+0000","last_fullsized":"2024-09-17T02:41:55.869994+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:28:47.106157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.859238+0000","last_change":"2024-09-17T02:41:56.867038+0000","last_active":"2024-09-17T02:41:57.859238+0000","last_peered":"2024-09-17T02:41:57.859238+0000","last_clean":"2024-09-17T02:41:57.859238+0000","last_became_active":"2024-09-17T02:41:56.865986+0000","last_became_peered":"2024-09-17T02:41:56.865986+0000","last_unstale":"2024-09-17T02:41:57.859238+0000","last_undegraded":"2024-09-17T02:41:57.859238+0000","last_fullsized":"2024-09-17T02:41:57.859238+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:31.091078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.862703+0000","last_change":"2024-09-17T02:41:55.859306+0000","last_active":"2024-09-17T02:41:55.862703+0000","last_peered":"2024-09-17T02:41:55.862703+0000","last_clean":"2024-09-17T02:41:55.862703+0000","last_became_active":"2024-09-17T02:41:55.858980+0000","last_became_peered":"2024-09-17T02:41:55.858980+0000","last_unstale":"2024-09-17T02:41:55.862703+0000","last_undegraded":"2024-09-17T02:41:55.862703+0000","last_fullsized":"2024-09-17T02:41:55.862703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:13.459554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.839286+0000","last_change":"2024-09-17T02:41:55.839286+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.839286+0000","last_undegraded":"2024-09-17T02:41:55.839286+0000","last_fullsized":"2024-09-17T02:41:55.839286+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.836960+0000","last_change":"2024-09-17T02:41:55.832205+0000","last_active":"2024-09-17T02:41:55.836960+0000","last_peered":"2024-09-17T02:41:55.836960+0000","last_clean":"2024-09-17T02:41:55.836960+0000","last_became_active":"2024-09-17T02:41:55.831773+0000","last_became_peered":"2024-09-17T02:41:55.831773+0000","last_unstale":"2024-09-17T02:41:55.836960+0000","last_undegraded":"2024-09-17T02:41:55.836960+0000","last_fullsized":"2024-09-17T02:41:55.836960+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:09.662328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822131+0000","last_change":"2024-09-17T02:41:55.822131+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822131+0000","last_undegraded":"2024-09-17T02:41:55.822131+0000","last_fullsized":"2024-09-17T02:41:55.822131+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.871586+0000","last_change":"2024-09-17T02:41:55.867371+0000","last_active":"2024-09-17T02:41:55.871586+0000","last_peered":"2024-09-17T02:41:55.871586+0000","last_clean":"2024-09-17T02:41:55.871586+0000","last_became_active":"2024-09-17T02:41:55.866967+0000","last_became_peered":"2024-09-17T02:41:55.866967+0000","last_unstale":"2024-09-17T02:41:55.871586+0000","last_undegraded":"2024-09-17T02:41:55.871586+0000","last_fullsized":"2024-09-17T02:41:55.871586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:08.169894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821328+0000","last_change":"2024-09-17T02:41:55.821328+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821328+0000","last_undegraded":"2024-09-17T02:41:55.821328+0000","last_fullsized":"2024-09-17T02:41:55.821328+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869429+0000","last_change":"2024-09-17T02:41:55.862221+0000","last_active":"2024-09-17T02:41:55.869429+0000","last_peered":"2024-09-17T02:41:55.869429+0000","last_clean":"2024-09-17T02:41:55.869429+0000","last_became_active":"2024-09-17T02:41:55.861013+0000","last_became_peered":"2024-09-17T02:41:55.861013+0000","last_unstale":"2024-09-17T02:41:55.869429+0000","last_undegraded":"2024-09-17T02:41:55.869429+0000","last_fullsized":"2024-09-17T02:41:55.869429+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:45.117350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.859026+0000","last_change":"2024-09-17T02:41:56.875750+0000","last_active":"2024-09-17T02:41:57.859026+0000","last_peered":"2024-09-17T02:41:57.859026+0000","last_clean":"2024-09-17T02:41:57.859026+0000","last_became_active":"2024-09-17T02:41:56.874828+0000","last_became_peered":"2024-09-17T02:41:56.874828+0000","last_unstale":"2024-09-17T02:41:57.859026+0000","last_undegraded":"2024-09-17T02:41:57.859026+0000","last_fullsized":"2024-09-17T02:41:57.859026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:39.693566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.835588+0000","last_change":"2024-09-17T02:41:55.833725+0000","last_active":"2024-09-17T02:41:55.835588+0000","last_peered":"2024-09-17T02:41:55.835588+0000","last_clean":"2024-09-17T02:41:55.835588+0000","last_became_active":"2024-09-17T02:41:55.833225+0000","last_became_peered":"2024-09-17T02:41:55.833225+0000","last_unstale":"2024-09-17T02:41:55.835588+0000","last_undegraded":"2024-09-17T02:41:55.835588+0000","last_fullsized":"2024-09-17T02:41:55.835588+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:13:19.961178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.837456+0000","last_change":"2024-09-17T02:41:55.837456+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.837456+0000","last_undegraded":"2024-09-17T02:41:55.837456+0000","last_fullsized":"2024-09-17T02:41:55.837456+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870175+0000","last_change":"2024-09-17T02:41:55.865849+0000","last_active":"2024-09-17T02:41:55.870175+0000","last_peered":"2024-09-17T02:41:55.870175+0000","last_clean":"2024-09-17T02:41:55.870175+0000","last_became_active":"2024-09-17T02:41:55.863121+0000","last_became_peered":"2024-09-17T02:41:55.863121+0000","last_unstale":"2024-09-17T02:41:55.870175+0000","last_undegraded":"2024-09-17T02:41:55.870175+0000","last_fullsized":"2024-09-17T02:41:55.870175+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:37.028521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822346+0000","last_change":"2024-09-17T02:41:55.822346+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822346+0000","last_undegraded":"2024-09-17T02:41:55.822346+0000","last_fullsized":"2024-09-17T02:41:55.822346+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848532+0000","last_change":"2024-09-17T02:41:55.846533+0000","last_active":"2024-09-17T02:41:55.848532+0000","last_peered":"2024-09-17T02:41:55.848532+0000","last_clean":"2024-09-17T02:41:55.848532+0000","last_became_active":"2024-09-17T02:41:55.846130+0000","last_became_peered":"2024-09-17T02:41:55.846130+0000","last_unstale":"2024-09-17T02:41:55.848532+0000","last_undegraded":"2024-09-17T02:41:55.848532+0000","last_fullsized":"2024-09-17T02:41:55.848532+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:31.071112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825732+0000","last_change":"2024-09-17T02:41:55.825732+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825732+0000","last_undegraded":"2024-09-17T02:41:55.825732+0000","last_fullsized":"2024-09-17T02:41:55.825732+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.871635+0000","last_change":"2024-09-17T02:41:55.866610+0000","last_active":"2024-09-17T02:41:55.871635+0000","last_peered":"2024-09-17T02:41:55.871635+0000","last_clean":"2024-09-17T02:41:55.871635+0000","last_became_active":"2024-09-17T02:41:55.866246+0000","last_became_peered":"2024-09-17T02:41:55.866246+0000","last_unstale":"2024-09-17T02:41:55.871635+0000","last_undegraded":"2024-09-17T02:41:55.871635+0000","last_fullsized":"2024-09-17T02:41:55.871635+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:45.605666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820352+0000","last_change":"2024-09-17T02:41:55.820352+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820352+0000","last_undegraded":"2024-09-17T02:41:55.820352+0000","last_fullsized":"2024-09-17T02:41:55.820352+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869861+0000","last_change":"2024-09-17T02:41:55.863340+0000","last_active":"2024-09-17T02:41:55.869861+0000","last_peered":"2024-09-17T02:41:55.869861+0000","last_clean":"2024-09-17T02:41:55.869861+0000","last_became_active":"2024-09-17T02:41:55.861822+0000","last_became_peered":"2024-09-17T02:41:55.861822+0000","last_unstale":"2024-09-17T02:41:55.869861+0000","last_undegraded":"2024-09-17T02:41:55.869861+0000","last_fullsized":"2024-09-17T02:41:55.869861+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:44.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.838002+0000","last_change":"2024-09-17T02:41:55.838002+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.838002+0000","last_undegraded":"2024-09-17T02:41:55.838002+0000","last_fullsized":"2024-09-17T02:41:55.838002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870109+0000","last_change":"2024-09-17T02:41:55.866575+0000","last_active":"2024-09-17T02:41:55.870109+0000","last_peered":"2024-09-17T02:41:55.870109+0000","last_clean":"2024-09-17T02:41:55.870109+0000","last_became_active":"2024-09-17T02:41:55.862842+0000","last_became_peered":"2024-09-17T02:41:55.862842+0000","last_unstale":"2024-09-17T02:41:55.870109+0000","last_undegraded":"2024-09-17T02:41:55.870109+0000","last_fullsized":"2024-09-17T02:41:55.870109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:46.934262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857825+0000","last_change":"2024-09-17T02:41:56.881999+0000","last_active":"2024-09-17T02:41:57.857825+0000","last_peered":"2024-09-17T02:41:57.857825+0000","last_clean":"2024-09-17T02:41:57.857825+0000","last_became_active":"2024-09-17T02:41:56.881115+0000","last_became_peered":"2024-09-17T02:41:56.881115+0000","last_unstale":"2024-09-17T02:41:57.857825+0000","last_undegraded":"2024-09-17T02:41:57.857825+0000","last_fullsized":"2024-09-17T02:41:57.857825+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:35.710278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857830+0000","last_change":"2024-09-17T02:41:55.839604+0000","last_active":"2024-09-17T02:41:57.857830+0000","last_peered":"2024-09-17T02:41:57.857830+0000","last_clean":"2024-09-17T02:41:57.857830+0000","last_became_active":"2024-09-17T02:41:55.835167+0000","last_became_peered":"2024-09-17T02:41:55.835167+0000","last_unstale":"2024-09-17T02:41:57.857830+0000","last_undegraded":"2024-09-17T02:41:57.857830+0000","last_fullsized":"2024-09-17T02:41:57.857830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:44.487692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823496+0000","last_change":"2024-09-17T02:41:55.823496+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823496+0000","last_undegraded":"2024-09-17T02:41:55.823496+0000","last_fullsized":"2024-09-17T02:41:55.823496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.840993+0000","last_change":"2024-09-17T02:41:55.830025+0000","last_active":"2024-09-17T02:41:55.840993+0000","last_peered":"2024-09-17T02:41:55.840993+0000","last_clean":"2024-09-17T02:41:55.840993+0000","last_became_active":"2024-09-17T02:41:55.829641+0000","last_became_peered":"2024-09-17T02:41:55.829641+0000","last_unstale":"2024-09-17T02:41:55.840993+0000","last_undegraded":"2024-09-17T02:41:55.840993+0000","last_fullsized":"2024-09-17T02:41:55.840993+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:33.231862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815645+0000","last_change":"2024-09-17T02:41:55.815645+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815645+0000","last_undegraded":"2024-09-17T02:41:55.815645+0000","last_fullsized":"2024-09-17T02:41:55.815645+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.830119+0000","last_change":"2024-09-17T02:41:55.824740+0000","last_active":"2024-09-17T02:41:55.830119+0000","last_peered":"2024-09-17T02:41:55.830119+0000","last_clean":"2024-09-17T02:41:55.830119+0000","last_became_active":"2024-09-17T02:41:55.824315+0000","last_became_peered":"2024-09-17T02:41:55.824315+0000","last_unstale":"2024-09-17T02:41:55.830119+0000","last_undegraded":"2024-09-17T02:41:55.830119+0000","last_fullsized":"2024-09-17T02:41:55.830119+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:20.013546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858509+0000","last_change":"2024-09-17T02:41:56.881490+0000","last_active":"2024-09-17T02:41:57.858509+0000","last_peered":"2024-09-17T02:41:57.858509+0000","last_clean":"2024-09-17T02:41:57.858509+0000","last_became_active":"2024-09-17T02:41:56.880621+0000","last_became_peered":"2024-09-17T02:41:56.880621+0000","last_unstale":"2024-09-17T02:41:57.858509+0000","last_undegraded":"2024-09-17T02:41:57.858509+0000","last_fullsized":"2024-09-17T02:41:57.858509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:27.688857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.835545+0000","last_change":"2024-09-17T02:41:55.833534+0000","last_active":"2024-09-17T02:41:55.835545+0000","last_peered":"2024-09-17T02:41:55.835545+0000","last_clean":"2024-09-17T02:41:55.835545+0000","last_became_active":"2024-09-17T02:41:55.833218+0000","last_became_peered":"2024-09-17T02:41:55.833218+0000","last_unstale":"2024-09-17T02:41:55.835545+0000","last_undegraded":"2024-09-17T02:41:55.835545+0000","last_fullsized":"2024-09-17T02:41:55.835545+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:40:57.836648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.821387+0000","last_change":"2024-09-17T02:41:55.821387+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.821387+0000","last_undegraded":"2024-09-17T02:41:55.821387+0000","last_fullsized":"2024-09-17T02:41:55.821387+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.863199+0000","last_change":"2024-09-17T02:41:55.861548+0000","last_active":"2024-09-17T02:41:55.863199+0000","last_peered":"2024-09-17T02:41:55.863199+0000","last_clean":"2024-09-17T02:41:55.863199+0000","last_became_active":"2024-09-17T02:41:55.861200+0000","last_became_peered":"2024-09-17T02:41:55.861200+0000","last_unstale":"2024-09-17T02:41:55.863199+0000","last_undegraded":"2024-09-17T02:41:55.863199+0000","last_fullsized":"2024-09-17T02:41:55.863199+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:08.387963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819800+0000","last_change":"2024-09-17T02:41:55.819800+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819800+0000","last_undegraded":"2024-09-17T02:41:55.819800+0000","last_fullsized":"2024-09-17T02:41:55.819800+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850416+0000","last_change":"2024-09-17T02:41:55.844139+0000","last_active":"2024-09-17T02:41:55.850416+0000","last_peered":"2024-09-17T02:41:55.850416+0000","last_clean":"2024-09-17T02:41:55.850416+0000","last_became_active":"2024-09-17T02:41:55.842953+0000","last_became_peered":"2024-09-17T02:41:55.842953+0000","last_unstale":"2024-09-17T02:41:55.850416+0000","last_undegraded":"2024-09-17T02:41:55.850416+0000","last_fullsized":"2024-09-17T02:41:55.850416+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:54:57.710104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.833247+0000","last_change":"2024-09-17T02:41:55.833247+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.833247+0000","last_undegraded":"2024-09-17T02:41:55.833247+0000","last_fullsized":"2024-09-17T02:41:55.833247+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.845102+0000","last_change":"2024-09-17T02:41:55.840256+0000","last_active":"2024-09-17T02:41:55.845102+0000","last_peered":"2024-09-17T02:41:55.845102+0000","last_clean":"2024-09-17T02:41:55.845102+0000","last_became_active":"2024-09-17T02:41:55.839277+0000","last_became_peered":"2024-09-17T02:41:55.839277+0000","last_unstale":"2024-09-17T02:41:55.845102+0000","last_undegraded":"2024-09-17T02:41:55.845102+0000","last_fullsized":"2024-09-17T02:41:55.845102+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:06:51.053427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.819364+0000","last_change":"2024-09-17T02:41:55.819364+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.819364+0000","last_undegraded":"2024-09-17T02:41:55.819364+0000","last_fullsized":"2024-09-17T02:41:55.819364+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.848889+0000","last_change":"2024-09-17T02:41:55.846024+0000","last_active":"2024-09-17T02:41:55.848889+0000","last_peered":"2024-09-17T02:41:55.848889+0000","last_clean":"2024-09-17T02:41:55.848889+0000","last_became_active":"2024-09-17T02:41:55.845095+0000","last_became_peered":"2024-09-17T02:41:55.845095+0000","last_unstale":"2024-09-17T02:41:55.848889+0000","last_undegraded":"2024-09-17T02:41:55.848889+0000","last_fullsized":"2024-09-17T02:41:55.848889+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:51.113568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.825036+0000","last_change":"2024-09-17T02:41:55.825036+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.825036+0000","last_undegraded":"2024-09-17T02:41:55.825036+0000","last_fullsized":"2024-09-17T02:41:55.825036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858145+0000","last_change":"2024-09-17T02:41:55.838665+0000","last_active":"2024-09-17T02:41:57.858145+0000","last_peered":"2024-09-17T02:41:57.858145+0000","last_clean":"2024-09-17T02:41:57.858145+0000","last_became_active":"2024-09-17T02:41:55.837817+0000","last_became_peered":"2024-09-17T02:41:55.837817+0000","last_unstale":"2024-09-17T02:41:57.858145+0000","last_undegraded":"2024-09-17T02:41:57.858145+0000","last_fullsized":"2024-09-17T02:41:57.858145+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:05.524969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826200+0000","last_change":"2024-09-17T02:41:55.826200+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826200+0000","last_undegraded":"2024-09-17T02:41:55.826200+0000","last_fullsized":"2024-09-17T02:41:55.826200+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.836148+0000","last_change":"2024-09-17T02:41:55.833701+0000","last_active":"2024-09-17T02:41:55.836148+0000","last_peered":"2024-09-17T02:41:55.836148+0000","last_clean":"2024-09-17T02:41:55.836148+0000","last_became_active":"2024-09-17T02:41:55.833254+0000","last_became_peered":"2024-09-17T02:41:55.833254+0000","last_unstale":"2024-09-17T02:41:55.836148+0000","last_undegraded":"2024-09-17T02:41:55.836148+0000","last_fullsized":"2024-09-17T02:41:55.836148+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:51.336142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.826130+0000","last_change":"2024-09-17T02:41:55.826130+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.826130+0000","last_undegraded":"2024-09-17T02:41:55.826130+0000","last_fullsized":"2024-09-17T02:41:55.826130+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857048+0000","last_change":"2024-09-17T02:41:55.863410+0000","last_active":"2024-09-17T02:41:57.857048+0000","last_peered":"2024-09-17T02:41:57.857048+0000","last_clean":"2024-09-17T02:41:57.857048+0000","last_became_active":"2024-09-17T02:41:55.862647+0000","last_became_peered":"2024-09-17T02:41:55.862647+0000","last_unstale":"2024-09-17T02:41:57.857048+0000","last_undegraded":"2024-09-17T02:41:57.857048+0000","last_fullsized":"2024-09-17T02:41:57.857048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:25.110212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815903+0000","last_change":"2024-09-17T02:41:55.815903+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815903+0000","last_undegraded":"2024-09-17T02:41:55.815903+0000","last_fullsized":"2024-09-17T02:41:55.815903+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850489+0000","last_change":"2024-09-17T02:41:55.849654+0000","last_active":"2024-09-17T02:41:55.850489+0000","last_peered":"2024-09-17T02:41:55.850489+0000","last_clean":"2024-09-17T02:41:55.850489+0000","last_became_active":"2024-09-17T02:41:55.849235+0000","last_became_peered":"2024-09-17T02:41:55.849235+0000","last_unstale":"2024-09-17T02:41:55.850489+0000","last_undegraded":"2024-09-17T02:41:55.850489+0000","last_fullsized":"2024-09-17T02:41:55.850489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:34.982736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.823333+0000","last_change":"2024-09-17T02:41:55.823333+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.823333+0000","last_undegraded":"2024-09-17T02:41:55.823333+0000","last_fullsized":"2024-09-17T02:41:55.823333+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.852875+0000","last_change":"2024-09-17T02:41:55.843817+0000","last_active":"2024-09-17T02:41:55.852875+0000","last_peered":"2024-09-17T02:41:55.852875+0000","last_clean":"2024-09-17T02:41:55.852875+0000","last_became_active":"2024-09-17T02:41:55.841737+0000","last_became_peered":"2024-09-17T02:41:55.841737+0000","last_unstale":"2024-09-17T02:41:55.852875+0000","last_undegraded":"2024-09-17T02:41:55.852875+0000","last_fullsized":"2024-09-17T02:41:55.852875+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:31.493951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.828622+0000","last_change":"2024-09-17T02:41:55.828622+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.828622+0000","last_undegraded":"2024-09-17T02:41:55.828622+0000","last_fullsized":"2024-09-17T02:41:55.828622+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.870027+0000","last_change":"2024-09-17T02:41:55.868286+0000","last_active":"2024-09-17T02:41:55.870027+0000","last_peered":"2024-09-17T02:41:55.870027+0000","last_clean":"2024-09-17T02:41:55.870027+0000","last_became_active":"2024-09-17T02:41:55.867258+0000","last_became_peered":"2024-09-17T02:41:55.867258+0000","last_unstale":"2024-09-17T02:41:55.870027+0000","last_undegraded":"2024-09-17T02:41:55.870027+0000","last_fullsized":"2024-09-17T02:41:55.870027+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:03.928821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.815674+0000","last_change":"2024-09-17T02:41:55.815674+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.815674+0000","last_undegraded":"2024-09-17T02:41:55.815674+0000","last_fullsized":"2024-09-17T02:41:55.815674+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869829+0000","last_change":"2024-09-17T02:41:55.863138+0000","last_active":"2024-09-17T02:41:55.869829+0000","last_peered":"2024-09-17T02:41:55.869829+0000","last_clean":"2024-09-17T02:41:55.869829+0000","last_became_active":"2024-09-17T02:41:55.862501+0000","last_became_peered":"2024-09-17T02:41:55.862501+0000","last_unstale":"2024-09-17T02:41:55.869829+0000","last_undegraded":"2024-09-17T02:41:55.869829+0000","last_fullsized":"2024-09-17T02:41:55.869829+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:16.370567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858000+0000","last_change":"2024-09-17T02:41:56.876227+0000","last_active":"2024-09-17T02:41:57.858000+0000","last_peered":"2024-09-17T02:41:57.858000+0000","last_clean":"2024-09-17T02:41:57.858000+0000","last_became_active":"2024-09-17T02:41:56.875330+0000","last_became_peered":"2024-09-17T02:41:56.875330+0000","last_unstale":"2024-09-17T02:41:57.858000+0000","last_undegraded":"2024-09-17T02:41:57.858000+0000","last_fullsized":"2024-09-17T02:41:57.858000+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:53.391175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.853308+0000","last_change":"2024-09-17T02:41:55.850201+0000","last_active":"2024-09-17T02:41:55.853308+0000","last_peered":"2024-09-17T02:41:55.853308+0000","last_clean":"2024-09-17T02:41:55.853308+0000","last_became_active":"2024-09-17T02:41:55.849734+0000","last_became_peered":"2024-09-17T02:41:55.849734+0000","last_unstale":"2024-09-17T02:41:55.853308+0000","last_undegraded":"2024-09-17T02:41:55.853308+0000","last_fullsized":"2024-09-17T02:41:55.853308+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:44.530383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.818667+0000","last_change":"2024-09-17T02:41:55.818667+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.818667+0000","last_undegraded":"2024-09-17T02:41:55.818667+0000","last_fullsized":"2024-09-17T02:41:55.818667+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.850540+0000","last_change":"2024-09-17T02:41:55.849649+0000","last_active":"2024-09-17T02:41:55.850540+0000","last_peered":"2024-09-17T02:41:55.850540+0000","last_clean":"2024-09-17T02:41:55.850540+0000","last_became_active":"2024-09-17T02:41:55.849237+0000","last_became_peered":"2024-09-17T02:41:55.849237+0000","last_unstale":"2024-09-17T02:41:55.850540+0000","last_undegraded":"2024-09-17T02:41:55.850540+0000","last_fullsized":"2024-09-17T02:41:55.850540+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:40.869732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.822074+0000","last_change":"2024-09-17T02:41:55.822074+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.822074+0000","last_undegraded":"2024-09-17T02:41:55.822074+0000","last_fullsized":"2024-09-17T02:41:55.822074+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.869389+0000","last_change":"2024-09-17T02:41:55.862706+0000","last_active":"2024-09-17T02:41:55.869389+0000","last_peered":"2024-09-17T02:41:55.869389+0000","last_clean":"2024-09-17T02:41:55.869389+0000","last_became_active":"2024-09-17T02:41:55.862091+0000","last_became_peered":"2024-09-17T02:41:55.862091+0000","last_unstale":"2024-09-17T02:41:55.869389+0000","last_undegraded":"2024-09-17T02:41:55.869389+0000","last_fullsized":"2024-09-17T02:41:55.869389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:16.349899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":38,"state":"creating+peering","last_fresh":"2024-09-17T02:41:55.820910+0000","last_change":"2024-09-17T02:41:55.820910+0000","last_active":"2024-09-17T02:41:55.802457+0000","last_peered":"2024-09-17T02:41:55.802457+0000","last_clean":"2024-09-17T02:41:55.802457+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:41:55.820910+0000","last_undegraded":"2024-09-17T02:41:55.820910+0000","last_fullsized":"2024-09-17T02:41:55.820910+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.852959+0000","last_change":"2024-09-17T02:41:55.850095+0000","last_active":"2024-09-17T02:41:55.852959+0000","last_peered":"2024-09-17T02:41:55.852959+0000","last_clean":"2024-09-17T02:41:55.852959+0000","last_became_active":"2024-09-17T02:41:55.849684+0000","last_became_peered":"2024-09-17T02:41:55.849684+0000","last_unstale":"2024-09-17T02:41:55.852959+0000","last_undegraded":"2024-09-17T02:41:55.852959+0000","last_fullsized":"2024-09-17T02:41:55.852959+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:58:16.370567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":141,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-17T02:41:55.825082+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:41:55.825082+0000","last_peered":"2024-09-17T02:41:55.825082+0000","last_clean":"2024-09-17T02:41:55.825082+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:41:55.825082+0000","last_undegraded":"2024-09-17T02:41:55.825082+0000","last_fullsized":"2024-09-17T02:41:55.825082+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":765006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":174638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1011712,"data_stored":765006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509460,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":430080,"data_stored":181305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":195198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:42:05.695 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:42:05.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.860+0000 7fd2e8954640 1 -- 172.21.15.131:0/3606476879 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2e40ced00 msgr2=0x7fd2e406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:05.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.860+0000 7fd2e8954640 1 --2- 172.21.15.131:0/3606476879 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2e40ced00 0x7fd2e406dea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd2d4009900 tx=0x7fd2d401c8e0 comp rx=0 tx=0).stop 2024-09-17T02:42:05.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.860+0000 7fd2e8954640 1 -- 172.21.15.131:0/3606476879 shutdown_connections 2024-09-17T02:42:05.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.860+0000 7fd2e8954640 1 --2- 172.21.15.131:0/3606476879 >> v2:172.21.15.189:3300/0 conn(0x7fd2e406e3e0 0x7fd2e415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:05.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.860+0000 7fd2e8954640 1 --2- 172.21.15.131:0/3606476879 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2e40ced00 0x7fd2e406dea0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:05.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.860+0000 7fd2e8954640 1 -- 172.21.15.131:0/3606476879 >> 172.21.15.131:0/3606476879 conn(0x7fd2e406eea0 msgr2=0x7fd2e406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:05.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.860+0000 7fd2e8954640 1 -- 172.21.15.131:0/3606476879 shutdown_connections 2024-09-17T02:42:05.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.860+0000 7fd2e8954640 1 -- 172.21.15.131:0/3606476879 wait complete. 2024-09-17T02:42:05.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e8954640 1 Processor -- start 2024-09-17T02:42:05.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e8954640 1 -- start start 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e8954640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2e406e3e0 0x7fd2e41f9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e8954640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2e40ced00 0x7fd2e4151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e8954640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2e415c3e0 0x7fd2e40a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e8954640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4168b60 con 0x7fd2e406e3e0 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e8954640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e41689e0 con 0x7fd2e40ced00 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e8954640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd2e4168ce0 con 0x7fd2e415c3e0 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2e406e3e0 0x7fd2e41f9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2e406e3e0 0x7fd2e41f9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57112/0 (socket says 172.21.15.131:57112) 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e259c640 1 -- 172.21.15.131:0/3487907565 learned_addr learned my addr 172.21.15.131:0/3487907565 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e2d9d640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2e415c3e0 0x7fd2e40a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e1d9b640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2e40ced00 0x7fd2e4151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:05.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e259c640 1 -- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2e415c3e0 msgr2=0x7fd2e40a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e259c640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2e415c3e0 0x7fd2e40a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e259c640 1 -- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2e40ced00 msgr2=0x7fd2e4151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e259c640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2e40ced00 0x7fd2e4151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e259c640 1 -- 172.21.15.131:0/3487907565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e41f9880 con 0x7fd2e406e3e0 2024-09-17T02:42:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e1d9b640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2e40ced00 0x7fd2e4151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e259c640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2e406e3e0 0x7fd2e41f9340 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd2d401af90 tx=0x7fd2d40a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2d2ffd640 1 -- 172.21.15.131:0/3487907565 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d40bc070 con 0x7fd2e406e3e0 2024-09-17T02:42:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2d2ffd640 1 -- 172.21.15.131:0/3487907565 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2d40043b0 con 0x7fd2e406e3e0 2024-09-17T02:42:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2d2ffd640 1 -- 172.21.15.131:0/3487907565 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2d40b2040 con 0x7fd2e406e3e0 2024-09-17T02:42:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e8954640 1 -- 172.21.15.131:0/3487907565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e4155af0 con 0x7fd2e406e3e0 2024-09-17T02:42:05.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e8954640 1 -- 172.21.15.131:0/3487907565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e4155810 con 0x7fd2e406e3e0 2024-09-17T02:42:05.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.864+0000 7fd2e8954640 1 -- 172.21.15.131:0/3487907565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e415bea0 con 0x7fd2e406e3e0 2024-09-17T02:42:05.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.868+0000 7fd2d2ffd640 1 -- 172.21.15.131:0/3487907565 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd2d4005280 con 0x7fd2e406e3e0 2024-09-17T02:42:05.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.868+0000 7fd2d2ffd640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd2c4073200 0x7fd2c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.868+0000 7fd2d2ffd640 1 -- 172.21.15.131:0/3487907565 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fd2d412e3e0 con 0x7fd2e406e3e0 2024-09-17T02:42:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.868+0000 7fd2e1d9b640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd2c4073200 0x7fd2c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.868+0000 7fd2e1d9b640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd2c4073200 0x7fd2c4075710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40af190 tx=0x7fd2d80091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:05.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:05.872+0000 7fd2d2ffd640 1 -- 172.21.15.131:0/3487907565 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd2e41f9880 con 0x7fd2e406e3e0 2024-09-17T02:42:06.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.040+0000 7fd2e8954640 1 -- 172.21.15.131:0/3487907565 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2e40743e0 con 0x7fd2c4073200 2024-09-17T02:42:06.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.052+0000 7fd2d2ffd640 1 -- 172.21.15.131:0/3487907565 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7fd2e40743e0 con 0x7fd2c4073200 2024-09-17T02:42:06.058 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:06.060 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:42:06.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 -- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd2c4073200 msgr2=0x7fd2c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd2c4073200 0x7fd2c4075710 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd2e40af190 tx=0x7fd2d80091e0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 -- 172.21.15.131:0/3487907565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2e406e3e0 msgr2=0x7fd2e41f9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2e406e3e0 0x7fd2e41f9340 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd2d401af90 tx=0x7fd2d40a5690 comp rx=0 tx=0).stop 2024-09-17T02:42:06.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 -- 172.21.15.131:0/3487907565 shutdown_connections 2024-09-17T02:42:06.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd2c4073200 0x7fd2c4075710 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2e415c3e0 0x7fd2e40a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2e40ced00 0x7fd2e4151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 --2- 172.21.15.131:0/3487907565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2e406e3e0 0x7fd2e41f9340 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 -- 172.21.15.131:0/3487907565 >> 172.21.15.131:0/3487907565 conn(0x7fd2e406eea0 msgr2=0x7fd2e40b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:06.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 -- 172.21.15.131:0/3487907565 shutdown_connections 2024-09-17T02:42:06.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.056+0000 7fd2e8954640 1 -- 172.21.15.131:0/3487907565 wait complete. 2024-09-17T02:42:06.078 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2024-09-17T02:42:05.145518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229324,"kb_used_data":4776,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745396,"statfs":{"total":773094113280,"available":772859285504,"internally_reserved":0,"allocated":4890624,"data_stored":2746868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":53,"apply_latency_ms":53,"commit_latency_ns":53000000,"apply_latency_ns":53000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.313752"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.854095+0000","last_change":"2024-09-17T02:41:56.884767+0000","last_active":"2024-09-17T02:41:57.854095+0000","last_peered":"2024-09-17T02:41:57.854095+0000","last_clean":"2024-09-17T02:41:57.854095+0000","last_became_active":"2024-09-17T02:41:56.884094+0000","last_became_peered":"2024-09-17T02:41:56.884094+0000","last_unstale":"2024-09-17T02:41:57.854095+0000","last_undegraded":"2024-09-17T02:41:57.854095+0000","last_fullsized":"2024-09-17T02:41:57.854095+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:22.832739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858600+0000","last_change":"2024-09-17T02:41:56.867810+0000","last_active":"2024-09-17T02:41:57.858600+0000","last_peered":"2024-09-17T02:41:57.858600+0000","last_clean":"2024-09-17T02:41:57.858600+0000","last_became_active":"2024-09-17T02:41:56.866944+0000","last_became_peered":"2024-09-17T02:41:56.866944+0000","last_unstale":"2024-09-17T02:41:57.858600+0000","last_undegraded":"2024-09-17T02:41:57.858600+0000","last_fullsized":"2024-09-17T02:41:57.858600+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:29:59.549330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.855566+0000","last_change":"2024-09-17T02:41:56.886043+0000","last_active":"2024-09-17T02:41:57.855566+0000","last_peered":"2024-09-17T02:41:57.855566+0000","last_clean":"2024-09-17T02:41:57.855566+0000","last_became_active":"2024-09-17T02:41:56.873813+0000","last_became_peered":"2024-09-17T02:41:56.873813+0000","last_unstale":"2024-09-17T02:41:57.855566+0000","last_undegraded":"2024-09-17T02:41:57.855566+0000","last_fullsized":"2024-09-17T02:41:57.855566+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:14:41.775259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.853367+0000","last_change":"2024-09-17T02:41:56.892113+0000","last_active":"2024-09-17T02:41:57.853367+0000","last_peered":"2024-09-17T02:41:57.853367+0000","last_clean":"2024-09-17T02:41:57.853367+0000","last_became_active":"2024-09-17T02:41:56.891556+0000","last_became_peered":"2024-09-17T02:41:56.891556+0000","last_unstale":"2024-09-17T02:41:57.853367+0000","last_undegraded":"2024-09-17T02:41:57.853367+0000","last_fullsized":"2024-09-17T02:41:57.853367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:38.726707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098582+0000","last_change":"2024-09-17T02:41:56.870576+0000","last_active":"2024-09-17T02:41:58.098582+0000","last_peered":"2024-09-17T02:41:58.098582+0000","last_clean":"2024-09-17T02:41:58.098582+0000","last_became_active":"2024-09-17T02:41:56.869441+0000","last_became_peered":"2024-09-17T02:41:56.869441+0000","last_unstale":"2024-09-17T02:41:58.098582+0000","last_undegraded":"2024-09-17T02:41:58.098582+0000","last_fullsized":"2024-09-17T02:41:58.098582+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:10.665693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857974+0000","last_change":"2024-09-17T02:41:56.880704+0000","last_active":"2024-09-17T02:41:57.857974+0000","last_peered":"2024-09-17T02:41:57.857974+0000","last_clean":"2024-09-17T02:41:57.857974+0000","last_became_active":"2024-09-17T02:41:56.879945+0000","last_became_peered":"2024-09-17T02:41:56.879945+0000","last_unstale":"2024-09-17T02:41:57.857974+0000","last_undegraded":"2024-09-17T02:41:57.857974+0000","last_fullsized":"2024-09-17T02:41:57.857974+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:36.026817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.096965+0000","last_change":"2024-09-17T02:41:56.860251+0000","last_active":"2024-09-17T02:41:58.096965+0000","last_peered":"2024-09-17T02:41:58.096965+0000","last_clean":"2024-09-17T02:41:58.096965+0000","last_became_active":"2024-09-17T02:41:56.859777+0000","last_became_peered":"2024-09-17T02:41:56.859777+0000","last_unstale":"2024-09-17T02:41:58.096965+0000","last_undegraded":"2024-09-17T02:41:58.096965+0000","last_fullsized":"2024-09-17T02:41:58.096965+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:28:10.665693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.852609+0000","last_change":"2024-09-17T02:41:56.884064+0000","last_active":"2024-09-17T02:41:57.852609+0000","last_peered":"2024-09-17T02:41:57.852609+0000","last_clean":"2024-09-17T02:41:57.852609+0000","last_became_active":"2024-09-17T02:41:56.874185+0000","last_became_peered":"2024-09-17T02:41:56.874185+0000","last_unstale":"2024-09-17T02:41:57.852609+0000","last_undegraded":"2024-09-17T02:41:57.852609+0000","last_fullsized":"2024-09-17T02:41:57.852609+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:18:50.391030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.100255+0000","last_change":"2024-09-17T02:41:56.878199+0000","last_active":"2024-09-17T02:41:58.100255+0000","last_peered":"2024-09-17T02:41:58.100255+0000","last_clean":"2024-09-17T02:41:58.100255+0000","last_became_active":"2024-09-17T02:41:56.876761+0000","last_became_peered":"2024-09-17T02:41:56.876761+0000","last_unstale":"2024-09-17T02:41:58.100255+0000","last_undegraded":"2024-09-17T02:41:58.100255+0000","last_fullsized":"2024-09-17T02:41:58.100255+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:35.532417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098834+0000","last_change":"2024-09-17T02:41:56.863721+0000","last_active":"2024-09-17T02:41:58.098834+0000","last_peered":"2024-09-17T02:41:58.098834+0000","last_clean":"2024-09-17T02:41:58.098834+0000","last_became_active":"2024-09-17T02:41:56.863256+0000","last_became_peered":"2024-09-17T02:41:56.863256+0000","last_unstale":"2024-09-17T02:41:58.098834+0000","last_undegraded":"2024-09-17T02:41:58.098834+0000","last_fullsized":"2024-09-17T02:41:58.098834+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:40.984139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.882881+0000","last_change":"2024-09-17T02:41:56.849568+0000","last_active":"2024-09-17T02:41:57.882881+0000","last_peered":"2024-09-17T02:41:57.882881+0000","last_clean":"2024-09-17T02:41:57.882881+0000","last_became_active":"2024-09-17T02:41:56.849045+0000","last_became_peered":"2024-09-17T02:41:56.849045+0000","last_unstale":"2024-09-17T02:41:57.882881+0000","last_undegraded":"2024-09-17T02:41:57.882881+0000","last_fullsized":"2024-09-17T02:41:57.882881+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:10.326577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.099487+0000","last_change":"2024-09-17T02:41:56.885089+0000","last_active":"2024-09-17T02:41:58.099487+0000","last_peered":"2024-09-17T02:41:58.099487+0000","last_clean":"2024-09-17T02:41:58.099487+0000","last_became_active":"2024-09-17T02:41:56.884324+0000","last_became_peered":"2024-09-17T02:41:56.884324+0000","last_unstale":"2024-09-17T02:41:58.099487+0000","last_undegraded":"2024-09-17T02:41:58.099487+0000","last_fullsized":"2024-09-17T02:41:58.099487+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:14.462920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.097572+0000","last_change":"2024-09-17T02:41:56.884549+0000","last_active":"2024-09-17T02:41:58.097572+0000","last_peered":"2024-09-17T02:41:58.097572+0000","last_clean":"2024-09-17T02:41:58.097572+0000","last_became_active":"2024-09-17T02:41:56.884210+0000","last_became_peered":"2024-09-17T02:41:56.884210+0000","last_unstale":"2024-09-17T02:41:58.097572+0000","last_undegraded":"2024-09-17T02:41:58.097572+0000","last_fullsized":"2024-09-17T02:41:58.097572+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:14.462920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098908+0000","last_change":"2024-09-17T02:41:56.872847+0000","last_active":"2024-09-17T02:41:58.098908+0000","last_peered":"2024-09-17T02:41:58.098908+0000","last_clean":"2024-09-17T02:41:58.098908+0000","last_became_active":"2024-09-17T02:41:56.867241+0000","last_became_peered":"2024-09-17T02:41:56.867241+0000","last_unstale":"2024-09-17T02:41:58.098908+0000","last_undegraded":"2024-09-17T02:41:58.098908+0000","last_fullsized":"2024-09-17T02:41:58.098908+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:46:06.859000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.849938+0000","last_change":"2024-09-17T02:41:56.875888+0000","last_active":"2024-09-17T02:41:57.849938+0000","last_peered":"2024-09-17T02:41:57.849938+0000","last_clean":"2024-09-17T02:41:57.849938+0000","last_became_active":"2024-09-17T02:41:56.875585+0000","last_became_peered":"2024-09-17T02:41:56.875585+0000","last_unstale":"2024-09-17T02:41:57.849938+0000","last_undegraded":"2024-09-17T02:41:57.849938+0000","last_fullsized":"2024-09-17T02:41:57.849938+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:05.453149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.096627+0000","last_change":"2024-09-17T02:41:56.887878+0000","last_active":"2024-09-17T02:41:58.096627+0000","last_peered":"2024-09-17T02:41:58.096627+0000","last_clean":"2024-09-17T02:41:58.096627+0000","last_became_active":"2024-09-17T02:41:56.887199+0000","last_became_peered":"2024-09-17T02:41:56.887199+0000","last_unstale":"2024-09-17T02:41:58.096627+0000","last_undegraded":"2024-09-17T02:41:58.096627+0000","last_fullsized":"2024-09-17T02:41:58.096627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:21.210898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.850945+0000","last_change":"2024-09-17T02:41:56.867306+0000","last_active":"2024-09-17T02:41:57.850945+0000","last_peered":"2024-09-17T02:41:57.850945+0000","last_clean":"2024-09-17T02:41:57.850945+0000","last_became_active":"2024-09-17T02:41:56.866978+0000","last_became_peered":"2024-09-17T02:41:56.866978+0000","last_unstale":"2024-09-17T02:41:57.850945+0000","last_undegraded":"2024-09-17T02:41:57.850945+0000","last_fullsized":"2024-09-17T02:41:57.850945+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:32:57.166128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.850416+0000","last_change":"2024-09-17T02:41:56.859795+0000","last_active":"2024-09-17T02:41:57.850416+0000","last_peered":"2024-09-17T02:41:57.850416+0000","last_clean":"2024-09-17T02:41:57.850416+0000","last_became_active":"2024-09-17T02:41:56.859462+0000","last_became_peered":"2024-09-17T02:41:56.859462+0000","last_unstale":"2024-09-17T02:41:57.850416+0000","last_undegraded":"2024-09-17T02:41:57.850416+0000","last_fullsized":"2024-09-17T02:41:57.850416+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:06:18.104414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098975+0000","last_change":"2024-09-17T02:41:56.895484+0000","last_active":"2024-09-17T02:41:58.098975+0000","last_peered":"2024-09-17T02:41:58.098975+0000","last_clean":"2024-09-17T02:41:58.098975+0000","last_became_active":"2024-09-17T02:41:56.894846+0000","last_became_peered":"2024-09-17T02:41:56.894846+0000","last_unstale":"2024-09-17T02:41:58.098975+0000","last_undegraded":"2024-09-17T02:41:58.098975+0000","last_fullsized":"2024-09-17T02:41:58.098975+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:12.663773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.882419+0000","last_change":"2024-09-17T02:41:56.861765+0000","last_active":"2024-09-17T02:41:57.882419+0000","last_peered":"2024-09-17T02:41:57.882419+0000","last_clean":"2024-09-17T02:41:57.882419+0000","last_became_active":"2024-09-17T02:41:56.861415+0000","last_became_peered":"2024-09-17T02:41:56.861415+0000","last_unstale":"2024-09-17T02:41:57.882419+0000","last_undegraded":"2024-09-17T02:41:57.882419+0000","last_fullsized":"2024-09-17T02:41:57.882419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:54:41.873098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.882358+0000","last_change":"2024-09-17T02:41:56.887565+0000","last_active":"2024-09-17T02:41:57.882358+0000","last_peered":"2024-09-17T02:41:57.882358+0000","last_clean":"2024-09-17T02:41:57.882358+0000","last_became_active":"2024-09-17T02:41:56.887199+0000","last_became_peered":"2024-09-17T02:41:56.887199+0000","last_unstale":"2024-09-17T02:41:57.882358+0000","last_undegraded":"2024-09-17T02:41:57.882358+0000","last_fullsized":"2024-09-17T02:41:57.882358+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:35.986102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.881795+0000","last_change":"2024-09-17T02:41:56.883993+0000","last_active":"2024-09-17T02:41:57.881795+0000","last_peered":"2024-09-17T02:41:57.881795+0000","last_clean":"2024-09-17T02:41:57.881795+0000","last_became_active":"2024-09-17T02:41:56.883729+0000","last_became_peered":"2024-09-17T02:41:56.883729+0000","last_unstale":"2024-09-17T02:41:57.881795+0000","last_undegraded":"2024-09-17T02:41:57.881795+0000","last_fullsized":"2024-09-17T02:41:57.881795+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:03.949057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.849804+0000","last_change":"2024-09-17T02:41:56.863169+0000","last_active":"2024-09-17T02:41:57.849804+0000","last_peered":"2024-09-17T02:41:57.849804+0000","last_clean":"2024-09-17T02:41:57.849804+0000","last_became_active":"2024-09-17T02:41:56.862841+0000","last_became_peered":"2024-09-17T02:41:56.862841+0000","last_unstale":"2024-09-17T02:41:57.849804+0000","last_undegraded":"2024-09-17T02:41:57.849804+0000","last_fullsized":"2024-09-17T02:41:57.849804+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:38:40.929441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.853350+0000","last_change":"2024-09-17T02:41:56.897213+0000","last_active":"2024-09-17T02:41:57.853350+0000","last_peered":"2024-09-17T02:41:57.853350+0000","last_clean":"2024-09-17T02:41:57.853350+0000","last_became_active":"2024-09-17T02:41:56.896396+0000","last_became_peered":"2024-09-17T02:41:56.896396+0000","last_unstale":"2024-09-17T02:41:57.853350+0000","last_undegraded":"2024-09-17T02:41:57.853350+0000","last_fullsized":"2024-09-17T02:41:57.853350+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:10.101272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098599+0000","last_change":"2024-09-17T02:41:56.884415+0000","last_active":"2024-09-17T02:41:58.098599+0000","last_peered":"2024-09-17T02:41:58.098599+0000","last_clean":"2024-09-17T02:41:58.098599+0000","last_became_active":"2024-09-17T02:41:56.883907+0000","last_became_peered":"2024-09-17T02:41:56.883907+0000","last_unstale":"2024-09-17T02:41:58.098599+0000","last_undegraded":"2024-09-17T02:41:58.098599+0000","last_fullsized":"2024-09-17T02:41:58.098599+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:52.116933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.853677+0000","last_change":"2024-09-17T02:41:56.896359+0000","last_active":"2024-09-17T02:41:57.853677+0000","last_peered":"2024-09-17T02:41:57.853677+0000","last_clean":"2024-09-17T02:41:57.853677+0000","last_became_active":"2024-09-17T02:41:56.895820+0000","last_became_peered":"2024-09-17T02:41:56.895820+0000","last_unstale":"2024-09-17T02:41:57.853677+0000","last_undegraded":"2024-09-17T02:41:57.853677+0000","last_fullsized":"2024-09-17T02:41:57.853677+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:50:56.642000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.882934+0000","last_change":"2024-09-17T02:41:56.880643+0000","last_active":"2024-09-17T02:41:57.882934+0000","last_peered":"2024-09-17T02:41:57.882934+0000","last_clean":"2024-09-17T02:41:57.882934+0000","last_became_active":"2024-09-17T02:41:56.880324+0000","last_became_peered":"2024-09-17T02:41:56.880324+0000","last_unstale":"2024-09-17T02:41:57.882934+0000","last_undegraded":"2024-09-17T02:41:57.882934+0000","last_fullsized":"2024-09-17T02:41:57.882934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:38.400600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098576+0000","last_change":"2024-09-17T02:41:56.887909+0000","last_active":"2024-09-17T02:41:58.098576+0000","last_peered":"2024-09-17T02:41:58.098576+0000","last_clean":"2024-09-17T02:41:58.098576+0000","last_became_active":"2024-09-17T02:41:56.887513+0000","last_became_peered":"2024-09-17T02:41:56.887513+0000","last_unstale":"2024-09-17T02:41:58.098576+0000","last_undegraded":"2024-09-17T02:41:58.098576+0000","last_fullsized":"2024-09-17T02:41:58.098576+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:35.532417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.851027+0000","last_change":"2024-09-17T02:41:56.858578+0000","last_active":"2024-09-17T02:41:57.851027+0000","last_peered":"2024-09-17T02:41:57.851027+0000","last_clean":"2024-09-17T02:41:57.851027+0000","last_became_active":"2024-09-17T02:41:56.857981+0000","last_became_peered":"2024-09-17T02:41:56.857981+0000","last_unstale":"2024-09-17T02:41:57.851027+0000","last_undegraded":"2024-09-17T02:41:57.851027+0000","last_fullsized":"2024-09-17T02:41:57.851027+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:46:08.323746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.850277+0000","last_change":"2024-09-17T02:41:56.868387+0000","last_active":"2024-09-17T02:41:57.850277+0000","last_peered":"2024-09-17T02:41:57.850277+0000","last_clean":"2024-09-17T02:41:57.850277+0000","last_became_active":"2024-09-17T02:41:56.867932+0000","last_became_peered":"2024-09-17T02:41:56.867932+0000","last_unstale":"2024-09-17T02:41:57.850277+0000","last_undegraded":"2024-09-17T02:41:57.850277+0000","last_fullsized":"2024-09-17T02:41:57.850277+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:42.027790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.097644+0000","last_change":"2024-09-17T02:41:56.884950+0000","last_active":"2024-09-17T02:41:58.097644+0000","last_peered":"2024-09-17T02:41:58.097644+0000","last_clean":"2024-09-17T02:41:58.097644+0000","last_became_active":"2024-09-17T02:41:56.884574+0000","last_became_peered":"2024-09-17T02:41:56.884574+0000","last_unstale":"2024-09-17T02:41:58.097644+0000","last_undegraded":"2024-09-17T02:41:58.097644+0000","last_fullsized":"2024-09-17T02:41:58.097644+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:15:36.404019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.854169+0000","last_change":"2024-09-17T02:41:56.886004+0000","last_active":"2024-09-17T02:41:57.854169+0000","last_peered":"2024-09-17T02:41:57.854169+0000","last_clean":"2024-09-17T02:41:57.854169+0000","last_became_active":"2024-09-17T02:41:56.874215+0000","last_became_peered":"2024-09-17T02:41:56.874215+0000","last_unstale":"2024-09-17T02:41:57.854169+0000","last_undegraded":"2024-09-17T02:41:57.854169+0000","last_fullsized":"2024-09-17T02:41:57.854169+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:14:54.367742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.851395+0000","last_change":"2024-09-17T02:41:56.874658+0000","last_active":"2024-09-17T02:41:57.851395+0000","last_peered":"2024-09-17T02:41:57.851395+0000","last_clean":"2024-09-17T02:41:57.851395+0000","last_became_active":"2024-09-17T02:41:56.874312+0000","last_became_peered":"2024-09-17T02:41:56.874312+0000","last_unstale":"2024-09-17T02:41:57.851395+0000","last_undegraded":"2024-09-17T02:41:57.851395+0000","last_fullsized":"2024-09-17T02:41:57.851395+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:50:21.424628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"39'4","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.099097+0000","last_change":"2024-09-17T02:41:55.843782+0000","last_active":"2024-09-17T02:41:58.099097+0000","last_peered":"2024-09-17T02:41:58.099097+0000","last_clean":"2024-09-17T02:41:58.099097+0000","last_became_active":"2024-09-17T02:41:55.843362+0000","last_became_peered":"2024-09-17T02:41:55.843362+0000","last_unstale":"2024-09-17T02:41:58.099097+0000","last_undegraded":"2024-09-17T02:41:58.099097+0000","last_fullsized":"2024-09-17T02:41:58.099097+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:39.980773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858515+0000","last_change":"2024-09-17T02:41:56.882518+0000","last_active":"2024-09-17T02:41:57.858515+0000","last_peered":"2024-09-17T02:41:57.858515+0000","last_clean":"2024-09-17T02:41:57.858515+0000","last_became_active":"2024-09-17T02:41:56.879984+0000","last_became_peered":"2024-09-17T02:41:56.879984+0000","last_unstale":"2024-09-17T02:41:57.858515+0000","last_undegraded":"2024-09-17T02:41:57.858515+0000","last_fullsized":"2024-09-17T02:41:57.858515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:35.041247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858745+0000","last_change":"2024-09-17T02:41:55.844353+0000","last_active":"2024-09-17T02:41:57.858745+0000","last_peered":"2024-09-17T02:41:57.858745+0000","last_clean":"2024-09-17T02:41:57.858745+0000","last_became_active":"2024-09-17T02:41:55.843505+0000","last_became_peered":"2024-09-17T02:41:55.843505+0000","last_unstale":"2024-09-17T02:41:57.858745+0000","last_undegraded":"2024-09-17T02:41:57.858745+0000","last_fullsized":"2024-09-17T02:41:57.858745+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:43.510214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.850643+0000","last_change":"2024-09-17T02:41:56.878765+0000","last_active":"2024-09-17T02:41:57.850643+0000","last_peered":"2024-09-17T02:41:57.850643+0000","last_clean":"2024-09-17T02:41:57.850643+0000","last_became_active":"2024-09-17T02:41:56.878435+0000","last_became_peered":"2024-09-17T02:41:56.878435+0000","last_unstale":"2024-09-17T02:41:57.850643+0000","last_undegraded":"2024-09-17T02:41:57.850643+0000","last_fullsized":"2024-09-17T02:41:57.850643+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:43:35.073657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.856294+0000","last_change":"2024-09-17T02:41:55.865708+0000","last_active":"2024-09-17T02:41:57.856294+0000","last_peered":"2024-09-17T02:41:57.856294+0000","last_clean":"2024-09-17T02:41:57.856294+0000","last_became_active":"2024-09-17T02:41:55.861337+0000","last_became_peered":"2024-09-17T02:41:55.861337+0000","last_unstale":"2024-09-17T02:41:57.856294+0000","last_undegraded":"2024-09-17T02:41:57.856294+0000","last_fullsized":"2024-09-17T02:41:57.856294+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:21:51.113568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.859238+0000","last_change":"2024-09-17T02:41:56.867038+0000","last_active":"2024-09-17T02:41:57.859238+0000","last_peered":"2024-09-17T02:41:57.859238+0000","last_clean":"2024-09-17T02:41:57.859238+0000","last_became_active":"2024-09-17T02:41:56.865986+0000","last_became_peered":"2024-09-17T02:41:56.865986+0000","last_unstale":"2024-09-17T02:41:57.859238+0000","last_undegraded":"2024-09-17T02:41:57.859238+0000","last_fullsized":"2024-09-17T02:41:57.859238+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:43:31.091078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098324+0000","last_change":"2024-09-17T02:41:55.859306+0000","last_active":"2024-09-17T02:41:58.098324+0000","last_peered":"2024-09-17T02:41:58.098324+0000","last_clean":"2024-09-17T02:41:58.098324+0000","last_became_active":"2024-09-17T02:41:55.858980+0000","last_became_peered":"2024-09-17T02:41:55.858980+0000","last_unstale":"2024-09-17T02:41:58.098324+0000","last_undegraded":"2024-09-17T02:41:58.098324+0000","last_fullsized":"2024-09-17T02:41:58.098324+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:20.207532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.854816+0000","last_change":"2024-09-17T02:41:56.861369+0000","last_active":"2024-09-17T02:41:57.854816+0000","last_peered":"2024-09-17T02:41:57.854816+0000","last_clean":"2024-09-17T02:41:57.854816+0000","last_became_active":"2024-09-17T02:41:56.860275+0000","last_became_peered":"2024-09-17T02:41:56.860275+0000","last_unstale":"2024-09-17T02:41:57.854816+0000","last_undegraded":"2024-09-17T02:41:57.854816+0000","last_fullsized":"2024-09-17T02:41:57.854816+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:53.499867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098031+0000","last_change":"2024-09-17T02:41:55.832205+0000","last_active":"2024-09-17T02:41:58.098031+0000","last_peered":"2024-09-17T02:41:58.098031+0000","last_clean":"2024-09-17T02:41:58.098031+0000","last_became_active":"2024-09-17T02:41:55.831773+0000","last_became_peered":"2024-09-17T02:41:55.831773+0000","last_unstale":"2024-09-17T02:41:58.098031+0000","last_undegraded":"2024-09-17T02:41:58.098031+0000","last_fullsized":"2024-09-17T02:41:58.098031+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:34.529052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.881818+0000","last_change":"2024-09-17T02:41:56.853587+0000","last_active":"2024-09-17T02:41:57.881818+0000","last_peered":"2024-09-17T02:41:57.881818+0000","last_clean":"2024-09-17T02:41:57.881818+0000","last_became_active":"2024-09-17T02:41:56.853265+0000","last_became_peered":"2024-09-17T02:41:56.853265+0000","last_unstale":"2024-09-17T02:41:57.881818+0000","last_undegraded":"2024-09-17T02:41:57.881818+0000","last_fullsized":"2024-09-17T02:41:57.881818+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:34.235228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.096893+0000","last_change":"2024-09-17T02:41:55.867371+0000","last_active":"2024-09-17T02:41:58.096893+0000","last_peered":"2024-09-17T02:41:58.096893+0000","last_clean":"2024-09-17T02:41:58.096893+0000","last_became_active":"2024-09-17T02:41:55.866967+0000","last_became_peered":"2024-09-17T02:41:55.866967+0000","last_unstale":"2024-09-17T02:41:58.096893+0000","last_undegraded":"2024-09-17T02:41:58.096893+0000","last_fullsized":"2024-09-17T02:41:58.096893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:33:12.978264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.851482+0000","last_change":"2024-09-17T02:41:56.858098+0000","last_active":"2024-09-17T02:41:57.851482+0000","last_peered":"2024-09-17T02:41:57.851482+0000","last_clean":"2024-09-17T02:41:57.851482+0000","last_became_active":"2024-09-17T02:41:56.857713+0000","last_became_peered":"2024-09-17T02:41:56.857713+0000","last_unstale":"2024-09-17T02:41:57.851482+0000","last_undegraded":"2024-09-17T02:41:57.851482+0000","last_fullsized":"2024-09-17T02:41:57.851482+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:19.122058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.852437+0000","last_change":"2024-09-17T02:41:55.862221+0000","last_active":"2024-09-17T02:41:57.852437+0000","last_peered":"2024-09-17T02:41:57.852437+0000","last_clean":"2024-09-17T02:41:57.852437+0000","last_became_active":"2024-09-17T02:41:55.861013+0000","last_became_peered":"2024-09-17T02:41:55.861013+0000","last_unstale":"2024-09-17T02:41:57.852437+0000","last_undegraded":"2024-09-17T02:41:57.852437+0000","last_fullsized":"2024-09-17T02:41:57.852437+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:28:27.751955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.859026+0000","last_change":"2024-09-17T02:41:56.875750+0000","last_active":"2024-09-17T02:41:57.859026+0000","last_peered":"2024-09-17T02:41:57.859026+0000","last_clean":"2024-09-17T02:41:57.859026+0000","last_became_active":"2024-09-17T02:41:56.874828+0000","last_became_peered":"2024-09-17T02:41:56.874828+0000","last_unstale":"2024-09-17T02:41:57.859026+0000","last_undegraded":"2024-09-17T02:41:57.859026+0000","last_fullsized":"2024-09-17T02:41:57.859026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:39.693566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.097891+0000","last_change":"2024-09-17T02:41:55.833725+0000","last_active":"2024-09-17T02:41:58.097891+0000","last_peered":"2024-09-17T02:41:58.097891+0000","last_clean":"2024-09-17T02:41:58.097891+0000","last_became_active":"2024-09-17T02:41:55.833225+0000","last_became_peered":"2024-09-17T02:41:55.833225+0000","last_unstale":"2024-09-17T02:41:58.097891+0000","last_undegraded":"2024-09-17T02:41:58.097891+0000","last_fullsized":"2024-09-17T02:41:58.097891+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:05.465016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.854026+0000","last_change":"2024-09-17T02:41:56.893169+0000","last_active":"2024-09-17T02:41:57.854026+0000","last_peered":"2024-09-17T02:41:57.854026+0000","last_clean":"2024-09-17T02:41:57.854026+0000","last_became_active":"2024-09-17T02:41:56.886848+0000","last_became_peered":"2024-09-17T02:41:56.886848+0000","last_unstale":"2024-09-17T02:41:57.854026+0000","last_undegraded":"2024-09-17T02:41:57.854026+0000","last_fullsized":"2024-09-17T02:41:57.854026+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:34.749887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.854615+0000","last_change":"2024-09-17T02:41:55.865849+0000","last_active":"2024-09-17T02:41:57.854615+0000","last_peered":"2024-09-17T02:41:57.854615+0000","last_clean":"2024-09-17T02:41:57.854615+0000","last_became_active":"2024-09-17T02:41:55.863121+0000","last_became_peered":"2024-09-17T02:41:55.863121+0000","last_unstale":"2024-09-17T02:41:57.854615+0000","last_undegraded":"2024-09-17T02:41:57.854615+0000","last_fullsized":"2024-09-17T02:41:57.854615+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:45.005426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098304+0000","last_change":"2024-09-17T02:41:56.854059+0000","last_active":"2024-09-17T02:41:58.098304+0000","last_peered":"2024-09-17T02:41:58.098304+0000","last_clean":"2024-09-17T02:41:58.098304+0000","last_became_active":"2024-09-17T02:41:56.853719+0000","last_became_peered":"2024-09-17T02:41:56.853719+0000","last_unstale":"2024-09-17T02:41:58.098304+0000","last_undegraded":"2024-09-17T02:41:58.098304+0000","last_fullsized":"2024-09-17T02:41:58.098304+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:06:52.056793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.850462+0000","last_change":"2024-09-17T02:41:55.846533+0000","last_active":"2024-09-17T02:41:57.850462+0000","last_peered":"2024-09-17T02:41:57.850462+0000","last_clean":"2024-09-17T02:41:57.850462+0000","last_became_active":"2024-09-17T02:41:55.846130+0000","last_became_peered":"2024-09-17T02:41:55.846130+0000","last_unstale":"2024-09-17T02:41:57.850462+0000","last_undegraded":"2024-09-17T02:41:57.850462+0000","last_fullsized":"2024-09-17T02:41:57.850462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:35.723537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.854310+0000","last_change":"2024-09-17T02:41:56.871843+0000","last_active":"2024-09-17T02:41:57.854310+0000","last_peered":"2024-09-17T02:41:57.854310+0000","last_clean":"2024-09-17T02:41:57.854310+0000","last_became_active":"2024-09-17T02:41:56.871008+0000","last_became_peered":"2024-09-17T02:41:56.871008+0000","last_unstale":"2024-09-17T02:41:57.854310+0000","last_undegraded":"2024-09-17T02:41:57.854310+0000","last_fullsized":"2024-09-17T02:41:57.854310+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:30:32.787222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"39'3","reported_seq":18,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.850406+0000","last_change":"2024-09-17T02:41:55.866610+0000","last_active":"2024-09-17T02:41:57.850406+0000","last_peered":"2024-09-17T02:41:57.850406+0000","last_clean":"2024-09-17T02:41:57.850406+0000","last_became_active":"2024-09-17T02:41:55.866246+0000","last_became_peered":"2024-09-17T02:41:55.866246+0000","last_unstale":"2024-09-17T02:41:57.850406+0000","last_undegraded":"2024-09-17T02:41:57.850406+0000","last_fullsized":"2024-09-17T02:41:57.850406+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:12.980554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.097260+0000","last_change":"2024-09-17T02:41:56.864537+0000","last_active":"2024-09-17T02:41:58.097260+0000","last_peered":"2024-09-17T02:41:58.097260+0000","last_clean":"2024-09-17T02:41:58.097260+0000","last_became_active":"2024-09-17T02:41:56.864272+0000","last_became_peered":"2024-09-17T02:41:56.864272+0000","last_unstale":"2024-09-17T02:41:58.097260+0000","last_undegraded":"2024-09-17T02:41:58.097260+0000","last_fullsized":"2024-09-17T02:41:58.097260+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:40.658405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"39'6","reported_seq":21,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.854669+0000","last_change":"2024-09-17T02:41:55.863340+0000","last_active":"2024-09-17T02:41:57.854669+0000","last_peered":"2024-09-17T02:41:57.854669+0000","last_clean":"2024-09-17T02:41:57.854669+0000","last_became_active":"2024-09-17T02:41:55.861822+0000","last_became_peered":"2024-09-17T02:41:55.861822+0000","last_unstale":"2024-09-17T02:41:57.854669+0000","last_undegraded":"2024-09-17T02:41:57.854669+0000","last_fullsized":"2024-09-17T02:41:57.854669+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:06:51.053427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.854978+0000","last_change":"2024-09-17T02:41:56.897910+0000","last_active":"2024-09-17T02:41:57.854978+0000","last_peered":"2024-09-17T02:41:57.854978+0000","last_clean":"2024-09-17T02:41:57.854978+0000","last_became_active":"2024-09-17T02:41:56.895646+0000","last_became_peered":"2024-09-17T02:41:56.895646+0000","last_unstale":"2024-09-17T02:41:57.854978+0000","last_undegraded":"2024-09-17T02:41:57.854978+0000","last_fullsized":"2024-09-17T02:41:57.854978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:34:54.300402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.852783+0000","last_change":"2024-09-17T02:41:55.866575+0000","last_active":"2024-09-17T02:41:57.852783+0000","last_peered":"2024-09-17T02:41:57.852783+0000","last_clean":"2024-09-17T02:41:57.852783+0000","last_became_active":"2024-09-17T02:41:55.862842+0000","last_became_peered":"2024-09-17T02:41:55.862842+0000","last_unstale":"2024-09-17T02:41:57.852783+0000","last_undegraded":"2024-09-17T02:41:57.852783+0000","last_fullsized":"2024-09-17T02:41:57.852783+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:19:24.726723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857825+0000","last_change":"2024-09-17T02:41:56.881999+0000","last_active":"2024-09-17T02:41:57.857825+0000","last_peered":"2024-09-17T02:41:57.857825+0000","last_clean":"2024-09-17T02:41:57.857825+0000","last_became_active":"2024-09-17T02:41:56.881115+0000","last_became_peered":"2024-09-17T02:41:56.881115+0000","last_unstale":"2024-09-17T02:41:57.857825+0000","last_undegraded":"2024-09-17T02:41:57.857825+0000","last_fullsized":"2024-09-17T02:41:57.857825+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:35.710278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857830+0000","last_change":"2024-09-17T02:41:55.839604+0000","last_active":"2024-09-17T02:41:57.857830+0000","last_peered":"2024-09-17T02:41:57.857830+0000","last_clean":"2024-09-17T02:41:57.857830+0000","last_became_active":"2024-09-17T02:41:55.835167+0000","last_became_peered":"2024-09-17T02:41:55.835167+0000","last_unstale":"2024-09-17T02:41:57.857830+0000","last_undegraded":"2024-09-17T02:41:57.857830+0000","last_fullsized":"2024-09-17T02:41:57.857830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:44.487692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.882734+0000","last_change":"2024-09-17T02:41:56.869703+0000","last_active":"2024-09-17T02:41:57.882734+0000","last_peered":"2024-09-17T02:41:57.882734+0000","last_clean":"2024-09-17T02:41:57.882734+0000","last_became_active":"2024-09-17T02:41:56.869385+0000","last_became_peered":"2024-09-17T02:41:56.869385+0000","last_unstale":"2024-09-17T02:41:57.882734+0000","last_undegraded":"2024-09-17T02:41:57.882734+0000","last_fullsized":"2024-09-17T02:41:57.882734+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:25:46.609031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.850748+0000","last_change":"2024-09-17T02:41:55.830025+0000","last_active":"2024-09-17T02:41:57.850748+0000","last_peered":"2024-09-17T02:41:57.850748+0000","last_clean":"2024-09-17T02:41:57.850748+0000","last_became_active":"2024-09-17T02:41:55.829641+0000","last_became_peered":"2024-09-17T02:41:55.829641+0000","last_unstale":"2024-09-17T02:41:57.850748+0000","last_undegraded":"2024-09-17T02:41:57.850748+0000","last_fullsized":"2024-09-17T02:41:57.850748+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:57:19.132964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.099163+0000","last_change":"2024-09-17T02:41:56.882737+0000","last_active":"2024-09-17T02:41:58.099163+0000","last_peered":"2024-09-17T02:41:58.099163+0000","last_clean":"2024-09-17T02:41:58.099163+0000","last_became_active":"2024-09-17T02:41:56.882026+0000","last_became_peered":"2024-09-17T02:41:56.882026+0000","last_unstale":"2024-09-17T02:41:58.099163+0000","last_undegraded":"2024-09-17T02:41:58.099163+0000","last_fullsized":"2024-09-17T02:41:58.099163+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:52:40.658405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"39'8","reported_seq":23,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.097746+0000","last_change":"2024-09-17T02:41:55.824740+0000","last_active":"2024-09-17T02:41:58.097746+0000","last_peered":"2024-09-17T02:41:58.097746+0000","last_clean":"2024-09-17T02:41:58.097746+0000","last_became_active":"2024-09-17T02:41:55.824315+0000","last_became_peered":"2024-09-17T02:41:55.824315+0000","last_unstale":"2024-09-17T02:41:58.097746+0000","last_undegraded":"2024-09-17T02:41:58.097746+0000","last_fullsized":"2024-09-17T02:41:58.097746+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:15:35.400654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858509+0000","last_change":"2024-09-17T02:41:56.881490+0000","last_active":"2024-09-17T02:41:57.858509+0000","last_peered":"2024-09-17T02:41:57.858509+0000","last_clean":"2024-09-17T02:41:57.858509+0000","last_became_active":"2024-09-17T02:41:56.880621+0000","last_became_peered":"2024-09-17T02:41:56.880621+0000","last_unstale":"2024-09-17T02:41:57.858509+0000","last_undegraded":"2024-09-17T02:41:57.858509+0000","last_fullsized":"2024-09-17T02:41:57.858509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:27.688857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.097136+0000","last_change":"2024-09-17T02:41:55.833534+0000","last_active":"2024-09-17T02:41:58.097136+0000","last_peered":"2024-09-17T02:41:58.097136+0000","last_clean":"2024-09-17T02:41:58.097136+0000","last_became_active":"2024-09-17T02:41:55.833218+0000","last_became_peered":"2024-09-17T02:41:55.833218+0000","last_unstale":"2024-09-17T02:41:58.097136+0000","last_undegraded":"2024-09-17T02:41:58.097136+0000","last_fullsized":"2024-09-17T02:41:58.097136+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:58.636057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.099322+0000","last_change":"2024-09-17T02:41:56.894182+0000","last_active":"2024-09-17T02:41:58.099322+0000","last_peered":"2024-09-17T02:41:58.099322+0000","last_clean":"2024-09-17T02:41:58.099322+0000","last_became_active":"2024-09-17T02:41:56.893842+0000","last_became_peered":"2024-09-17T02:41:56.893842+0000","last_unstale":"2024-09-17T02:41:58.099322+0000","last_undegraded":"2024-09-17T02:41:58.099322+0000","last_fullsized":"2024-09-17T02:41:58.099322+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:33.472544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.850820+0000","last_change":"2024-09-17T02:41:55.861548+0000","last_active":"2024-09-17T02:41:57.850820+0000","last_peered":"2024-09-17T02:41:57.850820+0000","last_clean":"2024-09-17T02:41:57.850820+0000","last_became_active":"2024-09-17T02:41:55.861200+0000","last_became_peered":"2024-09-17T02:41:55.861200+0000","last_unstale":"2024-09-17T02:41:57.850820+0000","last_undegraded":"2024-09-17T02:41:57.850820+0000","last_fullsized":"2024-09-17T02:41:57.850820+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:51.127549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.099346+0000","last_change":"2024-09-17T02:41:56.884595+0000","last_active":"2024-09-17T02:41:58.099346+0000","last_peered":"2024-09-17T02:41:58.099346+0000","last_clean":"2024-09-17T02:41:58.099346+0000","last_became_active":"2024-09-17T02:41:56.884263+0000","last_became_peered":"2024-09-17T02:41:56.884263+0000","last_unstale":"2024-09-17T02:41:58.099346+0000","last_undegraded":"2024-09-17T02:41:58.099346+0000","last_fullsized":"2024-09-17T02:41:58.099346+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:19:54.328345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"39'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098616+0000","last_change":"2024-09-17T02:41:55.844139+0000","last_active":"2024-09-17T02:41:58.098616+0000","last_peered":"2024-09-17T02:41:58.098616+0000","last_clean":"2024-09-17T02:41:58.098616+0000","last_became_active":"2024-09-17T02:41:55.842953+0000","last_became_peered":"2024-09-17T02:41:55.842953+0000","last_unstale":"2024-09-17T02:41:58.098616+0000","last_undegraded":"2024-09-17T02:41:58.098616+0000","last_fullsized":"2024-09-17T02:41:58.098616+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:13:07.376197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.099325+0000","last_change":"2024-09-17T02:41:56.895730+0000","last_active":"2024-09-17T02:41:58.099325+0000","last_peered":"2024-09-17T02:41:58.099325+0000","last_clean":"2024-09-17T02:41:58.099325+0000","last_became_active":"2024-09-17T02:41:56.895099+0000","last_became_peered":"2024-09-17T02:41:56.895099+0000","last_unstale":"2024-09-17T02:41:58.099325+0000","last_undegraded":"2024-09-17T02:41:58.099325+0000","last_fullsized":"2024-09-17T02:41:58.099325+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:42.343206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098700+0000","last_change":"2024-09-17T02:41:55.840256+0000","last_active":"2024-09-17T02:41:58.098700+0000","last_peered":"2024-09-17T02:41:58.098700+0000","last_clean":"2024-09-17T02:41:58.098700+0000","last_became_active":"2024-09-17T02:41:55.839277+0000","last_became_peered":"2024-09-17T02:41:55.839277+0000","last_unstale":"2024-09-17T02:41:58.098700+0000","last_undegraded":"2024-09-17T02:41:58.098700+0000","last_fullsized":"2024-09-17T02:41:58.098700+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:46:05.855634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.849811+0000","last_change":"2024-09-17T02:41:56.865932+0000","last_active":"2024-09-17T02:41:57.849811+0000","last_peered":"2024-09-17T02:41:57.849811+0000","last_clean":"2024-09-17T02:41:57.849811+0000","last_became_active":"2024-09-17T02:41:56.865491+0000","last_became_peered":"2024-09-17T02:41:56.865491+0000","last_unstale":"2024-09-17T02:41:57.849811+0000","last_undegraded":"2024-09-17T02:41:57.849811+0000","last_fullsized":"2024-09-17T02:41:57.849811+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:55.820989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.099110+0000","last_change":"2024-09-17T02:41:55.846024+0000","last_active":"2024-09-17T02:41:58.099110+0000","last_peered":"2024-09-17T02:41:58.099110+0000","last_clean":"2024-09-17T02:41:58.099110+0000","last_became_active":"2024-09-17T02:41:55.845095+0000","last_became_peered":"2024-09-17T02:41:55.845095+0000","last_unstale":"2024-09-17T02:41:58.099110+0000","last_undegraded":"2024-09-17T02:41:58.099110+0000","last_fullsized":"2024-09-17T02:41:58.099110+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:25.053792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.852638+0000","last_change":"2024-09-17T02:41:56.871439+0000","last_active":"2024-09-17T02:41:57.852638+0000","last_peered":"2024-09-17T02:41:57.852638+0000","last_clean":"2024-09-17T02:41:57.852638+0000","last_became_active":"2024-09-17T02:41:56.870577+0000","last_became_peered":"2024-09-17T02:41:56.870577+0000","last_unstale":"2024-09-17T02:41:57.852638+0000","last_undegraded":"2024-09-17T02:41:57.852638+0000","last_fullsized":"2024-09-17T02:41:57.852638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:48.541158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858145+0000","last_change":"2024-09-17T02:41:55.838665+0000","last_active":"2024-09-17T02:41:57.858145+0000","last_peered":"2024-09-17T02:41:57.858145+0000","last_clean":"2024-09-17T02:41:57.858145+0000","last_became_active":"2024-09-17T02:41:55.837817+0000","last_became_peered":"2024-09-17T02:41:55.837817+0000","last_unstale":"2024-09-17T02:41:57.858145+0000","last_undegraded":"2024-09-17T02:41:57.858145+0000","last_fullsized":"2024-09-17T02:41:57.858145+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:05.524969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.882226+0000","last_change":"2024-09-17T02:41:56.860294+0000","last_active":"2024-09-17T02:41:57.882226+0000","last_peered":"2024-09-17T02:41:57.882226+0000","last_clean":"2024-09-17T02:41:57.882226+0000","last_became_active":"2024-09-17T02:41:56.855828+0000","last_became_peered":"2024-09-17T02:41:56.855828+0000","last_unstale":"2024-09-17T02:41:57.882226+0000","last_undegraded":"2024-09-17T02:41:57.882226+0000","last_fullsized":"2024-09-17T02:41:57.882226+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:32.074477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098120+0000","last_change":"2024-09-17T02:41:55.833701+0000","last_active":"2024-09-17T02:41:58.098120+0000","last_peered":"2024-09-17T02:41:58.098120+0000","last_clean":"2024-09-17T02:41:58.098120+0000","last_became_active":"2024-09-17T02:41:55.833254+0000","last_became_peered":"2024-09-17T02:41:55.833254+0000","last_unstale":"2024-09-17T02:41:58.098120+0000","last_undegraded":"2024-09-17T02:41:58.098120+0000","last_fullsized":"2024-09-17T02:41:58.098120+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:45.082415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.852824+0000","last_change":"2024-09-17T02:41:56.861634+0000","last_active":"2024-09-17T02:41:57.852824+0000","last_peered":"2024-09-17T02:41:57.852824+0000","last_clean":"2024-09-17T02:41:57.852824+0000","last_became_active":"2024-09-17T02:41:56.860806+0000","last_became_peered":"2024-09-17T02:41:56.860806+0000","last_unstale":"2024-09-17T02:41:57.852824+0000","last_undegraded":"2024-09-17T02:41:57.852824+0000","last_fullsized":"2024-09-17T02:41:57.852824+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:04.162086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.857048+0000","last_change":"2024-09-17T02:41:55.863410+0000","last_active":"2024-09-17T02:41:57.857048+0000","last_peered":"2024-09-17T02:41:57.857048+0000","last_clean":"2024-09-17T02:41:57.857048+0000","last_became_active":"2024-09-17T02:41:55.862647+0000","last_became_peered":"2024-09-17T02:41:55.862647+0000","last_unstale":"2024-09-17T02:41:57.857048+0000","last_undegraded":"2024-09-17T02:41:57.857048+0000","last_fullsized":"2024-09-17T02:41:57.857048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:25.110212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.099905+0000","last_change":"2024-09-17T02:41:56.868470+0000","last_active":"2024-09-17T02:41:58.099905+0000","last_peered":"2024-09-17T02:41:58.099905+0000","last_clean":"2024-09-17T02:41:58.099905+0000","last_became_active":"2024-09-17T02:41:56.867299+0000","last_became_peered":"2024-09-17T02:41:56.867299+0000","last_unstale":"2024-09-17T02:41:58.099905+0000","last_undegraded":"2024-09-17T02:41:58.099905+0000","last_fullsized":"2024-09-17T02:41:58.099905+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:15:36.404019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"39'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.851209+0000","last_change":"2024-09-17T02:41:55.849654+0000","last_active":"2024-09-17T02:41:57.851209+0000","last_peered":"2024-09-17T02:41:57.851209+0000","last_clean":"2024-09-17T02:41:57.851209+0000","last_became_active":"2024-09-17T02:41:55.849235+0000","last_became_peered":"2024-09-17T02:41:55.849235+0000","last_unstale":"2024-09-17T02:41:57.851209+0000","last_undegraded":"2024-09-17T02:41:57.851209+0000","last_fullsized":"2024-09-17T02:41:57.851209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:47.537792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.883091+0000","last_change":"2024-09-17T02:41:56.856318+0000","last_active":"2024-09-17T02:41:57.883091+0000","last_peered":"2024-09-17T02:41:57.883091+0000","last_clean":"2024-09-17T02:41:57.883091+0000","last_became_active":"2024-09-17T02:41:56.855924+0000","last_became_peered":"2024-09-17T02:41:56.855924+0000","last_unstale":"2024-09-17T02:41:57.883091+0000","last_undegraded":"2024-09-17T02:41:57.883091+0000","last_fullsized":"2024-09-17T02:41:57.883091+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:22:10.298396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.099893+0000","last_change":"2024-09-17T02:41:55.843817+0000","last_active":"2024-09-17T02:41:58.099893+0000","last_peered":"2024-09-17T02:41:58.099893+0000","last_clean":"2024-09-17T02:41:58.099893+0000","last_became_active":"2024-09-17T02:41:55.841737+0000","last_became_peered":"2024-09-17T02:41:55.841737+0000","last_unstale":"2024-09-17T02:41:58.099893+0000","last_undegraded":"2024-09-17T02:41:58.099893+0000","last_fullsized":"2024-09-17T02:41:58.099893+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:01.752600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.099511+0000","last_change":"2024-09-17T02:41:56.868887+0000","last_active":"2024-09-17T02:41:58.099511+0000","last_peered":"2024-09-17T02:41:58.099511+0000","last_clean":"2024-09-17T02:41:58.099511+0000","last_became_active":"2024-09-17T02:41:56.867925+0000","last_became_peered":"2024-09-17T02:41:56.867925+0000","last_unstale":"2024-09-17T02:41:58.099511+0000","last_undegraded":"2024-09-17T02:41:58.099511+0000","last_fullsized":"2024-09-17T02:41:58.099511+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:43.679037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098965+0000","last_change":"2024-09-17T02:41:55.868286+0000","last_active":"2024-09-17T02:41:58.098965+0000","last_peered":"2024-09-17T02:41:58.098965+0000","last_clean":"2024-09-17T02:41:58.098965+0000","last_became_active":"2024-09-17T02:41:55.867258+0000","last_became_peered":"2024-09-17T02:41:55.867258+0000","last_unstale":"2024-09-17T02:41:58.098965+0000","last_undegraded":"2024-09-17T02:41:58.098965+0000","last_fullsized":"2024-09-17T02:41:58.098965+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:11.620060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098295+0000","last_change":"2024-09-17T02:41:56.899281+0000","last_active":"2024-09-17T02:41:58.098295+0000","last_peered":"2024-09-17T02:41:58.098295+0000","last_clean":"2024-09-17T02:41:58.098295+0000","last_became_active":"2024-09-17T02:41:56.898683+0000","last_became_peered":"2024-09-17T02:41:56.898683+0000","last_unstale":"2024-09-17T02:41:58.098295+0000","last_undegraded":"2024-09-17T02:41:58.098295+0000","last_fullsized":"2024-09-17T02:41:58.098295+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:21.016912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.853343+0000","last_change":"2024-09-17T02:41:55.863138+0000","last_active":"2024-09-17T02:41:57.853343+0000","last_peered":"2024-09-17T02:41:57.853343+0000","last_clean":"2024-09-17T02:41:57.853343+0000","last_became_active":"2024-09-17T02:41:55.862501+0000","last_became_peered":"2024-09-17T02:41:55.862501+0000","last_unstale":"2024-09-17T02:41:57.853343+0000","last_undegraded":"2024-09-17T02:41:57.853343+0000","last_fullsized":"2024-09-17T02:41:57.853343+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:46:05.855634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.858000+0000","last_change":"2024-09-17T02:41:56.876227+0000","last_active":"2024-09-17T02:41:57.858000+0000","last_peered":"2024-09-17T02:41:57.858000+0000","last_clean":"2024-09-17T02:41:57.858000+0000","last_became_active":"2024-09-17T02:41:56.875330+0000","last_became_peered":"2024-09-17T02:41:56.875330+0000","last_unstale":"2024-09-17T02:41:57.858000+0000","last_undegraded":"2024-09-17T02:41:57.858000+0000","last_fullsized":"2024-09-17T02:41:57.858000+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:53.391175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.882676+0000","last_change":"2024-09-17T02:41:55.850201+0000","last_active":"2024-09-17T02:41:57.882676+0000","last_peered":"2024-09-17T02:41:57.882676+0000","last_clean":"2024-09-17T02:41:57.882676+0000","last_became_active":"2024-09-17T02:41:55.849734+0000","last_became_peered":"2024-09-17T02:41:55.849734+0000","last_unstale":"2024-09-17T02:41:57.882676+0000","last_undegraded":"2024-09-17T02:41:57.882676+0000","last_fullsized":"2024-09-17T02:41:57.882676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:37.028521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.851215+0000","last_change":"2024-09-17T02:41:56.900360+0000","last_active":"2024-09-17T02:41:57.851215+0000","last_peered":"2024-09-17T02:41:57.851215+0000","last_clean":"2024-09-17T02:41:57.851215+0000","last_became_active":"2024-09-17T02:41:56.899991+0000","last_became_peered":"2024-09-17T02:41:56.899991+0000","last_unstale":"2024-09-17T02:41:57.851215+0000","last_undegraded":"2024-09-17T02:41:57.851215+0000","last_fullsized":"2024-09-17T02:41:57.851215+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:51:32.842976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.851053+0000","last_change":"2024-09-17T02:41:55.849649+0000","last_active":"2024-09-17T02:41:57.851053+0000","last_peered":"2024-09-17T02:41:57.851053+0000","last_clean":"2024-09-17T02:41:57.851053+0000","last_became_active":"2024-09-17T02:41:55.849237+0000","last_became_peered":"2024-09-17T02:41:55.849237+0000","last_unstale":"2024-09-17T02:41:57.851053+0000","last_undegraded":"2024-09-17T02:41:57.851053+0000","last_fullsized":"2024-09-17T02:41:57.851053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:57:34.286910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.882432+0000","last_change":"2024-09-17T02:41:56.856990+0000","last_active":"2024-09-17T02:41:57.882432+0000","last_peered":"2024-09-17T02:41:57.882432+0000","last_clean":"2024-09-17T02:41:57.882432+0000","last_became_active":"2024-09-17T02:41:56.856613+0000","last_became_peered":"2024-09-17T02:41:56.856613+0000","last_unstale":"2024-09-17T02:41:57.882432+0000","last_undegraded":"2024-09-17T02:41:57.882432+0000","last_fullsized":"2024-09-17T02:41:57.882432+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:13.656168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.855230+0000","last_change":"2024-09-17T02:41:55.862706+0000","last_active":"2024-09-17T02:41:57.855230+0000","last_peered":"2024-09-17T02:41:57.855230+0000","last_clean":"2024-09-17T02:41:57.855230+0000","last_became_active":"2024-09-17T02:41:55.862091+0000","last_became_peered":"2024-09-17T02:41:55.862091+0000","last_unstale":"2024-09-17T02:41:57.855230+0000","last_undegraded":"2024-09-17T02:41:57.855230+0000","last_fullsized":"2024-09-17T02:41:57.855230+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:03:25.053792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.850807+0000","last_change":"2024-09-17T02:41:56.881627+0000","last_active":"2024-09-17T02:41:57.850807+0000","last_peered":"2024-09-17T02:41:57.850807+0000","last_clean":"2024-09-17T02:41:57.850807+0000","last_became_active":"2024-09-17T02:41:56.881241+0000","last_became_peered":"2024-09-17T02:41:56.881241+0000","last_unstale":"2024-09-17T02:41:57.850807+0000","last_undegraded":"2024-09-17T02:41:57.850807+0000","last_fullsized":"2024-09-17T02:41:57.850807+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:55.802457+0000","last_clean_scrub_stamp":"2024-09-17T02:41:55.802457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:05.091936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:57.882611+0000","last_change":"2024-09-17T02:41:55.850095+0000","last_active":"2024-09-17T02:41:57.882611+0000","last_peered":"2024-09-17T02:41:57.882611+0000","last_clean":"2024-09-17T02:41:57.882611+0000","last_became_active":"2024-09-17T02:41:55.849684+0000","last_became_peered":"2024-09-17T02:41:55.849684+0000","last_unstale":"2024-09-17T02:41:57.882611+0000","last_undegraded":"2024-09-17T02:41:57.882611+0000","last_fullsized":"2024-09-17T02:41:57.882611+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:41:54.799092+0000","last_clean_scrub_stamp":"2024-09-17T02:41:54.799092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:28:47.106157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":145,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-17T02:41:58.098180+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:41:58.098180+0000","last_peered":"2024-09-17T02:41:58.098180+0000","last_clean":"2024-09-17T02:41:58.098180+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:41:58.098180+0000","last_undegraded":"2024-09-17T02:41:58.098180+0000","last_fullsized":"2024-09-17T02:41:58.098180+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":195234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509460,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1052672,"data_stored":786114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":475136,"data_stored":196900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509459,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":195254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27488,"kb_used_data":460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344352,"statfs":{"total":96636764160,"available":96608616448,"internally_reserved":0,"allocated":471040,"data_stored":197358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1052672,"data_stored":785554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":195256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":458752,"data_stored":195198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:42:06.083 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T02:42:06.084 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T02:42:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.284+0000 7f994b6b3640 1 -- 172.21.15.131:0/3217478469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9930005360 msgr2=0x7f993000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.284+0000 7f994b6b3640 1 --2- 172.21.15.131:0/3217478469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9930005360 0x7f993000b9d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f993c0016f0 tx=0x7f993c01c5d0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.284+0000 7f994b6b3640 1 -- 172.21.15.131:0/3217478469 shutdown_connections 2024-09-17T02:42:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.284+0000 7f994b6b3640 1 --2- 172.21.15.131:0/3217478469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9930005360 0x7f993000b9d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.284+0000 7f994b6b3640 1 -- 172.21.15.131:0/3217478469 >> 172.21.15.131:0/3217478469 conn(0x7f99440b2ce0 msgr2=0x7f99440b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.284+0000 7f994b6b3640 1 -- 172.21.15.131:0/3217478469 shutdown_connections 2024-09-17T02:42:06.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.284+0000 7f994b6b3640 1 -- 172.21.15.131:0/3217478469 wait complete. 2024-09-17T02:42:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.284+0000 7f994b6b3640 1 Processor -- start 2024-09-17T02:42:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.284+0000 7f994b6b3640 1 -- start start 2024-09-17T02:42:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f994b6b3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9930005360 0x7f99440cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:06.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f994b6b3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f994413f5b0 0x7f99441e9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f994b6b3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99441e95b0 0x7f99441f9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f994b6b3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f99440cc7c0 con 0x7f99441e95b0 2024-09-17T02:42:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f994b6b3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f99440c6e50 con 0x7f994413f5b0 2024-09-17T02:42:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f994b6b3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f99440cc940 con 0x7f9930005360 2024-09-17T02:42:06.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9949c50640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99441e95b0 0x7f99441f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9949c50640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99441e95b0 0x7f99441f9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57122/0 (socket says 172.21.15.131:57122) 2024-09-17T02:42:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9949c50640 1 -- 172.21.15.131:0/1940303927 learned_addr learned my addr 172.21.15.131:0/1940303927 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f994944f640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9930005360 0x7f99440cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9948c4e640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f994413f5b0 0x7f99441e9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:06.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9949c50640 1 -- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9930005360 msgr2=0x7f99440cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9949c50640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9930005360 0x7f99440cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9949c50640 1 -- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f994413f5b0 msgr2=0x7f99441e9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9949c50640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f994413f5b0 0x7f99441e9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9949c50640 1 -- 172.21.15.131:0/1940303927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99441e99c0 con 0x7f99441e95b0 2024-09-17T02:42:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f994944f640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9930005360 0x7f99440cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:06.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9949c50640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99441e95b0 0x7f99441f9240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f993c01acd0 tx=0x7f993c0010f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9939ffb640 1 -- 172.21.15.131:0/1940303927 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0b8070 con 0x7f99441e95b0 2024-09-17T02:42:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9939ffb640 1 -- 172.21.15.131:0/1940303927 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f993c0ad040 con 0x7f99441e95b0 2024-09-17T02:42:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f994b6b3640 1 -- 172.21.15.131:0/1940303927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994413ff10 con 0x7f99441e95b0 2024-09-17T02:42:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9939ffb640 1 -- 172.21.15.131:0/1940303927 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c0ac070 con 0x7f99441e95b0 2024-09-17T02:42:06.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f994b6b3640 1 -- 172.21.15.131:0/1940303927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99441434c0 con 0x7f99441e95b0 2024-09-17T02:42:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f994b6b3640 1 -- 172.21.15.131:0/1940303927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99440cb810 con 0x7f99441e95b0 2024-09-17T02:42:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.288+0000 7f9939ffb640 1 -- 172.21.15.131:0/1940303927 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f993c0bb070 con 0x7f99441e95b0 2024-09-17T02:42:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.292+0000 7f9939ffb640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9928073230 0x7f9928075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.292+0000 7f9939ffb640 1 -- 172.21.15.131:0/1940303927 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f993c12d2b0 con 0x7f99441e95b0 2024-09-17T02:42:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.292+0000 7f994944f640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9928073230 0x7f9928075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:06.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.292+0000 7f994944f640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9928073230 0x7f9928075740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9940004670 tx=0x7f994000c040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:06.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.296+0000 7f9939ffb640 1 -- 172.21.15.131:0/1940303927 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f99441e99c0 con 0x7f99441e95b0 2024-09-17T02:42:06.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.508+0000 7f994b6b3640 1 -- 172.21.15.131:0/1940303927 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f99440c67a0 con 0x7f99441e95b0 2024-09-17T02:42:06.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.512+0000 7f9939ffb640 1 -- 172.21.15.131:0/1940303927 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v29) ==== 190+0+147 (secure 0 0 0) 0x7f99440ccd00 con 0x7f99441e95b0 2024-09-17T02:42:06.517 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-09-17T02:42:06.517 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA== 2024-09-17T02:42:06.517 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-09-17T02:42:06.517 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-09-17T02:42:06.517 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T02:42:06.518 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-09-17T02:42:06.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 -- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9928073230 msgr2=0x7f9928075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9928073230 0x7f9928075740 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9940004670 tx=0x7f994000c040 comp rx=0 tx=0).stop 2024-09-17T02:42:06.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 -- 172.21.15.131:0/1940303927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99441e95b0 msgr2=0x7f99441f9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99441e95b0 0x7f99441f9240 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f993c01acd0 tx=0x7f993c0010f0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 -- 172.21.15.131:0/1940303927 shutdown_connections 2024-09-17T02:42:06.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9928073230 0x7f9928075740 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99441e95b0 0x7f99441f9240 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f994413f5b0 0x7f99441e9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 --2- 172.21.15.131:0/1940303927 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9930005360 0x7f99440cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 -- 172.21.15.131:0/1940303927 >> 172.21.15.131:0/1940303927 conn(0x7f99440b2ce0 msgr2=0x7f9944074410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:06.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 -- 172.21.15.131:0/1940303927 shutdown_connections 2024-09-17T02:42:06.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.516+0000 7f994b6b3640 1 -- 172.21.15.131:0/1940303927 wait complete. 2024-09-17T02:42:06.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.688+0000 7fdf4faf4640 1 -- 172.21.15.131:0/667203758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf34003a60 msgr2=0x7fdf34003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.688+0000 7fdf4faf4640 1 --2- 172.21.15.131:0/667203758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf34003a60 0x7fdf34003ed0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009b60 tx=0x7fdf3801d230 comp rx=0 tx=0).stop 2024-09-17T02:42:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.688+0000 7fdf4faf4640 1 -- 172.21.15.131:0/667203758 shutdown_connections 2024-09-17T02:42:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.688+0000 7fdf4faf4640 1 --2- 172.21.15.131:0/667203758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf34003a60 0x7fdf34003ed0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.688+0000 7fdf4faf4640 1 -- 172.21.15.131:0/667203758 >> 172.21.15.131:0/667203758 conn(0x7fdf4806eea0 msgr2=0x7fdf4806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.688+0000 7fdf4faf4640 1 -- 172.21.15.131:0/667203758 shutdown_connections 2024-09-17T02:42:06.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.688+0000 7fdf4faf4640 1 -- 172.21.15.131:0/667203758 wait complete. 2024-09-17T02:42:06.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4faf4640 1 Processor -- start 2024-09-17T02:42:06.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4faf4640 1 -- start start 2024-09-17T02:42:06.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4faf4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34003a60 0x7fdf4814f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:06.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4faf4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf4816b460 0x7fdf48159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:06.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4faf4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf4815a1b0 0x7fdf481fb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:06.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4faf4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4815e950 con 0x7fdf4816b460 2024-09-17T02:42:06.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4faf4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4815e7d0 con 0x7fdf34003a60 2024-09-17T02:42:06.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4faf4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdf4815ead0 con 0x7fdf4815a1b0 2024-09-17T02:42:06.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4d08f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf4816b460 0x7fdf48159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:06.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4d08f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf4816b460 0x7fdf48159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57134/0 (socket says 172.21.15.131:57134) 2024-09-17T02:42:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4d08f640 1 -- 172.21.15.131:0/2098679420 learned_addr learned my addr 172.21.15.131:0/2098679420 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4d890640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34003a60 0x7fdf4814f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4e091640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf4815a1b0 0x7fdf481fb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4d08f640 1 -- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf4815a1b0 msgr2=0x7fdf481fb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4d08f640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf4815a1b0 0x7fdf481fb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4d08f640 1 -- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34003a60 msgr2=0x7fdf4814f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4d08f640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34003a60 0x7fdf4814f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4d08f640 1 -- 172.21.15.131:0/2098679420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4815a5c0 con 0x7fdf4816b460 2024-09-17T02:42:06.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4d08f640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf4816b460 0x7fdf48159c70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400c730 tx=0x7fdf4400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:06.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf3e7fc640 1 -- 172.21.15.131:0/2098679420 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44017070 con 0x7fdf4816b460 2024-09-17T02:42:06.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf3e7fc640 1 -- 172.21.15.131:0/2098679420 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf440053a0 con 0x7fdf4816b460 2024-09-17T02:42:06.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4faf4640 1 -- 172.21.15.131:0/2098679420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf481532c0 con 0x7fdf4816b460 2024-09-17T02:42:06.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf3e7fc640 1 -- 172.21.15.131:0/2098679420 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400a8a0 con 0x7fdf4816b460 2024-09-17T02:42:06.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.692+0000 7fdf4faf4640 1 -- 172.21.15.131:0/2098679420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf48153880 con 0x7fdf4816b460 2024-09-17T02:42:06.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.696+0000 7fdf4faf4640 1 -- 172.21.15.131:0/2098679420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4816bbb0 con 0x7fdf4816b460 2024-09-17T02:42:06.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.696+0000 7fdf3e7fc640 1 -- 172.21.15.131:0/2098679420 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdf4401a070 con 0x7fdf4816b460 2024-09-17T02:42:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.696+0000 7fdf3e7fc640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf2c073200 0x7fdf2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.696+0000 7fdf3e7fc640 1 -- 172.21.15.131:0/2098679420 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fdf44093740 con 0x7fdf4816b460 2024-09-17T02:42:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.696+0000 7fdf4d890640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf2c073200 0x7fdf2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:06.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.696+0000 7fdf4d890640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf2c073200 0x7fdf2c075710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdf380ae000 tx=0x7fdf38019fd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:06.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.700+0000 7fdf3e7fc640 1 -- 172.21.15.131:0/2098679420 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdf4815a5c0 con 0x7fdf4816b460 2024-09-17T02:42:06.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.896+0000 7fdf4faf4640 1 -- 172.21.15.131:0/2098679420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf4815f030 con 0x7fdf4816b460 2024-09-17T02:42:06.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.896+0000 7fdf3e7fc640 1 -- 172.21.15.131:0/2098679420 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7fdf4405df80 con 0x7fdf4816b460 2024-09-17T02:42:06.899 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:06.899 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":18,"btime":"2024-09-17T02:41:59:873356+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3342719482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3342719482},{"type":"v1","addr":"172.21.15.131:6835","nonce":3342719482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/3070378350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3070378350},{"type":"v1","addr":"172.21.15.189:6836","nonce":3070378350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/575920659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":575920659},{"type":"v1","addr":"172.21.15.189:6837","nonce":575920659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:41:56.227947+0000","modified":"2024-09-17T02:41:59.279613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6833/1415478569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1415478569},{"type":"v1","addr":"172.21.15.131:6833","nonce":1415478569}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-09-17T02:42:06.900 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 18 2024-09-17T02:42:06.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 -- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf2c073200 msgr2=0x7fdf2c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf2c073200 0x7fdf2c075710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdf380ae000 tx=0x7fdf38019fd0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 -- 172.21.15.131:0/2098679420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf4816b460 msgr2=0x7fdf48159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:06.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf4816b460 0x7fdf48159c70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400c730 tx=0x7fdf4400cbf0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 -- 172.21.15.131:0/2098679420 shutdown_connections 2024-09-17T02:42:06.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf2c073200 0x7fdf2c075710 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf4815a1b0 0x7fdf481fb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf4816b460 0x7fdf48159c70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 --2- 172.21.15.131:0/2098679420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf34003a60 0x7fdf4814f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:06.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 -- 172.21.15.131:0/2098679420 >> 172.21.15.131:0/2098679420 conn(0x7fdf4806eea0 msgr2=0x7fdf480b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:06.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 -- 172.21.15.131:0/2098679420 shutdown_connections 2024-09-17T02:42:06.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:06.900+0000 7fdf4faf4640 1 -- 172.21.15.131:0/2098679420 wait complete. 2024-09-17T02:42:07.322 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:41:56.227947+0000', 'modified': '2024-09-17T02:41:59.279613+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5022}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5022': {'gid': 5022, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6833/1415478569', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 1415478569}, {'type': 'v1', 'addr': '172.21.15.131:6833', 'nonce': 1415478569}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5022, 'qdb_cluster': [5022]} 2024-09-17T02:42:07.322 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:42:07.322 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T02:42:07.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.484+0000 7f2e7be21640 1 -- 172.21.15.131:0/1233273513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e60003a60 msgr2=0x7f2e60003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.484+0000 7f2e7be21640 1 --2- 172.21.15.131:0/1233273513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e60003a60 0x7f2e60003ed0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2e70009b60 tx=0x7f2e7001d260 comp rx=0 tx=0).stop 2024-09-17T02:42:07.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.484+0000 7f2e7be21640 1 -- 172.21.15.131:0/1233273513 shutdown_connections 2024-09-17T02:42:07.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.484+0000 7f2e7be21640 1 --2- 172.21.15.131:0/1233273513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e60003a60 0x7f2e60003ed0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.484+0000 7f2e7be21640 1 -- 172.21.15.131:0/1233273513 >> 172.21.15.131:0/1233273513 conn(0x7f2e740cf0b0 msgr2=0x7f2e740cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:07.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.484+0000 7f2e7be21640 1 -- 172.21.15.131:0/1233273513 shutdown_connections 2024-09-17T02:42:07.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.484+0000 7f2e7be21640 1 -- 172.21.15.131:0/1233273513 wait complete. 2024-09-17T02:42:07.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7be21640 1 Processor -- start 2024-09-17T02:42:07.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7be21640 1 -- start start 2024-09-17T02:42:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7be21640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e60003a60 0x7f2e741ea610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7be21640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e7415ac20 0x7f2e7415a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7be21640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e740c5d60 0x7f2e740c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7be21640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74071400 con 0x7f2e60003a60 2024-09-17T02:42:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7be21640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74071280 con 0x7f2e7415ac20 2024-09-17T02:42:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7be21640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2e740b5d30 con 0x7f2e740c5d60 2024-09-17T02:42:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e79bbd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e60003a60 0x7f2e741ea610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e79bbd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e60003a60 0x7f2e741ea610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57146/0 (socket says 172.21.15.131:57146) 2024-09-17T02:42:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e79bbd640 1 -- 172.21.15.131:0/1885811767 learned_addr learned my addr 172.21.15.131:0/1885811767 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e793bc640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e7415ac20 0x7f2e7415a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7a3be640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e740c5d60 0x7f2e740c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e79bbd640 1 -- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e740c5d60 msgr2=0x7f2e740c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e79bbd640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e740c5d60 0x7f2e740c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e79bbd640 1 -- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e7415ac20 msgr2=0x7f2e7415a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e79bbd640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e7415ac20 0x7f2e7415a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e79bbd640 1 -- 172.21.15.131:0/1885811767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e741ece90 con 0x7f2e60003a60 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7a3be640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e740c5d60 0x7f2e740c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e79bbd640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e60003a60 0x7f2e741ea610 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2e7001d090 tx=0x7f2e700056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e6a7fc640 1 -- 172.21.15.131:0/1885811767 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e700ba020 con 0x7f2e60003a60 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e6a7fc640 1 -- 172.21.15.131:0/1885811767 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e700acaa0 con 0x7f2e60003a60 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7be21640 1 -- 172.21.15.131:0/1885811767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e741ecbb0 con 0x7f2e60003a60 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e7be21640 1 -- 172.21.15.131:0/1885811767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e741e9e10 con 0x7f2e60003a60 2024-09-17T02:42:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.488+0000 7f2e6a7fc640 1 -- 172.21.15.131:0/1885811767 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e700b1040 con 0x7f2e60003a60 2024-09-17T02:42:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e6a7fc640 1 -- 172.21.15.131:0/1885811767 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2e700bd020 con 0x7f2e60003a60 2024-09-17T02:42:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e6a7fc640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e58073200 0x7f2e58075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e6a7fc640 1 -- 172.21.15.131:0/1885811767 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2e7012f380 con 0x7f2e60003a60 2024-09-17T02:42:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e793bc640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e58073200 0x7f2e58075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e830ec000 1 Processor -- start 2024-09-17T02:42:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e830ec000 1 -- start start 2024-09-17T02:42:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e793bc640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e58073200 0x7f2e58075710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e7415b0b0 tx=0x7f2e64006b70 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:07.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dc969460 0x5632dceaa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dcd59950 0x5632dceaabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dceab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e830ec000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5632dcd7f410 con 0x5632dcd42ca0 2024-09-17T02:42:07.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e830ec000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5632dccbcb40 con 0x5632dc969460 2024-09-17T02:42:07.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e830ec000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5632dccabf70 con 0x5632dcd59950 2024-09-17T02:42:07.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e7a3be640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dceab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e7a3be640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dceab120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57162/0 (socket says 172.21.15.131:57162) 2024-09-17T02:42:07.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e7a3be640 1 -- 172.21.15.131:0/1997073530 learned_addr learned my addr 172.21.15.131:0/1997073530 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:07.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e79bbd640 1 --2- 172.21.15.131:0/1997073530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dc969460 0x5632dceaa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e793bc640 1 --2- 172.21.15.131:0/1997073530 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dcd59950 0x5632dceaabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e7a3be640 1 -- 172.21.15.131:0/1997073530 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dcd59950 msgr2=0x5632dceaabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e7a3be640 1 --2- 172.21.15.131:0/1997073530 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dcd59950 0x5632dceaabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e7a3be640 1 -- 172.21.15.131:0/1997073530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dc969460 msgr2=0x5632dceaa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e7a3be640 1 --2- 172.21.15.131:0/1997073530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dc969460 0x5632dceaa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e7a3be640 1 -- 172.21.15.131:0/1997073530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632dcc735b0 con 0x5632dcd42ca0 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.492+0000 7f2e793bc640 1 --2- 172.21.15.131:0/1997073530 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dcd59950 0x5632dceaabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e7a3be640 1 --2- 172.21.15.131:0/1997073530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dceab120 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2e740c5d30 tx=0x7f2e6c00e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8a16675564e113 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e577fe640 1 -- 172.21.15.131:0/1997073530 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c018070 con 0x5632dcd42ca0 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e577fe640 1 -- 172.21.15.131:0/1997073530 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e6c0020a0 con 0x5632dcd42ca0 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e577fe640 1 -- 172.21.15.131:0/1997073530 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00ee30 con 0x5632dcd42ca0 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 -- 172.21.15.131:0/1997073530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 msgr2=0x5632dceab120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 --2- 172.21.15.131:0/1997073530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dceab120 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2e740c5d30 tx=0x7f2e6c00e5a0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 -- 172.21.15.131:0/1997073530 shutdown_connections 2024-09-17T02:42:07.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 --2- 172.21.15.131:0/1997073530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dceab120 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 --2- 172.21.15.131:0/1997073530 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dcd59950 0x5632dceaabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 --2- 172.21.15.131:0/1997073530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dc969460 0x5632dceaa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 -- 172.21.15.131:0/1997073530 >> 172.21.15.131:0/1997073530 conn(0x5632dce2fd90 msgr2=0x5632dce5f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:07.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 -- 172.21.15.131:0/1997073530 shutdown_connections 2024-09-17T02:42:07.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 -- 172.21.15.131:0/1997073530 wait complete. 2024-09-17T02:42:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 -- start start 2024-09-17T02:42:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 Processor -- start 2024-09-17T02:42:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dc969460 0x5632dce6cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dcd42ca0 0x5632dce6d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd59950 0x5632dce6d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5632dcc465f0 con 0x5632dc969460 2024-09-17T02:42:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5632dccafd80 con 0x5632dcd59950 2024-09-17T02:42:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5632dcc52520 con 0x5632dcd42ca0 2024-09-17T02:42:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e830ec000 10 client.0 _ll_register_callbacks cb 0x5632dcc56830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:42:07.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e79bbd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dc969460 0x5632dce6cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e79bbd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dc969460 0x5632dce6cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57176/0 (socket says 172.21.15.131:57176) 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e79bbd640 1 -- 172.21.15.131:0/3375010460 learned_addr learned my addr 172.21.15.131:0/3375010460 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e7a3be640 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd59950 0x5632dce6d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e793bc640 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dcd42ca0 0x5632dce6d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e79bbd640 1 -- 172.21.15.131:0/3375010460 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dcd42ca0 msgr2=0x5632dce6d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e79bbd640 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dcd42ca0 0x5632dce6d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e79bbd640 1 -- 172.21.15.131:0/3375010460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd59950 msgr2=0x5632dce6d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e79bbd640 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd59950 0x5632dce6d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.496+0000 7f2e79bbd640 1 -- 172.21.15.131:0/3375010460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632dcc4a190 con 0x5632dc969460 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e79bbd640 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dc969460 0x5632dce6cad0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e7001d740 tx=0x7f2e700f9500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e52e3450e445e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:42:07.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3375010460 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e700fe030 con 0x5632dc969460 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3375010460 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2e700ac4c0 con 0x5632dc969460 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 -- 172.21.15.131:0/3375010460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5632dcc44110 con 0x5632dc969460 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3375010460 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2e700fa2b0 con 0x5632dc969460 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 10 client.5229 fetch_fsmap learned FSMap version 18 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 -- 172.21.15.131:0/3375010460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5632dcd30f10 con 0x5632dc969460 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3375010460 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5632dcc44110 con 0x5632dc969460 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3375010460 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 18) ==== 3502+0+0 (secure 0 0 0) 0x7f2e700e7080 con 0x5632dc969460 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 10 client.5229 fetch_fsmap finished waiting for FSMap version 18 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 10 client.5229 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 20 client.5229 populate_metadata read hostname 'smithi131' 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.131:6832/1415478569,v1:172.21.15.131:6833/1415478569] conn(0x5632dc94e7d0 0x5632dcebbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 4 client.5229 mds_command: new command op to 5022 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 -- 172.21.15.131:0/3375010460 --> [v2:172.21.15.131:6832/1415478569,v1:172.21.15.131:6833/1415478569] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632dccbea90 con 0x5632dc94e7d0 2024-09-17T02:42:07.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e793bc640 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.131:6832/1415478569,v1:172.21.15.131:6833/1415478569] conn(0x5632dc94e7d0 0x5632dcebbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e54ff9640 10 client.5229 ms_handle_connect on v2:172.21.15.131:6832/1415478569 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e793bc640 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.131:6832/1415478569,v1:172.21.15.131:6833/1415478569] conn(0x5632dc94e7d0 0x5632dcebbc60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714e2cb4c17b47ca server_cookie=e8661c82ee1f0763 in_seq=0 out_seq=0 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3375010460 <== mds.0 v2:172.21.15.131:6832/1415478569 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5632dccbea90 con 0x5632dc94e7d0 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e54ff9640 10 client.5229 handle_command_reply: tid=0 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 client.5229 shutdown 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 -- 172.21.15.131:0/3375010460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dc969460 msgr2=0x5632dce6cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dc969460 0x5632dce6cad0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e7001d740 tx=0x7f2e700f9500 comp rx=0 tx=0).stop 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 -- 172.21.15.131:0/3375010460 shutdown_connections 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.131:6832/1415478569,v1:172.21.15.131:6833/1415478569] conn(0x5632dc94e7d0 0x5632dcebbc60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd59950 0x5632dce6d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dcd42ca0 0x5632dce6d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3375010460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dc969460 0x5632dce6cad0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e54ff9640 0 client.5229 ms_handle_reset on v2:172.21.15.131:6832/1415478569 2024-09-17T02:42:07.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 -- 172.21.15.131:0/3375010460 >> 172.21.15.131:0/3375010460 conn(0x5632dce2fd90 msgr2=0x5632dce66440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:07.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 -- 172.21.15.131:0/3375010460 shutdown_connections 2024-09-17T02:42:07.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 1 -- 172.21.15.131:0/3375010460 wait complete. 2024-09-17T02:42:07.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.500+0000 7f2e830ec000 20 client.5229 trim_cache size 0 max 16384 2024-09-17T02:42:07.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 Processor -- start 2024-09-17T02:42:07.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 -- start start 2024-09-17T02:42:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dc969460 0x5632dcf1c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dcf1cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd59950 0x5632dcf1d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74204150 con 0x5632dcd42ca0 2024-09-17T02:42:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e742031d0 con 0x5632dcd59950 2024-09-17T02:42:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2e742025c0 con 0x5632dc969460 2024-09-17T02:42:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e793bc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dcf1cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e793bc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dcf1cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57182/0 (socket says 172.21.15.131:57182) 2024-09-17T02:42:07.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e793bc640 1 -- 172.21.15.131:0/3711385953 learned_addr learned my addr 172.21.15.131:0/3711385953 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e79bbd640 1 --2- 172.21.15.131:0/3711385953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dc969460 0x5632dcf1c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e7a3be640 1 --2- 172.21.15.131:0/3711385953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd59950 0x5632dcf1d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e793bc640 1 -- 172.21.15.131:0/3711385953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dc969460 msgr2=0x5632dcf1c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e793bc640 1 --2- 172.21.15.131:0/3711385953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dc969460 0x5632dcf1c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e793bc640 1 -- 172.21.15.131:0/3711385953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd59950 msgr2=0x5632dcf1d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e793bc640 1 --2- 172.21.15.131:0/3711385953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd59950 0x5632dcf1d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e793bc640 1 -- 172.21.15.131:0/3711385953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632dcd07db0 con 0x5632dcd42ca0 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e7a3be640 1 --2- 172.21.15.131:0/3711385953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd59950 0x5632dcf1d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e793bc640 1 --2- 172.21.15.131:0/3711385953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dcf1cad0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2e640125d0 tx=0x7f2e6400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd33964a56d5046c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e56ffd640 1 -- 172.21.15.131:0/3711385953 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6402a070 con 0x5632dcd42ca0 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e56ffd640 1 -- 172.21.15.131:0/3711385953 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e6401f960 con 0x5632dcd42ca0 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e56ffd640 1 -- 172.21.15.131:0/3711385953 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e64016070 con 0x5632dcd42ca0 2024-09-17T02:42:07.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 -- 172.21.15.131:0/3711385953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 msgr2=0x5632dcf1cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3711385953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dcf1cad0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2e640125d0 tx=0x7f2e6400c000 comp rx=0 tx=0).stop 2024-09-17T02:42:07.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 -- 172.21.15.131:0/3711385953 shutdown_connections 2024-09-17T02:42:07.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3711385953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd59950 0x5632dcf1d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3711385953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd42ca0 0x5632dcf1cad0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3711385953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dc969460 0x5632dcf1c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 -- 172.21.15.131:0/3711385953 >> 172.21.15.131:0/3711385953 conn(0x5632dce2fd90 msgr2=0x5632dce5ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:07.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 -- 172.21.15.131:0/3711385953 shutdown_connections 2024-09-17T02:42:07.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.520+0000 7f2e830ec000 1 -- 172.21.15.131:0/3711385953 wait complete. 2024-09-17T02:42:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 -- start start 2024-09-17T02:42:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 Processor -- start 2024-09-17T02:42:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dc969460 0x5632dcedbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd42ca0 0x5632dcedc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd59950 0x5632dcedc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5632dccafd80 con 0x5632dcd59950 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2e74202bb0 con 0x5632dcd42ca0 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5632dcc465f0 con 0x5632dc969460 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 10 client.0 _ll_register_callbacks cb 0x7f2e70005800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e7a3be640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd59950 0x5632dcedc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e7a3be640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd59950 0x5632dcedc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57194/0 (socket says 172.21.15.131:57194) 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e7a3be640 1 -- 172.21.15.131:0/3783528758 learned_addr learned my addr 172.21.15.131:0/3783528758 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e79bbd640 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dc969460 0x5632dcedbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e793bc640 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd42ca0 0x5632dcedc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e7a3be640 1 -- 172.21.15.131:0/3783528758 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dc969460 msgr2=0x5632dcedbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e7a3be640 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dc969460 0x5632dcedbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e7a3be640 1 -- 172.21.15.131:0/3783528758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd42ca0 msgr2=0x5632dcedc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:07.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e7a3be640 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd42ca0 0x5632dcedc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e7a3be640 1 -- 172.21.15.131:0/3783528758 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632dcc72ab0 con 0x5632dcd59950 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e793bc640 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd42ca0 0x5632dcedc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e7a3be640 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd59950 0x5632dcedc610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0015d0 tx=0x7f2e6c003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275ed2ac6737d70c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3783528758 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c018040 con 0x5632dcd59950 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3783528758 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2e6c0052d0 con 0x5632dcd59950 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 -- 172.21.15.131:0/3783528758 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e6401f960 con 0x5632dcd59950 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3783528758 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (secure 0 0 0) 0x7f2e6c029050 con 0x5632dcd59950 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 10 client.5235 fetch_fsmap learned FSMap version 18 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 -- 172.21.15.131:0/3783528758 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) -- 0x5632dcc51070 con 0x5632dcd59950 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3783528758 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e6401f960 con 0x5632dcd59950 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3783528758 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 18) ==== 3502+0+0 (secure 0 0 0) 0x7f2e6c02c020 con 0x5632dcd59950 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 10 client.5235 fetch_fsmap finished waiting for FSMap version 18 2024-09-17T02:42:07.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 10 client.5235 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 20 client.5235 populate_metadata read hostname 'smithi131' 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.131:6832/1415478569,v1:172.21.15.131:6833/1415478569] conn(0x5632dc93c830 0x5632dcedcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 4 client.5235 mds_command: new command op to 5022 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e830ec000 1 -- 172.21.15.131:0/3783528758 --> [v2:172.21.15.131:6832/1415478569,v1:172.21.15.131:6833/1415478569] -- command(tid 0: {"prefix": "status"}) -- 0x5632dccba540 con 0x5632dc93c830 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.524+0000 7f2e79bbd640 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.131:6832/1415478569,v1:172.21.15.131:6833/1415478569] conn(0x5632dc93c830 0x5632dcedcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e54ff9640 10 client.5235 ms_handle_connect on v2:172.21.15.131:6832/1415478569 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e79bbd640 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.131:6832/1415478569,v1:172.21.15.131:6833/1415478569] conn(0x5632dc93c830 0x5632dcedcb50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9554398143f0cd2 server_cookie=dee776e560c343f9 in_seq=0 out_seq=0 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e54ff9640 1 -- 172.21.15.131:0/3783528758 <== mds.0 v2:172.21.15.131:6832/1415478569 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5632dccba540 con 0x5632dc93c830 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e54ff9640 10 client.5235 handle_command_reply: tid=0 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 1 client.5235 shutdown 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 1 -- 172.21.15.131:0/3783528758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd59950 msgr2=0x5632dcedc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd59950 0x5632dcedc610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c0015d0 tx=0x7f2e6c003e10 comp rx=0 tx=0).stop 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 1 -- 172.21.15.131:0/3783528758 shutdown_connections 2024-09-17T02:42:07.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.131:6832/1415478569,v1:172.21.15.131:6833/1415478569] conn(0x5632dc93c830 0x5632dcedcb50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5632dcd59950 0x5632dcedc610 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e54ff9640 0 client.5235 ms_handle_reset on v2:172.21.15.131:6832/1415478569 2024-09-17T02:42:07.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5632dcd42ca0 0x5632dcedc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 1 --2- 172.21.15.131:0/3783528758 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5632dc969460 0x5632dcedbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 1 -- 172.21.15.131:0/3783528758 >> 172.21.15.131:0/3783528758 conn(0x5632dce2fd90 msgr2=0x5632dce5f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:07.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 1 -- 172.21.15.131:0/3783528758 shutdown_connections 2024-09-17T02:42:07.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 1 -- 172.21.15.131:0/3783528758 wait complete. 2024-09-17T02:42:07.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e830ec000 20 client.5235 trim_cache size 0 max 16384 2024-09-17T02:42:07.535 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5022, 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 10.703101953999999, 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 18, 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 40, 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 40, 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 18.048259967 2024-09-17T02:42:07.536 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:42:07.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 -- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e58073200 msgr2=0x7f2e58075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e58073200 0x7f2e58075710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2e7415b0b0 tx=0x7f2e64006b70 comp rx=0 tx=0).stop 2024-09-17T02:42:07.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 -- 172.21.15.131:0/1885811767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e60003a60 msgr2=0x7f2e741ea610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e60003a60 0x7f2e741ea610 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2e7001d090 tx=0x7f2e700056c0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 -- 172.21.15.131:0/1885811767 shutdown_connections 2024-09-17T02:42:07.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2e58073200 0x7f2e58075710 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2e740c5d60 0x7f2e740c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2e7415ac20 0x7f2e7415a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 --2- 172.21.15.131:0/1885811767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2e60003a60 0x7f2e741ea610 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 -- 172.21.15.131:0/1885811767 >> 172.21.15.131:0/1885811767 conn(0x7f2e740cf0b0 msgr2=0x7f2e74160600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 -- 172.21.15.131:0/1885811767 shutdown_connections 2024-09-17T02:42:07.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.528+0000 7f2e7be21640 1 -- 172.21.15.131:0/1885811767 wait complete. 2024-09-17T02:42:07.548 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:42:07.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.688+0000 7f55f520b640 1 -- 172.21.15.131:0/1231812872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f00cefa0 msgr2=0x7f55f00afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.688+0000 7f55f520b640 1 --2- 172.21.15.131:0/1231812872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f00cefa0 0x7f55f00afa00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f55f00d01d0 tx=0x7f55e001cc60 comp rx=0 tx=0).stop 2024-09-17T02:42:07.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.688+0000 7f55f520b640 1 -- 172.21.15.131:0/1231812872 shutdown_connections 2024-09-17T02:42:07.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.688+0000 7f55f520b640 1 --2- 172.21.15.131:0/1231812872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f00cefa0 0x7f55f00afa00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.688+0000 7f55f520b640 1 --2- 172.21.15.131:0/1231812872 >> v2:172.21.15.189:3300/0 conn(0x7f55f015ebc0 0x7f55f00af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.688+0000 7f55f520b640 1 -- 172.21.15.131:0/1231812872 >> 172.21.15.131:0/1231812872 conn(0x7f55f00cd870 msgr2=0x7f55f00d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:07.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.688+0000 7f55f520b640 1 -- 172.21.15.131:0/1231812872 shutdown_connections 2024-09-17T02:42:07.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.688+0000 7f55f520b640 1 -- 172.21.15.131:0/1231812872 wait complete. 2024-09-17T02:42:07.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55f520b640 1 Processor -- start 2024-09-17T02:42:07.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55f520b640 1 -- start start 2024-09-17T02:42:07.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55f520b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f00ceb90 0x7f55f00b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55f520b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f55f00cefa0 0x7f55f00b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55f520b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f00b17b0 0x7f55f00b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55f520b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0167040 con 0x7f55f00b17b0 2024-09-17T02:42:07.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55f520b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0166ec0 con 0x7f55f00ceb90 2024-09-17T02:42:07.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55f520b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f55f01671c0 con 0x7f55f00cefa0 2024-09-17T02:42:07.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55ef59e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f00b17b0 0x7f55f00b4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55ef59e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f00b17b0 0x7f55f00b4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57206/0 (socket says 172.21.15.131:57206) 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55ef59e640 1 -- 172.21.15.131:0/972970542 learned_addr learned my addr 172.21.15.131:0/972970542 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55eed9d640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f00ceb90 0x7f55f00b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55ee59c640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f55f00cefa0 0x7f55f00b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55ef59e640 1 -- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f55f00cefa0 msgr2=0x7f55f00b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55ef59e640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f55f00cefa0 0x7f55f00b4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55ef59e640 1 -- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f00ceb90 msgr2=0x7f55f00b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55ef59e640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f00ceb90 0x7f55f00b1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55ef59e640 1 -- 172.21.15.131:0/972970542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f00aee90 con 0x7f55f00b17b0 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55eed9d640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f00ceb90 0x7f55f00b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55ef59e640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f00b17b0 0x7f55f00b4db0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f55e0019870 tx=0x7f55e0003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55e77fe640 1 -- 172.21.15.131:0/972970542 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e00bc070 con 0x7f55f00b17b0 2024-09-17T02:42:07.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55e77fe640 1 -- 172.21.15.131:0/972970542 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55e00042f0 con 0x7f55f00b17b0 2024-09-17T02:42:07.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55e77fe640 1 -- 172.21.15.131:0/972970542 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e00b2040 con 0x7f55f00b17b0 2024-09-17T02:42:07.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55f520b640 1 -- 172.21.15.131:0/972970542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f00af170 con 0x7f55f00b17b0 2024-09-17T02:42:07.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55f520b640 1 -- 172.21.15.131:0/972970542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f015a5f0 con 0x7f55f00b17b0 2024-09-17T02:42:07.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.692+0000 7f55f520b640 1 -- 172.21.15.131:0/972970542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f00af4a0 con 0x7f55f00b17b0 2024-09-17T02:42:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.696+0000 7f55e77fe640 1 -- 172.21.15.131:0/972970542 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f55e0004720 con 0x7f55f00b17b0 2024-09-17T02:42:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.696+0000 7f55e77fe640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f55d0073200 0x7f55d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.696+0000 7f55e77fe640 1 -- 172.21.15.131:0/972970542 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f55e012e400 con 0x7f55f00b17b0 2024-09-17T02:42:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.696+0000 7f55eed9d640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f55d0073200 0x7f55d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.696+0000 7f55eed9d640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f55d0073200 0x7f55d0075710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f55e8000f70 tx=0x7f55e8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:07.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.700+0000 7f55e77fe640 1 -- 172.21.15.131:0/972970542 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f55e0016970 con 0x7f55f00b17b0 2024-09-17T02:42:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.892+0000 7f55f520b640 1 -- 172.21.15.131:0/972970542 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f55f014d6a0 con 0x7f55f00b17b0 2024-09-17T02:42:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.892+0000 7f55e77fe640 1 -- 172.21.15.131:0/972970542 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) ==== 53+0+73 (secure 0 0 0) 0x7f55e00f9290 con 0x7f55f00b17b0 2024-09-17T02:42:07.895 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:42:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 -- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f55d0073200 msgr2=0x7f55d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f55d0073200 0x7f55d0075710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f55e8000f70 tx=0x7f55e8009040 comp rx=0 tx=0).stop 2024-09-17T02:42:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 -- 172.21.15.131:0/972970542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f00b17b0 msgr2=0x7f55f00b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f00b17b0 0x7f55f00b4db0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f55e0019870 tx=0x7f55e0003f80 comp rx=0 tx=0).stop 2024-09-17T02:42:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 -- 172.21.15.131:0/972970542 shutdown_connections 2024-09-17T02:42:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f55d0073200 0x7f55d0075710 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55f00b17b0 0x7f55f00b4db0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f55f00cefa0 0x7f55f00b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 --2- 172.21.15.131:0/972970542 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f55f00ceb90 0x7f55f00b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 -- 172.21.15.131:0/972970542 >> 172.21.15.131:0/972970542 conn(0x7f55f00cd870 msgr2=0x7f55f0151210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 -- 172.21.15.131:0/972970542 shutdown_connections 2024-09-17T02:42:07.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:07.896+0000 7f55f520b640 1 -- 172.21.15.131:0/972970542 wait complete. 2024-09-17T02:42:07.917 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:42:07.917 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:42:07.917 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:42:07.917 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:42:07.917 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:07.917 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:42:07.917 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:42:07.917 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:42:07.917 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:42:07.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:07.918 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:42:07.924 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:42:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:42:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:42:07.924 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:07.925 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42378sec preferred_lft 42378sec 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:42:07.926 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:07.927 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:42:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:42:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:07.927 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:42:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:42:07.927 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:07.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:07.927 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:42:07.978 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:42:07.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:07.980 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:42:08.029 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:42:08.029 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:42:08.029 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:42:08.029 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:42:08.030 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:42:08.030 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:42:08.030 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:42:08.030 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:42:08.030 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:42:08.030 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:42:08.030 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:42:08.030 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:42:08.030 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:42:08.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:42:08.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:42:08.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:42:08.031 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:42:08.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:42:08.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:42:08.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:08.149 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:42:08.201 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:42:08.201 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:42:08.202 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:42:08.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:08.202 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:42:08.258 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:42:08.258 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:42:08.258 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:08.310 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:42:08.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:42:08.310 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:08.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:42:08.411 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:42:08.460 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:42:08.460 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:42:08.460 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:42:08.460 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:42:08.460 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:42:08.460 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:42:08.460 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:42:08.460 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:42:08.461 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:42:08.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:42:08.462 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:42:08.462 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:42:08.462 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:42:08.462 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:42:08.462 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:42:08.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:42:08.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:08.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:08.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:08.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:08.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:08.463 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:42:08.512 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:42:08.513 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:42:08.558 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:42:08.574 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:42:08.622 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:42:08.622 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:42:08.622 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:42:08.622 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:42:08.622 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:42:08.622 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:42:08.622 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:42:08.623 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:08.624 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:08.625 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:08.625 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:08.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:08.625 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:42:08.669 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:42:08.668+0000 7fa431033580 -1 init, newargv = 0x7fa424004a30 newargc=15 2024-09-17T02:42:08.669 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[38915]: starting ceph client 2024-09-17T02:42:08.676 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[38915]: starting fuse 2024-09-17T02:42:09.770 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:42:09.786 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:42:09.834 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:42:09.834 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:42:09.834 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:42:09.835 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:42:09.835 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:42:09.835 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:42:09.835 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:42:09.835 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:42:09.835 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:42:09.835 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:42:09.835 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:42:09.835 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:42:09.835 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:42:09.835 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:42:09.836 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:42:09.837 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:42:09.837 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:09.837 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:09.837 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:09.837 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:09.837 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:09.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:09.838 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:42:09.887 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:42:09.888 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:42:09.888 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:42:09.888 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:42:09.888 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:42:09.888 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:42:09.888 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:42:09.888 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:09.888 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:42:09.889 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:42:09.890 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:42:09.890 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:42:09.890 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:42:09.890 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:09.890 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:42:09.890 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:42:10.010 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.38915.asok 2024-09-17T02:42:10.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:42:10.022 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38915.asok 2024-09-17T02:42:10.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:10.022 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38915.asok status 2024-09-17T02:42:10.201 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:42:10.201 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:42:10.201 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:42:10.201 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:42:10.201 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "38915", 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5250, 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 5250 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:10.202 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 29657016 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 29657016 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.5250 192.168.0.1:0/29657016", 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/29657016", 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 18, 2024-09-17T02:42:10.203 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 40, 2024-09-17T02:42:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:42:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:42:10.204 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:42:10.204 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:42:10.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:10.218 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:42:10.218 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:10.224 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:42:10.224 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:10.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:10.224 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:10.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:10.283 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-09-17T02:42:10.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:10.339 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-09-17T02:42:10.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:10.389 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-09-17T02:42:10.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:10.442 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-09-17T02:42:10.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:10.499 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-09-17T02:42:10.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:10.553 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-17T02:42:10.620 INFO:teuthology.orchestra.run.smithi131.stdout:./ 2024-09-17T02:42:10.621 INFO:teuthology.orchestra.run.smithi131.stdout:./rfile 2024-09-17T02:42:10.621 INFO:teuthology.orchestra.run.smithi131.stdout:./parentdir 2024-09-17T02:42:10.621 INFO:teuthology.orchestra.run.smithi131.stdout:./parentdir/pfile 2024-09-17T02:42:10.621 INFO:teuthology.orchestra.run.smithi131.stdout:./parentdir/childdir 2024-09-17T02:42:10.621 INFO:teuthology.orchestra.run.smithi131.stdout:./parentdir/childdir/cfile 2024-09-17T02:42:10.622 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:42:10.622 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:42:10.622 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:42:10.622 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:10.622 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-17T02:42:10.623 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:42:10.721 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:42:10.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:42:10.731 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:42:10.731 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:42:10.731 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:42:10.731 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:10.732 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-09-17T02:42:10.732 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:42:10.832 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627778 2024-09-17T02:42:10.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:42:10.841 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:42:10.841 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:42:10.841 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:42:10.842 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:10.842 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-09-17T02:42:10.842 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:42:10.947 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627776 2024-09-17T02:42:10.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:42:10.958 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:42:10.958 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:42:10.959 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:42:10.959 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:10.959 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-09-17T02:42:10.959 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:42:11.064 INFO:teuthology.orchestra.run.smithi131.stdout:1099511628279 2024-09-17T02:42:11.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:42:11.074 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:42:11.075 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:42:11.075 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:42:11.075 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:11.075 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-09-17T02:42:11.075 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:42:11.181 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627777 2024-09-17T02:42:11.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:42:11.191 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:42:11.191 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:42:11.191 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:42:11.191 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:11.191 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-09-17T02:42:11.191 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:42:11.296 INFO:teuthology.orchestra.run.smithi131.stdout:1099511628280 2024-09-17T02:42:11.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:42:11.307 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:42:11.307 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:42:11.358 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:42:11.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.564+0000 7ff76b241640 1 -- 172.21.15.131:0/1793389743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff74c004010 msgr2=0x7ff74c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:11.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.564+0000 7ff76b241640 1 --2- 172.21.15.131:0/1793389743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff74c004010 0x7ff74c004810 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff7580016f0 tx=0x7ff75801c7b0 comp rx=0 tx=0).stop 2024-09-17T02:42:11.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.564+0000 7ff7697de640 1 -- 172.21.15.131:0/1793389743 reap_dead start 2024-09-17T02:42:11.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.564+0000 7ff76b241640 1 -- 172.21.15.131:0/1793389743 shutdown_connections 2024-09-17T02:42:11.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.564+0000 7ff76b241640 1 -- 172.21.15.131:0/1793389743 >> 172.21.15.131:0/1793389743 conn(0x7ff76406eea0 msgr2=0x7ff76406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:11.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.564+0000 7ff76b241640 1 -- 172.21.15.131:0/1793389743 shutdown_connections 2024-09-17T02:42:11.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.564+0000 7ff76b241640 1 -- 172.21.15.131:0/1793389743 wait complete. 2024-09-17T02:42:11.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.564+0000 7ff76b241640 1 Processor -- start 2024-09-17T02:42:11.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.564+0000 7ff76b241640 1 -- start start 2024-09-17T02:42:11.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff76b241640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff74c004010 0x7ff76413d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:11.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff76b241640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76413adb0 0x7ff76415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:11.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff76b241640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff76415bbe0 0x7ff7640c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:11.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff76b241640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff764168c60 con 0x7ff76413adb0 2024-09-17T02:42:11.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff76b241640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff764168ae0 con 0x7ff76415bbe0 2024-09-17T02:42:11.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff76b241640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff764168de0 con 0x7ff74c004010 2024-09-17T02:42:11.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff763fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76413adb0 0x7ff76415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:11.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff763fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76413adb0 0x7ff76415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51982/0 (socket says 172.21.15.131:51982) 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff763fff640 1 -- 172.21.15.131:0/2773573019 learned_addr learned my addr 172.21.15.131:0/2773573019 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff7697de640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff76415bbe0 0x7ff7640c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff768fdd640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff74c004010 0x7ff76413d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff763fff640 1 -- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff74c004010 msgr2=0x7ff76413d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff763fff640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff74c004010 0x7ff76413d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff763fff640 1 -- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff76415bbe0 msgr2=0x7ff7640c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff763fff640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff76415bbe0 0x7ff7640c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff763fff640 1 -- 172.21.15.131:0/2773573019 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7640c8d70 con 0x7ff76413adb0 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff7697de640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff76415bbe0 0x7ff7640c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff763fff640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76413adb0 0x7ff76415b6a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff758000c10 tx=0x7ff758003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff7617fa640 1 -- 172.21.15.131:0/2773573019 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7580bd070 con 0x7ff76413adb0 2024-09-17T02:42:11.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff7617fa640 1 -- 172.21.15.131:0/2773573019 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff758006500 con 0x7ff76413adb0 2024-09-17T02:42:11.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff76b241640 1 -- 172.21.15.131:0/2773573019 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76413dd50 con 0x7ff76413adb0 2024-09-17T02:42:11.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff7617fa640 1 -- 172.21.15.131:0/2773573019 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff758001310 con 0x7ff76413adb0 2024-09-17T02:42:11.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff76b241640 1 -- 172.21.15.131:0/2773573019 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7640c9010 con 0x7ff76413adb0 2024-09-17T02:42:11.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff76b241640 1 -- 172.21.15.131:0/2773573019 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7640c4580 con 0x7ff76413adb0 2024-09-17T02:42:11.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.568+0000 7ff7617fa640 1 -- 172.21.15.131:0/2773573019 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff7580c0070 con 0x7ff76413adb0 2024-09-17T02:42:11.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.572+0000 7ff7617fa640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 0x7ff750075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:11.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.572+0000 7ff7617fa640 1 -- 172.21.15.131:0/2773573019 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff75812dfa0 con 0x7ff76413adb0 2024-09-17T02:42:11.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.572+0000 7ff768fdd640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 0x7ff750075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:11.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.572+0000 7ff768fdd640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 0x7ff750075710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff7640c7050 tx=0x7ff754001500 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:11.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.576+0000 7ff7617fa640 1 -- 172.21.15.131:0/2773573019 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff7580aa050 con 0x7ff76413adb0 2024-09-17T02:42:11.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.752+0000 7ff76b241640 1 -- 172.21.15.131:0/2773573019 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff764073310 con 0x7ff76413adb0 2024-09-17T02:42:11.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.752+0000 7ff7617fa640 1 -- 172.21.15.131:0/2773573019 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) ==== 80+0+0 (secure 0 0 0) 0x7ff758016510 con 0x7ff76413adb0 2024-09-17T02:42:11.756 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:42:11.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 -- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 msgr2=0x7ff750075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:11.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 0x7ff750075710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff7640c7050 tx=0x7ff754001500 comp rx=0 tx=0).stop 2024-09-17T02:42:11.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 -- 172.21.15.131:0/2773573019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76413adb0 msgr2=0x7ff76415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:11.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76413adb0 0x7ff76415b6a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff758000c10 tx=0x7ff758003b90 comp rx=0 tx=0).stop 2024-09-17T02:42:11.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 -- 172.21.15.131:0/2773573019 shutdown_connections 2024-09-17T02:42:11.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 0x7ff750075710 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:11.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff76415bbe0 0x7ff7640c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:11.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff76413adb0 0x7ff76415b6a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:11.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 --2- 172.21.15.131:0/2773573019 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff74c004010 0x7ff76413d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:11.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 -- 172.21.15.131:0/2773573019 >> 172.21.15.131:0/2773573019 conn(0x7ff76406eea0 msgr2=0x7ff764159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:11.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 -- 172.21.15.131:0/2773573019 shutdown_connections 2024-09-17T02:42:11.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.756+0000 7ff76b241640 1 -- 172.21.15.131:0/2773573019 wait complete. 2024-09-17T02:42:11.777 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:42:11.777 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:42:11.782 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:42:11.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.972+0000 7f2cc1f75640 1 -- 172.21.15.131:0/1198683762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ca8004810 msgr2=0x7f2ca8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:11.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.972+0000 7f2cc1f75640 1 --2- 172.21.15.131:0/1198683762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ca8004810 0x7f2ca8004c20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00c260 tx=0x7f2cac01cee0 comp rx=0 tx=0).stop 2024-09-17T02:42:11.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.972+0000 7f2cc1f75640 1 -- 172.21.15.131:0/1198683762 shutdown_connections 2024-09-17T02:42:11.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.972+0000 7f2cc1f75640 1 --2- 172.21.15.131:0/1198683762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ca8004810 0x7f2ca8004c20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:11.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.972+0000 7f2cc1f75640 1 -- 172.21.15.131:0/1198683762 >> 172.21.15.131:0/1198683762 conn(0x7f2cbc06fbe0 msgr2=0x7f2cbc06fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:11.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.972+0000 7f2cc1f75640 1 -- 172.21.15.131:0/1198683762 shutdown_connections 2024-09-17T02:42:11.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.972+0000 7f2cc1f75640 1 -- 172.21.15.131:0/1198683762 wait complete. 2024-09-17T02:42:11.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cc1f75640 1 Processor -- start 2024-09-17T02:42:11.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cc1f75640 1 -- start start 2024-09-17T02:42:11.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cc1f75640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ca8004810 0x7f2cbc154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cc1f75640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cbc1f88b0 0x7f2cbc151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cc1f75640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cbc155f00 0x7f2cbc156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cc1f75640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc153380 con 0x7f2cbc155f00 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cc1f75640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2cbc153200 con 0x7f2cbc1f88b0 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cc1f75640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2cbc153500 con 0x7f2ca8004810 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cbbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cbc155f00 0x7f2cbc156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cbbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cbc155f00 0x7f2cbc156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52006/0 (socket says 172.21.15.131:52006) 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cbbfff640 1 -- 172.21.15.131:0/2046175738 learned_addr learned my addr 172.21.15.131:0/2046175738 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cbaffd640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cbc1f88b0 0x7f2cbc151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cbb7fe640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ca8004810 0x7f2cbc154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cbbfff640 1 -- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ca8004810 msgr2=0x7f2cbc154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cbbfff640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ca8004810 0x7f2cbc154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:11.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cbbfff640 1 -- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cbc1f88b0 msgr2=0x7f2cbc151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:11.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cbbfff640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cbc1f88b0 0x7f2cbc151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:11.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cbbfff640 1 -- 172.21.15.131:0/2046175738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cbc1f8cc0 con 0x7f2cbc155f00 2024-09-17T02:42:11.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cbbfff640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cbc155f00 0x7f2cbc156310 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc160f50 tx=0x7f2cb4010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:11.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cb3fff640 1 -- 172.21.15.131:0/2046175738 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4017070 con 0x7f2cbc155f00 2024-09-17T02:42:11.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cb3fff640 1 -- 172.21.15.131:0/2046175738 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cb4015070 con 0x7f2cbc155f00 2024-09-17T02:42:11.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cc1f75640 1 -- 172.21.15.131:0/2046175738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cbc153bb0 con 0x7f2cbc155f00 2024-09-17T02:42:11.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cb3fff640 1 -- 172.21.15.131:0/2046175738 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4004d90 con 0x7f2cbc155f00 2024-09-17T02:42:11.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.976+0000 7f2cc1f75640 1 -- 172.21.15.131:0/2046175738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cbc0c5db0 con 0x7f2cbc155f00 2024-09-17T02:42:11.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.980+0000 7f2cc1f75640 1 -- 172.21.15.131:0/2046175738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cbc1540d0 con 0x7f2cbc155f00 2024-09-17T02:42:11.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.980+0000 7f2cb3fff640 1 -- 172.21.15.131:0/2046175738 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2cb4003980 con 0x7f2cbc155f00 2024-09-17T02:42:11.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.980+0000 7f2cb3fff640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2c9c073200 0x7f2c9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:11.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.980+0000 7f2cb3fff640 1 -- 172.21.15.131:0/2046175738 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f2cb4094910 con 0x7f2cbc155f00 2024-09-17T02:42:11.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.980+0000 7f2cbb7fe640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2c9c073200 0x7f2c9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:11.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.980+0000 7f2cbb7fe640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2c9c073200 0x7f2c9c075710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4006630 tx=0x7f2ca4006b30 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:11.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:11.984+0000 7f2cb3fff640 1 -- 172.21.15.131:0/2046175738 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2cbc1540d0 con 0x7f2cbc155f00 2024-09-17T02:42:12.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.160+0000 7f2cc1f75640 1 -- 172.21.15.131:0/2046175738 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2cbc15d1b0 con 0x7f2cbc155f00 2024-09-17T02:42:12.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.160+0000 7f2cb3fff640 1 -- 172.21.15.131:0/2046175738 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) ==== 80+0+0 (secure 0 0 0) 0x7f2cb405f7a0 con 0x7f2cbc155f00 2024-09-17T02:42:12.165 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:42:12.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 -- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2c9c073200 msgr2=0x7f2c9c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:12.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2c9c073200 0x7f2c9c075710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4006630 tx=0x7f2ca4006b30 comp rx=0 tx=0).stop 2024-09-17T02:42:12.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 -- 172.21.15.131:0/2046175738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cbc155f00 msgr2=0x7f2cbc156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:12.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cbc155f00 0x7f2cbc156310 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc160f50 tx=0x7f2cb4010e90 comp rx=0 tx=0).stop 2024-09-17T02:42:12.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 -- 172.21.15.131:0/2046175738 shutdown_connections 2024-09-17T02:42:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2c9c073200 0x7f2c9c075710 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2cbc155f00 0x7f2cbc156310 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2cbc1f88b0 0x7f2cbc151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 --2- 172.21.15.131:0/2046175738 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ca8004810 0x7f2cbc154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 -- 172.21.15.131:0/2046175738 >> 172.21.15.131:0/2046175738 conn(0x7f2cbc06fbe0 msgr2=0x7f2cbc0b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 -- 172.21.15.131:0/2046175738 shutdown_connections 2024-09-17T02:42:12.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:12.164+0000 7f2cc1f75640 1 -- 172.21.15.131:0/2046175738 wait complete. 2024-09-17T02:42:12.184 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:42:12.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:12.185 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:12.197 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[38915]: fuse finished with error 0 and tester_r 0 2024-09-17T02:42:12.198 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:42:13.380 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:42:13.383 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:42:18.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:42:18.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:18.301 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:42:18.309 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.512+0000 7f913434a640 1 -- 172.21.15.131:0/1883054278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91140053b0 msgr2=0x7f911400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.512+0000 7f913434a640 1 --2- 172.21.15.131:0/1883054278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91140053b0 0x7f911400ba60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f911c00c470 tx=0x7f911c01cf20 comp rx=0 tx=0).stop 2024-09-17T02:42:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.512+0000 7f913434a640 1 -- 172.21.15.131:0/1883054278 shutdown_connections 2024-09-17T02:42:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.512+0000 7f913434a640 1 --2- 172.21.15.131:0/1883054278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f91140053b0 0x7f911400ba60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.512+0000 7f913434a640 1 -- 172.21.15.131:0/1883054278 >> 172.21.15.131:0/1883054278 conn(0x7f912c0d13d0 msgr2=0x7f912c0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f913434a640 1 -- 172.21.15.131:0/1883054278 shutdown_connections 2024-09-17T02:42:18.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f913434a640 1 -- 172.21.15.131:0/1883054278 wait complete. 2024-09-17T02:42:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f913434a640 1 Processor -- start 2024-09-17T02:42:18.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f913434a640 1 -- start start 2024-09-17T02:42:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f913434a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91140053b0 0x7f912c14ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f913434a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912c0b1870 0x7f912c157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f913434a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f912c14be60 0x7f912c15da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f913434a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f912c06fea0 con 0x7f912c0b1870 2024-09-17T02:42:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f913434a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f912c06fd20 con 0x7f912c14be60 2024-09-17T02:42:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f913434a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f912c070020 con 0x7f91140053b0 2024-09-17T02:42:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f91318e5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912c0b1870 0x7f912c157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:18.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f91318e5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912c0b1870 0x7f912c157fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52022/0 (socket says 172.21.15.131:52022) 2024-09-17T02:42:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f91318e5640 1 -- 172.21.15.131:0/2468937078 learned_addr learned my addr 172.21.15.131:0/2468937078 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f91320e6640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91140053b0 0x7f912c14ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f91328e7640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f912c14be60 0x7f912c15da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f91318e5640 1 -- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91140053b0 msgr2=0x7f912c14ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f91318e5640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91140053b0 0x7f912c14ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f91318e5640 1 -- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f912c14be60 msgr2=0x7f912c15da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f91318e5640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f912c14be60 0x7f912c15da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f91318e5640 1 -- 172.21.15.131:0/2468937078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c0b1ff0 con 0x7f912c0b1870 2024-09-17T02:42:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.516+0000 7f91328e7640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f912c14be60 0x7f912c15da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:42:18.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f91318e5640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912c0b1870 0x7f912c157fb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9120004da0 tx=0x7f9120001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:18.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f912affd640 1 -- 172.21.15.131:0/2468937078 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120001d50 con 0x7f912c0b1870 2024-09-17T02:42:18.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f913434a640 1 -- 172.21.15.131:0/2468937078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912c1574d0 con 0x7f912c0b1870 2024-09-17T02:42:18.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f912affd640 1 -- 172.21.15.131:0/2468937078 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9120016070 con 0x7f912c0b1870 2024-09-17T02:42:18.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f912affd640 1 -- 172.21.15.131:0/2468937078 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9120010b40 con 0x7f912c0b1870 2024-09-17T02:42:18.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f913434a640 1 -- 172.21.15.131:0/2468937078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912c1571a0 con 0x7f912c0b1870 2024-09-17T02:42:18.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f913434a640 1 -- 172.21.15.131:0/2468937078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912c0af630 con 0x7f912c0b1870 2024-09-17T02:42:18.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f912affd640 1 -- 172.21.15.131:0/2468937078 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9120010ce0 con 0x7f912c0b1870 2024-09-17T02:42:18.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f912affd640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9118073200 0x7f9118075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:18.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f912affd640 1 -- 172.21.15.131:0/2468937078 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f9120094c10 con 0x7f912c0b1870 2024-09-17T02:42:18.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f91320e6640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9118073200 0x7f9118075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:18.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.520+0000 7f91320e6640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9118073200 0x7f9118075710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9124009e90 tx=0x7f9124009500 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:18.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.524+0000 7f912affd640 1 -- 172.21.15.131:0/2468937078 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f912005fab0 con 0x7f912c0b1870 2024-09-17T02:42:18.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.720+0000 7f913434a640 1 -- 172.21.15.131:0/2468937078 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f912c074500 con 0x7f912c0b1870 2024-09-17T02:42:18.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f912affd640 1 -- 172.21.15.131:0/2468937078 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7f912005f3b0 con 0x7f912c0b1870 2024-09-17T02:42:18.725 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:18.725 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":18,"btime":"2024-09-17T02:41:59:873356+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3342719482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3342719482},{"type":"v1","addr":"172.21.15.131:6835","nonce":3342719482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/3070378350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3070378350},{"type":"v1","addr":"172.21.15.189:6836","nonce":3070378350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/575920659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":575920659},{"type":"v1","addr":"172.21.15.189:6837","nonce":575920659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:41:56.227947+0000","modified":"2024-09-17T02:41:59.279613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6833/1415478569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1415478569},{"type":"v1","addr":"172.21.15.131:6833","nonce":1415478569}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-09-17T02:42:18.725 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 18 2024-09-17T02:42:18.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 -- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9118073200 msgr2=0x7f9118075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9118073200 0x7f9118075710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9124009e90 tx=0x7f9124009500 comp rx=0 tx=0).stop 2024-09-17T02:42:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 -- 172.21.15.131:0/2468937078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912c0b1870 msgr2=0x7f912c157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912c0b1870 0x7f912c157fb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9120004da0 tx=0x7f9120001640 comp rx=0 tx=0).stop 2024-09-17T02:42:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 -- 172.21.15.131:0/2468937078 shutdown_connections 2024-09-17T02:42:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9118073200 0x7f9118075710 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f912c14be60 0x7f912c15da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f912c0b1870 0x7f912c157fb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 --2- 172.21.15.131:0/2468937078 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91140053b0 0x7f912c14ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 -- 172.21.15.131:0/2468937078 >> 172.21.15.131:0/2468937078 conn(0x7f912c0d13d0 msgr2=0x7f912c0b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 -- 172.21.15.131:0/2468937078 shutdown_connections 2024-09-17T02:42:18.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:18.724+0000 7f913434a640 1 -- 172.21.15.131:0/2468937078 wait complete. 2024-09-17T02:42:18.745 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-17T02:42:18.915 INFO:teuthology.orchestra.run.smithi131.stdout:{"message":"","return_code":0} 2024-09-17T02:42:18.915 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T02:42:18.916 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:19.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.068+0000 7f2427469640 1 -- 172.21.15.131:0/1582797146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f240c004d50 msgr2=0x7f240c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:19.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.068+0000 7f2427469640 1 --2- 172.21.15.131:0/1582797146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f240c004d50 0x7f240c005200 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2420145bb0 tx=0x7f241801cae0 comp rx=0 tx=0).stop 2024-09-17T02:42:19.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.068+0000 7f2425a06640 1 -- 172.21.15.131:0/1582797146 reap_dead start 2024-09-17T02:42:19.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.068+0000 7f2427469640 1 -- 172.21.15.131:0/1582797146 shutdown_connections 2024-09-17T02:42:19.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.068+0000 7f2427469640 1 -- 172.21.15.131:0/1582797146 >> 172.21.15.131:0/1582797146 conn(0x7f24200d2130 msgr2=0x7f24200d2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:19.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.068+0000 7f2427469640 1 -- 172.21.15.131:0/1582797146 shutdown_connections 2024-09-17T02:42:19.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.068+0000 7f2427469640 1 -- 172.21.15.131:0/1582797146 wait complete. 2024-09-17T02:42:19.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2427469640 1 Processor -- start 2024-09-17T02:42:19.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2427469640 1 -- start start 2024-09-17T02:42:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2427469640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f24200c50a0 0x7f24201f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2427469640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24201f73f0 0x7f24200c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2427469640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24200c61a0 0x7f24201f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2427469640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2420143e70 con 0x7f24200c61a0 2024-09-17T02:42:19.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2427469640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24200d0cd0 con 0x7f24201f73f0 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2427469640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2420143ff0 con 0x7f24200c50a0 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2425a06640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24200c61a0 0x7f24201f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2425a06640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24200c61a0 0x7f24201f38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52034/0 (socket says 172.21.15.131:52034) 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2425a06640 1 -- 172.21.15.131:0/3543004120 learned_addr learned my addr 172.21.15.131:0/3543004120 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2425205640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f24200c50a0 0x7f24201f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2424a04640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24201f73f0 0x7f24200c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2425a06640 1 -- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f24200c50a0 msgr2=0x7f24201f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2425a06640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f24200c50a0 0x7f24201f6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2425a06640 1 -- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24201f73f0 msgr2=0x7f24200c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2425a06640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24201f73f0 0x7f24200c5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2425a06640 1 -- 172.21.15.131:0/3543004120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24201f7800 con 0x7f24200c61a0 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2424a04640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24201f73f0 0x7f24200c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:19.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2425a06640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24200c61a0 0x7f24201f38c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f241801d380 tx=0x7f2418001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:19.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2415ffb640 1 -- 172.21.15.131:0/3543004120 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418001970 con 0x7f24200c61a0 2024-09-17T02:42:19.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2415ffb640 1 -- 172.21.15.131:0/3543004120 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24180a5ce0 con 0x7f24200c61a0 2024-09-17T02:42:19.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2427469640 1 -- 172.21.15.131:0/3543004120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24200c68b0 con 0x7f24200c61a0 2024-09-17T02:42:19.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2415ffb640 1 -- 172.21.15.131:0/3543004120 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418006c50 con 0x7f24200c61a0 2024-09-17T02:42:19.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.072+0000 7f2427469640 1 -- 172.21.15.131:0/3543004120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24200c6e70 con 0x7f24200c61a0 2024-09-17T02:42:19.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.076+0000 7f2427469640 1 -- 172.21.15.131:0/3543004120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2420144580 con 0x7f24200c61a0 2024-09-17T02:42:19.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.076+0000 7f2415ffb640 1 -- 172.21.15.131:0/3543004120 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2418006310 con 0x7f24200c61a0 2024-09-17T02:42:19.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.076+0000 7f2415ffb640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2404073200 0x7f2404075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:19.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.076+0000 7f2415ffb640 1 -- 172.21.15.131:0/3543004120 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f241812e2c0 con 0x7f24200c61a0 2024-09-17T02:42:19.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.076+0000 7f2425205640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2404073200 0x7f2404075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:19.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.076+0000 7f2425205640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2404073200 0x7f2404075710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f24200cc050 tx=0x7f241c0091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:19.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.080+0000 7f2415ffb640 1 -- 172.21.15.131:0/3543004120 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f24201f7800 con 0x7f24200c61a0 2024-09-17T02:42:19.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.284+0000 7f2427469640 1 -- 172.21.15.131:0/3543004120 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24200717d0 con 0x7f24200c61a0 2024-09-17T02:42:19.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2415ffb640 1 -- 172.21.15.131:0/3543004120 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7f24180167f0 con 0x7f24200c61a0 2024-09-17T02:42:19.290 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:19.290 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":18,"btime":"2024-09-17T02:41:59:873356+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3342719482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3342719482},{"type":"v1","addr":"172.21.15.131:6835","nonce":3342719482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/3070378350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3070378350},{"type":"v1","addr":"172.21.15.189:6836","nonce":3070378350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/575920659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":575920659},{"type":"v1","addr":"172.21.15.189:6837","nonce":575920659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:41:56.227947+0000","modified":"2024-09-17T02:41:59.279613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6833/1415478569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1415478569},{"type":"v1","addr":"172.21.15.131:6833","nonce":1415478569}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-09-17T02:42:19.290 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 18 2024-09-17T02:42:19.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 -- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2404073200 msgr2=0x7f2404075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:19.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2404073200 0x7f2404075710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f24200cc050 tx=0x7f241c0091e0 comp rx=0 tx=0).stop 2024-09-17T02:42:19.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 -- 172.21.15.131:0/3543004120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24200c61a0 msgr2=0x7f24201f38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:19.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24200c61a0 0x7f24201f38c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f241801d380 tx=0x7f2418001710 comp rx=0 tx=0).stop 2024-09-17T02:42:19.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 -- 172.21.15.131:0/3543004120 shutdown_connections 2024-09-17T02:42:19.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2404073200 0x7f2404075710 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:19.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24200c61a0 0x7f24201f38c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:19.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24201f73f0 0x7f24200c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:19.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 --2- 172.21.15.131:0/3543004120 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f24200c50a0 0x7f24201f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:19.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 -- 172.21.15.131:0/3543004120 >> 172.21.15.131:0/3543004120 conn(0x7f24200d2130 msgr2=0x7f24200b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:19.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 -- 172.21.15.131:0/3543004120 shutdown_connections 2024-09-17T02:42:19.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:19.288+0000 7f2427469640 1 -- 172.21.15.131:0/3543004120 wait complete. 2024-09-17T02:42:19.311 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-09-17T02:42:19.484 INFO:teuthology.orchestra.run.smithi131.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-09-17T02:42:19.484 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-09-17T02:42:29.485 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-09-17T02:42:29.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 Processor -- start 2024-09-17T02:42:29.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 -- start start 2024-09-17T02:42:29.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcdee3c780 0x55dcdef0a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee78b90 0x55dcdef0a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55dcdee7ac20 0x55dcdef04cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dcdede1f40 con 0x55dcdee78b90 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dcdedeed00 con 0x55dcdee3c780 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55dcded83a00 con 0x55dcdee7ac20 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b1736640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee78b90 0x55dcdef0a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b1736640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee78b90 0x55dcdef0a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35800/0 (socket says 172.21.15.131:35800) 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b1736640 1 -- 172.21.15.131:0/995304179 learned_addr learned my addr 172.21.15.131:0/995304179 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b1f37640 1 --2- 172.21.15.131:0/995304179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcdee3c780 0x55dcdef0a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b1736640 1 -- 172.21.15.131:0/995304179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55dcdee7ac20 msgr2=0x55dcdef04cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b1736640 1 --2- 172.21.15.131:0/995304179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55dcdee7ac20 0x55dcdef04cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b1736640 1 -- 172.21.15.131:0/995304179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcdee3c780 msgr2=0x55dcdef0a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b1736640 1 --2- 172.21.15.131:0/995304179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcdee3c780 0x55dcdef0a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b1736640 1 -- 172.21.15.131:0/995304179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcdee81ac0 con 0x55dcdee78b90 2024-09-17T02:42:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b1736640 1 --2- 172.21.15.131:0/995304179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee78b90 0x55dcdef0a7f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f67a800a440 tx=0x7f67a800a900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad4da7f8d569b7ef server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67a3fff640 1 -- 172.21.15.131:0/995304179 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a800b930 con 0x55dcdee78b90 2024-09-17T02:42:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67a3fff640 1 -- 172.21.15.131:0/995304179 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67a800f070 con 0x55dcdee78b90 2024-09-17T02:42:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67a3fff640 1 -- 172.21.15.131:0/995304179 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8014470 con 0x55dcdee78b90 2024-09-17T02:42:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 -- 172.21.15.131:0/995304179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee78b90 msgr2=0x55dcdef0a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 --2- 172.21.15.131:0/995304179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee78b90 0x55dcdef0a7f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f67a800a440 tx=0x7f67a800a900 comp rx=0 tx=0).stop 2024-09-17T02:42:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 -- 172.21.15.131:0/995304179 shutdown_connections 2024-09-17T02:42:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 --2- 172.21.15.131:0/995304179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55dcdee7ac20 0x55dcdef04cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 --2- 172.21.15.131:0/995304179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee78b90 0x55dcdef0a7f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 --2- 172.21.15.131:0/995304179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcdee3c780 0x55dcdef0a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 -- 172.21.15.131:0/995304179 >> 172.21.15.131:0/995304179 conn(0x55dcdee21d00 msgr2=0x55dcdee338a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 -- 172.21.15.131:0/995304179 shutdown_connections 2024-09-17T02:42:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.560+0000 7f67b8460080 1 -- 172.21.15.131:0/995304179 wait complete. 2024-09-17T02:42:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b8460080 1 Processor -- start 2024-09-17T02:42:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b8460080 1 -- start start 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b8460080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcdee3c780 0x55dcdef05cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b8460080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55dcdee78b90 0x55dcdef01860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b8460080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee7ac20 0x55dcdefa88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b8460080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55dcdef02de0 con 0x55dcdee7ac20 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b8460080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55dcdee81d60 con 0x55dcdee3c780 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b8460080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55dcdef03080 con 0x55dcdee78b90 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b2738640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee7ac20 0x55dcdefa88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b2738640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee7ac20 0x55dcdefa88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35816/0 (socket says 172.21.15.131:35816) 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b2738640 1 -- 172.21.15.131:0/1119436393 learned_addr learned my addr 172.21.15.131:0/1119436393 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b1736640 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55dcdee78b90 0x55dcdef01860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b1f37640 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcdee3c780 0x55dcdef05cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b2738640 1 -- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55dcdee78b90 msgr2=0x55dcdef01860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b2738640 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55dcdee78b90 0x55dcdef01860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b2738640 1 -- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcdee3c780 msgr2=0x55dcdef05cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b2738640 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcdee3c780 0x55dcdef05cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b2738640 1 -- 172.21.15.131:0/1119436393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcdef062a0 con 0x55dcdee7ac20 2024-09-17T02:42:29.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b1f37640 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55dcdee3c780 0x55dcdef05cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:29.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b2738640 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee7ac20 0x55dcdefa88c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f67ac006f80 tx=0x7f67ac00b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67a2ffd640 1 -- 172.21.15.131:0/1119436393 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac016070 con 0x55dcdee7ac20 2024-09-17T02:42:29.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b8460080 1 -- 172.21.15.131:0/1119436393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55dcdef0a990 con 0x55dcdee7ac20 2024-09-17T02:42:29.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67a2ffd640 1 -- 172.21.15.131:0/1119436393 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67ac011620 con 0x55dcdee7ac20 2024-09-17T02:42:29.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67a2ffd640 1 -- 172.21.15.131:0/1119436393 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac011a70 con 0x55dcdee7ac20 2024-09-17T02:42:29.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.564+0000 7f67b8460080 1 -- 172.21.15.131:0/1119436393 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcdefa8fe0 con 0x55dcdee7ac20 2024-09-17T02:42:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67a2ffd640 1 -- 172.21.15.131:0/1119436393 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f67ac019070 con 0x55dcdee7ac20 2024-09-17T02:42:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67a2ffd640 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6794071c10 0x7f6794074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67a2ffd640 1 -- 172.21.15.131:0/1119436393 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f67ac092700 con 0x55dcdee7ac20 2024-09-17T02:42:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55dcdeeff990 0x55dcdeeffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b1f37640 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6794071c10 0x7f6794074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 -- 172.21.15.131:0/1119436393 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x55dcdef002e0 con 0x55dcdeeff990 2024-09-17T02:42:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b1736640 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55dcdeeff990 0x55dcdeeffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b1736640 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55dcdeeff990 0x55dcdeeffda0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b1f37640 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6794071c10 0x7f6794074120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67a40ab000 tx=0x7f67a40ad040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b1736640 1 -- 172.21.15.131:0/1119436393 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f67a800cbd0 con 0x55dcdeeff990 2024-09-17T02:42:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 -- 172.21.15.131:0/1119436393 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55dcdeeff990 msgr2=0x55dcdeeffda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55dcdeeff990 0x55dcdeeffda0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 -- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6794071c10 msgr2=0x7f6794074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6794071c10 0x7f6794074120 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f67a40ab000 tx=0x7f67a40ad040 comp rx=0 tx=0).stop 2024-09-17T02:42:29.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 -- 172.21.15.131:0/1119436393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee7ac20 msgr2=0x55dcdefa88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 --2- 172.21.15.131:0/1119436393 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55dcdee7ac20 0x55dcdefa88c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f67ac006f80 tx=0x7f67ac00b4b0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b2738640 1 -- 172.21.15.131:0/1119436393 reap_dead start 2024-09-17T02:42:29.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 -- 172.21.15.131:0/1119436393 shutdown_connections 2024-09-17T02:42:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 -- 172.21.15.131:0/1119436393 >> 172.21.15.131:0/1119436393 conn(0x55dcdee21d00 msgr2=0x55dcdef16780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 -- 172.21.15.131:0/1119436393 shutdown_connections 2024-09-17T02:42:29.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.568+0000 7f67b8460080 1 -- 172.21.15.131:0/1119436393 wait complete. 2024-09-17T02:42:29.578 INFO:teuthology.orchestra.run.smithi131.stdout:mytag 2024-09-17T02:42:29.579 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-09-17T02:42:29.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 Processor -- start 2024-09-17T02:42:29.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 -- start start 2024-09-17T02:42:29.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556db6bc9360 0x556db6d2a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556db6c10890 0x556db6bca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c9aed0 0x556db6c84b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556db6b15a00 con 0x556db6c9aed0 2024-09-17T02:42:29.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556db6b73f40 con 0x556db6bc9360 2024-09-17T02:42:29.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x556db6c9bdb0 con 0x556db6c10890 2024-09-17T02:42:29.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff39f02640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c9aed0 0x556db6c84b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff39f02640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c9aed0 0x556db6c84b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35838/0 (socket says 172.21.15.131:35838) 2024-09-17T02:42:29.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff39f02640 1 -- 172.21.15.131:0/1011454014 learned_addr learned my addr 172.21.15.131:0/1011454014 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:29.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff39f02640 1 -- 172.21.15.131:0/1011454014 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556db6c10890 msgr2=0x556db6bca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:29.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff39f02640 1 --2- 172.21.15.131:0/1011454014 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556db6c10890 0x556db6bca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff39f02640 1 -- 172.21.15.131:0/1011454014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556db6bc9360 msgr2=0x556db6d2a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:29.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff39f02640 1 --2- 172.21.15.131:0/1011454014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556db6bc9360 0x556db6d2a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff39f02640 1 -- 172.21.15.131:0/1011454014 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556db6aeec10 con 0x556db6c9aed0 2024-09-17T02:42:29.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff39f02640 1 --2- 172.21.15.131:0/1011454014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c9aed0 0x556db6c84b90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7eff2c004ea0 tx=0x7eff2c01a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8810103590985b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff2b7fe640 1 -- 172.21.15.131:0/1011454014 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c004590 con 0x556db6c9aed0 2024-09-17T02:42:29.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff2b7fe640 1 -- 172.21.15.131:0/1011454014 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff2c0a7ab0 con 0x556db6c9aed0 2024-09-17T02:42:29.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff2b7fe640 1 -- 172.21.15.131:0/1011454014 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2c0b0470 con 0x556db6c9aed0 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1011454014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c9aed0 msgr2=0x556db6c84b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 --2- 172.21.15.131:0/1011454014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c9aed0 0x556db6c84b90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7eff2c004ea0 tx=0x7eff2c01a7b0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1011454014 shutdown_connections 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 --2- 172.21.15.131:0/1011454014 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c9aed0 0x556db6c84b90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 --2- 172.21.15.131:0/1011454014 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556db6c10890 0x556db6bca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 --2- 172.21.15.131:0/1011454014 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556db6bc9360 0x556db6d2a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1011454014 >> 172.21.15.131:0/1011454014 conn(0x556db6bb3d00 msgr2=0x556db6c0e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1011454014 shutdown_connections 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.640+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1011454014 wait complete. 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff3fc2a080 1 Processor -- start 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff3fc2a080 1 -- start start 2024-09-17T02:42:29.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff3fc2a080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556db6bc9360 0x556db6d31ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff3fc2a080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c10890 0x556db6c7d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff3fc2a080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556db6c9aed0 0x556db6d32cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff3fc2a080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556db6c86600 con 0x556db6c10890 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff3fc2a080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556db6c86380 con 0x556db6c9aed0 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff3fc2a080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x556db6c868a0 con 0x556db6bc9360 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff38f00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c10890 0x556db6c7d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff38f00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c10890 0x556db6c7d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35854/0 (socket says 172.21.15.131:35854) 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff38f00640 1 -- 172.21.15.131:0/1240980191 learned_addr learned my addr 172.21.15.131:0/1240980191 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff39701640 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556db6bc9360 0x556db6d31ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff38f00640 1 -- 172.21.15.131:0/1240980191 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556db6bc9360 msgr2=0x556db6d31ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff38f00640 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556db6bc9360 0x556db6d31ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff39f02640 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556db6c9aed0 0x556db6d32cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff38f00640 1 -- 172.21.15.131:0/1240980191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556db6c9aed0 msgr2=0x556db6d32cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff38f00640 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556db6c9aed0 0x556db6d32cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff38f00640 1 -- 172.21.15.131:0/1240980191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556db6d32390 con 0x556db6c10890 2024-09-17T02:42:29.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff38f00640 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c10890 0x556db6c7d270 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7eff3000bd20 tx=0x7eff30009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff2a7fc640 1 -- 172.21.15.131:0/1240980191 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3000e630 con 0x556db6c10890 2024-09-17T02:42:29.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff2a7fc640 1 -- 172.21.15.131:0/1240980191 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff3000e8e0 con 0x556db6c10890 2024-09-17T02:42:29.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1240980191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556db6d32630 con 0x556db6c10890 2024-09-17T02:42:29.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff2a7fc640 1 -- 172.21.15.131:0/1240980191 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3000ed30 con 0x556db6c10890 2024-09-17T02:42:29.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.644+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1240980191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556db6d35310 con 0x556db6c10890 2024-09-17T02:42:29.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff2a7fc640 1 -- 172.21.15.131:0/1240980191 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7eff3001f3f0 con 0x556db6c10890 2024-09-17T02:42:29.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff2a7fc640 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7eff1c071be0 0x7eff1c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff2a7fc640 1 -- 172.21.15.131:0/1240980191 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7eff30093350 con 0x556db6c10890 2024-09-17T02:42:29.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x556db6d357b0 0x556db6d35bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1240980191 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x556db6c08ed0 con 0x556db6d357b0 2024-09-17T02:42:29.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff39701640 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7eff1c071be0 0x7eff1c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff39f02640 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x556db6d357b0 0x556db6d35bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff39f02640 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x556db6d357b0 0x556db6d35bc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff39f02640 1 -- 172.21.15.131:0/1240980191 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7eff2c0a7c10 con 0x556db6d357b0 2024-09-17T02:42:29.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff39701640 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7eff1c071be0 0x7eff1c0740f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff3400d8c0 tx=0x7eff340043c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1240980191 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x556db6d357b0 msgr2=0x556db6d35bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x556db6d357b0 0x556db6d35bc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1240980191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7eff1c071be0 msgr2=0x7eff1c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7eff1c071be0 0x7eff1c0740f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7eff3400d8c0 tx=0x7eff340043c0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1240980191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c10890 msgr2=0x556db6c7d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 --2- 172.21.15.131:0/1240980191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556db6c10890 0x556db6c7d270 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7eff3000bd20 tx=0x7eff30009600 comp rx=0 tx=0).stop 2024-09-17T02:42:29.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff39f02640 1 -- 172.21.15.131:0/1240980191 reap_dead start 2024-09-17T02:42:29.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1240980191 shutdown_connections 2024-09-17T02:42:29.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1240980191 >> 172.21.15.131:0/1240980191 conn(0x556db6bb3d00 msgr2=0x556db6bcbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:29.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1240980191 shutdown_connections 2024-09-17T02:42:29.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.648+0000 7eff3fc2a080 1 -- 172.21.15.131:0/1240980191 wait complete. 2024-09-17T02:42:29.658 INFO:teuthology.orchestra.run.smithi131.stdout:mytag 2024-09-17T02:42:29.658 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-09-17T02:42:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.716+0000 7fcf73ef9080 1 Processor -- start 2024-09-17T02:42:29.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.716+0000 7fcf73ef9080 1 -- start start 2024-09-17T02:42:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.716+0000 7fcf73ef9080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55cb884b29f0 0x55cb884b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.716+0000 7fcf73ef9080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb884b2e00 0x55cb884b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.716+0000 7fcf73ef9080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb885590d0 0x55cb884c7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.716+0000 7fcf73ef9080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cb88432f40 con 0x55cb885590d0 2024-09-17T02:42:29.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.716+0000 7fcf73ef9080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cb8843fd00 con 0x55cb884b2e00 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.716+0000 7fcf73ef9080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55cb883d4a00 con 0x55cb884b29f0 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.716+0000 7fcf6e1d1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb885590d0 0x55cb884c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.716+0000 7fcf6e1d1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb885590d0 0x55cb884c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35866/0 (socket says 172.21.15.131:35866) 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.716+0000 7fcf6e1d1640 1 -- 172.21.15.131:0/2978817787 learned_addr learned my addr 172.21.15.131:0/2978817787 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6d1cf640 1 --2- 172.21.15.131:0/2978817787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb884b2e00 0x55cb884b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6d9d0640 1 --2- 172.21.15.131:0/2978817787 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55cb884b29f0 0x55cb884b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6e1d1640 1 -- 172.21.15.131:0/2978817787 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55cb884b29f0 msgr2=0x55cb884b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6e1d1640 1 --2- 172.21.15.131:0/2978817787 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55cb884b29f0 0x55cb884b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6e1d1640 1 -- 172.21.15.131:0/2978817787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb884b2e00 msgr2=0x55cb884b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6e1d1640 1 --2- 172.21.15.131:0/2978817787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb884b2e00 0x55cb884b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6e1d1640 1 -- 172.21.15.131:0/2978817787 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb884b48c0 con 0x55cb885590d0 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6d1cf640 1 --2- 172.21.15.131:0/2978817787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb884b2e00 0x55cb884b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:29.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6d9d0640 1 --2- 172.21.15.131:0/2978817787 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55cb884b29f0 0x55cb884b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6e1d1640 1 --2- 172.21.15.131:0/2978817787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb885590d0 0x55cb884c7460 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000a380 tx=0x7fcf6000a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6195faeef4c440c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf67fff640 1 -- 172.21.15.131:0/2978817787 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf60010070 con 0x55cb885590d0 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf67fff640 1 -- 172.21.15.131:0/2978817787 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf6000c9f0 con 0x55cb885590d0 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf67fff640 1 -- 172.21.15.131:0/2978817787 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6000ce40 con 0x55cb885590d0 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 -- 172.21.15.131:0/2978817787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb885590d0 msgr2=0x55cb884c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 --2- 172.21.15.131:0/2978817787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb885590d0 0x55cb884c7460 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000a380 tx=0x7fcf6000a840 comp rx=0 tx=0).stop 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 -- 172.21.15.131:0/2978817787 shutdown_connections 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 --2- 172.21.15.131:0/2978817787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb885590d0 0x55cb884c7460 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 --2- 172.21.15.131:0/2978817787 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb884b2e00 0x55cb884b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 --2- 172.21.15.131:0/2978817787 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55cb884b29f0 0x55cb884b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 -- 172.21.15.131:0/2978817787 >> 172.21.15.131:0/2978817787 conn(0x55cb88472d00 msgr2=0x55cb8855b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 -- 172.21.15.131:0/2978817787 shutdown_connections 2024-09-17T02:42:29.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 -- 172.21.15.131:0/2978817787 wait complete. 2024-09-17T02:42:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 Processor -- start 2024-09-17T02:42:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 -- start start 2024-09-17T02:42:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb884b29f0 0x55cb885f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb884b2e00 0x55cb884c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55cb885590d0 0x55cb88556590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55cb884c5560 con 0x55cb884b29f0 2024-09-17T02:42:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55cb884c7e30 con 0x55cb884b2e00 2024-09-17T02:42:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf73ef9080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55cb88554780 con 0x55cb885590d0 2024-09-17T02:42:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6d9d0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb884b29f0 0x55cb885f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6d9d0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb884b29f0 0x55cb885f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35882/0 (socket says 172.21.15.131:35882) 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.720+0000 7fcf6d9d0640 1 -- 172.21.15.131:0/371082884 learned_addr learned my addr 172.21.15.131:0/371082884 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf6e1d1640 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55cb885590d0 0x55cb88556590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf6d1cf640 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb884b2e00 0x55cb884c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf6d9d0640 1 -- 172.21.15.131:0/371082884 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55cb885590d0 msgr2=0x55cb88556590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf6d9d0640 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55cb885590d0 0x55cb88556590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf6d9d0640 1 -- 172.21.15.131:0/371082884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb884b2e00 msgr2=0x55cb884c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf6d9d0640 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55cb884b2e00 0x55cb884c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf6d9d0640 1 -- 172.21.15.131:0/371082884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb88554a20 con 0x55cb884b29f0 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf6d9d0640 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb884b29f0 0x55cb885f6f10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c150 tx=0x7fcf6800c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf66ffd640 1 -- 172.21.15.131:0/371082884 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68015030 con 0x55cb884b29f0 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf66ffd640 1 -- 172.21.15.131:0/371082884 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf6800ce40 con 0x55cb884b29f0 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf73ef9080 1 -- 172.21.15.131:0/371082884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cb884c85b0 con 0x55cb884b29f0 2024-09-17T02:42:29.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf66ffd640 1 -- 172.21.15.131:0/371082884 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf68011860 con 0x55cb884b29f0 2024-09-17T02:42:29.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf73ef9080 1 -- 172.21.15.131:0/371082884 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb884c7af0 con 0x55cb884b29f0 2024-09-17T02:42:29.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf66ffd640 1 -- 172.21.15.131:0/371082884 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcf68018070 con 0x55cb884b29f0 2024-09-17T02:42:29.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf66ffd640 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcf50071be0 0x7fcf500740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf66ffd640 1 -- 172.21.15.131:0/371082884 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fcf680931e0 con 0x55cb884b29f0 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf73ef9080 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55cb885f50e0 0x55cb88555670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf6d1cf640 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcf50071be0 0x7fcf500740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf73ef9080 1 -- 172.21.15.131:0/371082884 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x55cb88555bb0 con 0x55cb885f50e0 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf6e1d1640 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55cb885f50e0 0x55cb88555670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.724+0000 7fcf6e1d1640 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55cb885f50e0 0x55cb88555670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf6d1cf640 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcf50071be0 0x7fcf500740f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcf580ab000 tx=0x7fcf58019a40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf6e1d1640 1 -- 172.21.15.131:0/371082884 <== osd.1 v2:172.21.15.131:6824/1616383728 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fcf60012030 con 0x55cb885f50e0 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf73ef9080 1 -- 172.21.15.131:0/371082884 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55cb885f50e0 msgr2=0x55cb88555670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf73ef9080 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55cb885f50e0 0x55cb88555670 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf73ef9080 1 -- 172.21.15.131:0/371082884 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcf50071be0 msgr2=0x7fcf500740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf73ef9080 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcf50071be0 0x7fcf500740f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcf580ab000 tx=0x7fcf58019a40 comp rx=0 tx=0).stop 2024-09-17T02:42:29.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf73ef9080 1 -- 172.21.15.131:0/371082884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb884b29f0 msgr2=0x55cb885f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf73ef9080 1 --2- 172.21.15.131:0/371082884 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55cb884b29f0 0x55cb885f6f10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800c150 tx=0x7fcf6800c610 comp rx=0 tx=0).stop 2024-09-17T02:42:29.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf6e1d1640 1 -- 172.21.15.131:0/371082884 reap_dead start 2024-09-17T02:42:29.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf73ef9080 1 -- 172.21.15.131:0/371082884 shutdown_connections 2024-09-17T02:42:29.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf73ef9080 1 -- 172.21.15.131:0/371082884 >> 172.21.15.131:0/371082884 conn(0x55cb88472d00 msgr2=0x55cb884b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:29.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf73ef9080 1 -- 172.21.15.131:0/371082884 shutdown_connections 2024-09-17T02:42:29.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.728+0000 7fcf73ef9080 1 -- 172.21.15.131:0/371082884 wait complete. 2024-09-17T02:42:29.735 INFO:teuthology.orchestra.run.smithi131.stdout:mytag 2024-09-17T02:42:29.736 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-09-17T02:42:29.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe4974e080 1 Processor -- start 2024-09-17T02:42:29.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe4974e080 1 -- start start 2024-09-17T02:42:29.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe4974e080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 0x55871bc99590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe4974e080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871bcaa300 0x55871bc99ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe4974e080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55871bdcfb20 0x55871bdcff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe4974e080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55871bbb7a00 con 0x7efe30002300 2024-09-17T02:42:29.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe4974e080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55871bc15f40 con 0x55871bcaa300 2024-09-17T02:42:29.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe4974e080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55871bd3d2e0 con 0x55871bdcfb20 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe43225640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 0x55871bc99590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe43225640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 0x55871bc99590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35896/0 (socket says 172.21.15.131:35896) 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe43225640 1 -- 172.21.15.131:0/214996834 learned_addr learned my addr 172.21.15.131:0/214996834 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe3bfff640 1 --2- 172.21.15.131:0/214996834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871bcaa300 0x55871bc99ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe43225640 1 -- 172.21.15.131:0/214996834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55871bdcfb20 msgr2=0x55871bdcff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe43a26640 1 --2- 172.21.15.131:0/214996834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55871bdcfb20 0x55871bdcff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe43225640 1 --2- 172.21.15.131:0/214996834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55871bdcfb20 0x55871bdcff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe43225640 1 -- 172.21.15.131:0/214996834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871bcaa300 msgr2=0x55871bc99ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe43225640 1 --2- 172.21.15.131:0/214996834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871bcaa300 0x55871bc99ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe43225640 1 -- 172.21.15.131:0/214996834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871bb90c10 con 0x7efe30002300 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe3bfff640 1 --2- 172.21.15.131:0/214996834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871bcaa300 0x55871bc99ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe43225640 1 --2- 172.21.15.131:0/214996834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 0x55871bc99590 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b660 tx=0x7efe30004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8eb52ee5f79d5e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe42223640 1 -- 172.21.15.131:0/214996834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe340a4920 con 0x7efe30002300 2024-09-17T02:42:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe42223640 1 -- 172.21.15.131:0/214996834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe340a4ac0 con 0x7efe30002300 2024-09-17T02:42:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.792+0000 7efe42223640 1 -- 172.21.15.131:0/214996834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe340b4420 con 0x7efe30002300 2024-09-17T02:42:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 -- 172.21.15.131:0/214996834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 msgr2=0x55871bc99590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 --2- 172.21.15.131:0/214996834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 0x55871bc99590 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7efe3400b660 tx=0x7efe30004130 comp rx=0 tx=0).stop 2024-09-17T02:42:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 -- 172.21.15.131:0/214996834 shutdown_connections 2024-09-17T02:42:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 --2- 172.21.15.131:0/214996834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55871bdcfb20 0x55871bdcff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 --2- 172.21.15.131:0/214996834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871bcaa300 0x55871bc99ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 --2- 172.21.15.131:0/214996834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 0x55871bc99590 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 -- 172.21.15.131:0/214996834 >> 172.21.15.131:0/214996834 conn(0x55871bc55d00 msgr2=0x55871bd26c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 -- 172.21.15.131:0/214996834 shutdown_connections 2024-09-17T02:42:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 -- 172.21.15.131:0/214996834 wait complete. 2024-09-17T02:42:29.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 Processor -- start 2024-09-17T02:42:29.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 -- start start 2024-09-17T02:42:29.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55871bcaa300 0x55871bdd4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871bdcfb20 0x55871bc957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 0x55871bd21f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55871bd24440 con 0x7efe30002300 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55871bd279e0 con 0x55871bdcfb20 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55871bd246e0 con 0x55871bcaa300 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe43a26640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 0x55871bd21f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe43a26640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 0x55871bd21f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35898/0 (socket says 172.21.15.131:35898) 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe43a26640 1 -- 172.21.15.131:0/1104389492 learned_addr learned my addr 172.21.15.131:0/1104389492 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe3bfff640 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871bdcfb20 0x55871bc957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe43225640 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55871bcaa300 0x55871bdd4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe43a26640 1 -- 172.21.15.131:0/1104389492 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55871bcaa300 msgr2=0x55871bdd4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe43a26640 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55871bcaa300 0x55871bdd4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe43a26640 1 -- 172.21.15.131:0/1104389492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871bdcfb20 msgr2=0x55871bc957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe43a26640 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55871bdcfb20 0x55871bc957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe43a26640 1 -- 172.21.15.131:0/1104389492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871bdd4870 con 0x7efe30002300 2024-09-17T02:42:29.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe43a26640 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 0x55871bd21f70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55871bc6e8b0 tx=0x7efe3c00c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe41221640 1 -- 172.21.15.131:0/1104389492 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c010640 con 0x7efe30002300 2024-09-17T02:42:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe41221640 1 -- 172.21.15.131:0/1104389492 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe3c0108d0 con 0x7efe30002300 2024-09-17T02:42:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe41221640 1 -- 172.21.15.131:0/1104389492 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c010d20 con 0x7efe30002300 2024-09-17T02:42:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 -- 172.21.15.131:0/1104389492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55871bdd4b10 con 0x7efe30002300 2024-09-17T02:42:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.796+0000 7efe4974e080 1 -- 172.21.15.131:0/1104389492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871bdd5050 con 0x7efe30002300 2024-09-17T02:42:29.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe41221640 1 -- 172.21.15.131:0/1104389492 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efe3c00b030 con 0x7efe30002300 2024-09-17T02:42:29.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe41221640 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efe24071be0 0x7efe240740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe41221640 1 -- 172.21.15.131:0/1104389492 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7efe3c093880 con 0x7efe30002300 2024-09-17T02:42:29.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe43225640 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efe24071be0 0x7efe240740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55871bd22b70 0x55871bd22f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 -- 172.21.15.131:0/1104389492 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x55871bdd5350 con 0x55871bd22b70 2024-09-17T02:42:29.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe3bfff640 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55871bd22b70 0x55871bd22f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe3bfff640 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55871bd22b70 0x55871bd22f80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe43225640 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efe24071be0 0x7efe240740f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55871bcaade0 tx=0x7efe34004890 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe3bfff640 1 -- 172.21.15.131:0/1104389492 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7efe2c011490 con 0x55871bd22b70 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 -- 172.21.15.131:0/1104389492 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55871bd22b70 msgr2=0x55871bd22f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55871bd22b70 0x55871bd22f80 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 -- 172.21.15.131:0/1104389492 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efe24071be0 msgr2=0x7efe240740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efe24071be0 0x7efe240740f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55871bcaade0 tx=0x7efe34004890 comp rx=0 tx=0).stop 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 -- 172.21.15.131:0/1104389492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 msgr2=0x55871bd21f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 --2- 172.21.15.131:0/1104389492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efe30002300 0x55871bd21f70 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55871bc6e8b0 tx=0x7efe3c00c6b0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe43a26640 1 -- 172.21.15.131:0/1104389492 reap_dead start 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 -- 172.21.15.131:0/1104389492 shutdown_connections 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 -- 172.21.15.131:0/1104389492 >> 172.21.15.131:0/1104389492 conn(0x55871bc55d00 msgr2=0x55871bcac570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 -- 172.21.15.131:0/1104389492 shutdown_connections 2024-09-17T02:42:29.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.800+0000 7efe4974e080 1 -- 172.21.15.131:0/1104389492 wait complete. 2024-09-17T02:42:29.810 INFO:teuthology.orchestra.run.smithi131.stdout:mytag 2024-09-17T02:42:29.810 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-09-17T02:42:29.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc0ac03080 1 Processor -- start 2024-09-17T02:42:29.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc0ac03080 1 -- start start 2024-09-17T02:42:29.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc0ac03080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5630d12c0730 0x5630d137bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc0ac03080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d1309a10 0x5630d12ee9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc0ac03080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630d12eef30 0x5630d12c2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc0ac03080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5630d120da00 con 0x5630d1309a10 2024-09-17T02:42:29.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc0ac03080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5630d126bf40 con 0x5630d12eef30 2024-09-17T02:42:29.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc0ac03080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5630d13774b0 con 0x5630d12c0730 2024-09-17T02:42:29.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc03ed9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d1309a10 0x5630d12ee9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc03ed9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d1309a10 0x5630d12ee9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35912/0 (socket says 172.21.15.131:35912) 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc03ed9640 1 -- 172.21.15.131:0/3588970679 learned_addr learned my addr 172.21.15.131:0/3588970679 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc046da640 1 --2- 172.21.15.131:0/3588970679 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5630d12c0730 0x5630d137bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc04edb640 1 --2- 172.21.15.131:0/3588970679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630d12eef30 0x5630d12c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc03ed9640 1 -- 172.21.15.131:0/3588970679 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5630d12c0730 msgr2=0x5630d137bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc03ed9640 1 --2- 172.21.15.131:0/3588970679 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5630d12c0730 0x5630d137bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc03ed9640 1 -- 172.21.15.131:0/3588970679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630d12eef30 msgr2=0x5630d12c2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc03ed9640 1 --2- 172.21.15.131:0/3588970679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630d12eef30 0x5630d12c2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc03ed9640 1 -- 172.21.15.131:0/3588970679 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d11e6c10 con 0x5630d1309a10 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc03ed9640 1 --2- 172.21.15.131:0/3588970679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d1309a10 0x5630d12ee9f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0016c0 tx=0x7fbbfc001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa723dfbbad6256 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc02ed7640 1 -- 172.21.15.131:0/3588970679 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc016070 con 0x5630d1309a10 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc02ed7640 1 -- 172.21.15.131:0/3588970679 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbfc00f800 con 0x5630d1309a10 2024-09-17T02:42:29.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc02ed7640 1 -- 172.21.15.131:0/3588970679 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc00fc30 con 0x5630d1309a10 2024-09-17T02:42:29.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc0ac03080 1 -- 172.21.15.131:0/3588970679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d1309a10 msgr2=0x5630d12ee9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.868+0000 7fbc0ac03080 1 --2- 172.21.15.131:0/3588970679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d1309a10 0x5630d12ee9f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc0016c0 tx=0x7fbbfc001b80 comp rx=0 tx=0).stop 2024-09-17T02:42:29.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 -- 172.21.15.131:0/3588970679 shutdown_connections 2024-09-17T02:42:29.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 --2- 172.21.15.131:0/3588970679 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630d12eef30 0x5630d12c2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 --2- 172.21.15.131:0/3588970679 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d1309a10 0x5630d12ee9f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 --2- 172.21.15.131:0/3588970679 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5630d12c0730 0x5630d137bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 -- 172.21.15.131:0/3588970679 >> 172.21.15.131:0/3588970679 conn(0x5630d12abd00 msgr2=0x5630d13071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 -- 172.21.15.131:0/3588970679 shutdown_connections 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 -- 172.21.15.131:0/3588970679 wait complete. 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 Processor -- start 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 -- start start 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5630d12c0730 0x5630d1392ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d12eef30 0x5630d1302440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630d1309a10 0x5630d142d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5630d12c16e0 con 0x5630d12eef30 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5630d1396640 con 0x5630d1309a10 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5630d12c1980 con 0x5630d12c0730 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc03ed9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d12eef30 0x5630d1302440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc03ed9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d12eef30 0x5630d1302440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35922/0 (socket says 172.21.15.131:35922) 2024-09-17T02:42:29.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc03ed9640 1 -- 172.21.15.131:0/2748416944 learned_addr learned my addr 172.21.15.131:0/2748416944 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc04edb640 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630d1309a10 0x5630d142d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc046da640 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5630d12c0730 0x5630d1392ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc03ed9640 1 -- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5630d12c0730 msgr2=0x5630d1392ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc03ed9640 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5630d12c0730 0x5630d1392ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc03ed9640 1 -- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630d1309a10 msgr2=0x5630d142d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc03ed9640 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630d1309a10 0x5630d142d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc03ed9640 1 -- 172.21.15.131:0/2748416944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630d137e320 con 0x5630d12eef30 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc04edb640 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5630d1309a10 0x5630d142d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc03ed9640 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d12eef30 0x5630d1302440 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc002080 tx=0x7fbbfc006620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc01ed5640 1 -- 172.21.15.131:0/2748416944 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc021020 con 0x5630d12eef30 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc01ed5640 1 -- 172.21.15.131:0/2748416944 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbfc017040 con 0x5630d12eef30 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 -- 172.21.15.131:0/2748416944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5630d137d160 con 0x5630d12eef30 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc01ed5640 1 -- 172.21.15.131:0/2748416944 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc016070 con 0x5630d12eef30 2024-09-17T02:42:29.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.872+0000 7fbc0ac03080 1 -- 172.21.15.131:0/2748416944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630d137d6a0 con 0x5630d12eef30 2024-09-17T02:42:29.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc01ed5640 1 -- 172.21.15.131:0/2748416944 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbbfc024020 con 0x5630d12eef30 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc01ed5640 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbe4071c10 0x7fbbe4074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc01ed5640 1 -- 172.21.15.131:0/2748416944 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fbbfc094990 con 0x5630d12eef30 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5630d12ffb80 0x5630d12fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 -- 172.21.15.131:0/2748416944 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) -- 0x5630d13004d0 con 0x5630d12ffb80 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc046da640 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbe4071c10 0x7fbbe4074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc04edb640 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5630d12ffb80 0x5630d12fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc04edb640 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5630d12ffb80 0x5630d12fff90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc04edb640 1 -- 172.21.15.131:0/2748416944 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7fbbec0a7ab0 con 0x5630d12ffb80 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc046da640 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbe4071c10 0x7fbbe4074120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400d700 tx=0x7fbbf4002040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 -- 172.21.15.131:0/2748416944 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5630d12ffb80 msgr2=0x5630d12fff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5630d12ffb80 0x5630d12fff90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:29.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 -- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbe4071c10 msgr2=0x7fbbe4074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbe4071c10 0x7fbbe4074120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbbf400d700 tx=0x7fbbf4002040 comp rx=0 tx=0).stop 2024-09-17T02:42:29.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 -- 172.21.15.131:0/2748416944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d12eef30 msgr2=0x5630d1302440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:29.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 --2- 172.21.15.131:0/2748416944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5630d12eef30 0x5630d1302440 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc002080 tx=0x7fbbfc006620 comp rx=0 tx=0).stop 2024-09-17T02:42:29.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc04edb640 1 -- 172.21.15.131:0/2748416944 reap_dead start 2024-09-17T02:42:29.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 -- 172.21.15.131:0/2748416944 shutdown_connections 2024-09-17T02:42:29.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 -- 172.21.15.131:0/2748416944 >> 172.21.15.131:0/2748416944 conn(0x5630d12abd00 msgr2=0x5630d1306fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:29.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 -- 172.21.15.131:0/2748416944 shutdown_connections 2024-09-17T02:42:29.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:29.876+0000 7fbc0ac03080 1 -- 172.21.15.131:0/2748416944 wait complete. 2024-09-17T02:42:29.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:42:29.886 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:42:29.886 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:42:29.966 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:42:30.034 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:42:30.035 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:42:30.035 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:42:30.035 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:42:30.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:42:30.041 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:42:30.041 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:42:30.041 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:42:30.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:42:30.047 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:42:30.047 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:42:30.047 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:42:30.089 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:42:30.090 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:42:30.090 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:42:30.090 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:42:30.096 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:42:30.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:30.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.296+0000 7fccc6f41640 1 -- 172.21.15.131:0/921500447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccc006e2c0 msgr2=0x7fccc00c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:30.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.296+0000 7fccc6f41640 1 --2- 172.21.15.131:0/921500447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccc006e2c0 0x7fccc00c4580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009840 tx=0x7fccb001c790 comp rx=0 tx=0).stop 2024-09-17T02:42:30.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.296+0000 7fccc6f41640 1 -- 172.21.15.131:0/921500447 shutdown_connections 2024-09-17T02:42:30.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.296+0000 7fccc6f41640 1 --2- 172.21.15.131:0/921500447 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccc00c4ac0 0x7fccc00c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.296+0000 7fccc6f41640 1 --2- 172.21.15.131:0/921500447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccc006e2c0 0x7fccc00c4580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.296+0000 7fccc6f41640 1 -- 172.21.15.131:0/921500447 >> 172.21.15.131:0/921500447 conn(0x7fccc006eea0 msgr2=0x7fccc006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:30.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.296+0000 7fccc6f41640 1 -- 172.21.15.131:0/921500447 shutdown_connections 2024-09-17T02:42:30.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.296+0000 7fccc6f41640 1 -- 172.21.15.131:0/921500447 wait complete. 2024-09-17T02:42:30.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc6f41640 1 Processor -- start 2024-09-17T02:42:30.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc6f41640 1 -- start start 2024-09-17T02:42:30.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc6f41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc006e2c0 0x7fccc0159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:30.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc6f41640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccc00c4ac0 0x7fccc00c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc6f41640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccc00cdbf0 0x7fccc01fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc6f41640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fccc00cf0a0 con 0x7fccc006e2c0 2024-09-17T02:42:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc6f41640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fccc00cef20 con 0x7fccc00cdbf0 2024-09-17T02:42:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc6f41640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fccc00cf220 con 0x7fccc00c4ac0 2024-09-17T02:42:30.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc4cdd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc006e2c0 0x7fccc0159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:30.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc4cdd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc006e2c0 0x7fccc0159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35938/0 (socket says 172.21.15.131:35938) 2024-09-17T02:42:30.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc4cdd640 1 -- 172.21.15.131:0/766081693 learned_addr learned my addr 172.21.15.131:0/766081693 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:30.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccbffff640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccc00c4ac0 0x7fccc00c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:30.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc54de640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccc00cdbf0 0x7fccc01fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:30.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc4cdd640 1 -- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccc00c4ac0 msgr2=0x7fccc00c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:30.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc4cdd640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccc00c4ac0 0x7fccc00c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc4cdd640 1 -- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccc00cdbf0 msgr2=0x7fccc01fa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:30.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc4cdd640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccc00cdbf0 0x7fccc01fa000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc4cdd640 1 -- 172.21.15.131:0/766081693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc015fa90 con 0x7fccc006e2c0 2024-09-17T02:42:30.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc4cdd640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc006e2c0 0x7fccc0159860 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fccb001cc70 tx=0x7fccb0005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:30.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccbd7fa640 1 -- 172.21.15.131:0/766081693 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00ac070 con 0x7fccc006e2c0 2024-09-17T02:42:30.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccbd7fa640 1 -- 172.21.15.131:0/766081693 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccb0005330 con 0x7fccc006e2c0 2024-09-17T02:42:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc6f41640 1 -- 172.21.15.131:0/766081693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc015ef10 con 0x7fccc006e2c0 2024-09-17T02:42:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccbd7fa640 1 -- 172.21.15.131:0/766081693 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb00046d0 con 0x7fccc006e2c0 2024-09-17T02:42:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.300+0000 7fccc6f41640 1 -- 172.21.15.131:0/766081693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc015f4d0 con 0x7fccc006e2c0 2024-09-17T02:42:30.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.304+0000 7fccc6f41640 1 -- 172.21.15.131:0/766081693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc00c4580 con 0x7fccc006e2c0 2024-09-17T02:42:30.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.304+0000 7fccbd7fa640 1 -- 172.21.15.131:0/766081693 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fccb00af070 con 0x7fccc006e2c0 2024-09-17T02:42:30.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.304+0000 7fccbd7fa640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccac073200 0x7fccac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:30.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.304+0000 7fccbd7fa640 1 -- 172.21.15.131:0/766081693 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7fccb012ed70 con 0x7fccc006e2c0 2024-09-17T02:42:30.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.304+0000 7fccbffff640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccac073200 0x7fccac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:30.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.304+0000 7fccbffff640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccac073200 0x7fccac075710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fccc01f9f60 tx=0x7fccb40091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:30.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.308+0000 7fccbd7fa640 1 -- 172.21.15.131:0/766081693 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fccb00c9080 con 0x7fccc006e2c0 2024-09-17T02:42:30.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.500+0000 7fccc6f41640 1 -- 172.21.15.131:0/766081693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccc0152e50 con 0x7fccc006e2c0 2024-09-17T02:42:30.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.500+0000 7fccbd7fa640 1 -- 172.21.15.131:0/766081693 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7fccb0016410 con 0x7fccc006e2c0 2024-09-17T02:42:30.503 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:30.503 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":18,"btime":"2024-09-17T02:41:59:873356+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3342719482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3342719482},{"type":"v1","addr":"172.21.15.131:6835","nonce":3342719482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/3070378350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3070378350},{"type":"v1","addr":"172.21.15.189:6836","nonce":3070378350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/575920659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":575920659},{"type":"v1","addr":"172.21.15.189:6837","nonce":575920659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:41:56.227947+0000","modified":"2024-09-17T02:41:59.279613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6833/1415478569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1415478569},{"type":"v1","addr":"172.21.15.131:6833","nonce":1415478569}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-09-17T02:42:30.503 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 18 2024-09-17T02:42:30.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 -- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccac073200 msgr2=0x7fccac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:30.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccac073200 0x7fccac075710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fccc01f9f60 tx=0x7fccb40091e0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 -- 172.21.15.131:0/766081693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc006e2c0 msgr2=0x7fccc0159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:30.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc006e2c0 0x7fccc0159860 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fccb001cc70 tx=0x7fccb0005cc0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 -- 172.21.15.131:0/766081693 shutdown_connections 2024-09-17T02:42:30.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccac073200 0x7fccac075710 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fccc00cdbf0 0x7fccc01fa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fccc00c4ac0 0x7fccc00c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 --2- 172.21.15.131:0/766081693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fccc006e2c0 0x7fccc0159860 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 -- 172.21.15.131:0/766081693 >> 172.21.15.131:0/766081693 conn(0x7fccc006eea0 msgr2=0x7fccc01537c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:30.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 -- 172.21.15.131:0/766081693 shutdown_connections 2024-09-17T02:42:30.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.504+0000 7fccc6f41640 1 -- 172.21.15.131:0/766081693 wait complete. 2024-09-17T02:42:30.523 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.672+0000 7f77d83b4640 1 -- 172.21.15.131:0/1221332876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77d0074310 msgr2=0x7f77d00af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.672+0000 7f77d83b4640 1 --2- 172.21.15.131:0/1221332876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77d0074310 0x7f77d00af4c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f77d0072420 tx=0x7f77c001c8c0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.672+0000 7f77d83b4640 1 -- 172.21.15.131:0/1221332876 shutdown_connections 2024-09-17T02:42:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.672+0000 7f77d83b4640 1 --2- 172.21.15.131:0/1221332876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77d0074310 0x7f77d00af4c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.672+0000 7f77d83b4640 1 --2- 172.21.15.131:0/1221332876 >> v2:172.21.15.189:3300/0 conn(0x7f77d0070c80 0x7f77d015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.672+0000 7f77d83b4640 1 -- 172.21.15.131:0/1221332876 >> 172.21.15.131:0/1221332876 conn(0x7f77d00d13d0 msgr2=0x7f77d00734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:30.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.672+0000 7f77d83b4640 1 -- 172.21.15.131:0/1221332876 shutdown_connections 2024-09-17T02:42:30.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.672+0000 7f77d83b4640 1 -- 172.21.15.131:0/1221332876 wait complete. 2024-09-17T02:42:30.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d83b4640 1 Processor -- start 2024-09-17T02:42:30.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d83b4640 1 -- start start 2024-09-17T02:42:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d83b4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f77d0070c80 0x7f77d014d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d83b4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77d0151a90 0x7f77d0146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d83b4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77d0148010 0x7f77d006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d83b4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f77d00b0110 con 0x7f77d0151a90 2024-09-17T02:42:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d83b4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f77d006f270 con 0x7f77d0148010 2024-09-17T02:42:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d83b4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f77d0147c70 con 0x7f77d0070c80 2024-09-17T02:42:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d594f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77d0151a90 0x7f77d0146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d594f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77d0151a90 0x7f77d0146230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35960/0 (socket says 172.21.15.131:35960) 2024-09-17T02:42:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d594f640 1 -- 172.21.15.131:0/1549008613 learned_addr learned my addr 172.21.15.131:0/1549008613 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d6150640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f77d0070c80 0x7f77d014d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d6951640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77d0148010 0x7f77d006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d594f640 1 -- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f77d0070c80 msgr2=0x7f77d014d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d594f640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f77d0070c80 0x7f77d014d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d594f640 1 -- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77d0148010 msgr2=0x7f77d006f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d594f640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77d0148010 0x7f77d006f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d594f640 1 -- 172.21.15.131:0/1549008613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d0134c10 con 0x7f77d0151a90 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d594f640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77d0151a90 0x7f77d0146230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00c450 tx=0x7f77cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77c6ffd640 1 -- 172.21.15.131:0/1549008613 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0052d0 con 0x7f77d0151a90 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77c6ffd640 1 -- 172.21.15.131:0/1549008613 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77cc005470 con 0x7f77d0151a90 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d83b4640 1 -- 172.21.15.131:0/1549008613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d0134ef0 con 0x7f77d0151a90 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77c6ffd640 1 -- 172.21.15.131:0/1549008613 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc012870 con 0x7f77d0151a90 2024-09-17T02:42:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.676+0000 7f77d83b4640 1 -- 172.21.15.131:0/1549008613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d01479b0 con 0x7f77d0151a90 2024-09-17T02:42:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.680+0000 7f77d83b4640 1 -- 172.21.15.131:0/1549008613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d015bc50 con 0x7f77d0151a90 2024-09-17T02:42:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.680+0000 7f77c6ffd640 1 -- 172.21.15.131:0/1549008613 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f77cc003600 con 0x7f77d0151a90 2024-09-17T02:42:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.680+0000 7f77c6ffd640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f77bc073230 0x7f77bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.680+0000 7f77c6ffd640 1 -- 172.21.15.131:0/1549008613 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f77cc093ee0 con 0x7f77d0151a90 2024-09-17T02:42:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.680+0000 7f77d6150640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f77bc073230 0x7f77bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.680+0000 7f77d6150640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f77bc073230 0x7f77bc075740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005c50 tx=0x7f77c8005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:30.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.684+0000 7f77c6ffd640 1 -- 172.21.15.131:0/1549008613 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f77cc05ed60 con 0x7f77d0151a90 2024-09-17T02:42:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.884+0000 7f77d83b4640 1 -- 172.21.15.131:0/1549008613 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77d014dd40 con 0x7f77d0151a90 2024-09-17T02:42:30.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.884+0000 7f77c6ffd640 1 -- 172.21.15.131:0/1549008613 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7f77cc05e660 con 0x7f77d0151a90 2024-09-17T02:42:30.888 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:30.888 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":18,"btime":"2024-09-17T02:41:59:873356+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3342719482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3342719482},{"type":"v1","addr":"172.21.15.131:6835","nonce":3342719482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/3070378350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3070378350},{"type":"v1","addr":"172.21.15.189:6836","nonce":3070378350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/575920659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":575920659},{"type":"v1","addr":"172.21.15.189:6837","nonce":575920659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:41:56.227947+0000","modified":"2024-09-17T02:41:59.279613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6833/1415478569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1415478569},{"type":"v1","addr":"172.21.15.131:6833","nonce":1415478569}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-09-17T02:42:30.888 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 18 2024-09-17T02:42:30.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 -- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f77bc073230 msgr2=0x7f77bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:30.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f77bc073230 0x7f77bc075740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005c50 tx=0x7f77c8005be0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 -- 172.21.15.131:0/1549008613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77d0151a90 msgr2=0x7f77d0146230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77d0151a90 0x7f77d0146230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00c450 tx=0x7f77cc00c910 comp rx=0 tx=0).stop 2024-09-17T02:42:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 -- 172.21.15.131:0/1549008613 shutdown_connections 2024-09-17T02:42:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f77bc073230 0x7f77bc075740 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f77d0148010 0x7f77d006f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77d0151a90 0x7f77d0146230 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 --2- 172.21.15.131:0/1549008613 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f77d0070c80 0x7f77d014d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 -- 172.21.15.131:0/1549008613 >> 172.21.15.131:0/1549008613 conn(0x7f77d00d13d0 msgr2=0x7f77d0148e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 -- 172.21.15.131:0/1549008613 shutdown_connections 2024-09-17T02:42:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:30.888+0000 7f77d83b4640 1 -- 172.21.15.131:0/1549008613 wait complete. 2024-09-17T02:42:30.910 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:42:31.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.060+0000 7f84a8080640 1 -- 172.21.15.131:0/4106697229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a00c4c90 msgr2=0x7f84a00c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.060+0000 7f84a8080640 1 --2- 172.21.15.131:0/4106697229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a00c4c90 0x7f84a00c50a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f8498009bc0 tx=0x7f849801cb10 comp rx=0 tx=0).stop 2024-09-17T02:42:31.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.060+0000 7f84a8080640 1 -- 172.21.15.131:0/4106697229 shutdown_connections 2024-09-17T02:42:31.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.060+0000 7f84a8080640 1 --2- 172.21.15.131:0/4106697229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a00c4c90 0x7f84a00c50a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.060+0000 7f84a8080640 1 --2- 172.21.15.131:0/4106697229 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84a00cec00 0x7f84a00cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.060+0000 7f84a8080640 1 -- 172.21.15.131:0/4106697229 >> 172.21.15.131:0/4106697229 conn(0x7f84a006eea0 msgr2=0x7f84a006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:31.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.060+0000 7f84a8080640 1 -- 172.21.15.131:0/4106697229 shutdown_connections 2024-09-17T02:42:31.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.060+0000 7f84a8080640 1 -- 172.21.15.131:0/4106697229 wait complete. 2024-09-17T02:42:31.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a8080640 1 Processor -- start 2024-09-17T02:42:31.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a8080640 1 -- start start 2024-09-17T02:42:31.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a8080640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84a006e2c0 0x7f84a015f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a8080640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a00c4c90 0x7f84a015fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a8080640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84a00cec00 0x7f84a015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a8080640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0169e70 con 0x7f84a00c4c90 2024-09-17T02:42:31.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a8080640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84a0169cf0 con 0x7f84a00cec00 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a8080640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f84a0169ff0 con 0x7f84a006e2c0 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a561b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a00c4c90 0x7f84a015fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a561b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a00c4c90 0x7f84a015fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35980/0 (socket says 172.21.15.131:35980) 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a561b640 1 -- 172.21.15.131:0/1736737216 learned_addr learned my addr 172.21.15.131:0/1736737216 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a561b640 1 -- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84a006e2c0 msgr2=0x7f84a015f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a561b640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84a006e2c0 0x7f84a015f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a561b640 1 -- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84a00cec00 msgr2=0x7f84a015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a561b640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84a00cec00 0x7f84a015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a561b640 1 -- 172.21.15.131:0/1736737216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a014f610 con 0x7f84a00c4c90 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a561b640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a00c4c90 0x7f84a015fa70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f849000d680 tx=0x7f849000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84967fc640 1 -- 172.21.15.131:0/1736737216 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490017070 con 0x7f84a00c4c90 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84967fc640 1 -- 172.21.15.131:0/1736737216 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8490005310 con 0x7f84a00c4c90 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a8080640 1 -- 172.21.15.131:0/1736737216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a014f8f0 con 0x7f84a00c4c90 2024-09-17T02:42:31.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84967fc640 1 -- 172.21.15.131:0/1736737216 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490009820 con 0x7f84a00c4c90 2024-09-17T02:42:31.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a8080640 1 -- 172.21.15.131:0/1736737216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a014feb0 con 0x7f84a00c4c90 2024-09-17T02:42:31.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.064+0000 7f84a8080640 1 -- 172.21.15.131:0/1736737216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a00cf010 con 0x7f84a00c4c90 2024-09-17T02:42:31.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.068+0000 7f84967fc640 1 -- 172.21.15.131:0/1736737216 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f84900099c0 con 0x7f84a00c4c90 2024-09-17T02:42:31.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.068+0000 7f84967fc640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8484073200 0x7f8484075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.068+0000 7f84967fc640 1 -- 172.21.15.131:0/1736737216 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f8490094360 con 0x7f84a00c4c90 2024-09-17T02:42:31.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.068+0000 7f84a5e1c640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8484073200 0x7f8484075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:31.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.068+0000 7f84a5e1c640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8484073200 0x7f8484075710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f849c0046c0 tx=0x7f849c007910 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:31.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.072+0000 7f84967fc640 1 -- 172.21.15.131:0/1736737216 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f849005f280 con 0x7f84a00c4c90 2024-09-17T02:42:31.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.232+0000 7f84a8080640 1 -- 172.21.15.131:0/1736737216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84a00c8760 con 0x7f84a00c4c90 2024-09-17T02:42:31.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.232+0000 7f84967fc640 1 -- 172.21.15.131:0/1736737216 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17050 (secure 0 0 0) 0x7f849005eb80 con 0x7f84a00c4c90 2024-09-17T02:42:31.235 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:31.235 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:41:57.843697+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:41:53.913442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-17T02:41:55.184999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:42:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 -- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8484073200 msgr2=0x7f8484075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8484073200 0x7f8484075710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f849c0046c0 tx=0x7f849c007910 comp rx=0 tx=0).stop 2024-09-17T02:42:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 -- 172.21.15.131:0/1736737216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a00c4c90 msgr2=0x7f84a015fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a00c4c90 0x7f84a015fa70 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f849000d680 tx=0x7f849000db40 comp rx=0 tx=0).stop 2024-09-17T02:42:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 -- 172.21.15.131:0/1736737216 shutdown_connections 2024-09-17T02:42:31.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8484073200 0x7f8484075710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84a00cec00 0x7f84a015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84a00c4c90 0x7f84a015fa70 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 --2- 172.21.15.131:0/1736737216 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84a006e2c0 0x7f84a015f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 -- 172.21.15.131:0/1736737216 >> 172.21.15.131:0/1736737216 conn(0x7f84a006eea0 msgr2=0x7f84a00b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:31.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 -- 172.21.15.131:0/1736737216 shutdown_connections 2024-09-17T02:42:31.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.236+0000 7f84a8080640 1 -- 172.21.15.131:0/1736737216 wait complete. 2024-09-17T02:42:31.255 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T02:42:31.256 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:31.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.400+0000 7ff7c4b83640 1 -- 172.21.15.131:0/1982383783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a8003a40 msgr2=0x7ff7a8003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.400+0000 7ff7c4b83640 1 --2- 172.21.15.131:0/1982383783 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a8003a40 0x7ff7a8003eb0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00016f0 tx=0x7ff7b001c7b0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.400+0000 7ff7bed9d640 1 -- 172.21.15.131:0/1982383783 reap_dead start 2024-09-17T02:42:31.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.400+0000 7ff7c4b83640 1 -- 172.21.15.131:0/1982383783 shutdown_connections 2024-09-17T02:42:31.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.400+0000 7ff7c4b83640 1 -- 172.21.15.131:0/1982383783 >> 172.21.15.131:0/1982383783 conn(0x7ff7c006fbe0 msgr2=0x7ff7c006fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:31.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.400+0000 7ff7c4b83640 1 -- 172.21.15.131:0/1982383783 shutdown_connections 2024-09-17T02:42:31.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.400+0000 7ff7c4b83640 1 -- 172.21.15.131:0/1982383783 wait complete. 2024-09-17T02:42:31.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7c4b83640 1 Processor -- start 2024-09-17T02:42:31.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7c4b83640 1 -- start start 2024-09-17T02:42:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7c4b83640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a8003a40 0x7ff7c014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7c4b83640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7c0165710 0x7ff7c0165b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7c4b83640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff7c00b1ab0 0x7ff7c00b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7c4b83640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c01682a0 con 0x7ff7a8003a40 2024-09-17T02:42:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7c4b83640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0168120 con 0x7ff7c0165710 2024-09-17T02:42:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7c4b83640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c0168420 con 0x7ff7c00b1ab0 2024-09-17T02:42:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7be59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a8003a40 0x7ff7c014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:31.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7be59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a8003a40 0x7ff7c014f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43366/0 (socket says 172.21.15.131:43366) 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7be59c640 1 -- 172.21.15.131:0/3723401648 learned_addr learned my addr 172.21.15.131:0/3723401648 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7be59c640 1 -- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff7c00b1ab0 msgr2=0x7ff7c00b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7be59c640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff7c00b1ab0 0x7ff7c00b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7be59c640 1 -- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7c0165710 msgr2=0x7ff7c0165b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7be59c640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7c0165710 0x7ff7c0165b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7be59c640 1 -- 172.21.15.131:0/3723401648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c00b2400 con 0x7ff7a8003a40 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7be59c640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a8003a40 0x7ff7c014f360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff7b001aeb0 tx=0x7ff7b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7aeffd640 1 -- 172.21.15.131:0/3723401648 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b00bb070 con 0x7ff7a8003a40 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7aeffd640 1 -- 172.21.15.131:0/3723401648 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7b0006500 con 0x7ff7a8003a40 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7c4b83640 1 -- 172.21.15.131:0/3723401648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c00cec80 con 0x7ff7a8003a40 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7aeffd640 1 -- 172.21.15.131:0/3723401648 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b00ae070 con 0x7ff7a8003a40 2024-09-17T02:42:31.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.404+0000 7ff7c4b83640 1 -- 172.21.15.131:0/3723401648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c00ca380 con 0x7ff7a8003a40 2024-09-17T02:42:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.408+0000 7ff7c4b83640 1 -- 172.21.15.131:0/3723401648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c006db20 con 0x7ff7a8003a40 2024-09-17T02:42:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.408+0000 7ff7aeffd640 1 -- 172.21.15.131:0/3723401648 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff7b00a8480 con 0x7ff7a8003a40 2024-09-17T02:42:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.408+0000 7ff7aeffd640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff7a0073200 0x7ff7a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.408+0000 7ff7aeffd640 1 -- 172.21.15.131:0/3723401648 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7ff7b012df60 con 0x7ff7a8003a40 2024-09-17T02:42:31.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.408+0000 7ff7bdd9b640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff7a0073200 0x7ff7a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:31.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.408+0000 7ff7bdd9b640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff7a0073200 0x7ff7a0075710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4001870 tx=0x7ff7b40014c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:31.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.412+0000 7ff7aeffd640 1 -- 172.21.15.131:0/3723401648 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff7c00b2400 con 0x7ff7a8003a40 2024-09-17T02:42:31.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.600+0000 7ff7c4b83640 1 -- 172.21.15.131:0/3723401648 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7c006d0e0 con 0x7ff7a8003a40 2024-09-17T02:42:31.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.600+0000 7ff7aeffd640 1 -- 172.21.15.131:0/3723401648 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7ff7b0016510 con 0x7ff7a8003a40 2024-09-17T02:42:31.605 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:31.605 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":18,"btime":"2024-09-17T02:41:59:873356+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3342719482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3342719482},{"type":"v1","addr":"172.21.15.131:6835","nonce":3342719482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/3070378350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3070378350},{"type":"v1","addr":"172.21.15.189:6836","nonce":3070378350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/575920659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":575920659},{"type":"v1","addr":"172.21.15.189:6837","nonce":575920659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:41:56.227947+0000","modified":"2024-09-17T02:41:59.279613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6833/1415478569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1415478569},{"type":"v1","addr":"172.21.15.131:6833","nonce":1415478569}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-09-17T02:42:31.605 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 18 2024-09-17T02:42:31.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 -- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff7a0073200 msgr2=0x7ff7a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff7a0073200 0x7ff7a0075710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4001870 tx=0x7ff7b40014c0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 -- 172.21.15.131:0/3723401648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a8003a40 msgr2=0x7ff7c014f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a8003a40 0x7ff7c014f360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff7b001aeb0 tx=0x7ff7b0003c10 comp rx=0 tx=0).stop 2024-09-17T02:42:31.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 -- 172.21.15.131:0/3723401648 shutdown_connections 2024-09-17T02:42:31.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff7a0073200 0x7ff7a0075710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff7c00b1ab0 0x7ff7c00b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7c0165710 0x7ff7c0165b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 --2- 172.21.15.131:0/3723401648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff7a8003a40 0x7ff7c014f360 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 -- 172.21.15.131:0/3723401648 >> 172.21.15.131:0/3723401648 conn(0x7ff7c006fbe0 msgr2=0x7ff7c015f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:31.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 -- 172.21.15.131:0/3723401648 shutdown_connections 2024-09-17T02:42:31.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.604+0000 7ff7c4b83640 1 -- 172.21.15.131:0/3723401648 wait complete. 2024-09-17T02:42:31.627 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:31.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.764+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/3053209605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c70003a40 msgr2=0x7f8c70003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.764+0000 7f8c8b4e4640 1 --2- 172.21.15.131:0/3053209605 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c70003a40 0x7f8c70003eb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8c84073220 tx=0x7f8c8001cc60 comp rx=0 tx=0).stop 2024-09-17T02:42:31.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.764+0000 7f8c89a81640 1 -- 172.21.15.131:0/3053209605 reap_dead start 2024-09-17T02:42:31.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.764+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/3053209605 shutdown_connections 2024-09-17T02:42:31.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.764+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/3053209605 >> 172.21.15.131:0/3053209605 conn(0x7f8c8406fbe0 msgr2=0x7f8c8406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:31.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.764+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/3053209605 shutdown_connections 2024-09-17T02:42:31.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.764+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/3053209605 wait complete. 2024-09-17T02:42:31.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c8b4e4640 1 Processor -- start 2024-09-17T02:42:31.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c8b4e4640 1 -- start start 2024-09-17T02:42:31.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c8b4e4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c70003a40 0x7f8c841522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c8b4e4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c84155580 0x7f8c841f9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c8b4e4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c841f9b70 0x7f8c841f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c8b4e4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841689d0 con 0x7f8c70003a40 2024-09-17T02:42:31.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c8b4e4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84168850 con 0x7f8c841f9b70 2024-09-17T02:42:31.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c8b4e4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8c84168b50 con 0x7f8c84155580 2024-09-17T02:42:31.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c89280640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c70003a40 0x7f8c841522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:31.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c89280640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c70003a40 0x7f8c841522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43394/0 (socket says 172.21.15.131:43394) 2024-09-17T02:42:31.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c89280640 1 -- 172.21.15.131:0/2202798277 learned_addr learned my addr 172.21.15.131:0/2202798277 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c89a81640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c841f9b70 0x7f8c841f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c88a7f640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c84155580 0x7f8c841f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c89280640 1 -- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c84155580 msgr2=0x7f8c841f9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c89280640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c84155580 0x7f8c841f9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c89280640 1 -- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c841f9b70 msgr2=0x7f8c841f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c89280640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c841f9b70 0x7f8c841f9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c89280640 1 -- 172.21.15.131:0/2202798277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c84155b60 con 0x7f8c70003a40 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c89a81640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c841f9b70 0x7f8c841f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c89280640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c70003a40 0x7f8c841522d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8c80001ec0 tx=0x7f8c800056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c79ffb640 1 -- 172.21.15.131:0/2202798277 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c800ab040 con 0x7f8c70003a40 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c79ffb640 1 -- 172.21.15.131:0/2202798277 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c800be070 con 0x7f8c70003a40 2024-09-17T02:42:31.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/2202798277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c84155e40 con 0x7f8c70003a40 2024-09-17T02:42:31.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c79ffb640 1 -- 172.21.15.131:0/2202798277 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80004a20 con 0x7f8c70003a40 2024-09-17T02:42:31.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.768+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/2202798277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c840748a0 con 0x7f8c70003a40 2024-09-17T02:42:31.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.772+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/2202798277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c84167e20 con 0x7f8c70003a40 2024-09-17T02:42:31.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.772+0000 7f8c79ffb640 1 -- 172.21.15.131:0/2202798277 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8c800b6070 con 0x7f8c70003a40 2024-09-17T02:42:31.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.772+0000 7f8c79ffb640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c68073200 0x7f8c68075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:31.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.772+0000 7f8c79ffb640 1 -- 172.21.15.131:0/2202798277 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f8c8012f060 con 0x7f8c70003a40 2024-09-17T02:42:31.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.772+0000 7f8c88a7f640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c68073200 0x7f8c68075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:31.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.772+0000 7f8c88a7f640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c68073200 0x7f8c68075710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8c74001ec0 tx=0x7f8c74001e30 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:31.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.776+0000 7f8c79ffb640 1 -- 172.21.15.131:0/2202798277 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8c80016970 con 0x7f8c70003a40 2024-09-17T02:42:31.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.968+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/2202798277 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c84156010 con 0x7f8c70003a40 2024-09-17T02:42:31.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.968+0000 7f8c79ffb640 1 -- 172.21.15.131:0/2202798277 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5232 (secure 0 0 0) 0x7f8c800fa010 con 0x7f8c70003a40 2024-09-17T02:42:31.971 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:31.971 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":18,"btime":"2024-09-17T02:41:59:873356+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3342719482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3342719482},{"type":"v1","addr":"172.21.15.131:6835","nonce":3342719482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/3070378350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3070378350},{"type":"v1","addr":"172.21.15.189:6836","nonce":3070378350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/575920659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":575920659},{"type":"v1","addr":"172.21.15.189:6837","nonce":575920659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:41:56.227947+0000","modified":"2024-09-17T02:41:59.279613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5022":{"gid":5022,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6833/1415478569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1415478569},{"type":"v1","addr":"172.21.15.131:6833","nonce":1415478569}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5022,"qdb_cluster":[5022]},"id":2}]} 2024-09-17T02:42:31.971 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 18 2024-09-17T02:42:31.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c68073200 msgr2=0x7f8c68075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c68073200 0x7f8c68075710 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8c74001ec0 tx=0x7f8c74001e30 comp rx=0 tx=0).stop 2024-09-17T02:42:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/2202798277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c70003a40 msgr2=0x7f8c841522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c70003a40 0x7f8c841522d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8c80001ec0 tx=0x7f8c800056e0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/2202798277 shutdown_connections 2024-09-17T02:42:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c68073200 0x7f8c68075710 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c841f9b70 0x7f8c841f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c84155580 0x7f8c841f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 --2- 172.21.15.131:0/2202798277 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c70003a40 0x7f8c841522d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/2202798277 >> 172.21.15.131:0/2202798277 conn(0x7f8c8406fbe0 msgr2=0x7f8c840b4100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/2202798277 shutdown_connections 2024-09-17T02:42:31.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:31.972+0000 7f8c8b4e4640 1 -- 172.21.15.131:0/2202798277 wait complete. 2024-09-17T02:42:31.992 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:42:32.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.128+0000 7f9c13bd6640 1 -- 172.21.15.131:0/1573064628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c0c06e5b0 msgr2=0x7f9c0c15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.128+0000 7f9c13bd6640 1 --2- 172.21.15.131:0/1573064628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c0c06e5b0 0x7f9c0c15bea0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc009900 tx=0x7f9bfc01c8e0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.128+0000 7f9c13bd6640 1 -- 172.21.15.131:0/1573064628 shutdown_connections 2024-09-17T02:42:32.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.128+0000 7f9c13bd6640 1 --2- 172.21.15.131:0/1573064628 >> v2:172.21.15.189:3300/0 conn(0x7f9c0c15c3e0 0x7f9c0c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.128+0000 7f9c13bd6640 1 --2- 172.21.15.131:0/1573064628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c0c06e5b0 0x7f9c0c15bea0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.128+0000 7f9c13bd6640 1 -- 172.21.15.131:0/1573064628 >> 172.21.15.131:0/1573064628 conn(0x7f9c0c06eea0 msgr2=0x7f9c0c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.128+0000 7f9c13bd6640 1 -- 172.21.15.131:0/1573064628 shutdown_connections 2024-09-17T02:42:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.128+0000 7f9c13bd6640 1 -- 172.21.15.131:0/1573064628 wait complete. 2024-09-17T02:42:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c13bd6640 1 Processor -- start 2024-09-17T02:42:32.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c13bd6640 1 -- start start 2024-09-17T02:42:32.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c13bd6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c0c0ced00 0x7f9c0c14f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c13bd6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0c15c3e0 0x7f9c0c154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c13bd6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c0c150000 0x7f9c0c1554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c13bd6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c1687a0 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c13bd6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c0c168620 con 0x7f9c0c150000 2024-09-17T02:42:32.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c13bd6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9c0c168920 con 0x7f9c0c0ced00 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c11171640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0c15c3e0 0x7f9c0c154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c11171640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0c15c3e0 0x7f9c0c154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43402/0 (socket says 172.21.15.131:43402) 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c11171640 1 -- 172.21.15.131:0/3735483939 learned_addr learned my addr 172.21.15.131:0/3735483939 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c12173640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c0c150000 0x7f9c0c1554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c11972640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c0c0ced00 0x7f9c0c14f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c11171640 1 -- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c0c0ced00 msgr2=0x7f9c0c14f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c11171640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c0c0ced00 0x7f9c0c14f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c11171640 1 -- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c0c150000 msgr2=0x7f9c0c1554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c11171640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c0c150000 0x7f9c0c1554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c11171640 1 -- 172.21.15.131:0/3735483939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c0c1f9730 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c11972640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c0c0ced00 0x7f9c0c14f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c11171640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0c15c3e0 0x7f9c0c154f90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0b0040 tx=0x7f9bfc005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:32.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c027fc640 1 -- 172.21.15.131:0/3735483939 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc0b2040 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c027fc640 1 -- 172.21.15.131:0/3735483939 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bfc004330 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c13bd6640 1 -- 172.21.15.131:0/3735483939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c0c1f9a10 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c027fc640 1 -- 172.21.15.131:0/3735483939 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bfc005280 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c13bd6640 1 -- 172.21.15.131:0/3735483939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c0c14f820 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c13bd6640 1 -- 172.21.15.131:0/3735483939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c0c15bea0 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.132+0000 7f9c027fc640 1 -- 172.21.15.131:0/3735483939 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9bfc005420 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.136+0000 7f9c027fc640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bf0073230 0x7f9bf0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.136+0000 7f9c027fc640 1 -- 172.21.15.131:0/3735483939 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6101+0+0 (secure 0 0 0) 0x7f9bfc0bd070 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.136+0000 7f9c11972640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bf0073230 0x7f9bf0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:32.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.136+0000 7f9c11972640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bf0073230 0x7f9bf0075740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9c08001630 tx=0x7f9c08009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:32.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.140+0000 7f9c027fc640 1 -- 172.21.15.131:0/3735483939 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9bfc0164d0 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.316+0000 7f9c13bd6640 1 -- 172.21.15.131:0/3735483939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c0c15eb40 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c027fc640 1 -- 172.21.15.131:0/3735483939 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17193 (secure 0 0 0) 0x7f9bfc0f8f30 con 0x7f9c0c15c3e0 2024-09-17T02:42:32.321 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:32.321 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":41,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:42:32.285804+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:41:53.913442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-17T02:41:55.184999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.10","mappings":[{"from":4,"to":7}]},{"pgid":"6.23","mappings":[{"from":3,"to":2}]},{"pgid":"6.3d","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:42:32.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 -- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bf0073230 msgr2=0x7f9bf0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bf0073230 0x7f9bf0075740 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9c08001630 tx=0x7f9c08009040 comp rx=0 tx=0).stop 2024-09-17T02:42:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 -- 172.21.15.131:0/3735483939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0c15c3e0 msgr2=0x7f9c0c154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0c15c3e0 0x7f9c0c154f90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0b0040 tx=0x7f9bfc005e00 comp rx=0 tx=0).stop 2024-09-17T02:42:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 -- 172.21.15.131:0/3735483939 shutdown_connections 2024-09-17T02:42:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bf0073230 0x7f9bf0075740 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c0c150000 0x7f9c0c1554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c0c15c3e0 0x7f9c0c154f90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 --2- 172.21.15.131:0/3735483939 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c0c0ced00 0x7f9c0c14f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 -- 172.21.15.131:0/3735483939 >> 172.21.15.131:0/3735483939 conn(0x7f9c0c06eea0 msgr2=0x7f9c0c15b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 -- 172.21.15.131:0/3735483939 shutdown_connections 2024-09-17T02:42:32.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.320+0000 7f9c13bd6640 1 -- 172.21.15.131:0/3735483939 wait complete. 2024-09-17T02:42:32.342 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:42:32.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.480+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3054187575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0ac004810 msgr2=0x7fe0ac004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.480+0000 7fe0c90c1640 1 --2- 172.21.15.131:0/3054187575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0ac004810 0x7fe0ac004c20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800baf0 tx=0x7fe0b801cdc0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.480+0000 7fe0c359e640 1 -- 172.21.15.131:0/3054187575 reap_dead start 2024-09-17T02:42:32.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.480+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3054187575 shutdown_connections 2024-09-17T02:42:32.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.480+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3054187575 >> 172.21.15.131:0/3054187575 conn(0x7fe0c406eea0 msgr2=0x7fe0c406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:32.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.480+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3054187575 shutdown_connections 2024-09-17T02:42:32.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.480+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3054187575 wait complete. 2024-09-17T02:42:32.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c90c1640 1 Processor -- start 2024-09-17T02:42:32.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c90c1640 1 -- start start 2024-09-17T02:42:32.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c90c1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0ac004810 0x7fe0c4154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c90c1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0c41f46e0 0x7fe0c41f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c90c1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c41f5030 0x7fe0c41f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c90c1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c415c9b0 con 0x7fe0c41f5030 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c90c1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c40cf320 con 0x7fe0c41f46e0 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c90c1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c406dea0 con 0x7fe0ac004810 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c359e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c41f5030 0x7fe0c41f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c359e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c41f5030 0x7fe0c41f2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43424/0 (socket says 172.21.15.131:43424) 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c359e640 1 -- 172.21.15.131:0/3737165807 learned_addr learned my addr 172.21.15.131:0/3737165807 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c2d9d640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0ac004810 0x7fe0c4154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c259c640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0c41f46e0 0x7fe0c41f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c359e640 1 -- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0ac004810 msgr2=0x7fe0c4154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c359e640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0ac004810 0x7fe0c4154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c359e640 1 -- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0c41f46e0 msgr2=0x7fe0c41f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c359e640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0c41f46e0 0x7fe0c41f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c359e640 1 -- 172.21.15.131:0/3737165807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c4157fc0 con 0x7fe0c41f5030 2024-09-17T02:42:32.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c2d9d640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0ac004810 0x7fe0c4154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:32.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c259c640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0c41f46e0 0x7fe0c41f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:32.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c359e640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c41f5030 0x7fe0c41f2740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc002240 tx=0x7fe0bc010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:32.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0ab7fe640 1 -- 172.21.15.131:0/3737165807 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc00e9d0 con 0x7fe0c41f5030 2024-09-17T02:42:32.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0ab7fe640 1 -- 172.21.15.131:0/3737165807 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0bc00ec80 con 0x7fe0c41f5030 2024-09-17T02:42:32.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3737165807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c41582a0 con 0x7fe0c41f5030 2024-09-17T02:42:32.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0ab7fe640 1 -- 172.21.15.131:0/3737165807 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0136e0 con 0x7fe0c41f5030 2024-09-17T02:42:32.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.484+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3737165807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c4158860 con 0x7fe0c41f5030 2024-09-17T02:42:32.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.488+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3737165807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c4075b80 con 0x7fe0c41f5030 2024-09-17T02:42:32.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.488+0000 7fe0ab7fe640 1 -- 172.21.15.131:0/3737165807 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe0bc013880 con 0x7fe0c41f5030 2024-09-17T02:42:32.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.488+0000 7fe0ab7fe640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe0b0073230 0x7fe0b0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.488+0000 7fe0ab7fe640 1 -- 172.21.15.131:0/3737165807 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6188+0+0 (secure 0 0 0) 0x7fe0bc097f20 con 0x7fe0c41f5030 2024-09-17T02:42:32.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.488+0000 7fe0c2d9d640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe0b0073230 0x7fe0b0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:32.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.488+0000 7fe0c2d9d640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe0b0073230 0x7fe0b0075740 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005d90 tx=0x7fe0b4005d20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:32.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.492+0000 7fe0ab7fe640 1 -- 172.21.15.131:0/3737165807 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe0c406e7f0 con 0x7fe0c41f5030 2024-09-17T02:42:32.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.692+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3737165807 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe0c415bea0 con 0x7fe0c41f5030 2024-09-17T02:42:32.696 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:42:32.692+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:42:32.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.696+0000 7fe0ab7fe640 1 -- 172.21.15.131:0/3737165807 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7fe0bc062df0 con 0x7fe0c41f5030 2024-09-17T02:42:32.699 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:42:32.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe0b0073230 msgr2=0x7fe0b0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe0b0073230 0x7fe0b0075740 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005d90 tx=0x7fe0b4005d20 comp rx=0 tx=0).stop 2024-09-17T02:42:32.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3737165807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c41f5030 msgr2=0x7fe0c41f2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c41f5030 0x7fe0c41f2740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc002240 tx=0x7fe0bc010ba0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3737165807 shutdown_connections 2024-09-17T02:42:32.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe0b0073230 0x7fe0b0075740 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0c41f5030 0x7fe0c41f2740 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe0c41f46e0 0x7fe0c41f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 --2- 172.21.15.131:0/3737165807 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0ac004810 0x7fe0c4154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3737165807 >> 172.21.15.131:0/3737165807 conn(0x7fe0c406eea0 msgr2=0x7fe0c40cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:32.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3737165807 shutdown_connections 2024-09-17T02:42:32.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.700+0000 7fe0c90c1640 1 -- 172.21.15.131:0/3737165807 wait complete. 2024-09-17T02:42:32.711 INFO:tasks.ceph.mds.c.smithi131.stderr: -6792> 2024-09-17T02:42:00.328+0000 7fcb6c222640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:42:32.729 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T02:42:32.785 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:42:32.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.864+0000 7fc7ac070640 1 -- 172.21.15.131:0/2330324371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7a406e3e0 msgr2=0x7fc7a415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.864+0000 7fc7ac070640 1 --2- 172.21.15.131:0/2330324371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7a406e3e0 0x7fc7a415bea0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc798009900 tx=0x7fc79801c700 comp rx=0 tx=0).stop 2024-09-17T02:42:32.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.864+0000 7fc7ac070640 1 -- 172.21.15.131:0/2330324371 shutdown_connections 2024-09-17T02:42:32.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.864+0000 7fc7ac070640 1 --2- 172.21.15.131:0/2330324371 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7a406e3e0 0x7fc7a415bea0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.864+0000 7fc7ac070640 1 --2- 172.21.15.131:0/2330324371 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7a40ced00 0x7fc7a406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.864+0000 7fc7ac070640 1 -- 172.21.15.131:0/2330324371 >> 172.21.15.131:0/2330324371 conn(0x7fc7a406eea0 msgr2=0x7fc7a406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:32.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.864+0000 7fc7ac070640 1 -- 172.21.15.131:0/2330324371 shutdown_connections 2024-09-17T02:42:32.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.864+0000 7fc7ac070640 1 -- 172.21.15.131:0/2330324371 wait complete. 2024-09-17T02:42:32.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7ac070640 1 Processor -- start 2024-09-17T02:42:32.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7ac070640 1 -- start start 2024-09-17T02:42:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7ac070640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7a406e3e0 0x7fc7a414b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7ac070640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7a40ced00 0x7fc7a4153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7ac070640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7a415c3e0 0x7fc7a4154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7ac070640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a4168b60 con 0x7fc7a406e3e0 2024-09-17T02:42:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7ac070640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7a41689e0 con 0x7fc7a415c3e0 2024-09-17T02:42:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7ac070640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc7a4168ce0 con 0x7fc7a40ced00 2024-09-17T02:42:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a9e0c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7a406e3e0 0x7fc7a414b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a9e0c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7a406e3e0 0x7fc7a414b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43440/0 (socket says 172.21.15.131:43440) 2024-09-17T02:42:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a9e0c640 1 -- 172.21.15.131:0/1314989981 learned_addr learned my addr 172.21.15.131:0/1314989981 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a9e0c640 1 -- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7a40ced00 msgr2=0x7fc7a4153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a9e0c640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7a40ced00 0x7fc7a4153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a9e0c640 1 -- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7a415c3e0 msgr2=0x7fc7a4154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a9e0c640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7a415c3e0 0x7fc7a4154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a9e0c640 1 -- 172.21.15.131:0/1314989981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7a414ffb0 con 0x7fc7a406e3e0 2024-09-17T02:42:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a9e0c640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7a406e3e0 0x7fc7a414b2a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00c970 tx=0x7fc79c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1314989981 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c004220 con 0x7fc7a406e3e0 2024-09-17T02:42:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1314989981 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc79c0079b0 con 0x7fc7a406e3e0 2024-09-17T02:42:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7ac070640 1 -- 172.21.15.131:0/1314989981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7a4150290 con 0x7fc7a406e3e0 2024-09-17T02:42:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1314989981 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79c0044b0 con 0x7fc7a406e3e0 2024-09-17T02:42:32.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.868+0000 7fc7ac070640 1 -- 172.21.15.131:0/1314989981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7a414b7e0 con 0x7fc7a406e3e0 2024-09-17T02:42:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.872+0000 7fc7ac070640 1 -- 172.21.15.131:0/1314989981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a40b0c10 con 0x7fc7a406e3e0 2024-09-17T02:42:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.872+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1314989981 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc79c005020 con 0x7fc7a406e3e0 2024-09-17T02:42:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.872+0000 7fc7a27fc640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073200 0x7fc790075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:32.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.872+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1314989981 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6274+0+0 (secure 0 0 0) 0x7fc79c094990 con 0x7fc7a406e3e0 2024-09-17T02:42:32.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.872+0000 7fc7a960b640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073200 0x7fc790075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:32.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.876+0000 7fc7a960b640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073200 0x7fc790075710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc7980ae000 tx=0x7fc7980194a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:32.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:32.876+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1314989981 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc79c014030 con 0x7fc7a406e3e0 2024-09-17T02:42:33.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.068+0000 7fc7ac070640 1 -- 172.21.15.131:0/1314989981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc7a406af50 con 0x7fc7a406e3e0 2024-09-17T02:42:33.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.699+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1314989981 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7fc79c05f850 con 0x7fc7a406e3e0 2024-09-17T02:42:33.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 -- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073200 msgr2=0x7fc790075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:33.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073200 0x7fc790075710 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc7980ae000 tx=0x7fc7980194a0 comp rx=0 tx=0).stop 2024-09-17T02:42:33.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 -- 172.21.15.131:0/1314989981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7a406e3e0 msgr2=0x7fc7a414b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:33.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7a406e3e0 0x7fc7a414b2a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00c970 tx=0x7fc79c00ce30 comp rx=0 tx=0).stop 2024-09-17T02:42:33.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 -- 172.21.15.131:0/1314989981 shutdown_connections 2024-09-17T02:42:33.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073200 0x7fc790075710 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:33.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7a415c3e0 0x7fc7a4154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:33.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7a40ced00 0x7fc7a4153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:33.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 --2- 172.21.15.131:0/1314989981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7a406e3e0 0x7fc7a414b2a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:33.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 -- 172.21.15.131:0/1314989981 >> 172.21.15.131:0/1314989981 conn(0x7fc7a406eea0 msgr2=0x7fc7a40b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:33.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 -- 172.21.15.131:0/1314989981 shutdown_connections 2024-09-17T02:42:33.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.703+0000 7fc7ac070640 1 -- 172.21.15.131:0/1314989981 wait complete. 2024-09-17T02:42:33.732 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T02:42:33.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 -- 172.21.15.131:0/4031534494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc0731c0 msgr2=0x7ffacc0735d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:33.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 --2- 172.21.15.131:0/4031534494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc0731c0 0x7ffacc0735d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ffacc070ea0 tx=0x7ffac401cb10 comp rx=0 tx=0).stop 2024-09-17T02:42:33.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 -- 172.21.15.131:0/4031534494 shutdown_connections 2024-09-17T02:42:33.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 --2- 172.21.15.131:0/4031534494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc0731c0 0x7ffacc0735d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:33.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 --2- 172.21.15.131:0/4031534494 >> v2:172.21.15.189:3300/0 conn(0x7ffacc15f060 0x7ffacc15f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:33.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 -- 172.21.15.131:0/4031534494 >> 172.21.15.131:0/4031534494 conn(0x7ffacc0d05a0 msgr2=0x7ffacc0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:33.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 -- 172.21.15.131:0/4031534494 shutdown_connections 2024-09-17T02:42:33.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 -- 172.21.15.131:0/4031534494 wait complete. 2024-09-17T02:42:33.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 Processor -- start 2024-09-17T02:42:33.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 -- start start 2024-09-17T02:42:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffacc0731c0 0x7ffacc1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc1523d0 0x7ffacc158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffacc15f060 0x7ffacc158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc168a80 con 0x7ffacc1523d0 2024-09-17T02:42:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc168900 con 0x7ffacc15f060 2024-09-17T02:42:33.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffad0f8c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ffacc168c00 con 0x7ffacc0731c0 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffac1d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc1523d0 0x7ffacc158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffac1d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc1523d0 0x7ffacc158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43456/0 (socket says 172.21.15.131:43456) 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffac1d9b640 1 -- 172.21.15.131:0/2126058208 learned_addr learned my addr 172.21.15.131:0/2126058208 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffac1d9b640 1 -- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffacc0731c0 msgr2=0x7ffacc1560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffac1d9b640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffacc0731c0 0x7ffacc1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffac1d9b640 1 -- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffacc15f060 msgr2=0x7ffacc158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffac1d9b640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffacc15f060 0x7ffacc158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.883+0000 7ffac1d9b640 1 -- 172.21.15.131:0/2126058208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc1569a0 con 0x7ffacc1523d0 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffac1d9b640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc1523d0 0x7ffacc158400 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffab400c7d0 tx=0x7ffab400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffac37fe640 1 -- 172.21.15.131:0/2126058208 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4007bb0 con 0x7ffacc1523d0 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffac37fe640 1 -- 172.21.15.131:0/2126058208 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffab4007e40 con 0x7ffacc1523d0 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffad0f8c640 1 -- 172.21.15.131:0/2126058208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffacc156c80 con 0x7ffacc1523d0 2024-09-17T02:42:33.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffac37fe640 1 -- 172.21.15.131:0/2126058208 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4005280 con 0x7ffacc1523d0 2024-09-17T02:42:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffad0f8c640 1 -- 172.21.15.131:0/2126058208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffacc157240 con 0x7ffacc1523d0 2024-09-17T02:42:33.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffad0f8c640 1 -- 172.21.15.131:0/2126058208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffacc0735d0 con 0x7ffacc1523d0 2024-09-17T02:42:33.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffac37fe640 1 -- 172.21.15.131:0/2126058208 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffab4005420 con 0x7ffacc1523d0 2024-09-17T02:42:33.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffac37fe640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaac073200 0x7ffaac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:33.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffac37fe640 1 -- 172.21.15.131:0/2126058208 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6274+0+0 (secure 0 0 0) 0x7ffab4094c80 con 0x7ffacc1523d0 2024-09-17T02:42:33.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffaca59c640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaac073200 0x7ffaac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:33.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.887+0000 7ffaca59c640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaac073200 0x7ffaac075710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009670 tx=0x7ffabc006bd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:33.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:33.891+0000 7ffac37fe640 1 -- 172.21.15.131:0/2126058208 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffab4014030 con 0x7ffacc1523d0 2024-09-17T02:42:34.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.047+0000 7ffad0f8c640 1 -- 172.21.15.131:0/2126058208 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffacc147c60 con 0x7ffacc1523d0 2024-09-17T02:42:34.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.711+0000 7ffac37fe640 1 -- 172.21.15.131:0/2126058208 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7ffab405fb40 con 0x7ffacc1523d0 2024-09-17T02:42:34.715 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-09-17T02:42:34.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 -- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaac073200 msgr2=0x7ffaac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaac073200 0x7ffaac075710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffabc009670 tx=0x7ffabc006bd0 comp rx=0 tx=0).stop 2024-09-17T02:42:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 -- 172.21.15.131:0/2126058208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc1523d0 msgr2=0x7ffacc158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc1523d0 0x7ffacc158400 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffab400c7d0 tx=0x7ffab400cc90 comp rx=0 tx=0).stop 2024-09-17T02:42:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 -- 172.21.15.131:0/2126058208 shutdown_connections 2024-09-17T02:42:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffaac073200 0x7ffaac075710 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffacc15f060 0x7ffacc158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc1523d0 0x7ffacc158400 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 --2- 172.21.15.131:0/2126058208 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffacc0731c0 0x7ffacc1560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 -- 172.21.15.131:0/2126058208 >> 172.21.15.131:0/2126058208 conn(0x7ffacc0d05a0 msgr2=0x7ffacc06d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 -- 172.21.15.131:0/2126058208 shutdown_connections 2024-09-17T02:42:34.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.715+0000 7ffad0f8c640 1 -- 172.21.15.131:0/2126058208 wait complete. 2024-09-17T02:42:34.747 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T02:42:34.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.907+0000 7fcfe1703640 1 -- 172.21.15.131:0/2978398032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfc4003a60 msgr2=0x7fcfc4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:34.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.907+0000 7fcfe1703640 1 --2- 172.21.15.131:0/2978398032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfc4003a60 0x7fcfc4003ed0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00bb10 tx=0x7fcfcc01cd70 comp rx=0 tx=0).stop 2024-09-17T02:42:34.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.907+0000 7fcfe1703640 1 -- 172.21.15.131:0/2978398032 shutdown_connections 2024-09-17T02:42:34.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.907+0000 7fcfe1703640 1 --2- 172.21.15.131:0/2978398032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfc4003a60 0x7fcfc4003ed0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:34.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.907+0000 7fcfe1703640 1 -- 172.21.15.131:0/2978398032 >> 172.21.15.131:0/2978398032 conn(0x7fcfdc0b3e50 msgr2=0x7fcfdc0b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:34.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.907+0000 7fcfe1703640 1 -- 172.21.15.131:0/2978398032 shutdown_connections 2024-09-17T02:42:34.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.907+0000 7fcfe1703640 1 -- 172.21.15.131:0/2978398032 wait complete. 2024-09-17T02:42:34.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfe1703640 1 Processor -- start 2024-09-17T02:42:34.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfe1703640 1 -- start start 2024-09-17T02:42:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfe1703640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcfc4003a60 0x7fcfdc0c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfe1703640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfdc1522f0 0x7fcfdc0c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfe1703640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfdc0c6140 0x7fcfdc156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfe1703640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc0cffd0 con 0x7fcfdc1522f0 2024-09-17T02:42:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfe1703640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc0cf450 con 0x7fcfdc0c6140 2024-09-17T02:42:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfe1703640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcfdc0d0150 con 0x7fcfc4003a60 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfda7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfdc1522f0 0x7fcfdc0c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfda7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfdc1522f0 0x7fcfdc0c5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43470/0 (socket says 172.21.15.131:43470) 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfda7fc640 1 -- 172.21.15.131:0/34227410 learned_addr learned my addr 172.21.15.131:0/34227410 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfdb7fe640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfdc0c6140 0x7fcfdc156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfdaffd640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcfc4003a60 0x7fcfdc0c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfda7fc640 1 -- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcfc4003a60 msgr2=0x7fcfdc0c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfda7fc640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcfc4003a60 0x7fcfdc0c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfda7fc640 1 -- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfdc0c6140 msgr2=0x7fcfdc156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfda7fc640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfdc0c6140 0x7fcfdc156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfda7fc640 1 -- 172.21.15.131:0/34227410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc157b60 con 0x7fcfdc1522f0 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfdb7fe640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfdc0c6140 0x7fcfdc156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfda7fc640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfdc1522f0 0x7fcfdc0c5c00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000c760 tx=0x7fcfd000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfc37fe640 1 -- 172.21.15.131:0/34227410 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0017070 con 0x7fcfdc1522f0 2024-09-17T02:42:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfc37fe640 1 -- 172.21.15.131:0/34227410 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfd0005360 con 0x7fcfdc1522f0 2024-09-17T02:42:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfe1703640 1 -- 172.21.15.131:0/34227410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc157e40 con 0x7fcfdc1522f0 2024-09-17T02:42:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfc37fe640 1 -- 172.21.15.131:0/34227410 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000a8a0 con 0x7fcfdc1522f0 2024-09-17T02:42:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.911+0000 7fcfe1703640 1 -- 172.21.15.131:0/34227410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc158400 con 0x7fcfdc1522f0 2024-09-17T02:42:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.915+0000 7fcfe1703640 1 -- 172.21.15.131:0/34227410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfdc0d06b0 con 0x7fcfdc1522f0 2024-09-17T02:42:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.915+0000 7fcfc37fe640 1 -- 172.21.15.131:0/34227410 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcfd001a070 con 0x7fcfdc1522f0 2024-09-17T02:42:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.915+0000 7fcfc37fe640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfc8073200 0x7fcfc8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.915+0000 7fcfc37fe640 1 -- 172.21.15.131:0/34227410 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5836+0+0 (secure 0 0 0) 0x7fcfd0093dd0 con 0x7fcfdc1522f0 2024-09-17T02:42:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.915+0000 7fcfdaffd640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfc8073200 0x7fcfc8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.915+0000 7fcfdaffd640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfc8073200 0x7fcfc8075710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0b0000 tx=0x7fcfcc019b10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:34.919+0000 7fcfc37fe640 1 -- 172.21.15.131:0/34227410 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcfd0021070 con 0x7fcfdc1522f0 2024-09-17T02:42:35.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.079+0000 7fcfe1703640 1 -- 172.21.15.131:0/34227410 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcfdc15c230 con 0x7fcfdc1522f0 2024-09-17T02:42:35.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.719+0000 7fcfc37fe640 1 -- 172.21.15.131:0/34227410 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7fcfd005ee50 con 0x7fcfdc1522f0 2024-09-17T02:42:35.725 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-09-17T02:42:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.723+0000 7fcfe1703640 1 -- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfc8073200 msgr2=0x7fcfc8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.723+0000 7fcfe1703640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfc8073200 0x7fcfc8075710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0b0000 tx=0x7fcfcc019b10 comp rx=0 tx=0).stop 2024-09-17T02:42:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.723+0000 7fcfe1703640 1 -- 172.21.15.131:0/34227410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfdc1522f0 msgr2=0x7fcfdc0c5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:35.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.723+0000 7fcfe1703640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfdc1522f0 0x7fcfdc0c5c00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000c760 tx=0x7fcfd000cc20 comp rx=0 tx=0).stop 2024-09-17T02:42:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.723+0000 7fcfe1703640 1 -- 172.21.15.131:0/34227410 shutdown_connections 2024-09-17T02:42:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.727+0000 7fcfe1703640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfc8073200 0x7fcfc8075710 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.727+0000 7fcfe1703640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfdc0c6140 0x7fcfdc156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.727+0000 7fcfe1703640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfdc1522f0 0x7fcfdc0c5c00 secure :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000c760 tx=0x7fcfd000cc20 comp rx=0 tx=0).stop 2024-09-17T02:42:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.727+0000 7fcfe1703640 1 --2- 172.21.15.131:0/34227410 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcfc4003a60 0x7fcfdc0c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.727+0000 7fcfe1703640 1 -- 172.21.15.131:0/34227410 >> 172.21.15.131:0/34227410 conn(0x7fcfdc0b3e50 msgr2=0x7fcfdc161030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.727+0000 7fcfe1703640 1 -- 172.21.15.131:0/34227410 shutdown_connections 2024-09-17T02:42:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.727+0000 7fcfe1703640 1 -- 172.21.15.131:0/34227410 wait complete. 2024-09-17T02:42:35.753 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T02:42:35.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.911+0000 7ff3d6227640 1 -- 172.21.15.131:0/3334161871 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff3d015c5c0 msgr2=0x7ff3d00cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:35.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.911+0000 7ff3d6227640 1 --2- 172.21.15.131:0/3334161871 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff3d015c5c0 0x7ff3d00cba00 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009840 tx=0x7ff3c001c820 comp rx=0 tx=0).stop 2024-09-17T02:42:35.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.911+0000 7ff3d6227640 1 -- 172.21.15.131:0/3334161871 shutdown_connections 2024-09-17T02:42:35.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.911+0000 7ff3d6227640 1 --2- 172.21.15.131:0/3334161871 >> v2:172.21.15.189:3300/0 conn(0x7ff3d006efb0 0x7ff3d006f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:35.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.911+0000 7ff3d6227640 1 --2- 172.21.15.131:0/3334161871 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff3d015c5c0 0x7ff3d00cba00 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:35.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.911+0000 7ff3d6227640 1 -- 172.21.15.131:0/3334161871 >> 172.21.15.131:0/3334161871 conn(0x7ff3d0073d10 msgr2=0x7ff3d0074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:35.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.911+0000 7ff3d6227640 1 -- 172.21.15.131:0/3334161871 shutdown_connections 2024-09-17T02:42:35.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.911+0000 7ff3d6227640 1 -- 172.21.15.131:0/3334161871 wait complete. 2024-09-17T02:42:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3d6227640 1 Processor -- start 2024-09-17T02:42:35.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3d6227640 1 -- start start 2024-09-17T02:42:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3d6227640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d006ce90 0x7ff3d0155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3d6227640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3d006efb0 0x7ff3d00b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3d6227640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff3d015c5c0 0x7ff3d00c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3d6227640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d016a1d0 con 0x7ff3d006efb0 2024-09-17T02:42:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3d6227640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d016a050 con 0x7ff3d006ce90 2024-09-17T02:42:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3d6227640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d016a350 con 0x7ff3d015c5c0 2024-09-17T02:42:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3ceffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3d006efb0 0x7ff3d00b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:35.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3ceffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3d006efb0 0x7ff3d00b01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43474/0 (socket says 172.21.15.131:43474) 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3ceffd640 1 -- 172.21.15.131:0/1021539220 learned_addr learned my addr 172.21.15.131:0/1021539220 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3cf7fe640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d006ce90 0x7ff3d0155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3cffff640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff3d015c5c0 0x7ff3d00c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3ceffd640 1 -- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff3d015c5c0 msgr2=0x7ff3d00c95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3ceffd640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff3d015c5c0 0x7ff3d00c95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3ceffd640 1 -- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d006ce90 msgr2=0x7ff3d0155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3ceffd640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d006ce90 0x7ff3d0155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3ceffd640 1 -- 172.21.15.131:0/1021539220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d01f7090 con 0x7ff3d006efb0 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.915+0000 7ff3cffff640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff3d015c5c0 0x7ff3d00c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3ceffd640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3d006efb0 0x7ff3d00b01e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400e9f0 tx=0x7ff3c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3b7fff640 1 -- 172.21.15.131:0/1021539220 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c400cd80 con 0x7ff3d006efb0 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3b7fff640 1 -- 172.21.15.131:0/1021539220 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3c4004020 con 0x7ff3d006efb0 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3d6227640 1 -- 172.21.15.131:0/1021539220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d01f7370 con 0x7ff3d006efb0 2024-09-17T02:42:35.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3b7fff640 1 -- 172.21.15.131:0/1021539220 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4004450 con 0x7ff3d006efb0 2024-09-17T02:42:35.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3d6227640 1 -- 172.21.15.131:0/1021539220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d01f7930 con 0x7ff3d006efb0 2024-09-17T02:42:35.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3d6227640 1 -- 172.21.15.131:0/1021539220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d006f3c0 con 0x7ff3d006efb0 2024-09-17T02:42:35.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3b7fff640 1 -- 172.21.15.131:0/1021539220 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff3c40047b0 con 0x7ff3d006efb0 2024-09-17T02:42:35.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3b7fff640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3bc073200 0x7ff3bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:35.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3b7fff640 1 -- 172.21.15.131:0/1021539220 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7ff3c4094680 con 0x7ff3d006efb0 2024-09-17T02:42:35.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3cf7fe640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3bc073200 0x7ff3bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:35.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.919+0000 7ff3cf7fe640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3bc073200 0x7ff3bc075710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00af000 tx=0x7ff3c001b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:35.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:35.923+0000 7ff3b7fff640 1 -- 172.21.15.131:0/1021539220 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff3c405f810 con 0x7ff3d006efb0 2024-09-17T02:42:36.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.111+0000 7ff3d6227640 1 -- 172.21.15.131:0/1021539220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7ff3d0156e10 con 0x7ff3d006efb0 2024-09-17T02:42:36.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.115+0000 7ff3b7fff640 1 -- 172.21.15.131:0/1021539220 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7ff3c405f110 con 0x7ff3d006efb0 2024-09-17T02:42:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 -- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3bc073200 msgr2=0x7ff3bc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3bc073200 0x7ff3bc075710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00af000 tx=0x7ff3c001b060 comp rx=0 tx=0).stop 2024-09-17T02:42:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 -- 172.21.15.131:0/1021539220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3d006efb0 msgr2=0x7ff3d00b01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:36.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3d006efb0 0x7ff3d00b01e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400e9f0 tx=0x7ff3c400eeb0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 -- 172.21.15.131:0/1021539220 shutdown_connections 2024-09-17T02:42:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3bc073200 0x7ff3bc075710 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff3d015c5c0 0x7ff3d00c95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3d006efb0 0x7ff3d00b01e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 --2- 172.21.15.131:0/1021539220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3d006ce90 0x7ff3d0155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 -- 172.21.15.131:0/1021539220 >> 172.21.15.131:0/1021539220 conn(0x7ff3d0073d10 msgr2=0x7ff3d00c4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 -- 172.21.15.131:0/1021539220 shutdown_connections 2024-09-17T02:42:36.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.119+0000 7ff3d6227640 1 -- 172.21.15.131:0/1021539220 wait complete. 2024-09-17T02:42:36.142 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-09-17T02:42:36.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.291+0000 7fdeb854e640 1 -- 172.21.15.131:0/3365175336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde98003a10 msgr2=0x7fde98003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:36.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.291+0000 7fdeb854e640 1 --2- 172.21.15.131:0/3365175336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde98003a10 0x7fde98003e80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdea800c610 tx=0x7fdea801d320 comp rx=0 tx=0).stop 2024-09-17T02:42:36.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.291+0000 7fdeb6aeb640 1 -- 172.21.15.131:0/3365175336 reap_dead start 2024-09-17T02:42:36.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.291+0000 7fdeb854e640 1 -- 172.21.15.131:0/3365175336 shutdown_connections 2024-09-17T02:42:36.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.291+0000 7fdeb854e640 1 -- 172.21.15.131:0/3365175336 >> 172.21.15.131:0/3365175336 conn(0x7fdeb006eea0 msgr2=0x7fdeb006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:36.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.291+0000 7fdeb854e640 1 -- 172.21.15.131:0/3365175336 shutdown_connections 2024-09-17T02:42:36.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.291+0000 7fdeb854e640 1 -- 172.21.15.131:0/3365175336 wait complete. 2024-09-17T02:42:36.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb854e640 1 Processor -- start 2024-09-17T02:42:36.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb854e640 1 -- start start 2024-09-17T02:42:36.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb854e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde98003a10 0x7fdeb01544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:36.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb854e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdeb00cbf90 0x7fdeb0160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:36.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb854e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeb0160750 0x7fdeb00b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:36.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb854e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0167c50 con 0x7fdeb0160750 2024-09-17T02:42:36.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb854e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb0167ad0 con 0x7fde98003a10 2024-09-17T02:42:36.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb854e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdeb0167dd0 con 0x7fdeb00cbf90 2024-09-17T02:42:36.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb6aeb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeb0160750 0x7fdeb00b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb6aeb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeb0160750 0x7fdeb00b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43484/0 (socket says 172.21.15.131:43484) 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb6aeb640 1 -- 172.21.15.131:0/1051941767 learned_addr learned my addr 172.21.15.131:0/1051941767 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb62ea640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde98003a10 0x7fdeb01544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb5ae9640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdeb00cbf90 0x7fdeb0160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb6aeb640 1 -- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdeb00cbf90 msgr2=0x7fdeb0160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb6aeb640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdeb00cbf90 0x7fdeb0160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb6aeb640 1 -- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde98003a10 msgr2=0x7fdeb01544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb6aeb640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde98003a10 0x7fdeb01544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb6aeb640 1 -- 172.21.15.131:0/1051941767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb01f9880 con 0x7fdeb0160750 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb6aeb640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeb0160750 0x7fdeb00b08b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00727c0 tx=0x7fdea000d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdea6ffd640 1 -- 172.21.15.131:0/1051941767 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea001a070 con 0x7fdeb0160750 2024-09-17T02:42:36.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdea6ffd640 1 -- 172.21.15.131:0/1051941767 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdea0004340 con 0x7fdeb0160750 2024-09-17T02:42:36.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdea6ffd640 1 -- 172.21.15.131:0/1051941767 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea000f8f0 con 0x7fdeb0160750 2024-09-17T02:42:36.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb854e640 1 -- 172.21.15.131:0/1051941767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb00caef0 con 0x7fdeb0160750 2024-09-17T02:42:36.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb854e640 1 -- 172.21.15.131:0/1051941767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb00cb4b0 con 0x7fdeb0160750 2024-09-17T02:42:36.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.295+0000 7fdeb854e640 1 -- 172.21.15.131:0/1051941767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb00cac60 con 0x7fdeb0160750 2024-09-17T02:42:36.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.299+0000 7fdea6ffd640 1 -- 172.21.15.131:0/1051941767 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdea000fa90 con 0x7fdeb0160750 2024-09-17T02:42:36.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.299+0000 7fdea6ffd640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde9c073200 0x7fde9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:36.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.299+0000 7fdea6ffd640 1 -- 172.21.15.131:0/1051941767 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fdea0093350 con 0x7fdeb0160750 2024-09-17T02:42:36.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.299+0000 7fdeb62ea640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde9c073200 0x7fde9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:36.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.299+0000 7fdeb62ea640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde9c073200 0x7fde9c075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdea80ab000 tx=0x7fdea801a160 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:36.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.303+0000 7fdea6ffd640 1 -- 172.21.15.131:0/1051941767 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdea005e460 con 0x7fdeb0160750 2024-09-17T02:42:36.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.539+0000 7fdeb854e640 1 -- 172.21.15.131:0/1051941767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fdeb00c50a0 con 0x7fdeb0160750 2024-09-17T02:42:36.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.723+0000 7fdea6ffd640 1 -- 172.21.15.131:0/1051941767 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v108) ==== 142+0+0 (secure 0 0 0) 0x7fdea005dd60 con 0x7fdeb0160750 2024-09-17T02:42:36.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 -- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde9c073200 msgr2=0x7fde9c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:36.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde9c073200 0x7fde9c075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdea80ab000 tx=0x7fdea801a160 comp rx=0 tx=0).stop 2024-09-17T02:42:36.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 -- 172.21.15.131:0/1051941767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeb0160750 msgr2=0x7fdeb00b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:36.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeb0160750 0x7fdeb00b08b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fdeb00727c0 tx=0x7fdea000d8e0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 -- 172.21.15.131:0/1051941767 shutdown_connections 2024-09-17T02:42:36.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde9c073200 0x7fde9c075710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdeb0160750 0x7fdeb00b08b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdeb00cbf90 0x7fdeb0160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 --2- 172.21.15.131:0/1051941767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde98003a10 0x7fdeb01544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 -- 172.21.15.131:0/1051941767 >> 172.21.15.131:0/1051941767 conn(0x7fdeb006eea0 msgr2=0x7fdeb015ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:36.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 -- 172.21.15.131:0/1051941767 shutdown_connections 2024-09-17T02:42:36.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.727+0000 7fdeb854e640 1 -- 172.21.15.131:0/1051941767 wait complete. 2024-09-17T02:42:36.753 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-09-17T02:42:36.753 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-09-17T02:42:36.754 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-17T02:42:36.754 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:42:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.907+0000 7f96f0dc7640 1 -- 172.21.15.131:0/2225859656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0cab60 msgr2=0x7f96ec0caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.907+0000 7f96f0dc7640 1 --2- 172.21.15.131:0/2225859656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0cab60 0x7f96ec0caf70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009bc0 tx=0x7f96e401cb70 comp rx=0 tx=0).stop 2024-09-17T02:42:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.907+0000 7f96f0dc7640 1 -- 172.21.15.131:0/2225859656 shutdown_connections 2024-09-17T02:42:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.907+0000 7f96f0dc7640 1 --2- 172.21.15.131:0/2225859656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0cab60 0x7f96ec0caf70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.907+0000 7f96f0dc7640 1 --2- 172.21.15.131:0/2225859656 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ec06dea0 0x7f96ec06e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.907+0000 7f96f0dc7640 1 -- 172.21.15.131:0/2225859656 >> 172.21.15.131:0/2225859656 conn(0x7f96ec06eea0 msgr2=0x7f96ec06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.907+0000 7f96f0dc7640 1 -- 172.21.15.131:0/2225859656 shutdown_connections 2024-09-17T02:42:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.907+0000 7f96f0dc7640 1 -- 172.21.15.131:0/2225859656 wait complete. 2024-09-17T02:42:36.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96f0dc7640 1 Processor -- start 2024-09-17T02:42:36.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96f0dc7640 1 -- start start 2024-09-17T02:42:36.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96f0dc7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96ec06dea0 0x7f96ec0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:36.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96f0dc7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0cab60 0x7f96ec0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:36.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96f0dc7640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ec155240 0x7f96ec0ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:36.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96f0dc7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec168430 con 0x7f96ec0cab60 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96f0dc7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f96ec1682b0 con 0x7f96ec06dea0 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96f0dc7640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f96ec1685b0 con 0x7f96ec155240 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0cab60 0x7f96ec0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96e9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0cab60 0x7f96ec0cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43498/0 (socket says 172.21.15.131:43498) 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96e9d9b640 1 -- 172.21.15.131:0/571208878 learned_addr learned my addr 172.21.15.131:0/571208878 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96ead9d640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ec155240 0x7f96ec0ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96ea59c640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96ec06dea0 0x7f96ec0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96e9d9b640 1 -- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ec155240 msgr2=0x7f96ec0ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96e9d9b640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ec155240 0x7f96ec0ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96e9d9b640 1 -- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96ec06dea0 msgr2=0x7f96ec0c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96e9d9b640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96ec06dea0 0x7f96ec0c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96e9d9b640 1 -- 172.21.15.131:0/571208878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96ec1fcb80 con 0x7f96ec0cab60 2024-09-17T02:42:36.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96e9d9b640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0cab60 0x7f96ec0cdbf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00ca20 tx=0x7f96dc00cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:36.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96daffd640 1 -- 172.21.15.131:0/571208878 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc007c10 con 0x7f96ec0cab60 2024-09-17T02:42:36.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96daffd640 1 -- 172.21.15.131:0/571208878 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96dc00f450 con 0x7f96ec0cab60 2024-09-17T02:42:36.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96f0dc7640 1 -- 172.21.15.131:0/571208878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96ec140080 con 0x7f96ec0cab60 2024-09-17T02:42:36.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96daffd640 1 -- 172.21.15.131:0/571208878 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96dc00f8a0 con 0x7f96ec0cab60 2024-09-17T02:42:36.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96f0dc7640 1 -- 172.21.15.131:0/571208878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96ec140b60 con 0x7f96ec0cab60 2024-09-17T02:42:36.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.911+0000 7f96f0dc7640 1 -- 172.21.15.131:0/571208878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96ec13fea0 con 0x7f96ec0cab60 2024-09-17T02:42:36.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.915+0000 7f96daffd640 1 -- 172.21.15.131:0/571208878 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f96dc00fa40 con 0x7f96ec0cab60 2024-09-17T02:42:36.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.915+0000 7f96daffd640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f96cc073230 0x7f96cc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:36.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.915+0000 7f96daffd640 1 -- 172.21.15.131:0/571208878 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f96dc094710 con 0x7f96ec0cab60 2024-09-17T02:42:36.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.915+0000 7f96ea59c640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f96cc073230 0x7f96cc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:36.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.915+0000 7f96ea59c640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f96cc073230 0x7f96cc075740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f96e0000f70 tx=0x7f96e0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:36.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:36.919+0000 7f96daffd640 1 -- 172.21.15.131:0/571208878 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f96ec13fea0 con 0x7f96ec0cab60 2024-09-17T02:42:37.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.087+0000 7f96f0dc7640 1 -- 172.21.15.131:0/571208878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96ec06e2b0 con 0x7f96ec0cab60 2024-09-17T02:42:37.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96daffd640 1 -- 172.21.15.131:0/571208878 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+13355 (secure 0 0 0) 0x7f96dc014030 con 0x7f96ec0cab60 2024-09-17T02:42:37.094 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:37.094 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":44,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:42:35.717523+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6833/1415478569":"2024-09-18T02:42:32.697936+0000","172.21.15.131:6832/1415478569":"2024-09-18T02:42:32.697936+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:42:37.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96f0dc7640 1 -- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f96cc073230 msgr2=0x7f96cc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:37.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96f0dc7640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f96cc073230 0x7f96cc075740 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f96e0000f70 tx=0x7f96e0009040 comp rx=0 tx=0).stop 2024-09-17T02:42:37.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96f0dc7640 1 -- 172.21.15.131:0/571208878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0cab60 msgr2=0x7f96ec0cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:37.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96f0dc7640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0cab60 0x7f96ec0cdbf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f96dc00ca20 tx=0x7f96dc00cee0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96f0dc7640 1 -- 172.21.15.131:0/571208878 shutdown_connections 2024-09-17T02:42:37.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96f0dc7640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f96cc073230 0x7f96cc075740 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96f0dc7640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f96ec155240 0x7f96ec0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96f0dc7640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f96ec0cab60 0x7f96ec0cdbf0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96f0dc7640 1 --2- 172.21.15.131:0/571208878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f96ec06dea0 0x7f96ec0c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96f0dc7640 1 -- 172.21.15.131:0/571208878 >> 172.21.15.131:0/571208878 conn(0x7f96ec06eea0 msgr2=0x7f96ec1543f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:37.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.091+0000 7f96f0dc7640 1 -- 172.21.15.131:0/571208878 shutdown_connections 2024-09-17T02:42:37.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.095+0000 7f96f0dc7640 1 -- 172.21.15.131:0/571208878 wait complete. 2024-09-17T02:42:37.117 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T02:40:41.163614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T02:42:37.117 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T02:42:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.271+0000 7f1d6cd38640 1 -- 172.21.15.131:0/1849117527 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1d6815c740 msgr2=0x7f1d680d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.271+0000 7f1d6cd38640 1 --2- 172.21.15.131:0/1849117527 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1d6815c740 0x7f1d680d0130 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009840 tx=0x7f1d5801c7f0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.271+0000 7f1d6cd38640 1 -- 172.21.15.131:0/1849117527 shutdown_connections 2024-09-17T02:42:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.271+0000 7f1d6cd38640 1 --2- 172.21.15.131:0/1849117527 >> v2:172.21.15.189:3300/0 conn(0x7f1d680cca90 0x7f1d680ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.271+0000 7f1d6cd38640 1 --2- 172.21.15.131:0/1849117527 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1d6815c740 0x7f1d680d0130 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.271+0000 7f1d6cd38640 1 -- 172.21.15.131:0/1849117527 >> 172.21.15.131:0/1849117527 conn(0x7f1d680cf0b0 msgr2=0x7f1d680cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.271+0000 7f1d6cd38640 1 -- 172.21.15.131:0/1849117527 shutdown_connections 2024-09-17T02:42:37.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.271+0000 7f1d6cd38640 1 -- 172.21.15.131:0/1849117527 wait complete. 2024-09-17T02:42:37.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.275+0000 7f1d6cd38640 1 Processor -- start 2024-09-17T02:42:37.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.275+0000 7f1d6cd38640 1 -- start start 2024-09-17T02:42:37.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.275+0000 7f1d6cd38640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1d680706c0 0x7f1d680c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:37.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.275+0000 7f1d6cd38640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d680cca90 0x7f1d68159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:37.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.275+0000 7f1d6cd38640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d6815c740 0x7f1d680c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:37.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.275+0000 7f1d6cd38640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6816a160 con 0x7f1d680cca90 2024-09-17T02:42:37.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.275+0000 7f1d6cd38640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1d68169fe0 con 0x7f1d6815c740 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.275+0000 7f1d6cd38640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6816a2e0 con 0x7f1d680706c0 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.275+0000 7f1d65d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d680cca90 0x7f1d68159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.275+0000 7f1d65d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d680cca90 0x7f1d68159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43508/0 (socket says 172.21.15.131:43508) 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.275+0000 7f1d65d9b640 1 -- 172.21.15.131:0/401243453 learned_addr learned my addr 172.21.15.131:0/401243453 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d6659c640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1d680706c0 0x7f1d680c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d66d9d640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d6815c740 0x7f1d680c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d65d9b640 1 -- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1d680706c0 msgr2=0x7f1d680c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d65d9b640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1d680706c0 0x7f1d680c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d65d9b640 1 -- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d6815c740 msgr2=0x7f1d680c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d65d9b640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d6815c740 0x7f1d680c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d65d9b640 1 -- 172.21.15.131:0/401243453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d680b81b0 con 0x7f1d680cca90 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d6659c640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1d680706c0 0x7f1d680c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:42:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d65d9b640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d680cca90 0x7f1d68159e10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00ea20 tx=0x7f1d5c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:37.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d56ffd640 1 -- 172.21.15.131:0/401243453 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c00cd80 con 0x7f1d680cca90 2024-09-17T02:42:37.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d56ffd640 1 -- 172.21.15.131:0/401243453 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d5c004020 con 0x7f1d680cca90 2024-09-17T02:42:37.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d56ffd640 1 -- 172.21.15.131:0/401243453 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c004450 con 0x7f1d680cca90 2024-09-17T02:42:37.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d6cd38640 1 -- 172.21.15.131:0/401243453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6815a780 con 0x7f1d680cca90 2024-09-17T02:42:37.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d6cd38640 1 -- 172.21.15.131:0/401243453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6815ad40 con 0x7f1d680cca90 2024-09-17T02:42:37.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d6cd38640 1 -- 172.21.15.131:0/401243453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6815a4f0 con 0x7f1d680cca90 2024-09-17T02:42:37.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d56ffd640 1 -- 172.21.15.131:0/401243453 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1d5c0047b0 con 0x7f1d680cca90 2024-09-17T02:42:37.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d56ffd640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d48073200 0x7f1d48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:37.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d56ffd640 1 -- 172.21.15.131:0/401243453 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f1d5c0946c0 con 0x7f1d680cca90 2024-09-17T02:42:37.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.279+0000 7f1d6659c640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d48073200 0x7f1d48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:37.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.283+0000 7f1d6659c640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d48073200 0x7f1d48075710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009840 tx=0x7f1d5801b030 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:37.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.287+0000 7f1d56ffd640 1 -- 172.21.15.131:0/401243453 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1d5c05f8e0 con 0x7f1d680cca90 2024-09-17T02:42:37.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.451+0000 7f1d6cd38640 1 -- 172.21.15.131:0/401243453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1d680ccea0 con 0x7f1d680cca90 2024-09-17T02:42:37.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.451+0000 7f1d56ffd640 1 -- 172.21.15.131:0/401243453 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) ==== 93+0+10 (secure 0 0 0) 0x7f1d5c05f1e0 con 0x7f1d680cca90 2024-09-17T02:42:37.454 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-09-17T02:42:37.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 -- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d48073200 msgr2=0x7f1d48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:37.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d48073200 0x7f1d48075710 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009840 tx=0x7f1d5801b030 comp rx=0 tx=0).stop 2024-09-17T02:42:37.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 -- 172.21.15.131:0/401243453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d680cca90 msgr2=0x7f1d68159e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:37.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d680cca90 0x7f1d68159e10 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00ea20 tx=0x7f1d5c00eee0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 -- 172.21.15.131:0/401243453 shutdown_connections 2024-09-17T02:42:37.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d48073200 0x7f1d48075710 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d6815c740 0x7f1d680c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d680cca90 0x7f1d68159e10 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 --2- 172.21.15.131:0/401243453 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1d680706c0 0x7f1d680c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 -- 172.21.15.131:0/401243453 >> 172.21.15.131:0/401243453 conn(0x7f1d680cf0b0 msgr2=0x7f1d68152100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:37.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 -- 172.21.15.131:0/401243453 shutdown_connections 2024-09-17T02:42:37.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.455+0000 7f1d6cd38640 1 -- 172.21.15.131:0/401243453 wait complete. 2024-09-17T02:42:37.476 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-09-17T02:42:37.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 -- 172.21.15.131:0/989051264 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f849c0cab60 msgr2=0x7f849c0caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:37.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 --2- 172.21.15.131:0/989051264 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f849c0cab60 0x7f849c0caf70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f848c009840 tx=0x7f848c01c7f0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 -- 172.21.15.131:0/989051264 shutdown_connections 2024-09-17T02:42:37.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 --2- 172.21.15.131:0/989051264 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f849c0cab60 0x7f849c0caf70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 --2- 172.21.15.131:0/989051264 >> v2:172.21.15.189:3300/0 conn(0x7f849c06dea0 0x7f849c06e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 -- 172.21.15.131:0/989051264 >> 172.21.15.131:0/989051264 conn(0x7f849c06eea0 msgr2=0x7f849c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:37.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 -- 172.21.15.131:0/989051264 shutdown_connections 2024-09-17T02:42:37.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 -- 172.21.15.131:0/989051264 wait complete. 2024-09-17T02:42:37.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 Processor -- start 2024-09-17T02:42:37.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 -- start start 2024-09-17T02:42:37.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f849c06dea0 0x7f849c14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:37.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f849c0cab60 0x7f849c1f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:37.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f849c155240 0x7f849c1f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:37.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f849c0cbec0 con 0x7f849c06dea0 2024-09-17T02:42:37.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f849c0cbd40 con 0x7f849c155240 2024-09-17T02:42:37.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f84a1c7a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f849c168c80 con 0x7f849c0cab60 2024-09-17T02:42:37.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f849b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f849c06dea0 0x7f849c14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:37.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f849b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f849c06dea0 0x7f849c14ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43510/0 (socket says 172.21.15.131:43510) 2024-09-17T02:42:37.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f849b7fe640 1 -- 172.21.15.131:0/1482765146 learned_addr learned my addr 172.21.15.131:0/1482765146 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f849bfff640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f849c155240 0x7f849c1f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.615+0000 7f849affd640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f849c0cab60 0x7f849c1f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f849b7fe640 1 -- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f849c0cab60 msgr2=0x7f849c1f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f849b7fe640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f849c0cab60 0x7f849c1f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f849b7fe640 1 -- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f849c155240 msgr2=0x7f849c1f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f849b7fe640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f849c155240 0x7f849c1f7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f849b7fe640 1 -- 172.21.15.131:0/1482765146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849c1558a0 con 0x7f849c06dea0 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f849b7fe640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f849c06dea0 0x7f849c14ffd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f849400c760 tx=0x7f849400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f8493fff640 1 -- 172.21.15.131:0/1482765146 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494006aa0 con 0x7f849c06dea0 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f8493fff640 1 -- 172.21.15.131:0/1482765146 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8494006c40 con 0x7f849c06dea0 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f84a1c7a640 1 -- 172.21.15.131:0/1482765146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849c155da0 con 0x7f849c06dea0 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f8493fff640 1 -- 172.21.15.131:0/1482765146 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8494007290 con 0x7f849c06dea0 2024-09-17T02:42:37.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f84a1c7a640 1 -- 172.21.15.131:0/1482765146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f849c1f6770 con 0x7f849c06dea0 2024-09-17T02:42:37.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f84a1c7a640 1 -- 172.21.15.131:0/1482765146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f849c06e2b0 con 0x7f849c06dea0 2024-09-17T02:42:37.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f8493fff640 1 -- 172.21.15.131:0/1482765146 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8494007430 con 0x7f849c06dea0 2024-09-17T02:42:37.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f8493fff640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f847c073230 0x7f847c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:37.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f8493fff640 1 -- 172.21.15.131:0/1482765146 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f8494093b80 con 0x7f849c06dea0 2024-09-17T02:42:37.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f849affd640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f847c073230 0x7f847c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:37.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.619+0000 7f849affd640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f847c073230 0x7f847c075740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f849c15a140 tx=0x7f8484009500 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:37.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.623+0000 7f8493fff640 1 -- 172.21.15.131:0/1482765146 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f849c06e7f0 con 0x7f849c06dea0 2024-09-17T02:42:37.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:37.855+0000 7f84a1c7a640 1 -- 172.21.15.131:0/1482765146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f849c06e9d0 con 0x7f849c06dea0 2024-09-17T02:42:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.735+0000 7f8493fff640 1 -- 172.21.15.131:0/1482765146 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v110) ==== 152+0+0 (secure 0 0 0) 0x7f849405ec90 con 0x7f849c06dea0 2024-09-17T02:42:38.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 -- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f847c073230 msgr2=0x7f847c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:38.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f847c073230 0x7f847c075740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f849c15a140 tx=0x7f8484009500 comp rx=0 tx=0).stop 2024-09-17T02:42:38.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 -- 172.21.15.131:0/1482765146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f849c06dea0 msgr2=0x7f849c14ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:38.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f849c06dea0 0x7f849c14ffd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f849400c760 tx=0x7f849400cc20 comp rx=0 tx=0).stop 2024-09-17T02:42:38.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 -- 172.21.15.131:0/1482765146 shutdown_connections 2024-09-17T02:42:38.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f847c073230 0x7f847c075740 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:38.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f849c155240 0x7f849c1f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:38.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f849c0cab60 0x7f849c1f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:38.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 --2- 172.21.15.131:0/1482765146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f849c06dea0 0x7f849c14ffd0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:38.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 -- 172.21.15.131:0/1482765146 >> 172.21.15.131:0/1482765146 conn(0x7f849c06eea0 msgr2=0x7f849c0733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:38.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 -- 172.21.15.131:0/1482765146 shutdown_connections 2024-09-17T02:42:38.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.739+0000 7f84a1c7a640 1 -- 172.21.15.131:0/1482765146 wait complete. 2024-09-17T02:42:38.764 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T02:42:38.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.911+0000 7f399572c640 1 -- 172.21.15.131:0/3826899584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399006e3e0 msgr2=0x7f399015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:38.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.911+0000 7f399572c640 1 --2- 172.21.15.131:0/3826899584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399006e3e0 0x7f399015bea0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f399006a1c0 tx=0x7f397801c8c0 comp rx=0 tx=0).stop 2024-09-17T02:42:38.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.911+0000 7f399572c640 1 -- 172.21.15.131:0/3826899584 shutdown_connections 2024-09-17T02:42:38.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.911+0000 7f399572c640 1 --2- 172.21.15.131:0/3826899584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f399006e3e0 0x7f399015bea0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:38.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.911+0000 7f399572c640 1 --2- 172.21.15.131:0/3826899584 >> v2:172.21.15.189:3300/0 conn(0x7f39900ced00 0x7f399006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:38.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.911+0000 7f399572c640 1 -- 172.21.15.131:0/3826899584 >> 172.21.15.131:0/3826899584 conn(0x7f399006eea0 msgr2=0x7f399006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:38.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.911+0000 7f399572c640 1 -- 172.21.15.131:0/3826899584 shutdown_connections 2024-09-17T02:42:38.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.911+0000 7f399572c640 1 -- 172.21.15.131:0/3826899584 wait complete. 2024-09-17T02:42:38.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.911+0000 7f399572c640 1 Processor -- start 2024-09-17T02:42:38.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.911+0000 7f399572c640 1 -- start start 2024-09-17T02:42:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f399572c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f399006e3e0 0x7f399015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f399572c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39900ced00 0x7f39900a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f399572c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f399015c3e0 0x7f3990153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f399572c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3990168b60 con 0x7f39900ced00 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f399572c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f39901689e0 con 0x7f399006e3e0 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f399572c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3990168ce0 con 0x7f399015c3e0 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39900ced00 0x7f39900a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39900ced00 0x7f39900a95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43528/0 (socket says 172.21.15.131:43528) 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398e7fc640 1 -- 172.21.15.131:0/1332095717 learned_addr learned my addr 172.21.15.131:0/1332095717 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398e7fc640 1 -- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f399015c3e0 msgr2=0x7f3990153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398effd640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f399006e3e0 0x7f399015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398f7fe640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f399015c3e0 0x7f3990153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398e7fc640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f399015c3e0 0x7f3990153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398e7fc640 1 -- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f399006e3e0 msgr2=0x7f399015dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398e7fc640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f399006e3e0 0x7f399015dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398e7fc640 1 -- 172.21.15.131:0/1332095717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39901f8630 con 0x7f39900ced00 2024-09-17T02:42:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398effd640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f399006e3e0 0x7f399015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398f7fe640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f399015c3e0 0x7f3990153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f398e7fc640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39900ced00 0x7f39900a95c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f39780b0040 tx=0x7f39780a5780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f39877fe640 1 -- 172.21.15.131:0/1332095717 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39780b2040 con 0x7f39900ced00 2024-09-17T02:42:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f39877fe640 1 -- 172.21.15.131:0/1332095717 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3978004430 con 0x7f39900ced00 2024-09-17T02:42:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f399572c640 1 -- 172.21.15.131:0/1332095717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39901fbc20 con 0x7f39900ced00 2024-09-17T02:42:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f39877fe640 1 -- 172.21.15.131:0/1332095717 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978005280 con 0x7f39900ced00 2024-09-17T02:42:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f399572c640 1 -- 172.21.15.131:0/1332095717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39901fc400 con 0x7f39900ced00 2024-09-17T02:42:38.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f399572c640 1 -- 172.21.15.131:0/1332095717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39901fbe90 con 0x7f39900ced00 2024-09-17T02:42:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f39877fe640 1 -- 172.21.15.131:0/1332095717 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3978005420 con 0x7f39900ced00 2024-09-17T02:42:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f39877fe640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3970073230 0x7f3970075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:38.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.915+0000 7f39877fe640 1 -- 172.21.15.131:0/1332095717 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f397812e4f0 con 0x7f39900ced00 2024-09-17T02:42:38.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.919+0000 7f398effd640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3970073230 0x7f3970075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:38.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.919+0000 7f398effd640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3970073230 0x7f3970075740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3980009d90 tx=0x7f3980009320 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:38.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:38.923+0000 7f39877fe640 1 -- 172.21.15.131:0/1332095717 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f39901f8630 con 0x7f39900ced00 2024-09-17T02:42:39.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.087+0000 7f399572c640 1 -- 172.21.15.131:0/1332095717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f399015bea0 con 0x7f39900ced00 2024-09-17T02:42:39.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f39877fe640 1 -- 172.21.15.131:0/1332095717 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+2294 (secure 0 0 0) 0x7f39780164b0 con 0x7f39900ced00 2024-09-17T02:42:39.093 INFO:teuthology.orchestra.run.smithi131.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T02:42:39.093 INFO:teuthology.orchestra.run.smithi131.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T02:42:39.093 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced client_mount_timeout 30 2024-09-17T02:42:39.093 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_client 20/20 2024-09-17T02:42:39.093 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_mgr 20/20 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_ms 1/1 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3855.598857 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3598.912856 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3977.024117 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3823.557728 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4643.710284 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4452.598518 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6659.946233 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4539.778076 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds 20/20 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T02:42:39.094 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_ms 2/2 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_frag true 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_scatterstat true 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_verify_scatter true 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced client_mount_timeout 600 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_client 20/20 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_ms 1/1 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T02:42:39.095 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T02:42:39.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 -- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3970073230 msgr2=0x7f3970075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:39.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3970073230 0x7f3970075740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3980009d90 tx=0x7f3980009320 comp rx=0 tx=0).stop 2024-09-17T02:42:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 -- 172.21.15.131:0/1332095717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39900ced00 msgr2=0x7f39900a95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39900ced00 0x7f39900a95c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f39780b0040 tx=0x7f39780a5780 comp rx=0 tx=0).stop 2024-09-17T02:42:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 -- 172.21.15.131:0/1332095717 shutdown_connections 2024-09-17T02:42:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3970073230 0x7f3970075740 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f399015c3e0 0x7f3990153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f39900ced00 0x7f39900a95c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 --2- 172.21.15.131:0/1332095717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f399006e3e0 0x7f399015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 -- 172.21.15.131:0/1332095717 >> 172.21.15.131:0/1332095717 conn(0x7f399006eea0 msgr2=0x7f39900b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 -- 172.21.15.131:0/1332095717 shutdown_connections 2024-09-17T02:42:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.091+0000 7f399572c640 1 -- 172.21.15.131:0/1332095717 wait complete. 2024-09-17T02:42:39.114 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T02:42:39.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.259+0000 7f6bca3b5640 1 -- 172.21.15.131:0/1636757321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bac005360 msgr2=0x7f6bac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:39.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.259+0000 7f6bca3b5640 1 --2- 172.21.15.131:0/1636757321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bac005360 0x7f6bac00b9d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400ac50 tx=0x7f6bb401e0b0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.259+0000 7f6bca3b5640 1 -- 172.21.15.131:0/1636757321 shutdown_connections 2024-09-17T02:42:39.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.259+0000 7f6bca3b5640 1 --2- 172.21.15.131:0/1636757321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bac005360 0x7f6bac00b9d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.259+0000 7f6bca3b5640 1 -- 172.21.15.131:0/1636757321 >> 172.21.15.131:0/1636757321 conn(0x7f6bc406eea0 msgr2=0x7f6bc406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:39.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.259+0000 7f6bca3b5640 1 -- 172.21.15.131:0/1636757321 shutdown_connections 2024-09-17T02:42:39.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.259+0000 7f6bca3b5640 1 -- 172.21.15.131:0/1636757321 wait complete. 2024-09-17T02:42:39.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.259+0000 7f6bca3b5640 1 Processor -- start 2024-09-17T02:42:39.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.259+0000 7f6bca3b5640 1 -- start start 2024-09-17T02:42:39.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bca3b5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bac005360 0x7f6bc40b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:39.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bca3b5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc4153b50 0x7f6bc41533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:39.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bca3b5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6bc4158f20 0x7f6bc4159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:39.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bca3b5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4155120 con 0x7f6bac005360 2024-09-17T02:42:39.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bca3b5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4154fa0 con 0x7f6bc4153b50 2024-09-17T02:42:39.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bca3b5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc41552a0 con 0x7f6bc4158f20 2024-09-17T02:42:39.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bac005360 0x7f6bc40b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:39.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bac005360 0x7f6bc40b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43550/0 (socket says 172.21.15.131:43550) 2024-09-17T02:42:39.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc3fff640 1 -- 172.21.15.131:0/2386407996 learned_addr learned my addr 172.21.15.131:0/2386407996 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:39.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc37fe640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc4153b50 0x7f6bc41533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:39.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc8952640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6bc4158f20 0x7f6bc4159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:39.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc3fff640 1 -- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6bc4158f20 msgr2=0x7f6bc4159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:39.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc3fff640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6bc4158f20 0x7f6bc4159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc3fff640 1 -- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc4153b50 msgr2=0x7f6bc41533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:39.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc3fff640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc4153b50 0x7f6bc41533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc3fff640 1 -- 172.21.15.131:0/2386407996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc4159870 con 0x7f6bac005360 2024-09-17T02:42:39.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc37fe640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc4153b50 0x7f6bc41533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:39.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc8952640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6bc4158f20 0x7f6bc4159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:39.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc3fff640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bac005360 0x7f6bc40b5570 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00d920 tx=0x7f6bbc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:39.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc0ff9640 1 -- 172.21.15.131:0/2386407996 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc016020 con 0x7f6bac005360 2024-09-17T02:42:39.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc0ff9640 1 -- 172.21.15.131:0/2386407996 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bbc00b990 con 0x7f6bac005360 2024-09-17T02:42:39.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bca3b5640 1 -- 172.21.15.131:0/2386407996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc40b3fe0 con 0x7f6bac005360 2024-09-17T02:42:39.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc0ff9640 1 -- 172.21.15.131:0/2386407996 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00bdc0 con 0x7f6bac005360 2024-09-17T02:42:39.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bca3b5640 1 -- 172.21.15.131:0/2386407996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc40b4280 con 0x7f6bac005360 2024-09-17T02:42:39.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bca3b5640 1 -- 172.21.15.131:0/2386407996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc406e220 con 0x7f6bac005360 2024-09-17T02:42:39.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc0ff9640 1 -- 172.21.15.131:0/2386407996 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6bbc019020 con 0x7f6bac005360 2024-09-17T02:42:39.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc0ff9640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bb0073230 0x7f6bb0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:39.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc0ff9640 1 -- 172.21.15.131:0/2386407996 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f6bbc093e90 con 0x7f6bac005360 2024-09-17T02:42:39.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.263+0000 7f6bc37fe640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bb0073230 0x7f6bb0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:39.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.267+0000 7f6bc37fe640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bb0073230 0x7f6bb0075740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8001870 tx=0x7f6bb8001120 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:39.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.271+0000 7f6bc0ff9640 1 -- 172.21.15.131:0/2386407996 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6bbc020030 con 0x7f6bac005360 2024-09-17T02:42:39.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.427+0000 7f6bca3b5640 1 -- 172.21.15.131:0/2386407996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6bc406fa20 con 0x7f6bac005360 2024-09-17T02:42:39.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.431+0000 7f6bc0ff9640 1 -- 172.21.15.131:0/2386407996 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f6bc4159870 con 0x7f6bac005360 2024-09-17T02:42:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 -- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bb0073230 msgr2=0x7f6bb0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:39.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bb0073230 0x7f6bb0075740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8001870 tx=0x7f6bb8001120 comp rx=0 tx=0).stop 2024-09-17T02:42:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 -- 172.21.15.131:0/2386407996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bac005360 msgr2=0x7f6bc40b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bac005360 0x7f6bc40b5570 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00d920 tx=0x7f6bbc00dde0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 -- 172.21.15.131:0/2386407996 shutdown_connections 2024-09-17T02:42:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bb0073230 0x7f6bb0075740 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6bc4158f20 0x7f6bc4159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6bc4153b50 0x7f6bc41533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 --2- 172.21.15.131:0/2386407996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6bac005360 0x7f6bc40b5570 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 -- 172.21.15.131:0/2386407996 >> 172.21.15.131:0/2386407996 conn(0x7f6bc406eea0 msgr2=0x7f6bc40cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 -- 172.21.15.131:0/2386407996 shutdown_connections 2024-09-17T02:42:39.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.435+0000 7f6bca3b5640 1 -- 172.21.15.131:0/2386407996 wait complete. 2024-09-17T02:42:39.459 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:42:39.460 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:42:39.542 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:42:39.606 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:42:39.606 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:42:39.606 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:42:39.613 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:42:39.613 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:42:39.613 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:42:39.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:42:39.619 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:42:39.619 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:42:39.661 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:42:39.662 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:42:39.662 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:42:39.669 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:42:39.669 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T02:42:39.669 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:39.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.863+0000 7faefe7ae640 1 -- 172.21.15.131:0/4143411824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef80c4c90 msgr2=0x7faef80c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:39.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.863+0000 7faefe7ae640 1 --2- 172.21.15.131:0/4143411824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef80c4c90 0x7faef80c50a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7faef0009bc0 tx=0x7faef001cb10 comp rx=0 tx=0).stop 2024-09-17T02:42:39.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.863+0000 7faefe7ae640 1 -- 172.21.15.131:0/4143411824 shutdown_connections 2024-09-17T02:42:39.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.863+0000 7faefe7ae640 1 --2- 172.21.15.131:0/4143411824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef80c4c90 0x7faef80c50a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.863+0000 7faefe7ae640 1 --2- 172.21.15.131:0/4143411824 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faef80cec00 0x7faef80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.863+0000 7faefe7ae640 1 -- 172.21.15.131:0/4143411824 >> 172.21.15.131:0/4143411824 conn(0x7faef806eea0 msgr2=0x7faef806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:39.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.863+0000 7faefe7ae640 1 -- 172.21.15.131:0/4143411824 shutdown_connections 2024-09-17T02:42:39.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.863+0000 7faefe7ae640 1 -- 172.21.15.131:0/4143411824 wait complete. 2024-09-17T02:42:39.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefe7ae640 1 Processor -- start 2024-09-17T02:42:39.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefe7ae640 1 -- start start 2024-09-17T02:42:39.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefe7ae640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faef806e2c0 0x7faef8151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:39.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefe7ae640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef80c4c90 0x7faef815f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:39.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefe7ae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faef80cec00 0x7faef815f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:39.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefe7ae640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faef8169e70 con 0x7faef80c4c90 2024-09-17T02:42:39.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefe7ae640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faef8169cf0 con 0x7faef80cec00 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefe7ae640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7faef8169ff0 con 0x7faef806e2c0 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef80c4c90 0x7faef815f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef80c4c90 0x7faef815f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43558/0 (socket says 172.21.15.131:43558) 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef77fe640 1 -- 172.21.15.131:0/2037042352 learned_addr learned my addr 172.21.15.131:0/2037042352 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefcd4b640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faef80cec00 0x7faef815f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef7fff640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faef806e2c0 0x7faef8151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef77fe640 1 -- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faef806e2c0 msgr2=0x7faef8151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef77fe640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faef806e2c0 0x7faef8151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef77fe640 1 -- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faef80cec00 msgr2=0x7faef815f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef77fe640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faef80cec00 0x7faef815f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef77fe640 1 -- 172.21.15.131:0/2037042352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef815dc70 con 0x7faef80c4c90 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef77fe640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef80c4c90 0x7faef815f2d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c710 tx=0x7faeec00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef4ff9640 1 -- 172.21.15.131:0/2037042352 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec007bb0 con 0x7faef80c4c90 2024-09-17T02:42:39.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef4ff9640 1 -- 172.21.15.131:0/2037042352 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faeec007e40 con 0x7faef80c4c90 2024-09-17T02:42:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef4ff9640 1 -- 172.21.15.131:0/2037042352 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec005280 con 0x7faef80c4c90 2024-09-17T02:42:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefe7ae640 1 -- 172.21.15.131:0/2037042352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef8152300 con 0x7faef80c4c90 2024-09-17T02:42:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefe7ae640 1 -- 172.21.15.131:0/2037042352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef815e1c0 con 0x7faef80c4c90 2024-09-17T02:42:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faefe7ae640 1 -- 172.21.15.131:0/2037042352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef81550d0 con 0x7faef80c4c90 2024-09-17T02:42:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.867+0000 7faef4ff9640 1 -- 172.21.15.131:0/2037042352 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faeec005420 con 0x7faef80c4c90 2024-09-17T02:42:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.871+0000 7faef4ff9640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faee4073200 0x7faee4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.871+0000 7faef4ff9640 1 -- 172.21.15.131:0/2037042352 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7faeec0944c0 con 0x7faef80c4c90 2024-09-17T02:42:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.871+0000 7faef7fff640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faee4073200 0x7faee4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:39.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.871+0000 7faef7fff640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faee4073200 0x7faee4075710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faee8000f70 tx=0x7faee8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:39.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:39.875+0000 7faef4ff9640 1 -- 172.21.15.131:0/2037042352 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faef8152300 con 0x7faef80c4c90 2024-09-17T02:42:40.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.071+0000 7faefe7ae640 1 -- 172.21.15.131:0/2037042352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faef80cf010 con 0x7faef80c4c90 2024-09-17T02:42:40.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.071+0000 7faef4ff9640 1 -- 172.21.15.131:0/2037042352 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3920 (secure 0 0 0) 0x7faeec014030 con 0x7faef80c4c90 2024-09-17T02:42:40.077 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:40.077 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":20,"btime":"2024-09-17T02:42:33:701021+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3342719482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3342719482},{"type":"v1","addr":"172.21.15.131:6835","nonce":3342719482}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4804,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/3070378350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3070378350},{"type":"v1","addr":"172.21.15.189:6836","nonce":3070378350}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/575920659","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":575920659},{"type":"v1","addr":"172.21.15.189:6837","nonce":575920659}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5370,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2837315235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2837315235},{"type":"v1","addr":"172.21.15.131:6833","nonce":2837315235}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-17T02:42:40.077 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 20 2024-09-17T02:42:40.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 -- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faee4073200 msgr2=0x7faee4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:40.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faee4073200 0x7faee4075710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7faee8000f70 tx=0x7faee8009040 comp rx=0 tx=0).stop 2024-09-17T02:42:40.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 -- 172.21.15.131:0/2037042352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef80c4c90 msgr2=0x7faef815f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:40.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef80c4c90 0x7faef815f2d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c710 tx=0x7faeec00cbd0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 -- 172.21.15.131:0/2037042352 shutdown_connections 2024-09-17T02:42:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faee4073200 0x7faee4075710 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faef80cec00 0x7faef815f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faef80c4c90 0x7faef815f2d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 --2- 172.21.15.131:0/2037042352 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faef806e2c0 0x7faef8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 -- 172.21.15.131:0/2037042352 >> 172.21.15.131:0/2037042352 conn(0x7faef806eea0 msgr2=0x7faef80cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 -- 172.21.15.131:0/2037042352 shutdown_connections 2024-09-17T02:42:40.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.075+0000 7faefe7ae640 1 -- 172.21.15.131:0/2037042352 wait complete. 2024-09-17T02:42:40.097 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T02:42:40.097 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T02:42:40.097 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T02:42:40.097 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:42:40.097 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T02:42:40.098 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T02:42:40.098 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T02:42:40.098 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:42:40.098 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T02:42:40.098 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T02:42:40.098 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T02:42:40.099 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:42:40.099 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T02:42:40.099 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T02:42:40.099 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T02:42:40.099 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:42:40.110 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T02:42:40.110 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T02:42:40.113 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T02:42:40.113 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T02:42:40.125 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T02:42:40.126 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T02:42:40.126 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T02:42:40.127 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T02:42:40.157 INFO:tasks.ceph.mds.a:Started 2024-09-17T02:42:40.158 DEBUG:teuthology.parallel:result is None 2024-09-17T02:42:40.158 INFO:tasks.ceph.mds.c:Started 2024-09-17T02:42:40.159 DEBUG:teuthology.parallel:result is None 2024-09-17T02:42:40.174 INFO:tasks.ceph.mds.d:Started 2024-09-17T02:42:40.175 DEBUG:teuthology.parallel:result is None 2024-09-17T02:42:40.176 INFO:tasks.ceph.mds.b:Started 2024-09-17T02:42:40.176 DEBUG:teuthology.parallel:result is None 2024-09-17T02:42:40.176 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T02:42:40.245 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:42:40.249 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:42:40.277 INFO:tasks.ceph.mds.d.smithi189.stdout:starting mds.d at 2024-09-17T02:42:40.280 INFO:tasks.ceph.mds.b.smithi189.stdout:starting mds.b at 2024-09-17T02:42:40.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.319+0000 7f2ef42c4640 1 -- 172.21.15.131:0/1610189855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8005260 msgr2=0x7f2ed800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:40.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.319+0000 7f2ef42c4640 1 --2- 172.21.15.131:0/1610189855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8005260 0x7f2ed800b9d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400c470 tx=0x7f2ee401cf60 comp rx=0 tx=0).stop 2024-09-17T02:42:40.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.319+0000 7f2ef42c4640 1 -- 172.21.15.131:0/1610189855 shutdown_connections 2024-09-17T02:42:40.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.319+0000 7f2ef42c4640 1 --2- 172.21.15.131:0/1610189855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8005260 0x7f2ed800b9d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.319+0000 7f2ef42c4640 1 -- 172.21.15.131:0/1610189855 >> 172.21.15.131:0/1610189855 conn(0x7f2eec0d1130 msgr2=0x7f2eec0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:40.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.319+0000 7f2ef42c4640 1 -- 172.21.15.131:0/1610189855 shutdown_connections 2024-09-17T02:42:40.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.319+0000 7f2ef42c4640 1 -- 172.21.15.131:0/1610189855 wait complete. 2024-09-17T02:42:40.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef42c4640 1 Processor -- start 2024-09-17T02:42:40.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef42c4640 1 -- start start 2024-09-17T02:42:40.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef42c4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8005260 0x7f2eec0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef42c4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2eec0b54b0 0x7f2eec0b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef42c4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eec1f8bf0 0x7f2eec1f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef42c4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec0cdd10 con 0x7f2ed8005260 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef42c4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec0d0f40 con 0x7f2eec1f8bf0 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef42c4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2eec0cde90 con 0x7f2eec0b54b0 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef2060640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8005260 0x7f2eec0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef2060640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8005260 0x7f2eec0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43624/0 (socket says 172.21.15.131:43624) 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef2060640 1 -- 172.21.15.131:0/2348520651 learned_addr learned my addr 172.21.15.131:0/2348520651 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef185f640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2eec0b54b0 0x7f2eec0b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef2861640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eec1f8bf0 0x7f2eec1f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef2060640 1 -- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2eec0b54b0 msgr2=0x7f2eec0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:40.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef2060640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2eec0b54b0 0x7f2eec0b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef2060640 1 -- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eec1f8bf0 msgr2=0x7f2eec1f9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef2060640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eec1f8bf0 0x7f2eec1f9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef2060640 1 -- 172.21.15.131:0/2348520651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec0b41e0 con 0x7f2ed8005260 2024-09-17T02:42:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef2060640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8005260 0x7f2eec0b1e80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2edc011a50 tx=0x7f2edc011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ee2ffd640 1 -- 172.21.15.131:0/2348520651 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc01f070 con 0x7f2ed8005260 2024-09-17T02:42:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ee2ffd640 1 -- 172.21.15.131:0/2348520651 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2edc012450 con 0x7f2ed8005260 2024-09-17T02:42:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ee2ffd640 1 -- 172.21.15.131:0/2348520651 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc012840 con 0x7f2ed8005260 2024-09-17T02:42:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef42c4640 1 -- 172.21.15.131:0/2348520651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec0b44d0 con 0x7f2ed8005260 2024-09-17T02:42:40.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.323+0000 7f2ef42c4640 1 -- 172.21.15.131:0/2348520651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec0b4a90 con 0x7f2ed8005260 2024-09-17T02:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.327+0000 7f2ef42c4640 1 -- 172.21.15.131:0/2348520651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2eec071210 con 0x7f2ed8005260 2024-09-17T02:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.327+0000 7f2ee2ffd640 1 -- 172.21.15.131:0/2348520651 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2edc022070 con 0x7f2ed8005260 2024-09-17T02:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.327+0000 7f2ee2ffd640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ed0073230 0x7f2ed0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.327+0000 7f2ee2ffd640 1 -- 172.21.15.131:0/2348520651 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f2edc094b40 con 0x7f2ed8005260 2024-09-17T02:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.327+0000 7f2ef185f640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ed0073230 0x7f2ed0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:40.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.327+0000 7f2ef185f640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ed0073230 0x7f2ed0075740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2eec1f6370 tx=0x7f2ee8006f00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:40.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.331+0000 7f2ee2ffd640 1 -- 172.21.15.131:0/2348520651 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2edc05f710 con 0x7f2ed8005260 2024-09-17T02:42:40.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.507+0000 7f2ef42c4640 1 -- 172.21.15.131:0/2348520651 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2eec0b1960 con 0x7f2ed8005260 2024-09-17T02:42:40.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.763+0000 7f2ee2ffd640 1 -- 172.21.15.131:0/2348520651 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) ==== 97+0+0 (secure 0 0 0) 0x7f2edc064140 con 0x7f2ed8005260 2024-09-17T02:42:40.769 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-09-17T02:42:40.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.767+0000 7f2ef42c4640 1 -- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ed0073230 msgr2=0x7f2ed0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:40.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.767+0000 7f2ef42c4640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ed0073230 0x7f2ed0075740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2eec1f6370 tx=0x7f2ee8006f00 comp rx=0 tx=0).stop 2024-09-17T02:42:40.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.767+0000 7f2ef42c4640 1 -- 172.21.15.131:0/2348520651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8005260 msgr2=0x7f2eec0b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:40.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.767+0000 7f2ef42c4640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8005260 0x7f2eec0b1e80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2edc011a50 tx=0x7f2edc011f10 comp rx=0 tx=0).stop 2024-09-17T02:42:40.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.767+0000 7f2ef42c4640 1 -- 172.21.15.131:0/2348520651 shutdown_connections 2024-09-17T02:42:40.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.767+0000 7f2ef42c4640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ed0073230 0x7f2ed0075740 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.767+0000 7f2ef42c4640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2eec1f8bf0 0x7f2eec1f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.767+0000 7f2ef42c4640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2eec0b54b0 0x7f2eec0b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.767+0000 7f2ef42c4640 1 --2- 172.21.15.131:0/2348520651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed8005260 0x7f2eec0b1e80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.767+0000 7f2ef42c4640 1 -- 172.21.15.131:0/2348520651 >> 172.21.15.131:0/2348520651 conn(0x7f2eec0d1130 msgr2=0x7f2eec153fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:40.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.771+0000 7f2ef42c4640 1 -- 172.21.15.131:0/2348520651 shutdown_connections 2024-09-17T02:42:40.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.771+0000 7f2ef42c4640 1 -- 172.21.15.131:0/2348520651 wait complete. 2024-09-17T02:42:40.799 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T02:42:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.943+0000 7f8e10f5a640 1 -- 172.21.15.131:0/1513567323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df8003a10 msgr2=0x7f8df8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.943+0000 7f8e10f5a640 1 --2- 172.21.15.131:0/1513567323 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df8003a10 0x7f8df8003e80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0b7610 tx=0x7f8dfc01cda0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.943+0000 7f8e0ad9d640 1 -- 172.21.15.131:0/1513567323 reap_dead start 2024-09-17T02:42:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.943+0000 7f8e10f5a640 1 -- 172.21.15.131:0/1513567323 shutdown_connections 2024-09-17T02:42:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.943+0000 7f8e10f5a640 1 -- 172.21.15.131:0/1513567323 >> 172.21.15.131:0/1513567323 conn(0x7f8e0c0d1b10 msgr2=0x7f8e0c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.943+0000 7f8e10f5a640 1 -- 172.21.15.131:0/1513567323 shutdown_connections 2024-09-17T02:42:40.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.943+0000 7f8e10f5a640 1 -- 172.21.15.131:0/1513567323 wait complete. 2024-09-17T02:42:40.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e10f5a640 1 Processor -- start 2024-09-17T02:42:40.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e10f5a640 1 -- start start 2024-09-17T02:42:40.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e10f5a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8df8003a10 0x7f8e0c1fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e10f5a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e0c15c140 0x7f8e0c1fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e10f5a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e0c1fc630 0x7f8e0c1f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e10f5a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c15fd60 con 0x7f8e0c1fc630 2024-09-17T02:42:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e10f5a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c15fbe0 con 0x7f8df8003a10 2024-09-17T02:42:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e10f5a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0c169da0 con 0x7f8e0c15c140 2024-09-17T02:42:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e0ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e0c1fc630 0x7f8e0c1f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e0ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e0c1fc630 0x7f8e0c1f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43642/0 (socket says 172.21.15.131:43642) 2024-09-17T02:42:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e0ad9d640 1 -- 172.21.15.131:0/436445364 learned_addr learned my addr 172.21.15.131:0/436445364 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e0a59c640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8df8003a10 0x7f8e0c1fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e0ad9d640 1 -- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e0c15c140 msgr2=0x7f8e0c1fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:40.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e0ad9d640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e0c15c140 0x7f8e0c1fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e0ad9d640 1 -- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8df8003a10 msgr2=0x7f8e0c1fb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e0ad9d640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8df8003a10 0x7f8e0c1fb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e0ad9d640 1 -- 172.21.15.131:0/436445364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c1fca40 con 0x7f8e0c1fc630 2024-09-17T02:42:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e0ad9d640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e0c1fc630 0x7f8e0c1f8b00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c08d6d0 tx=0x7f8e04010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e037fe640 1 -- 172.21.15.131:0/436445364 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04003980 con 0x7f8e0c1fc630 2024-09-17T02:42:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e037fe640 1 -- 172.21.15.131:0/436445364 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e04015070 con 0x7f8e0c1fc630 2024-09-17T02:42:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e037fe640 1 -- 172.21.15.131:0/436445364 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04004030 con 0x7f8e0c1fc630 2024-09-17T02:42:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e10f5a640 1 -- 172.21.15.131:0/436445364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c0b85b0 con 0x7f8e0c1fc630 2024-09-17T02:42:40.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e10f5a640 1 -- 172.21.15.131:0/436445364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c159a60 con 0x7f8e0c1fc630 2024-09-17T02:42:40.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.947+0000 7f8e10f5a640 1 -- 172.21.15.131:0/436445364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0c162410 con 0x7f8e0c1fc630 2024-09-17T02:42:40.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.951+0000 7f8e037fe640 1 -- 172.21.15.131:0/436445364 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8e040041d0 con 0x7f8e0c1fc630 2024-09-17T02:42:40.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.951+0000 7f8e037fe640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dec073200 0x7f8dec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:40.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.951+0000 7f8e037fe640 1 -- 172.21.15.131:0/436445364 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f8e04094a90 con 0x7f8e0c1fc630 2024-09-17T02:42:40.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.951+0000 7f8e0a59c640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dec073200 0x7f8dec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:40.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.951+0000 7f8e0a59c640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dec073200 0x7f8dec075710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0b0000 tx=0x7f8dfc0a7040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:40.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:40.955+0000 7f8e037fe640 1 -- 172.21.15.131:0/436445364 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8e0c1fca40 con 0x7f8e0c1fc630 2024-09-17T02:42:41.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.167+0000 7f8e10f5a640 1 -- 172.21.15.131:0/436445364 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8e0c0ce930 con 0x7f8e0c1fc630 2024-09-17T02:42:41.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.167+0000 7f8e037fe640 1 -- 172.21.15.131:0/436445364 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v29) ==== 80+0+5304 (secure 0 0 0) 0x7f8e0405fd00 con 0x7f8e0c1fc630 2024-09-17T02:42:41.171 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:41.171 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm4U1cHhAATu86VrwT2vMjL/cW2SICMw==", 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.172 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmVpUSKxAA4l5zGHXg0VRasiyBI/j4ig==", 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmwHuCJRAAlOj+za8ZmqMI55PsShjKDw==", 2024-09-17T02:42:41.173 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm9f+9MhAAS8z9jZFYon3nrmu9/4JMPg==", 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:42:41.174 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD16+hmV9xkBxAAT+IDf2J4awJfqYcx2xDoNA==", 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.175 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD46+hmOhJsGBAAdlA4BSdOPk8BySUTfo1ghg==", 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD76+hm3mSoKBAAQPoGFy816yWZQW6vPIK6bA==", 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:42:41.176 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD+6+hmpZaPNxAAp0EV6bn5SugDMwA4RnnJwg==", 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.177 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAF7OhmgDfVAxAALACK3bk+PK3UhG20b498HQ==", 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAI7Ohm6w44FBAA90GkelUAbiNjAwn+6IJWcg==", 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:42:41.178 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAL7Ohm6S6TIxAArISJW//rmEONG2bC/sfkXA==", 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.179 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAO7OhmICwTMxAArAr7DhcO8A0FDSu7VEBehw==", 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA==", 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.180 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmduVoBRAAwr50jSXw6sCyo+5t+3iT7A==", 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:42:41.181 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmBBCDABAAMw9581QyaOxJk5fs35o+ng==", 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-09-17T02:42:41.182 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmgznRCRAA1Z8d32ADoSG+7RHtdPGmTQ==", 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmpUfvABAAzclxwjR4E+UxvtkbwVqDXw==", 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:42:41.183 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmnWmJGBAA6fzewrm8ELqUAz/MczTOGA==", 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.184 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-09-17T02:42:41.185 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmrTq3ExAAuR8+2bItlpHqqFzlvPxASA==", 2024-09-17T02:42:41.185 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:42:41.185 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:42:41.185 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:42:41.185 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:42:41.185 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.185 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.185 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-09-17T02:42:41.185 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:42:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 -- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dec073200 msgr2=0x7f8dec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dec073200 0x7f8dec075710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0b0000 tx=0x7f8dfc0a7040 comp rx=0 tx=0).stop 2024-09-17T02:42:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 -- 172.21.15.131:0/436445364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e0c1fc630 msgr2=0x7f8e0c1f8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e0c1fc630 0x7f8e0c1f8b00 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c08d6d0 tx=0x7f8e04010f40 comp rx=0 tx=0).stop 2024-09-17T02:42:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 -- 172.21.15.131:0/436445364 shutdown_connections 2024-09-17T02:42:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dec073200 0x7f8dec075710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e0c1fc630 0x7f8e0c1f8b00 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e0c15c140 0x7f8e0c1fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 --2- 172.21.15.131:0/436445364 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8df8003a10 0x7f8e0c1fb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 -- 172.21.15.131:0/436445364 >> 172.21.15.131:0/436445364 conn(0x7f8e0c0d1b10 msgr2=0x7f8e0c157ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:41.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 -- 172.21.15.131:0/436445364 shutdown_connections 2024-09-17T02:42:41.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.171+0000 7f8e10f5a640 1 -- 172.21.15.131:0/436445364 wait complete. 2024-09-17T02:42:41.193 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T02:42:41.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.347+0000 7f2b681a1640 1 -- 172.21.15.131:0/2731101148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b4c003a40 msgr2=0x7f2b4c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:41.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.347+0000 7f2b681a1640 1 --2- 172.21.15.131:0/2731101148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b4c003a40 0x7f2b4c003eb0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0016f0 tx=0x7f2b5c01c7b0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.347+0000 7f2b6673e640 1 -- 172.21.15.131:0/2731101148 reap_dead start 2024-09-17T02:42:41.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.347+0000 7f2b681a1640 1 -- 172.21.15.131:0/2731101148 shutdown_connections 2024-09-17T02:42:41.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.347+0000 7f2b681a1640 1 -- 172.21.15.131:0/2731101148 >> 172.21.15.131:0/2731101148 conn(0x7f2b6006eea0 msgr2=0x7f2b6006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:41.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.347+0000 7f2b681a1640 1 -- 172.21.15.131:0/2731101148 shutdown_connections 2024-09-17T02:42:41.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.347+0000 7f2b681a1640 1 -- 172.21.15.131:0/2731101148 wait complete. 2024-09-17T02:42:41.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b681a1640 1 Processor -- start 2024-09-17T02:42:41.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b681a1640 1 -- start start 2024-09-17T02:42:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b681a1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b4c003a40 0x7f2b6015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b681a1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6014f650 0x7f2b6015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b681a1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b60156500 0x7f2b60156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b681a1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2b60166ef0 con 0x7f2b6014f650 2024-09-17T02:42:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b681a1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b60166d70 con 0x7f2b60156500 2024-09-17T02:42:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b681a1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2b60167070 con 0x7f2b4c003a40 2024-09-17T02:42:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b6573c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6014f650 0x7f2b6015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b6573c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6014f650 0x7f2b6015a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57870/0 (socket says 172.21.15.131:57870) 2024-09-17T02:42:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b6573c640 1 -- 172.21.15.131:0/3992910501 learned_addr learned my addr 172.21.15.131:0/3992910501 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b6673e640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b60156500 0x7f2b60156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:41.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b65f3d640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b4c003a40 0x7f2b6015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b6573c640 1 -- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b4c003a40 msgr2=0x7f2b6015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b6573c640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b4c003a40 0x7f2b6015a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b6573c640 1 -- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b60156500 msgr2=0x7f2b60156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b6573c640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b60156500 0x7f2b60156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b6573c640 1 -- 172.21.15.131:0/3992910501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b600b2f90 con 0x7f2b6014f650 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b6573c640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6014f650 0x7f2b6015a750 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2b50004fd0 tx=0x7f2b50001890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b567fc640 1 -- 172.21.15.131:0/3992910501 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50015070 con 0x7f2b6014f650 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b567fc640 1 -- 172.21.15.131:0/3992910501 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b500109a0 con 0x7f2b6014f650 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b681a1640 1 -- 172.21.15.131:0/3992910501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b60151680 con 0x7f2b6014f650 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b567fc640 1 -- 172.21.15.131:0/3992910501 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b50010df0 con 0x7f2b6014f650 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b681a1640 1 -- 172.21.15.131:0/3992910501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b60155a20 con 0x7f2b6014f650 2024-09-17T02:42:41.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.351+0000 7f2b681a1640 1 -- 172.21.15.131:0/3992910501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b601513f0 con 0x7f2b6014f650 2024-09-17T02:42:41.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.355+0000 7f2b567fc640 1 -- 172.21.15.131:0/3992910501 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2b50018070 con 0x7f2b6014f650 2024-09-17T02:42:41.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.355+0000 7f2b567fc640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b44073230 0x7f2b44075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:41.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.355+0000 7f2b567fc640 1 -- 172.21.15.131:0/3992910501 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f2b500944a0 con 0x7f2b6014f650 2024-09-17T02:42:41.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.355+0000 7f2b65f3d640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b44073230 0x7f2b44075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:41.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.355+0000 7f2b65f3d640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b44073230 0x7f2b44075740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0016c0 tx=0x7f2b5c0195a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:41.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.359+0000 7f2b567fc640 1 -- 172.21.15.131:0/3992910501 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2b5005f690 con 0x7f2b6014f650 2024-09-17T02:42:41.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.531+0000 7f2b681a1640 1 -- 172.21.15.131:0/3992910501 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2b60073430 con 0x7f2b6014f650 2024-09-17T02:42:41.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.531+0000 7f2b567fc640 1 -- 172.21.15.131:0/3992910501 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) ==== 84+0+65 (secure 0 0 0) 0x7f2b5005ef90 con 0x7f2b6014f650 2024-09-17T02:42:41.536 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:41.536 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:42:41.536 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:41.536 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-09-17T02:42:41.536 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-09-17T02:42:41.536 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:41.536 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:42:41.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 -- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b44073230 msgr2=0x7f2b44075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:41.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b44073230 0x7f2b44075740 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c0016c0 tx=0x7f2b5c0195a0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 -- 172.21.15.131:0/3992910501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6014f650 msgr2=0x7f2b6015a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:41.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6014f650 0x7f2b6015a750 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2b50004fd0 tx=0x7f2b50001890 comp rx=0 tx=0).stop 2024-09-17T02:42:41.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 -- 172.21.15.131:0/3992910501 shutdown_connections 2024-09-17T02:42:41.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b44073230 0x7f2b44075740 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b60156500 0x7f2b60156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b6014f650 0x7f2b6015a750 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 --2- 172.21.15.131:0/3992910501 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b4c003a40 0x7f2b6015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 -- 172.21.15.131:0/3992910501 >> 172.21.15.131:0/3992910501 conn(0x7f2b6006eea0 msgr2=0x7f2b600b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:41.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 -- 172.21.15.131:0/3992910501 shutdown_connections 2024-09-17T02:42:41.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.535+0000 7f2b681a1640 1 -- 172.21.15.131:0/3992910501 wait complete. 2024-09-17T02:42:41.558 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T02:42:41.558 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T02:42:41.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.711+0000 7f4538076640 1 -- 172.21.15.131:0/1059896074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f453015f060 msgr2=0x7f453015f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:41.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.711+0000 7f4538076640 1 --2- 172.21.15.131:0/1059896074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f453015f060 0x7f453015f470 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4530070f60 tx=0x7f452001c740 comp rx=0 tx=0).stop 2024-09-17T02:42:41.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.711+0000 7f4538076640 1 -- 172.21.15.131:0/1059896074 shutdown_connections 2024-09-17T02:42:41.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.711+0000 7f4538076640 1 --2- 172.21.15.131:0/1059896074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f453015f060 0x7f453015f470 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.711+0000 7f4538076640 1 --2- 172.21.15.131:0/1059896074 >> v2:172.21.15.189:3300/0 conn(0x7f45301523d0 0x7f45300b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.711+0000 7f4538076640 1 -- 172.21.15.131:0/1059896074 >> 172.21.15.131:0/1059896074 conn(0x7f45300d05a0 msgr2=0x7f45300d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:41.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.711+0000 7f4538076640 1 -- 172.21.15.131:0/1059896074 shutdown_connections 2024-09-17T02:42:41.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.711+0000 7f4538076640 1 -- 172.21.15.131:0/1059896074 wait complete. 2024-09-17T02:42:41.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.711+0000 7f4538076640 1 Processor -- start 2024-09-17T02:42:41.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.711+0000 7f4538076640 1 -- start start 2024-09-17T02:42:41.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4538076640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45301523d0 0x7f4530144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:41.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4538076640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f453015ba70 0x7f4530146990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:41.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4538076640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f453015f060 0x7f4530146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:41.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4538076640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4530168ea0 con 0x7f453015ba70 2024-09-17T02:42:41.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4538076640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4530168d20 con 0x7f453015f060 2024-09-17T02:42:41.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4538076640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4530169020 con 0x7f45301523d0 2024-09-17T02:42:41.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4535611640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f453015ba70 0x7f4530146990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:41.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4535611640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f453015ba70 0x7f4530146990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57888/0 (socket says 172.21.15.131:57888) 2024-09-17T02:42:41.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4535611640 1 -- 172.21.15.131:0/1342494876 learned_addr learned my addr 172.21.15.131:0/1342494876 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:41.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4535e12640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45301523d0 0x7f4530144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4536613640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f453015f060 0x7f4530146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4535611640 1 -- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45301523d0 msgr2=0x7f4530144840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4535611640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45301523d0 0x7f4530144840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4535611640 1 -- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f453015f060 msgr2=0x7f4530146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4535611640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f453015f060 0x7f4530146ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4535611640 1 -- 172.21.15.131:0/1342494876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f453014a950 con 0x7f453015ba70 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4536613640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f453015f060 0x7f4530146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4535611640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f453015ba70 0x7f4530146990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4520000c10 tx=0x7f4520004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f45267fc640 1 -- 172.21.15.131:0/1342494876 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45200bb070 con 0x7f453015ba70 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f45267fc640 1 -- 172.21.15.131:0/1342494876 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4520005030 con 0x7f453015ba70 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4538076640 1 -- 172.21.15.131:0/1342494876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f453014abf0 con 0x7f453015ba70 2024-09-17T02:42:41.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f45267fc640 1 -- 172.21.15.131:0/1342494876 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45200b2040 con 0x7f453015ba70 2024-09-17T02:42:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4538076640 1 -- 172.21.15.131:0/1342494876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f453015b170 con 0x7f453015ba70 2024-09-17T02:42:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4538076640 1 -- 172.21.15.131:0/1342494876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453014b390 con 0x7f453015ba70 2024-09-17T02:42:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f45267fc640 1 -- 172.21.15.131:0/1342494876 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f45200be070 con 0x7f453015ba70 2024-09-17T02:42:41.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f45267fc640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4514073230 0x7f4514075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:41.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f45267fc640 1 -- 172.21.15.131:0/1342494876 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5249+0+0 (secure 0 0 0) 0x7f452012dc40 con 0x7f453015ba70 2024-09-17T02:42:41.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.715+0000 7f4535e12640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4514073230 0x7f4514075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:41.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.719+0000 7f4535e12640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4514073230 0x7f4514075740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f452c005fd0 tx=0x7f452c00c460 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:41.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.723+0000 7f45267fc640 1 -- 172.21.15.131:0/1342494876 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f45200163f0 con 0x7f453015ba70 2024-09-17T02:42:41.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:41.899+0000 7f4538076640 1 -- 172.21.15.131:0/1342494876 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f45300b2ef0 con 0x7f453015ba70 2024-09-17T02:42:42.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.787+0000 7f45267fc640 1 -- 172.21.15.131:0/1342494876 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) ==== 138+0+0 (secure 0 0 0) 0x7f45200f8f40 con 0x7f453015ba70 2024-09-17T02:42:42.792 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-09-17T02:42:42.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 -- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4514073230 msgr2=0x7f4514075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:42.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4514073230 0x7f4514075740 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f452c005fd0 tx=0x7f452c00c460 comp rx=0 tx=0).stop 2024-09-17T02:42:42.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 -- 172.21.15.131:0/1342494876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f453015ba70 msgr2=0x7f4530146990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:42.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f453015ba70 0x7f4530146990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4520000c10 tx=0x7f4520004040 comp rx=0 tx=0).stop 2024-09-17T02:42:42.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 -- 172.21.15.131:0/1342494876 shutdown_connections 2024-09-17T02:42:42.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4514073230 0x7f4514075740 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:42.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f453015f060 0x7f4530146ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:42.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f453015ba70 0x7f4530146990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:42.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 --2- 172.21.15.131:0/1342494876 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45301523d0 0x7f4530144840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:42.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 -- 172.21.15.131:0/1342494876 >> 172.21.15.131:0/1342494876 conn(0x7f45300d05a0 msgr2=0x7f453015be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:42.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 -- 172.21.15.131:0/1342494876 shutdown_connections 2024-09-17T02:42:42.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.791+0000 7f4538076640 1 -- 172.21.15.131:0/1342494876 wait complete. 2024-09-17T02:42:42.828 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T02:42:42.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.991+0000 7f02642bc640 1 -- 172.21.15.131:0/1778435251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c15cc00 msgr2=0x7f025c15d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:42.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.991+0000 7f02642bc640 1 --2- 172.21.15.131:0/1778435251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c15cc00 0x7f025c15d010 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f024c009900 tx=0x7f024c01c8b0 comp rx=0 tx=0).stop 2024-09-17T02:42:42.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.991+0000 7f02642bc640 1 -- 172.21.15.131:0/1778435251 shutdown_connections 2024-09-17T02:42:42.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.991+0000 7f02642bc640 1 --2- 172.21.15.131:0/1778435251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c15cc00 0x7f025c15d010 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:42.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.991+0000 7f02642bc640 1 --2- 172.21.15.131:0/1778435251 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f025c0b32a0 0x7f025c0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:42.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.991+0000 7f02642bc640 1 -- 172.21.15.131:0/1778435251 >> 172.21.15.131:0/1778435251 conn(0x7f025c0d0490 msgr2=0x7f025c06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:42.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.991+0000 7f02642bc640 1 -- 172.21.15.131:0/1778435251 shutdown_connections 2024-09-17T02:42:42.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.991+0000 7f02642bc640 1 -- 172.21.15.131:0/1778435251 wait complete. 2024-09-17T02:42:42.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.991+0000 7f02642bc640 1 Processor -- start 2024-09-17T02:42:42.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.991+0000 7f02642bc640 1 -- start start 2024-09-17T02:42:42.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f02642bc640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f025c0b32a0 0x7f025c136140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:42.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f02642bc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c15cc00 0x7f025c14ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:42.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f02642bc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f025c15d550 0x7f025c1f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:42.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f02642bc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f025c1697a0 con 0x7f025c15cc00 2024-09-17T02:42:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f02642bc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f025c169620 con 0x7f025c15d550 2024-09-17T02:42:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f02642bc640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f025c169920 con 0x7f025c0b32a0 2024-09-17T02:42:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f0261857640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c15cc00 0x7f025c14ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f0261857640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c15cc00 0x7f025c14ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57906/0 (socket says 172.21.15.131:57906) 2024-09-17T02:42:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f0261857640 1 -- 172.21.15.131:0/2746216193 learned_addr learned my addr 172.21.15.131:0/2746216193 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f0262058640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f025c0b32a0 0x7f025c136140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f0262859640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f025c15d550 0x7f025c1f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f0261857640 1 -- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f025c0b32a0 msgr2=0x7f025c136140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f0261857640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f025c0b32a0 0x7f025c136140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f0261857640 1 -- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f025c15d550 msgr2=0x7f025c1f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f0261857640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f025c15d550 0x7f025c1f96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:43.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f0261857640 1 -- 172.21.15.131:0/2746216193 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c154a90 con 0x7f025c15cc00 2024-09-17T02:42:43.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f0261857640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c15cc00 0x7f025c14ab80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f024c009850 tx=0x7f024c0a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:43.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f025affd640 1 -- 172.21.15.131:0/2746216193 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c0bc020 con 0x7f025c15cc00 2024-09-17T02:42:43.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f025affd640 1 -- 172.21.15.131:0/2746216193 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f024c0044b0 con 0x7f025c15cc00 2024-09-17T02:42:43.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f02642bc640 1 -- 172.21.15.131:0/2746216193 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c14b840 con 0x7f025c15cc00 2024-09-17T02:42:43.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f025affd640 1 -- 172.21.15.131:0/2746216193 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c0b2040 con 0x7f025c15cc00 2024-09-17T02:42:43.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f02642bc640 1 -- 172.21.15.131:0/2746216193 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c14be00 con 0x7f025c15cc00 2024-09-17T02:42:43.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f02642bc640 1 -- 172.21.15.131:0/2746216193 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025c0ccb30 con 0x7f025c15cc00 2024-09-17T02:42:43.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f025affd640 1 -- 172.21.15.131:0/2746216193 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f024c0bf070 con 0x7f025c15cc00 2024-09-17T02:42:43.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f025affd640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0248073200 0x7f0248075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:43.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.995+0000 7f025affd640 1 -- 172.21.15.131:0/2746216193 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5619+0+0 (secure 0 0 0) 0x7f024c12e1f0 con 0x7f025c15cc00 2024-09-17T02:42:43.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.999+0000 7f0262058640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0248073200 0x7f0248075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:43.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:42.999+0000 7f0262058640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0248073200 0x7f0248075710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0250005c50 tx=0x7f025000a390 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:43.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.003+0000 7f025affd640 1 -- 172.21.15.131:0/2746216193 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f024c0164d0 con 0x7f025c15cc00 2024-09-17T02:42:43.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.171+0000 7f02642bc640 1 -- 172.21.15.131:0/2746216193 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f025c151530 con 0x7f025c15cc00 2024-09-17T02:42:43.496 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:42:43.499 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:42:43.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.795+0000 7f025affd640 1 -- 172.21.15.131:0/2746216193 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) ==== 170+0+0 (secure 0 0 0) 0x7f024c0f92f0 con 0x7f025c15cc00 2024-09-17T02:42:43.798 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-09-17T02:42:43.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 -- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0248073200 msgr2=0x7f0248075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:43.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0248073200 0x7f0248075710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0250005c50 tx=0x7f025000a390 comp rx=0 tx=0).stop 2024-09-17T02:42:43.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 -- 172.21.15.131:0/2746216193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c15cc00 msgr2=0x7f025c14ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:43.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c15cc00 0x7f025c14ab80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f024c009850 tx=0x7f024c0a5770 comp rx=0 tx=0).stop 2024-09-17T02:42:43.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 -- 172.21.15.131:0/2746216193 shutdown_connections 2024-09-17T02:42:43.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0248073200 0x7f0248075710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:43.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f025c15d550 0x7f025c1f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:43.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f025c15cc00 0x7f025c14ab80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:43.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 --2- 172.21.15.131:0/2746216193 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f025c0b32a0 0x7f025c136140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:43.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 -- 172.21.15.131:0/2746216193 >> 172.21.15.131:0/2746216193 conn(0x7f025c0d0490 msgr2=0x7f025c1507a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:43.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 -- 172.21.15.131:0/2746216193 shutdown_connections 2024-09-17T02:42:43.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.799+0000 7f02642bc640 1 -- 172.21.15.131:0/2746216193 wait complete. 2024-09-17T02:42:43.838 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T02:42:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 -- 172.21.15.131:0/2114884385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc140051c0 msgr2=0x7fcc14005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 --2- 172.21.15.131:0/2114884385 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc140051c0 0x7fcc14005920 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00bae0 tx=0x7fcc1c01ca30 comp rx=0 tx=0).stop 2024-09-17T02:42:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc32d9d640 1 -- 172.21.15.131:0/2114884385 reap_dead start 2024-09-17T02:42:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 -- 172.21.15.131:0/2114884385 shutdown_connections 2024-09-17T02:42:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 -- 172.21.15.131:0/2114884385 >> 172.21.15.131:0/2114884385 conn(0x7fcc3406eea0 msgr2=0x7fcc3406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:43.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 -- 172.21.15.131:0/2114884385 shutdown_connections 2024-09-17T02:42:43.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 -- 172.21.15.131:0/2114884385 wait complete. 2024-09-17T02:42:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 Processor -- start 2024-09-17T02:42:43.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 -- start start 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc140051c0 0x7fcc341f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc340c96c0 0x7fcc34153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc340c9ad0 0x7fcc34153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3415f060 con 0x7fcc340c9ad0 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3415eee0 con 0x7fcc340c96c0 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc38e87640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3415f1e0 con 0x7fcc140051c0 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc340c9ad0 0x7fcc34153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc32d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc340c9ad0 0x7fcc34153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57926/0 (socket says 172.21.15.131:57926) 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.995+0000 7fcc32d9d640 1 -- 172.21.15.131:0/1631847822 learned_addr learned my addr 172.21.15.131:0/1631847822 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc31d9b640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc340c96c0 0x7fcc34153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc3259c640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc140051c0 0x7fcc341f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc32d9d640 1 -- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc140051c0 msgr2=0x7fcc341f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:44.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc32d9d640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc140051c0 0x7fcc341f6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:44.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc32d9d640 1 -- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc340c96c0 msgr2=0x7fcc34153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:44.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc32d9d640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc340c96c0 0x7fcc34153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:44.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc32d9d640 1 -- 172.21.15.131:0/1631847822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc340cdbf0 con 0x7fcc340c9ad0 2024-09-17T02:42:44.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc3259c640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc140051c0 0x7fcc341f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:44.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc32d9d640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc340c9ad0 0x7fcc34153980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcc340cdba0 tx=0x7fcc2400d640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:44.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc2affd640 1 -- 172.21.15.131:0/1631847822 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24012670 con 0x7fcc340c9ad0 2024-09-17T02:42:44.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc2affd640 1 -- 172.21.15.131:0/1631847822 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc24019070 con 0x7fcc340c9ad0 2024-09-17T02:42:44.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc2affd640 1 -- 172.21.15.131:0/1631847822 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24012b80 con 0x7fcc340c9ad0 2024-09-17T02:42:44.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc38e87640 1 -- 172.21.15.131:0/1631847822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc341f72c0 con 0x7fcc340c9ad0 2024-09-17T02:42:44.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc38e87640 1 -- 172.21.15.131:0/1631847822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc340ce3d0 con 0x7fcc340c9ad0 2024-09-17T02:42:44.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc38e87640 1 -- 172.21.15.131:0/1631847822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc34073430 con 0x7fcc340c9ad0 2024-09-17T02:42:44.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc2affd640 1 -- 172.21.15.131:0/1631847822 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcc2401e460 con 0x7fcc340c9ad0 2024-09-17T02:42:44.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc2affd640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc18073200 0x7fcc18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:44.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc2affd640 1 -- 172.21.15.131:0/1631847822 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 6001+0+0 (secure 0 0 0) 0x7fcc24097d30 con 0x7fcc340c9ad0 2024-09-17T02:42:44.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:43.999+0000 7fcc3259c640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc18073200 0x7fcc18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:44.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.003+0000 7fcc3259c640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc18073200 0x7fcc18075710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009f40 tx=0x7fcc2c0094f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:44.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.003+0000 7fcc2affd640 1 -- 172.21.15.131:0/1631847822 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcc3406e7f0 con 0x7fcc340c9ad0 2024-09-17T02:42:44.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.211+0000 7fcc38e87640 1 -- 172.21.15.131:0/1631847822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcc340cde80 con 0x7fcc340c9ad0 2024-09-17T02:42:44.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.863+0000 7fcc2affd640 1 -- 172.21.15.131:0/1631847822 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v23) ==== 172+0+0 (secure 0 0 0) 0x7fcc24062c20 con 0x7fcc340c9ad0 2024-09-17T02:42:44.868 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 7 and data pool 8 2024-09-17T02:42:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 -- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc18073200 msgr2=0x7fcc18075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc18073200 0x7fcc18075710 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c009f40 tx=0x7fcc2c0094f0 comp rx=0 tx=0).stop 2024-09-17T02:42:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 -- 172.21.15.131:0/1631847822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc340c9ad0 msgr2=0x7fcc34153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:44.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc340c9ad0 0x7fcc34153980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcc340cdba0 tx=0x7fcc2400d640 comp rx=0 tx=0).stop 2024-09-17T02:42:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 -- 172.21.15.131:0/1631847822 shutdown_connections 2024-09-17T02:42:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc18073200 0x7fcc18075710 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc340c9ad0 0x7fcc34153980 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc340c96c0 0x7fcc34153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 --2- 172.21.15.131:0/1631847822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc140051c0 0x7fcc341f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 -- 172.21.15.131:0/1631847822 >> 172.21.15.131:0/1631847822 conn(0x7fcc3406eea0 msgr2=0x7fcc34154540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 -- 172.21.15.131:0/1631847822 shutdown_connections 2024-09-17T02:42:44.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:44.875+0000 7fcc38e87640 1 -- 172.21.15.131:0/1631847822 wait complete. 2024-09-17T02:42:44.914 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:42:45.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.063+0000 7fa2807c0640 1 -- 172.21.15.131:0/2751022873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa27806dea0 msgr2=0x7fa27806e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.063+0000 7fa2807c0640 1 --2- 172.21.15.131:0/2751022873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa27806dea0 0x7fa27806e2b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa2780727c0 tx=0x7fa27401c650 comp rx=0 tx=0).stop 2024-09-17T02:42:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.063+0000 7fa2807c0640 1 -- 172.21.15.131:0/2751022873 shutdown_connections 2024-09-17T02:42:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.063+0000 7fa2807c0640 1 --2- 172.21.15.131:0/2751022873 >> v2:172.21.15.189:3300/0 conn(0x7fa2780ce470 0x7fa2780ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.063+0000 7fa2807c0640 1 --2- 172.21.15.131:0/2751022873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa27806dea0 0x7fa27806e2b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.063+0000 7fa2807c0640 1 -- 172.21.15.131:0/2751022873 >> 172.21.15.131:0/2751022873 conn(0x7fa27806eea0 msgr2=0x7fa27806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.063+0000 7fa2807c0640 1 -- 172.21.15.131:0/2751022873 shutdown_connections 2024-09-17T02:42:45.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.063+0000 7fa2807c0640 1 -- 172.21.15.131:0/2751022873 wait complete. 2024-09-17T02:42:45.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.063+0000 7fa2807c0640 1 Processor -- start 2024-09-17T02:42:45.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.063+0000 7fa2807c0640 1 -- start start 2024-09-17T02:42:45.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa2807c0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27806dea0 0x7fa2781534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa2807c0640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa2780cac80 0x7fa2780b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa2807c0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2780ce470 0x7fa2780b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa2807c0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa27816c060 con 0x7fa2780ce470 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa2807c0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa27816bee0 con 0x7fa27806dea0 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa2807c0640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa27816c1e0 con 0x7fa2780cac80 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27ed5d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2780ce470 0x7fa2780b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27ed5d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2780ce470 0x7fa2780b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57954/0 (socket says 172.21.15.131:57954) 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27ed5d640 1 -- 172.21.15.131:0/3786051800 learned_addr learned my addr 172.21.15.131:0/3786051800 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27e55c640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27806dea0 0x7fa2781534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27dd5b640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa2780cac80 0x7fa2780b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27ed5d640 1 -- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa2780cac80 msgr2=0x7fa2780b48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27ed5d640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa2780cac80 0x7fa2780b48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27ed5d640 1 -- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27806dea0 msgr2=0x7fa2781534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27ed5d640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27806dea0 0x7fa2781534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27ed5d640 1 -- 172.21.15.131:0/3786051800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa278153d10 con 0x7fa2780ce470 2024-09-17T02:42:45.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27dd5b640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa2780cac80 0x7fa2780b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:45.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa27ed5d640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2780ce470 0x7fa2780b4e30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa27000ca60 tx=0x7fa27000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:45.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa26effd640 1 -- 172.21.15.131:0/3786051800 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270007d90 con 0x7fa2780ce470 2024-09-17T02:42:45.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa26effd640 1 -- 172.21.15.131:0/3786051800 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa270004c20 con 0x7fa2780ce470 2024-09-17T02:42:45.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa2807c0640 1 -- 172.21.15.131:0/3786051800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa278153ff0 con 0x7fa2780ce470 2024-09-17T02:42:45.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa26effd640 1 -- 172.21.15.131:0/3786051800 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa270005280 con 0x7fa2780ce470 2024-09-17T02:42:45.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa2807c0640 1 -- 172.21.15.131:0/3786051800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2781545b0 con 0x7fa2780ce470 2024-09-17T02:42:45.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa2807c0640 1 -- 172.21.15.131:0/3786051800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa27806e2b0 con 0x7fa2780ce470 2024-09-17T02:42:45.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa26effd640 1 -- 172.21.15.131:0/3786051800 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa270005420 con 0x7fa2780ce470 2024-09-17T02:42:45.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.067+0000 7fa26effd640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa25c073200 0x7fa25c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.071+0000 7fa26effd640 1 -- 172.21.15.131:0/3786051800 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fa270094bd0 con 0x7fa2780ce470 2024-09-17T02:42:45.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.071+0000 7fa27e55c640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa25c073200 0x7fa25c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.071+0000 7fa27e55c640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa25c073200 0x7fa25c075710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2740ae000 tx=0x7fa27401ae90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:45.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.075+0000 7fa26effd640 1 -- 172.21.15.131:0/3786051800 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa270014030 con 0x7fa2780ce470 2024-09-17T02:42:45.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.243+0000 7fa2807c0640 1 -- 172.21.15.131:0/3786051800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa278070c70 con 0x7fa2780ce470 2024-09-17T02:42:45.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.247+0000 7fa26effd640 1 -- 172.21.15.131:0/3786051800 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+17003 (secure 0 0 0) 0x7fa27005fa60 con 0x7fa2780ce470 2024-09-17T02:42:45.250 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:45.251 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":51,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:42:44.218636+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:42:41.917939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T02:42:43.186384+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:42:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.247+0000 7fa2807c0640 1 -- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa25c073200 msgr2=0x7fa25c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.247+0000 7fa2807c0640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa25c073200 0x7fa25c075710 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2740ae000 tx=0x7fa27401ae90 comp rx=0 tx=0).stop 2024-09-17T02:42:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.247+0000 7fa2807c0640 1 -- 172.21.15.131:0/3786051800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2780ce470 msgr2=0x7fa2780b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.247+0000 7fa2807c0640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2780ce470 0x7fa2780b4e30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa27000ca60 tx=0x7fa27000cf20 comp rx=0 tx=0).stop 2024-09-17T02:42:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.251+0000 7fa2807c0640 1 -- 172.21.15.131:0/3786051800 shutdown_connections 2024-09-17T02:42:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.251+0000 7fa2807c0640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa25c073200 0x7fa25c075710 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.251+0000 7fa2807c0640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa2780ce470 0x7fa2780b4e30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.251+0000 7fa2807c0640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa2780cac80 0x7fa2780b48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.251+0000 7fa2807c0640 1 --2- 172.21.15.131:0/3786051800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa27806dea0 0x7fa2781534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.251+0000 7fa2807c0640 1 -- 172.21.15.131:0/3786051800 >> 172.21.15.131:0/3786051800 conn(0x7fa27806eea0 msgr2=0x7fa2780b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:45.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.251+0000 7fa2807c0640 1 -- 172.21.15.131:0/3786051800 shutdown_connections 2024-09-17T02:42:45.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.251+0000 7fa2807c0640 1 -- 172.21.15.131:0/3786051800 wait complete. 2024-09-17T02:42:45.273 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:42:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.419+0000 7f16b35ef640 1 -- 172.21.15.131:0/2504726637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16ac0cec00 msgr2=0x7f16ac0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.419+0000 7f16b35ef640 1 --2- 172.21.15.131:0/2504726637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16ac0cec00 0x7f16ac0cf010 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009900 tx=0x7f16a801c850 comp rx=0 tx=0).stop 2024-09-17T02:42:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.419+0000 7f16b35ef640 1 -- 172.21.15.131:0/2504726637 shutdown_connections 2024-09-17T02:42:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.419+0000 7f16b35ef640 1 --2- 172.21.15.131:0/2504726637 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f16ac0c4c90 0x7f16ac0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.419+0000 7f16b35ef640 1 --2- 172.21.15.131:0/2504726637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16ac0cec00 0x7f16ac0cf010 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.419+0000 7f16b35ef640 1 -- 172.21.15.131:0/2504726637 >> 172.21.15.131:0/2504726637 conn(0x7f16ac06eea0 msgr2=0x7f16ac06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.419+0000 7f16b35ef640 1 -- 172.21.15.131:0/2504726637 shutdown_connections 2024-09-17T02:42:45.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.419+0000 7f16b35ef640 1 -- 172.21.15.131:0/2504726637 wait complete. 2024-09-17T02:42:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b35ef640 1 Processor -- start 2024-09-17T02:42:45.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b35ef640 1 -- start start 2024-09-17T02:42:45.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b35ef640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16ac06e2c0 0x7f16ac0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b35ef640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac0c4c90 0x7f16ac150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b35ef640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f16ac0cec00 0x7f16ac0b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b35ef640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac169cf0 con 0x7f16ac06e2c0 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b35ef640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac15cb90 con 0x7f16ac0c4c90 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b35ef640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f16ac169e70 con 0x7f16ac0cec00 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b138b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16ac06e2c0 0x7f16ac0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b138b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16ac06e2c0 0x7f16ac0b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57980/0 (socket says 172.21.15.131:57980) 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b138b640 1 -- 172.21.15.131:0/487021814 learned_addr learned my addr 172.21.15.131:0/487021814 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b0b8a640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac0c4c90 0x7f16ac150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b1b8c640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f16ac0cec00 0x7f16ac0b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b138b640 1 -- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f16ac0cec00 msgr2=0x7f16ac0b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b138b640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f16ac0cec00 0x7f16ac0b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b138b640 1 -- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac0c4c90 msgr2=0x7f16ac150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b138b640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac0c4c90 0x7f16ac150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b138b640 1 -- 172.21.15.131:0/487021814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac0b3430 con 0x7f16ac06e2c0 2024-09-17T02:42:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b138b640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16ac06e2c0 0x7f16ac0b2ef0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f169c00c3c0 tx=0x7f169c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16a1ffb640 1 -- 172.21.15.131:0/487021814 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c0052d0 con 0x7f16ac06e2c0 2024-09-17T02:42:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16a1ffb640 1 -- 172.21.15.131:0/487021814 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f169c007020 con 0x7f16ac06e2c0 2024-09-17T02:42:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b35ef640 1 -- 172.21.15.131:0/487021814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ac0b41e0 con 0x7f16ac06e2c0 2024-09-17T02:42:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16a1ffb640 1 -- 172.21.15.131:0/487021814 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c007470 con 0x7f16ac06e2c0 2024-09-17T02:42:45.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.423+0000 7f16b35ef640 1 -- 172.21.15.131:0/487021814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ac0b3c20 con 0x7f16ac06e2c0 2024-09-17T02:42:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.427+0000 7f16b35ef640 1 -- 172.21.15.131:0/487021814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ac0c79d0 con 0x7f16ac06e2c0 2024-09-17T02:42:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.427+0000 7f16a1ffb640 1 -- 172.21.15.131:0/487021814 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f169c0067e0 con 0x7f16ac06e2c0 2024-09-17T02:42:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.427+0000 7f16a1ffb640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1690073200 0x7f1690075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.427+0000 7f16a1ffb640 1 -- 172.21.15.131:0/487021814 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7f169c093860 con 0x7f16ac06e2c0 2024-09-17T02:42:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.427+0000 7f16b0b8a640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1690073200 0x7f1690075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.427+0000 7f16b0b8a640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1690073200 0x7f1690075710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f16a80af000 tx=0x7f16a801b090 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:45.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.431+0000 7f16a1ffb640 1 -- 172.21.15.131:0/487021814 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f16ac0c79d0 con 0x7f16ac06e2c0 2024-09-17T02:42:45.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.603+0000 7f16b35ef640 1 -- 172.21.15.131:0/487021814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16ac0c50a0 con 0x7f16ac06e2c0 2024-09-17T02:42:45.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.607+0000 7f16a1ffb640 1 -- 172.21.15.131:0/487021814 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) ==== 74+0+17003 (secure 0 0 0) 0x7f169c05e780 con 0x7f16ac06e2c0 2024-09-17T02:42:45.610 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:45.610 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":51,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:42:44.218636+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:42:41.917939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T02:42:43.186384+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.607+0000 7f16b35ef640 1 -- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1690073200 msgr2=0x7f1690075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.607+0000 7f16b35ef640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1690073200 0x7f1690075710 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f16a80af000 tx=0x7f16a801b090 comp rx=0 tx=0).stop 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.607+0000 7f16b35ef640 1 -- 172.21.15.131:0/487021814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16ac06e2c0 msgr2=0x7f16ac0b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.607+0000 7f16b35ef640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16ac06e2c0 0x7f16ac0b2ef0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f169c00c3c0 tx=0x7f169c00c880 comp rx=0 tx=0).stop 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.611+0000 7f16b35ef640 1 -- 172.21.15.131:0/487021814 shutdown_connections 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.611+0000 7f16b35ef640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1690073200 0x7f1690075710 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.611+0000 7f16b35ef640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f16ac0cec00 0x7f16ac0b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.611+0000 7f16b35ef640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16ac0c4c90 0x7f16ac150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.611+0000 7f16b35ef640 1 --2- 172.21.15.131:0/487021814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16ac06e2c0 0x7f16ac0b2ef0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.611+0000 7f16b35ef640 1 -- 172.21.15.131:0/487021814 >> 172.21.15.131:0/487021814 conn(0x7f16ac06eea0 msgr2=0x7f16ac159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.611+0000 7f16b35ef640 1 -- 172.21.15.131:0/487021814 shutdown_connections 2024-09-17T02:42:45.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.611+0000 7f16b35ef640 1 -- 172.21.15.131:0/487021814 wait complete. 2024-09-17T02:42:45.633 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T02:42:45.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.783+0000 7fe139423640 1 -- 172.21.15.131:0/3462808714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe120003a40 msgr2=0x7fe120003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.783+0000 7fe139423640 1 --2- 172.21.15.131:0/3462808714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe120003a40 0x7fe120003eb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe12400c2e0 tx=0x7fe12401cf20 comp rx=0 tx=0).stop 2024-09-17T02:42:45.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.783+0000 7fe1337fe640 1 -- 172.21.15.131:0/3462808714 reap_dead start 2024-09-17T02:42:45.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.783+0000 7fe139423640 1 -- 172.21.15.131:0/3462808714 shutdown_connections 2024-09-17T02:42:45.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.783+0000 7fe139423640 1 -- 172.21.15.131:0/3462808714 >> 172.21.15.131:0/3462808714 conn(0x7fe13406eea0 msgr2=0x7fe13406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:45.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.783+0000 7fe139423640 1 -- 172.21.15.131:0/3462808714 shutdown_connections 2024-09-17T02:42:45.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.783+0000 7fe139423640 1 -- 172.21.15.131:0/3462808714 wait complete. 2024-09-17T02:42:45.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe139423640 1 Processor -- start 2024-09-17T02:42:45.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe139423640 1 -- start start 2024-09-17T02:42:45.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe139423640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe120003a40 0x7fe1340cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe139423640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1341fc0d0 0x7fe1341fc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe139423640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1340c8120 0x7fe1340c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe139423640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe134168c60 con 0x7fe1340c8120 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe139423640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe134168ae0 con 0x7fe1341fc0d0 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe139423640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe134168de0 con 0x7fe120003a40 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1340c8120 0x7fe1340c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe1337fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1340c8120 0x7fe1340c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58000/0 (socket says 172.21.15.131:58000) 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe1337fe640 1 -- 172.21.15.131:0/1615984476 learned_addr learned my addr 172.21.15.131:0/1615984476 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe132ffd640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe120003a40 0x7fe1340cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe1337fe640 1 -- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe120003a40 msgr2=0x7fe1340cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe12bfff640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1341fc0d0 0x7fe1341fc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe1337fe640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe120003a40 0x7fe1340cc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe1337fe640 1 -- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1341fc0d0 msgr2=0x7fe1341fc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe1337fe640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1341fc0d0 0x7fe1341fc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe1337fe640 1 -- 172.21.15.131:0/1615984476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1341fce60 con 0x7fe1340c8120 2024-09-17T02:42:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe1337fe640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1340c8120 0x7fe1340c8530 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe1340722f0 tx=0x7fe12c00fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe12b7fe640 1 -- 172.21.15.131:0/1615984476 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c00ddd0 con 0x7fe1340c8120 2024-09-17T02:42:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe12b7fe640 1 -- 172.21.15.131:0/1615984476 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe12c011570 con 0x7fe1340c8120 2024-09-17T02:42:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe12b7fe640 1 -- 172.21.15.131:0/1615984476 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0119c0 con 0x7fe1340c8120 2024-09-17T02:42:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe139423640 1 -- 172.21.15.131:0/1615984476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1340cb4e0 con 0x7fe1340c8120 2024-09-17T02:42:45.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.787+0000 7fe139423640 1 -- 172.21.15.131:0/1615984476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1340cbaa0 con 0x7fe1340c8120 2024-09-17T02:42:45.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.791+0000 7fe139423640 1 -- 172.21.15.131:0/1615984476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1341fcbd0 con 0x7fe1340c8120 2024-09-17T02:42:45.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.791+0000 7fe12b7fe640 1 -- 172.21.15.131:0/1615984476 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe12c011b60 con 0x7fe1340c8120 2024-09-17T02:42:45.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.791+0000 7fe12b7fe640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe114073200 0x7fe114075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:45.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.791+0000 7fe12b7fe640 1 -- 172.21.15.131:0/1615984476 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6101+0+0 (secure 0 0 0) 0x7fe12c094d60 con 0x7fe1340c8120 2024-09-17T02:42:45.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.791+0000 7fe132ffd640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe114073200 0x7fe114075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:45.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.791+0000 7fe132ffd640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe114073200 0x7fe114075710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe1240ab000 tx=0x7fe1240a7040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:45.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:45.795+0000 7fe12b7fe640 1 -- 172.21.15.131:0/1615984476 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe12c021060 con 0x7fe1340c8120 2024-09-17T02:42:46.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.003+0000 7fe139423640 1 -- 172.21.15.131:0/1615984476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe1340c4580 con 0x7fe1340c8120 2024-09-17T02:42:46.780 INFO:tasks.ceph.mds.b.smithi189.stderr:2024-09-17T02:42:46.778+0000 7f45b9496640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:42:46.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.887+0000 7fe12b7fe640 1 -- 172.21.15.131:0/1615984476 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7fe12c05fc80 con 0x7fe1340c8120 2024-09-17T02:42:46.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 -- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe114073200 msgr2=0x7fe114075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:46.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe114073200 0x7fe114075710 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe1240ab000 tx=0x7fe1240a7040 comp rx=0 tx=0).stop 2024-09-17T02:42:46.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 -- 172.21.15.131:0/1615984476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1340c8120 msgr2=0x7fe1340c8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:46.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1340c8120 0x7fe1340c8530 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe1340722f0 tx=0x7fe12c00fed0 comp rx=0 tx=0).stop 2024-09-17T02:42:46.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 -- 172.21.15.131:0/1615984476 shutdown_connections 2024-09-17T02:42:46.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe114073200 0x7fe114075710 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:46.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe1340c8120 0x7fe1340c8530 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:46.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe1341fc0d0 0x7fe1341fc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:46.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 --2- 172.21.15.131:0/1615984476 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe120003a40 0x7fe1340cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:46.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 -- 172.21.15.131:0/1615984476 >> 172.21.15.131:0/1615984476 conn(0x7fe13406eea0 msgr2=0x7fe13415a580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:46.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 -- 172.21.15.131:0/1615984476 shutdown_connections 2024-09-17T02:42:46.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:46.891+0000 7fe139423640 1 -- 172.21.15.131:0/1615984476 wait complete. 2024-09-17T02:42:46.919 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T02:42:46.919 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T02:42:47.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.079+0000 7fa917b0c640 1 -- 172.21.15.131:0/2516417102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8fc004d50 msgr2=0x7fa8fc005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:47.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.079+0000 7fa917b0c640 1 --2- 172.21.15.131:0/2516417102 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8fc004d50 0x7fa8fc005160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa90800c610 tx=0x7fa90801fa40 comp rx=0 tx=0).stop 2024-09-17T02:42:47.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.079+0000 7fa9160a9640 1 -- 172.21.15.131:0/2516417102 reap_dead start 2024-09-17T02:42:47.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.079+0000 7fa917b0c640 1 -- 172.21.15.131:0/2516417102 shutdown_connections 2024-09-17T02:42:47.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.079+0000 7fa917b0c640 1 -- 172.21.15.131:0/2516417102 >> 172.21.15.131:0/2516417102 conn(0x7fa91006eea0 msgr2=0x7fa91006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:47.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.079+0000 7fa917b0c640 1 -- 172.21.15.131:0/2516417102 shutdown_connections 2024-09-17T02:42:47.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.079+0000 7fa917b0c640 1 -- 172.21.15.131:0/2516417102 wait complete. 2024-09-17T02:42:47.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa917b0c640 1 Processor -- start 2024-09-17T02:42:47.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa917b0c640 1 -- start start 2024-09-17T02:42:47.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa917b0c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc004d50 0x7fa91015e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa917b0c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9101f9a40 0x7fa9101544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa917b0c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa9100cc030 0x7fa9100c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa917b0c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa9100c5410 con 0x7fa9101f9a40 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa917b0c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa91006e7e0 con 0x7fa8fc004d50 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa917b0c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa9100c5590 con 0x7fa9100cc030 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9150a7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9101f9a40 0x7fa9101544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9150a7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9101f9a40 0x7fa9101544e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58010/0 (socket says 172.21.15.131:58010) 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9150a7640 1 -- 172.21.15.131:0/2348252205 learned_addr learned my addr 172.21.15.131:0/2348252205 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9160a9640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa9100cc030 0x7fa9100c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9158a8640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc004d50 0x7fa91015e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9150a7640 1 -- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa9100cc030 msgr2=0x7fa9100c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9150a7640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa9100cc030 0x7fa9100c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9150a7640 1 -- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc004d50 msgr2=0x7fa91015e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9150a7640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc004d50 0x7fa91015e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9150a7640 1 -- 172.21.15.131:0/2348252205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91015eb40 con 0x7fa9101f9a40 2024-09-17T02:42:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9150a7640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9101f9a40 0x7fa9101544e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0117a0 tx=0x7fa90c011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9067fc640 1 -- 172.21.15.131:0/2348252205 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c004460 con 0x7fa9101f9a40 2024-09-17T02:42:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9067fc640 1 -- 172.21.15.131:0/2348252205 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa90c016070 con 0x7fa9101f9a40 2024-09-17T02:42:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa917b0c640 1 -- 172.21.15.131:0/2348252205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9101f8f60 con 0x7fa9101f9a40 2024-09-17T02:42:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa9067fc640 1 -- 172.21.15.131:0/2348252205 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00fba0 con 0x7fa9101f9a40 2024-09-17T02:42:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa917b0c640 1 -- 172.21.15.131:0/2348252205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9101f9520 con 0x7fa9101f9a40 2024-09-17T02:42:47.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.083+0000 7fa917b0c640 1 -- 172.21.15.131:0/2348252205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91015e8b0 con 0x7fa9101f9a40 2024-09-17T02:42:47.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.087+0000 7fa9067fc640 1 -- 172.21.15.131:0/2348252205 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa90c018030 con 0x7fa9101f9a40 2024-09-17T02:42:47.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.087+0000 7fa9067fc640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8f4073230 0x7fa8f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:47.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.087+0000 7fa9067fc640 1 -- 172.21.15.131:0/2348252205 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fa90c095310 con 0x7fa9101f9a40 2024-09-17T02:42:47.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.087+0000 7fa9158a8640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8f4073230 0x7fa8f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:47.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.087+0000 7fa9158a8640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8f4073230 0x7fa8f4075740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa900009ab0 tx=0x7fa9000091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.091+0000 7fa9067fc640 1 -- 172.21.15.131:0/2348252205 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa90c060280 con 0x7fa9101f9a40 2024-09-17T02:42:47.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.299+0000 7fa917b0c640 1 -- 172.21.15.131:0/2348252205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fa9100c4790 con 0x7fa9101f9a40 2024-09-17T02:42:47.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.899+0000 7fa9067fc640 1 -- 172.21.15.131:0/2348252205 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v27) ==== 122+0+0 (secure 0 0 0) 0x7fa90c05fb80 con 0x7fa9101f9a40 2024-09-17T02:42:47.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 -- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8f4073230 msgr2=0x7fa8f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:47.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8f4073230 0x7fa8f4075740 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa900009ab0 tx=0x7fa9000091e0 comp rx=0 tx=0).stop 2024-09-17T02:42:47.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 -- 172.21.15.131:0/2348252205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9101f9a40 msgr2=0x7fa9101544e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:47.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9101f9a40 0x7fa9101544e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0117a0 tx=0x7fa90c011c60 comp rx=0 tx=0).stop 2024-09-17T02:42:47.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 -- 172.21.15.131:0/2348252205 shutdown_connections 2024-09-17T02:42:47.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8f4073230 0x7fa8f4075740 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:47.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa9100cc030 0x7fa9100c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:47.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa9101f9a40 0x7fa9101544e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:47.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 --2- 172.21.15.131:0/2348252205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc004d50 0x7fa91015e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:47.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 -- 172.21.15.131:0/2348252205 >> 172.21.15.131:0/2348252205 conn(0x7fa91006eea0 msgr2=0x7fa91015c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:47.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 -- 172.21.15.131:0/2348252205 shutdown_connections 2024-09-17T02:42:47.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:47.903+0000 7fa917b0c640 1 -- 172.21.15.131:0/2348252205 wait complete. 2024-09-17T02:42:47.930 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T02:42:47.930 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.087+0000 7f3e31a66640 1 -- 172.21.15.131:0/3041254671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e14004810 msgr2=0x7f3e14004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.087+0000 7f3e31a66640 1 --2- 172.21.15.131:0/3041254671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e14004810 0x7f3e14004c20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009b60 tx=0x7f3e2001d270 comp rx=0 tx=0).stop 2024-09-17T02:42:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.087+0000 7f3e31a66640 1 -- 172.21.15.131:0/3041254671 shutdown_connections 2024-09-17T02:42:48.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.087+0000 7f3e31a66640 1 --2- 172.21.15.131:0/3041254671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e14004810 0x7f3e14004c20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.087+0000 7f3e31a66640 1 -- 172.21.15.131:0/3041254671 >> 172.21.15.131:0/3041254671 conn(0x7f3e2c06eea0 msgr2=0x7f3e2c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:48.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.087+0000 7f3e31a66640 1 -- 172.21.15.131:0/3041254671 shutdown_connections 2024-09-17T02:42:48.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.087+0000 7f3e31a66640 1 -- 172.21.15.131:0/3041254671 wait complete. 2024-09-17T02:42:48.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.087+0000 7f3e31a66640 1 Processor -- start 2024-09-17T02:42:48.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.087+0000 7f3e31a66640 1 -- start start 2024-09-17T02:42:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e31a66640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e14004810 0x7f3e2c14dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e31a66640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e2c150970 0x7f3e2c0b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e31a66640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e2c0b3e50 0x7f3e2c1fc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e31a66640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c15cbb0 con 0x7f3e14004810 2024-09-17T02:42:48.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e31a66640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2c15ca30 con 0x7f3e2c150970 2024-09-17T02:42:48.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e31a66640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3e2c0c50a0 con 0x7f3e2c0b3e50 2024-09-17T02:42:48.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e2b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e14004810 0x7f3e2c14dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e2b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e14004810 0x7f3e2c14dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58022/0 (socket says 172.21.15.131:58022) 2024-09-17T02:42:48.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e2b7fe640 1 -- 172.21.15.131:0/2445816741 learned_addr learned my addr 172.21.15.131:0/2445816741 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:48.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e2bfff640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e2c0b3e50 0x7f3e2c1fc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e2b7fe640 1 -- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e2c0b3e50 msgr2=0x7f3e2c1fc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e2b7fe640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e2c0b3e50 0x7f3e2c1fc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e2b7fe640 1 -- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e2c150970 msgr2=0x7f3e2c0b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e2affd640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e2c150970 0x7f3e2c0b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e2b7fe640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e2c150970 0x7f3e2c0b3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e2b7fe640 1 -- 172.21.15.131:0/2445816741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2c0b4260 con 0x7f3e14004810 2024-09-17T02:42:48.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e2b7fe640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e14004810 0x7f3e2c14dd80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00d780 tx=0x7f3e1c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:48.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e13fff640 1 -- 172.21.15.131:0/2445816741 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c018070 con 0x7f3e14004810 2024-09-17T02:42:48.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e13fff640 1 -- 172.21.15.131:0/2445816741 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e1c0050d0 con 0x7f3e14004810 2024-09-17T02:42:48.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e31a66640 1 -- 172.21.15.131:0/2445816741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2c0af1d0 con 0x7f3e14004810 2024-09-17T02:42:48.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e13fff640 1 -- 172.21.15.131:0/2445816741 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c00b6c0 con 0x7f3e14004810 2024-09-17T02:42:48.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e31a66640 1 -- 172.21.15.131:0/2445816741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2c0ae930 con 0x7f3e14004810 2024-09-17T02:42:48.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e31a66640 1 -- 172.21.15.131:0/2445816741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e2c0c5550 con 0x7f3e14004810 2024-09-17T02:42:48.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.091+0000 7f3e13fff640 1 -- 172.21.15.131:0/2445816741 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3e1c01b070 con 0x7f3e14004810 2024-09-17T02:42:48.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.095+0000 7f3e13fff640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3e18073200 0x7f3e18075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.095+0000 7f3e13fff640 1 -- 172.21.15.131:0/2445816741 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f3e1c093960 con 0x7f3e14004810 2024-09-17T02:42:48.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.095+0000 7f3e2affd640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3e18073200 0x7f3e18075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.095+0000 7f3e2affd640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3e18073200 0x7f3e18075710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3e200ae000 tx=0x7f3e2001a010 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:48.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.099+0000 7f3e13fff640 1 -- 172.21.15.131:0/2445816741 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3e2c0b4260 con 0x7f3e14004810 2024-09-17T02:42:48.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.295+0000 7f3e31a66640 1 -- 172.21.15.131:0/2445816741 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e2c073310 con 0x7f3e14004810 2024-09-17T02:42:48.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.295+0000 7f3e13fff640 1 -- 172.21.15.131:0/2445816741 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f3e1c05e2f0 con 0x7f3e14004810 2024-09-17T02:42:48.298 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:48.298 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"btime":"2024-09-17T02:42:47:892534+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/434043926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":434043926},{"type":"v1","addr":"172.21.15.189:6836","nonce":434043926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5445,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1496962214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1496962214},{"type":"v1","addr":"172.21.15.131:6833","nonce":1496962214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2196282015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2196282015},{"type":"v1","addr":"172.21.15.131:6835","nonce":2196282015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:42:44.218540+0000","modified":"2024-09-17T02:42:47.307497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/4030750882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":4030750882},{"type":"v1","addr":"172.21.15.189:6837","nonce":4030750882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5463,"qdb_cluster":[5463]},"id":3}]} 2024-09-17T02:42:48.299 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-09-17T02:42:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.295+0000 7f3e31a66640 1 -- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3e18073200 msgr2=0x7f3e18075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.295+0000 7f3e31a66640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3e18073200 0x7f3e18075710 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3e200ae000 tx=0x7f3e2001a010 comp rx=0 tx=0).stop 2024-09-17T02:42:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.295+0000 7f3e31a66640 1 -- 172.21.15.131:0/2445816741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e14004810 msgr2=0x7f3e2c14dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.295+0000 7f3e31a66640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e14004810 0x7f3e2c14dd80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00d780 tx=0x7f3e1c00dc40 comp rx=0 tx=0).stop 2024-09-17T02:42:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.295+0000 7f3e31a66640 1 -- 172.21.15.131:0/2445816741 shutdown_connections 2024-09-17T02:42:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.295+0000 7f3e31a66640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3e18073200 0x7f3e18075710 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.295+0000 7f3e31a66640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3e2c0b3e50 0x7f3e2c1fc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.295+0000 7f3e31a66640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3e2c150970 0x7f3e2c0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.295+0000 7f3e31a66640 1 --2- 172.21.15.131:0/2445816741 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3e14004810 0x7f3e2c14dd80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.299+0000 7f3e31a66640 1 -- 172.21.15.131:0/2445816741 >> 172.21.15.131:0/2445816741 conn(0x7f3e2c06eea0 msgr2=0x7f3e2c154b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.299+0000 7f3e31a66640 1 -- 172.21.15.131:0/2445816741 shutdown_connections 2024-09-17T02:42:48.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.299+0000 7f3e31a66640 1 -- 172.21.15.131:0/2445816741 wait complete. 2024-09-17T02:42:48.321 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:42:48.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.471+0000 7faddf8f8640 1 -- 172.21.15.131:0/1115783906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd80c4ac0 msgr2=0x7fadd80c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.471+0000 7faddf8f8640 1 --2- 172.21.15.131:0/1115783906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd80c4ac0 0x7fadd80c4ed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fadd806a1c0 tx=0x7fadc801c770 comp rx=0 tx=0).stop 2024-09-17T02:42:48.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.471+0000 7faddf8f8640 1 -- 172.21.15.131:0/1115783906 shutdown_connections 2024-09-17T02:42:48.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.471+0000 7faddf8f8640 1 --2- 172.21.15.131:0/1115783906 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd80c4ac0 0x7fadd80c4ed0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.471+0000 7faddf8f8640 1 --2- 172.21.15.131:0/1115783906 >> v2:172.21.15.189:3300/0 conn(0x7fadd806e2c0 0x7fadd80c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.471+0000 7faddf8f8640 1 -- 172.21.15.131:0/1115783906 >> 172.21.15.131:0/1115783906 conn(0x7fadd806eea0 msgr2=0x7fadd806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:48.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.471+0000 7faddf8f8640 1 -- 172.21.15.131:0/1115783906 shutdown_connections 2024-09-17T02:42:48.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.471+0000 7faddf8f8640 1 -- 172.21.15.131:0/1115783906 wait complete. 2024-09-17T02:42:48.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.471+0000 7faddf8f8640 1 Processor -- start 2024-09-17T02:42:48.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.471+0000 7faddf8f8640 1 -- start start 2024-09-17T02:42:48.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddf8f8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd806e2c0 0x7fadd80ca670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddf8f8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd80c4ac0 0x7fadd80c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddf8f8640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadd80cdbf0 0x7fadd81fc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddf8f8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fadd80cef20 con 0x7fadd806e2c0 2024-09-17T02:42:48.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddf8f8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fadd80cf0a0 con 0x7fadd80c4ac0 2024-09-17T02:42:48.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddf8f8640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fadd80cf220 con 0x7fadd80cdbf0 2024-09-17T02:42:48.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddd694640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd806e2c0 0x7fadd80ca670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddd694640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd806e2c0 0x7fadd80ca670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58046/0 (socket says 172.21.15.131:58046) 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddd694640 1 -- 172.21.15.131:0/2044353079 learned_addr learned my addr 172.21.15.131:0/2044353079 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddce93640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd80c4ac0 0x7fadd80c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddde95640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadd80cdbf0 0x7fadd81fc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddd694640 1 -- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadd80cdbf0 msgr2=0x7fadd81fc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddd694640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadd80cdbf0 0x7fadd81fc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddd694640 1 -- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd80c4ac0 msgr2=0x7fadd80c3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddd694640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd80c4ac0 0x7fadd80c3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddd694640 1 -- 172.21.15.131:0/2044353079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd80cabb0 con 0x7fadd806e2c0 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddce93640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd80c4ac0 0x7fadd80c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddd694640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd806e2c0 0x7fadd80ca670 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fadd400d780 tx=0x7fadd400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7fadcdffb640 1 -- 172.21.15.131:0/2044353079 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4017070 con 0x7fadd806e2c0 2024-09-17T02:42:48.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7fadcdffb640 1 -- 172.21.15.131:0/2044353079 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadd400de90 con 0x7fadd806e2c0 2024-09-17T02:42:48.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddf8f8640 1 -- 172.21.15.131:0/2044353079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd8151650 con 0x7fadd806e2c0 2024-09-17T02:42:48.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7fadcdffb640 1 -- 172.21.15.131:0/2044353079 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4004020 con 0x7fadd806e2c0 2024-09-17T02:42:48.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddf8f8640 1 -- 172.21.15.131:0/2044353079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd8151c10 con 0x7fadd806e2c0 2024-09-17T02:42:48.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.475+0000 7faddf8f8640 1 -- 172.21.15.131:0/2044353079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd80c4580 con 0x7fadd806e2c0 2024-09-17T02:42:48.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.479+0000 7fadcdffb640 1 -- 172.21.15.131:0/2044353079 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fadd40041c0 con 0x7fadd806e2c0 2024-09-17T02:42:48.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.479+0000 7fadcdffb640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fadbc073230 0x7fadbc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.479+0000 7fadcdffb640 1 -- 172.21.15.131:0/2044353079 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fadd4094db0 con 0x7fadd806e2c0 2024-09-17T02:42:48.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.479+0000 7faddce93640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fadbc073230 0x7fadbc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.479+0000 7faddce93640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fadbc073230 0x7fadbc075740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fadc80af000 tx=0x7fadc801afb0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:48.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.483+0000 7fadcdffb640 1 -- 172.21.15.131:0/2044353079 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fadd4020020 con 0x7fadd806e2c0 2024-09-17T02:42:48.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.651+0000 7faddf8f8640 1 -- 172.21.15.131:0/2044353079 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadd8073310 con 0x7fadd806e2c0 2024-09-17T02:42:48.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.655+0000 7fadcdffb640 1 -- 172.21.15.131:0/2044353079 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16990 (secure 0 0 0) 0x7fadd405fcd0 con 0x7fadd806e2c0 2024-09-17T02:42:48.659 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:48.659 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":52,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:42:45.860734+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:42:41.917939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T02:42:43.186384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:42:48.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 -- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fadbc073230 msgr2=0x7fadbc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fadbc073230 0x7fadbc075740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fadc80af000 tx=0x7fadc801afb0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 -- 172.21.15.131:0/2044353079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd806e2c0 msgr2=0x7fadd80ca670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd806e2c0 0x7fadd80ca670 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fadd400d780 tx=0x7fadd400dc40 comp rx=0 tx=0).stop 2024-09-17T02:42:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 -- 172.21.15.131:0/2044353079 shutdown_connections 2024-09-17T02:42:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fadbc073230 0x7fadbc075740 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fadd80cdbf0 0x7fadd81fc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fadd80c4ac0 0x7fadd80c3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 --2- 172.21.15.131:0/2044353079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fadd806e2c0 0x7fadd80ca670 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 -- 172.21.15.131:0/2044353079 >> 172.21.15.131:0/2044353079 conn(0x7fadd806eea0 msgr2=0x7fadd80b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 -- 172.21.15.131:0/2044353079 shutdown_connections 2024-09-17T02:42:48.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.659+0000 7faddf8f8640 1 -- 172.21.15.131:0/2044353079 wait complete. 2024-09-17T02:42:48.682 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T02:42:48.682 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:42:48.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.835+0000 7f2b11bee640 1 -- 172.21.15.131:0/1039119697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b0c06e5b0 msgr2=0x7f2b0c15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.835+0000 7f2b11bee640 1 --2- 172.21.15.131:0/1039119697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b0c06e5b0 0x7f2b0c15bea0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2af4009900 tx=0x7f2af401c8e0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.835+0000 7f2b11bee640 1 -- 172.21.15.131:0/1039119697 shutdown_connections 2024-09-17T02:42:48.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.835+0000 7f2b11bee640 1 --2- 172.21.15.131:0/1039119697 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b0c15c3e0 0x7f2b0c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.835+0000 7f2b11bee640 1 --2- 172.21.15.131:0/1039119697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b0c06e5b0 0x7f2b0c15bea0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.835+0000 7f2b11bee640 1 -- 172.21.15.131:0/1039119697 >> 172.21.15.131:0/1039119697 conn(0x7f2b0c06eea0 msgr2=0x7f2b0c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:48.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.835+0000 7f2b11bee640 1 -- 172.21.15.131:0/1039119697 shutdown_connections 2024-09-17T02:42:48.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.835+0000 7f2b11bee640 1 -- 172.21.15.131:0/1039119697 wait complete. 2024-09-17T02:42:48.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b11bee640 1 Processor -- start 2024-09-17T02:42:48.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b11bee640 1 -- start start 2024-09-17T02:42:48.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b11bee640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b0c0ced00 0x7f2b0c152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b11bee640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b0c15c3e0 0x7f2b0c156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b11bee640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b0c156ff0 0x7f2b0c153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b11bee640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c168890 con 0x7f2b0c0ced00 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b11bee640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c168710 con 0x7f2b0c15c3e0 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b11bee640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0c168a10 con 0x7f2b0c156ff0 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b0b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b0c0ced00 0x7f2b0c152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b0b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b0c0ced00 0x7f2b0c152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58058/0 (socket says 172.21.15.131:58058) 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b0b7fe640 1 -- 172.21.15.131:0/3496148725 learned_addr learned my addr 172.21.15.131:0/3496148725 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b03fff640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b0c15c3e0 0x7f2b0c156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b0bfff640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b0c156ff0 0x7f2b0c153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b0b7fe640 1 -- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b0c156ff0 msgr2=0x7f2b0c153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b0b7fe640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b0c156ff0 0x7f2b0c153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b0b7fe640 1 -- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b0c15c3e0 msgr2=0x7f2b0c156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:48.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b0b7fe640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b0c15c3e0 0x7f2b0c156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:48.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b0b7fe640 1 -- 172.21.15.131:0/3496148725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c160c20 con 0x7f2b0c0ced00 2024-09-17T02:42:48.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b0b7fe640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b0c0ced00 0x7f2b0c152a00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b440 tx=0x7f2afc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:48.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b08ff9640 1 -- 172.21.15.131:0/3496148725 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc015020 con 0x7f2b0c0ced00 2024-09-17T02:42:48.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b08ff9640 1 -- 172.21.15.131:0/3496148725 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2afc011980 con 0x7f2b0c0ced00 2024-09-17T02:42:48.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b08ff9640 1 -- 172.21.15.131:0/3496148725 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc011dd0 con 0x7f2b0c0ced00 2024-09-17T02:42:48.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b11bee640 1 -- 172.21.15.131:0/3496148725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0c158b90 con 0x7f2b0c0ced00 2024-09-17T02:42:48.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b11bee640 1 -- 172.21.15.131:0/3496148725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0c1578c0 con 0x7f2b0c0ced00 2024-09-17T02:42:48.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.839+0000 7f2b11bee640 1 -- 172.21.15.131:0/3496148725 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0c15bea0 con 0x7f2b0c0ced00 2024-09-17T02:42:48.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.843+0000 7f2b08ff9640 1 -- 172.21.15.131:0/3496148725 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2afc018020 con 0x7f2b0c0ced00 2024-09-17T02:42:48.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.843+0000 7f2b08ff9640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2aec073200 0x7f2aec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:48.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.843+0000 7f2b08ff9640 1 -- 172.21.15.131:0/3496148725 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f2afc0941f0 con 0x7f2b0c0ced00 2024-09-17T02:42:48.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.843+0000 7f2b03fff640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2aec073200 0x7f2aec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:48.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.843+0000 7f2b03fff640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2aec073200 0x7f2aec075710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2af40af000 tx=0x7f2af401b120 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:48.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:48.847+0000 7f2b08ff9640 1 -- 172.21.15.131:0/3496148725 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2afc01f030 con 0x7f2b0c0ced00 2024-09-17T02:42:49.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.007+0000 7f2b11bee640 1 -- 172.21.15.131:0/3496148725 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b0c15ee40 con 0x7f2aec073200 2024-09-17T02:42:49.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.023+0000 7f2b08ff9640 1 -- 172.21.15.131:0/3496148725 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251128 (secure 0 0 0) 0x7f2b0c15ee40 con 0x7f2aec073200 2024-09-17T02:42:49.028 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:49.030 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:42:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 -- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2aec073200 msgr2=0x7f2aec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2aec073200 0x7f2aec075710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2af40af000 tx=0x7f2af401b120 comp rx=0 tx=0).stop 2024-09-17T02:42:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 -- 172.21.15.131:0/3496148725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b0c0ced00 msgr2=0x7f2b0c152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b0c0ced00 0x7f2b0c152a00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b440 tx=0x7f2afc00b900 comp rx=0 tx=0).stop 2024-09-17T02:42:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 -- 172.21.15.131:0/3496148725 shutdown_connections 2024-09-17T02:42:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2aec073200 0x7f2aec075710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b0c156ff0 0x7f2b0c153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b0c15c3e0 0x7f2b0c156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 --2- 172.21.15.131:0/3496148725 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b0c0ced00 0x7f2b0c152a00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 -- 172.21.15.131:0/3496148725 >> 172.21.15.131:0/3496148725 conn(0x7f2b0c06eea0 msgr2=0x7f2b0c06dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:49.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 -- 172.21.15.131:0/3496148725 shutdown_connections 2024-09-17T02:42:49.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.027+0000 7f2b11bee640 1 -- 172.21.15.131:0/3496148725 wait complete. 2024-09-17T02:42:49.048 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-17T02:42:47.161488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238792,"kb_used_data":5892,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735928,"statfs":{"total":773094113280,"available":772849590272,"internally_reserved":0,"allocated":6033408,"data_stored":3286124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.363434"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.828438+0000","last_change":"2024-09-17T02:42:43.828438+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.828438+0000","last_undegraded":"2024-09-17T02:42:43.828438+0000","last_fullsized":"2024-09-17T02:42:43.828438+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883357+0000","last_change":"2024-09-17T02:42:44.884814+0000","last_active":"2024-09-17T02:42:45.883357+0000","last_peered":"2024-09-17T02:42:45.883357+0000","last_clean":"2024-09-17T02:42:45.883357+0000","last_became_active":"2024-09-17T02:42:44.884432+0000","last_became_peered":"2024-09-17T02:42:44.884432+0000","last_unstale":"2024-09-17T02:42:45.883357+0000","last_undegraded":"2024-09-17T02:42:45.883357+0000","last_fullsized":"2024-09-17T02:42:45.883357+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:23.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884491+0000","last_change":"2024-09-17T02:42:44.854032+0000","last_active":"2024-09-17T02:42:45.884491+0000","last_peered":"2024-09-17T02:42:45.884491+0000","last_clean":"2024-09-17T02:42:45.884491+0000","last_became_active":"2024-09-17T02:42:44.853205+0000","last_became_peered":"2024-09-17T02:42:44.853205+0000","last_unstale":"2024-09-17T02:42:45.884491+0000","last_undegraded":"2024-09-17T02:42:45.884491+0000","last_fullsized":"2024-09-17T02:42:45.884491+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:18.997673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867302+0000","last_change":"2024-09-17T02:42:44.878169+0000","last_active":"2024-09-17T02:42:45.867302+0000","last_peered":"2024-09-17T02:42:45.867302+0000","last_clean":"2024-09-17T02:42:45.867302+0000","last_became_active":"2024-09-17T02:42:44.877402+0000","last_became_peered":"2024-09-17T02:42:44.877402+0000","last_unstale":"2024-09-17T02:42:45.867302+0000","last_undegraded":"2024-09-17T02:42:45.867302+0000","last_fullsized":"2024-09-17T02:42:45.867302+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:55.948606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882642+0000","last_change":"2024-09-17T02:42:44.872147+0000","last_active":"2024-09-17T02:42:45.882642+0000","last_peered":"2024-09-17T02:42:45.882642+0000","last_clean":"2024-09-17T02:42:45.882642+0000","last_became_active":"2024-09-17T02:42:44.871745+0000","last_became_peered":"2024-09-17T02:42:44.871745+0000","last_unstale":"2024-09-17T02:42:45.882642+0000","last_undegraded":"2024-09-17T02:42:45.882642+0000","last_fullsized":"2024-09-17T02:42:45.882642+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:29.767271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882210+0000","last_change":"2024-09-17T02:42:44.885865+0000","last_active":"2024-09-17T02:42:45.882210+0000","last_peered":"2024-09-17T02:42:45.882210+0000","last_clean":"2024-09-17T02:42:45.882210+0000","last_became_active":"2024-09-17T02:42:44.885512+0000","last_became_peered":"2024-09-17T02:42:44.885512+0000","last_unstale":"2024-09-17T02:42:45.882210+0000","last_undegraded":"2024-09-17T02:42:45.882210+0000","last_fullsized":"2024-09-17T02:42:45.882210+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:56.895371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870233+0000","last_change":"2024-09-17T02:42:44.886317+0000","last_active":"2024-09-17T02:42:45.870233+0000","last_peered":"2024-09-17T02:42:45.870233+0000","last_clean":"2024-09-17T02:42:45.870233+0000","last_became_active":"2024-09-17T02:42:44.885948+0000","last_became_peered":"2024-09-17T02:42:44.885948+0000","last_unstale":"2024-09-17T02:42:45.870233+0000","last_undegraded":"2024-09-17T02:42:45.870233+0000","last_fullsized":"2024-09-17T02:42:45.870233+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:44.318617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881521+0000","last_change":"2024-09-17T02:42:44.876332+0000","last_active":"2024-09-17T02:42:45.881521+0000","last_peered":"2024-09-17T02:42:45.881521+0000","last_clean":"2024-09-17T02:42:45.881521+0000","last_became_active":"2024-09-17T02:42:44.875594+0000","last_became_peered":"2024-09-17T02:42:44.875594+0000","last_unstale":"2024-09-17T02:42:45.881521+0000","last_undegraded":"2024-09-17T02:42:45.881521+0000","last_fullsized":"2024-09-17T02:42:45.881521+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:34.865339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884540+0000","last_change":"2024-09-17T02:42:44.854764+0000","last_active":"2024-09-17T02:42:45.884540+0000","last_peered":"2024-09-17T02:42:45.884540+0000","last_clean":"2024-09-17T02:42:45.884540+0000","last_became_active":"2024-09-17T02:42:44.853822+0000","last_became_peered":"2024-09-17T02:42:44.853822+0000","last_unstale":"2024-09-17T02:42:45.884540+0000","last_undegraded":"2024-09-17T02:42:45.884540+0000","last_fullsized":"2024-09-17T02:42:45.884540+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:18.705995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873645+0000","last_change":"2024-09-17T02:42:44.889606+0000","last_active":"2024-09-17T02:42:45.873645+0000","last_peered":"2024-09-17T02:42:45.873645+0000","last_clean":"2024-09-17T02:42:45.873645+0000","last_became_active":"2024-09-17T02:42:44.889274+0000","last_became_peered":"2024-09-17T02:42:44.889274+0000","last_unstale":"2024-09-17T02:42:45.873645+0000","last_undegraded":"2024-09-17T02:42:45.873645+0000","last_fullsized":"2024-09-17T02:42:45.873645+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:12.214981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809880+0000","last_change":"2024-09-17T02:42:43.809880+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809880+0000","last_undegraded":"2024-09-17T02:42:43.809880+0000","last_fullsized":"2024-09-17T02:42:43.809880+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882022+0000","last_change":"2024-09-17T02:42:44.880065+0000","last_active":"2024-09-17T02:42:45.882022+0000","last_peered":"2024-09-17T02:42:45.882022+0000","last_clean":"2024-09-17T02:42:45.882022+0000","last_became_active":"2024-09-17T02:42:44.876260+0000","last_became_peered":"2024-09-17T02:42:44.876260+0000","last_unstale":"2024-09-17T02:42:45.882022+0000","last_undegraded":"2024-09-17T02:42:45.882022+0000","last_fullsized":"2024-09-17T02:42:45.882022+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:34:02.925523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883058+0000","last_change":"2024-09-17T02:42:44.882550+0000","last_active":"2024-09-17T02:42:45.883058+0000","last_peered":"2024-09-17T02:42:45.883058+0000","last_clean":"2024-09-17T02:42:45.883058+0000","last_became_active":"2024-09-17T02:42:44.882123+0000","last_became_peered":"2024-09-17T02:42:44.882123+0000","last_unstale":"2024-09-17T02:42:45.883058+0000","last_undegraded":"2024-09-17T02:42:45.883058+0000","last_fullsized":"2024-09-17T02:42:45.883058+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:24.890536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.812574+0000","last_change":"2024-09-17T02:42:43.812574+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.812574+0000","last_undegraded":"2024-09-17T02:42:43.812574+0000","last_fullsized":"2024-09-17T02:42:43.812574+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866923+0000","last_change":"2024-09-17T02:42:44.877349+0000","last_active":"2024-09-17T02:42:45.866923+0000","last_peered":"2024-09-17T02:42:45.866923+0000","last_clean":"2024-09-17T02:42:45.866923+0000","last_became_active":"2024-09-17T02:42:44.876816+0000","last_became_peered":"2024-09-17T02:42:44.876816+0000","last_unstale":"2024-09-17T02:42:45.866923+0000","last_undegraded":"2024-09-17T02:42:45.866923+0000","last_fullsized":"2024-09-17T02:42:45.866923+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:22.741898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881823+0000","last_change":"2024-09-17T02:42:44.876906+0000","last_active":"2024-09-17T02:42:45.881823+0000","last_peered":"2024-09-17T02:42:45.881823+0000","last_clean":"2024-09-17T02:42:45.881823+0000","last_became_active":"2024-09-17T02:42:44.875966+0000","last_became_peered":"2024-09-17T02:42:44.875966+0000","last_unstale":"2024-09-17T02:42:45.881823+0000","last_undegraded":"2024-09-17T02:42:45.881823+0000","last_fullsized":"2024-09-17T02:42:45.881823+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:06.983520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.813038+0000","last_change":"2024-09-17T02:42:43.813038+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.813038+0000","last_undegraded":"2024-09-17T02:42:43.813038+0000","last_fullsized":"2024-09-17T02:42:43.813038+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867528+0000","last_change":"2024-09-17T02:42:44.888487+0000","last_active":"2024-09-17T02:42:45.867528+0000","last_peered":"2024-09-17T02:42:45.867528+0000","last_clean":"2024-09-17T02:42:45.867528+0000","last_became_active":"2024-09-17T02:42:44.887970+0000","last_became_peered":"2024-09-17T02:42:44.887970+0000","last_unstale":"2024-09-17T02:42:45.867528+0000","last_undegraded":"2024-09-17T02:42:45.867528+0000","last_fullsized":"2024-09-17T02:42:45.867528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:42.359754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884599+0000","last_change":"2024-09-17T02:42:44.881307+0000","last_active":"2024-09-17T02:42:45.884599+0000","last_peered":"2024-09-17T02:42:45.884599+0000","last_clean":"2024-09-17T02:42:45.884599+0000","last_became_active":"2024-09-17T02:42:44.880605+0000","last_became_peered":"2024-09-17T02:42:44.880605+0000","last_unstale":"2024-09-17T02:42:45.884599+0000","last_undegraded":"2024-09-17T02:42:45.884599+0000","last_fullsized":"2024-09-17T02:42:45.884599+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:49.563614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883282+0000","last_change":"2024-09-17T02:42:44.887466+0000","last_active":"2024-09-17T02:42:45.883282+0000","last_peered":"2024-09-17T02:42:45.883282+0000","last_clean":"2024-09-17T02:42:45.883282+0000","last_became_active":"2024-09-17T02:42:44.885044+0000","last_became_peered":"2024-09-17T02:42:44.885044+0000","last_unstale":"2024-09-17T02:42:45.883282+0000","last_undegraded":"2024-09-17T02:42:45.883282+0000","last_fullsized":"2024-09-17T02:42:45.883282+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.263210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867244+0000","last_change":"2024-09-17T02:42:44.876044+0000","last_active":"2024-09-17T02:42:45.867244+0000","last_peered":"2024-09-17T02:42:45.867244+0000","last_clean":"2024-09-17T02:42:45.867244+0000","last_became_active":"2024-09-17T02:42:44.873541+0000","last_became_peered":"2024-09-17T02:42:44.873541+0000","last_unstale":"2024-09-17T02:42:45.867244+0000","last_undegraded":"2024-09-17T02:42:45.867244+0000","last_fullsized":"2024-09-17T02:42:45.867244+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:56.895371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883856+0000","last_change":"2024-09-17T02:42:44.879337+0000","last_active":"2024-09-17T02:42:45.883856+0000","last_peered":"2024-09-17T02:42:45.883856+0000","last_clean":"2024-09-17T02:42:45.883856+0000","last_became_active":"2024-09-17T02:42:44.878982+0000","last_became_peered":"2024-09-17T02:42:44.878982+0000","last_unstale":"2024-09-17T02:42:45.883856+0000","last_undegraded":"2024-09-17T02:42:45.883856+0000","last_fullsized":"2024-09-17T02:42:45.883856+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:59.473102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882578+0000","last_change":"2024-09-17T02:42:44.856410+0000","last_active":"2024-09-17T02:42:45.882578+0000","last_peered":"2024-09-17T02:42:45.882578+0000","last_clean":"2024-09-17T02:42:45.882578+0000","last_became_active":"2024-09-17T02:42:44.854487+0000","last_became_peered":"2024-09-17T02:42:44.854487+0000","last_unstale":"2024-09-17T02:42:45.882578+0000","last_undegraded":"2024-09-17T02:42:45.882578+0000","last_fullsized":"2024-09-17T02:42:45.882578+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:07.107071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882985+0000","last_change":"2024-09-17T02:42:44.870968+0000","last_active":"2024-09-17T02:42:45.882985+0000","last_peered":"2024-09-17T02:42:45.882985+0000","last_clean":"2024-09-17T02:42:45.882985+0000","last_became_active":"2024-09-17T02:42:44.870254+0000","last_became_peered":"2024-09-17T02:42:44.870254+0000","last_unstale":"2024-09-17T02:42:45.882985+0000","last_undegraded":"2024-09-17T02:42:45.882985+0000","last_fullsized":"2024-09-17T02:42:45.882985+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:15.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.826441+0000","last_change":"2024-09-17T02:42:43.826441+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.826441+0000","last_undegraded":"2024-09-17T02:42:43.826441+0000","last_fullsized":"2024-09-17T02:42:43.826441+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884665+0000","last_change":"2024-09-17T02:42:44.875582+0000","last_active":"2024-09-17T02:42:45.884665+0000","last_peered":"2024-09-17T02:42:45.884665+0000","last_clean":"2024-09-17T02:42:45.884665+0000","last_became_active":"2024-09-17T02:42:44.875205+0000","last_became_peered":"2024-09-17T02:42:44.875205+0000","last_unstale":"2024-09-17T02:42:45.884665+0000","last_undegraded":"2024-09-17T02:42:45.884665+0000","last_fullsized":"2024-09-17T02:42:45.884665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:22:23.818837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881878+0000","last_change":"2024-09-17T02:42:44.885311+0000","last_active":"2024-09-17T02:42:45.881878+0000","last_peered":"2024-09-17T02:42:45.881878+0000","last_clean":"2024-09-17T02:42:45.881878+0000","last_became_active":"2024-09-17T02:42:44.884647+0000","last_became_peered":"2024-09-17T02:42:44.884647+0000","last_unstale":"2024-09-17T02:42:45.881878+0000","last_undegraded":"2024-09-17T02:42:45.881878+0000","last_fullsized":"2024-09-17T02:42:45.881878+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:20.906053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883665+0000","last_change":"2024-09-17T02:42:44.877183+0000","last_active":"2024-09-17T02:42:45.883665+0000","last_peered":"2024-09-17T02:42:45.883665+0000","last_clean":"2024-09-17T02:42:45.883665+0000","last_became_active":"2024-09-17T02:42:44.875043+0000","last_became_peered":"2024-09-17T02:42:44.875043+0000","last_unstale":"2024-09-17T02:42:45.883665+0000","last_undegraded":"2024-09-17T02:42:45.883665+0000","last_fullsized":"2024-09-17T02:42:45.883665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:53.445161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809918+0000","last_change":"2024-09-17T02:42:43.809918+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809918+0000","last_undegraded":"2024-09-17T02:42:43.809918+0000","last_fullsized":"2024-09-17T02:42:43.809918+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883971+0000","last_change":"2024-09-17T02:42:44.869378+0000","last_active":"2024-09-17T02:42:45.883971+0000","last_peered":"2024-09-17T02:42:45.883971+0000","last_clean":"2024-09-17T02:42:45.883971+0000","last_became_active":"2024-09-17T02:42:44.868147+0000","last_became_peered":"2024-09-17T02:42:44.868147+0000","last_unstale":"2024-09-17T02:42:45.883971+0000","last_undegraded":"2024-09-17T02:42:45.883971+0000","last_fullsized":"2024-09-17T02:42:45.883971+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:03.959077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882887+0000","last_change":"2024-09-17T02:42:44.879236+0000","last_active":"2024-09-17T02:42:45.882887+0000","last_peered":"2024-09-17T02:42:45.882887+0000","last_clean":"2024-09-17T02:42:45.882887+0000","last_became_active":"2024-09-17T02:42:44.878872+0000","last_became_peered":"2024-09-17T02:42:44.878872+0000","last_unstale":"2024-09-17T02:42:45.882887+0000","last_undegraded":"2024-09-17T02:42:45.882887+0000","last_fullsized":"2024-09-17T02:42:45.882887+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:39.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882661+0000","last_change":"2024-09-17T02:42:44.853789+0000","last_active":"2024-09-17T02:42:45.882661+0000","last_peered":"2024-09-17T02:42:45.882661+0000","last_clean":"2024-09-17T02:42:45.882661+0000","last_became_active":"2024-09-17T02:42:44.852870+0000","last_became_peered":"2024-09-17T02:42:44.852870+0000","last_unstale":"2024-09-17T02:42:45.882661+0000","last_undegraded":"2024-09-17T02:42:45.882661+0000","last_fullsized":"2024-09-17T02:42:45.882661+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.016019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882301+0000","last_change":"2024-09-17T02:42:44.884160+0000","last_active":"2024-09-17T02:42:45.882301+0000","last_peered":"2024-09-17T02:42:45.882301+0000","last_clean":"2024-09-17T02:42:45.882301+0000","last_became_active":"2024-09-17T02:42:44.883295+0000","last_became_peered":"2024-09-17T02:42:44.883295+0000","last_unstale":"2024-09-17T02:42:45.882301+0000","last_undegraded":"2024-09-17T02:42:45.882301+0000","last_fullsized":"2024-09-17T02:42:45.882301+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:13.871419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":19,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885185+0000","last_change":"2024-09-17T02:42:43.845735+0000","last_active":"2024-09-17T02:42:45.885185+0000","last_peered":"2024-09-17T02:42:45.885185+0000","last_clean":"2024-09-17T02:42:45.885185+0000","last_became_active":"2024-09-17T02:42:43.845485+0000","last_became_peered":"2024-09-17T02:42:43.845485+0000","last_unstale":"2024-09-17T02:42:45.885185+0000","last_undegraded":"2024-09-17T02:42:45.885185+0000","last_fullsized":"2024-09-17T02:42:45.885185+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:19.775974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881559+0000","last_change":"2024-09-17T02:42:44.875131+0000","last_active":"2024-09-17T02:42:45.881559+0000","last_peered":"2024-09-17T02:42:45.881559+0000","last_clean":"2024-09-17T02:42:45.881559+0000","last_became_active":"2024-09-17T02:42:44.874308+0000","last_became_peered":"2024-09-17T02:42:44.874308+0000","last_unstale":"2024-09-17T02:42:45.881559+0000","last_undegraded":"2024-09-17T02:42:45.881559+0000","last_fullsized":"2024-09-17T02:42:45.881559+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:05.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882119+0000","last_change":"2024-09-17T02:42:43.865164+0000","last_active":"2024-09-17T02:42:45.882119+0000","last_peered":"2024-09-17T02:42:45.882119+0000","last_clean":"2024-09-17T02:42:45.882119+0000","last_became_active":"2024-09-17T02:42:43.862678+0000","last_became_peered":"2024-09-17T02:42:43.862678+0000","last_unstale":"2024-09-17T02:42:45.882119+0000","last_undegraded":"2024-09-17T02:42:45.882119+0000","last_fullsized":"2024-09-17T02:42:45.882119+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:46.632441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883731+0000","last_change":"2024-09-17T02:42:44.870094+0000","last_active":"2024-09-17T02:42:45.883731+0000","last_peered":"2024-09-17T02:42:45.883731+0000","last_clean":"2024-09-17T02:42:45.883731+0000","last_became_active":"2024-09-17T02:42:44.869245+0000","last_became_peered":"2024-09-17T02:42:44.869245+0000","last_unstale":"2024-09-17T02:42:45.883731+0000","last_undegraded":"2024-09-17T02:42:45.883731+0000","last_fullsized":"2024-09-17T02:42:45.883731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:18.419470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873178+0000","last_change":"2024-09-17T02:42:43.837893+0000","last_active":"2024-09-17T02:42:45.873178+0000","last_peered":"2024-09-17T02:42:45.873178+0000","last_clean":"2024-09-17T02:42:45.873178+0000","last_became_active":"2024-09-17T02:42:43.833094+0000","last_became_peered":"2024-09-17T02:42:43.833094+0000","last_unstale":"2024-09-17T02:42:45.873178+0000","last_undegraded":"2024-09-17T02:42:45.873178+0000","last_fullsized":"2024-09-17T02:42:45.873178+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:27.547510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870591+0000","last_change":"2024-09-17T02:42:44.876821+0000","last_active":"2024-09-17T02:42:45.870591+0000","last_peered":"2024-09-17T02:42:45.870591+0000","last_clean":"2024-09-17T02:42:45.870591+0000","last_became_active":"2024-09-17T02:42:44.876427+0000","last_became_peered":"2024-09-17T02:42:44.876427+0000","last_unstale":"2024-09-17T02:42:45.870591+0000","last_undegraded":"2024-09-17T02:42:45.870591+0000","last_fullsized":"2024-09-17T02:42:45.870591+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:29.285476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884021+0000","last_change":"2024-09-17T02:42:43.859536+0000","last_active":"2024-09-17T02:42:45.884021+0000","last_peered":"2024-09-17T02:42:45.884021+0000","last_clean":"2024-09-17T02:42:45.884021+0000","last_became_active":"2024-09-17T02:42:43.859191+0000","last_became_peered":"2024-09-17T02:42:43.859191+0000","last_unstale":"2024-09-17T02:42:45.884021+0000","last_undegraded":"2024-09-17T02:42:45.884021+0000","last_fullsized":"2024-09-17T02:42:45.884021+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:34.772218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.886237+0000","last_change":"2024-09-17T02:42:44.881758+0000","last_active":"2024-09-17T02:42:45.886237+0000","last_peered":"2024-09-17T02:42:45.886237+0000","last_clean":"2024-09-17T02:42:45.886237+0000","last_became_active":"2024-09-17T02:42:44.880037+0000","last_became_peered":"2024-09-17T02:42:44.880037+0000","last_unstale":"2024-09-17T02:42:45.886237+0000","last_undegraded":"2024-09-17T02:42:45.886237+0000","last_fullsized":"2024-09-17T02:42:45.886237+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:49.224409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884002+0000","last_change":"2024-09-17T02:42:43.859603+0000","last_active":"2024-09-17T02:42:45.884002+0000","last_peered":"2024-09-17T02:42:45.884002+0000","last_clean":"2024-09-17T02:42:45.884002+0000","last_became_active":"2024-09-17T02:42:43.856481+0000","last_became_peered":"2024-09-17T02:42:43.856481+0000","last_unstale":"2024-09-17T02:42:45.884002+0000","last_undegraded":"2024-09-17T02:42:45.884002+0000","last_fullsized":"2024-09-17T02:42:45.884002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:31.985885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884898+0000","last_change":"2024-09-17T02:42:44.891687+0000","last_active":"2024-09-17T02:42:45.884898+0000","last_peered":"2024-09-17T02:42:45.884898+0000","last_clean":"2024-09-17T02:42:45.884898+0000","last_became_active":"2024-09-17T02:42:44.890077+0000","last_became_peered":"2024-09-17T02:42:44.890077+0000","last_unstale":"2024-09-17T02:42:45.884898+0000","last_undegraded":"2024-09-17T02:42:45.884898+0000","last_fullsized":"2024-09-17T02:42:45.884898+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:46:21.265517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884101+0000","last_change":"2024-09-17T02:42:43.822676+0000","last_active":"2024-09-17T02:42:45.884101+0000","last_peered":"2024-09-17T02:42:45.884101+0000","last_clean":"2024-09-17T02:42:45.884101+0000","last_became_active":"2024-09-17T02:42:43.822225+0000","last_became_peered":"2024-09-17T02:42:43.822225+0000","last_unstale":"2024-09-17T02:42:45.884101+0000","last_undegraded":"2024-09-17T02:42:45.884101+0000","last_fullsized":"2024-09-17T02:42:45.884101+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:38.838275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873695+0000","last_change":"2024-09-17T02:42:44.880963+0000","last_active":"2024-09-17T02:42:45.873695+0000","last_peered":"2024-09-17T02:42:45.873695+0000","last_clean":"2024-09-17T02:42:45.873695+0000","last_became_active":"2024-09-17T02:42:44.880257+0000","last_became_peered":"2024-09-17T02:42:44.880257+0000","last_unstale":"2024-09-17T02:42:45.873695+0000","last_undegraded":"2024-09-17T02:42:45.873695+0000","last_fullsized":"2024-09-17T02:42:45.873695+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:09.416640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885098+0000","last_change":"2024-09-17T02:42:43.814090+0000","last_active":"2024-09-17T02:42:45.885098+0000","last_peered":"2024-09-17T02:42:45.885098+0000","last_clean":"2024-09-17T02:42:45.885098+0000","last_became_active":"2024-09-17T02:42:43.813685+0000","last_became_peered":"2024-09-17T02:42:43.813685+0000","last_unstale":"2024-09-17T02:42:45.885098+0000","last_undegraded":"2024-09-17T02:42:45.885098+0000","last_fullsized":"2024-09-17T02:42:45.885098+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:06.103811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867600+0000","last_change":"2024-09-17T02:42:44.878439+0000","last_active":"2024-09-17T02:42:45.867600+0000","last_peered":"2024-09-17T02:42:45.867600+0000","last_clean":"2024-09-17T02:42:45.867600+0000","last_became_active":"2024-09-17T02:42:44.875651+0000","last_became_peered":"2024-09-17T02:42:44.875651+0000","last_unstale":"2024-09-17T02:42:45.867600+0000","last_undegraded":"2024-09-17T02:42:45.867600+0000","last_fullsized":"2024-09-17T02:42:45.867600+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:29.767271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883310+0000","last_change":"2024-09-17T02:42:43.855843+0000","last_active":"2024-09-17T02:42:45.883310+0000","last_peered":"2024-09-17T02:42:45.883310+0000","last_clean":"2024-09-17T02:42:45.883310+0000","last_became_active":"2024-09-17T02:42:43.854712+0000","last_became_peered":"2024-09-17T02:42:43.854712+0000","last_unstale":"2024-09-17T02:42:45.883310+0000","last_undegraded":"2024-09-17T02:42:45.883310+0000","last_fullsized":"2024-09-17T02:42:45.883310+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:19:02.429366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884260+0000","last_change":"2024-09-17T02:42:44.875385+0000","last_active":"2024-09-17T02:42:45.884260+0000","last_peered":"2024-09-17T02:42:45.884260+0000","last_clean":"2024-09-17T02:42:45.884260+0000","last_became_active":"2024-09-17T02:42:44.874816+0000","last_became_peered":"2024-09-17T02:42:44.874816+0000","last_unstale":"2024-09-17T02:42:45.884260+0000","last_undegraded":"2024-09-17T02:42:45.884260+0000","last_fullsized":"2024-09-17T02:42:45.884260+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:36.940445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867765+0000","last_change":"2024-09-17T02:42:43.826860+0000","last_active":"2024-09-17T02:42:45.867765+0000","last_peered":"2024-09-17T02:42:45.867765+0000","last_clean":"2024-09-17T02:42:45.867765+0000","last_became_active":"2024-09-17T02:42:43.826423+0000","last_became_peered":"2024-09-17T02:42:43.826423+0000","last_unstale":"2024-09-17T02:42:45.867765+0000","last_undegraded":"2024-09-17T02:42:45.867765+0000","last_fullsized":"2024-09-17T02:42:45.867765+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:57.315329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866970+0000","last_change":"2024-09-17T02:42:44.889033+0000","last_active":"2024-09-17T02:42:45.866970+0000","last_peered":"2024-09-17T02:42:45.866970+0000","last_clean":"2024-09-17T02:42:45.866970+0000","last_became_active":"2024-09-17T02:42:44.888575+0000","last_became_peered":"2024-09-17T02:42:44.888575+0000","last_unstale":"2024-09-17T02:42:45.866970+0000","last_undegraded":"2024-09-17T02:42:45.866970+0000","last_fullsized":"2024-09-17T02:42:45.866970+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:10.824751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867254+0000","last_change":"2024-09-17T02:42:43.835429+0000","last_active":"2024-09-17T02:42:45.867254+0000","last_peered":"2024-09-17T02:42:45.867254+0000","last_clean":"2024-09-17T02:42:45.867254+0000","last_became_active":"2024-09-17T02:42:43.835072+0000","last_became_peered":"2024-09-17T02:42:43.835072+0000","last_unstale":"2024-09-17T02:42:45.867254+0000","last_undegraded":"2024-09-17T02:42:45.867254+0000","last_fullsized":"2024-09-17T02:42:45.867254+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:28.861850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.807944+0000","last_change":"2024-09-17T02:42:43.807944+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.807944+0000","last_undegraded":"2024-09-17T02:42:43.807944+0000","last_fullsized":"2024-09-17T02:42:43.807944+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885035+0000","last_change":"2024-09-17T02:42:43.844104+0000","last_active":"2024-09-17T02:42:45.885035+0000","last_peered":"2024-09-17T02:42:45.885035+0000","last_clean":"2024-09-17T02:42:45.885035+0000","last_became_active":"2024-09-17T02:42:43.843661+0000","last_became_peered":"2024-09-17T02:42:43.843661+0000","last_unstale":"2024-09-17T02:42:45.885035+0000","last_undegraded":"2024-09-17T02:42:45.885035+0000","last_fullsized":"2024-09-17T02:42:45.885035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:14.987141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882393+0000","last_change":"2024-09-17T02:42:44.853449+0000","last_active":"2024-09-17T02:42:45.882393+0000","last_peered":"2024-09-17T02:42:45.882393+0000","last_clean":"2024-09-17T02:42:45.882393+0000","last_became_active":"2024-09-17T02:42:44.852561+0000","last_became_peered":"2024-09-17T02:42:44.852561+0000","last_unstale":"2024-09-17T02:42:45.882393+0000","last_undegraded":"2024-09-17T02:42:45.882393+0000","last_fullsized":"2024-09-17T02:42:45.882393+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.871956+0000","last_change":"2024-09-17T02:42:43.864191+0000","last_active":"2024-09-17T02:42:45.871956+0000","last_peered":"2024-09-17T02:42:45.871956+0000","last_clean":"2024-09-17T02:42:45.871956+0000","last_became_active":"2024-09-17T02:42:43.863400+0000","last_became_peered":"2024-09-17T02:42:43.863400+0000","last_unstale":"2024-09-17T02:42:45.871956+0000","last_undegraded":"2024-09-17T02:42:45.871956+0000","last_fullsized":"2024-09-17T02:42:45.871956+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:09.821491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885484+0000","last_change":"2024-09-17T02:42:44.880368+0000","last_active":"2024-09-17T02:42:45.885484+0000","last_peered":"2024-09-17T02:42:45.885484+0000","last_clean":"2024-09-17T02:42:45.885484+0000","last_became_active":"2024-09-17T02:42:44.879881+0000","last_became_peered":"2024-09-17T02:42:44.879881+0000","last_unstale":"2024-09-17T02:42:45.885484+0000","last_undegraded":"2024-09-17T02:42:45.885484+0000","last_fullsized":"2024-09-17T02:42:45.885484+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:58.252210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883189+0000","last_change":"2024-09-17T02:42:43.841976+0000","last_active":"2024-09-17T02:42:45.883189+0000","last_peered":"2024-09-17T02:42:45.883189+0000","last_clean":"2024-09-17T02:42:45.883189+0000","last_became_active":"2024-09-17T02:42:43.841094+0000","last_became_peered":"2024-09-17T02:42:43.841094+0000","last_unstale":"2024-09-17T02:42:45.883189+0000","last_undegraded":"2024-09-17T02:42:45.883189+0000","last_fullsized":"2024-09-17T02:42:45.883189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:29.331958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.808614+0000","last_change":"2024-09-17T02:42:43.808614+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.808614+0000","last_undegraded":"2024-09-17T02:42:43.808614+0000","last_fullsized":"2024-09-17T02:42:43.808614+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866984+0000","last_change":"2024-09-17T02:42:43.843349+0000","last_active":"2024-09-17T02:42:45.866984+0000","last_peered":"2024-09-17T02:42:45.866984+0000","last_clean":"2024-09-17T02:42:45.866984+0000","last_became_active":"2024-09-17T02:42:43.842950+0000","last_became_peered":"2024-09-17T02:42:43.842950+0000","last_unstale":"2024-09-17T02:42:45.866984+0000","last_undegraded":"2024-09-17T02:42:45.866984+0000","last_fullsized":"2024-09-17T02:42:45.866984+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:22.974854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882439+0000","last_change":"2024-09-17T02:42:44.886488+0000","last_active":"2024-09-17T02:42:45.882439+0000","last_peered":"2024-09-17T02:42:45.882439+0000","last_clean":"2024-09-17T02:42:45.882439+0000","last_became_active":"2024-09-17T02:42:44.886106+0000","last_became_peered":"2024-09-17T02:42:44.886106+0000","last_unstale":"2024-09-17T02:42:45.882439+0000","last_undegraded":"2024-09-17T02:42:45.882439+0000","last_fullsized":"2024-09-17T02:42:45.882439+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:55.948606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881989+0000","last_change":"2024-09-17T02:42:43.825883+0000","last_active":"2024-09-17T02:42:45.881989+0000","last_peered":"2024-09-17T02:42:45.881989+0000","last_clean":"2024-09-17T02:42:45.881989+0000","last_became_active":"2024-09-17T02:42:43.825522+0000","last_became_peered":"2024-09-17T02:42:43.825522+0000","last_unstale":"2024-09-17T02:42:45.881989+0000","last_undegraded":"2024-09-17T02:42:45.881989+0000","last_fullsized":"2024-09-17T02:42:45.881989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:22:57.287148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885037+0000","last_change":"2024-09-17T02:42:44.878272+0000","last_active":"2024-09-17T02:42:45.885037+0000","last_peered":"2024-09-17T02:42:45.885037+0000","last_clean":"2024-09-17T02:42:45.885037+0000","last_became_active":"2024-09-17T02:42:44.877921+0000","last_became_peered":"2024-09-17T02:42:44.877921+0000","last_unstale":"2024-09-17T02:42:45.885037+0000","last_undegraded":"2024-09-17T02:42:45.885037+0000","last_fullsized":"2024-09-17T02:42:45.885037+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:14.742444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":23,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881967+0000","last_change":"2024-09-17T02:42:43.864045+0000","last_active":"2024-09-17T02:42:45.881967+0000","last_peered":"2024-09-17T02:42:45.881967+0000","last_clean":"2024-09-17T02:42:45.881967+0000","last_became_active":"2024-09-17T02:42:43.862689+0000","last_became_peered":"2024-09-17T02:42:43.862689+0000","last_unstale":"2024-09-17T02:42:45.881967+0000","last_undegraded":"2024-09-17T02:42:45.881967+0000","last_fullsized":"2024-09-17T02:42:45.881967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:30.667789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870056+0000","last_change":"2024-09-17T02:42:44.871420+0000","last_active":"2024-09-17T02:42:45.870056+0000","last_peered":"2024-09-17T02:42:45.870056+0000","last_clean":"2024-09-17T02:42:45.870056+0000","last_became_active":"2024-09-17T02:42:44.871075+0000","last_became_peered":"2024-09-17T02:42:44.871075+0000","last_unstale":"2024-09-17T02:42:45.870056+0000","last_undegraded":"2024-09-17T02:42:45.870056+0000","last_fullsized":"2024-09-17T02:42:45.870056+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:21.340145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882977+0000","last_change":"2024-09-17T02:42:43.865760+0000","last_active":"2024-09-17T02:42:45.882977+0000","last_peered":"2024-09-17T02:42:45.882977+0000","last_clean":"2024-09-17T02:42:45.882977+0000","last_became_active":"2024-09-17T02:42:43.862428+0000","last_became_peered":"2024-09-17T02:42:43.862428+0000","last_unstale":"2024-09-17T02:42:45.882977+0000","last_undegraded":"2024-09-17T02:42:45.882977+0000","last_fullsized":"2024-09-17T02:42:45.882977+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:00.644920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809542+0000","last_change":"2024-09-17T02:42:43.809542+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809542+0000","last_undegraded":"2024-09-17T02:42:43.809542+0000","last_fullsized":"2024-09-17T02:42:43.809542+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882280+0000","last_change":"2024-09-17T02:42:43.839506+0000","last_active":"2024-09-17T02:42:45.882280+0000","last_peered":"2024-09-17T02:42:45.882280+0000","last_clean":"2024-09-17T02:42:45.882280+0000","last_became_active":"2024-09-17T02:42:43.838617+0000","last_became_peered":"2024-09-17T02:42:43.838617+0000","last_unstale":"2024-09-17T02:42:45.882280+0000","last_undegraded":"2024-09-17T02:42:45.882280+0000","last_fullsized":"2024-09-17T02:42:45.882280+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:04.244931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883433+0000","last_change":"2024-09-17T02:42:44.876771+0000","last_active":"2024-09-17T02:42:45.883433+0000","last_peered":"2024-09-17T02:42:45.883433+0000","last_clean":"2024-09-17T02:42:45.883433+0000","last_became_active":"2024-09-17T02:42:44.876438+0000","last_became_peered":"2024-09-17T02:42:44.876438+0000","last_unstale":"2024-09-17T02:42:45.883433+0000","last_undegraded":"2024-09-17T02:42:45.883433+0000","last_fullsized":"2024-09-17T02:42:45.883433+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:28.921453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867493+0000","last_change":"2024-09-17T02:42:43.862172+0000","last_active":"2024-09-17T02:42:45.867493+0000","last_peered":"2024-09-17T02:42:45.867493+0000","last_clean":"2024-09-17T02:42:45.867493+0000","last_became_active":"2024-09-17T02:42:43.861853+0000","last_became_peered":"2024-09-17T02:42:43.861853+0000","last_unstale":"2024-09-17T02:42:45.867493+0000","last_undegraded":"2024-09-17T02:42:45.867493+0000","last_fullsized":"2024-09-17T02:42:45.867493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:00.644920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883740+0000","last_change":"2024-09-17T02:42:44.875816+0000","last_active":"2024-09-17T02:42:45.883740+0000","last_peered":"2024-09-17T02:42:45.883740+0000","last_clean":"2024-09-17T02:42:45.883740+0000","last_became_active":"2024-09-17T02:42:44.875443+0000","last_became_peered":"2024-09-17T02:42:44.875443+0000","last_unstale":"2024-09-17T02:42:45.883740+0000","last_undegraded":"2024-09-17T02:42:45.883740+0000","last_fullsized":"2024-09-17T02:42:45.883740+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.263210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884349+0000","last_change":"2024-09-17T02:42:43.842888+0000","last_active":"2024-09-17T02:42:45.884349+0000","last_peered":"2024-09-17T02:42:45.884349+0000","last_clean":"2024-09-17T02:42:45.884349+0000","last_became_active":"2024-09-17T02:42:43.842563+0000","last_became_peered":"2024-09-17T02:42:43.842563+0000","last_unstale":"2024-09-17T02:42:45.884349+0000","last_undegraded":"2024-09-17T02:42:45.884349+0000","last_fullsized":"2024-09-17T02:42:45.884349+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:37.379782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883984+0000","last_change":"2024-09-17T02:42:44.875239+0000","last_active":"2024-09-17T02:42:45.883984+0000","last_peered":"2024-09-17T02:42:45.883984+0000","last_clean":"2024-09-17T02:42:45.883984+0000","last_became_active":"2024-09-17T02:42:44.874268+0000","last_became_peered":"2024-09-17T02:42:44.874268+0000","last_unstale":"2024-09-17T02:42:45.883984+0000","last_undegraded":"2024-09-17T02:42:45.883984+0000","last_fullsized":"2024-09-17T02:42:45.883984+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:30.019802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T02:42:43.849754+0000","last_change":"2024-09-17T02:42:43.840810+0000","last_active":"2024-09-17T02:42:43.849754+0000","last_peered":"2024-09-17T02:42:43.849754+0000","last_clean":"2024-09-17T02:42:43.849754+0000","last_became_active":"2024-09-17T02:42:43.839861+0000","last_became_peered":"2024-09-17T02:42:43.839861+0000","last_unstale":"2024-09-17T02:42:43.849754+0000","last_undegraded":"2024-09-17T02:42:43.849754+0000","last_fullsized":"2024-09-17T02:42:43.849754+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:14.684387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867465+0000","last_change":"2024-09-17T02:42:44.877843+0000","last_active":"2024-09-17T02:42:45.867465+0000","last_peered":"2024-09-17T02:42:45.867465+0000","last_clean":"2024-09-17T02:42:45.867465+0000","last_became_active":"2024-09-17T02:42:44.877029+0000","last_became_peered":"2024-09-17T02:42:44.877029+0000","last_unstale":"2024-09-17T02:42:45.867465+0000","last_undegraded":"2024-09-17T02:42:45.867465+0000","last_fullsized":"2024-09-17T02:42:45.867465+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883764+0000","last_change":"2024-09-17T02:42:43.860370+0000","last_active":"2024-09-17T02:42:45.883764+0000","last_peered":"2024-09-17T02:42:45.883764+0000","last_clean":"2024-09-17T02:42:45.883764+0000","last_became_active":"2024-09-17T02:42:43.859094+0000","last_became_peered":"2024-09-17T02:42:43.859094+0000","last_unstale":"2024-09-17T02:42:45.883764+0000","last_undegraded":"2024-09-17T02:42:45.883764+0000","last_fullsized":"2024-09-17T02:42:45.883764+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:15.987844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.810796+0000","last_change":"2024-09-17T02:42:43.810796+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.810796+0000","last_undegraded":"2024-09-17T02:42:43.810796+0000","last_fullsized":"2024-09-17T02:42:43.810796+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870452+0000","last_change":"2024-09-17T02:42:43.862720+0000","last_active":"2024-09-17T02:42:45.870452+0000","last_peered":"2024-09-17T02:42:45.870452+0000","last_clean":"2024-09-17T02:42:45.870452+0000","last_became_active":"2024-09-17T02:42:43.862004+0000","last_became_peered":"2024-09-17T02:42:43.862004+0000","last_unstale":"2024-09-17T02:42:45.870452+0000","last_undegraded":"2024-09-17T02:42:45.870452+0000","last_fullsized":"2024-09-17T02:42:45.870452+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:41.356494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882795+0000","last_change":"2024-09-17T02:42:44.883533+0000","last_active":"2024-09-17T02:42:45.882795+0000","last_peered":"2024-09-17T02:42:45.882795+0000","last_clean":"2024-09-17T02:42:45.882795+0000","last_became_active":"2024-09-17T02:42:44.883166+0000","last_became_peered":"2024-09-17T02:42:44.883166+0000","last_unstale":"2024-09-17T02:42:45.882795+0000","last_undegraded":"2024-09-17T02:42:45.882795+0000","last_fullsized":"2024-09-17T02:42:45.882795+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.016019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.869776+0000","last_change":"2024-09-17T02:42:43.831772+0000","last_active":"2024-09-17T02:42:45.869776+0000","last_peered":"2024-09-17T02:42:45.869776+0000","last_clean":"2024-09-17T02:42:45.869776+0000","last_became_active":"2024-09-17T02:42:43.830906+0000","last_became_peered":"2024-09-17T02:42:43.830906+0000","last_unstale":"2024-09-17T02:42:45.869776+0000","last_undegraded":"2024-09-17T02:42:45.869776+0000","last_fullsized":"2024-09-17T02:42:45.869776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:43.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866625+0000","last_change":"2024-09-17T02:42:44.878871+0000","last_active":"2024-09-17T02:42:45.866625+0000","last_peered":"2024-09-17T02:42:45.866625+0000","last_clean":"2024-09-17T02:42:45.866625+0000","last_became_active":"2024-09-17T02:42:44.878370+0000","last_became_peered":"2024-09-17T02:42:44.878370+0000","last_unstale":"2024-09-17T02:42:45.866625+0000","last_undegraded":"2024-09-17T02:42:45.866625+0000","last_fullsized":"2024-09-17T02:42:45.866625+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:44.634012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884758+0000","last_change":"2024-09-17T02:42:43.837536+0000","last_active":"2024-09-17T02:42:45.884758+0000","last_peered":"2024-09-17T02:42:45.884758+0000","last_clean":"2024-09-17T02:42:45.884758+0000","last_became_active":"2024-09-17T02:42:43.837182+0000","last_became_peered":"2024-09-17T02:42:43.837182+0000","last_unstale":"2024-09-17T02:42:45.884758+0000","last_undegraded":"2024-09-17T02:42:45.884758+0000","last_fullsized":"2024-09-17T02:42:45.884758+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:38.099906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870485+0000","last_change":"2024-09-17T02:42:44.870506+0000","last_active":"2024-09-17T02:42:45.870485+0000","last_peered":"2024-09-17T02:42:45.870485+0000","last_clean":"2024-09-17T02:42:45.870485+0000","last_became_active":"2024-09-17T02:42:44.870091+0000","last_became_peered":"2024-09-17T02:42:44.870091+0000","last_unstale":"2024-09-17T02:42:45.870485+0000","last_undegraded":"2024-09-17T02:42:45.870485+0000","last_fullsized":"2024-09-17T02:42:45.870485+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:14.136518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885151+0000","last_change":"2024-09-17T02:42:43.835536+0000","last_active":"2024-09-17T02:42:45.885151+0000","last_peered":"2024-09-17T02:42:45.885151+0000","last_clean":"2024-09-17T02:42:45.885151+0000","last_became_active":"2024-09-17T02:42:43.835141+0000","last_became_peered":"2024-09-17T02:42:43.835141+0000","last_unstale":"2024-09-17T02:42:45.885151+0000","last_undegraded":"2024-09-17T02:42:45.885151+0000","last_fullsized":"2024-09-17T02:42:45.885151+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:58.469842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884026+0000","last_change":"2024-09-17T02:42:44.869196+0000","last_active":"2024-09-17T02:42:45.884026+0000","last_peered":"2024-09-17T02:42:45.884026+0000","last_clean":"2024-09-17T02:42:45.884026+0000","last_became_active":"2024-09-17T02:42:44.868845+0000","last_became_peered":"2024-09-17T02:42:44.868845+0000","last_unstale":"2024-09-17T02:42:45.884026+0000","last_undegraded":"2024-09-17T02:42:45.884026+0000","last_fullsized":"2024-09-17T02:42:45.884026+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:44.069731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884247+0000","last_change":"2024-09-17T02:42:43.856625+0000","last_active":"2024-09-17T02:42:45.884247+0000","last_peered":"2024-09-17T02:42:45.884247+0000","last_clean":"2024-09-17T02:42:45.884247+0000","last_became_active":"2024-09-17T02:42:43.855851+0000","last_became_peered":"2024-09-17T02:42:43.855851+0000","last_unstale":"2024-09-17T02:42:45.884247+0000","last_undegraded":"2024-09-17T02:42:45.884247+0000","last_fullsized":"2024-09-17T02:42:45.884247+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:11.579876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.872710+0000","last_change":"2024-09-17T02:42:44.869981+0000","last_active":"2024-09-17T02:42:45.872710+0000","last_peered":"2024-09-17T02:42:45.872710+0000","last_clean":"2024-09-17T02:42:45.872710+0000","last_became_active":"2024-09-17T02:42:44.869502+0000","last_became_peered":"2024-09-17T02:42:44.869502+0000","last_unstale":"2024-09-17T02:42:45.872710+0000","last_undegraded":"2024-09-17T02:42:45.872710+0000","last_fullsized":"2024-09-17T02:42:45.872710+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:22.269723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870525+0000","last_change":"2024-09-17T02:42:43.847400+0000","last_active":"2024-09-17T02:42:45.870525+0000","last_peered":"2024-09-17T02:42:45.870525+0000","last_clean":"2024-09-17T02:42:45.870525+0000","last_became_active":"2024-09-17T02:42:43.843509+0000","last_became_peered":"2024-09-17T02:42:43.843509+0000","last_unstale":"2024-09-17T02:42:45.870525+0000","last_undegraded":"2024-09-17T02:42:45.870525+0000","last_fullsized":"2024-09-17T02:42:45.870525+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:21.738639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884317+0000","last_change":"2024-09-17T02:42:44.887685+0000","last_active":"2024-09-17T02:42:45.884317+0000","last_peered":"2024-09-17T02:42:45.884317+0000","last_clean":"2024-09-17T02:42:45.884317+0000","last_became_active":"2024-09-17T02:42:44.887127+0000","last_became_peered":"2024-09-17T02:42:44.887127+0000","last_unstale":"2024-09-17T02:42:45.884317+0000","last_undegraded":"2024-09-17T02:42:45.884317+0000","last_fullsized":"2024-09-17T02:42:45.884317+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:23.065669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883962+0000","last_change":"2024-09-17T02:42:43.859365+0000","last_active":"2024-09-17T02:42:45.883962+0000","last_peered":"2024-09-17T02:42:45.883962+0000","last_clean":"2024-09-17T02:42:45.883962+0000","last_became_active":"2024-09-17T02:42:43.856287+0000","last_became_peered":"2024-09-17T02:42:43.856287+0000","last_unstale":"2024-09-17T02:42:45.883962+0000","last_undegraded":"2024-09-17T02:42:45.883962+0000","last_fullsized":"2024-09-17T02:42:45.883962+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:18.332340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.874248+0000","last_change":"2024-09-17T02:42:44.880311+0000","last_active":"2024-09-17T02:42:45.874248+0000","last_peered":"2024-09-17T02:42:45.874248+0000","last_clean":"2024-09-17T02:42:45.874248+0000","last_became_active":"2024-09-17T02:42:44.879946+0000","last_became_peered":"2024-09-17T02:42:44.879946+0000","last_unstale":"2024-09-17T02:42:45.874248+0000","last_undegraded":"2024-09-17T02:42:45.874248+0000","last_fullsized":"2024-09-17T02:42:45.874248+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:32.989145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882301+0000","last_change":"2024-09-17T02:42:43.822424+0000","last_active":"2024-09-17T02:42:45.882301+0000","last_peered":"2024-09-17T02:42:45.882301+0000","last_clean":"2024-09-17T02:42:45.882301+0000","last_became_active":"2024-09-17T02:42:43.822051+0000","last_became_peered":"2024-09-17T02:42:43.822051+0000","last_unstale":"2024-09-17T02:42:45.882301+0000","last_undegraded":"2024-09-17T02:42:45.882301+0000","last_fullsized":"2024-09-17T02:42:45.882301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:25.389352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882609+0000","last_change":"2024-09-17T02:42:44.881443+0000","last_active":"2024-09-17T02:42:45.882609+0000","last_peered":"2024-09-17T02:42:45.882609+0000","last_clean":"2024-09-17T02:42:45.882609+0000","last_became_active":"2024-09-17T02:42:44.881026+0000","last_became_peered":"2024-09-17T02:42:44.881026+0000","last_unstale":"2024-09-17T02:42:45.882609+0000","last_undegraded":"2024-09-17T02:42:45.882609+0000","last_fullsized":"2024-09-17T02:42:45.882609+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866487+0000","last_change":"2024-09-17T02:42:43.845560+0000","last_active":"2024-09-17T02:42:45.866487+0000","last_peered":"2024-09-17T02:42:45.866487+0000","last_clean":"2024-09-17T02:42:45.866487+0000","last_became_active":"2024-09-17T02:42:43.845154+0000","last_became_peered":"2024-09-17T02:42:43.845154+0000","last_unstale":"2024-09-17T02:42:45.866487+0000","last_undegraded":"2024-09-17T02:42:45.866487+0000","last_fullsized":"2024-09-17T02:42:45.866487+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:33.597784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.887379+0000","last_change":"2024-09-17T02:42:44.881402+0000","last_active":"2024-09-17T02:42:45.887379+0000","last_peered":"2024-09-17T02:42:45.887379+0000","last_clean":"2024-09-17T02:42:45.887379+0000","last_became_active":"2024-09-17T02:42:44.880572+0000","last_became_peered":"2024-09-17T02:42:44.880572+0000","last_unstale":"2024-09-17T02:42:45.887379+0000","last_undegraded":"2024-09-17T02:42:45.887379+0000","last_fullsized":"2024-09-17T02:42:45.887379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:23.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.874020+0000","last_change":"2024-09-17T02:42:43.834185+0000","last_active":"2024-09-17T02:42:45.874020+0000","last_peered":"2024-09-17T02:42:45.874020+0000","last_clean":"2024-09-17T02:42:45.874020+0000","last_became_active":"2024-09-17T02:42:43.832748+0000","last_became_peered":"2024-09-17T02:42:43.832748+0000","last_unstale":"2024-09-17T02:42:45.874020+0000","last_undegraded":"2024-09-17T02:42:45.874020+0000","last_fullsized":"2024-09-17T02:42:45.874020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:19.775974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":168,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866609+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:42:45.866609+0000","last_peered":"2024-09-17T02:42:45.866609+0000","last_clean":"2024-09-17T02:42:45.866609+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:42:45.866609+0000","last_undegraded":"2024-09-17T02:42:45.866609+0000","last_fullsized":"2024-09-17T02:42:45.866609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":606208,"data_stored":262756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1208320,"data_stored":854770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":271096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":602112,"data_stored":262790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":614400,"data_stored":269426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1196032,"data_stored":853138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":602112,"data_stored":263282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":573440,"data_stored":248866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:42:49.052 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:42:49.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.235+0000 7fb3b07ae640 1 -- 172.21.15.131:0/1292417451 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb3a80c4ac0 msgr2=0x7fb3a80c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.235+0000 7fb3b07ae640 1 --2- 172.21.15.131:0/1292417451 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb3a80c4ac0 0x7fb3a80c4ed0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb398009840 tx=0x7fb39801c790 comp rx=0 tx=0).stop 2024-09-17T02:42:49.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.235+0000 7fb3b07ae640 1 -- 172.21.15.131:0/1292417451 shutdown_connections 2024-09-17T02:42:49.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.235+0000 7fb3b07ae640 1 --2- 172.21.15.131:0/1292417451 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb3a80c4ac0 0x7fb3a80c4ed0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.235+0000 7fb3b07ae640 1 --2- 172.21.15.131:0/1292417451 >> v2:172.21.15.189:3300/0 conn(0x7fb3a806e2c0 0x7fb3a80c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.235+0000 7fb3b07ae640 1 -- 172.21.15.131:0/1292417451 >> 172.21.15.131:0/1292417451 conn(0x7fb3a806eea0 msgr2=0x7fb3a806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:49.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.235+0000 7fb3b07ae640 1 -- 172.21.15.131:0/1292417451 shutdown_connections 2024-09-17T02:42:49.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.235+0000 7fb3b07ae640 1 -- 172.21.15.131:0/1292417451 wait complete. 2024-09-17T02:42:49.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.239+0000 7fb3b07ae640 1 Processor -- start 2024-09-17T02:42:49.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.239+0000 7fb3b07ae640 1 -- start start 2024-09-17T02:42:49.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.239+0000 7fb3b07ae640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb3a806e2c0 0x7fb3a8157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.239+0000 7fb3b07ae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3a80c4ac0 0x7fb3a80c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.239+0000 7fb3b07ae640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a80cdbf0 0x7fb3a80b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.239+0000 7fb3b07ae640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a80cf0a0 con 0x7fb3a80cdbf0 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.239+0000 7fb3b07ae640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a80cef20 con 0x7fb3a80c4ac0 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.239+0000 7fb3b07ae640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a80cf220 con 0x7fb3a806e2c0 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.239+0000 7fb3aed4b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a80cdbf0 0x7fb3a80b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.239+0000 7fb3aed4b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a80cdbf0 0x7fb3a80b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58064/0 (socket says 172.21.15.131:58064) 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.239+0000 7fb3aed4b640 1 -- 172.21.15.131:0/443503912 learned_addr learned my addr 172.21.15.131:0/443503912 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3add49640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3a80c4ac0 0x7fb3a80c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3ae54a640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb3a806e2c0 0x7fb3a8157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3aed4b640 1 -- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb3a806e2c0 msgr2=0x7fb3a8157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3aed4b640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb3a806e2c0 0x7fb3a8157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3aed4b640 1 -- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3a80c4ac0 msgr2=0x7fb3a80c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3aed4b640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3a80c4ac0 0x7fb3a80c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3aed4b640 1 -- 172.21.15.131:0/443503912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a8158980 con 0x7fb3a80cdbf0 2024-09-17T02:42:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3aed4b640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a80cdbf0 0x7fb3a80b4800 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000cb10 tx=0x7fb3a000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb39effd640 1 -- 172.21.15.131:0/443503912 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0016020 con 0x7fb3a80cdbf0 2024-09-17T02:42:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb39effd640 1 -- 172.21.15.131:0/443503912 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3a000ce50 con 0x7fb3a80cdbf0 2024-09-17T02:42:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3b07ae640 1 -- 172.21.15.131:0/443503912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a81583c0 con 0x7fb3a80cdbf0 2024-09-17T02:42:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb39effd640 1 -- 172.21.15.131:0/443503912 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a000adc0 con 0x7fb3a80cdbf0 2024-09-17T02:42:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3b07ae640 1 -- 172.21.15.131:0/443503912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a814d890 con 0x7fb3a80cdbf0 2024-09-17T02:42:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3b07ae640 1 -- 172.21.15.131:0/443503912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a80c4580 con 0x7fb3a80cdbf0 2024-09-17T02:42:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb39effd640 1 -- 172.21.15.131:0/443503912 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb3a0019020 con 0x7fb3a80cdbf0 2024-09-17T02:42:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb39effd640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb38c073200 0x7fb38c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb39effd640 1 -- 172.21.15.131:0/443503912 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fb3a0094350 con 0x7fb3a80cdbf0 2024-09-17T02:42:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3ae54a640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb38c073200 0x7fb38c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.243+0000 7fb3ae54a640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb38c073200 0x7fb38c075710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009ad0 tx=0x7fb3a40091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:49.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.247+0000 7fb39effd640 1 -- 172.21.15.131:0/443503912 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb3a0020030 con 0x7fb3a80cdbf0 2024-09-17T02:42:49.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.415+0000 7fb3b07ae640 1 -- 172.21.15.131:0/443503912 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3a8073310 con 0x7fb38c073200 2024-09-17T02:42:49.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.431+0000 7fb39effd640 1 -- 172.21.15.131:0/443503912 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251128 (secure 0 0 0) 0x7fb3a8073310 con 0x7fb38c073200 2024-09-17T02:42:49.435 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:49.436 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:42:49.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 -- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb38c073200 msgr2=0x7fb38c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb38c073200 0x7fb38c075710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009ad0 tx=0x7fb3a40091e0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 -- 172.21.15.131:0/443503912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a80cdbf0 msgr2=0x7fb3a80b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a80cdbf0 0x7fb3a80b4800 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000cb10 tx=0x7fb3a000cc00 comp rx=0 tx=0).stop 2024-09-17T02:42:49.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 -- 172.21.15.131:0/443503912 shutdown_connections 2024-09-17T02:42:49.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb38c073200 0x7fb38c075710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb3a80cdbf0 0x7fb3a80b4800 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb3a80c4ac0 0x7fb3a80c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 --2- 172.21.15.131:0/443503912 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb3a806e2c0 0x7fb3a8157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 -- 172.21.15.131:0/443503912 >> 172.21.15.131:0/443503912 conn(0x7fb3a806eea0 msgr2=0x7fb3a80b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:49.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 -- 172.21.15.131:0/443503912 shutdown_connections 2024-09-17T02:42:49.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.435+0000 7fb3b07ae640 1 -- 172.21.15.131:0/443503912 wait complete. 2024-09-17T02:42:49.457 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-17T02:42:49.162416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238792,"kb_used_data":5892,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735928,"statfs":{"total":773094113280,"available":772849590272,"internally_reserved":0,"allocated":6033408,"data_stored":3286124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002204"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.828438+0000","last_change":"2024-09-17T02:42:43.828438+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.828438+0000","last_undegraded":"2024-09-17T02:42:43.828438+0000","last_fullsized":"2024-09-17T02:42:43.828438+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883357+0000","last_change":"2024-09-17T02:42:44.884814+0000","last_active":"2024-09-17T02:42:45.883357+0000","last_peered":"2024-09-17T02:42:45.883357+0000","last_clean":"2024-09-17T02:42:45.883357+0000","last_became_active":"2024-09-17T02:42:44.884432+0000","last_became_peered":"2024-09-17T02:42:44.884432+0000","last_unstale":"2024-09-17T02:42:45.883357+0000","last_undegraded":"2024-09-17T02:42:45.883357+0000","last_fullsized":"2024-09-17T02:42:45.883357+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:23.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884491+0000","last_change":"2024-09-17T02:42:44.854032+0000","last_active":"2024-09-17T02:42:45.884491+0000","last_peered":"2024-09-17T02:42:45.884491+0000","last_clean":"2024-09-17T02:42:45.884491+0000","last_became_active":"2024-09-17T02:42:44.853205+0000","last_became_peered":"2024-09-17T02:42:44.853205+0000","last_unstale":"2024-09-17T02:42:45.884491+0000","last_undegraded":"2024-09-17T02:42:45.884491+0000","last_fullsized":"2024-09-17T02:42:45.884491+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:18.997673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867302+0000","last_change":"2024-09-17T02:42:44.878169+0000","last_active":"2024-09-17T02:42:45.867302+0000","last_peered":"2024-09-17T02:42:45.867302+0000","last_clean":"2024-09-17T02:42:45.867302+0000","last_became_active":"2024-09-17T02:42:44.877402+0000","last_became_peered":"2024-09-17T02:42:44.877402+0000","last_unstale":"2024-09-17T02:42:45.867302+0000","last_undegraded":"2024-09-17T02:42:45.867302+0000","last_fullsized":"2024-09-17T02:42:45.867302+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:55.948606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882642+0000","last_change":"2024-09-17T02:42:44.872147+0000","last_active":"2024-09-17T02:42:45.882642+0000","last_peered":"2024-09-17T02:42:45.882642+0000","last_clean":"2024-09-17T02:42:45.882642+0000","last_became_active":"2024-09-17T02:42:44.871745+0000","last_became_peered":"2024-09-17T02:42:44.871745+0000","last_unstale":"2024-09-17T02:42:45.882642+0000","last_undegraded":"2024-09-17T02:42:45.882642+0000","last_fullsized":"2024-09-17T02:42:45.882642+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:29.767271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882210+0000","last_change":"2024-09-17T02:42:44.885865+0000","last_active":"2024-09-17T02:42:45.882210+0000","last_peered":"2024-09-17T02:42:45.882210+0000","last_clean":"2024-09-17T02:42:45.882210+0000","last_became_active":"2024-09-17T02:42:44.885512+0000","last_became_peered":"2024-09-17T02:42:44.885512+0000","last_unstale":"2024-09-17T02:42:45.882210+0000","last_undegraded":"2024-09-17T02:42:45.882210+0000","last_fullsized":"2024-09-17T02:42:45.882210+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:56.895371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870233+0000","last_change":"2024-09-17T02:42:44.886317+0000","last_active":"2024-09-17T02:42:45.870233+0000","last_peered":"2024-09-17T02:42:45.870233+0000","last_clean":"2024-09-17T02:42:45.870233+0000","last_became_active":"2024-09-17T02:42:44.885948+0000","last_became_peered":"2024-09-17T02:42:44.885948+0000","last_unstale":"2024-09-17T02:42:45.870233+0000","last_undegraded":"2024-09-17T02:42:45.870233+0000","last_fullsized":"2024-09-17T02:42:45.870233+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:44.318617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881521+0000","last_change":"2024-09-17T02:42:44.876332+0000","last_active":"2024-09-17T02:42:45.881521+0000","last_peered":"2024-09-17T02:42:45.881521+0000","last_clean":"2024-09-17T02:42:45.881521+0000","last_became_active":"2024-09-17T02:42:44.875594+0000","last_became_peered":"2024-09-17T02:42:44.875594+0000","last_unstale":"2024-09-17T02:42:45.881521+0000","last_undegraded":"2024-09-17T02:42:45.881521+0000","last_fullsized":"2024-09-17T02:42:45.881521+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:34.865339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884540+0000","last_change":"2024-09-17T02:42:44.854764+0000","last_active":"2024-09-17T02:42:45.884540+0000","last_peered":"2024-09-17T02:42:45.884540+0000","last_clean":"2024-09-17T02:42:45.884540+0000","last_became_active":"2024-09-17T02:42:44.853822+0000","last_became_peered":"2024-09-17T02:42:44.853822+0000","last_unstale":"2024-09-17T02:42:45.884540+0000","last_undegraded":"2024-09-17T02:42:45.884540+0000","last_fullsized":"2024-09-17T02:42:45.884540+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:18.705995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873645+0000","last_change":"2024-09-17T02:42:44.889606+0000","last_active":"2024-09-17T02:42:45.873645+0000","last_peered":"2024-09-17T02:42:45.873645+0000","last_clean":"2024-09-17T02:42:45.873645+0000","last_became_active":"2024-09-17T02:42:44.889274+0000","last_became_peered":"2024-09-17T02:42:44.889274+0000","last_unstale":"2024-09-17T02:42:45.873645+0000","last_undegraded":"2024-09-17T02:42:45.873645+0000","last_fullsized":"2024-09-17T02:42:45.873645+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:12.214981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809880+0000","last_change":"2024-09-17T02:42:43.809880+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809880+0000","last_undegraded":"2024-09-17T02:42:43.809880+0000","last_fullsized":"2024-09-17T02:42:43.809880+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882022+0000","last_change":"2024-09-17T02:42:44.880065+0000","last_active":"2024-09-17T02:42:45.882022+0000","last_peered":"2024-09-17T02:42:45.882022+0000","last_clean":"2024-09-17T02:42:45.882022+0000","last_became_active":"2024-09-17T02:42:44.876260+0000","last_became_peered":"2024-09-17T02:42:44.876260+0000","last_unstale":"2024-09-17T02:42:45.882022+0000","last_undegraded":"2024-09-17T02:42:45.882022+0000","last_fullsized":"2024-09-17T02:42:45.882022+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:34:02.925523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883058+0000","last_change":"2024-09-17T02:42:44.882550+0000","last_active":"2024-09-17T02:42:45.883058+0000","last_peered":"2024-09-17T02:42:45.883058+0000","last_clean":"2024-09-17T02:42:45.883058+0000","last_became_active":"2024-09-17T02:42:44.882123+0000","last_became_peered":"2024-09-17T02:42:44.882123+0000","last_unstale":"2024-09-17T02:42:45.883058+0000","last_undegraded":"2024-09-17T02:42:45.883058+0000","last_fullsized":"2024-09-17T02:42:45.883058+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:24.890536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.812574+0000","last_change":"2024-09-17T02:42:43.812574+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.812574+0000","last_undegraded":"2024-09-17T02:42:43.812574+0000","last_fullsized":"2024-09-17T02:42:43.812574+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866923+0000","last_change":"2024-09-17T02:42:44.877349+0000","last_active":"2024-09-17T02:42:45.866923+0000","last_peered":"2024-09-17T02:42:45.866923+0000","last_clean":"2024-09-17T02:42:45.866923+0000","last_became_active":"2024-09-17T02:42:44.876816+0000","last_became_peered":"2024-09-17T02:42:44.876816+0000","last_unstale":"2024-09-17T02:42:45.866923+0000","last_undegraded":"2024-09-17T02:42:45.866923+0000","last_fullsized":"2024-09-17T02:42:45.866923+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:22.741898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881823+0000","last_change":"2024-09-17T02:42:44.876906+0000","last_active":"2024-09-17T02:42:45.881823+0000","last_peered":"2024-09-17T02:42:45.881823+0000","last_clean":"2024-09-17T02:42:45.881823+0000","last_became_active":"2024-09-17T02:42:44.875966+0000","last_became_peered":"2024-09-17T02:42:44.875966+0000","last_unstale":"2024-09-17T02:42:45.881823+0000","last_undegraded":"2024-09-17T02:42:45.881823+0000","last_fullsized":"2024-09-17T02:42:45.881823+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:06.983520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.813038+0000","last_change":"2024-09-17T02:42:43.813038+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.813038+0000","last_undegraded":"2024-09-17T02:42:43.813038+0000","last_fullsized":"2024-09-17T02:42:43.813038+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867528+0000","last_change":"2024-09-17T02:42:44.888487+0000","last_active":"2024-09-17T02:42:45.867528+0000","last_peered":"2024-09-17T02:42:45.867528+0000","last_clean":"2024-09-17T02:42:45.867528+0000","last_became_active":"2024-09-17T02:42:44.887970+0000","last_became_peered":"2024-09-17T02:42:44.887970+0000","last_unstale":"2024-09-17T02:42:45.867528+0000","last_undegraded":"2024-09-17T02:42:45.867528+0000","last_fullsized":"2024-09-17T02:42:45.867528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:42.359754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884599+0000","last_change":"2024-09-17T02:42:44.881307+0000","last_active":"2024-09-17T02:42:45.884599+0000","last_peered":"2024-09-17T02:42:45.884599+0000","last_clean":"2024-09-17T02:42:45.884599+0000","last_became_active":"2024-09-17T02:42:44.880605+0000","last_became_peered":"2024-09-17T02:42:44.880605+0000","last_unstale":"2024-09-17T02:42:45.884599+0000","last_undegraded":"2024-09-17T02:42:45.884599+0000","last_fullsized":"2024-09-17T02:42:45.884599+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:49.563614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883282+0000","last_change":"2024-09-17T02:42:44.887466+0000","last_active":"2024-09-17T02:42:45.883282+0000","last_peered":"2024-09-17T02:42:45.883282+0000","last_clean":"2024-09-17T02:42:45.883282+0000","last_became_active":"2024-09-17T02:42:44.885044+0000","last_became_peered":"2024-09-17T02:42:44.885044+0000","last_unstale":"2024-09-17T02:42:45.883282+0000","last_undegraded":"2024-09-17T02:42:45.883282+0000","last_fullsized":"2024-09-17T02:42:45.883282+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.263210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867244+0000","last_change":"2024-09-17T02:42:44.876044+0000","last_active":"2024-09-17T02:42:45.867244+0000","last_peered":"2024-09-17T02:42:45.867244+0000","last_clean":"2024-09-17T02:42:45.867244+0000","last_became_active":"2024-09-17T02:42:44.873541+0000","last_became_peered":"2024-09-17T02:42:44.873541+0000","last_unstale":"2024-09-17T02:42:45.867244+0000","last_undegraded":"2024-09-17T02:42:45.867244+0000","last_fullsized":"2024-09-17T02:42:45.867244+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:56.895371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883856+0000","last_change":"2024-09-17T02:42:44.879337+0000","last_active":"2024-09-17T02:42:45.883856+0000","last_peered":"2024-09-17T02:42:45.883856+0000","last_clean":"2024-09-17T02:42:45.883856+0000","last_became_active":"2024-09-17T02:42:44.878982+0000","last_became_peered":"2024-09-17T02:42:44.878982+0000","last_unstale":"2024-09-17T02:42:45.883856+0000","last_undegraded":"2024-09-17T02:42:45.883856+0000","last_fullsized":"2024-09-17T02:42:45.883856+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:59.473102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882578+0000","last_change":"2024-09-17T02:42:44.856410+0000","last_active":"2024-09-17T02:42:45.882578+0000","last_peered":"2024-09-17T02:42:45.882578+0000","last_clean":"2024-09-17T02:42:45.882578+0000","last_became_active":"2024-09-17T02:42:44.854487+0000","last_became_peered":"2024-09-17T02:42:44.854487+0000","last_unstale":"2024-09-17T02:42:45.882578+0000","last_undegraded":"2024-09-17T02:42:45.882578+0000","last_fullsized":"2024-09-17T02:42:45.882578+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:07.107071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882985+0000","last_change":"2024-09-17T02:42:44.870968+0000","last_active":"2024-09-17T02:42:45.882985+0000","last_peered":"2024-09-17T02:42:45.882985+0000","last_clean":"2024-09-17T02:42:45.882985+0000","last_became_active":"2024-09-17T02:42:44.870254+0000","last_became_peered":"2024-09-17T02:42:44.870254+0000","last_unstale":"2024-09-17T02:42:45.882985+0000","last_undegraded":"2024-09-17T02:42:45.882985+0000","last_fullsized":"2024-09-17T02:42:45.882985+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:15.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.826441+0000","last_change":"2024-09-17T02:42:43.826441+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.826441+0000","last_undegraded":"2024-09-17T02:42:43.826441+0000","last_fullsized":"2024-09-17T02:42:43.826441+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884665+0000","last_change":"2024-09-17T02:42:44.875582+0000","last_active":"2024-09-17T02:42:45.884665+0000","last_peered":"2024-09-17T02:42:45.884665+0000","last_clean":"2024-09-17T02:42:45.884665+0000","last_became_active":"2024-09-17T02:42:44.875205+0000","last_became_peered":"2024-09-17T02:42:44.875205+0000","last_unstale":"2024-09-17T02:42:45.884665+0000","last_undegraded":"2024-09-17T02:42:45.884665+0000","last_fullsized":"2024-09-17T02:42:45.884665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:22:23.818837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881878+0000","last_change":"2024-09-17T02:42:44.885311+0000","last_active":"2024-09-17T02:42:45.881878+0000","last_peered":"2024-09-17T02:42:45.881878+0000","last_clean":"2024-09-17T02:42:45.881878+0000","last_became_active":"2024-09-17T02:42:44.884647+0000","last_became_peered":"2024-09-17T02:42:44.884647+0000","last_unstale":"2024-09-17T02:42:45.881878+0000","last_undegraded":"2024-09-17T02:42:45.881878+0000","last_fullsized":"2024-09-17T02:42:45.881878+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:20.906053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883665+0000","last_change":"2024-09-17T02:42:44.877183+0000","last_active":"2024-09-17T02:42:45.883665+0000","last_peered":"2024-09-17T02:42:45.883665+0000","last_clean":"2024-09-17T02:42:45.883665+0000","last_became_active":"2024-09-17T02:42:44.875043+0000","last_became_peered":"2024-09-17T02:42:44.875043+0000","last_unstale":"2024-09-17T02:42:45.883665+0000","last_undegraded":"2024-09-17T02:42:45.883665+0000","last_fullsized":"2024-09-17T02:42:45.883665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:53.445161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809918+0000","last_change":"2024-09-17T02:42:43.809918+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809918+0000","last_undegraded":"2024-09-17T02:42:43.809918+0000","last_fullsized":"2024-09-17T02:42:43.809918+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883971+0000","last_change":"2024-09-17T02:42:44.869378+0000","last_active":"2024-09-17T02:42:45.883971+0000","last_peered":"2024-09-17T02:42:45.883971+0000","last_clean":"2024-09-17T02:42:45.883971+0000","last_became_active":"2024-09-17T02:42:44.868147+0000","last_became_peered":"2024-09-17T02:42:44.868147+0000","last_unstale":"2024-09-17T02:42:45.883971+0000","last_undegraded":"2024-09-17T02:42:45.883971+0000","last_fullsized":"2024-09-17T02:42:45.883971+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:03.959077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882887+0000","last_change":"2024-09-17T02:42:44.879236+0000","last_active":"2024-09-17T02:42:45.882887+0000","last_peered":"2024-09-17T02:42:45.882887+0000","last_clean":"2024-09-17T02:42:45.882887+0000","last_became_active":"2024-09-17T02:42:44.878872+0000","last_became_peered":"2024-09-17T02:42:44.878872+0000","last_unstale":"2024-09-17T02:42:45.882887+0000","last_undegraded":"2024-09-17T02:42:45.882887+0000","last_fullsized":"2024-09-17T02:42:45.882887+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:39.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882661+0000","last_change":"2024-09-17T02:42:44.853789+0000","last_active":"2024-09-17T02:42:45.882661+0000","last_peered":"2024-09-17T02:42:45.882661+0000","last_clean":"2024-09-17T02:42:45.882661+0000","last_became_active":"2024-09-17T02:42:44.852870+0000","last_became_peered":"2024-09-17T02:42:44.852870+0000","last_unstale":"2024-09-17T02:42:45.882661+0000","last_undegraded":"2024-09-17T02:42:45.882661+0000","last_fullsized":"2024-09-17T02:42:45.882661+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.016019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882301+0000","last_change":"2024-09-17T02:42:44.884160+0000","last_active":"2024-09-17T02:42:45.882301+0000","last_peered":"2024-09-17T02:42:45.882301+0000","last_clean":"2024-09-17T02:42:45.882301+0000","last_became_active":"2024-09-17T02:42:44.883295+0000","last_became_peered":"2024-09-17T02:42:44.883295+0000","last_unstale":"2024-09-17T02:42:45.882301+0000","last_undegraded":"2024-09-17T02:42:45.882301+0000","last_fullsized":"2024-09-17T02:42:45.882301+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:13.871419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":19,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885185+0000","last_change":"2024-09-17T02:42:43.845735+0000","last_active":"2024-09-17T02:42:45.885185+0000","last_peered":"2024-09-17T02:42:45.885185+0000","last_clean":"2024-09-17T02:42:45.885185+0000","last_became_active":"2024-09-17T02:42:43.845485+0000","last_became_peered":"2024-09-17T02:42:43.845485+0000","last_unstale":"2024-09-17T02:42:45.885185+0000","last_undegraded":"2024-09-17T02:42:45.885185+0000","last_fullsized":"2024-09-17T02:42:45.885185+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:19.775974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881559+0000","last_change":"2024-09-17T02:42:44.875131+0000","last_active":"2024-09-17T02:42:45.881559+0000","last_peered":"2024-09-17T02:42:45.881559+0000","last_clean":"2024-09-17T02:42:45.881559+0000","last_became_active":"2024-09-17T02:42:44.874308+0000","last_became_peered":"2024-09-17T02:42:44.874308+0000","last_unstale":"2024-09-17T02:42:45.881559+0000","last_undegraded":"2024-09-17T02:42:45.881559+0000","last_fullsized":"2024-09-17T02:42:45.881559+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:05.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882119+0000","last_change":"2024-09-17T02:42:43.865164+0000","last_active":"2024-09-17T02:42:45.882119+0000","last_peered":"2024-09-17T02:42:45.882119+0000","last_clean":"2024-09-17T02:42:45.882119+0000","last_became_active":"2024-09-17T02:42:43.862678+0000","last_became_peered":"2024-09-17T02:42:43.862678+0000","last_unstale":"2024-09-17T02:42:45.882119+0000","last_undegraded":"2024-09-17T02:42:45.882119+0000","last_fullsized":"2024-09-17T02:42:45.882119+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:46.632441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883731+0000","last_change":"2024-09-17T02:42:44.870094+0000","last_active":"2024-09-17T02:42:45.883731+0000","last_peered":"2024-09-17T02:42:45.883731+0000","last_clean":"2024-09-17T02:42:45.883731+0000","last_became_active":"2024-09-17T02:42:44.869245+0000","last_became_peered":"2024-09-17T02:42:44.869245+0000","last_unstale":"2024-09-17T02:42:45.883731+0000","last_undegraded":"2024-09-17T02:42:45.883731+0000","last_fullsized":"2024-09-17T02:42:45.883731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:18.419470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873178+0000","last_change":"2024-09-17T02:42:43.837893+0000","last_active":"2024-09-17T02:42:45.873178+0000","last_peered":"2024-09-17T02:42:45.873178+0000","last_clean":"2024-09-17T02:42:45.873178+0000","last_became_active":"2024-09-17T02:42:43.833094+0000","last_became_peered":"2024-09-17T02:42:43.833094+0000","last_unstale":"2024-09-17T02:42:45.873178+0000","last_undegraded":"2024-09-17T02:42:45.873178+0000","last_fullsized":"2024-09-17T02:42:45.873178+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:27.547510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870591+0000","last_change":"2024-09-17T02:42:44.876821+0000","last_active":"2024-09-17T02:42:45.870591+0000","last_peered":"2024-09-17T02:42:45.870591+0000","last_clean":"2024-09-17T02:42:45.870591+0000","last_became_active":"2024-09-17T02:42:44.876427+0000","last_became_peered":"2024-09-17T02:42:44.876427+0000","last_unstale":"2024-09-17T02:42:45.870591+0000","last_undegraded":"2024-09-17T02:42:45.870591+0000","last_fullsized":"2024-09-17T02:42:45.870591+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:29.285476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884021+0000","last_change":"2024-09-17T02:42:43.859536+0000","last_active":"2024-09-17T02:42:45.884021+0000","last_peered":"2024-09-17T02:42:45.884021+0000","last_clean":"2024-09-17T02:42:45.884021+0000","last_became_active":"2024-09-17T02:42:43.859191+0000","last_became_peered":"2024-09-17T02:42:43.859191+0000","last_unstale":"2024-09-17T02:42:45.884021+0000","last_undegraded":"2024-09-17T02:42:45.884021+0000","last_fullsized":"2024-09-17T02:42:45.884021+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:34.772218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.886237+0000","last_change":"2024-09-17T02:42:44.881758+0000","last_active":"2024-09-17T02:42:45.886237+0000","last_peered":"2024-09-17T02:42:45.886237+0000","last_clean":"2024-09-17T02:42:45.886237+0000","last_became_active":"2024-09-17T02:42:44.880037+0000","last_became_peered":"2024-09-17T02:42:44.880037+0000","last_unstale":"2024-09-17T02:42:45.886237+0000","last_undegraded":"2024-09-17T02:42:45.886237+0000","last_fullsized":"2024-09-17T02:42:45.886237+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:49.224409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884002+0000","last_change":"2024-09-17T02:42:43.859603+0000","last_active":"2024-09-17T02:42:45.884002+0000","last_peered":"2024-09-17T02:42:45.884002+0000","last_clean":"2024-09-17T02:42:45.884002+0000","last_became_active":"2024-09-17T02:42:43.856481+0000","last_became_peered":"2024-09-17T02:42:43.856481+0000","last_unstale":"2024-09-17T02:42:45.884002+0000","last_undegraded":"2024-09-17T02:42:45.884002+0000","last_fullsized":"2024-09-17T02:42:45.884002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:31.985885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884898+0000","last_change":"2024-09-17T02:42:44.891687+0000","last_active":"2024-09-17T02:42:45.884898+0000","last_peered":"2024-09-17T02:42:45.884898+0000","last_clean":"2024-09-17T02:42:45.884898+0000","last_became_active":"2024-09-17T02:42:44.890077+0000","last_became_peered":"2024-09-17T02:42:44.890077+0000","last_unstale":"2024-09-17T02:42:45.884898+0000","last_undegraded":"2024-09-17T02:42:45.884898+0000","last_fullsized":"2024-09-17T02:42:45.884898+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:46:21.265517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884101+0000","last_change":"2024-09-17T02:42:43.822676+0000","last_active":"2024-09-17T02:42:45.884101+0000","last_peered":"2024-09-17T02:42:45.884101+0000","last_clean":"2024-09-17T02:42:45.884101+0000","last_became_active":"2024-09-17T02:42:43.822225+0000","last_became_peered":"2024-09-17T02:42:43.822225+0000","last_unstale":"2024-09-17T02:42:45.884101+0000","last_undegraded":"2024-09-17T02:42:45.884101+0000","last_fullsized":"2024-09-17T02:42:45.884101+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:38.838275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873695+0000","last_change":"2024-09-17T02:42:44.880963+0000","last_active":"2024-09-17T02:42:45.873695+0000","last_peered":"2024-09-17T02:42:45.873695+0000","last_clean":"2024-09-17T02:42:45.873695+0000","last_became_active":"2024-09-17T02:42:44.880257+0000","last_became_peered":"2024-09-17T02:42:44.880257+0000","last_unstale":"2024-09-17T02:42:45.873695+0000","last_undegraded":"2024-09-17T02:42:45.873695+0000","last_fullsized":"2024-09-17T02:42:45.873695+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:09.416640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885098+0000","last_change":"2024-09-17T02:42:43.814090+0000","last_active":"2024-09-17T02:42:45.885098+0000","last_peered":"2024-09-17T02:42:45.885098+0000","last_clean":"2024-09-17T02:42:45.885098+0000","last_became_active":"2024-09-17T02:42:43.813685+0000","last_became_peered":"2024-09-17T02:42:43.813685+0000","last_unstale":"2024-09-17T02:42:45.885098+0000","last_undegraded":"2024-09-17T02:42:45.885098+0000","last_fullsized":"2024-09-17T02:42:45.885098+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:06.103811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867600+0000","last_change":"2024-09-17T02:42:44.878439+0000","last_active":"2024-09-17T02:42:45.867600+0000","last_peered":"2024-09-17T02:42:45.867600+0000","last_clean":"2024-09-17T02:42:45.867600+0000","last_became_active":"2024-09-17T02:42:44.875651+0000","last_became_peered":"2024-09-17T02:42:44.875651+0000","last_unstale":"2024-09-17T02:42:45.867600+0000","last_undegraded":"2024-09-17T02:42:45.867600+0000","last_fullsized":"2024-09-17T02:42:45.867600+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:29.767271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883310+0000","last_change":"2024-09-17T02:42:43.855843+0000","last_active":"2024-09-17T02:42:45.883310+0000","last_peered":"2024-09-17T02:42:45.883310+0000","last_clean":"2024-09-17T02:42:45.883310+0000","last_became_active":"2024-09-17T02:42:43.854712+0000","last_became_peered":"2024-09-17T02:42:43.854712+0000","last_unstale":"2024-09-17T02:42:45.883310+0000","last_undegraded":"2024-09-17T02:42:45.883310+0000","last_fullsized":"2024-09-17T02:42:45.883310+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:19:02.429366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884260+0000","last_change":"2024-09-17T02:42:44.875385+0000","last_active":"2024-09-17T02:42:45.884260+0000","last_peered":"2024-09-17T02:42:45.884260+0000","last_clean":"2024-09-17T02:42:45.884260+0000","last_became_active":"2024-09-17T02:42:44.874816+0000","last_became_peered":"2024-09-17T02:42:44.874816+0000","last_unstale":"2024-09-17T02:42:45.884260+0000","last_undegraded":"2024-09-17T02:42:45.884260+0000","last_fullsized":"2024-09-17T02:42:45.884260+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:36.940445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867765+0000","last_change":"2024-09-17T02:42:43.826860+0000","last_active":"2024-09-17T02:42:45.867765+0000","last_peered":"2024-09-17T02:42:45.867765+0000","last_clean":"2024-09-17T02:42:45.867765+0000","last_became_active":"2024-09-17T02:42:43.826423+0000","last_became_peered":"2024-09-17T02:42:43.826423+0000","last_unstale":"2024-09-17T02:42:45.867765+0000","last_undegraded":"2024-09-17T02:42:45.867765+0000","last_fullsized":"2024-09-17T02:42:45.867765+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:57.315329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866970+0000","last_change":"2024-09-17T02:42:44.889033+0000","last_active":"2024-09-17T02:42:45.866970+0000","last_peered":"2024-09-17T02:42:45.866970+0000","last_clean":"2024-09-17T02:42:45.866970+0000","last_became_active":"2024-09-17T02:42:44.888575+0000","last_became_peered":"2024-09-17T02:42:44.888575+0000","last_unstale":"2024-09-17T02:42:45.866970+0000","last_undegraded":"2024-09-17T02:42:45.866970+0000","last_fullsized":"2024-09-17T02:42:45.866970+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:10.824751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867254+0000","last_change":"2024-09-17T02:42:43.835429+0000","last_active":"2024-09-17T02:42:45.867254+0000","last_peered":"2024-09-17T02:42:45.867254+0000","last_clean":"2024-09-17T02:42:45.867254+0000","last_became_active":"2024-09-17T02:42:43.835072+0000","last_became_peered":"2024-09-17T02:42:43.835072+0000","last_unstale":"2024-09-17T02:42:45.867254+0000","last_undegraded":"2024-09-17T02:42:45.867254+0000","last_fullsized":"2024-09-17T02:42:45.867254+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:28.861850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.807944+0000","last_change":"2024-09-17T02:42:43.807944+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.807944+0000","last_undegraded":"2024-09-17T02:42:43.807944+0000","last_fullsized":"2024-09-17T02:42:43.807944+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885035+0000","last_change":"2024-09-17T02:42:43.844104+0000","last_active":"2024-09-17T02:42:45.885035+0000","last_peered":"2024-09-17T02:42:45.885035+0000","last_clean":"2024-09-17T02:42:45.885035+0000","last_became_active":"2024-09-17T02:42:43.843661+0000","last_became_peered":"2024-09-17T02:42:43.843661+0000","last_unstale":"2024-09-17T02:42:45.885035+0000","last_undegraded":"2024-09-17T02:42:45.885035+0000","last_fullsized":"2024-09-17T02:42:45.885035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:14.987141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882393+0000","last_change":"2024-09-17T02:42:44.853449+0000","last_active":"2024-09-17T02:42:45.882393+0000","last_peered":"2024-09-17T02:42:45.882393+0000","last_clean":"2024-09-17T02:42:45.882393+0000","last_became_active":"2024-09-17T02:42:44.852561+0000","last_became_peered":"2024-09-17T02:42:44.852561+0000","last_unstale":"2024-09-17T02:42:45.882393+0000","last_undegraded":"2024-09-17T02:42:45.882393+0000","last_fullsized":"2024-09-17T02:42:45.882393+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.871956+0000","last_change":"2024-09-17T02:42:43.864191+0000","last_active":"2024-09-17T02:42:45.871956+0000","last_peered":"2024-09-17T02:42:45.871956+0000","last_clean":"2024-09-17T02:42:45.871956+0000","last_became_active":"2024-09-17T02:42:43.863400+0000","last_became_peered":"2024-09-17T02:42:43.863400+0000","last_unstale":"2024-09-17T02:42:45.871956+0000","last_undegraded":"2024-09-17T02:42:45.871956+0000","last_fullsized":"2024-09-17T02:42:45.871956+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:09.821491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885484+0000","last_change":"2024-09-17T02:42:44.880368+0000","last_active":"2024-09-17T02:42:45.885484+0000","last_peered":"2024-09-17T02:42:45.885484+0000","last_clean":"2024-09-17T02:42:45.885484+0000","last_became_active":"2024-09-17T02:42:44.879881+0000","last_became_peered":"2024-09-17T02:42:44.879881+0000","last_unstale":"2024-09-17T02:42:45.885484+0000","last_undegraded":"2024-09-17T02:42:45.885484+0000","last_fullsized":"2024-09-17T02:42:45.885484+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:58.252210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883189+0000","last_change":"2024-09-17T02:42:43.841976+0000","last_active":"2024-09-17T02:42:45.883189+0000","last_peered":"2024-09-17T02:42:45.883189+0000","last_clean":"2024-09-17T02:42:45.883189+0000","last_became_active":"2024-09-17T02:42:43.841094+0000","last_became_peered":"2024-09-17T02:42:43.841094+0000","last_unstale":"2024-09-17T02:42:45.883189+0000","last_undegraded":"2024-09-17T02:42:45.883189+0000","last_fullsized":"2024-09-17T02:42:45.883189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:29.331958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.808614+0000","last_change":"2024-09-17T02:42:43.808614+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.808614+0000","last_undegraded":"2024-09-17T02:42:43.808614+0000","last_fullsized":"2024-09-17T02:42:43.808614+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866984+0000","last_change":"2024-09-17T02:42:43.843349+0000","last_active":"2024-09-17T02:42:45.866984+0000","last_peered":"2024-09-17T02:42:45.866984+0000","last_clean":"2024-09-17T02:42:45.866984+0000","last_became_active":"2024-09-17T02:42:43.842950+0000","last_became_peered":"2024-09-17T02:42:43.842950+0000","last_unstale":"2024-09-17T02:42:45.866984+0000","last_undegraded":"2024-09-17T02:42:45.866984+0000","last_fullsized":"2024-09-17T02:42:45.866984+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:22.974854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882439+0000","last_change":"2024-09-17T02:42:44.886488+0000","last_active":"2024-09-17T02:42:45.882439+0000","last_peered":"2024-09-17T02:42:45.882439+0000","last_clean":"2024-09-17T02:42:45.882439+0000","last_became_active":"2024-09-17T02:42:44.886106+0000","last_became_peered":"2024-09-17T02:42:44.886106+0000","last_unstale":"2024-09-17T02:42:45.882439+0000","last_undegraded":"2024-09-17T02:42:45.882439+0000","last_fullsized":"2024-09-17T02:42:45.882439+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:55.948606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881989+0000","last_change":"2024-09-17T02:42:43.825883+0000","last_active":"2024-09-17T02:42:45.881989+0000","last_peered":"2024-09-17T02:42:45.881989+0000","last_clean":"2024-09-17T02:42:45.881989+0000","last_became_active":"2024-09-17T02:42:43.825522+0000","last_became_peered":"2024-09-17T02:42:43.825522+0000","last_unstale":"2024-09-17T02:42:45.881989+0000","last_undegraded":"2024-09-17T02:42:45.881989+0000","last_fullsized":"2024-09-17T02:42:45.881989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:22:57.287148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885037+0000","last_change":"2024-09-17T02:42:44.878272+0000","last_active":"2024-09-17T02:42:45.885037+0000","last_peered":"2024-09-17T02:42:45.885037+0000","last_clean":"2024-09-17T02:42:45.885037+0000","last_became_active":"2024-09-17T02:42:44.877921+0000","last_became_peered":"2024-09-17T02:42:44.877921+0000","last_unstale":"2024-09-17T02:42:45.885037+0000","last_undegraded":"2024-09-17T02:42:45.885037+0000","last_fullsized":"2024-09-17T02:42:45.885037+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:14.742444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":23,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881967+0000","last_change":"2024-09-17T02:42:43.864045+0000","last_active":"2024-09-17T02:42:45.881967+0000","last_peered":"2024-09-17T02:42:45.881967+0000","last_clean":"2024-09-17T02:42:45.881967+0000","last_became_active":"2024-09-17T02:42:43.862689+0000","last_became_peered":"2024-09-17T02:42:43.862689+0000","last_unstale":"2024-09-17T02:42:45.881967+0000","last_undegraded":"2024-09-17T02:42:45.881967+0000","last_fullsized":"2024-09-17T02:42:45.881967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:30.667789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870056+0000","last_change":"2024-09-17T02:42:44.871420+0000","last_active":"2024-09-17T02:42:45.870056+0000","last_peered":"2024-09-17T02:42:45.870056+0000","last_clean":"2024-09-17T02:42:45.870056+0000","last_became_active":"2024-09-17T02:42:44.871075+0000","last_became_peered":"2024-09-17T02:42:44.871075+0000","last_unstale":"2024-09-17T02:42:45.870056+0000","last_undegraded":"2024-09-17T02:42:45.870056+0000","last_fullsized":"2024-09-17T02:42:45.870056+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:21.340145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882977+0000","last_change":"2024-09-17T02:42:43.865760+0000","last_active":"2024-09-17T02:42:45.882977+0000","last_peered":"2024-09-17T02:42:45.882977+0000","last_clean":"2024-09-17T02:42:45.882977+0000","last_became_active":"2024-09-17T02:42:43.862428+0000","last_became_peered":"2024-09-17T02:42:43.862428+0000","last_unstale":"2024-09-17T02:42:45.882977+0000","last_undegraded":"2024-09-17T02:42:45.882977+0000","last_fullsized":"2024-09-17T02:42:45.882977+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:00.644920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809542+0000","last_change":"2024-09-17T02:42:43.809542+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809542+0000","last_undegraded":"2024-09-17T02:42:43.809542+0000","last_fullsized":"2024-09-17T02:42:43.809542+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882280+0000","last_change":"2024-09-17T02:42:43.839506+0000","last_active":"2024-09-17T02:42:45.882280+0000","last_peered":"2024-09-17T02:42:45.882280+0000","last_clean":"2024-09-17T02:42:45.882280+0000","last_became_active":"2024-09-17T02:42:43.838617+0000","last_became_peered":"2024-09-17T02:42:43.838617+0000","last_unstale":"2024-09-17T02:42:45.882280+0000","last_undegraded":"2024-09-17T02:42:45.882280+0000","last_fullsized":"2024-09-17T02:42:45.882280+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:04.244931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883433+0000","last_change":"2024-09-17T02:42:44.876771+0000","last_active":"2024-09-17T02:42:45.883433+0000","last_peered":"2024-09-17T02:42:45.883433+0000","last_clean":"2024-09-17T02:42:45.883433+0000","last_became_active":"2024-09-17T02:42:44.876438+0000","last_became_peered":"2024-09-17T02:42:44.876438+0000","last_unstale":"2024-09-17T02:42:45.883433+0000","last_undegraded":"2024-09-17T02:42:45.883433+0000","last_fullsized":"2024-09-17T02:42:45.883433+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:28.921453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867493+0000","last_change":"2024-09-17T02:42:43.862172+0000","last_active":"2024-09-17T02:42:45.867493+0000","last_peered":"2024-09-17T02:42:45.867493+0000","last_clean":"2024-09-17T02:42:45.867493+0000","last_became_active":"2024-09-17T02:42:43.861853+0000","last_became_peered":"2024-09-17T02:42:43.861853+0000","last_unstale":"2024-09-17T02:42:45.867493+0000","last_undegraded":"2024-09-17T02:42:45.867493+0000","last_fullsized":"2024-09-17T02:42:45.867493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:00.644920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883740+0000","last_change":"2024-09-17T02:42:44.875816+0000","last_active":"2024-09-17T02:42:45.883740+0000","last_peered":"2024-09-17T02:42:45.883740+0000","last_clean":"2024-09-17T02:42:45.883740+0000","last_became_active":"2024-09-17T02:42:44.875443+0000","last_became_peered":"2024-09-17T02:42:44.875443+0000","last_unstale":"2024-09-17T02:42:45.883740+0000","last_undegraded":"2024-09-17T02:42:45.883740+0000","last_fullsized":"2024-09-17T02:42:45.883740+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.263210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884349+0000","last_change":"2024-09-17T02:42:43.842888+0000","last_active":"2024-09-17T02:42:45.884349+0000","last_peered":"2024-09-17T02:42:45.884349+0000","last_clean":"2024-09-17T02:42:45.884349+0000","last_became_active":"2024-09-17T02:42:43.842563+0000","last_became_peered":"2024-09-17T02:42:43.842563+0000","last_unstale":"2024-09-17T02:42:45.884349+0000","last_undegraded":"2024-09-17T02:42:45.884349+0000","last_fullsized":"2024-09-17T02:42:45.884349+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:37.379782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883984+0000","last_change":"2024-09-17T02:42:44.875239+0000","last_active":"2024-09-17T02:42:45.883984+0000","last_peered":"2024-09-17T02:42:45.883984+0000","last_clean":"2024-09-17T02:42:45.883984+0000","last_became_active":"2024-09-17T02:42:44.874268+0000","last_became_peered":"2024-09-17T02:42:44.874268+0000","last_unstale":"2024-09-17T02:42:45.883984+0000","last_undegraded":"2024-09-17T02:42:45.883984+0000","last_fullsized":"2024-09-17T02:42:45.883984+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:30.019802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T02:42:43.849754+0000","last_change":"2024-09-17T02:42:43.840810+0000","last_active":"2024-09-17T02:42:43.849754+0000","last_peered":"2024-09-17T02:42:43.849754+0000","last_clean":"2024-09-17T02:42:43.849754+0000","last_became_active":"2024-09-17T02:42:43.839861+0000","last_became_peered":"2024-09-17T02:42:43.839861+0000","last_unstale":"2024-09-17T02:42:43.849754+0000","last_undegraded":"2024-09-17T02:42:43.849754+0000","last_fullsized":"2024-09-17T02:42:43.849754+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:14.684387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867465+0000","last_change":"2024-09-17T02:42:44.877843+0000","last_active":"2024-09-17T02:42:45.867465+0000","last_peered":"2024-09-17T02:42:45.867465+0000","last_clean":"2024-09-17T02:42:45.867465+0000","last_became_active":"2024-09-17T02:42:44.877029+0000","last_became_peered":"2024-09-17T02:42:44.877029+0000","last_unstale":"2024-09-17T02:42:45.867465+0000","last_undegraded":"2024-09-17T02:42:45.867465+0000","last_fullsized":"2024-09-17T02:42:45.867465+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883764+0000","last_change":"2024-09-17T02:42:43.860370+0000","last_active":"2024-09-17T02:42:45.883764+0000","last_peered":"2024-09-17T02:42:45.883764+0000","last_clean":"2024-09-17T02:42:45.883764+0000","last_became_active":"2024-09-17T02:42:43.859094+0000","last_became_peered":"2024-09-17T02:42:43.859094+0000","last_unstale":"2024-09-17T02:42:45.883764+0000","last_undegraded":"2024-09-17T02:42:45.883764+0000","last_fullsized":"2024-09-17T02:42:45.883764+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:15.987844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.810796+0000","last_change":"2024-09-17T02:42:43.810796+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.810796+0000","last_undegraded":"2024-09-17T02:42:43.810796+0000","last_fullsized":"2024-09-17T02:42:43.810796+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870452+0000","last_change":"2024-09-17T02:42:43.862720+0000","last_active":"2024-09-17T02:42:45.870452+0000","last_peered":"2024-09-17T02:42:45.870452+0000","last_clean":"2024-09-17T02:42:45.870452+0000","last_became_active":"2024-09-17T02:42:43.862004+0000","last_became_peered":"2024-09-17T02:42:43.862004+0000","last_unstale":"2024-09-17T02:42:45.870452+0000","last_undegraded":"2024-09-17T02:42:45.870452+0000","last_fullsized":"2024-09-17T02:42:45.870452+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:41.356494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882795+0000","last_change":"2024-09-17T02:42:44.883533+0000","last_active":"2024-09-17T02:42:45.882795+0000","last_peered":"2024-09-17T02:42:45.882795+0000","last_clean":"2024-09-17T02:42:45.882795+0000","last_became_active":"2024-09-17T02:42:44.883166+0000","last_became_peered":"2024-09-17T02:42:44.883166+0000","last_unstale":"2024-09-17T02:42:45.882795+0000","last_undegraded":"2024-09-17T02:42:45.882795+0000","last_fullsized":"2024-09-17T02:42:45.882795+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.016019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.869776+0000","last_change":"2024-09-17T02:42:43.831772+0000","last_active":"2024-09-17T02:42:45.869776+0000","last_peered":"2024-09-17T02:42:45.869776+0000","last_clean":"2024-09-17T02:42:45.869776+0000","last_became_active":"2024-09-17T02:42:43.830906+0000","last_became_peered":"2024-09-17T02:42:43.830906+0000","last_unstale":"2024-09-17T02:42:45.869776+0000","last_undegraded":"2024-09-17T02:42:45.869776+0000","last_fullsized":"2024-09-17T02:42:45.869776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:43.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866625+0000","last_change":"2024-09-17T02:42:44.878871+0000","last_active":"2024-09-17T02:42:45.866625+0000","last_peered":"2024-09-17T02:42:45.866625+0000","last_clean":"2024-09-17T02:42:45.866625+0000","last_became_active":"2024-09-17T02:42:44.878370+0000","last_became_peered":"2024-09-17T02:42:44.878370+0000","last_unstale":"2024-09-17T02:42:45.866625+0000","last_undegraded":"2024-09-17T02:42:45.866625+0000","last_fullsized":"2024-09-17T02:42:45.866625+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:44.634012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884758+0000","last_change":"2024-09-17T02:42:43.837536+0000","last_active":"2024-09-17T02:42:45.884758+0000","last_peered":"2024-09-17T02:42:45.884758+0000","last_clean":"2024-09-17T02:42:45.884758+0000","last_became_active":"2024-09-17T02:42:43.837182+0000","last_became_peered":"2024-09-17T02:42:43.837182+0000","last_unstale":"2024-09-17T02:42:45.884758+0000","last_undegraded":"2024-09-17T02:42:45.884758+0000","last_fullsized":"2024-09-17T02:42:45.884758+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:38.099906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870485+0000","last_change":"2024-09-17T02:42:44.870506+0000","last_active":"2024-09-17T02:42:45.870485+0000","last_peered":"2024-09-17T02:42:45.870485+0000","last_clean":"2024-09-17T02:42:45.870485+0000","last_became_active":"2024-09-17T02:42:44.870091+0000","last_became_peered":"2024-09-17T02:42:44.870091+0000","last_unstale":"2024-09-17T02:42:45.870485+0000","last_undegraded":"2024-09-17T02:42:45.870485+0000","last_fullsized":"2024-09-17T02:42:45.870485+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:14.136518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885151+0000","last_change":"2024-09-17T02:42:43.835536+0000","last_active":"2024-09-17T02:42:45.885151+0000","last_peered":"2024-09-17T02:42:45.885151+0000","last_clean":"2024-09-17T02:42:45.885151+0000","last_became_active":"2024-09-17T02:42:43.835141+0000","last_became_peered":"2024-09-17T02:42:43.835141+0000","last_unstale":"2024-09-17T02:42:45.885151+0000","last_undegraded":"2024-09-17T02:42:45.885151+0000","last_fullsized":"2024-09-17T02:42:45.885151+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:58.469842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884026+0000","last_change":"2024-09-17T02:42:44.869196+0000","last_active":"2024-09-17T02:42:45.884026+0000","last_peered":"2024-09-17T02:42:45.884026+0000","last_clean":"2024-09-17T02:42:45.884026+0000","last_became_active":"2024-09-17T02:42:44.868845+0000","last_became_peered":"2024-09-17T02:42:44.868845+0000","last_unstale":"2024-09-17T02:42:45.884026+0000","last_undegraded":"2024-09-17T02:42:45.884026+0000","last_fullsized":"2024-09-17T02:42:45.884026+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:44.069731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884247+0000","last_change":"2024-09-17T02:42:43.856625+0000","last_active":"2024-09-17T02:42:45.884247+0000","last_peered":"2024-09-17T02:42:45.884247+0000","last_clean":"2024-09-17T02:42:45.884247+0000","last_became_active":"2024-09-17T02:42:43.855851+0000","last_became_peered":"2024-09-17T02:42:43.855851+0000","last_unstale":"2024-09-17T02:42:45.884247+0000","last_undegraded":"2024-09-17T02:42:45.884247+0000","last_fullsized":"2024-09-17T02:42:45.884247+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:11.579876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.872710+0000","last_change":"2024-09-17T02:42:44.869981+0000","last_active":"2024-09-17T02:42:45.872710+0000","last_peered":"2024-09-17T02:42:45.872710+0000","last_clean":"2024-09-17T02:42:45.872710+0000","last_became_active":"2024-09-17T02:42:44.869502+0000","last_became_peered":"2024-09-17T02:42:44.869502+0000","last_unstale":"2024-09-17T02:42:45.872710+0000","last_undegraded":"2024-09-17T02:42:45.872710+0000","last_fullsized":"2024-09-17T02:42:45.872710+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:22.269723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870525+0000","last_change":"2024-09-17T02:42:43.847400+0000","last_active":"2024-09-17T02:42:45.870525+0000","last_peered":"2024-09-17T02:42:45.870525+0000","last_clean":"2024-09-17T02:42:45.870525+0000","last_became_active":"2024-09-17T02:42:43.843509+0000","last_became_peered":"2024-09-17T02:42:43.843509+0000","last_unstale":"2024-09-17T02:42:45.870525+0000","last_undegraded":"2024-09-17T02:42:45.870525+0000","last_fullsized":"2024-09-17T02:42:45.870525+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:21.738639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884317+0000","last_change":"2024-09-17T02:42:44.887685+0000","last_active":"2024-09-17T02:42:45.884317+0000","last_peered":"2024-09-17T02:42:45.884317+0000","last_clean":"2024-09-17T02:42:45.884317+0000","last_became_active":"2024-09-17T02:42:44.887127+0000","last_became_peered":"2024-09-17T02:42:44.887127+0000","last_unstale":"2024-09-17T02:42:45.884317+0000","last_undegraded":"2024-09-17T02:42:45.884317+0000","last_fullsized":"2024-09-17T02:42:45.884317+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:23.065669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883962+0000","last_change":"2024-09-17T02:42:43.859365+0000","last_active":"2024-09-17T02:42:45.883962+0000","last_peered":"2024-09-17T02:42:45.883962+0000","last_clean":"2024-09-17T02:42:45.883962+0000","last_became_active":"2024-09-17T02:42:43.856287+0000","last_became_peered":"2024-09-17T02:42:43.856287+0000","last_unstale":"2024-09-17T02:42:45.883962+0000","last_undegraded":"2024-09-17T02:42:45.883962+0000","last_fullsized":"2024-09-17T02:42:45.883962+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:18.332340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.874248+0000","last_change":"2024-09-17T02:42:44.880311+0000","last_active":"2024-09-17T02:42:45.874248+0000","last_peered":"2024-09-17T02:42:45.874248+0000","last_clean":"2024-09-17T02:42:45.874248+0000","last_became_active":"2024-09-17T02:42:44.879946+0000","last_became_peered":"2024-09-17T02:42:44.879946+0000","last_unstale":"2024-09-17T02:42:45.874248+0000","last_undegraded":"2024-09-17T02:42:45.874248+0000","last_fullsized":"2024-09-17T02:42:45.874248+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:32.989145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882301+0000","last_change":"2024-09-17T02:42:43.822424+0000","last_active":"2024-09-17T02:42:45.882301+0000","last_peered":"2024-09-17T02:42:45.882301+0000","last_clean":"2024-09-17T02:42:45.882301+0000","last_became_active":"2024-09-17T02:42:43.822051+0000","last_became_peered":"2024-09-17T02:42:43.822051+0000","last_unstale":"2024-09-17T02:42:45.882301+0000","last_undegraded":"2024-09-17T02:42:45.882301+0000","last_fullsized":"2024-09-17T02:42:45.882301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:25.389352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882609+0000","last_change":"2024-09-17T02:42:44.881443+0000","last_active":"2024-09-17T02:42:45.882609+0000","last_peered":"2024-09-17T02:42:45.882609+0000","last_clean":"2024-09-17T02:42:45.882609+0000","last_became_active":"2024-09-17T02:42:44.881026+0000","last_became_peered":"2024-09-17T02:42:44.881026+0000","last_unstale":"2024-09-17T02:42:45.882609+0000","last_undegraded":"2024-09-17T02:42:45.882609+0000","last_fullsized":"2024-09-17T02:42:45.882609+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866487+0000","last_change":"2024-09-17T02:42:43.845560+0000","last_active":"2024-09-17T02:42:45.866487+0000","last_peered":"2024-09-17T02:42:45.866487+0000","last_clean":"2024-09-17T02:42:45.866487+0000","last_became_active":"2024-09-17T02:42:43.845154+0000","last_became_peered":"2024-09-17T02:42:43.845154+0000","last_unstale":"2024-09-17T02:42:45.866487+0000","last_undegraded":"2024-09-17T02:42:45.866487+0000","last_fullsized":"2024-09-17T02:42:45.866487+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:33.597784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.887379+0000","last_change":"2024-09-17T02:42:44.881402+0000","last_active":"2024-09-17T02:42:45.887379+0000","last_peered":"2024-09-17T02:42:45.887379+0000","last_clean":"2024-09-17T02:42:45.887379+0000","last_became_active":"2024-09-17T02:42:44.880572+0000","last_became_peered":"2024-09-17T02:42:44.880572+0000","last_unstale":"2024-09-17T02:42:45.887379+0000","last_undegraded":"2024-09-17T02:42:45.887379+0000","last_fullsized":"2024-09-17T02:42:45.887379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:23.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.874020+0000","last_change":"2024-09-17T02:42:43.834185+0000","last_active":"2024-09-17T02:42:45.874020+0000","last_peered":"2024-09-17T02:42:45.874020+0000","last_clean":"2024-09-17T02:42:45.874020+0000","last_became_active":"2024-09-17T02:42:43.832748+0000","last_became_peered":"2024-09-17T02:42:43.832748+0000","last_unstale":"2024-09-17T02:42:45.874020+0000","last_undegraded":"2024-09-17T02:42:45.874020+0000","last_fullsized":"2024-09-17T02:42:45.874020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:19.775974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":168,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866609+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:42:45.866609+0000","last_peered":"2024-09-17T02:42:45.866609+0000","last_clean":"2024-09-17T02:42:45.866609+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:42:45.866609+0000","last_undegraded":"2024-09-17T02:42:45.866609+0000","last_fullsized":"2024-09-17T02:42:45.866609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":606208,"data_stored":262756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1208320,"data_stored":854770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":271096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":602112,"data_stored":262790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":614400,"data_stored":269426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1196032,"data_stored":853138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":602112,"data_stored":263282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":573440,"data_stored":248866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:42:49.461 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:42:49.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.647+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/1097053586 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5d080cfed0 msgr2=0x7f5d08073c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.647+0000 7f5d0e2d6640 1 --2- 172.21.15.131:0/1097053586 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5d080cfed0 0x7f5d08073c00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009900 tx=0x7f5cf001c7c0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.647+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/1097053586 shutdown_connections 2024-09-17T02:42:49.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.647+0000 7f5d0e2d6640 1 --2- 172.21.15.131:0/1097053586 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5d080cfed0 0x7f5d08073c00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.647+0000 7f5d0e2d6640 1 --2- 172.21.15.131:0/1097053586 >> v2:172.21.15.189:3300/0 conn(0x7f5d080cfac0 0x7f5d0815ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.647+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/1097053586 >> 172.21.15.131:0/1097053586 conn(0x7f5d080d13d0 msgr2=0x7f5d080734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:49.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.647+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/1097053586 shutdown_connections 2024-09-17T02:42:49.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.647+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/1097053586 wait complete. 2024-09-17T02:42:49.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.647+0000 7f5d0e2d6640 1 Processor -- start 2024-09-17T02:42:49.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.647+0000 7f5d0e2d6640 1 -- start start 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d0e2d6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d080cfac0 0x7f5d080b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d0e2d6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5d080cfed0 0x7f5d0815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d0e2d6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d0815f4b0 0x7f5d0815a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d0e2d6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08169110 con 0x7f5d080cfac0 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d0e2d6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08070170 con 0x7f5d0815f4b0 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d0e2d6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5d08169290 con 0x7f5d080cfed0 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d080cfac0 0x7f5d080b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d080cfac0 0x7f5d080b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58066/0 (socket says 172.21.15.131:58066) 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d07fff640 1 -- 172.21.15.131:0/2964299505 learned_addr learned my addr 172.21.15.131:0/2964299505 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d0c873640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d0815f4b0 0x7f5d0815a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d077fe640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5d080cfed0 0x7f5d0815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:49.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d07fff640 1 -- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5d080cfed0 msgr2=0x7f5d0815a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d07fff640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5d080cfed0 0x7f5d0815a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d07fff640 1 -- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d0815f4b0 msgr2=0x7f5d0815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d07fff640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d0815f4b0 0x7f5d0815a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d07fff640 1 -- 172.21.15.131:0/2964299505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d080b1170 con 0x7f5d080cfac0 2024-09-17T02:42:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d07fff640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d080cfac0 0x7f5d080b0c30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e7c0 tx=0x7f5d0000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d04ff9640 1 -- 172.21.15.131:0/2964299505 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000ccc0 con 0x7f5d080cfac0 2024-09-17T02:42:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d04ff9640 1 -- 172.21.15.131:0/2964299505 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d0000ce80 con 0x7f5d080cfac0 2024-09-17T02:42:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/2964299505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d081f9a70 con 0x7f5d080cfac0 2024-09-17T02:42:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d04ff9640 1 -- 172.21.15.131:0/2964299505 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00005280 con 0x7f5d080cfac0 2024-09-17T02:42:49.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/2964299505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d080b0430 con 0x7f5d080cfac0 2024-09-17T02:42:49.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/2964299505 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0815ef70 con 0x7f5d080cfac0 2024-09-17T02:42:49.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d04ff9640 1 -- 172.21.15.131:0/2964299505 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5d00005420 con 0x7f5d080cfac0 2024-09-17T02:42:49.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d04ff9640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5ce8073200 0x7f5ce8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:49.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.651+0000 7f5d04ff9640 1 -- 172.21.15.131:0/2964299505 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f5d00094d50 con 0x7f5d080cfac0 2024-09-17T02:42:49.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.655+0000 7f5d077fe640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5ce8073200 0x7f5ce8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:49.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.655+0000 7f5d077fe640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5ce8073200 0x7f5ce8075710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00ae000 tx=0x7f5cf001b000 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:49.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.659+0000 7f5d04ff9640 1 -- 172.21.15.131:0/2964299505 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5d00098030 con 0x7f5d080cfac0 2024-09-17T02:42:49.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.811+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/2964299505 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5d08150e00 con 0x7f5ce8073200 2024-09-17T02:42:49.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.823+0000 7f5d04ff9640 1 -- 172.21.15.131:0/2964299505 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251128 (secure 0 0 0) 0x7f5d08150e00 con 0x7f5ce8073200 2024-09-17T02:42:49.828 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:49.829 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:42:49.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5ce8073200 msgr2=0x7f5ce8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5ce8073200 0x7f5ce8075710 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00ae000 tx=0x7f5cf001b000 comp rx=0 tx=0).stop 2024-09-17T02:42:49.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/2964299505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d080cfac0 msgr2=0x7f5d080b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:49.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d080cfac0 0x7f5d080b0c30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000e7c0 tx=0x7f5d0000ec80 comp rx=0 tx=0).stop 2024-09-17T02:42:49.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/2964299505 shutdown_connections 2024-09-17T02:42:49.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5ce8073200 0x7f5ce8075710 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5d0815f4b0 0x7f5d0815a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5d080cfed0 0x7f5d0815a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 --2- 172.21.15.131:0/2964299505 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5d080cfac0 0x7f5d080b0c30 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:49.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/2964299505 >> 172.21.15.131:0/2964299505 conn(0x7f5d080d13d0 msgr2=0x7f5d080b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:49.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/2964299505 shutdown_connections 2024-09-17T02:42:49.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:49.827+0000 7f5d0e2d6640 1 -- 172.21.15.131:0/2964299505 wait complete. 2024-09-17T02:42:49.848 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-17T02:42:49.162416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238792,"kb_used_data":5892,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735928,"statfs":{"total":773094113280,"available":772849590272,"internally_reserved":0,"allocated":6033408,"data_stored":3286124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002204"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.828438+0000","last_change":"2024-09-17T02:42:43.828438+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.828438+0000","last_undegraded":"2024-09-17T02:42:43.828438+0000","last_fullsized":"2024-09-17T02:42:43.828438+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883357+0000","last_change":"2024-09-17T02:42:44.884814+0000","last_active":"2024-09-17T02:42:45.883357+0000","last_peered":"2024-09-17T02:42:45.883357+0000","last_clean":"2024-09-17T02:42:45.883357+0000","last_became_active":"2024-09-17T02:42:44.884432+0000","last_became_peered":"2024-09-17T02:42:44.884432+0000","last_unstale":"2024-09-17T02:42:45.883357+0000","last_undegraded":"2024-09-17T02:42:45.883357+0000","last_fullsized":"2024-09-17T02:42:45.883357+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:23.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884491+0000","last_change":"2024-09-17T02:42:44.854032+0000","last_active":"2024-09-17T02:42:45.884491+0000","last_peered":"2024-09-17T02:42:45.884491+0000","last_clean":"2024-09-17T02:42:45.884491+0000","last_became_active":"2024-09-17T02:42:44.853205+0000","last_became_peered":"2024-09-17T02:42:44.853205+0000","last_unstale":"2024-09-17T02:42:45.884491+0000","last_undegraded":"2024-09-17T02:42:45.884491+0000","last_fullsized":"2024-09-17T02:42:45.884491+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:18.997673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867302+0000","last_change":"2024-09-17T02:42:44.878169+0000","last_active":"2024-09-17T02:42:45.867302+0000","last_peered":"2024-09-17T02:42:45.867302+0000","last_clean":"2024-09-17T02:42:45.867302+0000","last_became_active":"2024-09-17T02:42:44.877402+0000","last_became_peered":"2024-09-17T02:42:44.877402+0000","last_unstale":"2024-09-17T02:42:45.867302+0000","last_undegraded":"2024-09-17T02:42:45.867302+0000","last_fullsized":"2024-09-17T02:42:45.867302+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:55.948606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882642+0000","last_change":"2024-09-17T02:42:44.872147+0000","last_active":"2024-09-17T02:42:45.882642+0000","last_peered":"2024-09-17T02:42:45.882642+0000","last_clean":"2024-09-17T02:42:45.882642+0000","last_became_active":"2024-09-17T02:42:44.871745+0000","last_became_peered":"2024-09-17T02:42:44.871745+0000","last_unstale":"2024-09-17T02:42:45.882642+0000","last_undegraded":"2024-09-17T02:42:45.882642+0000","last_fullsized":"2024-09-17T02:42:45.882642+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:29.767271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882210+0000","last_change":"2024-09-17T02:42:44.885865+0000","last_active":"2024-09-17T02:42:45.882210+0000","last_peered":"2024-09-17T02:42:45.882210+0000","last_clean":"2024-09-17T02:42:45.882210+0000","last_became_active":"2024-09-17T02:42:44.885512+0000","last_became_peered":"2024-09-17T02:42:44.885512+0000","last_unstale":"2024-09-17T02:42:45.882210+0000","last_undegraded":"2024-09-17T02:42:45.882210+0000","last_fullsized":"2024-09-17T02:42:45.882210+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:56.895371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870233+0000","last_change":"2024-09-17T02:42:44.886317+0000","last_active":"2024-09-17T02:42:45.870233+0000","last_peered":"2024-09-17T02:42:45.870233+0000","last_clean":"2024-09-17T02:42:45.870233+0000","last_became_active":"2024-09-17T02:42:44.885948+0000","last_became_peered":"2024-09-17T02:42:44.885948+0000","last_unstale":"2024-09-17T02:42:45.870233+0000","last_undegraded":"2024-09-17T02:42:45.870233+0000","last_fullsized":"2024-09-17T02:42:45.870233+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:44.318617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881521+0000","last_change":"2024-09-17T02:42:44.876332+0000","last_active":"2024-09-17T02:42:45.881521+0000","last_peered":"2024-09-17T02:42:45.881521+0000","last_clean":"2024-09-17T02:42:45.881521+0000","last_became_active":"2024-09-17T02:42:44.875594+0000","last_became_peered":"2024-09-17T02:42:44.875594+0000","last_unstale":"2024-09-17T02:42:45.881521+0000","last_undegraded":"2024-09-17T02:42:45.881521+0000","last_fullsized":"2024-09-17T02:42:45.881521+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:34.865339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884540+0000","last_change":"2024-09-17T02:42:44.854764+0000","last_active":"2024-09-17T02:42:45.884540+0000","last_peered":"2024-09-17T02:42:45.884540+0000","last_clean":"2024-09-17T02:42:45.884540+0000","last_became_active":"2024-09-17T02:42:44.853822+0000","last_became_peered":"2024-09-17T02:42:44.853822+0000","last_unstale":"2024-09-17T02:42:45.884540+0000","last_undegraded":"2024-09-17T02:42:45.884540+0000","last_fullsized":"2024-09-17T02:42:45.884540+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:18.705995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873645+0000","last_change":"2024-09-17T02:42:44.889606+0000","last_active":"2024-09-17T02:42:45.873645+0000","last_peered":"2024-09-17T02:42:45.873645+0000","last_clean":"2024-09-17T02:42:45.873645+0000","last_became_active":"2024-09-17T02:42:44.889274+0000","last_became_peered":"2024-09-17T02:42:44.889274+0000","last_unstale":"2024-09-17T02:42:45.873645+0000","last_undegraded":"2024-09-17T02:42:45.873645+0000","last_fullsized":"2024-09-17T02:42:45.873645+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:12.214981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809880+0000","last_change":"2024-09-17T02:42:43.809880+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809880+0000","last_undegraded":"2024-09-17T02:42:43.809880+0000","last_fullsized":"2024-09-17T02:42:43.809880+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882022+0000","last_change":"2024-09-17T02:42:44.880065+0000","last_active":"2024-09-17T02:42:45.882022+0000","last_peered":"2024-09-17T02:42:45.882022+0000","last_clean":"2024-09-17T02:42:45.882022+0000","last_became_active":"2024-09-17T02:42:44.876260+0000","last_became_peered":"2024-09-17T02:42:44.876260+0000","last_unstale":"2024-09-17T02:42:45.882022+0000","last_undegraded":"2024-09-17T02:42:45.882022+0000","last_fullsized":"2024-09-17T02:42:45.882022+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:34:02.925523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883058+0000","last_change":"2024-09-17T02:42:44.882550+0000","last_active":"2024-09-17T02:42:45.883058+0000","last_peered":"2024-09-17T02:42:45.883058+0000","last_clean":"2024-09-17T02:42:45.883058+0000","last_became_active":"2024-09-17T02:42:44.882123+0000","last_became_peered":"2024-09-17T02:42:44.882123+0000","last_unstale":"2024-09-17T02:42:45.883058+0000","last_undegraded":"2024-09-17T02:42:45.883058+0000","last_fullsized":"2024-09-17T02:42:45.883058+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:24.890536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.812574+0000","last_change":"2024-09-17T02:42:43.812574+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.812574+0000","last_undegraded":"2024-09-17T02:42:43.812574+0000","last_fullsized":"2024-09-17T02:42:43.812574+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866923+0000","last_change":"2024-09-17T02:42:44.877349+0000","last_active":"2024-09-17T02:42:45.866923+0000","last_peered":"2024-09-17T02:42:45.866923+0000","last_clean":"2024-09-17T02:42:45.866923+0000","last_became_active":"2024-09-17T02:42:44.876816+0000","last_became_peered":"2024-09-17T02:42:44.876816+0000","last_unstale":"2024-09-17T02:42:45.866923+0000","last_undegraded":"2024-09-17T02:42:45.866923+0000","last_fullsized":"2024-09-17T02:42:45.866923+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:22.741898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881823+0000","last_change":"2024-09-17T02:42:44.876906+0000","last_active":"2024-09-17T02:42:45.881823+0000","last_peered":"2024-09-17T02:42:45.881823+0000","last_clean":"2024-09-17T02:42:45.881823+0000","last_became_active":"2024-09-17T02:42:44.875966+0000","last_became_peered":"2024-09-17T02:42:44.875966+0000","last_unstale":"2024-09-17T02:42:45.881823+0000","last_undegraded":"2024-09-17T02:42:45.881823+0000","last_fullsized":"2024-09-17T02:42:45.881823+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:06.983520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.813038+0000","last_change":"2024-09-17T02:42:43.813038+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.813038+0000","last_undegraded":"2024-09-17T02:42:43.813038+0000","last_fullsized":"2024-09-17T02:42:43.813038+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867528+0000","last_change":"2024-09-17T02:42:44.888487+0000","last_active":"2024-09-17T02:42:45.867528+0000","last_peered":"2024-09-17T02:42:45.867528+0000","last_clean":"2024-09-17T02:42:45.867528+0000","last_became_active":"2024-09-17T02:42:44.887970+0000","last_became_peered":"2024-09-17T02:42:44.887970+0000","last_unstale":"2024-09-17T02:42:45.867528+0000","last_undegraded":"2024-09-17T02:42:45.867528+0000","last_fullsized":"2024-09-17T02:42:45.867528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:42.359754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884599+0000","last_change":"2024-09-17T02:42:44.881307+0000","last_active":"2024-09-17T02:42:45.884599+0000","last_peered":"2024-09-17T02:42:45.884599+0000","last_clean":"2024-09-17T02:42:45.884599+0000","last_became_active":"2024-09-17T02:42:44.880605+0000","last_became_peered":"2024-09-17T02:42:44.880605+0000","last_unstale":"2024-09-17T02:42:45.884599+0000","last_undegraded":"2024-09-17T02:42:45.884599+0000","last_fullsized":"2024-09-17T02:42:45.884599+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:49.563614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883282+0000","last_change":"2024-09-17T02:42:44.887466+0000","last_active":"2024-09-17T02:42:45.883282+0000","last_peered":"2024-09-17T02:42:45.883282+0000","last_clean":"2024-09-17T02:42:45.883282+0000","last_became_active":"2024-09-17T02:42:44.885044+0000","last_became_peered":"2024-09-17T02:42:44.885044+0000","last_unstale":"2024-09-17T02:42:45.883282+0000","last_undegraded":"2024-09-17T02:42:45.883282+0000","last_fullsized":"2024-09-17T02:42:45.883282+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.263210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867244+0000","last_change":"2024-09-17T02:42:44.876044+0000","last_active":"2024-09-17T02:42:45.867244+0000","last_peered":"2024-09-17T02:42:45.867244+0000","last_clean":"2024-09-17T02:42:45.867244+0000","last_became_active":"2024-09-17T02:42:44.873541+0000","last_became_peered":"2024-09-17T02:42:44.873541+0000","last_unstale":"2024-09-17T02:42:45.867244+0000","last_undegraded":"2024-09-17T02:42:45.867244+0000","last_fullsized":"2024-09-17T02:42:45.867244+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:56.895371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883856+0000","last_change":"2024-09-17T02:42:44.879337+0000","last_active":"2024-09-17T02:42:45.883856+0000","last_peered":"2024-09-17T02:42:45.883856+0000","last_clean":"2024-09-17T02:42:45.883856+0000","last_became_active":"2024-09-17T02:42:44.878982+0000","last_became_peered":"2024-09-17T02:42:44.878982+0000","last_unstale":"2024-09-17T02:42:45.883856+0000","last_undegraded":"2024-09-17T02:42:45.883856+0000","last_fullsized":"2024-09-17T02:42:45.883856+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:59.473102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882578+0000","last_change":"2024-09-17T02:42:44.856410+0000","last_active":"2024-09-17T02:42:45.882578+0000","last_peered":"2024-09-17T02:42:45.882578+0000","last_clean":"2024-09-17T02:42:45.882578+0000","last_became_active":"2024-09-17T02:42:44.854487+0000","last_became_peered":"2024-09-17T02:42:44.854487+0000","last_unstale":"2024-09-17T02:42:45.882578+0000","last_undegraded":"2024-09-17T02:42:45.882578+0000","last_fullsized":"2024-09-17T02:42:45.882578+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:07.107071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882985+0000","last_change":"2024-09-17T02:42:44.870968+0000","last_active":"2024-09-17T02:42:45.882985+0000","last_peered":"2024-09-17T02:42:45.882985+0000","last_clean":"2024-09-17T02:42:45.882985+0000","last_became_active":"2024-09-17T02:42:44.870254+0000","last_became_peered":"2024-09-17T02:42:44.870254+0000","last_unstale":"2024-09-17T02:42:45.882985+0000","last_undegraded":"2024-09-17T02:42:45.882985+0000","last_fullsized":"2024-09-17T02:42:45.882985+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:15.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.826441+0000","last_change":"2024-09-17T02:42:43.826441+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.826441+0000","last_undegraded":"2024-09-17T02:42:43.826441+0000","last_fullsized":"2024-09-17T02:42:43.826441+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884665+0000","last_change":"2024-09-17T02:42:44.875582+0000","last_active":"2024-09-17T02:42:45.884665+0000","last_peered":"2024-09-17T02:42:45.884665+0000","last_clean":"2024-09-17T02:42:45.884665+0000","last_became_active":"2024-09-17T02:42:44.875205+0000","last_became_peered":"2024-09-17T02:42:44.875205+0000","last_unstale":"2024-09-17T02:42:45.884665+0000","last_undegraded":"2024-09-17T02:42:45.884665+0000","last_fullsized":"2024-09-17T02:42:45.884665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:22:23.818837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881878+0000","last_change":"2024-09-17T02:42:44.885311+0000","last_active":"2024-09-17T02:42:45.881878+0000","last_peered":"2024-09-17T02:42:45.881878+0000","last_clean":"2024-09-17T02:42:45.881878+0000","last_became_active":"2024-09-17T02:42:44.884647+0000","last_became_peered":"2024-09-17T02:42:44.884647+0000","last_unstale":"2024-09-17T02:42:45.881878+0000","last_undegraded":"2024-09-17T02:42:45.881878+0000","last_fullsized":"2024-09-17T02:42:45.881878+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:20.906053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883665+0000","last_change":"2024-09-17T02:42:44.877183+0000","last_active":"2024-09-17T02:42:45.883665+0000","last_peered":"2024-09-17T02:42:45.883665+0000","last_clean":"2024-09-17T02:42:45.883665+0000","last_became_active":"2024-09-17T02:42:44.875043+0000","last_became_peered":"2024-09-17T02:42:44.875043+0000","last_unstale":"2024-09-17T02:42:45.883665+0000","last_undegraded":"2024-09-17T02:42:45.883665+0000","last_fullsized":"2024-09-17T02:42:45.883665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:53.445161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809918+0000","last_change":"2024-09-17T02:42:43.809918+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809918+0000","last_undegraded":"2024-09-17T02:42:43.809918+0000","last_fullsized":"2024-09-17T02:42:43.809918+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883971+0000","last_change":"2024-09-17T02:42:44.869378+0000","last_active":"2024-09-17T02:42:45.883971+0000","last_peered":"2024-09-17T02:42:45.883971+0000","last_clean":"2024-09-17T02:42:45.883971+0000","last_became_active":"2024-09-17T02:42:44.868147+0000","last_became_peered":"2024-09-17T02:42:44.868147+0000","last_unstale":"2024-09-17T02:42:45.883971+0000","last_undegraded":"2024-09-17T02:42:45.883971+0000","last_fullsized":"2024-09-17T02:42:45.883971+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:03.959077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882887+0000","last_change":"2024-09-17T02:42:44.879236+0000","last_active":"2024-09-17T02:42:45.882887+0000","last_peered":"2024-09-17T02:42:45.882887+0000","last_clean":"2024-09-17T02:42:45.882887+0000","last_became_active":"2024-09-17T02:42:44.878872+0000","last_became_peered":"2024-09-17T02:42:44.878872+0000","last_unstale":"2024-09-17T02:42:45.882887+0000","last_undegraded":"2024-09-17T02:42:45.882887+0000","last_fullsized":"2024-09-17T02:42:45.882887+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:39.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882661+0000","last_change":"2024-09-17T02:42:44.853789+0000","last_active":"2024-09-17T02:42:45.882661+0000","last_peered":"2024-09-17T02:42:45.882661+0000","last_clean":"2024-09-17T02:42:45.882661+0000","last_became_active":"2024-09-17T02:42:44.852870+0000","last_became_peered":"2024-09-17T02:42:44.852870+0000","last_unstale":"2024-09-17T02:42:45.882661+0000","last_undegraded":"2024-09-17T02:42:45.882661+0000","last_fullsized":"2024-09-17T02:42:45.882661+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.016019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882301+0000","last_change":"2024-09-17T02:42:44.884160+0000","last_active":"2024-09-17T02:42:45.882301+0000","last_peered":"2024-09-17T02:42:45.882301+0000","last_clean":"2024-09-17T02:42:45.882301+0000","last_became_active":"2024-09-17T02:42:44.883295+0000","last_became_peered":"2024-09-17T02:42:44.883295+0000","last_unstale":"2024-09-17T02:42:45.882301+0000","last_undegraded":"2024-09-17T02:42:45.882301+0000","last_fullsized":"2024-09-17T02:42:45.882301+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:13.871419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":19,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885185+0000","last_change":"2024-09-17T02:42:43.845735+0000","last_active":"2024-09-17T02:42:45.885185+0000","last_peered":"2024-09-17T02:42:45.885185+0000","last_clean":"2024-09-17T02:42:45.885185+0000","last_became_active":"2024-09-17T02:42:43.845485+0000","last_became_peered":"2024-09-17T02:42:43.845485+0000","last_unstale":"2024-09-17T02:42:45.885185+0000","last_undegraded":"2024-09-17T02:42:45.885185+0000","last_fullsized":"2024-09-17T02:42:45.885185+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:19.775974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881559+0000","last_change":"2024-09-17T02:42:44.875131+0000","last_active":"2024-09-17T02:42:45.881559+0000","last_peered":"2024-09-17T02:42:45.881559+0000","last_clean":"2024-09-17T02:42:45.881559+0000","last_became_active":"2024-09-17T02:42:44.874308+0000","last_became_peered":"2024-09-17T02:42:44.874308+0000","last_unstale":"2024-09-17T02:42:45.881559+0000","last_undegraded":"2024-09-17T02:42:45.881559+0000","last_fullsized":"2024-09-17T02:42:45.881559+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:05.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882119+0000","last_change":"2024-09-17T02:42:43.865164+0000","last_active":"2024-09-17T02:42:45.882119+0000","last_peered":"2024-09-17T02:42:45.882119+0000","last_clean":"2024-09-17T02:42:45.882119+0000","last_became_active":"2024-09-17T02:42:43.862678+0000","last_became_peered":"2024-09-17T02:42:43.862678+0000","last_unstale":"2024-09-17T02:42:45.882119+0000","last_undegraded":"2024-09-17T02:42:45.882119+0000","last_fullsized":"2024-09-17T02:42:45.882119+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:46.632441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883731+0000","last_change":"2024-09-17T02:42:44.870094+0000","last_active":"2024-09-17T02:42:45.883731+0000","last_peered":"2024-09-17T02:42:45.883731+0000","last_clean":"2024-09-17T02:42:45.883731+0000","last_became_active":"2024-09-17T02:42:44.869245+0000","last_became_peered":"2024-09-17T02:42:44.869245+0000","last_unstale":"2024-09-17T02:42:45.883731+0000","last_undegraded":"2024-09-17T02:42:45.883731+0000","last_fullsized":"2024-09-17T02:42:45.883731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:18.419470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873178+0000","last_change":"2024-09-17T02:42:43.837893+0000","last_active":"2024-09-17T02:42:45.873178+0000","last_peered":"2024-09-17T02:42:45.873178+0000","last_clean":"2024-09-17T02:42:45.873178+0000","last_became_active":"2024-09-17T02:42:43.833094+0000","last_became_peered":"2024-09-17T02:42:43.833094+0000","last_unstale":"2024-09-17T02:42:45.873178+0000","last_undegraded":"2024-09-17T02:42:45.873178+0000","last_fullsized":"2024-09-17T02:42:45.873178+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:27.547510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870591+0000","last_change":"2024-09-17T02:42:44.876821+0000","last_active":"2024-09-17T02:42:45.870591+0000","last_peered":"2024-09-17T02:42:45.870591+0000","last_clean":"2024-09-17T02:42:45.870591+0000","last_became_active":"2024-09-17T02:42:44.876427+0000","last_became_peered":"2024-09-17T02:42:44.876427+0000","last_unstale":"2024-09-17T02:42:45.870591+0000","last_undegraded":"2024-09-17T02:42:45.870591+0000","last_fullsized":"2024-09-17T02:42:45.870591+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:29.285476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884021+0000","last_change":"2024-09-17T02:42:43.859536+0000","last_active":"2024-09-17T02:42:45.884021+0000","last_peered":"2024-09-17T02:42:45.884021+0000","last_clean":"2024-09-17T02:42:45.884021+0000","last_became_active":"2024-09-17T02:42:43.859191+0000","last_became_peered":"2024-09-17T02:42:43.859191+0000","last_unstale":"2024-09-17T02:42:45.884021+0000","last_undegraded":"2024-09-17T02:42:45.884021+0000","last_fullsized":"2024-09-17T02:42:45.884021+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:34.772218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.886237+0000","last_change":"2024-09-17T02:42:44.881758+0000","last_active":"2024-09-17T02:42:45.886237+0000","last_peered":"2024-09-17T02:42:45.886237+0000","last_clean":"2024-09-17T02:42:45.886237+0000","last_became_active":"2024-09-17T02:42:44.880037+0000","last_became_peered":"2024-09-17T02:42:44.880037+0000","last_unstale":"2024-09-17T02:42:45.886237+0000","last_undegraded":"2024-09-17T02:42:45.886237+0000","last_fullsized":"2024-09-17T02:42:45.886237+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:49.224409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884002+0000","last_change":"2024-09-17T02:42:43.859603+0000","last_active":"2024-09-17T02:42:45.884002+0000","last_peered":"2024-09-17T02:42:45.884002+0000","last_clean":"2024-09-17T02:42:45.884002+0000","last_became_active":"2024-09-17T02:42:43.856481+0000","last_became_peered":"2024-09-17T02:42:43.856481+0000","last_unstale":"2024-09-17T02:42:45.884002+0000","last_undegraded":"2024-09-17T02:42:45.884002+0000","last_fullsized":"2024-09-17T02:42:45.884002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:31.985885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884898+0000","last_change":"2024-09-17T02:42:44.891687+0000","last_active":"2024-09-17T02:42:45.884898+0000","last_peered":"2024-09-17T02:42:45.884898+0000","last_clean":"2024-09-17T02:42:45.884898+0000","last_became_active":"2024-09-17T02:42:44.890077+0000","last_became_peered":"2024-09-17T02:42:44.890077+0000","last_unstale":"2024-09-17T02:42:45.884898+0000","last_undegraded":"2024-09-17T02:42:45.884898+0000","last_fullsized":"2024-09-17T02:42:45.884898+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:46:21.265517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884101+0000","last_change":"2024-09-17T02:42:43.822676+0000","last_active":"2024-09-17T02:42:45.884101+0000","last_peered":"2024-09-17T02:42:45.884101+0000","last_clean":"2024-09-17T02:42:45.884101+0000","last_became_active":"2024-09-17T02:42:43.822225+0000","last_became_peered":"2024-09-17T02:42:43.822225+0000","last_unstale":"2024-09-17T02:42:45.884101+0000","last_undegraded":"2024-09-17T02:42:45.884101+0000","last_fullsized":"2024-09-17T02:42:45.884101+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:38.838275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873695+0000","last_change":"2024-09-17T02:42:44.880963+0000","last_active":"2024-09-17T02:42:45.873695+0000","last_peered":"2024-09-17T02:42:45.873695+0000","last_clean":"2024-09-17T02:42:45.873695+0000","last_became_active":"2024-09-17T02:42:44.880257+0000","last_became_peered":"2024-09-17T02:42:44.880257+0000","last_unstale":"2024-09-17T02:42:45.873695+0000","last_undegraded":"2024-09-17T02:42:45.873695+0000","last_fullsized":"2024-09-17T02:42:45.873695+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:09.416640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885098+0000","last_change":"2024-09-17T02:42:43.814090+0000","last_active":"2024-09-17T02:42:45.885098+0000","last_peered":"2024-09-17T02:42:45.885098+0000","last_clean":"2024-09-17T02:42:45.885098+0000","last_became_active":"2024-09-17T02:42:43.813685+0000","last_became_peered":"2024-09-17T02:42:43.813685+0000","last_unstale":"2024-09-17T02:42:45.885098+0000","last_undegraded":"2024-09-17T02:42:45.885098+0000","last_fullsized":"2024-09-17T02:42:45.885098+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:06.103811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867600+0000","last_change":"2024-09-17T02:42:44.878439+0000","last_active":"2024-09-17T02:42:45.867600+0000","last_peered":"2024-09-17T02:42:45.867600+0000","last_clean":"2024-09-17T02:42:45.867600+0000","last_became_active":"2024-09-17T02:42:44.875651+0000","last_became_peered":"2024-09-17T02:42:44.875651+0000","last_unstale":"2024-09-17T02:42:45.867600+0000","last_undegraded":"2024-09-17T02:42:45.867600+0000","last_fullsized":"2024-09-17T02:42:45.867600+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:29.767271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883310+0000","last_change":"2024-09-17T02:42:43.855843+0000","last_active":"2024-09-17T02:42:45.883310+0000","last_peered":"2024-09-17T02:42:45.883310+0000","last_clean":"2024-09-17T02:42:45.883310+0000","last_became_active":"2024-09-17T02:42:43.854712+0000","last_became_peered":"2024-09-17T02:42:43.854712+0000","last_unstale":"2024-09-17T02:42:45.883310+0000","last_undegraded":"2024-09-17T02:42:45.883310+0000","last_fullsized":"2024-09-17T02:42:45.883310+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:19:02.429366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884260+0000","last_change":"2024-09-17T02:42:44.875385+0000","last_active":"2024-09-17T02:42:45.884260+0000","last_peered":"2024-09-17T02:42:45.884260+0000","last_clean":"2024-09-17T02:42:45.884260+0000","last_became_active":"2024-09-17T02:42:44.874816+0000","last_became_peered":"2024-09-17T02:42:44.874816+0000","last_unstale":"2024-09-17T02:42:45.884260+0000","last_undegraded":"2024-09-17T02:42:45.884260+0000","last_fullsized":"2024-09-17T02:42:45.884260+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:36.940445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867765+0000","last_change":"2024-09-17T02:42:43.826860+0000","last_active":"2024-09-17T02:42:45.867765+0000","last_peered":"2024-09-17T02:42:45.867765+0000","last_clean":"2024-09-17T02:42:45.867765+0000","last_became_active":"2024-09-17T02:42:43.826423+0000","last_became_peered":"2024-09-17T02:42:43.826423+0000","last_unstale":"2024-09-17T02:42:45.867765+0000","last_undegraded":"2024-09-17T02:42:45.867765+0000","last_fullsized":"2024-09-17T02:42:45.867765+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:57.315329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866970+0000","last_change":"2024-09-17T02:42:44.889033+0000","last_active":"2024-09-17T02:42:45.866970+0000","last_peered":"2024-09-17T02:42:45.866970+0000","last_clean":"2024-09-17T02:42:45.866970+0000","last_became_active":"2024-09-17T02:42:44.888575+0000","last_became_peered":"2024-09-17T02:42:44.888575+0000","last_unstale":"2024-09-17T02:42:45.866970+0000","last_undegraded":"2024-09-17T02:42:45.866970+0000","last_fullsized":"2024-09-17T02:42:45.866970+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:10.824751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867254+0000","last_change":"2024-09-17T02:42:43.835429+0000","last_active":"2024-09-17T02:42:45.867254+0000","last_peered":"2024-09-17T02:42:45.867254+0000","last_clean":"2024-09-17T02:42:45.867254+0000","last_became_active":"2024-09-17T02:42:43.835072+0000","last_became_peered":"2024-09-17T02:42:43.835072+0000","last_unstale":"2024-09-17T02:42:45.867254+0000","last_undegraded":"2024-09-17T02:42:45.867254+0000","last_fullsized":"2024-09-17T02:42:45.867254+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:28.861850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.807944+0000","last_change":"2024-09-17T02:42:43.807944+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.807944+0000","last_undegraded":"2024-09-17T02:42:43.807944+0000","last_fullsized":"2024-09-17T02:42:43.807944+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885035+0000","last_change":"2024-09-17T02:42:43.844104+0000","last_active":"2024-09-17T02:42:45.885035+0000","last_peered":"2024-09-17T02:42:45.885035+0000","last_clean":"2024-09-17T02:42:45.885035+0000","last_became_active":"2024-09-17T02:42:43.843661+0000","last_became_peered":"2024-09-17T02:42:43.843661+0000","last_unstale":"2024-09-17T02:42:45.885035+0000","last_undegraded":"2024-09-17T02:42:45.885035+0000","last_fullsized":"2024-09-17T02:42:45.885035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:14.987141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882393+0000","last_change":"2024-09-17T02:42:44.853449+0000","last_active":"2024-09-17T02:42:45.882393+0000","last_peered":"2024-09-17T02:42:45.882393+0000","last_clean":"2024-09-17T02:42:45.882393+0000","last_became_active":"2024-09-17T02:42:44.852561+0000","last_became_peered":"2024-09-17T02:42:44.852561+0000","last_unstale":"2024-09-17T02:42:45.882393+0000","last_undegraded":"2024-09-17T02:42:45.882393+0000","last_fullsized":"2024-09-17T02:42:45.882393+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.871956+0000","last_change":"2024-09-17T02:42:43.864191+0000","last_active":"2024-09-17T02:42:45.871956+0000","last_peered":"2024-09-17T02:42:45.871956+0000","last_clean":"2024-09-17T02:42:45.871956+0000","last_became_active":"2024-09-17T02:42:43.863400+0000","last_became_peered":"2024-09-17T02:42:43.863400+0000","last_unstale":"2024-09-17T02:42:45.871956+0000","last_undegraded":"2024-09-17T02:42:45.871956+0000","last_fullsized":"2024-09-17T02:42:45.871956+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:09.821491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885484+0000","last_change":"2024-09-17T02:42:44.880368+0000","last_active":"2024-09-17T02:42:45.885484+0000","last_peered":"2024-09-17T02:42:45.885484+0000","last_clean":"2024-09-17T02:42:45.885484+0000","last_became_active":"2024-09-17T02:42:44.879881+0000","last_became_peered":"2024-09-17T02:42:44.879881+0000","last_unstale":"2024-09-17T02:42:45.885484+0000","last_undegraded":"2024-09-17T02:42:45.885484+0000","last_fullsized":"2024-09-17T02:42:45.885484+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:58.252210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883189+0000","last_change":"2024-09-17T02:42:43.841976+0000","last_active":"2024-09-17T02:42:45.883189+0000","last_peered":"2024-09-17T02:42:45.883189+0000","last_clean":"2024-09-17T02:42:45.883189+0000","last_became_active":"2024-09-17T02:42:43.841094+0000","last_became_peered":"2024-09-17T02:42:43.841094+0000","last_unstale":"2024-09-17T02:42:45.883189+0000","last_undegraded":"2024-09-17T02:42:45.883189+0000","last_fullsized":"2024-09-17T02:42:45.883189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:29.331958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.808614+0000","last_change":"2024-09-17T02:42:43.808614+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.808614+0000","last_undegraded":"2024-09-17T02:42:43.808614+0000","last_fullsized":"2024-09-17T02:42:43.808614+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866984+0000","last_change":"2024-09-17T02:42:43.843349+0000","last_active":"2024-09-17T02:42:45.866984+0000","last_peered":"2024-09-17T02:42:45.866984+0000","last_clean":"2024-09-17T02:42:45.866984+0000","last_became_active":"2024-09-17T02:42:43.842950+0000","last_became_peered":"2024-09-17T02:42:43.842950+0000","last_unstale":"2024-09-17T02:42:45.866984+0000","last_undegraded":"2024-09-17T02:42:45.866984+0000","last_fullsized":"2024-09-17T02:42:45.866984+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:22.974854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882439+0000","last_change":"2024-09-17T02:42:44.886488+0000","last_active":"2024-09-17T02:42:45.882439+0000","last_peered":"2024-09-17T02:42:45.882439+0000","last_clean":"2024-09-17T02:42:45.882439+0000","last_became_active":"2024-09-17T02:42:44.886106+0000","last_became_peered":"2024-09-17T02:42:44.886106+0000","last_unstale":"2024-09-17T02:42:45.882439+0000","last_undegraded":"2024-09-17T02:42:45.882439+0000","last_fullsized":"2024-09-17T02:42:45.882439+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:55.948606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881989+0000","last_change":"2024-09-17T02:42:43.825883+0000","last_active":"2024-09-17T02:42:45.881989+0000","last_peered":"2024-09-17T02:42:45.881989+0000","last_clean":"2024-09-17T02:42:45.881989+0000","last_became_active":"2024-09-17T02:42:43.825522+0000","last_became_peered":"2024-09-17T02:42:43.825522+0000","last_unstale":"2024-09-17T02:42:45.881989+0000","last_undegraded":"2024-09-17T02:42:45.881989+0000","last_fullsized":"2024-09-17T02:42:45.881989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:22:57.287148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885037+0000","last_change":"2024-09-17T02:42:44.878272+0000","last_active":"2024-09-17T02:42:45.885037+0000","last_peered":"2024-09-17T02:42:45.885037+0000","last_clean":"2024-09-17T02:42:45.885037+0000","last_became_active":"2024-09-17T02:42:44.877921+0000","last_became_peered":"2024-09-17T02:42:44.877921+0000","last_unstale":"2024-09-17T02:42:45.885037+0000","last_undegraded":"2024-09-17T02:42:45.885037+0000","last_fullsized":"2024-09-17T02:42:45.885037+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:14.742444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":23,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881967+0000","last_change":"2024-09-17T02:42:43.864045+0000","last_active":"2024-09-17T02:42:45.881967+0000","last_peered":"2024-09-17T02:42:45.881967+0000","last_clean":"2024-09-17T02:42:45.881967+0000","last_became_active":"2024-09-17T02:42:43.862689+0000","last_became_peered":"2024-09-17T02:42:43.862689+0000","last_unstale":"2024-09-17T02:42:45.881967+0000","last_undegraded":"2024-09-17T02:42:45.881967+0000","last_fullsized":"2024-09-17T02:42:45.881967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:30.667789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870056+0000","last_change":"2024-09-17T02:42:44.871420+0000","last_active":"2024-09-17T02:42:45.870056+0000","last_peered":"2024-09-17T02:42:45.870056+0000","last_clean":"2024-09-17T02:42:45.870056+0000","last_became_active":"2024-09-17T02:42:44.871075+0000","last_became_peered":"2024-09-17T02:42:44.871075+0000","last_unstale":"2024-09-17T02:42:45.870056+0000","last_undegraded":"2024-09-17T02:42:45.870056+0000","last_fullsized":"2024-09-17T02:42:45.870056+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:21.340145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882977+0000","last_change":"2024-09-17T02:42:43.865760+0000","last_active":"2024-09-17T02:42:45.882977+0000","last_peered":"2024-09-17T02:42:45.882977+0000","last_clean":"2024-09-17T02:42:45.882977+0000","last_became_active":"2024-09-17T02:42:43.862428+0000","last_became_peered":"2024-09-17T02:42:43.862428+0000","last_unstale":"2024-09-17T02:42:45.882977+0000","last_undegraded":"2024-09-17T02:42:45.882977+0000","last_fullsized":"2024-09-17T02:42:45.882977+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:00.644920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809542+0000","last_change":"2024-09-17T02:42:43.809542+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809542+0000","last_undegraded":"2024-09-17T02:42:43.809542+0000","last_fullsized":"2024-09-17T02:42:43.809542+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882280+0000","last_change":"2024-09-17T02:42:43.839506+0000","last_active":"2024-09-17T02:42:45.882280+0000","last_peered":"2024-09-17T02:42:45.882280+0000","last_clean":"2024-09-17T02:42:45.882280+0000","last_became_active":"2024-09-17T02:42:43.838617+0000","last_became_peered":"2024-09-17T02:42:43.838617+0000","last_unstale":"2024-09-17T02:42:45.882280+0000","last_undegraded":"2024-09-17T02:42:45.882280+0000","last_fullsized":"2024-09-17T02:42:45.882280+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:04.244931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883433+0000","last_change":"2024-09-17T02:42:44.876771+0000","last_active":"2024-09-17T02:42:45.883433+0000","last_peered":"2024-09-17T02:42:45.883433+0000","last_clean":"2024-09-17T02:42:45.883433+0000","last_became_active":"2024-09-17T02:42:44.876438+0000","last_became_peered":"2024-09-17T02:42:44.876438+0000","last_unstale":"2024-09-17T02:42:45.883433+0000","last_undegraded":"2024-09-17T02:42:45.883433+0000","last_fullsized":"2024-09-17T02:42:45.883433+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:28.921453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867493+0000","last_change":"2024-09-17T02:42:43.862172+0000","last_active":"2024-09-17T02:42:45.867493+0000","last_peered":"2024-09-17T02:42:45.867493+0000","last_clean":"2024-09-17T02:42:45.867493+0000","last_became_active":"2024-09-17T02:42:43.861853+0000","last_became_peered":"2024-09-17T02:42:43.861853+0000","last_unstale":"2024-09-17T02:42:45.867493+0000","last_undegraded":"2024-09-17T02:42:45.867493+0000","last_fullsized":"2024-09-17T02:42:45.867493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:00.644920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883740+0000","last_change":"2024-09-17T02:42:44.875816+0000","last_active":"2024-09-17T02:42:45.883740+0000","last_peered":"2024-09-17T02:42:45.883740+0000","last_clean":"2024-09-17T02:42:45.883740+0000","last_became_active":"2024-09-17T02:42:44.875443+0000","last_became_peered":"2024-09-17T02:42:44.875443+0000","last_unstale":"2024-09-17T02:42:45.883740+0000","last_undegraded":"2024-09-17T02:42:45.883740+0000","last_fullsized":"2024-09-17T02:42:45.883740+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.263210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884349+0000","last_change":"2024-09-17T02:42:43.842888+0000","last_active":"2024-09-17T02:42:45.884349+0000","last_peered":"2024-09-17T02:42:45.884349+0000","last_clean":"2024-09-17T02:42:45.884349+0000","last_became_active":"2024-09-17T02:42:43.842563+0000","last_became_peered":"2024-09-17T02:42:43.842563+0000","last_unstale":"2024-09-17T02:42:45.884349+0000","last_undegraded":"2024-09-17T02:42:45.884349+0000","last_fullsized":"2024-09-17T02:42:45.884349+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:37.379782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883984+0000","last_change":"2024-09-17T02:42:44.875239+0000","last_active":"2024-09-17T02:42:45.883984+0000","last_peered":"2024-09-17T02:42:45.883984+0000","last_clean":"2024-09-17T02:42:45.883984+0000","last_became_active":"2024-09-17T02:42:44.874268+0000","last_became_peered":"2024-09-17T02:42:44.874268+0000","last_unstale":"2024-09-17T02:42:45.883984+0000","last_undegraded":"2024-09-17T02:42:45.883984+0000","last_fullsized":"2024-09-17T02:42:45.883984+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:30.019802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T02:42:43.849754+0000","last_change":"2024-09-17T02:42:43.840810+0000","last_active":"2024-09-17T02:42:43.849754+0000","last_peered":"2024-09-17T02:42:43.849754+0000","last_clean":"2024-09-17T02:42:43.849754+0000","last_became_active":"2024-09-17T02:42:43.839861+0000","last_became_peered":"2024-09-17T02:42:43.839861+0000","last_unstale":"2024-09-17T02:42:43.849754+0000","last_undegraded":"2024-09-17T02:42:43.849754+0000","last_fullsized":"2024-09-17T02:42:43.849754+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:14.684387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867465+0000","last_change":"2024-09-17T02:42:44.877843+0000","last_active":"2024-09-17T02:42:45.867465+0000","last_peered":"2024-09-17T02:42:45.867465+0000","last_clean":"2024-09-17T02:42:45.867465+0000","last_became_active":"2024-09-17T02:42:44.877029+0000","last_became_peered":"2024-09-17T02:42:44.877029+0000","last_unstale":"2024-09-17T02:42:45.867465+0000","last_undegraded":"2024-09-17T02:42:45.867465+0000","last_fullsized":"2024-09-17T02:42:45.867465+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883764+0000","last_change":"2024-09-17T02:42:43.860370+0000","last_active":"2024-09-17T02:42:45.883764+0000","last_peered":"2024-09-17T02:42:45.883764+0000","last_clean":"2024-09-17T02:42:45.883764+0000","last_became_active":"2024-09-17T02:42:43.859094+0000","last_became_peered":"2024-09-17T02:42:43.859094+0000","last_unstale":"2024-09-17T02:42:45.883764+0000","last_undegraded":"2024-09-17T02:42:45.883764+0000","last_fullsized":"2024-09-17T02:42:45.883764+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:15.987844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.810796+0000","last_change":"2024-09-17T02:42:43.810796+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.810796+0000","last_undegraded":"2024-09-17T02:42:43.810796+0000","last_fullsized":"2024-09-17T02:42:43.810796+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870452+0000","last_change":"2024-09-17T02:42:43.862720+0000","last_active":"2024-09-17T02:42:45.870452+0000","last_peered":"2024-09-17T02:42:45.870452+0000","last_clean":"2024-09-17T02:42:45.870452+0000","last_became_active":"2024-09-17T02:42:43.862004+0000","last_became_peered":"2024-09-17T02:42:43.862004+0000","last_unstale":"2024-09-17T02:42:45.870452+0000","last_undegraded":"2024-09-17T02:42:45.870452+0000","last_fullsized":"2024-09-17T02:42:45.870452+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:41.356494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882795+0000","last_change":"2024-09-17T02:42:44.883533+0000","last_active":"2024-09-17T02:42:45.882795+0000","last_peered":"2024-09-17T02:42:45.882795+0000","last_clean":"2024-09-17T02:42:45.882795+0000","last_became_active":"2024-09-17T02:42:44.883166+0000","last_became_peered":"2024-09-17T02:42:44.883166+0000","last_unstale":"2024-09-17T02:42:45.882795+0000","last_undegraded":"2024-09-17T02:42:45.882795+0000","last_fullsized":"2024-09-17T02:42:45.882795+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.016019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.869776+0000","last_change":"2024-09-17T02:42:43.831772+0000","last_active":"2024-09-17T02:42:45.869776+0000","last_peered":"2024-09-17T02:42:45.869776+0000","last_clean":"2024-09-17T02:42:45.869776+0000","last_became_active":"2024-09-17T02:42:43.830906+0000","last_became_peered":"2024-09-17T02:42:43.830906+0000","last_unstale":"2024-09-17T02:42:45.869776+0000","last_undegraded":"2024-09-17T02:42:45.869776+0000","last_fullsized":"2024-09-17T02:42:45.869776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:43.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866625+0000","last_change":"2024-09-17T02:42:44.878871+0000","last_active":"2024-09-17T02:42:45.866625+0000","last_peered":"2024-09-17T02:42:45.866625+0000","last_clean":"2024-09-17T02:42:45.866625+0000","last_became_active":"2024-09-17T02:42:44.878370+0000","last_became_peered":"2024-09-17T02:42:44.878370+0000","last_unstale":"2024-09-17T02:42:45.866625+0000","last_undegraded":"2024-09-17T02:42:45.866625+0000","last_fullsized":"2024-09-17T02:42:45.866625+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:44.634012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884758+0000","last_change":"2024-09-17T02:42:43.837536+0000","last_active":"2024-09-17T02:42:45.884758+0000","last_peered":"2024-09-17T02:42:45.884758+0000","last_clean":"2024-09-17T02:42:45.884758+0000","last_became_active":"2024-09-17T02:42:43.837182+0000","last_became_peered":"2024-09-17T02:42:43.837182+0000","last_unstale":"2024-09-17T02:42:45.884758+0000","last_undegraded":"2024-09-17T02:42:45.884758+0000","last_fullsized":"2024-09-17T02:42:45.884758+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:38.099906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870485+0000","last_change":"2024-09-17T02:42:44.870506+0000","last_active":"2024-09-17T02:42:45.870485+0000","last_peered":"2024-09-17T02:42:45.870485+0000","last_clean":"2024-09-17T02:42:45.870485+0000","last_became_active":"2024-09-17T02:42:44.870091+0000","last_became_peered":"2024-09-17T02:42:44.870091+0000","last_unstale":"2024-09-17T02:42:45.870485+0000","last_undegraded":"2024-09-17T02:42:45.870485+0000","last_fullsized":"2024-09-17T02:42:45.870485+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:14.136518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885151+0000","last_change":"2024-09-17T02:42:43.835536+0000","last_active":"2024-09-17T02:42:45.885151+0000","last_peered":"2024-09-17T02:42:45.885151+0000","last_clean":"2024-09-17T02:42:45.885151+0000","last_became_active":"2024-09-17T02:42:43.835141+0000","last_became_peered":"2024-09-17T02:42:43.835141+0000","last_unstale":"2024-09-17T02:42:45.885151+0000","last_undegraded":"2024-09-17T02:42:45.885151+0000","last_fullsized":"2024-09-17T02:42:45.885151+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:58.469842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884026+0000","last_change":"2024-09-17T02:42:44.869196+0000","last_active":"2024-09-17T02:42:45.884026+0000","last_peered":"2024-09-17T02:42:45.884026+0000","last_clean":"2024-09-17T02:42:45.884026+0000","last_became_active":"2024-09-17T02:42:44.868845+0000","last_became_peered":"2024-09-17T02:42:44.868845+0000","last_unstale":"2024-09-17T02:42:45.884026+0000","last_undegraded":"2024-09-17T02:42:45.884026+0000","last_fullsized":"2024-09-17T02:42:45.884026+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:44.069731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884247+0000","last_change":"2024-09-17T02:42:43.856625+0000","last_active":"2024-09-17T02:42:45.884247+0000","last_peered":"2024-09-17T02:42:45.884247+0000","last_clean":"2024-09-17T02:42:45.884247+0000","last_became_active":"2024-09-17T02:42:43.855851+0000","last_became_peered":"2024-09-17T02:42:43.855851+0000","last_unstale":"2024-09-17T02:42:45.884247+0000","last_undegraded":"2024-09-17T02:42:45.884247+0000","last_fullsized":"2024-09-17T02:42:45.884247+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:11.579876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.872710+0000","last_change":"2024-09-17T02:42:44.869981+0000","last_active":"2024-09-17T02:42:45.872710+0000","last_peered":"2024-09-17T02:42:45.872710+0000","last_clean":"2024-09-17T02:42:45.872710+0000","last_became_active":"2024-09-17T02:42:44.869502+0000","last_became_peered":"2024-09-17T02:42:44.869502+0000","last_unstale":"2024-09-17T02:42:45.872710+0000","last_undegraded":"2024-09-17T02:42:45.872710+0000","last_fullsized":"2024-09-17T02:42:45.872710+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:22.269723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870525+0000","last_change":"2024-09-17T02:42:43.847400+0000","last_active":"2024-09-17T02:42:45.870525+0000","last_peered":"2024-09-17T02:42:45.870525+0000","last_clean":"2024-09-17T02:42:45.870525+0000","last_became_active":"2024-09-17T02:42:43.843509+0000","last_became_peered":"2024-09-17T02:42:43.843509+0000","last_unstale":"2024-09-17T02:42:45.870525+0000","last_undegraded":"2024-09-17T02:42:45.870525+0000","last_fullsized":"2024-09-17T02:42:45.870525+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:21.738639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884317+0000","last_change":"2024-09-17T02:42:44.887685+0000","last_active":"2024-09-17T02:42:45.884317+0000","last_peered":"2024-09-17T02:42:45.884317+0000","last_clean":"2024-09-17T02:42:45.884317+0000","last_became_active":"2024-09-17T02:42:44.887127+0000","last_became_peered":"2024-09-17T02:42:44.887127+0000","last_unstale":"2024-09-17T02:42:45.884317+0000","last_undegraded":"2024-09-17T02:42:45.884317+0000","last_fullsized":"2024-09-17T02:42:45.884317+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:23.065669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883962+0000","last_change":"2024-09-17T02:42:43.859365+0000","last_active":"2024-09-17T02:42:45.883962+0000","last_peered":"2024-09-17T02:42:45.883962+0000","last_clean":"2024-09-17T02:42:45.883962+0000","last_became_active":"2024-09-17T02:42:43.856287+0000","last_became_peered":"2024-09-17T02:42:43.856287+0000","last_unstale":"2024-09-17T02:42:45.883962+0000","last_undegraded":"2024-09-17T02:42:45.883962+0000","last_fullsized":"2024-09-17T02:42:45.883962+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:18.332340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.874248+0000","last_change":"2024-09-17T02:42:44.880311+0000","last_active":"2024-09-17T02:42:45.874248+0000","last_peered":"2024-09-17T02:42:45.874248+0000","last_clean":"2024-09-17T02:42:45.874248+0000","last_became_active":"2024-09-17T02:42:44.879946+0000","last_became_peered":"2024-09-17T02:42:44.879946+0000","last_unstale":"2024-09-17T02:42:45.874248+0000","last_undegraded":"2024-09-17T02:42:45.874248+0000","last_fullsized":"2024-09-17T02:42:45.874248+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:32.989145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882301+0000","last_change":"2024-09-17T02:42:43.822424+0000","last_active":"2024-09-17T02:42:45.882301+0000","last_peered":"2024-09-17T02:42:45.882301+0000","last_clean":"2024-09-17T02:42:45.882301+0000","last_became_active":"2024-09-17T02:42:43.822051+0000","last_became_peered":"2024-09-17T02:42:43.822051+0000","last_unstale":"2024-09-17T02:42:45.882301+0000","last_undegraded":"2024-09-17T02:42:45.882301+0000","last_fullsized":"2024-09-17T02:42:45.882301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:25.389352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882609+0000","last_change":"2024-09-17T02:42:44.881443+0000","last_active":"2024-09-17T02:42:45.882609+0000","last_peered":"2024-09-17T02:42:45.882609+0000","last_clean":"2024-09-17T02:42:45.882609+0000","last_became_active":"2024-09-17T02:42:44.881026+0000","last_became_peered":"2024-09-17T02:42:44.881026+0000","last_unstale":"2024-09-17T02:42:45.882609+0000","last_undegraded":"2024-09-17T02:42:45.882609+0000","last_fullsized":"2024-09-17T02:42:45.882609+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866487+0000","last_change":"2024-09-17T02:42:43.845560+0000","last_active":"2024-09-17T02:42:45.866487+0000","last_peered":"2024-09-17T02:42:45.866487+0000","last_clean":"2024-09-17T02:42:45.866487+0000","last_became_active":"2024-09-17T02:42:43.845154+0000","last_became_peered":"2024-09-17T02:42:43.845154+0000","last_unstale":"2024-09-17T02:42:45.866487+0000","last_undegraded":"2024-09-17T02:42:45.866487+0000","last_fullsized":"2024-09-17T02:42:45.866487+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:33.597784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.887379+0000","last_change":"2024-09-17T02:42:44.881402+0000","last_active":"2024-09-17T02:42:45.887379+0000","last_peered":"2024-09-17T02:42:45.887379+0000","last_clean":"2024-09-17T02:42:45.887379+0000","last_became_active":"2024-09-17T02:42:44.880572+0000","last_became_peered":"2024-09-17T02:42:44.880572+0000","last_unstale":"2024-09-17T02:42:45.887379+0000","last_undegraded":"2024-09-17T02:42:45.887379+0000","last_fullsized":"2024-09-17T02:42:45.887379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:23.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.874020+0000","last_change":"2024-09-17T02:42:43.834185+0000","last_active":"2024-09-17T02:42:45.874020+0000","last_peered":"2024-09-17T02:42:45.874020+0000","last_clean":"2024-09-17T02:42:45.874020+0000","last_became_active":"2024-09-17T02:42:43.832748+0000","last_became_peered":"2024-09-17T02:42:43.832748+0000","last_unstale":"2024-09-17T02:42:45.874020+0000","last_undegraded":"2024-09-17T02:42:45.874020+0000","last_fullsized":"2024-09-17T02:42:45.874020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:19.775974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":168,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866609+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:42:45.866609+0000","last_peered":"2024-09-17T02:42:45.866609+0000","last_clean":"2024-09-17T02:42:45.866609+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:42:45.866609+0000","last_undegraded":"2024-09-17T02:42:45.866609+0000","last_fullsized":"2024-09-17T02:42:45.866609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":606208,"data_stored":262756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1208320,"data_stored":854770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":271096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":602112,"data_stored":262790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":614400,"data_stored":269426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1196032,"data_stored":853138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":602112,"data_stored":263282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":573440,"data_stored":248866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:42:49.910 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3f is not active+clean 2024-09-17T02:42:49.910 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:42:43.828438+0000', 'last_change': '2024-09-17T02:42:43.828438+0000', 'last_active': '2024-09-17T02:42:43.794469+0000', 'last_peered': '2024-09-17T02:42:43.794469+0000', 'last_clean': '2024-09-17T02:42:43.794469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:42:43.828438+0000', 'last_undegraded': '2024-09-17T02:42:43.828438+0000', 'last_fullsized': '2024-09-17T02:42:43.828438+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_clean_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:42:49.910 INFO:tasks.ceph.ceph_manager.ceph:PG 8.35 is not active+clean 2024-09-17T02:42:49.910 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:42:43.809880+0000', 'last_change': '2024-09-17T02:42:43.809880+0000', 'last_active': '2024-09-17T02:42:43.794469+0000', 'last_peered': '2024-09-17T02:42:43.794469+0000', 'last_clean': '2024-09-17T02:42:43.794469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:42:43.809880+0000', 'last_undegraded': '2024-09-17T02:42:43.809880+0000', 'last_fullsized': '2024-09-17T02:42:43.809880+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_clean_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:42:49.910 INFO:tasks.ceph.ceph_manager.ceph:PG 8.32 is not active+clean 2024-09-17T02:42:49.910 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:42:43.812574+0000', 'last_change': '2024-09-17T02:42:43.812574+0000', 'last_active': '2024-09-17T02:42:43.794469+0000', 'last_peered': '2024-09-17T02:42:43.794469+0000', 'last_clean': '2024-09-17T02:42:43.794469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:42:43.812574+0000', 'last_undegraded': '2024-09-17T02:42:43.812574+0000', 'last_fullsized': '2024-09-17T02:42:43.812574+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_clean_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:42:49.910 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2f is not active+clean 2024-09-17T02:42:49.910 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:42:43.813038+0000', 'last_change': '2024-09-17T02:42:43.813038+0000', 'last_active': '2024-09-17T02:42:43.794469+0000', 'last_peered': '2024-09-17T02:42:43.794469+0000', 'last_clean': '2024-09-17T02:42:43.794469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:42:43.813038+0000', 'last_undegraded': '2024-09-17T02:42:43.813038+0000', 'last_fullsized': '2024-09-17T02:42:43.813038+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_clean_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:42:49.911 INFO:tasks.ceph.ceph_manager.ceph:PG 8.27 is not active+clean 2024-09-17T02:42:49.911 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:42:43.826441+0000', 'last_change': '2024-09-17T02:42:43.826441+0000', 'last_active': '2024-09-17T02:42:43.794469+0000', 'last_peered': '2024-09-17T02:42:43.794469+0000', 'last_clean': '2024-09-17T02:42:43.794469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:42:43.826441+0000', 'last_undegraded': '2024-09-17T02:42:43.826441+0000', 'last_fullsized': '2024-09-17T02:42:43.826441+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_clean_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:42:49.911 INFO:tasks.ceph.ceph_manager.ceph:PG 8.23 is not active+clean 2024-09-17T02:42:49.911 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:42:43.809918+0000', 'last_change': '2024-09-17T02:42:43.809918+0000', 'last_active': '2024-09-17T02:42:43.794469+0000', 'last_peered': '2024-09-17T02:42:43.794469+0000', 'last_clean': '2024-09-17T02:42:43.794469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:42:43.809918+0000', 'last_undegraded': '2024-09-17T02:42:43.809918+0000', 'last_fullsized': '2024-09-17T02:42:43.809918+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_clean_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:42:49.911 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1a is not active+clean 2024-09-17T02:42:49.911 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:42:43.807944+0000', 'last_change': '2024-09-17T02:42:43.807944+0000', 'last_active': '2024-09-17T02:42:43.794469+0000', 'last_peered': '2024-09-17T02:42:43.794469+0000', 'last_clean': '2024-09-17T02:42:43.794469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:42:43.807944+0000', 'last_undegraded': '2024-09-17T02:42:43.807944+0000', 'last_fullsized': '2024-09-17T02:42:43.807944+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_clean_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:42:49.911 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1d is not active+clean 2024-09-17T02:42:49.911 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:42:43.808614+0000', 'last_change': '2024-09-17T02:42:43.808614+0000', 'last_active': '2024-09-17T02:42:43.794469+0000', 'last_peered': '2024-09-17T02:42:43.794469+0000', 'last_clean': '2024-09-17T02:42:43.794469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:42:43.808614+0000', 'last_undegraded': '2024-09-17T02:42:43.808614+0000', 'last_fullsized': '2024-09-17T02:42:43.808614+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_clean_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:42:49.911 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1 is not active+clean 2024-09-17T02:42:49.911 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:42:43.809542+0000', 'last_change': '2024-09-17T02:42:43.809542+0000', 'last_active': '2024-09-17T02:42:43.794469+0000', 'last_peered': '2024-09-17T02:42:43.794469+0000', 'last_clean': '2024-09-17T02:42:43.794469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:42:43.809542+0000', 'last_undegraded': '2024-09-17T02:42:43.809542+0000', 'last_fullsized': '2024-09-17T02:42:43.809542+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_clean_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:42:49.912 INFO:tasks.ceph.ceph_manager.ceph:PG 8.6 is not active+clean 2024-09-17T02:42:49.912 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 50, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:42:43.810796+0000', 'last_change': '2024-09-17T02:42:43.810796+0000', 'last_active': '2024-09-17T02:42:43.794469+0000', 'last_peered': '2024-09-17T02:42:43.794469+0000', 'last_clean': '2024-09-17T02:42:43.794469+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:42:43.810796+0000', 'last_undegraded': '2024-09-17T02:42:43.810796+0000', 'last_fullsized': '2024-09-17T02:42:43.810796+0000', 'mapping_epoch': 50, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 50, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'last_clean_scrub_stamp': '2024-09-17T02:42:43.794469+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:42:49.912 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T02:42:50.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 -- 172.21.15.131:0/1536010275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8003a10 msgr2=0x7fa8e8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:50.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 --2- 172.21.15.131:0/1536010275 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8003a10 0x7fa8e8003e80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0096f0 tx=0x7fa8ec01f490 comp rx=0 tx=0).stop 2024-09-17T02:42:50.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa8fb7fe640 1 -- 172.21.15.131:0/1536010275 reap_dead start 2024-09-17T02:42:50.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 -- 172.21.15.131:0/1536010275 shutdown_connections 2024-09-17T02:42:50.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 -- 172.21.15.131:0/1536010275 >> 172.21.15.131:0/1536010275 conn(0x7fa8fc074cb0 msgr2=0x7fa8fc06dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:50.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 -- 172.21.15.131:0/1536010275 shutdown_connections 2024-09-17T02:42:50.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 -- 172.21.15.131:0/1536010275 wait complete. 2024-09-17T02:42:50.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 Processor -- start 2024-09-17T02:42:50.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 -- start start 2024-09-17T02:42:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8003a10 0x7fa8fc15d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa8fc140a80 0x7fa8fc141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc1419b0 0x7fa8fc1f7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:50.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc0b4300 con 0x7fa8e8003a10 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc0b4180 con 0x7fa8fc1419b0 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa9013c3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa8fc0712b0 con 0x7fa8fc140a80 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa8faffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8003a10 0x7fa8fc15d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa8faffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8003a10 0x7fa8fc15d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58092/0 (socket says 172.21.15.131:58092) 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.067+0000 7fa8faffd640 1 -- 172.21.15.131:0/863668020 learned_addr learned my addr 172.21.15.131:0/863668020 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8fb7fe640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc1419b0 0x7fa8fc1f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8fa7fc640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa8fc140a80 0x7fa8fc141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8faffd640 1 -- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa8fc140a80 msgr2=0x7fa8fc141470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8faffd640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa8fc140a80 0x7fa8fc141470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8faffd640 1 -- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc1419b0 msgr2=0x7fa8fc1f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8faffd640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc1419b0 0x7fa8fc1f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8faffd640 1 -- 172.21.15.131:0/863668020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8fc15c240 con 0x7fa8e8003a10 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8faffd640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8003a10 0x7fa8fc15d2e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec01f2c0 tx=0x7fa8ec005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:50.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8f37fe640 1 -- 172.21.15.131:0/863668020 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec0b4040 con 0x7fa8e8003a10 2024-09-17T02:42:50.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8f37fe640 1 -- 172.21.15.131:0/863668020 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8ec0be070 con 0x7fa8e8003a10 2024-09-17T02:42:50.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa9013c3640 1 -- 172.21.15.131:0/863668020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8fc15c520 con 0x7fa8e8003a10 2024-09-17T02:42:50.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8f37fe640 1 -- 172.21.15.131:0/863668020 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec009730 con 0x7fa8e8003a10 2024-09-17T02:42:50.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa9013c3640 1 -- 172.21.15.131:0/863668020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8fc15cae0 con 0x7fa8e8003a10 2024-09-17T02:42:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa9013c3640 1 -- 172.21.15.131:0/863668020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8fc070d70 con 0x7fa8e8003a10 2024-09-17T02:42:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8f37fe640 1 -- 172.21.15.131:0/863668020 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa8ec004270 con 0x7fa8e8003a10 2024-09-17T02:42:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8f37fe640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8dc073200 0x7fa8dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8f37fe640 1 -- 172.21.15.131:0/863668020 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fa8ec1315f0 con 0x7fa8e8003a10 2024-09-17T02:42:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8fa7fc640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8dc073200 0x7fa8dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:50.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.071+0000 7fa8fa7fc640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8dc073200 0x7fa8dc075710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc15fbd0 tx=0x7fa8e4005e40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:50.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.079+0000 7fa8f37fe640 1 -- 172.21.15.131:0/863668020 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa8ec0191d0 con 0x7fa8e8003a10 2024-09-17T02:42:50.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.307+0000 7fa9013c3640 1 -- 172.21.15.131:0/863668020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fa8fc15d820 con 0x7fa8e8003a10 2024-09-17T02:42:50.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.307+0000 7fa8f37fe640 1 -- 172.21.15.131:0/863668020 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1560 (secure 0 0 0) 0x7fa8ec0fc400 con 0x7fa8e8003a10 2024-09-17T02:42:50.311 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:50.311 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":144,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1726540840,"num_in_osds":8,"osd_in_since":1726540830,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":87},{"state_name":"creating+peering","count":10}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592778,"bytes_used":244523008,"bytes_avail":772849590272,"bytes_total":773094113280,"inactive_pgs_ratio":0.10309278219938278,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":27,"btime":"2024-09-17T02:42:47:892534+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":5463}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-17T02:42:33.157208+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T02:42:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 -- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8dc073200 msgr2=0x7fa8dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8dc073200 0x7fa8dc075710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc15fbd0 tx=0x7fa8e4005e40 comp rx=0 tx=0).stop 2024-09-17T02:42:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 -- 172.21.15.131:0/863668020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8003a10 msgr2=0x7fa8fc15d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8003a10 0x7fa8fc15d2e0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec01f2c0 tx=0x7fa8ec005590 comp rx=0 tx=0).stop 2024-09-17T02:42:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 -- 172.21.15.131:0/863668020 shutdown_connections 2024-09-17T02:42:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8dc073200 0x7fa8dc075710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8fc1419b0 0x7fa8fc1f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa8fc140a80 0x7fa8fc141470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 --2- 172.21.15.131:0/863668020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8e8003a10 0x7fa8fc15d2e0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 -- 172.21.15.131:0/863668020 >> 172.21.15.131:0/863668020 conn(0x7fa8fc074cb0 msgr2=0x7fa8fc0cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:50.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 -- 172.21.15.131:0/863668020 shutdown_connections 2024-09-17T02:42:50.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.311+0000 7fa9013c3640 1 -- 172.21.15.131:0/863668020 wait complete. 2024-09-17T02:42:50.331 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T02:42:50.331 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:42:50.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.475+0000 7f9080f63640 1 -- 172.21.15.131:0/4086147639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9064003fe0 msgr2=0x7f9064004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:50.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.475+0000 7f9080f63640 1 --2- 172.21.15.131:0/4086147639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9064003fe0 0x7f9064004810 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f907000c310 tx=0x7f907001f720 comp rx=0 tx=0).stop 2024-09-17T02:42:50.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.475+0000 7f907ad9d640 1 -- 172.21.15.131:0/4086147639 reap_dead start 2024-09-17T02:42:50.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.475+0000 7f9080f63640 1 -- 172.21.15.131:0/4086147639 shutdown_connections 2024-09-17T02:42:50.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.475+0000 7f9080f63640 1 -- 172.21.15.131:0/4086147639 >> 172.21.15.131:0/4086147639 conn(0x7f907c0cd6a0 msgr2=0x7f907c0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:50.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.475+0000 7f9080f63640 1 -- 172.21.15.131:0/4086147639 shutdown_connections 2024-09-17T02:42:50.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.475+0000 7f9080f63640 1 -- 172.21.15.131:0/4086147639 wait complete. 2024-09-17T02:42:50.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.475+0000 7f9080f63640 1 Processor -- start 2024-09-17T02:42:50.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.475+0000 7f9080f63640 1 -- start start 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9080f63640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9064003fe0 0x7f907c14ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9080f63640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f907c0cb930 0x7f907c1f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9080f63640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f907c1f7310 0x7f907c0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9080f63640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f907c15ac10 con 0x7f907c0cb930 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9080f63640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f907c15aa90 con 0x7f9064003fe0 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9080f63640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f907c15ad90 con 0x7f907c1f7310 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9079d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f907c0cb930 0x7f907c1f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9079d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f907c0cb930 0x7f907c1f6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58106/0 (socket says 172.21.15.131:58106) 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9079d9b640 1 -- 172.21.15.131:0/2653872015 learned_addr learned my addr 172.21.15.131:0/2653872015 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f907a59c640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9064003fe0 0x7f907c14ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f907ad9d640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f907c1f7310 0x7f907c0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9079d9b640 1 -- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f907c1f7310 msgr2=0x7f907c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:50.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9079d9b640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f907c1f7310 0x7f907c0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9079d9b640 1 -- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9064003fe0 msgr2=0x7f907c14ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:50.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9079d9b640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9064003fe0 0x7f907c14ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9079d9b640 1 -- 172.21.15.131:0/2653872015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c14b240 con 0x7f907c0cb930 2024-09-17T02:42:50.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f907ad9d640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f907c1f7310 0x7f907c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:50.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9079d9b640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f907c0cb930 0x7f907c1f6dd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f90700b0040 tx=0x7f9070006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:50.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f906affd640 1 -- 172.21.15.131:0/2653872015 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90700bf070 con 0x7f907c0cb930 2024-09-17T02:42:50.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f906affd640 1 -- 172.21.15.131:0/2653872015 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90700036f0 con 0x7f907c0cb930 2024-09-17T02:42:50.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9080f63640 1 -- 172.21.15.131:0/2653872015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c0b49f0 con 0x7f907c0cb930 2024-09-17T02:42:50.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f906affd640 1 -- 172.21.15.131:0/2653872015 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90700b2070 con 0x7f907c0cb930 2024-09-17T02:42:50.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9080f63640 1 -- 172.21.15.131:0/2653872015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c0b4fb0 con 0x7f907c0cb930 2024-09-17T02:42:50.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f9080f63640 1 -- 172.21.15.131:0/2653872015 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907c06e2a0 con 0x7f907c0cb930 2024-09-17T02:42:50.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.479+0000 7f906affd640 1 -- 172.21.15.131:0/2653872015 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f90700c2070 con 0x7f907c0cb930 2024-09-17T02:42:50.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.483+0000 7f906affd640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f905c073200 0x7f905c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:50.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.483+0000 7f906affd640 1 -- 172.21.15.131:0/2653872015 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f90701319a0 con 0x7f907c0cb930 2024-09-17T02:42:50.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.483+0000 7f907a59c640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f905c073200 0x7f905c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:50.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.483+0000 7f907a59c640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f905c073200 0x7f905c075710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f906c009b30 tx=0x7f906c0091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:50.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.487+0000 7f906affd640 1 -- 172.21.15.131:0/2653872015 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f90700194d0 con 0x7f907c0cb930 2024-09-17T02:42:50.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.639+0000 7f9080f63640 1 -- 172.21.15.131:0/2653872015 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f907c15df20 con 0x7f905c073200 2024-09-17T02:42:50.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.651+0000 7f906affd640 1 -- 172.21.15.131:0/2653872015 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251128 (secure 0 0 0) 0x7f907c15df20 con 0x7f905c073200 2024-09-17T02:42:50.657 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:50.658 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:42:50.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 -- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f905c073200 msgr2=0x7f905c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:50.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f905c073200 0x7f905c075710 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f906c009b30 tx=0x7f906c0091e0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 -- 172.21.15.131:0/2653872015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f907c0cb930 msgr2=0x7f907c1f6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:50.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f907c0cb930 0x7f907c1f6dd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f90700b0040 tx=0x7f9070006040 comp rx=0 tx=0).stop 2024-09-17T02:42:50.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 -- 172.21.15.131:0/2653872015 shutdown_connections 2024-09-17T02:42:50.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f905c073200 0x7f905c075710 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f907c1f7310 0x7f907c0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f907c0cb930 0x7f907c1f6dd0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 --2- 172.21.15.131:0/2653872015 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9064003fe0 0x7f907c14ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:50.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 -- 172.21.15.131:0/2653872015 >> 172.21.15.131:0/2653872015 conn(0x7f907c0cd6a0 msgr2=0x7f907c071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:50.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 -- 172.21.15.131:0/2653872015 shutdown_connections 2024-09-17T02:42:50.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:50.655+0000 7f9080f63640 1 -- 172.21.15.131:0/2653872015 wait complete. 2024-09-17T02:42:50.677 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-17T02:42:49.162416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108,"ondisk_log_size":108,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":117,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238792,"kb_used_data":5892,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735928,"statfs":{"total":773094113280,"available":772849590272,"internally_reserved":0,"allocated":6033408,"data_stored":3286124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002204"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.828438+0000","last_change":"2024-09-17T02:42:43.828438+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.828438+0000","last_undegraded":"2024-09-17T02:42:43.828438+0000","last_fullsized":"2024-09-17T02:42:43.828438+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883357+0000","last_change":"2024-09-17T02:42:44.884814+0000","last_active":"2024-09-17T02:42:45.883357+0000","last_peered":"2024-09-17T02:42:45.883357+0000","last_clean":"2024-09-17T02:42:45.883357+0000","last_became_active":"2024-09-17T02:42:44.884432+0000","last_became_peered":"2024-09-17T02:42:44.884432+0000","last_unstale":"2024-09-17T02:42:45.883357+0000","last_undegraded":"2024-09-17T02:42:45.883357+0000","last_fullsized":"2024-09-17T02:42:45.883357+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:23.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884491+0000","last_change":"2024-09-17T02:42:44.854032+0000","last_active":"2024-09-17T02:42:45.884491+0000","last_peered":"2024-09-17T02:42:45.884491+0000","last_clean":"2024-09-17T02:42:45.884491+0000","last_became_active":"2024-09-17T02:42:44.853205+0000","last_became_peered":"2024-09-17T02:42:44.853205+0000","last_unstale":"2024-09-17T02:42:45.884491+0000","last_undegraded":"2024-09-17T02:42:45.884491+0000","last_fullsized":"2024-09-17T02:42:45.884491+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:18.997673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867302+0000","last_change":"2024-09-17T02:42:44.878169+0000","last_active":"2024-09-17T02:42:45.867302+0000","last_peered":"2024-09-17T02:42:45.867302+0000","last_clean":"2024-09-17T02:42:45.867302+0000","last_became_active":"2024-09-17T02:42:44.877402+0000","last_became_peered":"2024-09-17T02:42:44.877402+0000","last_unstale":"2024-09-17T02:42:45.867302+0000","last_undegraded":"2024-09-17T02:42:45.867302+0000","last_fullsized":"2024-09-17T02:42:45.867302+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:55.948606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882642+0000","last_change":"2024-09-17T02:42:44.872147+0000","last_active":"2024-09-17T02:42:45.882642+0000","last_peered":"2024-09-17T02:42:45.882642+0000","last_clean":"2024-09-17T02:42:45.882642+0000","last_became_active":"2024-09-17T02:42:44.871745+0000","last_became_peered":"2024-09-17T02:42:44.871745+0000","last_unstale":"2024-09-17T02:42:45.882642+0000","last_undegraded":"2024-09-17T02:42:45.882642+0000","last_fullsized":"2024-09-17T02:42:45.882642+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:29.767271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882210+0000","last_change":"2024-09-17T02:42:44.885865+0000","last_active":"2024-09-17T02:42:45.882210+0000","last_peered":"2024-09-17T02:42:45.882210+0000","last_clean":"2024-09-17T02:42:45.882210+0000","last_became_active":"2024-09-17T02:42:44.885512+0000","last_became_peered":"2024-09-17T02:42:44.885512+0000","last_unstale":"2024-09-17T02:42:45.882210+0000","last_undegraded":"2024-09-17T02:42:45.882210+0000","last_fullsized":"2024-09-17T02:42:45.882210+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:56.895371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870233+0000","last_change":"2024-09-17T02:42:44.886317+0000","last_active":"2024-09-17T02:42:45.870233+0000","last_peered":"2024-09-17T02:42:45.870233+0000","last_clean":"2024-09-17T02:42:45.870233+0000","last_became_active":"2024-09-17T02:42:44.885948+0000","last_became_peered":"2024-09-17T02:42:44.885948+0000","last_unstale":"2024-09-17T02:42:45.870233+0000","last_undegraded":"2024-09-17T02:42:45.870233+0000","last_fullsized":"2024-09-17T02:42:45.870233+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:44.318617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881521+0000","last_change":"2024-09-17T02:42:44.876332+0000","last_active":"2024-09-17T02:42:45.881521+0000","last_peered":"2024-09-17T02:42:45.881521+0000","last_clean":"2024-09-17T02:42:45.881521+0000","last_became_active":"2024-09-17T02:42:44.875594+0000","last_became_peered":"2024-09-17T02:42:44.875594+0000","last_unstale":"2024-09-17T02:42:45.881521+0000","last_undegraded":"2024-09-17T02:42:45.881521+0000","last_fullsized":"2024-09-17T02:42:45.881521+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:34.865339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884540+0000","last_change":"2024-09-17T02:42:44.854764+0000","last_active":"2024-09-17T02:42:45.884540+0000","last_peered":"2024-09-17T02:42:45.884540+0000","last_clean":"2024-09-17T02:42:45.884540+0000","last_became_active":"2024-09-17T02:42:44.853822+0000","last_became_peered":"2024-09-17T02:42:44.853822+0000","last_unstale":"2024-09-17T02:42:45.884540+0000","last_undegraded":"2024-09-17T02:42:45.884540+0000","last_fullsized":"2024-09-17T02:42:45.884540+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:18.705995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873645+0000","last_change":"2024-09-17T02:42:44.889606+0000","last_active":"2024-09-17T02:42:45.873645+0000","last_peered":"2024-09-17T02:42:45.873645+0000","last_clean":"2024-09-17T02:42:45.873645+0000","last_became_active":"2024-09-17T02:42:44.889274+0000","last_became_peered":"2024-09-17T02:42:44.889274+0000","last_unstale":"2024-09-17T02:42:45.873645+0000","last_undegraded":"2024-09-17T02:42:45.873645+0000","last_fullsized":"2024-09-17T02:42:45.873645+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:12.214981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809880+0000","last_change":"2024-09-17T02:42:43.809880+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809880+0000","last_undegraded":"2024-09-17T02:42:43.809880+0000","last_fullsized":"2024-09-17T02:42:43.809880+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882022+0000","last_change":"2024-09-17T02:42:44.880065+0000","last_active":"2024-09-17T02:42:45.882022+0000","last_peered":"2024-09-17T02:42:45.882022+0000","last_clean":"2024-09-17T02:42:45.882022+0000","last_became_active":"2024-09-17T02:42:44.876260+0000","last_became_peered":"2024-09-17T02:42:44.876260+0000","last_unstale":"2024-09-17T02:42:45.882022+0000","last_undegraded":"2024-09-17T02:42:45.882022+0000","last_fullsized":"2024-09-17T02:42:45.882022+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:34:02.925523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883058+0000","last_change":"2024-09-17T02:42:44.882550+0000","last_active":"2024-09-17T02:42:45.883058+0000","last_peered":"2024-09-17T02:42:45.883058+0000","last_clean":"2024-09-17T02:42:45.883058+0000","last_became_active":"2024-09-17T02:42:44.882123+0000","last_became_peered":"2024-09-17T02:42:44.882123+0000","last_unstale":"2024-09-17T02:42:45.883058+0000","last_undegraded":"2024-09-17T02:42:45.883058+0000","last_fullsized":"2024-09-17T02:42:45.883058+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:24.890536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.812574+0000","last_change":"2024-09-17T02:42:43.812574+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.812574+0000","last_undegraded":"2024-09-17T02:42:43.812574+0000","last_fullsized":"2024-09-17T02:42:43.812574+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866923+0000","last_change":"2024-09-17T02:42:44.877349+0000","last_active":"2024-09-17T02:42:45.866923+0000","last_peered":"2024-09-17T02:42:45.866923+0000","last_clean":"2024-09-17T02:42:45.866923+0000","last_became_active":"2024-09-17T02:42:44.876816+0000","last_became_peered":"2024-09-17T02:42:44.876816+0000","last_unstale":"2024-09-17T02:42:45.866923+0000","last_undegraded":"2024-09-17T02:42:45.866923+0000","last_fullsized":"2024-09-17T02:42:45.866923+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:22.741898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881823+0000","last_change":"2024-09-17T02:42:44.876906+0000","last_active":"2024-09-17T02:42:45.881823+0000","last_peered":"2024-09-17T02:42:45.881823+0000","last_clean":"2024-09-17T02:42:45.881823+0000","last_became_active":"2024-09-17T02:42:44.875966+0000","last_became_peered":"2024-09-17T02:42:44.875966+0000","last_unstale":"2024-09-17T02:42:45.881823+0000","last_undegraded":"2024-09-17T02:42:45.881823+0000","last_fullsized":"2024-09-17T02:42:45.881823+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:06.983520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.813038+0000","last_change":"2024-09-17T02:42:43.813038+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.813038+0000","last_undegraded":"2024-09-17T02:42:43.813038+0000","last_fullsized":"2024-09-17T02:42:43.813038+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867528+0000","last_change":"2024-09-17T02:42:44.888487+0000","last_active":"2024-09-17T02:42:45.867528+0000","last_peered":"2024-09-17T02:42:45.867528+0000","last_clean":"2024-09-17T02:42:45.867528+0000","last_became_active":"2024-09-17T02:42:44.887970+0000","last_became_peered":"2024-09-17T02:42:44.887970+0000","last_unstale":"2024-09-17T02:42:45.867528+0000","last_undegraded":"2024-09-17T02:42:45.867528+0000","last_fullsized":"2024-09-17T02:42:45.867528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:42.359754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884599+0000","last_change":"2024-09-17T02:42:44.881307+0000","last_active":"2024-09-17T02:42:45.884599+0000","last_peered":"2024-09-17T02:42:45.884599+0000","last_clean":"2024-09-17T02:42:45.884599+0000","last_became_active":"2024-09-17T02:42:44.880605+0000","last_became_peered":"2024-09-17T02:42:44.880605+0000","last_unstale":"2024-09-17T02:42:45.884599+0000","last_undegraded":"2024-09-17T02:42:45.884599+0000","last_fullsized":"2024-09-17T02:42:45.884599+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:49.563614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883282+0000","last_change":"2024-09-17T02:42:44.887466+0000","last_active":"2024-09-17T02:42:45.883282+0000","last_peered":"2024-09-17T02:42:45.883282+0000","last_clean":"2024-09-17T02:42:45.883282+0000","last_became_active":"2024-09-17T02:42:44.885044+0000","last_became_peered":"2024-09-17T02:42:44.885044+0000","last_unstale":"2024-09-17T02:42:45.883282+0000","last_undegraded":"2024-09-17T02:42:45.883282+0000","last_fullsized":"2024-09-17T02:42:45.883282+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.263210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867244+0000","last_change":"2024-09-17T02:42:44.876044+0000","last_active":"2024-09-17T02:42:45.867244+0000","last_peered":"2024-09-17T02:42:45.867244+0000","last_clean":"2024-09-17T02:42:45.867244+0000","last_became_active":"2024-09-17T02:42:44.873541+0000","last_became_peered":"2024-09-17T02:42:44.873541+0000","last_unstale":"2024-09-17T02:42:45.867244+0000","last_undegraded":"2024-09-17T02:42:45.867244+0000","last_fullsized":"2024-09-17T02:42:45.867244+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:56.895371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883856+0000","last_change":"2024-09-17T02:42:44.879337+0000","last_active":"2024-09-17T02:42:45.883856+0000","last_peered":"2024-09-17T02:42:45.883856+0000","last_clean":"2024-09-17T02:42:45.883856+0000","last_became_active":"2024-09-17T02:42:44.878982+0000","last_became_peered":"2024-09-17T02:42:44.878982+0000","last_unstale":"2024-09-17T02:42:45.883856+0000","last_undegraded":"2024-09-17T02:42:45.883856+0000","last_fullsized":"2024-09-17T02:42:45.883856+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:59.473102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882578+0000","last_change":"2024-09-17T02:42:44.856410+0000","last_active":"2024-09-17T02:42:45.882578+0000","last_peered":"2024-09-17T02:42:45.882578+0000","last_clean":"2024-09-17T02:42:45.882578+0000","last_became_active":"2024-09-17T02:42:44.854487+0000","last_became_peered":"2024-09-17T02:42:44.854487+0000","last_unstale":"2024-09-17T02:42:45.882578+0000","last_undegraded":"2024-09-17T02:42:45.882578+0000","last_fullsized":"2024-09-17T02:42:45.882578+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:07.107071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882985+0000","last_change":"2024-09-17T02:42:44.870968+0000","last_active":"2024-09-17T02:42:45.882985+0000","last_peered":"2024-09-17T02:42:45.882985+0000","last_clean":"2024-09-17T02:42:45.882985+0000","last_became_active":"2024-09-17T02:42:44.870254+0000","last_became_peered":"2024-09-17T02:42:44.870254+0000","last_unstale":"2024-09-17T02:42:45.882985+0000","last_undegraded":"2024-09-17T02:42:45.882985+0000","last_fullsized":"2024-09-17T02:42:45.882985+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:15.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.826441+0000","last_change":"2024-09-17T02:42:43.826441+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.826441+0000","last_undegraded":"2024-09-17T02:42:43.826441+0000","last_fullsized":"2024-09-17T02:42:43.826441+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884665+0000","last_change":"2024-09-17T02:42:44.875582+0000","last_active":"2024-09-17T02:42:45.884665+0000","last_peered":"2024-09-17T02:42:45.884665+0000","last_clean":"2024-09-17T02:42:45.884665+0000","last_became_active":"2024-09-17T02:42:44.875205+0000","last_became_peered":"2024-09-17T02:42:44.875205+0000","last_unstale":"2024-09-17T02:42:45.884665+0000","last_undegraded":"2024-09-17T02:42:45.884665+0000","last_fullsized":"2024-09-17T02:42:45.884665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:22:23.818837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881878+0000","last_change":"2024-09-17T02:42:44.885311+0000","last_active":"2024-09-17T02:42:45.881878+0000","last_peered":"2024-09-17T02:42:45.881878+0000","last_clean":"2024-09-17T02:42:45.881878+0000","last_became_active":"2024-09-17T02:42:44.884647+0000","last_became_peered":"2024-09-17T02:42:44.884647+0000","last_unstale":"2024-09-17T02:42:45.881878+0000","last_undegraded":"2024-09-17T02:42:45.881878+0000","last_fullsized":"2024-09-17T02:42:45.881878+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:20.906053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883665+0000","last_change":"2024-09-17T02:42:44.877183+0000","last_active":"2024-09-17T02:42:45.883665+0000","last_peered":"2024-09-17T02:42:45.883665+0000","last_clean":"2024-09-17T02:42:45.883665+0000","last_became_active":"2024-09-17T02:42:44.875043+0000","last_became_peered":"2024-09-17T02:42:44.875043+0000","last_unstale":"2024-09-17T02:42:45.883665+0000","last_undegraded":"2024-09-17T02:42:45.883665+0000","last_fullsized":"2024-09-17T02:42:45.883665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:53.445161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809918+0000","last_change":"2024-09-17T02:42:43.809918+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809918+0000","last_undegraded":"2024-09-17T02:42:43.809918+0000","last_fullsized":"2024-09-17T02:42:43.809918+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883971+0000","last_change":"2024-09-17T02:42:44.869378+0000","last_active":"2024-09-17T02:42:45.883971+0000","last_peered":"2024-09-17T02:42:45.883971+0000","last_clean":"2024-09-17T02:42:45.883971+0000","last_became_active":"2024-09-17T02:42:44.868147+0000","last_became_peered":"2024-09-17T02:42:44.868147+0000","last_unstale":"2024-09-17T02:42:45.883971+0000","last_undegraded":"2024-09-17T02:42:45.883971+0000","last_fullsized":"2024-09-17T02:42:45.883971+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:03.959077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882887+0000","last_change":"2024-09-17T02:42:44.879236+0000","last_active":"2024-09-17T02:42:45.882887+0000","last_peered":"2024-09-17T02:42:45.882887+0000","last_clean":"2024-09-17T02:42:45.882887+0000","last_became_active":"2024-09-17T02:42:44.878872+0000","last_became_peered":"2024-09-17T02:42:44.878872+0000","last_unstale":"2024-09-17T02:42:45.882887+0000","last_undegraded":"2024-09-17T02:42:45.882887+0000","last_fullsized":"2024-09-17T02:42:45.882887+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:39.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882661+0000","last_change":"2024-09-17T02:42:44.853789+0000","last_active":"2024-09-17T02:42:45.882661+0000","last_peered":"2024-09-17T02:42:45.882661+0000","last_clean":"2024-09-17T02:42:45.882661+0000","last_became_active":"2024-09-17T02:42:44.852870+0000","last_became_peered":"2024-09-17T02:42:44.852870+0000","last_unstale":"2024-09-17T02:42:45.882661+0000","last_undegraded":"2024-09-17T02:42:45.882661+0000","last_fullsized":"2024-09-17T02:42:45.882661+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.016019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882301+0000","last_change":"2024-09-17T02:42:44.884160+0000","last_active":"2024-09-17T02:42:45.882301+0000","last_peered":"2024-09-17T02:42:45.882301+0000","last_clean":"2024-09-17T02:42:45.882301+0000","last_became_active":"2024-09-17T02:42:44.883295+0000","last_became_peered":"2024-09-17T02:42:44.883295+0000","last_unstale":"2024-09-17T02:42:45.882301+0000","last_undegraded":"2024-09-17T02:42:45.882301+0000","last_fullsized":"2024-09-17T02:42:45.882301+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:13.871419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":19,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885185+0000","last_change":"2024-09-17T02:42:43.845735+0000","last_active":"2024-09-17T02:42:45.885185+0000","last_peered":"2024-09-17T02:42:45.885185+0000","last_clean":"2024-09-17T02:42:45.885185+0000","last_became_active":"2024-09-17T02:42:43.845485+0000","last_became_peered":"2024-09-17T02:42:43.845485+0000","last_unstale":"2024-09-17T02:42:45.885185+0000","last_undegraded":"2024-09-17T02:42:45.885185+0000","last_fullsized":"2024-09-17T02:42:45.885185+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:19.775974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881559+0000","last_change":"2024-09-17T02:42:44.875131+0000","last_active":"2024-09-17T02:42:45.881559+0000","last_peered":"2024-09-17T02:42:45.881559+0000","last_clean":"2024-09-17T02:42:45.881559+0000","last_became_active":"2024-09-17T02:42:44.874308+0000","last_became_peered":"2024-09-17T02:42:44.874308+0000","last_unstale":"2024-09-17T02:42:45.881559+0000","last_undegraded":"2024-09-17T02:42:45.881559+0000","last_fullsized":"2024-09-17T02:42:45.881559+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:05.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882119+0000","last_change":"2024-09-17T02:42:43.865164+0000","last_active":"2024-09-17T02:42:45.882119+0000","last_peered":"2024-09-17T02:42:45.882119+0000","last_clean":"2024-09-17T02:42:45.882119+0000","last_became_active":"2024-09-17T02:42:43.862678+0000","last_became_peered":"2024-09-17T02:42:43.862678+0000","last_unstale":"2024-09-17T02:42:45.882119+0000","last_undegraded":"2024-09-17T02:42:45.882119+0000","last_fullsized":"2024-09-17T02:42:45.882119+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:46.632441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883731+0000","last_change":"2024-09-17T02:42:44.870094+0000","last_active":"2024-09-17T02:42:45.883731+0000","last_peered":"2024-09-17T02:42:45.883731+0000","last_clean":"2024-09-17T02:42:45.883731+0000","last_became_active":"2024-09-17T02:42:44.869245+0000","last_became_peered":"2024-09-17T02:42:44.869245+0000","last_unstale":"2024-09-17T02:42:45.883731+0000","last_undegraded":"2024-09-17T02:42:45.883731+0000","last_fullsized":"2024-09-17T02:42:45.883731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:18.419470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873178+0000","last_change":"2024-09-17T02:42:43.837893+0000","last_active":"2024-09-17T02:42:45.873178+0000","last_peered":"2024-09-17T02:42:45.873178+0000","last_clean":"2024-09-17T02:42:45.873178+0000","last_became_active":"2024-09-17T02:42:43.833094+0000","last_became_peered":"2024-09-17T02:42:43.833094+0000","last_unstale":"2024-09-17T02:42:45.873178+0000","last_undegraded":"2024-09-17T02:42:45.873178+0000","last_fullsized":"2024-09-17T02:42:45.873178+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:27.547510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870591+0000","last_change":"2024-09-17T02:42:44.876821+0000","last_active":"2024-09-17T02:42:45.870591+0000","last_peered":"2024-09-17T02:42:45.870591+0000","last_clean":"2024-09-17T02:42:45.870591+0000","last_became_active":"2024-09-17T02:42:44.876427+0000","last_became_peered":"2024-09-17T02:42:44.876427+0000","last_unstale":"2024-09-17T02:42:45.870591+0000","last_undegraded":"2024-09-17T02:42:45.870591+0000","last_fullsized":"2024-09-17T02:42:45.870591+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:29.285476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884021+0000","last_change":"2024-09-17T02:42:43.859536+0000","last_active":"2024-09-17T02:42:45.884021+0000","last_peered":"2024-09-17T02:42:45.884021+0000","last_clean":"2024-09-17T02:42:45.884021+0000","last_became_active":"2024-09-17T02:42:43.859191+0000","last_became_peered":"2024-09-17T02:42:43.859191+0000","last_unstale":"2024-09-17T02:42:45.884021+0000","last_undegraded":"2024-09-17T02:42:45.884021+0000","last_fullsized":"2024-09-17T02:42:45.884021+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:34.772218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.886237+0000","last_change":"2024-09-17T02:42:44.881758+0000","last_active":"2024-09-17T02:42:45.886237+0000","last_peered":"2024-09-17T02:42:45.886237+0000","last_clean":"2024-09-17T02:42:45.886237+0000","last_became_active":"2024-09-17T02:42:44.880037+0000","last_became_peered":"2024-09-17T02:42:44.880037+0000","last_unstale":"2024-09-17T02:42:45.886237+0000","last_undegraded":"2024-09-17T02:42:45.886237+0000","last_fullsized":"2024-09-17T02:42:45.886237+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:49.224409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884002+0000","last_change":"2024-09-17T02:42:43.859603+0000","last_active":"2024-09-17T02:42:45.884002+0000","last_peered":"2024-09-17T02:42:45.884002+0000","last_clean":"2024-09-17T02:42:45.884002+0000","last_became_active":"2024-09-17T02:42:43.856481+0000","last_became_peered":"2024-09-17T02:42:43.856481+0000","last_unstale":"2024-09-17T02:42:45.884002+0000","last_undegraded":"2024-09-17T02:42:45.884002+0000","last_fullsized":"2024-09-17T02:42:45.884002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:31.985885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884898+0000","last_change":"2024-09-17T02:42:44.891687+0000","last_active":"2024-09-17T02:42:45.884898+0000","last_peered":"2024-09-17T02:42:45.884898+0000","last_clean":"2024-09-17T02:42:45.884898+0000","last_became_active":"2024-09-17T02:42:44.890077+0000","last_became_peered":"2024-09-17T02:42:44.890077+0000","last_unstale":"2024-09-17T02:42:45.884898+0000","last_undegraded":"2024-09-17T02:42:45.884898+0000","last_fullsized":"2024-09-17T02:42:45.884898+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:46:21.265517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884101+0000","last_change":"2024-09-17T02:42:43.822676+0000","last_active":"2024-09-17T02:42:45.884101+0000","last_peered":"2024-09-17T02:42:45.884101+0000","last_clean":"2024-09-17T02:42:45.884101+0000","last_became_active":"2024-09-17T02:42:43.822225+0000","last_became_peered":"2024-09-17T02:42:43.822225+0000","last_unstale":"2024-09-17T02:42:45.884101+0000","last_undegraded":"2024-09-17T02:42:45.884101+0000","last_fullsized":"2024-09-17T02:42:45.884101+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:38.838275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873695+0000","last_change":"2024-09-17T02:42:44.880963+0000","last_active":"2024-09-17T02:42:45.873695+0000","last_peered":"2024-09-17T02:42:45.873695+0000","last_clean":"2024-09-17T02:42:45.873695+0000","last_became_active":"2024-09-17T02:42:44.880257+0000","last_became_peered":"2024-09-17T02:42:44.880257+0000","last_unstale":"2024-09-17T02:42:45.873695+0000","last_undegraded":"2024-09-17T02:42:45.873695+0000","last_fullsized":"2024-09-17T02:42:45.873695+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:09.416640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885098+0000","last_change":"2024-09-17T02:42:43.814090+0000","last_active":"2024-09-17T02:42:45.885098+0000","last_peered":"2024-09-17T02:42:45.885098+0000","last_clean":"2024-09-17T02:42:45.885098+0000","last_became_active":"2024-09-17T02:42:43.813685+0000","last_became_peered":"2024-09-17T02:42:43.813685+0000","last_unstale":"2024-09-17T02:42:45.885098+0000","last_undegraded":"2024-09-17T02:42:45.885098+0000","last_fullsized":"2024-09-17T02:42:45.885098+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:06.103811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867600+0000","last_change":"2024-09-17T02:42:44.878439+0000","last_active":"2024-09-17T02:42:45.867600+0000","last_peered":"2024-09-17T02:42:45.867600+0000","last_clean":"2024-09-17T02:42:45.867600+0000","last_became_active":"2024-09-17T02:42:44.875651+0000","last_became_peered":"2024-09-17T02:42:44.875651+0000","last_unstale":"2024-09-17T02:42:45.867600+0000","last_undegraded":"2024-09-17T02:42:45.867600+0000","last_fullsized":"2024-09-17T02:42:45.867600+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:29.767271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883310+0000","last_change":"2024-09-17T02:42:43.855843+0000","last_active":"2024-09-17T02:42:45.883310+0000","last_peered":"2024-09-17T02:42:45.883310+0000","last_clean":"2024-09-17T02:42:45.883310+0000","last_became_active":"2024-09-17T02:42:43.854712+0000","last_became_peered":"2024-09-17T02:42:43.854712+0000","last_unstale":"2024-09-17T02:42:45.883310+0000","last_undegraded":"2024-09-17T02:42:45.883310+0000","last_fullsized":"2024-09-17T02:42:45.883310+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:19:02.429366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884260+0000","last_change":"2024-09-17T02:42:44.875385+0000","last_active":"2024-09-17T02:42:45.884260+0000","last_peered":"2024-09-17T02:42:45.884260+0000","last_clean":"2024-09-17T02:42:45.884260+0000","last_became_active":"2024-09-17T02:42:44.874816+0000","last_became_peered":"2024-09-17T02:42:44.874816+0000","last_unstale":"2024-09-17T02:42:45.884260+0000","last_undegraded":"2024-09-17T02:42:45.884260+0000","last_fullsized":"2024-09-17T02:42:45.884260+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:36.940445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867765+0000","last_change":"2024-09-17T02:42:43.826860+0000","last_active":"2024-09-17T02:42:45.867765+0000","last_peered":"2024-09-17T02:42:45.867765+0000","last_clean":"2024-09-17T02:42:45.867765+0000","last_became_active":"2024-09-17T02:42:43.826423+0000","last_became_peered":"2024-09-17T02:42:43.826423+0000","last_unstale":"2024-09-17T02:42:45.867765+0000","last_undegraded":"2024-09-17T02:42:45.867765+0000","last_fullsized":"2024-09-17T02:42:45.867765+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:57.315329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866970+0000","last_change":"2024-09-17T02:42:44.889033+0000","last_active":"2024-09-17T02:42:45.866970+0000","last_peered":"2024-09-17T02:42:45.866970+0000","last_clean":"2024-09-17T02:42:45.866970+0000","last_became_active":"2024-09-17T02:42:44.888575+0000","last_became_peered":"2024-09-17T02:42:44.888575+0000","last_unstale":"2024-09-17T02:42:45.866970+0000","last_undegraded":"2024-09-17T02:42:45.866970+0000","last_fullsized":"2024-09-17T02:42:45.866970+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:10.824751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867254+0000","last_change":"2024-09-17T02:42:43.835429+0000","last_active":"2024-09-17T02:42:45.867254+0000","last_peered":"2024-09-17T02:42:45.867254+0000","last_clean":"2024-09-17T02:42:45.867254+0000","last_became_active":"2024-09-17T02:42:43.835072+0000","last_became_peered":"2024-09-17T02:42:43.835072+0000","last_unstale":"2024-09-17T02:42:45.867254+0000","last_undegraded":"2024-09-17T02:42:45.867254+0000","last_fullsized":"2024-09-17T02:42:45.867254+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:28.861850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.807944+0000","last_change":"2024-09-17T02:42:43.807944+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.807944+0000","last_undegraded":"2024-09-17T02:42:43.807944+0000","last_fullsized":"2024-09-17T02:42:43.807944+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885035+0000","last_change":"2024-09-17T02:42:43.844104+0000","last_active":"2024-09-17T02:42:45.885035+0000","last_peered":"2024-09-17T02:42:45.885035+0000","last_clean":"2024-09-17T02:42:45.885035+0000","last_became_active":"2024-09-17T02:42:43.843661+0000","last_became_peered":"2024-09-17T02:42:43.843661+0000","last_unstale":"2024-09-17T02:42:45.885035+0000","last_undegraded":"2024-09-17T02:42:45.885035+0000","last_fullsized":"2024-09-17T02:42:45.885035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:14.987141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882393+0000","last_change":"2024-09-17T02:42:44.853449+0000","last_active":"2024-09-17T02:42:45.882393+0000","last_peered":"2024-09-17T02:42:45.882393+0000","last_clean":"2024-09-17T02:42:45.882393+0000","last_became_active":"2024-09-17T02:42:44.852561+0000","last_became_peered":"2024-09-17T02:42:44.852561+0000","last_unstale":"2024-09-17T02:42:45.882393+0000","last_undegraded":"2024-09-17T02:42:45.882393+0000","last_fullsized":"2024-09-17T02:42:45.882393+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.871956+0000","last_change":"2024-09-17T02:42:43.864191+0000","last_active":"2024-09-17T02:42:45.871956+0000","last_peered":"2024-09-17T02:42:45.871956+0000","last_clean":"2024-09-17T02:42:45.871956+0000","last_became_active":"2024-09-17T02:42:43.863400+0000","last_became_peered":"2024-09-17T02:42:43.863400+0000","last_unstale":"2024-09-17T02:42:45.871956+0000","last_undegraded":"2024-09-17T02:42:45.871956+0000","last_fullsized":"2024-09-17T02:42:45.871956+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:09.821491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885484+0000","last_change":"2024-09-17T02:42:44.880368+0000","last_active":"2024-09-17T02:42:45.885484+0000","last_peered":"2024-09-17T02:42:45.885484+0000","last_clean":"2024-09-17T02:42:45.885484+0000","last_became_active":"2024-09-17T02:42:44.879881+0000","last_became_peered":"2024-09-17T02:42:44.879881+0000","last_unstale":"2024-09-17T02:42:45.885484+0000","last_undegraded":"2024-09-17T02:42:45.885484+0000","last_fullsized":"2024-09-17T02:42:45.885484+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:58.252210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883189+0000","last_change":"2024-09-17T02:42:43.841976+0000","last_active":"2024-09-17T02:42:45.883189+0000","last_peered":"2024-09-17T02:42:45.883189+0000","last_clean":"2024-09-17T02:42:45.883189+0000","last_became_active":"2024-09-17T02:42:43.841094+0000","last_became_peered":"2024-09-17T02:42:43.841094+0000","last_unstale":"2024-09-17T02:42:45.883189+0000","last_undegraded":"2024-09-17T02:42:45.883189+0000","last_fullsized":"2024-09-17T02:42:45.883189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:29.331958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.808614+0000","last_change":"2024-09-17T02:42:43.808614+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.808614+0000","last_undegraded":"2024-09-17T02:42:43.808614+0000","last_fullsized":"2024-09-17T02:42:43.808614+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866984+0000","last_change":"2024-09-17T02:42:43.843349+0000","last_active":"2024-09-17T02:42:45.866984+0000","last_peered":"2024-09-17T02:42:45.866984+0000","last_clean":"2024-09-17T02:42:45.866984+0000","last_became_active":"2024-09-17T02:42:43.842950+0000","last_became_peered":"2024-09-17T02:42:43.842950+0000","last_unstale":"2024-09-17T02:42:45.866984+0000","last_undegraded":"2024-09-17T02:42:45.866984+0000","last_fullsized":"2024-09-17T02:42:45.866984+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:22.974854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882439+0000","last_change":"2024-09-17T02:42:44.886488+0000","last_active":"2024-09-17T02:42:45.882439+0000","last_peered":"2024-09-17T02:42:45.882439+0000","last_clean":"2024-09-17T02:42:45.882439+0000","last_became_active":"2024-09-17T02:42:44.886106+0000","last_became_peered":"2024-09-17T02:42:44.886106+0000","last_unstale":"2024-09-17T02:42:45.882439+0000","last_undegraded":"2024-09-17T02:42:45.882439+0000","last_fullsized":"2024-09-17T02:42:45.882439+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:55.948606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881989+0000","last_change":"2024-09-17T02:42:43.825883+0000","last_active":"2024-09-17T02:42:45.881989+0000","last_peered":"2024-09-17T02:42:45.881989+0000","last_clean":"2024-09-17T02:42:45.881989+0000","last_became_active":"2024-09-17T02:42:43.825522+0000","last_became_peered":"2024-09-17T02:42:43.825522+0000","last_unstale":"2024-09-17T02:42:45.881989+0000","last_undegraded":"2024-09-17T02:42:45.881989+0000","last_fullsized":"2024-09-17T02:42:45.881989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:22:57.287148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885037+0000","last_change":"2024-09-17T02:42:44.878272+0000","last_active":"2024-09-17T02:42:45.885037+0000","last_peered":"2024-09-17T02:42:45.885037+0000","last_clean":"2024-09-17T02:42:45.885037+0000","last_became_active":"2024-09-17T02:42:44.877921+0000","last_became_peered":"2024-09-17T02:42:44.877921+0000","last_unstale":"2024-09-17T02:42:45.885037+0000","last_undegraded":"2024-09-17T02:42:45.885037+0000","last_fullsized":"2024-09-17T02:42:45.885037+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:14.742444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":23,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881967+0000","last_change":"2024-09-17T02:42:43.864045+0000","last_active":"2024-09-17T02:42:45.881967+0000","last_peered":"2024-09-17T02:42:45.881967+0000","last_clean":"2024-09-17T02:42:45.881967+0000","last_became_active":"2024-09-17T02:42:43.862689+0000","last_became_peered":"2024-09-17T02:42:43.862689+0000","last_unstale":"2024-09-17T02:42:45.881967+0000","last_undegraded":"2024-09-17T02:42:45.881967+0000","last_fullsized":"2024-09-17T02:42:45.881967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:30.667789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870056+0000","last_change":"2024-09-17T02:42:44.871420+0000","last_active":"2024-09-17T02:42:45.870056+0000","last_peered":"2024-09-17T02:42:45.870056+0000","last_clean":"2024-09-17T02:42:45.870056+0000","last_became_active":"2024-09-17T02:42:44.871075+0000","last_became_peered":"2024-09-17T02:42:44.871075+0000","last_unstale":"2024-09-17T02:42:45.870056+0000","last_undegraded":"2024-09-17T02:42:45.870056+0000","last_fullsized":"2024-09-17T02:42:45.870056+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:21.340145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882977+0000","last_change":"2024-09-17T02:42:43.865760+0000","last_active":"2024-09-17T02:42:45.882977+0000","last_peered":"2024-09-17T02:42:45.882977+0000","last_clean":"2024-09-17T02:42:45.882977+0000","last_became_active":"2024-09-17T02:42:43.862428+0000","last_became_peered":"2024-09-17T02:42:43.862428+0000","last_unstale":"2024-09-17T02:42:45.882977+0000","last_undegraded":"2024-09-17T02:42:45.882977+0000","last_fullsized":"2024-09-17T02:42:45.882977+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:00.644920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.809542+0000","last_change":"2024-09-17T02:42:43.809542+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.809542+0000","last_undegraded":"2024-09-17T02:42:43.809542+0000","last_fullsized":"2024-09-17T02:42:43.809542+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882280+0000","last_change":"2024-09-17T02:42:43.839506+0000","last_active":"2024-09-17T02:42:45.882280+0000","last_peered":"2024-09-17T02:42:45.882280+0000","last_clean":"2024-09-17T02:42:45.882280+0000","last_became_active":"2024-09-17T02:42:43.838617+0000","last_became_peered":"2024-09-17T02:42:43.838617+0000","last_unstale":"2024-09-17T02:42:45.882280+0000","last_undegraded":"2024-09-17T02:42:45.882280+0000","last_fullsized":"2024-09-17T02:42:45.882280+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:04.244931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883433+0000","last_change":"2024-09-17T02:42:44.876771+0000","last_active":"2024-09-17T02:42:45.883433+0000","last_peered":"2024-09-17T02:42:45.883433+0000","last_clean":"2024-09-17T02:42:45.883433+0000","last_became_active":"2024-09-17T02:42:44.876438+0000","last_became_peered":"2024-09-17T02:42:44.876438+0000","last_unstale":"2024-09-17T02:42:45.883433+0000","last_undegraded":"2024-09-17T02:42:45.883433+0000","last_fullsized":"2024-09-17T02:42:45.883433+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:28.921453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867493+0000","last_change":"2024-09-17T02:42:43.862172+0000","last_active":"2024-09-17T02:42:45.867493+0000","last_peered":"2024-09-17T02:42:45.867493+0000","last_clean":"2024-09-17T02:42:45.867493+0000","last_became_active":"2024-09-17T02:42:43.861853+0000","last_became_peered":"2024-09-17T02:42:43.861853+0000","last_unstale":"2024-09-17T02:42:45.867493+0000","last_undegraded":"2024-09-17T02:42:45.867493+0000","last_fullsized":"2024-09-17T02:42:45.867493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:00.644920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883740+0000","last_change":"2024-09-17T02:42:44.875816+0000","last_active":"2024-09-17T02:42:45.883740+0000","last_peered":"2024-09-17T02:42:45.883740+0000","last_clean":"2024-09-17T02:42:45.883740+0000","last_became_active":"2024-09-17T02:42:44.875443+0000","last_became_peered":"2024-09-17T02:42:44.875443+0000","last_unstale":"2024-09-17T02:42:45.883740+0000","last_undegraded":"2024-09-17T02:42:45.883740+0000","last_fullsized":"2024-09-17T02:42:45.883740+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.263210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884349+0000","last_change":"2024-09-17T02:42:43.842888+0000","last_active":"2024-09-17T02:42:45.884349+0000","last_peered":"2024-09-17T02:42:45.884349+0000","last_clean":"2024-09-17T02:42:45.884349+0000","last_became_active":"2024-09-17T02:42:43.842563+0000","last_became_peered":"2024-09-17T02:42:43.842563+0000","last_unstale":"2024-09-17T02:42:45.884349+0000","last_undegraded":"2024-09-17T02:42:45.884349+0000","last_fullsized":"2024-09-17T02:42:45.884349+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:37.379782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883984+0000","last_change":"2024-09-17T02:42:44.875239+0000","last_active":"2024-09-17T02:42:45.883984+0000","last_peered":"2024-09-17T02:42:45.883984+0000","last_clean":"2024-09-17T02:42:45.883984+0000","last_became_active":"2024-09-17T02:42:44.874268+0000","last_became_peered":"2024-09-17T02:42:44.874268+0000","last_unstale":"2024-09-17T02:42:45.883984+0000","last_undegraded":"2024-09-17T02:42:45.883984+0000","last_fullsized":"2024-09-17T02:42:45.883984+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:30.019802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T02:42:43.849754+0000","last_change":"2024-09-17T02:42:43.840810+0000","last_active":"2024-09-17T02:42:43.849754+0000","last_peered":"2024-09-17T02:42:43.849754+0000","last_clean":"2024-09-17T02:42:43.849754+0000","last_became_active":"2024-09-17T02:42:43.839861+0000","last_became_peered":"2024-09-17T02:42:43.839861+0000","last_unstale":"2024-09-17T02:42:43.849754+0000","last_undegraded":"2024-09-17T02:42:43.849754+0000","last_fullsized":"2024-09-17T02:42:43.849754+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:48:14.684387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867465+0000","last_change":"2024-09-17T02:42:44.877843+0000","last_active":"2024-09-17T02:42:45.867465+0000","last_peered":"2024-09-17T02:42:45.867465+0000","last_clean":"2024-09-17T02:42:45.867465+0000","last_became_active":"2024-09-17T02:42:44.877029+0000","last_became_peered":"2024-09-17T02:42:44.877029+0000","last_unstale":"2024-09-17T02:42:45.867465+0000","last_undegraded":"2024-09-17T02:42:45.867465+0000","last_fullsized":"2024-09-17T02:42:45.867465+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883764+0000","last_change":"2024-09-17T02:42:43.860370+0000","last_active":"2024-09-17T02:42:45.883764+0000","last_peered":"2024-09-17T02:42:45.883764+0000","last_clean":"2024-09-17T02:42:45.883764+0000","last_became_active":"2024-09-17T02:42:43.859094+0000","last_became_peered":"2024-09-17T02:42:43.859094+0000","last_unstale":"2024-09-17T02:42:45.883764+0000","last_undegraded":"2024-09-17T02:42:45.883764+0000","last_fullsized":"2024-09-17T02:42:45.883764+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:15.987844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":50,"state":"creating+peering","last_fresh":"2024-09-17T02:42:43.810796+0000","last_change":"2024-09-17T02:42:43.810796+0000","last_active":"2024-09-17T02:42:43.794469+0000","last_peered":"2024-09-17T02:42:43.794469+0000","last_clean":"2024-09-17T02:42:43.794469+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:42:43.810796+0000","last_undegraded":"2024-09-17T02:42:43.810796+0000","last_fullsized":"2024-09-17T02:42:43.810796+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870452+0000","last_change":"2024-09-17T02:42:43.862720+0000","last_active":"2024-09-17T02:42:45.870452+0000","last_peered":"2024-09-17T02:42:45.870452+0000","last_clean":"2024-09-17T02:42:45.870452+0000","last_became_active":"2024-09-17T02:42:43.862004+0000","last_became_peered":"2024-09-17T02:42:43.862004+0000","last_unstale":"2024-09-17T02:42:45.870452+0000","last_undegraded":"2024-09-17T02:42:45.870452+0000","last_fullsized":"2024-09-17T02:42:45.870452+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:41.356494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882795+0000","last_change":"2024-09-17T02:42:44.883533+0000","last_active":"2024-09-17T02:42:45.882795+0000","last_peered":"2024-09-17T02:42:45.882795+0000","last_clean":"2024-09-17T02:42:45.882795+0000","last_became_active":"2024-09-17T02:42:44.883166+0000","last_became_peered":"2024-09-17T02:42:44.883166+0000","last_unstale":"2024-09-17T02:42:45.882795+0000","last_undegraded":"2024-09-17T02:42:45.882795+0000","last_fullsized":"2024-09-17T02:42:45.882795+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.016019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.869776+0000","last_change":"2024-09-17T02:42:43.831772+0000","last_active":"2024-09-17T02:42:45.869776+0000","last_peered":"2024-09-17T02:42:45.869776+0000","last_clean":"2024-09-17T02:42:45.869776+0000","last_became_active":"2024-09-17T02:42:43.830906+0000","last_became_peered":"2024-09-17T02:42:43.830906+0000","last_unstale":"2024-09-17T02:42:45.869776+0000","last_undegraded":"2024-09-17T02:42:45.869776+0000","last_fullsized":"2024-09-17T02:42:45.869776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:43.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866625+0000","last_change":"2024-09-17T02:42:44.878871+0000","last_active":"2024-09-17T02:42:45.866625+0000","last_peered":"2024-09-17T02:42:45.866625+0000","last_clean":"2024-09-17T02:42:45.866625+0000","last_became_active":"2024-09-17T02:42:44.878370+0000","last_became_peered":"2024-09-17T02:42:44.878370+0000","last_unstale":"2024-09-17T02:42:45.866625+0000","last_undegraded":"2024-09-17T02:42:45.866625+0000","last_fullsized":"2024-09-17T02:42:45.866625+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:44.634012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884758+0000","last_change":"2024-09-17T02:42:43.837536+0000","last_active":"2024-09-17T02:42:45.884758+0000","last_peered":"2024-09-17T02:42:45.884758+0000","last_clean":"2024-09-17T02:42:45.884758+0000","last_became_active":"2024-09-17T02:42:43.837182+0000","last_became_peered":"2024-09-17T02:42:43.837182+0000","last_unstale":"2024-09-17T02:42:45.884758+0000","last_undegraded":"2024-09-17T02:42:45.884758+0000","last_fullsized":"2024-09-17T02:42:45.884758+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:38.099906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870485+0000","last_change":"2024-09-17T02:42:44.870506+0000","last_active":"2024-09-17T02:42:45.870485+0000","last_peered":"2024-09-17T02:42:45.870485+0000","last_clean":"2024-09-17T02:42:45.870485+0000","last_became_active":"2024-09-17T02:42:44.870091+0000","last_became_peered":"2024-09-17T02:42:44.870091+0000","last_unstale":"2024-09-17T02:42:45.870485+0000","last_undegraded":"2024-09-17T02:42:45.870485+0000","last_fullsized":"2024-09-17T02:42:45.870485+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:14.136518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885151+0000","last_change":"2024-09-17T02:42:43.835536+0000","last_active":"2024-09-17T02:42:45.885151+0000","last_peered":"2024-09-17T02:42:45.885151+0000","last_clean":"2024-09-17T02:42:45.885151+0000","last_became_active":"2024-09-17T02:42:43.835141+0000","last_became_peered":"2024-09-17T02:42:43.835141+0000","last_unstale":"2024-09-17T02:42:45.885151+0000","last_undegraded":"2024-09-17T02:42:45.885151+0000","last_fullsized":"2024-09-17T02:42:45.885151+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:58.469842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884026+0000","last_change":"2024-09-17T02:42:44.869196+0000","last_active":"2024-09-17T02:42:45.884026+0000","last_peered":"2024-09-17T02:42:45.884026+0000","last_clean":"2024-09-17T02:42:45.884026+0000","last_became_active":"2024-09-17T02:42:44.868845+0000","last_became_peered":"2024-09-17T02:42:44.868845+0000","last_unstale":"2024-09-17T02:42:45.884026+0000","last_undegraded":"2024-09-17T02:42:45.884026+0000","last_fullsized":"2024-09-17T02:42:45.884026+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:44.069731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884247+0000","last_change":"2024-09-17T02:42:43.856625+0000","last_active":"2024-09-17T02:42:45.884247+0000","last_peered":"2024-09-17T02:42:45.884247+0000","last_clean":"2024-09-17T02:42:45.884247+0000","last_became_active":"2024-09-17T02:42:43.855851+0000","last_became_peered":"2024-09-17T02:42:43.855851+0000","last_unstale":"2024-09-17T02:42:45.884247+0000","last_undegraded":"2024-09-17T02:42:45.884247+0000","last_fullsized":"2024-09-17T02:42:45.884247+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:11.579876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.872710+0000","last_change":"2024-09-17T02:42:44.869981+0000","last_active":"2024-09-17T02:42:45.872710+0000","last_peered":"2024-09-17T02:42:45.872710+0000","last_clean":"2024-09-17T02:42:45.872710+0000","last_became_active":"2024-09-17T02:42:44.869502+0000","last_became_peered":"2024-09-17T02:42:44.869502+0000","last_unstale":"2024-09-17T02:42:45.872710+0000","last_undegraded":"2024-09-17T02:42:45.872710+0000","last_fullsized":"2024-09-17T02:42:45.872710+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:22.269723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870525+0000","last_change":"2024-09-17T02:42:43.847400+0000","last_active":"2024-09-17T02:42:45.870525+0000","last_peered":"2024-09-17T02:42:45.870525+0000","last_clean":"2024-09-17T02:42:45.870525+0000","last_became_active":"2024-09-17T02:42:43.843509+0000","last_became_peered":"2024-09-17T02:42:43.843509+0000","last_unstale":"2024-09-17T02:42:45.870525+0000","last_undegraded":"2024-09-17T02:42:45.870525+0000","last_fullsized":"2024-09-17T02:42:45.870525+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:21.738639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884317+0000","last_change":"2024-09-17T02:42:44.887685+0000","last_active":"2024-09-17T02:42:45.884317+0000","last_peered":"2024-09-17T02:42:45.884317+0000","last_clean":"2024-09-17T02:42:45.884317+0000","last_became_active":"2024-09-17T02:42:44.887127+0000","last_became_peered":"2024-09-17T02:42:44.887127+0000","last_unstale":"2024-09-17T02:42:45.884317+0000","last_undegraded":"2024-09-17T02:42:45.884317+0000","last_fullsized":"2024-09-17T02:42:45.884317+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:23.065669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883962+0000","last_change":"2024-09-17T02:42:43.859365+0000","last_active":"2024-09-17T02:42:45.883962+0000","last_peered":"2024-09-17T02:42:45.883962+0000","last_clean":"2024-09-17T02:42:45.883962+0000","last_became_active":"2024-09-17T02:42:43.856287+0000","last_became_peered":"2024-09-17T02:42:43.856287+0000","last_unstale":"2024-09-17T02:42:45.883962+0000","last_undegraded":"2024-09-17T02:42:45.883962+0000","last_fullsized":"2024-09-17T02:42:45.883962+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:18.332340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.874248+0000","last_change":"2024-09-17T02:42:44.880311+0000","last_active":"2024-09-17T02:42:45.874248+0000","last_peered":"2024-09-17T02:42:45.874248+0000","last_clean":"2024-09-17T02:42:45.874248+0000","last_became_active":"2024-09-17T02:42:44.879946+0000","last_became_peered":"2024-09-17T02:42:44.879946+0000","last_unstale":"2024-09-17T02:42:45.874248+0000","last_undegraded":"2024-09-17T02:42:45.874248+0000","last_fullsized":"2024-09-17T02:42:45.874248+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:32.989145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882301+0000","last_change":"2024-09-17T02:42:43.822424+0000","last_active":"2024-09-17T02:42:45.882301+0000","last_peered":"2024-09-17T02:42:45.882301+0000","last_clean":"2024-09-17T02:42:45.882301+0000","last_became_active":"2024-09-17T02:42:43.822051+0000","last_became_peered":"2024-09-17T02:42:43.822051+0000","last_unstale":"2024-09-17T02:42:45.882301+0000","last_undegraded":"2024-09-17T02:42:45.882301+0000","last_fullsized":"2024-09-17T02:42:45.882301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:25.389352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882609+0000","last_change":"2024-09-17T02:42:44.881443+0000","last_active":"2024-09-17T02:42:45.882609+0000","last_peered":"2024-09-17T02:42:45.882609+0000","last_clean":"2024-09-17T02:42:45.882609+0000","last_became_active":"2024-09-17T02:42:44.881026+0000","last_became_peered":"2024-09-17T02:42:44.881026+0000","last_unstale":"2024-09-17T02:42:45.882609+0000","last_undegraded":"2024-09-17T02:42:45.882609+0000","last_fullsized":"2024-09-17T02:42:45.882609+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866487+0000","last_change":"2024-09-17T02:42:43.845560+0000","last_active":"2024-09-17T02:42:45.866487+0000","last_peered":"2024-09-17T02:42:45.866487+0000","last_clean":"2024-09-17T02:42:45.866487+0000","last_became_active":"2024-09-17T02:42:43.845154+0000","last_became_peered":"2024-09-17T02:42:43.845154+0000","last_unstale":"2024-09-17T02:42:45.866487+0000","last_undegraded":"2024-09-17T02:42:45.866487+0000","last_fullsized":"2024-09-17T02:42:45.866487+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:33.597784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.887379+0000","last_change":"2024-09-17T02:42:44.881402+0000","last_active":"2024-09-17T02:42:45.887379+0000","last_peered":"2024-09-17T02:42:45.887379+0000","last_clean":"2024-09-17T02:42:45.887379+0000","last_became_active":"2024-09-17T02:42:44.880572+0000","last_became_peered":"2024-09-17T02:42:44.880572+0000","last_unstale":"2024-09-17T02:42:45.887379+0000","last_undegraded":"2024-09-17T02:42:45.887379+0000","last_fullsized":"2024-09-17T02:42:45.887379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:23.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.874020+0000","last_change":"2024-09-17T02:42:43.834185+0000","last_active":"2024-09-17T02:42:45.874020+0000","last_peered":"2024-09-17T02:42:45.874020+0000","last_clean":"2024-09-17T02:42:45.874020+0000","last_became_active":"2024-09-17T02:42:43.832748+0000","last_became_peered":"2024-09-17T02:42:43.832748+0000","last_unstale":"2024-09-17T02:42:45.874020+0000","last_undegraded":"2024-09-17T02:42:45.874020+0000","last_fullsized":"2024-09-17T02:42:45.874020+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:19.775974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":168,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866609+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:42:45.866609+0000","last_peered":"2024-09-17T02:42:45.866609+0000","last_clean":"2024-09-17T02:42:45.866609+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:42:45.866609+0000","last_undegraded":"2024-09-17T02:42:45.866609+0000","last_fullsized":"2024-09-17T02:42:45.866609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27628,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344212,"statfs":{"total":96636764160,"available":96608473088,"internally_reserved":0,"allocated":606208,"data_stored":262756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28216,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343624,"statfs":{"total":96636764160,"available":96607870976,"internally_reserved":0,"allocated":1208320,"data_stored":854770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":271096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27620,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344220,"statfs":{"total":96636764160,"available":96608481280,"internally_reserved":0,"allocated":602112,"data_stored":262790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":614400,"data_stored":269426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1196032,"data_stored":853138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":602112,"data_stored":263282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":573440,"data_stored":248866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:42:53.682 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:42:53.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.847+0000 7f76a578e640 1 -- 172.21.15.131:0/1156912236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76a00cf480 msgr2=0x7f76a0169700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:53.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.847+0000 7f76a578e640 1 --2- 172.21.15.131:0/1156912236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76a00cf480 0x7f76a0169700 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7698009bc0 tx=0x7f769801cb10 comp rx=0 tx=0).stop 2024-09-17T02:42:53.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.847+0000 7f76a578e640 1 -- 172.21.15.131:0/1156912236 shutdown_connections 2024-09-17T02:42:53.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.847+0000 7f76a578e640 1 --2- 172.21.15.131:0/1156912236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76a00cf480 0x7f76a0169700 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:53.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.847+0000 7f76a578e640 1 -- 172.21.15.131:0/1156912236 >> 172.21.15.131:0/1156912236 conn(0x7f76a006eea0 msgr2=0x7f76a006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:53.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.847+0000 7f76a578e640 1 -- 172.21.15.131:0/1156912236 shutdown_connections 2024-09-17T02:42:53.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.847+0000 7f76a578e640 1 -- 172.21.15.131:0/1156912236 wait complete. 2024-09-17T02:42:53.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.847+0000 7f76a578e640 1 Processor -- start 2024-09-17T02:42:53.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.847+0000 7f76a578e640 1 -- start start 2024-09-17T02:42:53.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76a578e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76a006dea0 0x7f76a01fc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:53.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76a578e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f76a00cac80 0x7f76a00c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:53.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76a578e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a00cf480 0x7f76a00b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:53.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76a578e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f76a016cf40 con 0x7f76a006dea0 2024-09-17T02:42:53.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76a578e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76a016cdc0 con 0x7f76a00cf480 2024-09-17T02:42:53.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76a578e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f76a016d0c0 con 0x7f76a00cac80 2024-09-17T02:42:53.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76a006dea0 0x7f76a01fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:53.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76a006dea0 0x7f76a01fc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56954/0 (socket says 172.21.15.131:56954) 2024-09-17T02:42:53.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769effd640 1 -- 172.21.15.131:0/597221867 learned_addr learned my addr 172.21.15.131:0/597221867 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:53.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769f7fe640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a00cf480 0x7f76a00b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:53.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769e7fc640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f76a00cac80 0x7f76a00c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:53.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769effd640 1 -- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f76a00cac80 msgr2=0x7f76a00c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:53.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769effd640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f76a00cac80 0x7f76a00c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:53.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769effd640 1 -- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a00cf480 msgr2=0x7f76a00b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:53.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769effd640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a00cf480 0x7f76a00b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:53.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769effd640 1 -- 172.21.15.131:0/597221867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a0150e20 con 0x7f76a006dea0 2024-09-17T02:42:53.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769f7fe640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a00cf480 0x7f76a00b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:53.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f769effd640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76a006dea0 0x7f76a01fc770 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f769000c400 tx=0x7f769000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:53.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76877fe640 1 -- 172.21.15.131:0/597221867 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690017070 con 0x7f76a006dea0 2024-09-17T02:42:53.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76877fe640 1 -- 172.21.15.131:0/597221867 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7690003600 con 0x7f76a006dea0 2024-09-17T02:42:53.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76a578e640 1 -- 172.21.15.131:0/597221867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a01fccb0 con 0x7f76a006dea0 2024-09-17T02:42:53.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76877fe640 1 -- 172.21.15.131:0/597221867 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7690003a50 con 0x7f76a006dea0 2024-09-17T02:42:53.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76a578e640 1 -- 172.21.15.131:0/597221867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a00c6370 con 0x7f76a006dea0 2024-09-17T02:42:53.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.851+0000 7f76a578e640 1 -- 172.21.15.131:0/597221867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a00cb090 con 0x7f76a006dea0 2024-09-17T02:42:53.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.855+0000 7f76877fe640 1 -- 172.21.15.131:0/597221867 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f769001a070 con 0x7f76a006dea0 2024-09-17T02:42:53.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.855+0000 7f76877fe640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f768c073230 0x7f768c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:53.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.855+0000 7f76877fe640 1 -- 172.21.15.131:0/597221867 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f7690093ad0 con 0x7f76a006dea0 2024-09-17T02:42:53.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.855+0000 7f769e7fc640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f768c073230 0x7f768c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:53.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.855+0000 7f769e7fc640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f768c073230 0x7f768c075740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f76a014dbf0 tx=0x7f7694008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:53.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:53.859+0000 7f76877fe640 1 -- 172.21.15.131:0/597221867 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7690093e20 con 0x7f76a006dea0 2024-09-17T02:42:54.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.019+0000 7f76a578e640 1 -- 172.21.15.131:0/597221867 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76a0150350 con 0x7f768c073230 2024-09-17T02:42:54.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.031+0000 7f76877fe640 1 -- 172.21.15.131:0/597221867 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251872 (secure 0 0 0) 0x7f769400c4f0 con 0x7f768c073230 2024-09-17T02:42:54.038 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:54.040 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:42:54.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 -- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f768c073230 msgr2=0x7f768c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f768c073230 0x7f768c075740 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f76a014dbf0 tx=0x7f7694008040 comp rx=0 tx=0).stop 2024-09-17T02:42:54.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 -- 172.21.15.131:0/597221867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76a006dea0 msgr2=0x7f76a01fc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76a006dea0 0x7f76a01fc770 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f769000c400 tx=0x7f769000c8c0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 -- 172.21.15.131:0/597221867 shutdown_connections 2024-09-17T02:42:54.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f768c073230 0x7f768c075740 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76a00cf480 0x7f76a00b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f76a00cac80 0x7f76a00c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 --2- 172.21.15.131:0/597221867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76a006dea0 0x7f76a01fc770 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 -- 172.21.15.131:0/597221867 >> 172.21.15.131:0/597221867 conn(0x7f76a006eea0 msgr2=0x7f76a0072b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:54.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 -- 172.21.15.131:0/597221867 shutdown_connections 2024-09-17T02:42:54.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.035+0000 7f76a578e640 1 -- 172.21.15.131:0/597221867 wait complete. 2024-09-17T02:42:54.059 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-17T02:42:53.163896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238868,"kb_used_data":5992,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735852,"statfs":{"total":773094113280,"available":772849512448,"internally_reserved":0,"allocated":6135808,"data_stored":3340692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.327727"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.871082+0000","last_change":"2024-09-17T02:42:44.876606+0000","last_active":"2024-09-17T02:42:45.871082+0000","last_peered":"2024-09-17T02:42:45.871082+0000","last_clean":"2024-09-17T02:42:45.871082+0000","last_became_active":"2024-09-17T02:42:44.876193+0000","last_became_peered":"2024-09-17T02:42:44.876193+0000","last_unstale":"2024-09-17T02:42:45.871082+0000","last_undegraded":"2024-09-17T02:42:45.871082+0000","last_fullsized":"2024-09-17T02:42:45.871082+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:32.049412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883357+0000","last_change":"2024-09-17T02:42:44.884814+0000","last_active":"2024-09-17T02:42:45.883357+0000","last_peered":"2024-09-17T02:42:45.883357+0000","last_clean":"2024-09-17T02:42:45.883357+0000","last_became_active":"2024-09-17T02:42:44.884432+0000","last_became_peered":"2024-09-17T02:42:44.884432+0000","last_unstale":"2024-09-17T02:42:45.883357+0000","last_undegraded":"2024-09-17T02:42:45.883357+0000","last_fullsized":"2024-09-17T02:42:45.883357+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:23.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884491+0000","last_change":"2024-09-17T02:42:44.854032+0000","last_active":"2024-09-17T02:42:45.884491+0000","last_peered":"2024-09-17T02:42:45.884491+0000","last_clean":"2024-09-17T02:42:45.884491+0000","last_became_active":"2024-09-17T02:42:44.853205+0000","last_became_peered":"2024-09-17T02:42:44.853205+0000","last_unstale":"2024-09-17T02:42:45.884491+0000","last_undegraded":"2024-09-17T02:42:45.884491+0000","last_fullsized":"2024-09-17T02:42:45.884491+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:18.997673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867302+0000","last_change":"2024-09-17T02:42:44.878169+0000","last_active":"2024-09-17T02:42:45.867302+0000","last_peered":"2024-09-17T02:42:45.867302+0000","last_clean":"2024-09-17T02:42:45.867302+0000","last_became_active":"2024-09-17T02:42:44.877402+0000","last_became_peered":"2024-09-17T02:42:44.877402+0000","last_unstale":"2024-09-17T02:42:45.867302+0000","last_undegraded":"2024-09-17T02:42:45.867302+0000","last_fullsized":"2024-09-17T02:42:45.867302+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:55.948606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882642+0000","last_change":"2024-09-17T02:42:44.872147+0000","last_active":"2024-09-17T02:42:45.882642+0000","last_peered":"2024-09-17T02:42:45.882642+0000","last_clean":"2024-09-17T02:42:45.882642+0000","last_became_active":"2024-09-17T02:42:44.871745+0000","last_became_peered":"2024-09-17T02:42:44.871745+0000","last_unstale":"2024-09-17T02:42:45.882642+0000","last_undegraded":"2024-09-17T02:42:45.882642+0000","last_fullsized":"2024-09-17T02:42:45.882642+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:29.767271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882210+0000","last_change":"2024-09-17T02:42:44.885865+0000","last_active":"2024-09-17T02:42:45.882210+0000","last_peered":"2024-09-17T02:42:45.882210+0000","last_clean":"2024-09-17T02:42:45.882210+0000","last_became_active":"2024-09-17T02:42:44.885512+0000","last_became_peered":"2024-09-17T02:42:44.885512+0000","last_unstale":"2024-09-17T02:42:45.882210+0000","last_undegraded":"2024-09-17T02:42:45.882210+0000","last_fullsized":"2024-09-17T02:42:45.882210+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:56.895371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870233+0000","last_change":"2024-09-17T02:42:44.886317+0000","last_active":"2024-09-17T02:42:45.870233+0000","last_peered":"2024-09-17T02:42:45.870233+0000","last_clean":"2024-09-17T02:42:45.870233+0000","last_became_active":"2024-09-17T02:42:44.885948+0000","last_became_peered":"2024-09-17T02:42:44.885948+0000","last_unstale":"2024-09-17T02:42:45.870233+0000","last_undegraded":"2024-09-17T02:42:45.870233+0000","last_fullsized":"2024-09-17T02:42:45.870233+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:44.318617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881521+0000","last_change":"2024-09-17T02:42:44.876332+0000","last_active":"2024-09-17T02:42:45.881521+0000","last_peered":"2024-09-17T02:42:45.881521+0000","last_clean":"2024-09-17T02:42:45.881521+0000","last_became_active":"2024-09-17T02:42:44.875594+0000","last_became_peered":"2024-09-17T02:42:44.875594+0000","last_unstale":"2024-09-17T02:42:45.881521+0000","last_undegraded":"2024-09-17T02:42:45.881521+0000","last_fullsized":"2024-09-17T02:42:45.881521+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:34.865339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884540+0000","last_change":"2024-09-17T02:42:44.854764+0000","last_active":"2024-09-17T02:42:45.884540+0000","last_peered":"2024-09-17T02:42:45.884540+0000","last_clean":"2024-09-17T02:42:45.884540+0000","last_became_active":"2024-09-17T02:42:44.853822+0000","last_became_peered":"2024-09-17T02:42:44.853822+0000","last_unstale":"2024-09-17T02:42:45.884540+0000","last_undegraded":"2024-09-17T02:42:45.884540+0000","last_fullsized":"2024-09-17T02:42:45.884540+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:18.705995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873645+0000","last_change":"2024-09-17T02:42:44.889606+0000","last_active":"2024-09-17T02:42:45.873645+0000","last_peered":"2024-09-17T02:42:45.873645+0000","last_clean":"2024-09-17T02:42:45.873645+0000","last_became_active":"2024-09-17T02:42:44.889274+0000","last_became_peered":"2024-09-17T02:42:44.889274+0000","last_unstale":"2024-09-17T02:42:45.873645+0000","last_undegraded":"2024-09-17T02:42:45.873645+0000","last_fullsized":"2024-09-17T02:42:45.873645+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:12:12.214981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.871610+0000","last_change":"2024-09-17T02:42:44.871373+0000","last_active":"2024-09-17T02:42:45.871610+0000","last_peered":"2024-09-17T02:42:45.871610+0000","last_clean":"2024-09-17T02:42:45.871610+0000","last_became_active":"2024-09-17T02:42:44.867897+0000","last_became_peered":"2024-09-17T02:42:44.867897+0000","last_unstale":"2024-09-17T02:42:45.871610+0000","last_undegraded":"2024-09-17T02:42:45.871610+0000","last_fullsized":"2024-09-17T02:42:45.871610+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:10.649860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882022+0000","last_change":"2024-09-17T02:42:44.880065+0000","last_active":"2024-09-17T02:42:45.882022+0000","last_peered":"2024-09-17T02:42:45.882022+0000","last_clean":"2024-09-17T02:42:45.882022+0000","last_became_active":"2024-09-17T02:42:44.876260+0000","last_became_peered":"2024-09-17T02:42:44.876260+0000","last_unstale":"2024-09-17T02:42:45.882022+0000","last_undegraded":"2024-09-17T02:42:45.882022+0000","last_fullsized":"2024-09-17T02:42:45.882022+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:34:02.925523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883058+0000","last_change":"2024-09-17T02:42:44.882550+0000","last_active":"2024-09-17T02:42:45.883058+0000","last_peered":"2024-09-17T02:42:45.883058+0000","last_clean":"2024-09-17T02:42:45.883058+0000","last_became_active":"2024-09-17T02:42:44.882123+0000","last_became_peered":"2024-09-17T02:42:44.882123+0000","last_unstale":"2024-09-17T02:42:45.883058+0000","last_undegraded":"2024-09-17T02:42:45.883058+0000","last_fullsized":"2024-09-17T02:42:45.883058+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:27:24.890536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.872105+0000","last_change":"2024-09-17T02:42:44.877777+0000","last_active":"2024-09-17T02:42:45.872105+0000","last_peered":"2024-09-17T02:42:45.872105+0000","last_clean":"2024-09-17T02:42:45.872105+0000","last_became_active":"2024-09-17T02:42:44.877406+0000","last_became_peered":"2024-09-17T02:42:44.877406+0000","last_unstale":"2024-09-17T02:42:45.872105+0000","last_undegraded":"2024-09-17T02:42:45.872105+0000","last_fullsized":"2024-09-17T02:42:45.872105+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:48.472139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866923+0000","last_change":"2024-09-17T02:42:44.877349+0000","last_active":"2024-09-17T02:42:45.866923+0000","last_peered":"2024-09-17T02:42:45.866923+0000","last_clean":"2024-09-17T02:42:45.866923+0000","last_became_active":"2024-09-17T02:42:44.876816+0000","last_became_peered":"2024-09-17T02:42:44.876816+0000","last_unstale":"2024-09-17T02:42:45.866923+0000","last_undegraded":"2024-09-17T02:42:45.866923+0000","last_fullsized":"2024-09-17T02:42:45.866923+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:22.741898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881823+0000","last_change":"2024-09-17T02:42:44.876906+0000","last_active":"2024-09-17T02:42:45.881823+0000","last_peered":"2024-09-17T02:42:45.881823+0000","last_clean":"2024-09-17T02:42:45.881823+0000","last_became_active":"2024-09-17T02:42:44.875966+0000","last_became_peered":"2024-09-17T02:42:44.875966+0000","last_unstale":"2024-09-17T02:42:45.881823+0000","last_undegraded":"2024-09-17T02:42:45.881823+0000","last_fullsized":"2024-09-17T02:42:45.881823+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:06.983520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.871767+0000","last_change":"2024-09-17T02:42:44.875869+0000","last_active":"2024-09-17T02:42:45.871767+0000","last_peered":"2024-09-17T02:42:45.871767+0000","last_clean":"2024-09-17T02:42:45.871767+0000","last_became_active":"2024-09-17T02:42:44.875533+0000","last_became_peered":"2024-09-17T02:42:44.875533+0000","last_unstale":"2024-09-17T02:42:45.871767+0000","last_undegraded":"2024-09-17T02:42:45.871767+0000","last_fullsized":"2024-09-17T02:42:45.871767+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:18:17.207909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867528+0000","last_change":"2024-09-17T02:42:44.888487+0000","last_active":"2024-09-17T02:42:45.867528+0000","last_peered":"2024-09-17T02:42:45.867528+0000","last_clean":"2024-09-17T02:42:45.867528+0000","last_became_active":"2024-09-17T02:42:44.887970+0000","last_became_peered":"2024-09-17T02:42:44.887970+0000","last_unstale":"2024-09-17T02:42:45.867528+0000","last_undegraded":"2024-09-17T02:42:45.867528+0000","last_fullsized":"2024-09-17T02:42:45.867528+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:42.359754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884599+0000","last_change":"2024-09-17T02:42:44.881307+0000","last_active":"2024-09-17T02:42:45.884599+0000","last_peered":"2024-09-17T02:42:45.884599+0000","last_clean":"2024-09-17T02:42:45.884599+0000","last_became_active":"2024-09-17T02:42:44.880605+0000","last_became_peered":"2024-09-17T02:42:44.880605+0000","last_unstale":"2024-09-17T02:42:45.884599+0000","last_undegraded":"2024-09-17T02:42:45.884599+0000","last_fullsized":"2024-09-17T02:42:45.884599+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:49.563614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883282+0000","last_change":"2024-09-17T02:42:44.887466+0000","last_active":"2024-09-17T02:42:45.883282+0000","last_peered":"2024-09-17T02:42:45.883282+0000","last_clean":"2024-09-17T02:42:45.883282+0000","last_became_active":"2024-09-17T02:42:44.885044+0000","last_became_peered":"2024-09-17T02:42:44.885044+0000","last_unstale":"2024-09-17T02:42:45.883282+0000","last_undegraded":"2024-09-17T02:42:45.883282+0000","last_fullsized":"2024-09-17T02:42:45.883282+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.263210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867244+0000","last_change":"2024-09-17T02:42:44.876044+0000","last_active":"2024-09-17T02:42:45.867244+0000","last_peered":"2024-09-17T02:42:45.867244+0000","last_clean":"2024-09-17T02:42:45.867244+0000","last_became_active":"2024-09-17T02:42:44.873541+0000","last_became_peered":"2024-09-17T02:42:44.873541+0000","last_unstale":"2024-09-17T02:42:45.867244+0000","last_undegraded":"2024-09-17T02:42:45.867244+0000","last_fullsized":"2024-09-17T02:42:45.867244+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:56.895371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883856+0000","last_change":"2024-09-17T02:42:44.879337+0000","last_active":"2024-09-17T02:42:45.883856+0000","last_peered":"2024-09-17T02:42:45.883856+0000","last_clean":"2024-09-17T02:42:45.883856+0000","last_became_active":"2024-09-17T02:42:44.878982+0000","last_became_peered":"2024-09-17T02:42:44.878982+0000","last_unstale":"2024-09-17T02:42:45.883856+0000","last_undegraded":"2024-09-17T02:42:45.883856+0000","last_fullsized":"2024-09-17T02:42:45.883856+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:59.473102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882578+0000","last_change":"2024-09-17T02:42:44.856410+0000","last_active":"2024-09-17T02:42:45.882578+0000","last_peered":"2024-09-17T02:42:45.882578+0000","last_clean":"2024-09-17T02:42:45.882578+0000","last_became_active":"2024-09-17T02:42:44.854487+0000","last_became_peered":"2024-09-17T02:42:44.854487+0000","last_unstale":"2024-09-17T02:42:45.882578+0000","last_undegraded":"2024-09-17T02:42:45.882578+0000","last_fullsized":"2024-09-17T02:42:45.882578+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:07.107071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882985+0000","last_change":"2024-09-17T02:42:44.870968+0000","last_active":"2024-09-17T02:42:45.882985+0000","last_peered":"2024-09-17T02:42:45.882985+0000","last_clean":"2024-09-17T02:42:45.882985+0000","last_became_active":"2024-09-17T02:42:44.870254+0000","last_became_peered":"2024-09-17T02:42:44.870254+0000","last_unstale":"2024-09-17T02:42:45.882985+0000","last_undegraded":"2024-09-17T02:42:45.882985+0000","last_fullsized":"2024-09-17T02:42:45.882985+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:15.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.871246+0000","last_change":"2024-09-17T02:42:44.878272+0000","last_active":"2024-09-17T02:42:45.871246+0000","last_peered":"2024-09-17T02:42:45.871246+0000","last_clean":"2024-09-17T02:42:45.871246+0000","last_became_active":"2024-09-17T02:42:44.877913+0000","last_became_peered":"2024-09-17T02:42:44.877913+0000","last_unstale":"2024-09-17T02:42:45.871246+0000","last_undegraded":"2024-09-17T02:42:45.871246+0000","last_fullsized":"2024-09-17T02:42:45.871246+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:44:39.672455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884665+0000","last_change":"2024-09-17T02:42:44.875582+0000","last_active":"2024-09-17T02:42:45.884665+0000","last_peered":"2024-09-17T02:42:45.884665+0000","last_clean":"2024-09-17T02:42:45.884665+0000","last_became_active":"2024-09-17T02:42:44.875205+0000","last_became_peered":"2024-09-17T02:42:44.875205+0000","last_unstale":"2024-09-17T02:42:45.884665+0000","last_undegraded":"2024-09-17T02:42:45.884665+0000","last_fullsized":"2024-09-17T02:42:45.884665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:22:23.818837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881878+0000","last_change":"2024-09-17T02:42:44.885311+0000","last_active":"2024-09-17T02:42:45.881878+0000","last_peered":"2024-09-17T02:42:45.881878+0000","last_clean":"2024-09-17T02:42:45.881878+0000","last_became_active":"2024-09-17T02:42:44.884647+0000","last_became_peered":"2024-09-17T02:42:44.884647+0000","last_unstale":"2024-09-17T02:42:45.881878+0000","last_undegraded":"2024-09-17T02:42:45.881878+0000","last_fullsized":"2024-09-17T02:42:45.881878+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:20.906053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883665+0000","last_change":"2024-09-17T02:42:44.877183+0000","last_active":"2024-09-17T02:42:45.883665+0000","last_peered":"2024-09-17T02:42:45.883665+0000","last_clean":"2024-09-17T02:42:45.883665+0000","last_became_active":"2024-09-17T02:42:44.875043+0000","last_became_peered":"2024-09-17T02:42:44.875043+0000","last_unstale":"2024-09-17T02:42:45.883665+0000","last_undegraded":"2024-09-17T02:42:45.883665+0000","last_fullsized":"2024-09-17T02:42:45.883665+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:54:53.445161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.872024+0000","last_change":"2024-09-17T02:42:44.873911+0000","last_active":"2024-09-17T02:42:45.872024+0000","last_peered":"2024-09-17T02:42:45.872024+0000","last_clean":"2024-09-17T02:42:45.872024+0000","last_became_active":"2024-09-17T02:42:44.873448+0000","last_became_peered":"2024-09-17T02:42:44.873448+0000","last_unstale":"2024-09-17T02:42:45.872024+0000","last_undegraded":"2024-09-17T02:42:45.872024+0000","last_fullsized":"2024-09-17T02:42:45.872024+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:34.136269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883971+0000","last_change":"2024-09-17T02:42:44.869378+0000","last_active":"2024-09-17T02:42:45.883971+0000","last_peered":"2024-09-17T02:42:45.883971+0000","last_clean":"2024-09-17T02:42:45.883971+0000","last_became_active":"2024-09-17T02:42:44.868147+0000","last_became_peered":"2024-09-17T02:42:44.868147+0000","last_unstale":"2024-09-17T02:42:45.883971+0000","last_undegraded":"2024-09-17T02:42:45.883971+0000","last_fullsized":"2024-09-17T02:42:45.883971+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:03.959077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882887+0000","last_change":"2024-09-17T02:42:44.879236+0000","last_active":"2024-09-17T02:42:45.882887+0000","last_peered":"2024-09-17T02:42:45.882887+0000","last_clean":"2024-09-17T02:42:45.882887+0000","last_became_active":"2024-09-17T02:42:44.878872+0000","last_became_peered":"2024-09-17T02:42:44.878872+0000","last_unstale":"2024-09-17T02:42:45.882887+0000","last_undegraded":"2024-09-17T02:42:45.882887+0000","last_fullsized":"2024-09-17T02:42:45.882887+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:39.841535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882661+0000","last_change":"2024-09-17T02:42:44.853789+0000","last_active":"2024-09-17T02:42:45.882661+0000","last_peered":"2024-09-17T02:42:45.882661+0000","last_clean":"2024-09-17T02:42:45.882661+0000","last_became_active":"2024-09-17T02:42:44.852870+0000","last_became_peered":"2024-09-17T02:42:44.852870+0000","last_unstale":"2024-09-17T02:42:45.882661+0000","last_undegraded":"2024-09-17T02:42:45.882661+0000","last_fullsized":"2024-09-17T02:42:45.882661+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.016019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882301+0000","last_change":"2024-09-17T02:42:44.884160+0000","last_active":"2024-09-17T02:42:45.882301+0000","last_peered":"2024-09-17T02:42:45.882301+0000","last_clean":"2024-09-17T02:42:45.882301+0000","last_became_active":"2024-09-17T02:42:44.883295+0000","last_became_peered":"2024-09-17T02:42:44.883295+0000","last_unstale":"2024-09-17T02:42:45.882301+0000","last_undegraded":"2024-09-17T02:42:45.882301+0000","last_fullsized":"2024-09-17T02:42:45.882301+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:59:13.871419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":19,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885185+0000","last_change":"2024-09-17T02:42:43.845735+0000","last_active":"2024-09-17T02:42:45.885185+0000","last_peered":"2024-09-17T02:42:45.885185+0000","last_clean":"2024-09-17T02:42:45.885185+0000","last_became_active":"2024-09-17T02:42:43.845485+0000","last_became_peered":"2024-09-17T02:42:43.845485+0000","last_unstale":"2024-09-17T02:42:45.885185+0000","last_undegraded":"2024-09-17T02:42:45.885185+0000","last_fullsized":"2024-09-17T02:42:45.885185+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:19.775974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881559+0000","last_change":"2024-09-17T02:42:44.875131+0000","last_active":"2024-09-17T02:42:45.881559+0000","last_peered":"2024-09-17T02:42:45.881559+0000","last_clean":"2024-09-17T02:42:45.881559+0000","last_became_active":"2024-09-17T02:42:44.874308+0000","last_became_peered":"2024-09-17T02:42:44.874308+0000","last_unstale":"2024-09-17T02:42:45.881559+0000","last_undegraded":"2024-09-17T02:42:45.881559+0000","last_fullsized":"2024-09-17T02:42:45.881559+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:20:05.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882119+0000","last_change":"2024-09-17T02:42:43.865164+0000","last_active":"2024-09-17T02:42:45.882119+0000","last_peered":"2024-09-17T02:42:45.882119+0000","last_clean":"2024-09-17T02:42:45.882119+0000","last_became_active":"2024-09-17T02:42:43.862678+0000","last_became_peered":"2024-09-17T02:42:43.862678+0000","last_unstale":"2024-09-17T02:42:45.882119+0000","last_undegraded":"2024-09-17T02:42:45.882119+0000","last_fullsized":"2024-09-17T02:42:45.882119+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:45:46.632441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883731+0000","last_change":"2024-09-17T02:42:44.870094+0000","last_active":"2024-09-17T02:42:45.883731+0000","last_peered":"2024-09-17T02:42:45.883731+0000","last_clean":"2024-09-17T02:42:45.883731+0000","last_became_active":"2024-09-17T02:42:44.869245+0000","last_became_peered":"2024-09-17T02:42:44.869245+0000","last_unstale":"2024-09-17T02:42:45.883731+0000","last_undegraded":"2024-09-17T02:42:45.883731+0000","last_fullsized":"2024-09-17T02:42:45.883731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:18.419470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873178+0000","last_change":"2024-09-17T02:42:43.837893+0000","last_active":"2024-09-17T02:42:45.873178+0000","last_peered":"2024-09-17T02:42:45.873178+0000","last_clean":"2024-09-17T02:42:45.873178+0000","last_became_active":"2024-09-17T02:42:43.833094+0000","last_became_peered":"2024-09-17T02:42:43.833094+0000","last_unstale":"2024-09-17T02:42:45.873178+0000","last_undegraded":"2024-09-17T02:42:45.873178+0000","last_fullsized":"2024-09-17T02:42:45.873178+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:17:27.547510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870591+0000","last_change":"2024-09-17T02:42:44.876821+0000","last_active":"2024-09-17T02:42:45.870591+0000","last_peered":"2024-09-17T02:42:45.870591+0000","last_clean":"2024-09-17T02:42:45.870591+0000","last_became_active":"2024-09-17T02:42:44.876427+0000","last_became_peered":"2024-09-17T02:42:44.876427+0000","last_unstale":"2024-09-17T02:42:45.870591+0000","last_undegraded":"2024-09-17T02:42:45.870591+0000","last_fullsized":"2024-09-17T02:42:45.870591+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:29.285476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884021+0000","last_change":"2024-09-17T02:42:43.859536+0000","last_active":"2024-09-17T02:42:45.884021+0000","last_peered":"2024-09-17T02:42:45.884021+0000","last_clean":"2024-09-17T02:42:45.884021+0000","last_became_active":"2024-09-17T02:42:43.859191+0000","last_became_peered":"2024-09-17T02:42:43.859191+0000","last_unstale":"2024-09-17T02:42:45.884021+0000","last_undegraded":"2024-09-17T02:42:45.884021+0000","last_fullsized":"2024-09-17T02:42:45.884021+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:31:34.772218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.886237+0000","last_change":"2024-09-17T02:42:44.881758+0000","last_active":"2024-09-17T02:42:45.886237+0000","last_peered":"2024-09-17T02:42:45.886237+0000","last_clean":"2024-09-17T02:42:45.886237+0000","last_became_active":"2024-09-17T02:42:44.880037+0000","last_became_peered":"2024-09-17T02:42:44.880037+0000","last_unstale":"2024-09-17T02:42:45.886237+0000","last_undegraded":"2024-09-17T02:42:45.886237+0000","last_fullsized":"2024-09-17T02:42:45.886237+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:49.224409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884002+0000","last_change":"2024-09-17T02:42:43.859603+0000","last_active":"2024-09-17T02:42:45.884002+0000","last_peered":"2024-09-17T02:42:45.884002+0000","last_clean":"2024-09-17T02:42:45.884002+0000","last_became_active":"2024-09-17T02:42:43.856481+0000","last_became_peered":"2024-09-17T02:42:43.856481+0000","last_unstale":"2024-09-17T02:42:45.884002+0000","last_undegraded":"2024-09-17T02:42:45.884002+0000","last_fullsized":"2024-09-17T02:42:45.884002+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:31.985885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884898+0000","last_change":"2024-09-17T02:42:44.891687+0000","last_active":"2024-09-17T02:42:45.884898+0000","last_peered":"2024-09-17T02:42:45.884898+0000","last_clean":"2024-09-17T02:42:45.884898+0000","last_became_active":"2024-09-17T02:42:44.890077+0000","last_became_peered":"2024-09-17T02:42:44.890077+0000","last_unstale":"2024-09-17T02:42:45.884898+0000","last_undegraded":"2024-09-17T02:42:45.884898+0000","last_fullsized":"2024-09-17T02:42:45.884898+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:46:21.265517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884101+0000","last_change":"2024-09-17T02:42:43.822676+0000","last_active":"2024-09-17T02:42:45.884101+0000","last_peered":"2024-09-17T02:42:45.884101+0000","last_clean":"2024-09-17T02:42:45.884101+0000","last_became_active":"2024-09-17T02:42:43.822225+0000","last_became_peered":"2024-09-17T02:42:43.822225+0000","last_unstale":"2024-09-17T02:42:45.884101+0000","last_undegraded":"2024-09-17T02:42:45.884101+0000","last_fullsized":"2024-09-17T02:42:45.884101+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:42:38.838275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.873695+0000","last_change":"2024-09-17T02:42:44.880963+0000","last_active":"2024-09-17T02:42:45.873695+0000","last_peered":"2024-09-17T02:42:45.873695+0000","last_clean":"2024-09-17T02:42:45.873695+0000","last_became_active":"2024-09-17T02:42:44.880257+0000","last_became_peered":"2024-09-17T02:42:44.880257+0000","last_unstale":"2024-09-17T02:42:45.873695+0000","last_undegraded":"2024-09-17T02:42:45.873695+0000","last_fullsized":"2024-09-17T02:42:45.873695+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:51:09.416640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885098+0000","last_change":"2024-09-17T02:42:43.814090+0000","last_active":"2024-09-17T02:42:45.885098+0000","last_peered":"2024-09-17T02:42:45.885098+0000","last_clean":"2024-09-17T02:42:45.885098+0000","last_became_active":"2024-09-17T02:42:43.813685+0000","last_became_peered":"2024-09-17T02:42:43.813685+0000","last_unstale":"2024-09-17T02:42:45.885098+0000","last_undegraded":"2024-09-17T02:42:45.885098+0000","last_fullsized":"2024-09-17T02:42:45.885098+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:39:06.103811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867600+0000","last_change":"2024-09-17T02:42:44.878439+0000","last_active":"2024-09-17T02:42:45.867600+0000","last_peered":"2024-09-17T02:42:45.867600+0000","last_clean":"2024-09-17T02:42:45.867600+0000","last_became_active":"2024-09-17T02:42:44.875651+0000","last_became_peered":"2024-09-17T02:42:44.875651+0000","last_unstale":"2024-09-17T02:42:45.867600+0000","last_undegraded":"2024-09-17T02:42:45.867600+0000","last_fullsized":"2024-09-17T02:42:45.867600+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:29.767271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883310+0000","last_change":"2024-09-17T02:42:43.855843+0000","last_active":"2024-09-17T02:42:45.883310+0000","last_peered":"2024-09-17T02:42:45.883310+0000","last_clean":"2024-09-17T02:42:45.883310+0000","last_became_active":"2024-09-17T02:42:43.854712+0000","last_became_peered":"2024-09-17T02:42:43.854712+0000","last_unstale":"2024-09-17T02:42:45.883310+0000","last_undegraded":"2024-09-17T02:42:45.883310+0000","last_fullsized":"2024-09-17T02:42:45.883310+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:19:02.429366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884260+0000","last_change":"2024-09-17T02:42:44.875385+0000","last_active":"2024-09-17T02:42:45.884260+0000","last_peered":"2024-09-17T02:42:45.884260+0000","last_clean":"2024-09-17T02:42:45.884260+0000","last_became_active":"2024-09-17T02:42:44.874816+0000","last_became_peered":"2024-09-17T02:42:44.874816+0000","last_unstale":"2024-09-17T02:42:45.884260+0000","last_undegraded":"2024-09-17T02:42:45.884260+0000","last_fullsized":"2024-09-17T02:42:45.884260+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:50:36.940445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867765+0000","last_change":"2024-09-17T02:42:43.826860+0000","last_active":"2024-09-17T02:42:45.867765+0000","last_peered":"2024-09-17T02:42:45.867765+0000","last_clean":"2024-09-17T02:42:45.867765+0000","last_became_active":"2024-09-17T02:42:43.826423+0000","last_became_peered":"2024-09-17T02:42:43.826423+0000","last_unstale":"2024-09-17T02:42:45.867765+0000","last_undegraded":"2024-09-17T02:42:45.867765+0000","last_fullsized":"2024-09-17T02:42:45.867765+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:57.315329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866970+0000","last_change":"2024-09-17T02:42:44.889033+0000","last_active":"2024-09-17T02:42:45.866970+0000","last_peered":"2024-09-17T02:42:45.866970+0000","last_clean":"2024-09-17T02:42:45.866970+0000","last_became_active":"2024-09-17T02:42:44.888575+0000","last_became_peered":"2024-09-17T02:42:44.888575+0000","last_unstale":"2024-09-17T02:42:45.866970+0000","last_undegraded":"2024-09-17T02:42:45.866970+0000","last_fullsized":"2024-09-17T02:42:45.866970+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:10.824751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867254+0000","last_change":"2024-09-17T02:42:43.835429+0000","last_active":"2024-09-17T02:42:45.867254+0000","last_peered":"2024-09-17T02:42:45.867254+0000","last_clean":"2024-09-17T02:42:45.867254+0000","last_became_active":"2024-09-17T02:42:43.835072+0000","last_became_peered":"2024-09-17T02:42:43.835072+0000","last_unstale":"2024-09-17T02:42:45.867254+0000","last_undegraded":"2024-09-17T02:42:45.867254+0000","last_fullsized":"2024-09-17T02:42:45.867254+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:55:28.861850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.872498+0000","last_change":"2024-09-17T02:42:44.866035+0000","last_active":"2024-09-17T02:42:45.872498+0000","last_peered":"2024-09-17T02:42:45.872498+0000","last_clean":"2024-09-17T02:42:45.872498+0000","last_became_active":"2024-09-17T02:42:44.865469+0000","last_became_peered":"2024-09-17T02:42:44.865469+0000","last_unstale":"2024-09-17T02:42:45.872498+0000","last_undegraded":"2024-09-17T02:42:45.872498+0000","last_fullsized":"2024-09-17T02:42:45.872498+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:07.790571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885035+0000","last_change":"2024-09-17T02:42:43.844104+0000","last_active":"2024-09-17T02:42:45.885035+0000","last_peered":"2024-09-17T02:42:45.885035+0000","last_clean":"2024-09-17T02:42:45.885035+0000","last_became_active":"2024-09-17T02:42:43.843661+0000","last_became_peered":"2024-09-17T02:42:43.843661+0000","last_unstale":"2024-09-17T02:42:45.885035+0000","last_undegraded":"2024-09-17T02:42:45.885035+0000","last_fullsized":"2024-09-17T02:42:45.885035+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:14.987141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882393+0000","last_change":"2024-09-17T02:42:44.853449+0000","last_active":"2024-09-17T02:42:45.882393+0000","last_peered":"2024-09-17T02:42:45.882393+0000","last_clean":"2024-09-17T02:42:45.882393+0000","last_became_active":"2024-09-17T02:42:44.852561+0000","last_became_peered":"2024-09-17T02:42:44.852561+0000","last_unstale":"2024-09-17T02:42:45.882393+0000","last_undegraded":"2024-09-17T02:42:45.882393+0000","last_fullsized":"2024-09-17T02:42:45.882393+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":21,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.871956+0000","last_change":"2024-09-17T02:42:43.864191+0000","last_active":"2024-09-17T02:42:45.871956+0000","last_peered":"2024-09-17T02:42:45.871956+0000","last_clean":"2024-09-17T02:42:45.871956+0000","last_became_active":"2024-09-17T02:42:43.863400+0000","last_became_peered":"2024-09-17T02:42:43.863400+0000","last_unstale":"2024-09-17T02:42:45.871956+0000","last_undegraded":"2024-09-17T02:42:45.871956+0000","last_fullsized":"2024-09-17T02:42:45.871956+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:09.821491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885484+0000","last_change":"2024-09-17T02:42:44.880368+0000","last_active":"2024-09-17T02:42:45.885484+0000","last_peered":"2024-09-17T02:42:45.885484+0000","last_clean":"2024-09-17T02:42:45.885484+0000","last_became_active":"2024-09-17T02:42:44.879881+0000","last_became_peered":"2024-09-17T02:42:44.879881+0000","last_unstale":"2024-09-17T02:42:45.885484+0000","last_undegraded":"2024-09-17T02:42:45.885484+0000","last_fullsized":"2024-09-17T02:42:45.885484+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:58.252210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883189+0000","last_change":"2024-09-17T02:42:43.841976+0000","last_active":"2024-09-17T02:42:45.883189+0000","last_peered":"2024-09-17T02:42:45.883189+0000","last_clean":"2024-09-17T02:42:45.883189+0000","last_became_active":"2024-09-17T02:42:43.841094+0000","last_became_peered":"2024-09-17T02:42:43.841094+0000","last_unstale":"2024-09-17T02:42:45.883189+0000","last_undegraded":"2024-09-17T02:42:45.883189+0000","last_fullsized":"2024-09-17T02:42:45.883189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:29.331958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.872289+0000","last_change":"2024-09-17T02:42:44.884710+0000","last_active":"2024-09-17T02:42:45.872289+0000","last_peered":"2024-09-17T02:42:45.872289+0000","last_clean":"2024-09-17T02:42:45.872289+0000","last_became_active":"2024-09-17T02:42:44.884359+0000","last_became_peered":"2024-09-17T02:42:44.884359+0000","last_unstale":"2024-09-17T02:42:45.872289+0000","last_undegraded":"2024-09-17T02:42:45.872289+0000","last_fullsized":"2024-09-17T02:42:45.872289+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:56.639098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866984+0000","last_change":"2024-09-17T02:42:43.843349+0000","last_active":"2024-09-17T02:42:45.866984+0000","last_peered":"2024-09-17T02:42:45.866984+0000","last_clean":"2024-09-17T02:42:45.866984+0000","last_became_active":"2024-09-17T02:42:43.842950+0000","last_became_peered":"2024-09-17T02:42:43.842950+0000","last_unstale":"2024-09-17T02:42:45.866984+0000","last_undegraded":"2024-09-17T02:42:45.866984+0000","last_fullsized":"2024-09-17T02:42:45.866984+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:22.974854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882439+0000","last_change":"2024-09-17T02:42:44.886488+0000","last_active":"2024-09-17T02:42:45.882439+0000","last_peered":"2024-09-17T02:42:45.882439+0000","last_clean":"2024-09-17T02:42:45.882439+0000","last_became_active":"2024-09-17T02:42:44.886106+0000","last_became_peered":"2024-09-17T02:42:44.886106+0000","last_unstale":"2024-09-17T02:42:45.882439+0000","last_undegraded":"2024-09-17T02:42:45.882439+0000","last_fullsized":"2024-09-17T02:42:45.882439+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:55.948606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881989+0000","last_change":"2024-09-17T02:42:43.825883+0000","last_active":"2024-09-17T02:42:45.881989+0000","last_peered":"2024-09-17T02:42:45.881989+0000","last_clean":"2024-09-17T02:42:45.881989+0000","last_became_active":"2024-09-17T02:42:43.825522+0000","last_became_peered":"2024-09-17T02:42:43.825522+0000","last_unstale":"2024-09-17T02:42:45.881989+0000","last_undegraded":"2024-09-17T02:42:45.881989+0000","last_fullsized":"2024-09-17T02:42:45.881989+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:22:57.287148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885037+0000","last_change":"2024-09-17T02:42:44.878272+0000","last_active":"2024-09-17T02:42:45.885037+0000","last_peered":"2024-09-17T02:42:45.885037+0000","last_clean":"2024-09-17T02:42:45.885037+0000","last_became_active":"2024-09-17T02:42:44.877921+0000","last_became_peered":"2024-09-17T02:42:44.877921+0000","last_unstale":"2024-09-17T02:42:45.885037+0000","last_undegraded":"2024-09-17T02:42:45.885037+0000","last_fullsized":"2024-09-17T02:42:45.885037+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:14.742444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":23,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.881967+0000","last_change":"2024-09-17T02:42:43.864045+0000","last_active":"2024-09-17T02:42:45.881967+0000","last_peered":"2024-09-17T02:42:45.881967+0000","last_clean":"2024-09-17T02:42:45.881967+0000","last_became_active":"2024-09-17T02:42:43.862689+0000","last_became_peered":"2024-09-17T02:42:43.862689+0000","last_unstale":"2024-09-17T02:42:45.881967+0000","last_undegraded":"2024-09-17T02:42:45.881967+0000","last_fullsized":"2024-09-17T02:42:45.881967+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:30.667789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870056+0000","last_change":"2024-09-17T02:42:44.871420+0000","last_active":"2024-09-17T02:42:45.870056+0000","last_peered":"2024-09-17T02:42:45.870056+0000","last_clean":"2024-09-17T02:42:45.870056+0000","last_became_active":"2024-09-17T02:42:44.871075+0000","last_became_peered":"2024-09-17T02:42:44.871075+0000","last_unstale":"2024-09-17T02:42:45.870056+0000","last_undegraded":"2024-09-17T02:42:45.870056+0000","last_fullsized":"2024-09-17T02:42:45.870056+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:32:21.340145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882977+0000","last_change":"2024-09-17T02:42:43.865760+0000","last_active":"2024-09-17T02:42:45.882977+0000","last_peered":"2024-09-17T02:42:45.882977+0000","last_clean":"2024-09-17T02:42:45.882977+0000","last_became_active":"2024-09-17T02:42:43.862428+0000","last_became_peered":"2024-09-17T02:42:43.862428+0000","last_unstale":"2024-09-17T02:42:45.882977+0000","last_undegraded":"2024-09-17T02:42:45.882977+0000","last_fullsized":"2024-09-17T02:42:45.882977+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:00.644920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870833+0000","last_change":"2024-09-17T02:42:44.853914+0000","last_active":"2024-09-17T02:42:45.870833+0000","last_peered":"2024-09-17T02:42:45.870833+0000","last_clean":"2024-09-17T02:42:45.870833+0000","last_became_active":"2024-09-17T02:42:44.853056+0000","last_became_peered":"2024-09-17T02:42:44.853056+0000","last_unstale":"2024-09-17T02:42:45.870833+0000","last_undegraded":"2024-09-17T02:42:45.870833+0000","last_fullsized":"2024-09-17T02:42:45.870833+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:09:55.764038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882280+0000","last_change":"2024-09-17T02:42:43.839506+0000","last_active":"2024-09-17T02:42:45.882280+0000","last_peered":"2024-09-17T02:42:45.882280+0000","last_clean":"2024-09-17T02:42:45.882280+0000","last_became_active":"2024-09-17T02:42:43.838617+0000","last_became_peered":"2024-09-17T02:42:43.838617+0000","last_unstale":"2024-09-17T02:42:45.882280+0000","last_undegraded":"2024-09-17T02:42:45.882280+0000","last_fullsized":"2024-09-17T02:42:45.882280+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:30:04.244931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883433+0000","last_change":"2024-09-17T02:42:44.876771+0000","last_active":"2024-09-17T02:42:45.883433+0000","last_peered":"2024-09-17T02:42:45.883433+0000","last_clean":"2024-09-17T02:42:45.883433+0000","last_became_active":"2024-09-17T02:42:44.876438+0000","last_became_peered":"2024-09-17T02:42:44.876438+0000","last_unstale":"2024-09-17T02:42:45.883433+0000","last_undegraded":"2024-09-17T02:42:45.883433+0000","last_fullsized":"2024-09-17T02:42:45.883433+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:39:28.921453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867493+0000","last_change":"2024-09-17T02:42:43.862172+0000","last_active":"2024-09-17T02:42:45.867493+0000","last_peered":"2024-09-17T02:42:45.867493+0000","last_clean":"2024-09-17T02:42:45.867493+0000","last_became_active":"2024-09-17T02:42:43.861853+0000","last_became_peered":"2024-09-17T02:42:43.861853+0000","last_unstale":"2024-09-17T02:42:45.867493+0000","last_undegraded":"2024-09-17T02:42:45.867493+0000","last_fullsized":"2024-09-17T02:42:45.867493+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:40:00.644920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883740+0000","last_change":"2024-09-17T02:42:44.875816+0000","last_active":"2024-09-17T02:42:45.883740+0000","last_peered":"2024-09-17T02:42:45.883740+0000","last_clean":"2024-09-17T02:42:45.883740+0000","last_became_active":"2024-09-17T02:42:44.875443+0000","last_became_peered":"2024-09-17T02:42:44.875443+0000","last_unstale":"2024-09-17T02:42:45.883740+0000","last_undegraded":"2024-09-17T02:42:45.883740+0000","last_fullsized":"2024-09-17T02:42:45.883740+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:55:44.263210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884349+0000","last_change":"2024-09-17T02:42:43.842888+0000","last_active":"2024-09-17T02:42:45.884349+0000","last_peered":"2024-09-17T02:42:45.884349+0000","last_clean":"2024-09-17T02:42:45.884349+0000","last_became_active":"2024-09-17T02:42:43.842563+0000","last_became_peered":"2024-09-17T02:42:43.842563+0000","last_unstale":"2024-09-17T02:42:45.884349+0000","last_undegraded":"2024-09-17T02:42:45.884349+0000","last_fullsized":"2024-09-17T02:42:45.884349+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:37.379782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883984+0000","last_change":"2024-09-17T02:42:44.875239+0000","last_active":"2024-09-17T02:42:45.883984+0000","last_peered":"2024-09-17T02:42:45.883984+0000","last_clean":"2024-09-17T02:42:45.883984+0000","last_became_active":"2024-09-17T02:42:44.874268+0000","last_became_peered":"2024-09-17T02:42:44.874268+0000","last_unstale":"2024-09-17T02:42:45.883984+0000","last_undegraded":"2024-09-17T02:42:45.883984+0000","last_fullsized":"2024-09-17T02:42:45.883984+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:30.019802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.872189+0000","last_change":"2024-09-17T02:42:43.840810+0000","last_active":"2024-09-17T02:42:45.872189+0000","last_peered":"2024-09-17T02:42:45.872189+0000","last_clean":"2024-09-17T02:42:45.872189+0000","last_became_active":"2024-09-17T02:42:43.839861+0000","last_became_peered":"2024-09-17T02:42:43.839861+0000","last_unstale":"2024-09-17T02:42:45.872189+0000","last_undegraded":"2024-09-17T02:42:45.872189+0000","last_fullsized":"2024-09-17T02:42:45.872189+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:48:20.403722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.867465+0000","last_change":"2024-09-17T02:42:44.877843+0000","last_active":"2024-09-17T02:42:45.867465+0000","last_peered":"2024-09-17T02:42:45.867465+0000","last_clean":"2024-09-17T02:42:45.867465+0000","last_became_active":"2024-09-17T02:42:44.877029+0000","last_became_peered":"2024-09-17T02:42:44.877029+0000","last_unstale":"2024-09-17T02:42:45.867465+0000","last_undegraded":"2024-09-17T02:42:45.867465+0000","last_fullsized":"2024-09-17T02:42:45.867465+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883764+0000","last_change":"2024-09-17T02:42:43.860370+0000","last_active":"2024-09-17T02:42:45.883764+0000","last_peered":"2024-09-17T02:42:45.883764+0000","last_clean":"2024-09-17T02:42:45.883764+0000","last_became_active":"2024-09-17T02:42:43.859094+0000","last_became_peered":"2024-09-17T02:42:43.859094+0000","last_unstale":"2024-09-17T02:42:45.883764+0000","last_undegraded":"2024-09-17T02:42:45.883764+0000","last_fullsized":"2024-09-17T02:42:45.883764+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:15.987844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870431+0000","last_change":"2024-09-17T02:42:44.868753+0000","last_active":"2024-09-17T02:42:45.870431+0000","last_peered":"2024-09-17T02:42:45.870431+0000","last_clean":"2024-09-17T02:42:45.870431+0000","last_became_active":"2024-09-17T02:42:44.868273+0000","last_became_peered":"2024-09-17T02:42:44.868273+0000","last_unstale":"2024-09-17T02:42:45.870431+0000","last_undegraded":"2024-09-17T02:42:45.870431+0000","last_fullsized":"2024-09-17T02:42:45.870431+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:05:37.566054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870452+0000","last_change":"2024-09-17T02:42:43.862720+0000","last_active":"2024-09-17T02:42:45.870452+0000","last_peered":"2024-09-17T02:42:45.870452+0000","last_clean":"2024-09-17T02:42:45.870452+0000","last_became_active":"2024-09-17T02:42:43.862004+0000","last_became_peered":"2024-09-17T02:42:43.862004+0000","last_unstale":"2024-09-17T02:42:45.870452+0000","last_undegraded":"2024-09-17T02:42:45.870452+0000","last_fullsized":"2024-09-17T02:42:45.870452+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:41.356494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882795+0000","last_change":"2024-09-17T02:42:44.883533+0000","last_active":"2024-09-17T02:42:45.882795+0000","last_peered":"2024-09-17T02:42:45.882795+0000","last_clean":"2024-09-17T02:42:45.882795+0000","last_became_active":"2024-09-17T02:42:44.883166+0000","last_became_peered":"2024-09-17T02:42:44.883166+0000","last_unstale":"2024-09-17T02:42:45.882795+0000","last_undegraded":"2024-09-17T02:42:45.882795+0000","last_fullsized":"2024-09-17T02:42:45.882795+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:56.016019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.869776+0000","last_change":"2024-09-17T02:42:43.831772+0000","last_active":"2024-09-17T02:42:45.869776+0000","last_peered":"2024-09-17T02:42:45.869776+0000","last_clean":"2024-09-17T02:42:45.869776+0000","last_became_active":"2024-09-17T02:42:43.830906+0000","last_became_peered":"2024-09-17T02:42:43.830906+0000","last_unstale":"2024-09-17T02:42:45.869776+0000","last_undegraded":"2024-09-17T02:42:45.869776+0000","last_fullsized":"2024-09-17T02:42:45.869776+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:43.630752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866625+0000","last_change":"2024-09-17T02:42:44.878871+0000","last_active":"2024-09-17T02:42:45.866625+0000","last_peered":"2024-09-17T02:42:45.866625+0000","last_clean":"2024-09-17T02:42:45.866625+0000","last_became_active":"2024-09-17T02:42:44.878370+0000","last_became_peered":"2024-09-17T02:42:44.878370+0000","last_unstale":"2024-09-17T02:42:45.866625+0000","last_undegraded":"2024-09-17T02:42:45.866625+0000","last_fullsized":"2024-09-17T02:42:45.866625+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:44.634012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884758+0000","last_change":"2024-09-17T02:42:43.837536+0000","last_active":"2024-09-17T02:42:45.884758+0000","last_peered":"2024-09-17T02:42:45.884758+0000","last_clean":"2024-09-17T02:42:45.884758+0000","last_became_active":"2024-09-17T02:42:43.837182+0000","last_became_peered":"2024-09-17T02:42:43.837182+0000","last_unstale":"2024-09-17T02:42:45.884758+0000","last_undegraded":"2024-09-17T02:42:45.884758+0000","last_fullsized":"2024-09-17T02:42:45.884758+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:38.099906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870485+0000","last_change":"2024-09-17T02:42:44.870506+0000","last_active":"2024-09-17T02:42:45.870485+0000","last_peered":"2024-09-17T02:42:45.870485+0000","last_clean":"2024-09-17T02:42:45.870485+0000","last_became_active":"2024-09-17T02:42:44.870091+0000","last_became_peered":"2024-09-17T02:42:44.870091+0000","last_unstale":"2024-09-17T02:42:45.870485+0000","last_undegraded":"2024-09-17T02:42:45.870485+0000","last_fullsized":"2024-09-17T02:42:45.870485+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:14.136518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.885151+0000","last_change":"2024-09-17T02:42:43.835536+0000","last_active":"2024-09-17T02:42:45.885151+0000","last_peered":"2024-09-17T02:42:45.885151+0000","last_clean":"2024-09-17T02:42:45.885151+0000","last_became_active":"2024-09-17T02:42:43.835141+0000","last_became_peered":"2024-09-17T02:42:43.835141+0000","last_unstale":"2024-09-17T02:42:45.885151+0000","last_undegraded":"2024-09-17T02:42:45.885151+0000","last_fullsized":"2024-09-17T02:42:45.885151+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:52:58.469842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884026+0000","last_change":"2024-09-17T02:42:44.869196+0000","last_active":"2024-09-17T02:42:45.884026+0000","last_peered":"2024-09-17T02:42:45.884026+0000","last_clean":"2024-09-17T02:42:45.884026+0000","last_became_active":"2024-09-17T02:42:44.868845+0000","last_became_peered":"2024-09-17T02:42:44.868845+0000","last_unstale":"2024-09-17T02:42:45.884026+0000","last_undegraded":"2024-09-17T02:42:45.884026+0000","last_fullsized":"2024-09-17T02:42:45.884026+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:43:44.069731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884247+0000","last_change":"2024-09-17T02:42:43.856625+0000","last_active":"2024-09-17T02:42:45.884247+0000","last_peered":"2024-09-17T02:42:45.884247+0000","last_clean":"2024-09-17T02:42:45.884247+0000","last_became_active":"2024-09-17T02:42:43.855851+0000","last_became_peered":"2024-09-17T02:42:43.855851+0000","last_unstale":"2024-09-17T02:42:45.884247+0000","last_undegraded":"2024-09-17T02:42:45.884247+0000","last_fullsized":"2024-09-17T02:42:45.884247+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:11.579876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.872710+0000","last_change":"2024-09-17T02:42:44.869981+0000","last_active":"2024-09-17T02:42:45.872710+0000","last_peered":"2024-09-17T02:42:45.872710+0000","last_clean":"2024-09-17T02:42:45.872710+0000","last_became_active":"2024-09-17T02:42:44.869502+0000","last_became_peered":"2024-09-17T02:42:44.869502+0000","last_unstale":"2024-09-17T02:42:45.872710+0000","last_undegraded":"2024-09-17T02:42:45.872710+0000","last_fullsized":"2024-09-17T02:42:45.872710+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:22.269723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.870525+0000","last_change":"2024-09-17T02:42:43.847400+0000","last_active":"2024-09-17T02:42:45.870525+0000","last_peered":"2024-09-17T02:42:45.870525+0000","last_clean":"2024-09-17T02:42:45.870525+0000","last_became_active":"2024-09-17T02:42:43.843509+0000","last_became_peered":"2024-09-17T02:42:43.843509+0000","last_unstale":"2024-09-17T02:42:45.870525+0000","last_undegraded":"2024-09-17T02:42:45.870525+0000","last_fullsized":"2024-09-17T02:42:45.870525+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:56:21.738639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.884317+0000","last_change":"2024-09-17T02:42:44.887685+0000","last_active":"2024-09-17T02:42:45.884317+0000","last_peered":"2024-09-17T02:42:45.884317+0000","last_clean":"2024-09-17T02:42:45.884317+0000","last_became_active":"2024-09-17T02:42:44.887127+0000","last_became_peered":"2024-09-17T02:42:44.887127+0000","last_unstale":"2024-09-17T02:42:45.884317+0000","last_undegraded":"2024-09-17T02:42:45.884317+0000","last_fullsized":"2024-09-17T02:42:45.884317+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:23.065669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.883962+0000","last_change":"2024-09-17T02:42:43.859365+0000","last_active":"2024-09-17T02:42:45.883962+0000","last_peered":"2024-09-17T02:42:45.883962+0000","last_clean":"2024-09-17T02:42:45.883962+0000","last_became_active":"2024-09-17T02:42:43.856287+0000","last_became_peered":"2024-09-17T02:42:43.856287+0000","last_unstale":"2024-09-17T02:42:45.883962+0000","last_undegraded":"2024-09-17T02:42:45.883962+0000","last_fullsized":"2024-09-17T02:42:45.883962+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:18.332340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.874248+0000","last_change":"2024-09-17T02:42:44.880311+0000","last_active":"2024-09-17T02:42:45.874248+0000","last_peered":"2024-09-17T02:42:45.874248+0000","last_clean":"2024-09-17T02:42:45.874248+0000","last_became_active":"2024-09-17T02:42:44.879946+0000","last_became_peered":"2024-09-17T02:42:44.879946+0000","last_unstale":"2024-09-17T02:42:45.874248+0000","last_undegraded":"2024-09-17T02:42:45.874248+0000","last_fullsized":"2024-09-17T02:42:45.874248+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:18:32.989145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882301+0000","last_change":"2024-09-17T02:42:43.822424+0000","last_active":"2024-09-17T02:42:45.882301+0000","last_peered":"2024-09-17T02:42:45.882301+0000","last_clean":"2024-09-17T02:42:45.882301+0000","last_became_active":"2024-09-17T02:42:43.822051+0000","last_became_peered":"2024-09-17T02:42:43.822051+0000","last_unstale":"2024-09-17T02:42:45.882301+0000","last_undegraded":"2024-09-17T02:42:45.882301+0000","last_fullsized":"2024-09-17T02:42:45.882301+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:25.389352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.882609+0000","last_change":"2024-09-17T02:42:44.881443+0000","last_active":"2024-09-17T02:42:45.882609+0000","last_peered":"2024-09-17T02:42:45.882609+0000","last_clean":"2024-09-17T02:42:45.882609+0000","last_became_active":"2024-09-17T02:42:44.881026+0000","last_became_peered":"2024-09-17T02:42:44.881026+0000","last_unstale":"2024-09-17T02:42:45.882609+0000","last_undegraded":"2024-09-17T02:42:45.882609+0000","last_fullsized":"2024-09-17T02:42:45.882609+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:15:11.585828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866487+0000","last_change":"2024-09-17T02:42:43.845560+0000","last_active":"2024-09-17T02:42:45.866487+0000","last_peered":"2024-09-17T02:42:45.866487+0000","last_clean":"2024-09-17T02:42:45.866487+0000","last_became_active":"2024-09-17T02:42:43.845154+0000","last_became_peered":"2024-09-17T02:42:43.845154+0000","last_unstale":"2024-09-17T02:42:45.866487+0000","last_undegraded":"2024-09-17T02:42:45.866487+0000","last_fullsized":"2024-09-17T02:42:45.866487+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:33.597784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.887379+0000","last_change":"2024-09-17T02:42:44.881402+0000","last_active":"2024-09-17T02:42:45.887379+0000","last_peered":"2024-09-17T02:42:45.887379+0000","last_clean":"2024-09-17T02:42:45.887379+0000","last_became_active":"2024-09-17T02:42:44.880572+0000","last_became_peered":"2024-09-17T02:42:44.880572+0000","last_unstale":"2024-09-17T02:42:45.887379+0000","last_undegraded":"2024-09-17T02:42:45.887379+0000","last_fullsized":"2024-09-17T02:42:45.887379+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:43.794469+0000","last_clean_scrub_stamp":"2024-09-17T02:42:43.794469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:11:23.751028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"52'1","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:46.177671+0000","last_change":"2024-09-17T02:42:43.834185+0000","last_active":"2024-09-17T02:42:46.177671+0000","last_peered":"2024-09-17T02:42:46.177671+0000","last_clean":"2024-09-17T02:42:46.177671+0000","last_became_active":"2024-09-17T02:42:43.832748+0000","last_became_peered":"2024-09-17T02:42:43.832748+0000","last_unstale":"2024-09-17T02:42:46.177671+0000","last_undegraded":"2024-09-17T02:42:46.177671+0000","last_fullsized":"2024-09-17T02:42:46.177671+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:42:42.791209+0000","last_clean_scrub_stamp":"2024-09-17T02:42:42.791209+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:19.775974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":168,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-17T02:42:45.866609+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:42:45.866609+0000","last_peered":"2024-09-17T02:42:45.866609+0000","last_clean":"2024-09-17T02:42:45.866609+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:42:45.866609+0000","last_undegraded":"2024-09-17T02:42:45.866609+0000","last_fullsized":"2024-09-17T02:42:45.866609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27632,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344208,"statfs":{"total":96636764160,"available":96608468992,"internally_reserved":0,"allocated":618496,"data_stored":269448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1220608,"data_stored":861462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509471,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":271096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":614400,"data_stored":269482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":614400,"data_stored":269426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1208320,"data_stored":859830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31788,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340052,"statfs":{"total":96636764160,"available":96604213248,"internally_reserved":0,"allocated":614400,"data_stored":269974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":614400,"data_stored":269974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:42:54.066 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T02:42:54.066 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T02:42:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.263+0000 7f9c77e78640 1 -- 172.21.15.131:0/2129421704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c700cdf20 msgr2=0x7f9c700709a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.263+0000 7f9c77e78640 1 --2- 172.21.15.131:0/2129421704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c700cdf20 0x7f9c700709a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009c80 tx=0x7f9c6801cd10 comp rx=0 tx=0).stop 2024-09-17T02:42:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.263+0000 7f9c77e78640 1 -- 172.21.15.131:0/2129421704 shutdown_connections 2024-09-17T02:42:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.263+0000 7f9c77e78640 1 --2- 172.21.15.131:0/2129421704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c700cdf20 0x7f9c700709a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.263+0000 7f9c77e78640 1 --2- 172.21.15.131:0/2129421704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c7006f6c0 0x7f9c700cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.263+0000 7f9c77e78640 1 -- 172.21.15.131:0/2129421704 >> 172.21.15.131:0/2129421704 conn(0x7f9c70073ca0 msgr2=0x7f9c700740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:54.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.263+0000 7f9c77e78640 1 -- 172.21.15.131:0/2129421704 shutdown_connections 2024-09-17T02:42:54.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.263+0000 7f9c77e78640 1 -- 172.21.15.131:0/2129421704 wait complete. 2024-09-17T02:42:54.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c77e78640 1 Processor -- start 2024-09-17T02:42:54.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c77e78640 1 -- start start 2024-09-17T02:42:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c77e78640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c700cdf20 0x7f9c70155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c77e78640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c70145cf0 0x7f9c701ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c77e78640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c70155c20 0x7f9c70200090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c77e78640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c70169c70 con 0x7f9c70145cf0 2024-09-17T02:42:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c77e78640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c70169af0 con 0x7f9c700cdf20 2024-09-17T02:42:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c77e78640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9c70169df0 con 0x7f9c70155c20 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c75413640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c70145cf0 0x7f9c701ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c75413640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c70145cf0 0x7f9c701ffb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56980/0 (socket says 172.21.15.131:56980) 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c75413640 1 -- 172.21.15.131:0/346909157 learned_addr learned my addr 172.21.15.131:0/346909157 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c76415640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c70155c20 0x7f9c70200090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c75c14640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c700cdf20 0x7f9c70155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c75413640 1 -- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c70155c20 msgr2=0x7f9c70200090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c75413640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c70155c20 0x7f9c70200090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c75413640 1 -- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c700cdf20 msgr2=0x7f9c70155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c75413640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c700cdf20 0x7f9c70155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c75413640 1 -- 172.21.15.131:0/346909157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c701424d0 con 0x7f9c70145cf0 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c75413640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c70145cf0 0x7f9c701ffb50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000d6b0 tx=0x7f9c6000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c667fc640 1 -- 172.21.15.131:0/346909157 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60017070 con 0x7f9c70145cf0 2024-09-17T02:42:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c667fc640 1 -- 172.21.15.131:0/346909157 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c60005310 con 0x7f9c70145cf0 2024-09-17T02:42:54.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c77e78640 1 -- 172.21.15.131:0/346909157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c70142770 con 0x7f9c70145cf0 2024-09-17T02:42:54.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c667fc640 1 -- 172.21.15.131:0/346909157 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c600097c0 con 0x7f9c70145cf0 2024-09-17T02:42:54.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c77e78640 1 -- 172.21.15.131:0/346909157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c70142d30 con 0x7f9c70145cf0 2024-09-17T02:42:54.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.267+0000 7f9c77e78640 1 -- 172.21.15.131:0/346909157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c700cd9e0 con 0x7f9c70145cf0 2024-09-17T02:42:54.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.271+0000 7f9c667fc640 1 -- 172.21.15.131:0/346909157 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9c60009960 con 0x7f9c70145cf0 2024-09-17T02:42:54.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.271+0000 7f9c667fc640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c54073230 0x7f9c54075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:54.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.271+0000 7f9c667fc640 1 -- 172.21.15.131:0/346909157 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f9c60094270 con 0x7f9c70145cf0 2024-09-17T02:42:54.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.271+0000 7f9c75c14640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c54073230 0x7f9c54075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:54.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.271+0000 7f9c75c14640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c54073230 0x7f9c54075740 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c000f70 tx=0x7f9c6c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:54.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.275+0000 7f9c667fc640 1 -- 172.21.15.131:0/346909157 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9c6005f220 con 0x7f9c70145cf0 2024-09-17T02:42:54.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.507+0000 7f9c77e78640 1 -- 172.21.15.131:0/346909157 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f9c700744c0 con 0x7f9c70145cf0 2024-09-17T02:42:54.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.511+0000 7f9c667fc640 1 -- 172.21.15.131:0/346909157 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v30) ==== 190+0+147 (secure 0 0 0) 0x7f9c6005eb20 con 0x7f9c70145cf0 2024-09-17T02:42:54.517 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-09-17T02:42:54.517 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA== 2024-09-17T02:42:54.517 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-09-17T02:42:54.517 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-09-17T02:42:54.517 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T02:42:54.517 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-09-17T02:42:54.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 -- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c54073230 msgr2=0x7f9c54075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c54073230 0x7f9c54075740 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c000f70 tx=0x7f9c6c009040 comp rx=0 tx=0).stop 2024-09-17T02:42:54.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 -- 172.21.15.131:0/346909157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c70145cf0 msgr2=0x7f9c701ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c70145cf0 0x7f9c701ffb50 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000d6b0 tx=0x7f9c6000db70 comp rx=0 tx=0).stop 2024-09-17T02:42:54.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 -- 172.21.15.131:0/346909157 shutdown_connections 2024-09-17T02:42:54.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9c54073230 0x7f9c54075740 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c70155c20 0x7f9c70200090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9c70145cf0 0x7f9c701ffb50 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 --2- 172.21.15.131:0/346909157 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c700cdf20 0x7f9c70155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 -- 172.21.15.131:0/346909157 >> 172.21.15.131:0/346909157 conn(0x7f9c70073ca0 msgr2=0x7f9c700b1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 -- 172.21.15.131:0/346909157 shutdown_connections 2024-09-17T02:42:54.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.515+0000 7f9c77e78640 1 -- 172.21.15.131:0/346909157 wait complete. 2024-09-17T02:42:54.538 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:54.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.687+0000 7f13cfd84640 1 -- 172.21.15.131:0/1736665836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13c80cf710 msgr2=0x7f13c815f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.687+0000 7f13cfd84640 1 --2- 172.21.15.131:0/1736665836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13c80cf710 0x7f13c815f290 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009900 tx=0x7f13c401c8e0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.687+0000 7f13cfd84640 1 -- 172.21.15.131:0/1736665836 shutdown_connections 2024-09-17T02:42:54.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.687+0000 7f13cfd84640 1 --2- 172.21.15.131:0/1736665836 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13c80cf710 0x7f13c815f290 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.687+0000 7f13cfd84640 1 -- 172.21.15.131:0/1736665836 >> 172.21.15.131:0/1736665836 conn(0x7f13c80d1b10 msgr2=0x7f13c80d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:54.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.687+0000 7f13cfd84640 1 -- 172.21.15.131:0/1736665836 shutdown_connections 2024-09-17T02:42:54.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.687+0000 7f13cfd84640 1 -- 172.21.15.131:0/1736665836 wait complete. 2024-09-17T02:42:54.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cfd84640 1 Processor -- start 2024-09-17T02:42:54.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cfd84640 1 -- start start 2024-09-17T02:42:54.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cfd84640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13c80cf710 0x7f13c815bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:54.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cfd84640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13c80d3470 0x7f13c80cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:54.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cfd84640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f13c815f7d0 0x7f13c80b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:54.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cfd84640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f13c816cfd0 con 0x7f13c80d3470 2024-09-17T02:42:54.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cfd84640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f13c815fdf0 con 0x7f13c80cf710 2024-09-17T02:42:54.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cfd84640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f13c816d150 con 0x7f13c815f7d0 2024-09-17T02:42:54.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cd31f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13c80d3470 0x7f13c80cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:54.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cd31f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13c80d3470 0x7f13c80cbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56996/0 (socket says 172.21.15.131:56996) 2024-09-17T02:42:54.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cd31f640 1 -- 172.21.15.131:0/915135048 learned_addr learned my addr 172.21.15.131:0/915135048 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:54.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cdb20640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13c80cf710 0x7f13c815bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13ce321640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f13c815f7d0 0x7f13c80b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cd31f640 1 -- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f13c815f7d0 msgr2=0x7f13c80b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cd31f640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f13c815f7d0 0x7f13c80b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cd31f640 1 -- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13c80cf710 msgr2=0x7f13c815bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cd31f640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13c80cf710 0x7f13c815bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cd31f640 1 -- 172.21.15.131:0/915135048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c8156a20 con 0x7f13c80d3470 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cd31f640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13c80d3470 0x7f13c80cbf90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c420 tx=0x7f13b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13be7fc640 1 -- 172.21.15.131:0/915135048 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8017070 con 0x7f13c80d3470 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13be7fc640 1 -- 172.21.15.131:0/915135048 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13b80053c0 con 0x7f13c80d3470 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13be7fc640 1 -- 172.21.15.131:0/915135048 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8003860 con 0x7f13c80d3470 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cfd84640 1 -- 172.21.15.131:0/915135048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c80cd2a0 con 0x7f13c80d3470 2024-09-17T02:42:54.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.691+0000 7f13cfd84640 1 -- 172.21.15.131:0/915135048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c80cd830 con 0x7f13c80d3470 2024-09-17T02:42:54.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.695+0000 7f13cfd84640 1 -- 172.21.15.131:0/915135048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c815d960 con 0x7f13c80d3470 2024-09-17T02:42:54.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.695+0000 7f13be7fc640 1 -- 172.21.15.131:0/915135048 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f13b801a070 con 0x7f13c80d3470 2024-09-17T02:42:54.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.695+0000 7f13be7fc640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f13ac073200 0x7f13ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:54.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.695+0000 7f13be7fc640 1 -- 172.21.15.131:0/915135048 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f13b8093700 con 0x7f13c80d3470 2024-09-17T02:42:54.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.695+0000 7f13cdb20640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f13ac073200 0x7f13ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:54.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.695+0000 7f13cdb20640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f13ac073200 0x7f13ac075710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f13c40af000 tx=0x7f13c401b120 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:54.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.699+0000 7f13be7fc640 1 -- 172.21.15.131:0/915135048 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f13b805e620 con 0x7f13c80d3470 2024-09-17T02:42:54.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.891+0000 7f13cfd84640 1 -- 172.21.15.131:0/915135048 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13c815c550 con 0x7f13c80d3470 2024-09-17T02:42:54.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.891+0000 7f13be7fc640 1 -- 172.21.15.131:0/915135048 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f13b805df20 con 0x7f13c80d3470 2024-09-17T02:42:54.895 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:54.895 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"btime":"2024-09-17T02:42:47:892534+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/434043926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":434043926},{"type":"v1","addr":"172.21.15.189:6836","nonce":434043926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5445,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1496962214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1496962214},{"type":"v1","addr":"172.21.15.131:6833","nonce":1496962214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2196282015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2196282015},{"type":"v1","addr":"172.21.15.131:6835","nonce":2196282015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:42:44.218540+0000","modified":"2024-09-17T02:42:47.307497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/4030750882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":4030750882},{"type":"v1","addr":"172.21.15.189:6837","nonce":4030750882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5463,"qdb_cluster":[5463]},"id":3}]} 2024-09-17T02:42:54.895 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-09-17T02:42:54.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 -- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f13ac073200 msgr2=0x7f13ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f13ac073200 0x7f13ac075710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f13c40af000 tx=0x7f13c401b120 comp rx=0 tx=0).stop 2024-09-17T02:42:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 -- 172.21.15.131:0/915135048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13c80d3470 msgr2=0x7f13c80cbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13c80d3470 0x7f13c80cbf90 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c420 tx=0x7f13b800c8e0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 -- 172.21.15.131:0/915135048 shutdown_connections 2024-09-17T02:42:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f13ac073200 0x7f13ac075710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f13c815f7d0 0x7f13c80b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f13c80d3470 0x7f13c80cbf90 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 --2- 172.21.15.131:0/915135048 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f13c80cf710 0x7f13c815bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 -- 172.21.15.131:0/915135048 >> 172.21.15.131:0/915135048 conn(0x7f13c80d1b10 msgr2=0x7f13c8151a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 -- 172.21.15.131:0/915135048 shutdown_connections 2024-09-17T02:42:54.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:54.895+0000 7f13cfd84640 1 -- 172.21.15.131:0/915135048 wait complete. 2024-09-17T02:42:55.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:42:44.218540+0000', 'modified': '2024-09-17T02:42:47.307497+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5463}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5463': {'gid': 5463, 'name': 'b', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.189:6837/4030750882', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6835', 'nonce': 4030750882}, {'type': 'v1', 'addr': '172.21.15.189:6837', 'nonce': 4030750882}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5463, 'qdb_cluster': [5463]} 2024-09-17T02:42:55.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:42:55.318 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T02:42:55.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.483+0000 7f34494a1640 1 -- 172.21.15.131:0/3080106084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3430003a40 msgr2=0x7f3430003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.483+0000 7f34494a1640 1 --2- 172.21.15.131:0/3080106084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3430003a40 0x7f3430003eb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f34340016f0 tx=0x7f343401c7b0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.483+0000 7f34437fe640 1 -- 172.21.15.131:0/3080106084 reap_dead start 2024-09-17T02:42:55.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.483+0000 7f34494a1640 1 -- 172.21.15.131:0/3080106084 shutdown_connections 2024-09-17T02:42:55.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.483+0000 7f34494a1640 1 -- 172.21.15.131:0/3080106084 >> 172.21.15.131:0/3080106084 conn(0x7f344406eea0 msgr2=0x7f344406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:55.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.483+0000 7f34494a1640 1 -- 172.21.15.131:0/3080106084 shutdown_connections 2024-09-17T02:42:55.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.483+0000 7f34494a1640 1 -- 172.21.15.131:0/3080106084 wait complete. 2024-09-17T02:42:55.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.483+0000 7f34494a1640 1 Processor -- start 2024-09-17T02:42:55.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f34494a1640 1 -- start start 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f34494a1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3444154860 0x7f3444166550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f34494a1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3444166a90 0x7f3444153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f34494a1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f34440c8710 0x7f3444154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f34494a1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f344415c230 con 0x7f3444166a90 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f34494a1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f344415c0b0 con 0x7f3444154860 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f34494a1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f34440cc200 con 0x7f34440c8710 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3444166a90 0x7f3444153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3444166a90 0x7f3444153c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57008/0 (socket says 172.21.15.131:57008) 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343bfff640 1 -- 172.21.15.131:0/3616455965 learned_addr learned my addr 172.21.15.131:0/3616455965 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f3442ffd640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3444154860 0x7f3444166550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f34437fe640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f34440c8710 0x7f3444154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343bfff640 1 -- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f34440c8710 msgr2=0x7f3444154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343bfff640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f34440c8710 0x7f3444154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343bfff640 1 -- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3444154860 msgr2=0x7f3444166550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343bfff640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3444154860 0x7f3444166550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343bfff640 1 -- 172.21.15.131:0/3616455965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3444158a30 con 0x7f3444166a90 2024-09-17T02:42:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343bfff640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3444166a90 0x7f3444153c10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f342c011ae0 tx=0x7f342c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343b7fe640 1 -- 172.21.15.131:0/3616455965 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c01e070 con 0x7f3444166a90 2024-09-17T02:42:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343b7fe640 1 -- 172.21.15.131:0/3616455965 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f342c00fe40 con 0x7f3444166a90 2024-09-17T02:42:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f343b7fe640 1 -- 172.21.15.131:0/3616455965 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0042b0 con 0x7f3444166a90 2024-09-17T02:42:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f34494a1640 1 -- 172.21.15.131:0/3616455965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3444158d10 con 0x7f3444166a90 2024-09-17T02:42:55.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.487+0000 7f34494a1640 1 -- 172.21.15.131:0/3616455965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34440b35c0 con 0x7f3444166a90 2024-09-17T02:42:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f343b7fe640 1 -- 172.21.15.131:0/3616455965 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f342c021070 con 0x7f3444166a90 2024-09-17T02:42:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f343b7fe640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3424073200 0x7f3424075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f343b7fe640 1 -- 172.21.15.131:0/3616455965 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f342c094da0 con 0x7f3444166a90 2024-09-17T02:42:55.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f3442ffd640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3424073200 0x7f3424075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f345076c000 1 Processor -- start 2024-09-17T02:42:55.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f345076c000 1 -- start start 2024-09-17T02:42:55.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f3442ffd640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3424073200 0x7f3424075710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f34340016c0 tx=0x7f34340195a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:55.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f345076c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f9441f460 0x557f949606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f345076c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9480f950 0x557f94960be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f345076c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94961120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f345076c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557f94835410 con 0x557f947f8ca0 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f345076c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557f94772b40 con 0x557f9480f950 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f345076c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557f94761f70 con 0x557f9441f460 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f34437fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94961120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f34437fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94961120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57020/0 (socket says 172.21.15.131:57020) 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f34437fe640 1 -- 172.21.15.131:0/841923002 learned_addr learned my addr 172.21.15.131:0/841923002 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f3442ffd640 1 --2- 172.21.15.131:0/841923002 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f9441f460 0x557f949606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f343bfff640 1 --2- 172.21.15.131:0/841923002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9480f950 0x557f94960be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f34437fe640 1 -- 172.21.15.131:0/841923002 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f9441f460 msgr2=0x557f949606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f34437fe640 1 --2- 172.21.15.131:0/841923002 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f9441f460 0x557f949606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f34437fe640 1 -- 172.21.15.131:0/841923002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9480f950 msgr2=0x557f94960be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f34437fe640 1 --2- 172.21.15.131:0/841923002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9480f950 0x557f94960be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f34437fe640 1 -- 172.21.15.131:0/841923002 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f947295b0 con 0x557f947f8ca0 2024-09-17T02:42:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f34437fe640 1 --2- 172.21.15.131:0/841923002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94961120 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f343c001340 tx=0x7f343c003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222f26aa026ad181 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f3438ff9640 1 -- 172.21.15.131:0/841923002 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c006790 con 0x557f947f8ca0 2024-09-17T02:42:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f3438ff9640 1 -- 172.21.15.131:0/841923002 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f343c016070 con 0x557f947f8ca0 2024-09-17T02:42:55.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.491+0000 7f3438ff9640 1 -- 172.21.15.131:0/841923002 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c0126a0 con 0x557f947f8ca0 2024-09-17T02:42:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 -- 172.21.15.131:0/841923002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 msgr2=0x557f94961120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 --2- 172.21.15.131:0/841923002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94961120 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f343c001340 tx=0x7f343c003e30 comp rx=0 tx=0).stop 2024-09-17T02:42:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 -- 172.21.15.131:0/841923002 shutdown_connections 2024-09-17T02:42:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 --2- 172.21.15.131:0/841923002 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94961120 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 --2- 172.21.15.131:0/841923002 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9480f950 0x557f94960be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 --2- 172.21.15.131:0/841923002 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f9441f460 0x557f949606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 -- 172.21.15.131:0/841923002 >> 172.21.15.131:0/841923002 conn(0x557f948e5d90 msgr2=0x557f94915690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 -- 172.21.15.131:0/841923002 shutdown_connections 2024-09-17T02:42:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 -- 172.21.15.131:0/841923002 wait complete. 2024-09-17T02:42:55.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 -- start start 2024-09-17T02:42:55.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 Processor -- start 2024-09-17T02:42:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f94967920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94967e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f9480f950 0x557f949683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557f946fc5f0 con 0x557f947f8ca0 2024-09-17T02:42:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557f94765d80 con 0x557f9441f460 2024-09-17T02:42:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557f94708520 con 0x557f9480f950 2024-09-17T02:42:55.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 10 client.0 _ll_register_callbacks cb 0x557f9470c830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f343bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94967e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f343bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94967e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57026/0 (socket says 172.21.15.131:57026) 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f343bfff640 1 -- 172.21.15.131:0/3604355797 learned_addr learned my addr 172.21.15.131:0/3604355797 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f3442ffd640 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f94967920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f34437fe640 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f9480f950 0x557f949683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f343bfff640 1 -- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f9480f950 msgr2=0x557f949683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f343bfff640 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f9480f950 0x557f949683a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f343bfff640 1 -- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 msgr2=0x557f94967920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f343bfff640 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f94967920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f343bfff640 1 -- 172.21.15.131:0/3604355797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f94700190 con 0x557f947f8ca0 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f3442ffd640 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f94967920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f343bfff640 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94967e60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f342c093140 tx=0x7f342c05f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ba47a0d0827ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f3421ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:42:55.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f3421ffb640 1 -- 172.21.15.131:0/3604355797 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c069080 con 0x557f947f8ca0 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 -- 172.21.15.131:0/3604355797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557f946fa110 con 0x557f947f8ca0 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f3421ffb640 1 -- 172.21.15.131:0/3604355797 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f342c0663d0 con 0x557f947f8ca0 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f3421ffb640 1 -- 172.21.15.131:0/3604355797 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f342c0665d0 con 0x557f947f8ca0 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 10 client.5604 fetch_fsmap learned FSMap version 27 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f345076c000 1 -- 172.21.15.131:0/3604355797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x557f947e6f10 con 0x557f947f8ca0 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.495+0000 7f3421ffb640 1 -- 172.21.15.131:0/3604355797 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557f946fa110 con 0x557f947f8ca0 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f3421ffb640 1 -- 172.21.15.131:0/3604355797 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f342c068040 con 0x557f947f8ca0 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 10 client.5604 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 10 client.5604 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 20 client.5604 populate_metadata read hostname 'smithi131' 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x557f944047d0 0x557f94971d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 4 client.5604 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 -- 172.21.15.131:0/3604355797 --> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f94774a90 con 0x557f944047d0 2024-09-17T02:42:55.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f3442ffd640 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x557f944047d0 0x557f94971d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f3442ffd640 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x557f944047d0 0x557f94971d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b11ef6980b49b9a server_cookie=b0a2263c087283b7 in_seq=0 out_seq=0 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f3421ffb640 10 client.5604 ms_handle_connect on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f3421ffb640 1 -- 172.21.15.131:0/3604355797 <== mds.0 v2:172.21.15.189:6835/4030750882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557f94774a90 con 0x557f944047d0 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f3421ffb640 10 client.5604 handle_command_reply: tid=0 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 client.5604 shutdown 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 -- 172.21.15.131:0/3604355797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 msgr2=0x557f94967e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94967e60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f342c093140 tx=0x7f342c05f2b0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 -- 172.21.15.131:0/3604355797 shutdown_connections 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x557f944047d0 0x557f94971d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f9480f950 0x557f949683a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f947f8ca0 0x557f94967e60 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 --2- 172.21.15.131:0/3604355797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f94967920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f3421ffb640 0 client.5604 ms_handle_reset on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 -- 172.21.15.131:0/3604355797 >> 172.21.15.131:0/3604355797 conn(0x557f948e5d90 msgr2=0x557f9491c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 -- 172.21.15.131:0/3604355797 shutdown_connections 2024-09-17T02:42:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 1 -- 172.21.15.131:0/3604355797 wait complete. 2024-09-17T02:42:55.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.499+0000 7f345076c000 20 client.5604 trim_cache size 0 max 16384 2024-09-17T02:42:55.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.519+0000 7f345076c000 1 Processor -- start 2024-09-17T02:42:55.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.519+0000 7f345076c000 1 -- start start 2024-09-17T02:42:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.519+0000 7f345076c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f949d2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.519+0000 7f345076c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f947f8ca0 0x557f949d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.519+0000 7f345076c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.519+0000 7f345076c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3444205d20 con 0x557f9480f950 2024-09-17T02:42:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.519+0000 7f345076c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3444204dc0 con 0x557f9441f460 2024-09-17T02:42:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.519+0000 7f345076c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f34441fbe20 con 0x557f947f8ca0 2024-09-17T02:42:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.519+0000 7f34437fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.519+0000 7f34437fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57038/0 (socket says 172.21.15.131:57038) 2024-09-17T02:42:55.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.519+0000 7f34437fe640 1 -- 172.21.15.131:0/2213851236 learned_addr learned my addr 172.21.15.131:0/2213851236 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f3442ffd640 1 --2- 172.21.15.131:0/2213851236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f949d2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f343bfff640 1 --2- 172.21.15.131:0/2213851236 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f947f8ca0 0x557f949d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f34437fe640 1 -- 172.21.15.131:0/2213851236 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f947f8ca0 msgr2=0x557f949d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f34437fe640 1 --2- 172.21.15.131:0/2213851236 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f947f8ca0 0x557f949d2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f34437fe640 1 -- 172.21.15.131:0/2213851236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 msgr2=0x557f949d2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f34437fe640 1 --2- 172.21.15.131:0/2213851236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f949d2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f34437fe640 1 -- 172.21.15.131:0/2213851236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f947bddb0 con 0x557f9480f950 2024-09-17T02:42:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f34437fe640 1 --2- 172.21.15.131:0/2213851236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d30a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f34440cca70 tx=0x7f344415fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335039a9e2b36113 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f3423fff640 1 -- 172.21.15.131:0/2213851236 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c006590 con 0x557f9480f950 2024-09-17T02:42:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f3423fff640 1 -- 172.21.15.131:0/2213851236 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f343c016040 con 0x557f9480f950 2024-09-17T02:42:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f3423fff640 1 -- 172.21.15.131:0/2213851236 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c01c470 con 0x557f9480f950 2024-09-17T02:42:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 -- 172.21.15.131:0/2213851236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 msgr2=0x557f949d30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 --2- 172.21.15.131:0/2213851236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d30a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f34440cca70 tx=0x7f344415fe90 comp rx=0 tx=0).stop 2024-09-17T02:42:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 -- 172.21.15.131:0/2213851236 shutdown_connections 2024-09-17T02:42:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 --2- 172.21.15.131:0/2213851236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d30a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 --2- 172.21.15.131:0/2213851236 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f947f8ca0 0x557f949d2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 --2- 172.21.15.131:0/2213851236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f949d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 -- 172.21.15.131:0/2213851236 >> 172.21.15.131:0/2213851236 conn(0x557f948e5d90 msgr2=0x557f9495e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 -- 172.21.15.131:0/2213851236 shutdown_connections 2024-09-17T02:42:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 -- 172.21.15.131:0/2213851236 wait complete. 2024-09-17T02:42:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 -- start start 2024-09-17T02:42:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 Processor -- start 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f949d65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f947f8ca0 0x557f949d6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3444204770 con 0x557f9480f950 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557f94765d80 con 0x557f9441f460 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557f947e6a80 con 0x557f947f8ca0 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f345076c000 10 client.0 _ll_register_callbacks cb 0x7f342c05bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f34437fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f34437fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57050/0 (socket says 172.21.15.131:57050) 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.523+0000 7f34437fe640 1 -- 172.21.15.131:0/1913790261 learned_addr learned my addr 172.21.15.131:0/1913790261 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f343bfff640 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f947f8ca0 0x557f949d6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3442ffd640 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f949d65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34437fe640 1 -- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f947f8ca0 msgr2=0x557f949d6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34437fe640 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f947f8ca0 0x557f949d6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34437fe640 1 -- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 msgr2=0x557f949d65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34437fe640 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f949d65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34437fe640 1 -- 172.21.15.131:0/1913790261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f94728ab0 con 0x557f9480f950 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f343bfff640 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f947f8ca0 0x557f949d6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34437fe640 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d7020 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f344406eb30 tx=0x7f343c001240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66cf2eadbcff5c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3421ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3421ffb640 1 -- 172.21.15.131:0/1913790261 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c01c470 con 0x557f9480f950 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3421ffb640 1 -- 172.21.15.131:0/1913790261 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f343c02fc50 con 0x557f9480f950 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 -- 172.21.15.131:0/1913790261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f343c016040 con 0x557f9480f950 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3421ffb640 1 -- 172.21.15.131:0/1913790261 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f343c02fe50 con 0x557f9480f950 2024-09-17T02:42:55.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 10 client.5610 fetch_fsmap learned FSMap version 27 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 -- 172.21.15.131:0/1913790261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x557f94707070 con 0x557f9480f950 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3421ffb640 1 -- 172.21.15.131:0/1913790261 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f343c016040 con 0x557f9480f950 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3421ffb640 1 -- 172.21.15.131:0/1913790261 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f343c01c610 con 0x557f9480f950 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 10 client.5610 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 10 client.5610 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 20 client.5610 populate_metadata read hostname 'smithi131' 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x557f943f2830 0x557f94992610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 4 client.5610 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 -- 172.21.15.131:0/1913790261 --> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] -- command(tid 0: {"prefix": "status"}) -- 0x557f94770540 con 0x557f943f2830 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3442ffd640 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x557f943f2830 0x557f94992610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3421ffb640 10 client.5610 ms_handle_connect on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3442ffd640 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x557f943f2830 0x557f94992610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b2827ff261514ee server_cookie=485c7f46bfdf2eb3 in_seq=0 out_seq=0 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3421ffb640 1 -- 172.21.15.131:0/1913790261 <== mds.0 v2:172.21.15.189:6835/4030750882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x557f94770540 con 0x557f943f2830 2024-09-17T02:42:55.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3421ffb640 10 client.5610 handle_command_reply: tid=0 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 client.5610 shutdown 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 -- 172.21.15.131:0/1913790261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 msgr2=0x557f949d7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d7020 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f344406eb30 tx=0x7f343c001240 comp rx=0 tx=0).stop 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 -- 172.21.15.131:0/1913790261 shutdown_connections 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x557f943f2830 0x557f94992610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557f9480f950 0x557f949d7020 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f3421ffb640 0 client.5610 ms_handle_reset on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557f947f8ca0 0x557f949d6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 --2- 172.21.15.131:0/1913790261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557f9441f460 0x557f949d65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 -- 172.21.15.131:0/1913790261 >> 172.21.15.131:0/1913790261 conn(0x557f948e5d90 msgr2=0x557f949716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 -- 172.21.15.131:0/1913790261 shutdown_connections 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 1 -- 172.21.15.131:0/1913790261 wait complete. 2024-09-17T02:42:55.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f345076c000 20 client.5610 trim_cache size 0 max 16384 2024-09-17T02:42:55.536 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:42:55.536 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:42:55.536 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:42:55.536 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5463, 2024-09-17T02:42:55.536 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:42:55.536 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:42:55.536 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:42:55.536 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 10.670091275000001, 2024-09-17T02:42:55.536 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 27, 2024-09-17T02:42:55.537 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 52, 2024-09-17T02:42:55.537 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 52, 2024-09-17T02:42:55.537 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 15.252016756 2024-09-17T02:42:55.537 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:42:55.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 -- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3424073200 msgr2=0x7f3424075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3424073200 0x7f3424075710 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f34340016c0 tx=0x7f34340195a0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 -- 172.21.15.131:0/3616455965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3444166a90 msgr2=0x7f3444153c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3444166a90 0x7f3444153c10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f342c011ae0 tx=0x7f342c009ea0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 -- 172.21.15.131:0/3616455965 shutdown_connections 2024-09-17T02:42:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3424073200 0x7f3424075710 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f34440c8710 0x7f3444154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3444166a90 0x7f3444153c10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 --2- 172.21.15.131:0/3616455965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3444154860 0x7f3444166550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 -- 172.21.15.131:0/3616455965 >> 172.21.15.131:0/3616455965 conn(0x7f344406eea0 msgr2=0x7f344406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 -- 172.21.15.131:0/3616455965 shutdown_connections 2024-09-17T02:42:55.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.527+0000 7f34494a1640 1 -- 172.21.15.131:0/3616455965 wait complete. 2024-09-17T02:42:55.551 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:42:55.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.691+0000 7f28347cb640 1 -- 172.21.15.131:0/1435631526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282c0ced00 msgr2=0x7f282c06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.691+0000 7f28347cb640 1 --2- 172.21.15.131:0/1435631526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282c0ced00 0x7f282c06dea0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f282c0b61b0 tx=0x7f281c01c8c0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.691+0000 7f28347cb640 1 -- 172.21.15.131:0/1435631526 shutdown_connections 2024-09-17T02:42:55.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.691+0000 7f28347cb640 1 --2- 172.21.15.131:0/1435631526 >> v2:172.21.15.189:3300/0 conn(0x7f282c06e3e0 0x7f282c15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.691+0000 7f28347cb640 1 --2- 172.21.15.131:0/1435631526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282c0ced00 0x7f282c06dea0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.691+0000 7f28347cb640 1 -- 172.21.15.131:0/1435631526 >> 172.21.15.131:0/1435631526 conn(0x7f282c06eea0 msgr2=0x7f282c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:55.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.691+0000 7f28347cb640 1 -- 172.21.15.131:0/1435631526 shutdown_connections 2024-09-17T02:42:55.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.691+0000 7f28347cb640 1 -- 172.21.15.131:0/1435631526 wait complete. 2024-09-17T02:42:55.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f28347cb640 1 Processor -- start 2024-09-17T02:42:55.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f28347cb640 1 -- start start 2024-09-17T02:42:55.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f28347cb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282c06e3e0 0x7f282c1516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f28347cb640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f282c0ced00 0x7f282c1528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f28347cb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f282c15c3e0 0x7f282c15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f28347cb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f282c0cdf80 con 0x7f282c06e3e0 2024-09-17T02:42:55.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f28347cb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f282c0cde00 con 0x7f282c15c3e0 2024-09-17T02:42:55.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f28347cb640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f282c0ce100 con 0x7f282c0ced00 2024-09-17T02:42:55.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2832567640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282c06e3e0 0x7f282c1516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2832567640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282c06e3e0 0x7f282c1516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57060/0 (socket says 172.21.15.131:57060) 2024-09-17T02:42:55.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2832567640 1 -- 172.21.15.131:0/2087147 learned_addr learned my addr 172.21.15.131:0/2087147 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:55.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2832d68640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f282c15c3e0 0x7f282c15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2831d66640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f282c0ced00 0x7f282c1528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2832567640 1 -- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f282c0ced00 msgr2=0x7f282c1528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2832567640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f282c0ced00 0x7f282c1528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2832567640 1 -- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f282c15c3e0 msgr2=0x7f282c15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2832567640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f282c15c3e0 0x7f282c15a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2832567640 1 -- 172.21.15.131:0/2087147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282c0aa080 con 0x7f282c06e3e0 2024-09-17T02:42:55.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2832567640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282c06e3e0 0x7f282c1516c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f281c0b0040 tx=0x7f281c0a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:55.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2822ffd640 1 -- 172.21.15.131:0/2087147 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c0bb070 con 0x7f282c06e3e0 2024-09-17T02:42:55.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2822ffd640 1 -- 172.21.15.131:0/2087147 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f281c004430 con 0x7f282c06e3e0 2024-09-17T02:42:55.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f28347cb640 1 -- 172.21.15.131:0/2087147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282c0a9ac0 con 0x7f282c06e3e0 2024-09-17T02:42:55.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.695+0000 7f2822ffd640 1 -- 172.21.15.131:0/2087147 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c0b2040 con 0x7f282c06e3e0 2024-09-17T02:42:55.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.699+0000 7f28347cb640 1 -- 172.21.15.131:0/2087147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282c1f8da0 con 0x7f282c06e3e0 2024-09-17T02:42:55.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.699+0000 7f28347cb640 1 -- 172.21.15.131:0/2087147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f282c1f8b60 con 0x7f282c06e3e0 2024-09-17T02:42:55.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.699+0000 7f2822ffd640 1 -- 172.21.15.131:0/2087147 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f281c005290 con 0x7f282c06e3e0 2024-09-17T02:42:55.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.699+0000 7f2822ffd640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2810073230 0x7f2810075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:55.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.699+0000 7f2822ffd640 1 -- 172.21.15.131:0/2087147 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f281c12db60 con 0x7f282c06e3e0 2024-09-17T02:42:55.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.699+0000 7f2831d66640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2810073230 0x7f2810075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:55.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.699+0000 7f2831d66640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2810073230 0x7f2810075740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28280058d0 tx=0x7f2828005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:55.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.703+0000 7f2822ffd640 1 -- 172.21.15.131:0/2087147 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f281c0164b0 con 0x7f282c06e3e0 2024-09-17T02:42:55.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.895+0000 7f28347cb640 1 -- 172.21.15.131:0/2087147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f282c06af50 con 0x7f282c06e3e0 2024-09-17T02:42:55.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.895+0000 7f2822ffd640 1 -- 172.21.15.131:0/2087147 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f281c0f8a80 con 0x7f282c06e3e0 2024-09-17T02:42:55.898 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:42:55.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.895+0000 7f28347cb640 1 -- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2810073230 msgr2=0x7f2810075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.895+0000 7f28347cb640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2810073230 0x7f2810075740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28280058d0 tx=0x7f2828005860 comp rx=0 tx=0).stop 2024-09-17T02:42:55.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.895+0000 7f28347cb640 1 -- 172.21.15.131:0/2087147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282c06e3e0 msgr2=0x7f282c1516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:55.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.895+0000 7f28347cb640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282c06e3e0 0x7f282c1516c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f281c0b0040 tx=0x7f281c0a57d0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.895+0000 7f28347cb640 1 -- 172.21.15.131:0/2087147 shutdown_connections 2024-09-17T02:42:55.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.895+0000 7f28347cb640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2810073230 0x7f2810075740 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.895+0000 7f28347cb640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f282c15c3e0 0x7f282c15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.899+0000 7f28347cb640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f282c0ced00 0x7f282c1528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.899+0000 7f28347cb640 1 --2- 172.21.15.131:0/2087147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282c06e3e0 0x7f282c1516c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:55.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.899+0000 7f28347cb640 1 -- 172.21.15.131:0/2087147 >> 172.21.15.131:0/2087147 conn(0x7f282c06eea0 msgr2=0x7f282c0cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:55.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.899+0000 7f28347cb640 1 -- 172.21.15.131:0/2087147 shutdown_connections 2024-09-17T02:42:55.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:55.899+0000 7f28347cb640 1 -- 172.21.15.131:0/2087147 wait complete. 2024-09-17T02:42:55.918 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:42:55.919 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:42:55.919 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:42:55.919 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:42:55.919 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:55.919 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:42:55.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:42:55.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:42:55.919 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:42:55.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:55.919 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:42:55.925 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:42:55.925 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:42:55.925 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:42:55.925 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:55.925 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:42:55.926 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42330sec preferred_lft 42330sec 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:42:55.927 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:55.928 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:42:55.928 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:55.928 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:42:55.928 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:55.928 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:42:55.928 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:42:55.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:55.928 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:42:55.977 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:42:55.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:55.978 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:42:56.028 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:42:56.029 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:42:56.029 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:42:56.029 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:42:56.029 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:42:56.029 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:42:56.029 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:42:56.029 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:42:56.029 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:42:56.029 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:42:56.029 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:42:56.030 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:42:56.030 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:42:56.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:42:56.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:42:56.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:42:56.030 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:42:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:42:56.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:42:56.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:56.149 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:42:56.198 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:42:56.198 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:42:56.198 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:42:56.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:56.199 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:42:56.253 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:42:56.253 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:42:56.253 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:56.301 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:42:56.302 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:42:56.302 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:56.349 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:42:56.406 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:42:56.454 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:42:56.454 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:42:56.454 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:42:56.454 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:42:56.454 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:42:56.454 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:42:56.454 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:42:56.454 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:42:56.454 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:42:56.455 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:42:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:42:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:42:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:42:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:42:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:42:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:42:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:56.456 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:56.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:56.457 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:42:56.505 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:42:56.505 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:42:56.550 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:42:56.565 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:42:56.613 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:42:56.613 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:42:56.613 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:42:56.614 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:42:56.615 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:42:56.615 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:42:56.615 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:42:56.615 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:42:56.615 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:42:56.615 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:42:56.615 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:42:56.615 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:42:56.615 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:42:56.615 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:42:56.616 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:42:56.616 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:42:56.616 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:42:56.616 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:56.616 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:56.616 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:56.616 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:56.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:56.617 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:42:56.661 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:42:56.655+0000 7f57c1716580 -1 init, newargv = 0x7f57b0004370 newargc=15 2024-09-17T02:42:56.661 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[40405]: starting ceph client 2024-09-17T02:42:56.666 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[40405]: starting fuse 2024-09-17T02:42:57.766 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:42:57.783 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:42:57.831 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:42:57.832 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:42:57.833 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:42:57.833 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:42:57.833 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:42:57.833 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:42:57.833 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:42:57.833 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:42:57.833 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:42:57.833 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:57.834 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:42:57.835 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:57.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:57.836 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:42:57.885 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:42:57.886 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:42:57.886 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:42:57.886 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:42:57.886 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:42:57.886 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:42:57.886 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:42:57.886 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:57.886 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:42:57.886 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:42:57.887 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:42:57.888 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:42:57.888 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:42:57.888 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:57.888 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:42:57.888 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:42:58.010 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.40405.asok 2024-09-17T02:42:58.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:42:58.022 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40405.asok 2024-09-17T02:42:58.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:58.022 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40405.asok status 2024-09-17T02:42:58.200 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:42:58.200 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "40405", 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5625, 2024-09-17T02:42:58.201 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 5625 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1081408616 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:42:58.202 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:42:58.203 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1081408616 2024-09-17T02:42:58.203 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:58.203 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.5625 192.168.0.1:0/1081408616", 2024-09-17T02:42:58.203 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/1081408616", 2024-09-17T02:42:58.203 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:42:58.203 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 27, 2024-09-17T02:42:58.203 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 52, 2024-09-17T02:42:58.203 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:42:58.203 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:42:58.203 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:42:58.203 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:42:58.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:58.216 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:42:58.217 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:42:58.223 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:58.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:58.223 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:42:58.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:58.279 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-09-17T02:42:58.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:58.344 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-09-17T02:42:58.401 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:42:58.401 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:42:58.401 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:42:58.401 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:42:58.402 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-09-17T02:42:58.402 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:42:58.511 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627777 2024-09-17T02:42:58.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:42:58.521 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:42:58.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 -- 172.21.15.131:0/1632606616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c0cdbb0 msgr2=0x7f865c071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:58.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 --2- 172.21.15.131:0/1632606616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c0cdbb0 0x7f865c071480 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f865c15df80 tx=0x7f865001c570 comp rx=0 tx=0).stop 2024-09-17T02:42:58.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 -- 172.21.15.131:0/1632606616 shutdown_connections 2024-09-17T02:42:58.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 --2- 172.21.15.131:0/1632606616 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c0cdbb0 0x7f865c071480 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:58.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 --2- 172.21.15.131:0/1632606616 >> v2:172.21.15.189:3300/0 conn(0x7f865c0719c0 0x7f865c15ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:58.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 --2- 172.21.15.131:0/1632606616 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f865c074500 0x7f865c15f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:58.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 -- 172.21.15.131:0/1632606616 >> 172.21.15.131:0/1632606616 conn(0x7f865c0d13d0 msgr2=0x7f865c0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:58.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 -- 172.21.15.131:0/1632606616 shutdown_connections 2024-09-17T02:42:58.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 -- 172.21.15.131:0/1632606616 wait complete. 2024-09-17T02:42:58.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 Processor -- start 2024-09-17T02:42:58.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 -- start start 2024-09-17T02:42:58.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f865c0719c0 0x7f865c166740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:58.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f865c074500 0x7f865c1fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:58.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c0cdbb0 0x7f865c15b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:58.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f865c0cfed0 con 0x7f865c0cdbb0 2024-09-17T02:42:58.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f865c0cfd50 con 0x7f865c074500 2024-09-17T02:42:58.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f866259b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f865c0d0050 con 0x7f865c0719c0 2024-09-17T02:42:58.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f8660b38640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c0cdbb0 0x7f865c15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:58.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f8660b38640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c0cdbb0 0x7f865c15b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57072/0 (socket says 172.21.15.131:57072) 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.723+0000 7f8660b38640 1 -- 172.21.15.131:0/3841342186 learned_addr learned my addr 172.21.15.131:0/3841342186 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f865b7fe640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f865c074500 0x7f865c1fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f865bfff640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f865c0719c0 0x7f865c166740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8660b38640 1 -- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f865c0719c0 msgr2=0x7f865c166740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8660b38640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f865c0719c0 0x7f865c166740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8660b38640 1 -- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f865c074500 msgr2=0x7f865c1fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8660b38640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f865c074500 0x7f865c1fbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8660b38640 1 -- 172.21.15.131:0/3841342186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865c06f270 con 0x7f865c0cdbb0 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f865b7fe640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f865c074500 0x7f865c1fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8660b38640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c0cdbb0 0x7f865c15b490 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f865400d630 tx=0x7f865400daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8658ff9640 1 -- 172.21.15.131:0/3841342186 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8654017070 con 0x7f865c0cdbb0 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8658ff9640 1 -- 172.21.15.131:0/3841342186 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8654005310 con 0x7f865c0cdbb0 2024-09-17T02:42:58.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8658ff9640 1 -- 172.21.15.131:0/3841342186 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86540047b0 con 0x7f865c0cdbb0 2024-09-17T02:42:58.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f866259b640 1 -- 172.21.15.131:0/3841342186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865c1fc780 con 0x7f865c0cdbb0 2024-09-17T02:42:58.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f866259b640 1 -- 172.21.15.131:0/3841342186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865c06f530 con 0x7f865c0cdbb0 2024-09-17T02:42:58.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f866259b640 1 -- 172.21.15.131:0/3841342186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865c0d0570 con 0x7f865c0cdbb0 2024-09-17T02:42:58.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8658ff9640 1 -- 172.21.15.131:0/3841342186 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8654004950 con 0x7f865c0cdbb0 2024-09-17T02:42:58.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8658ff9640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8648073200 0x7f8648075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:58.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f8658ff9640 1 -- 172.21.15.131:0/3841342186 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f8654094450 con 0x7f865c0cdbb0 2024-09-17T02:42:58.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.727+0000 7f865bfff640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8648073200 0x7f8648075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:58.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.731+0000 7f865bfff640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8648073200 0x7f8648075710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f864c009d30 tx=0x7f864c009710 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:58.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.731+0000 7f8658ff9640 1 -- 172.21.15.131:0/3841342186 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f865405f370 con 0x7f865c0cdbb0 2024-09-17T02:42:58.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.939+0000 7f866259b640 1 -- 172.21.15.131:0/3841342186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f865c15c520 con 0x7f865c0cdbb0 2024-09-17T02:42:58.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.939+0000 7f8658ff9640 1 -- 172.21.15.131:0/3841342186 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f865405ec70 con 0x7f865c0cdbb0 2024-09-17T02:42:58.943 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:58.943 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"btime":"2024-09-17T02:42:47:892534+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/434043926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":434043926},{"type":"v1","addr":"172.21.15.189:6836","nonce":434043926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5445,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1496962214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1496962214},{"type":"v1","addr":"172.21.15.131:6833","nonce":1496962214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2196282015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2196282015},{"type":"v1","addr":"172.21.15.131:6835","nonce":2196282015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:42:44.218540+0000","modified":"2024-09-17T02:42:47.307497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/4030750882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":4030750882},{"type":"v1","addr":"172.21.15.189:6837","nonce":4030750882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5463,"qdb_cluster":[5463]},"id":3}]} 2024-09-17T02:42:58.943 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-09-17T02:42:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 -- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8648073200 msgr2=0x7f8648075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8648073200 0x7f8648075710 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f864c009d30 tx=0x7f864c009710 comp rx=0 tx=0).stop 2024-09-17T02:42:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 -- 172.21.15.131:0/3841342186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c0cdbb0 msgr2=0x7f865c15b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c0cdbb0 0x7f865c15b490 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f865400d630 tx=0x7f865400daf0 comp rx=0 tx=0).stop 2024-09-17T02:42:58.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 -- 172.21.15.131:0/3841342186 shutdown_connections 2024-09-17T02:42:58.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8648073200 0x7f8648075710 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:58.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f865c0cdbb0 0x7f865c15b490 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:58.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f865c074500 0x7f865c1fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:58.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 --2- 172.21.15.131:0/3841342186 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f865c0719c0 0x7f865c166740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:58.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 -- 172.21.15.131:0/3841342186 >> 172.21.15.131:0/3841342186 conn(0x7f865c0d13d0 msgr2=0x7f865c0b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:58.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 -- 172.21.15.131:0/3841342186 shutdown_connections 2024-09-17T02:42:58.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:58.943+0000 7f866259b640 1 -- 172.21.15.131:0/3841342186 wait complete. 2024-09-17T02:42:58.965 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-09-17T02:42:59.156 INFO:teuthology.orchestra.run.smithi189.stdout:{"message":"","return_code":0} 2024-09-17T02:42:59.157 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T02:42:59.157 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-17T02:42:59.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.227+0000 7f9e28724080 1 Processor -- start 2024-09-17T02:42:59.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.227+0000 7f9e28724080 1 -- start start 2024-09-17T02:42:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e28724080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe72157780 0x55fe7221ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e28724080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fe72195c20 0x55fe72223340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e28724080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 0x55fe7219b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e28724080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fe7209ea00 con 0x55fe72197b70 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e28724080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fe720fcf40 con 0x55fe72157780 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e28724080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fe72193450 con 0x55fe72195c20 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e229fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 0x55fe7219b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e229fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 0x55fe7219b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57098/0 (socket says 172.21.15.131:57098) 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e229fc640 1 -- 172.21.15.131:0/3270127786 learned_addr learned my addr 172.21.15.131:0/3270127786 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e219fa640 1 --2- 172.21.15.131:0/3270127786 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fe72195c20 0x55fe72223340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e221fb640 1 --2- 172.21.15.131:0/3270127786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe72157780 0x55fe7221ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e229fc640 1 -- 172.21.15.131:0/3270127786 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fe72195c20 msgr2=0x55fe72223340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e229fc640 1 --2- 172.21.15.131:0/3270127786 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fe72195c20 0x55fe72223340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e229fc640 1 -- 172.21.15.131:0/3270127786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe72157780 msgr2=0x55fe7221ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e229fc640 1 --2- 172.21.15.131:0/3270127786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe72157780 0x55fe7221ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e229fc640 1 -- 172.21.15.131:0/3270127786 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe72077c10 con 0x55fe72197b70 2024-09-17T02:42:59.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.231+0000 7f9e229fc640 1 --2- 172.21.15.131:0/3270127786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 0x55fe7219b580 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400a380 tx=0x7f9e1400a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e584199ef14f8286 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e209f8640 1 -- 172.21.15.131:0/3270127786 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14010070 con 0x55fe72197b70 2024-09-17T02:42:59.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e209f8640 1 -- 172.21.15.131:0/3270127786 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e1400c9f0 con 0x55fe72197b70 2024-09-17T02:42:59.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e209f8640 1 -- 172.21.15.131:0/3270127786 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1400ce40 con 0x55fe72197b70 2024-09-17T02:42:59.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e28724080 1 -- 172.21.15.131:0/3270127786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 msgr2=0x55fe7219b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e28724080 1 --2- 172.21.15.131:0/3270127786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 0x55fe7219b580 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9e1400a380 tx=0x7f9e1400a840 comp rx=0 tx=0).stop 2024-09-17T02:42:59.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e28724080 1 -- 172.21.15.131:0/3270127786 shutdown_connections 2024-09-17T02:42:59.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e28724080 1 --2- 172.21.15.131:0/3270127786 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 0x55fe7219b580 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e28724080 1 --2- 172.21.15.131:0/3270127786 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fe72195c20 0x55fe72223340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e28724080 1 --2- 172.21.15.131:0/3270127786 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe72157780 0x55fe7221ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e28724080 1 -- 172.21.15.131:0/3270127786 >> 172.21.15.131:0/3270127786 conn(0x55fe7213cd00 msgr2=0x55fe72226ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e28724080 1 -- 172.21.15.131:0/3270127786 shutdown_connections 2024-09-17T02:42:59.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e28724080 1 -- 172.21.15.131:0/3270127786 wait complete. 2024-09-17T02:42:59.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e28724080 1 Processor -- start 2024-09-17T02:42:59.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.235+0000 7f9e28724080 1 -- start start 2024-09-17T02:42:59.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e28724080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe72157780 0x55fe7222ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e28724080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fe72195c20 0x55fe7221c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e28724080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 0x55fe72225200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e28724080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fe7221f4d0 con 0x55fe72197b70 2024-09-17T02:42:59.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e28724080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fe72227c60 con 0x55fe72157780 2024-09-17T02:42:59.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e28724080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fe72227ec0 con 0x55fe72195c20 2024-09-17T02:42:59.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e229fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 0x55fe72225200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e229fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 0x55fe72225200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57110/0 (socket says 172.21.15.131:57110) 2024-09-17T02:42:59.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e229fc640 1 -- 172.21.15.131:0/1773453716 learned_addr learned my addr 172.21.15.131:0/1773453716 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e221fb640 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe72157780 0x55fe7222ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e219fa640 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fe72195c20 0x55fe7221c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e229fc640 1 -- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fe72195c20 msgr2=0x55fe7221c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e229fc640 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fe72195c20 0x55fe7221c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e229fc640 1 -- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe72157780 msgr2=0x55fe7222ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e229fc640 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe72157780 0x55fe7222ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e229fc640 1 -- 172.21.15.131:0/1773453716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe7222c250 con 0x55fe72197b70 2024-09-17T02:42:59.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e221fb640 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fe72157780 0x55fe7222ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:42:59.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e229fc640 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 0x55fe72225200 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9e14015a50 tx=0x7f9e14015a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e1b7fe640 1 -- 172.21.15.131:0/1773453716 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e14010040 con 0x55fe72197b70 2024-09-17T02:42:59.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e1b7fe640 1 -- 172.21.15.131:0/1773453716 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e1400c700 con 0x55fe72197b70 2024-09-17T02:42:59.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e28724080 1 -- 172.21.15.131:0/1773453716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fe7222c4f0 con 0x55fe72197b70 2024-09-17T02:42:59.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e1b7fe640 1 -- 172.21.15.131:0/1773453716 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1400ce60 con 0x55fe72197b70 2024-09-17T02:42:59.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e28724080 1 -- 172.21.15.131:0/1773453716 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe72229610 con 0x55fe72197b70 2024-09-17T02:42:59.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e1b7fe640 1 -- 172.21.15.131:0/1773453716 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9e14013070 con 0x55fe72197b70 2024-09-17T02:42:59.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e1b7fe640 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9e04071be0 0x7f9e040740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.239+0000 7f9e1b7fe640 1 -- 172.21.15.131:0/1773453716 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f9e14094a00 con 0x55fe72197b70 2024-09-17T02:42:59.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55fe72229ab0 0x55fe72229ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e221fb640 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9e04071be0 0x7f9e040740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 -- 172.21.15.131:0/1773453716 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52) -- 0x55fe7222a400 con 0x55fe72229ab0 2024-09-17T02:42:59.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e219fa640 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55fe72229ab0 0x55fe72229ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e221fb640 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9e04071be0 0x7f9e040740f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0ad000 tx=0x7f9e1c019cd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e219fa640 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55fe72229ab0 0x55fe72229ec0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e219fa640 1 -- 172.21.15.131:0/1773453716 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f9e0c00b6e0 con 0x55fe72229ab0 2024-09-17T02:42:59.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 -- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55fe72229ab0 msgr2=0x55fe72229ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55fe72229ab0 0x55fe72229ec0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 -- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9e04071be0 msgr2=0x7f9e040740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9e04071be0 0x7f9e040740f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c0ad000 tx=0x7f9e1c019cd0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 -- 172.21.15.131:0/1773453716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 msgr2=0x55fe72225200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 --2- 172.21.15.131:0/1773453716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fe72197b70 0x55fe72225200 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9e14015a50 tx=0x7f9e14015a80 comp rx=0 tx=0).stop 2024-09-17T02:42:59.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e229fc640 1 -- 172.21.15.131:0/1773453716 reap_dead start 2024-09-17T02:42:59.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 -- 172.21.15.131:0/1773453716 shutdown_connections 2024-09-17T02:42:59.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 -- 172.21.15.131:0/1773453716 >> 172.21.15.131:0/1773453716 conn(0x55fe7213cd00 msgr2=0x55fe7219b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 -- 172.21.15.131:0/1773453716 shutdown_connections 2024-09-17T02:42:59.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.243+0000 7f9e28724080 1 -- 172.21.15.131:0/1773453716 wait complete. 2024-09-17T02:42:59.252 INFO:teuthology.orchestra.run.smithi131.stdout:Yalphaparent_a 2024-09-17T02:42:59.252 DEBUG:teuthology.orchestra.run.smithi131:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-17T02:42:59.317 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:42:59.317 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 1099511627777, 2024-09-17T02:42:59.317 INFO:teuthology.orchestra.run.smithi131.stdout: "ancestors": [ 2024-09-17T02:42:59.318 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:59.318 INFO:teuthology.orchestra.run.smithi131.stdout: "dirino": 1099511627776, 2024-09-17T02:42:59.318 INFO:teuthology.orchestra.run.smithi131.stdout: "dname": "alpha", 2024-09-17T02:42:59.318 INFO:teuthology.orchestra.run.smithi131.stdout: "version": 2 2024-09-17T02:42:59.318 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:59.318 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:42:59.318 INFO:teuthology.orchestra.run.smithi131.stdout: "dirino": 1, 2024-09-17T02:42:59.319 INFO:teuthology.orchestra.run.smithi131.stdout: "dname": "parent_a", 2024-09-17T02:42:59.319 INFO:teuthology.orchestra.run.smithi131.stdout: "version": 4 2024-09-17T02:42:59.319 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:59.319 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-17T02:42:59.319 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 8, 2024-09-17T02:42:59.319 INFO:teuthology.orchestra.run.smithi131.stdout: "old_pools": [] 2024-09-17T02:42:59.319 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:42:59.319 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:42:59.322 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-09-17T02:42:59.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f0855c88080 1 Processor -- start 2024-09-17T02:42:59.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f0855c88080 1 -- start start 2024-09-17T02:42:59.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f0855c88080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56378175d780 0x5637818251e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f0855c88080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 0x56378182dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f0855c88080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56378179bc20 0x56378182b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f0855c88080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5637816a4a00 con 0x563781799b90 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f0855c88080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563781702f40 con 0x56378175d780 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f0855c88080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5637817996f0 con 0x56378179bc20 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084ef5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 0x56378182dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084ef5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 0x56378182dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57124/0 (socket says 172.21.15.131:57124) 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084ef5e640 1 -- 172.21.15.131:0/1849553718 learned_addr learned my addr 172.21.15.131:0/1849553718 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084ff60640 1 --2- 172.21.15.131:0/1849553718 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56378179bc20 0x56378182b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084f75f640 1 --2- 172.21.15.131:0/1849553718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56378175d780 0x5637818251e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084ef5e640 1 -- 172.21.15.131:0/1849553718 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56378179bc20 msgr2=0x56378182b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084ef5e640 1 --2- 172.21.15.131:0/1849553718 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56378179bc20 0x56378182b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084ef5e640 1 -- 172.21.15.131:0/1849553718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56378175d780 msgr2=0x5637818251e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084ef5e640 1 --2- 172.21.15.131:0/1849553718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56378175d780 0x5637818251e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084ef5e640 1 -- 172.21.15.131:0/1849553718 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56378167dc10 con 0x563781799b90 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084ef5e640 1 --2- 172.21.15.131:0/1849553718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 0x56378182dc20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f083800b220 tx=0x7f083800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eda369995070501 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084df5c640 1 -- 172.21.15.131:0/1849553718 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083800a040 con 0x563781799b90 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084df5c640 1 -- 172.21.15.131:0/1849553718 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f083800f620 con 0x563781799b90 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.431+0000 7f084df5c640 1 -- 172.21.15.131:0/1849553718 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083800fa70 con 0x563781799b90 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 -- 172.21.15.131:0/1849553718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 msgr2=0x56378182dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 --2- 172.21.15.131:0/1849553718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 0x56378182dc20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f083800b220 tx=0x7f083800b6e0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 -- 172.21.15.131:0/1849553718 shutdown_connections 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 --2- 172.21.15.131:0/1849553718 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56378179bc20 0x56378182b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 --2- 172.21.15.131:0/1849553718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 0x56378182dc20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 --2- 172.21.15.131:0/1849553718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56378175d780 0x5637818251e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 -- 172.21.15.131:0/1849553718 >> 172.21.15.131:0/1849553718 conn(0x563781742d00 msgr2=0x563781754980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 -- 172.21.15.131:0/1849553718 shutdown_connections 2024-09-17T02:42:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 -- 172.21.15.131:0/1849553718 wait complete. 2024-09-17T02:42:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 Processor -- start 2024-09-17T02:42:59.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 -- start start 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56378175d780 0x563781812510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 0x56378182f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56378179bc20 0x56378182f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56378182ce50 con 0x563781799b90 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56378182af10 con 0x56378179bc20 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56378182d0f0 con 0x56378175d780 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084ef5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 0x56378182f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084ef5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 0x56378182f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57130/0 (socket says 172.21.15.131:57130) 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084ef5e640 1 -- 172.21.15.131:0/610031626 learned_addr learned my addr 172.21.15.131:0/610031626 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084f75f640 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56378175d780 0x563781812510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084ef5e640 1 -- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56378175d780 msgr2=0x563781812510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084ef5e640 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56378175d780 0x563781812510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084ff60640 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56378179bc20 0x56378182f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084ef5e640 1 -- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56378179bc20 msgr2=0x56378182f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084ef5e640 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56378179bc20 0x56378182f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084ef5e640 1 -- 172.21.15.131:0/610031626 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637818301c0 con 0x563781799b90 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084f75f640 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56378175d780 0x563781812510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084ef5e640 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 0x56378182f3a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0838019630 tx=0x7f0838019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084cf5a640 1 -- 172.21.15.131:0/610031626 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083801f070 con 0x563781799b90 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084cf5a640 1 -- 172.21.15.131:0/610031626 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0838019a90 con 0x563781799b90 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 -- 172.21.15.131:0/610031626 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563781834e60 con 0x563781799b90 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f084cf5a640 1 -- 172.21.15.131:0/610031626 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083801b430 con 0x563781799b90 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.435+0000 7f0855c88080 1 -- 172.21.15.131:0/610031626 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563781824060 con 0x563781799b90 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.439+0000 7f084cf5a640 1 -- 172.21.15.131:0/610031626 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0838022070 con 0x563781799b90 2024-09-17T02:42:59.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.439+0000 7f084cf5a640 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0830071be0 0x7f08300740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.439+0000 7f084cf5a640 1 -- 172.21.15.131:0/610031626 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f0838094730 con 0x563781799b90 2024-09-17T02:42:59.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.439+0000 7f0855c88080 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x5637818365b0 0x5637818d2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.439+0000 7f0855c88080 1 -- 172.21.15.131:0/610031626 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) -- 0x5637818d2ad0 con 0x5637818365b0 2024-09-17T02:42:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.439+0000 7f084f75f640 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0830071be0 0x7f08300740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.439+0000 7f084ff60640 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x5637818365b0 0x5637818d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.439+0000 7f084f75f640 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0830071be0 0x7f08300740f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0848004fd0 tx=0x7f0848019d10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.439+0000 7f084ff60640 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x5637818365b0 0x5637818d2590 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f084ff60640 1 -- 172.21.15.131:0/610031626 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v52'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f084000b5f0 con 0x5637818365b0 2024-09-17T02:42:59.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f0855c88080 1 -- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x5637818365b0 msgr2=0x5637818d2590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f0855c88080 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x5637818365b0 0x5637818d2590 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f0855c88080 1 -- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0830071be0 msgr2=0x7f08300740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f0855c88080 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0830071be0 0x7f08300740f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f0848004fd0 tx=0x7f0848019d10 comp rx=0 tx=0).stop 2024-09-17T02:42:59.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f0855c88080 1 -- 172.21.15.131:0/610031626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 msgr2=0x56378182f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f0855c88080 1 --2- 172.21.15.131:0/610031626 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563781799b90 0x56378182f3a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0838019630 tx=0x7f0838019660 comp rx=0 tx=0).stop 2024-09-17T02:42:59.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f084ff60640 1 -- 172.21.15.131:0/610031626 reap_dead start 2024-09-17T02:42:59.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f0855c88080 1 -- 172.21.15.131:0/610031626 shutdown_connections 2024-09-17T02:42:59.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f0855c88080 1 -- 172.21.15.131:0/610031626 >> 172.21.15.131:0/610031626 conn(0x563781742d00 msgr2=0x563781754840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f0855c88080 1 -- 172.21.15.131:0/610031626 shutdown_connections 2024-09-17T02:42:59.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.443+0000 7f0855c88080 1 -- 172.21.15.131:0/610031626 wait complete. 2024-09-17T02:42:59.453 DEBUG:teuthology.orchestra.run.smithi131:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-17T02:42:59.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:59.455 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-09-17T02:42:59.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.607+0000 7f3d8872a640 1 -- 172.21.15.131:0/1670001240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d80069770 msgr2=0x7f3d8006dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.607+0000 7f3d8872a640 1 --2- 172.21.15.131:0/1670001240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d80069770 0x7f3d8006dfa0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3d80155500 tx=0x7f3d7001c830 comp rx=0 tx=0).stop 2024-09-17T02:42:59.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.607+0000 7f3d8872a640 1 -- 172.21.15.131:0/1670001240 shutdown_connections 2024-09-17T02:42:59.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.607+0000 7f3d8872a640 1 --2- 172.21.15.131:0/1670001240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d80069770 0x7f3d8006dfa0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.607+0000 7f3d8872a640 1 --2- 172.21.15.131:0/1670001240 >> v2:172.21.15.189:3300/0 conn(0x7f3d80069b80 0x7f3d8006e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.607+0000 7f3d8872a640 1 --2- 172.21.15.131:0/1670001240 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d8015b9c0 0x7f3d800b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.607+0000 7f3d8872a640 1 -- 172.21.15.131:0/1670001240 >> 172.21.15.131:0/1670001240 conn(0x7f3d800d07c0 msgr2=0x7f3d800d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.607+0000 7f3d8872a640 1 -- 172.21.15.131:0/1670001240 shutdown_connections 2024-09-17T02:42:59.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.607+0000 7f3d8872a640 1 -- 172.21.15.131:0/1670001240 wait complete. 2024-09-17T02:42:59.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.607+0000 7f3d8872a640 1 Processor -- start 2024-09-17T02:42:59.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.607+0000 7f3d8872a640 1 -- start start 2024-09-17T02:42:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d8872a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d80069770 0x7f3d80149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d8872a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d80069b80 0x7f3d801f8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d8872a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d8015b9c0 0x7f3d801f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d8872a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8015e8c0 con 0x7f3d80069b80 2024-09-17T02:42:59.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d8872a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8015e740 con 0x7f3d80069770 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d8872a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8015ea40 con 0x7f3d8015b9c0 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d85cc5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d80069b80 0x7f3d801f8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d85cc5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d80069b80 0x7f3d801f8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57144/0 (socket says 172.21.15.131:57144) 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d85cc5640 1 -- 172.21.15.131:0/2707331981 learned_addr learned my addr 172.21.15.131:0/2707331981 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d86cc7640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d8015b9c0 0x7f3d801f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d864c6640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d80069770 0x7f3d80149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d85cc5640 1 -- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d8015b9c0 msgr2=0x7f3d801f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d8015b9c0 0x7f3d801f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d85cc5640 1 -- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d80069770 msgr2=0x7f3d80149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d80069770 0x7f3d80149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d85cc5640 1 -- 172.21.15.131:0/2707331981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d800aea20 con 0x7f3d80069b80 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d80069b80 0x7f3d801f8810 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3d7001aee0 tx=0x7f3d700a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d76ffd640 1 -- 172.21.15.131:0/2707331981 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d700ba020 con 0x7f3d80069b80 2024-09-17T02:42:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d76ffd640 1 -- 172.21.15.131:0/2707331981 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d700043b0 con 0x7f3d80069b80 2024-09-17T02:42:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d8872a640 1 -- 172.21.15.131:0/2707331981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d800aed00 con 0x7f3d80069b80 2024-09-17T02:42:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d76ffd640 1 -- 172.21.15.131:0/2707331981 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d700b2040 con 0x7f3d80069b80 2024-09-17T02:42:59.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.611+0000 7f3d8872a640 1 -- 172.21.15.131:0/2707331981 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8014a740 con 0x7f3d80069b80 2024-09-17T02:42:59.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d76ffd640 1 -- 172.21.15.131:0/2707331981 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3d700052d0 con 0x7f3d80069b80 2024-09-17T02:42:59.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d76ffd640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3d64073200 0x7f3d64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d76ffd640 1 -- 172.21.15.131:0/2707331981 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f3d7012e330 con 0x7f3d80069b80 2024-09-17T02:42:59.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d864c6640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3d64073200 0x7f3d64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d8f9f5000 1 Processor -- start 2024-09-17T02:42:59.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d8f9f5000 1 -- start start 2024-09-17T02:42:59.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d864c6640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3d64073200 0x7f3d64075710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005c50 tx=0x7f3d7c005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c7b11a0 0x55f18c8fa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7a9ae0 0x55f18c8fab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c793970 0x55f18c8fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f18c6bfbd0 con 0x55f18c7b11a0 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f18c6b0430 con 0x55f18c7a9ae0 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f18c70dce0 con 0x55f18c793970 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d864c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c7b11a0 0x55f18c8fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d864c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c7b11a0 0x55f18c8fa640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57160/0 (socket says 172.21.15.131:57160) 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d864c6640 1 -- 172.21.15.131:0/3013235472 learned_addr learned my addr 172.21.15.131:0/3013235472 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d86cc7640 1 --2- 172.21.15.131:0/3013235472 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c793970 0x55f18c8fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/3013235472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7a9ae0 0x55f18c8fab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d864c6640 1 -- 172.21.15.131:0/3013235472 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c793970 msgr2=0x55f18c8fb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d864c6640 1 --2- 172.21.15.131:0/3013235472 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c793970 0x55f18c8fb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d864c6640 1 -- 172.21.15.131:0/3013235472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7a9ae0 msgr2=0x55f18c8fab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.615+0000 7f3d864c6640 1 --2- 172.21.15.131:0/3013235472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7a9ae0 0x55f18c8fab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d864c6640 1 -- 172.21.15.131:0/3013235472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18c6b1d80 con 0x55f18c7b11a0 2024-09-17T02:42:59.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 -- 172.21.15.131:0/343385826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a500b7a40 msgr2=0x7f8a500b7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 --2- 172.21.15.131:0/343385826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a500b7a40 0x7f8a500b7e50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8a500d3400 tx=0x7f8a4801ca40 comp rx=0 tx=0).stop 2024-09-17T02:42:59.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 -- 172.21.15.131:0/343385826 shutdown_connections 2024-09-17T02:42:59.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 --2- 172.21.15.131:0/343385826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a500b7a40 0x7f8a500b7e50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 --2- 172.21.15.131:0/343385826 >> v2:172.21.15.189:3300/0 conn(0x7f8a500cf230 0x7f8a500cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 -- 172.21.15.131:0/343385826 >> 172.21.15.131:0/343385826 conn(0x7f8a500d43f0 msgr2=0x7f8a500d4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 -- 172.21.15.131:0/343385826 shutdown_connections 2024-09-17T02:42:59.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 -- 172.21.15.131:0/343385826 wait complete. 2024-09-17T02:42:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d864c6640 1 --2- 172.21.15.131:0/3013235472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c7b11a0 0x55f18c8fa640 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c012550 tx=0x7f3d7c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1cc0f6d70f917b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d63fff640 1 -- 172.21.15.131:0/3013235472 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c023070 con 0x55f18c7b11a0 2024-09-17T02:42:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d63fff640 1 -- 172.21.15.131:0/3013235472 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d7c011790 con 0x55f18c7b11a0 2024-09-17T02:42:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d63fff640 1 -- 172.21.15.131:0/3013235472 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c011b60 con 0x55f18c7b11a0 2024-09-17T02:42:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/3013235472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c7b11a0 msgr2=0x55f18c8fa640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/3013235472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c7b11a0 0x55f18c8fa640 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c012550 tx=0x7f3d7c013000 comp rx=0 tx=0).stop 2024-09-17T02:42:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/3013235472 shutdown_connections 2024-09-17T02:42:59.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/3013235472 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c793970 0x55f18c8fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/3013235472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7a9ae0 0x55f18c8fab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/3013235472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c7b11a0 0x55f18c8fa640 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/3013235472 >> 172.21.15.131:0/3013235472 conn(0x55f18c7b78c0 msgr2=0x55f18c8af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/3013235472 shutdown_connections 2024-09-17T02:42:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/3013235472 wait complete. 2024-09-17T02:42:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 Processor -- start 2024-09-17T02:42:59.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 -- start start 2024-09-17T02:42:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d8f9f5000 1 -- start start 2024-09-17T02:42:59.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f3d8f9f5000 1 Processor -- start 2024-09-17T02:42:59.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a500b7a40 0x7f8a50153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a500cf230 0x7f8a5015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8a50161be0 0x7f8a5015cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5016bf20 con 0x7f8a500cf230 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5016bda0 con 0x7f8a500b7a40 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a54e97640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8a5016c0a0 con 0x7f8a50161be0 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a4dd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a500cf230 0x7f8a5015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a4dd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a500cf230 0x7f8a5015aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57172/0 (socket says 172.21.15.131:57172) 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a4dd9b640 1 -- 172.21.15.131:0/2439293179 learned_addr learned my addr 172.21.15.131:0/2439293179 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a4ed9d640 1 --2- 172.21.15.131:0/2439293179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8a50161be0 0x7f8a5015cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a4e59c640 1 --2- 172.21.15.131:0/2439293179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a500b7a40 0x7f8a50153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a4dd9b640 1 -- 172.21.15.131:0/2439293179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8a50161be0 msgr2=0x7f8a5015cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a4dd9b640 1 --2- 172.21.15.131:0/2439293179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8a50161be0 0x7f8a5015cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a4dd9b640 1 -- 172.21.15.131:0/2439293179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a500b7a40 msgr2=0x7f8a50153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a4dd9b640 1 --2- 172.21.15.131:0/2439293179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a500b7a40 0x7f8a50153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.619+0000 7f8a4dd9b640 1 -- 172.21.15.131:0/2439293179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a50141310 con 0x7f8a500cf230 2024-09-17T02:42:59.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8b65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c7a9ae0 0x55f18c8b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7b11a0 0x55f18c8e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f18c700f20 con 0x55f18c793970 2024-09-17T02:42:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f18c6fd110 con 0x55f18c7b11a0 2024-09-17T02:42:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f18c6a26d0 con 0x55f18c7a9ae0 2024-09-17T02:42:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 10 client.0 _ll_register_callbacks cb 0x55f18c6a1330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:42:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d864c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8b65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d864c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8b65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57178/0 (socket says 172.21.15.131:57178) 2024-09-17T02:42:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d864c6640 1 -- 172.21.15.131:0/2869002067 learned_addr learned my addr 172.21.15.131:0/2869002067 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a4dd9b640 1 --2- 172.21.15.131:0/2439293179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a500cf230 0x7f8a5015aba0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8a4400d750 tx=0x7f8a4400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d86cc7640 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7b11a0 0x55f18c8e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c7a9ae0 0x55f18c8b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a3effd640 1 -- 172.21.15.131:0/2439293179 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a44017070 con 0x7f8a500cf230 2024-09-17T02:42:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a3effd640 1 -- 172.21.15.131:0/2439293179 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a44004310 con 0x7f8a500cf230 2024-09-17T02:42:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a54e97640 1 -- 172.21.15.131:0/2439293179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a501415f0 con 0x7f8a500cf230 2024-09-17T02:42:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d864c6640 1 -- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c7a9ae0 msgr2=0x55f18c8b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d864c6640 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c7a9ae0 0x55f18c8b6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d864c6640 1 -- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7b11a0 msgr2=0x55f18c8e5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d864c6640 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7b11a0 0x55f18c8e5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d864c6640 1 -- 172.21.15.131:0/2869002067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18c69b310 con 0x55f18c793970 2024-09-17T02:42:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a3effd640 1 -- 172.21.15.131:0/2439293179 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a4400de60 con 0x7f8a500cf230 2024-09-17T02:42:59.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a54e97640 1 -- 172.21.15.131:0/2439293179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a50141bb0 con 0x7f8a500cf230 2024-09-17T02:42:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d864c6640 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8b65e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c010bd0 tx=0x7f3d7c0118e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48cc0585f1ce7c01 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d617fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:42:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d617fa640 1 -- 172.21.15.131:0/2869002067 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c023040 con 0x55f18c793970 2024-09-17T02:42:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d617fa640 1 -- 172.21.15.131:0/2869002067 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f3d7c011ce0 con 0x55f18c793970 2024-09-17T02:42:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/2869002067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f18c695450 con 0x55f18c793970 2024-09-17T02:42:59.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a54e97640 1 -- 172.21.15.131:0/2439293179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f8a500b7e50 con 0x7f8a500cf230 2024-09-17T02:42:59.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d617fa640 1 -- 172.21.15.131:0/2869002067 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d7c01f450 con 0x55f18c793970 2024-09-17T02:42:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 10 client.5667 fetch_fsmap learned FSMap version 27 2024-09-17T02:42:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/2869002067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55f18c6b8f60 con 0x55f18c793970 2024-09-17T02:42:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d617fa640 1 -- 172.21.15.131:0/2869002067 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f18c695450 con 0x55f18c793970 2024-09-17T02:42:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d617fa640 1 -- 172.21.15.131:0/2869002067 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f3d7c00f040 con 0x55f18c793970 2024-09-17T02:42:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 10 client.5667 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T02:42:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 10 client.5667 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-17T02:42:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 20 client.5667 populate_metadata read hostname 'smithi131' 2024-09-17T02:42:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55f18c39fb80 0x55f18c9018c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a3effd640 1 -- 172.21.15.131:0/2439293179 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8a44009c30 con 0x7f8a500cf230 2024-09-17T02:42:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 4 client.5667 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:42:59.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/2869002067 --> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f18c70fc30 con 0x55f18c39fb80 2024-09-17T02:42:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55f18c39fb80 0x55f18c9018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a3effd640 1 --2- 172.21.15.131:0/2439293179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8a30073200 0x7f8a30075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a3effd640 1 -- 172.21.15.131:0/2439293179 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f8a44094470 con 0x7f8a500cf230 2024-09-17T02:42:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a3effd640 1 -- 172.21.15.131:0/2439293179 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+498 (secure 0 0 0) 0x7f8a44096af0 con 0x7f8a500cf230 2024-09-17T02:42:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a4e59c640 1 --2- 172.21.15.131:0/2439293179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8a30073200 0x7f8a30075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f8a54e97640 1 -- 172.21.15.131:0/2439293179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f8a50160090 con 0x7f8a500cf230 2024-09-17T02:42:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55f18c39fb80 0x55f18c9018c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bd63d6b4d5c3231 server_cookie=a7913454bdc316ff in_seq=0 out_seq=0 2024-09-17T02:42:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.623+0000 7f3d617fa640 10 client.5667 ms_handle_connect on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f8a4e59c640 1 --2- 172.21.15.131:0/2439293179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8a30073200 0x7f8a30075710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8a40000f70 tx=0x7f8a40009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d617fa640 1 -- 172.21.15.131:0/2869002067 <== mds.0 v2:172.21.15.189:6835/4030750882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f18c70fc30 con 0x55f18c39fb80 2024-09-17T02:42:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d617fa640 10 client.5667 handle_command_reply: tid=0 2024-09-17T02:42:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 1 client.5667 shutdown 2024-09-17T02:42:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/2869002067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 msgr2=0x55f18c8b65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8b65e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c010bd0 tx=0x7f3d7c0118e0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/2869002067 shutdown_connections 2024-09-17T02:42:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55f18c39fb80 0x55f18c9018c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7b11a0 0x55f18c8e5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d617fa640 0 client.5667 ms_handle_reset on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c7a9ae0 0x55f18c8b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/2869002067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8b65e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/2869002067 >> 172.21.15.131:0/2869002067 conn(0x55f18c7b78c0 msgr2=0x55f18c8afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/2869002067 shutdown_connections 2024-09-17T02:42:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/2869002067 wait complete. 2024-09-17T02:42:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.627+0000 7f3d8f9f5000 20 client.5667 trim_cache size 0 max 16384 2024-09-17T02:42:59.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.643+0000 7f3d8f9f5000 1 Processor -- start 2024-09-17T02:42:59.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.643+0000 7f3d8f9f5000 1 -- start start 2024-09-17T02:42:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c39fb80 0x55f18c8aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7a9ae0 0x55f18c8af570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80205990 con 0x55f18c793970 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3d802049f0 con 0x55f18c7a9ae0 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3d800ae860 con 0x55f18c39fb80 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d85cc5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d85cc5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57196/0 (socket says 172.21.15.131:57196) 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d85cc5640 1 -- 172.21.15.131:0/4036563758 learned_addr learned my addr 172.21.15.131:0/4036563758 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d85cc5640 1 -- 172.21.15.131:0/4036563758 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c39fb80 msgr2=0x55f18c8aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/4036563758 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c39fb80 0x55f18c8aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d85cc5640 1 -- 172.21.15.131:0/4036563758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7a9ae0 msgr2=0x55f18c8af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/4036563758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7a9ae0 0x55f18c8af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d85cc5640 1 -- 172.21.15.131:0/4036563758 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18c781f30 con 0x55f18c793970 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/4036563758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8af030 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3d700f8af0 tx=0x7f3d7012c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cf2ac038c6af86 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d637fe640 1 -- 172.21.15.131:0/4036563758 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d700f8670 con 0x55f18c793970 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d637fe640 1 -- 172.21.15.131:0/4036563758 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d700f9260 con 0x55f18c793970 2024-09-17T02:42:59.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d637fe640 1 -- 172.21.15.131:0/4036563758 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d700f8b60 con 0x55f18c793970 2024-09-17T02:42:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036563758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 msgr2=0x55f18c8af030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/4036563758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8af030 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3d700f8af0 tx=0x7f3d7012c630 comp rx=0 tx=0).stop 2024-09-17T02:42:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036563758 shutdown_connections 2024-09-17T02:42:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/4036563758 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c7a9ae0 0x55f18c8af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/4036563758 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c793970 0x55f18c8af030 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/4036563758 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c39fb80 0x55f18c8aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036563758 >> 172.21.15.131:0/4036563758 conn(0x55f18c7b78c0 msgr2=0x55f18c8f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036563758 shutdown_connections 2024-09-17T02:42:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036563758 wait complete. 2024-09-17T02:42:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 -- start start 2024-09-17T02:42:59.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.647+0000 7f3d8f9f5000 1 Processor -- start 2024-09-17T02:42:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c39fb80 0x55f18c96fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c793970 0x55f18c9703c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c7a9ae0 0x55f18c970900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f18c6bfbd0 con 0x55f18c39fb80 2024-09-17T02:42:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f18c6fd110 con 0x55f18c793970 2024-09-17T02:42:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f18c781aa0 con 0x55f18c7a9ae0 2024-09-17T02:42:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 10 client.0 _ll_register_callbacks cb 0x7f3d7c012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:42:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d864c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c39fb80 0x55f18c96fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d864c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c39fb80 0x55f18c96fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57208/0 (socket says 172.21.15.131:57208) 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d864c6640 1 -- 172.21.15.131:0/4036284756 learned_addr learned my addr 172.21.15.131:0/4036284756 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c793970 0x55f18c9703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d864c6640 1 -- 172.21.15.131:0/4036284756 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c7a9ae0 msgr2=0x55f18c970900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d864c6640 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c7a9ae0 0x55f18c970900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d864c6640 1 -- 172.21.15.131:0/4036284756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c793970 msgr2=0x55f18c9703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d864c6640 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c793970 0x55f18c9703c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d864c6640 1 -- 172.21.15.131:0/4036284756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18c758dd0 con 0x55f18c39fb80 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d864c6640 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c39fb80 0x55f18c96fe80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c01f680 tx=0x7f3d7c009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3aa96e17ab02ad server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d617fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d617fa640 1 -- 172.21.15.131:0/4036284756 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7c02c070 con 0x55f18c39fb80 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d617fa640 1 -- 172.21.15.131:0/4036284756 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f3d7c02e040 con 0x55f18c39fb80 2024-09-17T02:42:59.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036284756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d700f9260 con 0x55f18c39fb80 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d617fa640 1 -- 172.21.15.131:0/4036284756 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3d7c048050 con 0x55f18c39fb80 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 10 client.5673 fetch_fsmap learned FSMap version 27 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036284756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55f18c693840 con 0x55f18c39fb80 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d617fa640 1 -- 172.21.15.131:0/4036284756 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d700f9260 con 0x55f18c39fb80 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d617fa640 1 -- 172.21.15.131:0/4036284756 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f3d7c04b020 con 0x55f18c39fb80 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 10 client.5673 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 10 client.5673 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 20 client.5673 populate_metadata read hostname 'smithi131' 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55f18c3b9230 0x55f18c8b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 4 client.5673 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036284756 --> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55f18c70b6e0 con 0x55f18c3b9230 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55f18c3b9230 0x55f18c8b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d617fa640 10 client.5673 ms_handle_connect on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:59.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.651+0000 7f3d85cc5640 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55f18c3b9230 0x55f18c8b1ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a4216cf68e7f62 server_cookie=3321b02684e15660 in_seq=0 out_seq=0 2024-09-17T02:42:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d617fa640 1 -- 172.21.15.131:0/4036284756 <== mds.0 v2:172.21.15.189:6835/4030750882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f18c70b6e0 con 0x55f18c3b9230 2024-09-17T02:42:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d617fa640 10 client.5673 handle_command_reply: tid=0 2024-09-17T02:42:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 1 client.5673 shutdown 2024-09-17T02:42:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036284756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c39fb80 msgr2=0x55f18c96fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c39fb80 0x55f18c96fe80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c01f680 tx=0x7f3d7c009bf0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036284756 shutdown_connections 2024-09-17T02:42:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55f18c3b9230 0x55f18c8b1ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f18c7a9ae0 0x55f18c970900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f18c793970 0x55f18c9703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d617fa640 0 client.5673 ms_handle_reset on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 1 --2- 172.21.15.131:0/4036284756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f18c39fb80 0x55f18c96fe80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036284756 >> 172.21.15.131:0/4036284756 conn(0x55f18c7b78c0 msgr2=0x55f18c8f9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036284756 shutdown_connections 2024-09-17T02:42:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 1 -- 172.21.15.131:0/4036284756 wait complete. 2024-09-17T02:42:59.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8f9f5000 20 client.5673 trim_cache size 0 max 16384 2024-09-17T02:42:59.661 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:42:59.661 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-09-17T02:42:59.661 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "87131ed3-d4e1-440a-b0c9-3ff18ff3d0d0", 2024-09-17T02:42:59.661 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-09-17T02:42:59.662 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:42:59.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 -- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3d64073200 msgr2=0x7f3d64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3d64073200 0x7f3d64075710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005c50 tx=0x7f3d7c005be0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 -- 172.21.15.131:0/2707331981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d80069b80 msgr2=0x7f3d801f8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d80069b80 0x7f3d801f8810 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3d7001aee0 tx=0x7f3d700a5620 comp rx=0 tx=0).stop 2024-09-17T02:42:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 -- 172.21.15.131:0/2707331981 shutdown_connections 2024-09-17T02:42:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3d64073200 0x7f3d64075710 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3d8015b9c0 0x7f3d801f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3d80069b80 0x7f3d801f8810 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 --2- 172.21.15.131:0/2707331981 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3d80069770 0x7f3d80149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 -- 172.21.15.131:0/2707331981 >> 172.21.15.131:0/2707331981 conn(0x7f3d800d07c0 msgr2=0x7f3d800ceb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 -- 172.21.15.131:0/2707331981 shutdown_connections 2024-09-17T02:42:59.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.655+0000 7f3d8872a640 1 -- 172.21.15.131:0/2707331981 wait complete. 2024-09-17T02:42:59.664 INFO:tasks.ceph.mds.b.smithi189.stderr:2024-09-17T02:42:59.662+0000 7f45b548e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 pv4 ap=4 DIRTYPARENT s=0 n(v0 1=1+0)->n(v0 rc2024-09-17T02:42:58.402946+0000 1=1+0) (iauth excl w=1) (ifile lock->xlock w=1) (iquiesce lock w=1) cr={5625=0-4194304@1} caps={5625=pAsLsXs/-@3},l=5625(-1) | ptrwaiter=1 request=1 lock=4 caps=1 dirtyparent=1 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x5569be6e4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-17T02:42:59.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:42:59.676 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-09-17T02:42:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 -- 172.21.15.131:0/3531541079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc06dea0 msgr2=0x7fdcdc06e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 --2- 172.21.15.131:0/3531541079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc06dea0 0x7fdcdc06e2b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0afaf0 tx=0x7fdcd401cb70 comp rx=0 tx=0).stop 2024-09-17T02:42:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 -- 172.21.15.131:0/3531541079 shutdown_connections 2024-09-17T02:42:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 --2- 172.21.15.131:0/3531541079 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdcdc0cab60 0x7fdcdc0caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 --2- 172.21.15.131:0/3531541079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc06dea0 0x7fdcdc06e2b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 -- 172.21.15.131:0/3531541079 >> 172.21.15.131:0/3531541079 conn(0x7fdcdc06eea0 msgr2=0x7fdcdc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 -- 172.21.15.131:0/3531541079 shutdown_connections 2024-09-17T02:42:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 -- 172.21.15.131:0/3531541079 wait complete. 2024-09-17T02:42:59.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 Processor -- start 2024-09-17T02:42:59.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 -- start start 2024-09-17T02:42:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc06dea0 0x7fdcdc14fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdcdc0cab60 0x7fdcdc1f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcdc155240 0x7fdcdc153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc168520 con 0x7fdcdc06dea0 2024-09-17T02:42:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc1683a0 con 0x7fdcdc155240 2024-09-17T02:42:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdce0f66640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdcdc1686a0 con 0x7fdcdc0cab60 2024-09-17T02:42:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdcda59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc06dea0 0x7fdcdc14fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdcda59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc06dea0 0x7fdcdc14fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57224/0 (socket says 172.21.15.131:57224) 2024-09-17T02:42:59.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.811+0000 7fdcda59c640 1 -- 172.21.15.131:0/3239931046 learned_addr learned my addr 172.21.15.131:0/3239931046 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdcdc0cab60 0x7fdcdc1f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdcdad9d640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcdc155240 0x7fdcdc153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdcda59c640 1 -- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdcdc0cab60 msgr2=0x7fdcdc1f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdcda59c640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdcdc0cab60 0x7fdcdc1f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdcda59c640 1 -- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcdc155240 msgr2=0x7fdcdc153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdcda59c640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcdc155240 0x7fdcdc153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdcda59c640 1 -- 172.21.15.131:0/3239931046 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc1f6960 con 0x7fdcdc06dea0 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdcda59c640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc06dea0 0x7fdcdc14fbd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000b410 tx=0x7fdcd000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdccaffd640 1 -- 172.21.15.131:0/3239931046 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0015030 con 0x7fdcdc06dea0 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdccaffd640 1 -- 172.21.15.131:0/3239931046 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcd0004020 con 0x7fdcdc06dea0 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdce0f66640 1 -- 172.21.15.131:0/3239931046 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc1f6e70 con 0x7fdcdc06dea0 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdccaffd640 1 -- 172.21.15.131:0/3239931046 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd00044b0 con 0x7fdcdc06dea0 2024-09-17T02:42:59.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdce0f66640 1 -- 172.21.15.131:0/3239931046 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc0c7060 con 0x7fdcdc06dea0 2024-09-17T02:42:59.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdccaffd640 1 -- 172.21.15.131:0/3239931046 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdcd0018070 con 0x7fdcdc06dea0 2024-09-17T02:42:59.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdccaffd640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdcbc073200 0x7fdcbc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdccaffd640 1 -- 172.21.15.131:0/3239931046 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fdcd0093f00 con 0x7fdcdc06dea0 2024-09-17T02:42:59.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdcbc073200 0x7fdcbc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdce8231000 1 Processor -- start 2024-09-17T02:42:59.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.815+0000 7fdce8231000 1 -- start start 2024-09-17T02:42:59.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdcbc073200 0x7fdcbc075710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40af000 tx=0x7fdcd401b3b0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb68f6d0 0x562bbb7ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb800530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb3aaaa0 0x562bbb800a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562bbb69acd0 con 0x562bbb68f6d0 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bbb5b0330 con 0x562bbb2e6890 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562bbb60dbb0 con 0x562bbb3aaaa0 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcda59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb68f6d0 0x562bbb7ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcda59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb68f6d0 0x562bbb7ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57238/0 (socket says 172.21.15.131:57238) 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcda59c640 1 -- 172.21.15.131:0/353848223 learned_addr learned my addr 172.21.15.131:0/353848223 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/353848223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb800530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcdad9d640 1 --2- 172.21.15.131:0/353848223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb3aaaa0 0x562bbb800a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcda59c640 1 -- 172.21.15.131:0/353848223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb3aaaa0 msgr2=0x562bbb800a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcda59c640 1 --2- 172.21.15.131:0/353848223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb3aaaa0 0x562bbb800a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcda59c640 1 -- 172.21.15.131:0/353848223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 msgr2=0x562bbb800530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcda59c640 1 --2- 172.21.15.131:0/353848223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb800530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcda59c640 1 -- 172.21.15.131:0/353848223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bbb5b1c80 con 0x562bbb68f6d0 2024-09-17T02:42:59.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcda59c640 1 --2- 172.21.15.131:0/353848223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb68f6d0 0x562bbb7ffff0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0092240 tx=0x7fdcd005eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e54d72bcea37023 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcbbfff640 1 -- 172.21.15.131:0/353848223 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd00680b0 con 0x562bbb68f6d0 2024-09-17T02:42:59.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcbbfff640 1 -- 172.21.15.131:0/353848223 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcd000fb60 con 0x562bbb68f6d0 2024-09-17T02:42:59.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdcbbfff640 1 -- 172.21.15.131:0/353848223 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd005d980 con 0x562bbb68f6d0 2024-09-17T02:42:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 -- 172.21.15.131:0/353848223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb68f6d0 msgr2=0x562bbb7ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 --2- 172.21.15.131:0/353848223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb68f6d0 0x562bbb7ffff0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0092240 tx=0x7fdcd005eee0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 -- 172.21.15.131:0/353848223 shutdown_connections 2024-09-17T02:42:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 --2- 172.21.15.131:0/353848223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb3aaaa0 0x562bbb800a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 --2- 172.21.15.131:0/353848223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb800530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 --2- 172.21.15.131:0/353848223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb68f6d0 0x562bbb7ffff0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 -- 172.21.15.131:0/353848223 >> 172.21.15.131:0/353848223 conn(0x562bbb2ba5c0 msgr2=0x562bbb7b8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 -- 172.21.15.131:0/353848223 shutdown_connections 2024-09-17T02:42:59.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.819+0000 7fdce8231000 1 -- 172.21.15.131:0/353848223 wait complete. 2024-09-17T02:42:59.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 -- start start 2024-09-17T02:42:59.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 Processor -- start 2024-09-17T02:42:59.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb7c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7c21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb7c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562bbb600df0 con 0x562bbb3aaaa0 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562bbb5fcfe0 con 0x562bbb2e6890 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562bbb5a24d0 con 0x562bbb68f6d0 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcd9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7c21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 10 client.0 _ll_register_callbacks cb 0x562bbb5807f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcd9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7c21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57248/0 (socket says 172.21.15.131:57248) 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcd9d9b640 1 -- 172.21.15.131:0/2655141109 learned_addr learned my addr 172.21.15.131:0/2655141109 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcda59c640 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb7c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcdad9d640 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb7c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcd9d9b640 1 -- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 msgr2=0x562bbb7c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb7c2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcd9d9b640 1 -- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 msgr2=0x562bbb7c1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb7c1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcd9d9b640 1 -- 172.21.15.131:0/2655141109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bbb59b210 con 0x562bbb3aaaa0 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7c21f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdcd401c5e0 tx=0x7fdcd40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9cf4592c68827df server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcb97fa640 1 -- 172.21.15.131:0/2655141109 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40bf040 con 0x562bbb3aaaa0 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcb97fa640 1 -- 172.21.15.131:0/2655141109 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fdcd4004240 con 0x562bbb3aaaa0 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 -- 172.21.15.131:0/2655141109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562bbb595350 con 0x562bbb3aaaa0 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcb97fa640 1 -- 172.21.15.131:0/2655141109 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcd40d1050 con 0x562bbb3aaaa0 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 10 client.5685 fetch_fsmap learned FSMap version 27 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 -- 172.21.15.131:0/2655141109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x562bbb581fc0 con 0x562bbb3aaaa0 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcb97fa640 1 -- 172.21.15.131:0/2655141109 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562bbb595350 con 0x562bbb3aaaa0 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcb97fa640 1 -- 172.21.15.131:0/2655141109 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7fdcd40d4020 con 0x562bbb3aaaa0 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 10 client.5685 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 10 client.5685 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-17T02:42:59.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 20 client.5685 populate_metadata read hostname 'smithi131' 2024-09-17T02:42:59.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x562bbb29f930 0x562bbb8115b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 4 client.5685 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:42:59.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdce8231000 1 -- 172.21.15.131:0/2655141109 --> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562bbb60fb00 con 0x562bbb29f930 2024-09-17T02:42:59.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcda59c640 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x562bbb29f930 0x562bbb8115b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcb97fa640 10 client.5685 ms_handle_connect on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:59.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.823+0000 7fdcda59c640 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x562bbb29f930 0x562bbb8115b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc19180ff69496e1 server_cookie=1cc7fece2933b25e in_seq=0 out_seq=0 2024-09-17T02:42:59.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdcb97fa640 1 -- 172.21.15.131:0/2655141109 <== mds.0 v2:172.21.15.189:6835/4030750882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562bbb60fb00 con 0x562bbb29f930 2024-09-17T02:42:59.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdcb97fa640 10 client.5685 handle_command_reply: tid=0 2024-09-17T02:42:59.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 1 client.5685 shutdown 2024-09-17T02:42:59.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 1 -- 172.21.15.131:0/2655141109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 msgr2=0x562bbb7c21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7c21f0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdcd401c5e0 tx=0x7fdcd40095c0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 1 -- 172.21.15.131:0/2655141109 shutdown_connections 2024-09-17T02:42:59.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x562bbb29f930 0x562bbb8115b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb7c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7c21f0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 1 --2- 172.21.15.131:0/2655141109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb7c1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdcb97fa640 0 client.5685 ms_handle_reset on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:59.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 1 -- 172.21.15.131:0/2655141109 >> 172.21.15.131:0/2655141109 conn(0x562bbb2ba5c0 msgr2=0x562bbb7b0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 1 -- 172.21.15.131:0/2655141109 shutdown_connections 2024-09-17T02:42:59.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 1 -- 172.21.15.131:0/2655141109 wait complete. 2024-09-17T02:42:59.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.827+0000 7fdce8231000 20 client.5685 trim_cache size 0 max 16384 2024-09-17T02:42:59.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 Processor -- start 2024-09-17T02:42:59.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 -- start start 2024-09-17T02:42:59.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb7b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb7b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562bbb7bfd80 con 0x562bbb3aaaa0 2024-09-17T02:42:59.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc201650 con 0x562bbb2e6890 2024-09-17T02:42:59.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdcdc152100 con 0x562bbb68f6d0 2024-09-17T02:42:59.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcd9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcd9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7b8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57262/0 (socket says 172.21.15.131:57262) 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcd9d9b640 1 -- 172.21.15.131:0/1252347227 learned_addr learned my addr 172.21.15.131:0/1252347227 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcd9d9b640 1 -- 172.21.15.131:0/1252347227 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 msgr2=0x562bbb7b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/1252347227 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb7b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcda59c640 1 --2- 172.21.15.131:0/1252347227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb7b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcd9d9b640 1 -- 172.21.15.131:0/1252347227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 msgr2=0x562bbb7b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/1252347227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb7b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcd9d9b640 1 -- 172.21.15.131:0/1252347227 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562bbb65a680 con 0x562bbb3aaaa0 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/1252347227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7b8ab0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40a8020 tx=0x7fdcd4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831f1d85d31c2b21 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcbb7fe640 1 -- 172.21.15.131:0/1252347227 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40b2040 con 0x562bbb3aaaa0 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcbb7fe640 1 -- 172.21.15.131:0/1252347227 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcd40db070 con 0x562bbb3aaaa0 2024-09-17T02:42:59.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdcbb7fe640 1 -- 172.21.15.131:0/1252347227 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4005440 con 0x562bbb3aaaa0 2024-09-17T02:42:59.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 -- 172.21.15.131:0/1252347227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 msgr2=0x562bbb7b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 --2- 172.21.15.131:0/1252347227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7b8ab0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40a8020 tx=0x7fdcd4009b00 comp rx=0 tx=0).stop 2024-09-17T02:42:59.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 -- 172.21.15.131:0/1252347227 shutdown_connections 2024-09-17T02:42:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 --2- 172.21.15.131:0/1252347227 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb7b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 --2- 172.21.15.131:0/1252347227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb3aaaa0 0x562bbb7b8ab0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 --2- 172.21.15.131:0/1252347227 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb2e6890 0x562bbb7b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 -- 172.21.15.131:0/1252347227 >> 172.21.15.131:0/1252347227 conn(0x562bbb2ba5c0 msgr2=0x562bbb7b7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 -- 172.21.15.131:0/1252347227 shutdown_connections 2024-09-17T02:42:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.847+0000 7fdce8231000 1 -- 172.21.15.131:0/1252347227 wait complete. 2024-09-17T02:42:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 -- start start 2024-09-17T02:42:59.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 Processor -- start 2024-09-17T02:42:59.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb2e6890 0x562bbb875940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb3aaaa0 0x562bbb875e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb8763c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc151a90 con 0x562bbb2e6890 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc151440 con 0x562bbb3aaaa0 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562bbb2c2680 con 0x562bbb68f6d0 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 10 client.0 _ll_register_callbacks cb 0x7fdcd40b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcda59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb2e6890 0x562bbb875940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcda59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb2e6890 0x562bbb875940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57264/0 (socket says 172.21.15.131:57264) 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcda59c640 1 -- 172.21.15.131:0/3486035085 learned_addr learned my addr 172.21.15.131:0/3486035085 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb3aaaa0 0x562bbb875e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcdad9d640 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb8763c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcda59c640 1 -- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 msgr2=0x562bbb8763c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcda59c640 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb8763c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcda59c640 1 -- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb3aaaa0 msgr2=0x562bbb875e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:42:59.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcda59c640 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb3aaaa0 0x562bbb875e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcda59c640 1 -- 172.21.15.131:0/3486035085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562bbb5c70b0 con 0x562bbb2e6890 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcdad9d640 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb8763c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb3aaaa0 0x562bbb875e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcda59c640 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb2e6890 0x562bbb875940 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdcd005a6c0 tx=0x7fdcd0061040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecbf907c76782be server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcb97fa640 1 -- 172.21.15.131:0/3486035085 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd0078de0 con 0x562bbb2e6890 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcb97fa640 1 -- 172.21.15.131:0/3486035085 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fdcd000fb60 con 0x562bbb2e6890 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 -- 172.21.15.131:0/3486035085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd40db070 con 0x562bbb2e6890 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcb97fa640 1 -- 172.21.15.131:0/3486035085 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcd000fd60 con 0x562bbb2e6890 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 10 client.5691 fetch_fsmap learned FSMap version 27 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 -- 172.21.15.131:0/3486035085 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x562bbb593610 con 0x562bbb2e6890 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcb97fa640 1 -- 172.21.15.131:0/3486035085 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcd40db070 con 0x562bbb2e6890 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcb97fa640 1 -- 172.21.15.131:0/3486035085 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7fdcd0022060 con 0x562bbb2e6890 2024-09-17T02:42:59.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 10 client.5691 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 10 client.5691 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 20 client.5691 populate_metadata read hostname 'smithi131' 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x562bbb28d990 0x562bbb8129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 4 client.5691 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdce8231000 1 -- 172.21.15.131:0/3486035085 --> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562bbb60b5b0 con 0x562bbb28d990 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.851+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x562bbb28d990 0x562bbb8129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdcb97fa640 10 client.5691 ms_handle_connect on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdcd9d9b640 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x562bbb28d990 0x562bbb8129f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3a92d18c115d58 server_cookie=a0725d5966d6387e in_seq=0 out_seq=0 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdcb97fa640 1 -- 172.21.15.131:0/3486035085 <== mds.0 v2:172.21.15.189:6835/4030750882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+571 (crc 0 0 0) 0x562bbb60b5b0 con 0x562bbb28d990 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdcb97fa640 10 client.5691 handle_command_reply: tid=0 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 1 client.5691 shutdown 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 1 -- 172.21.15.131:0/3486035085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb2e6890 msgr2=0x562bbb875940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb2e6890 0x562bbb875940 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdcd005a6c0 tx=0x7fdcd0061040 comp rx=0 tx=0).stop 2024-09-17T02:42:59.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 1 -- 172.21.15.131:0/3486035085 shutdown_connections 2024-09-17T02:42:59.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x562bbb28d990 0x562bbb8129f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562bbb68f6d0 0x562bbb8763c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562bbb3aaaa0 0x562bbb875e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 1 --2- 172.21.15.131:0/3486035085 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562bbb2e6890 0x562bbb875940 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdcb97fa640 0 client.5691 ms_handle_reset on v2:172.21.15.189:6835/4030750882 2024-09-17T02:42:59.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 1 -- 172.21.15.131:0/3486035085 >> 172.21.15.131:0/3486035085 conn(0x562bbb2ba5c0 msgr2=0x562bbb7b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 1 -- 172.21.15.131:0/3486035085 shutdown_connections 2024-09-17T02:42:59.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 1 -- 172.21.15.131:0/3486035085 wait complete. 2024-09-17T02:42:59.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce8231000 20 client.5691 trim_cache size 0 max 16384 2024-09-17T02:42:59.862 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:42:59.862 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-17T02:42:59.863 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-09-17T02:42:59.863 INFO:teuthology.orchestra.run.smithi131.stdout: "87131ed3-d4e1-440a-b0c9-3ff18ff3d0d0": { 2024-09-17T02:42:59.863 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:42:59.863 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "87131ed3-d4e1-440a-b0c9-3ff18ff3d0d0", 2024-09-17T02:42:59.863 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,repair" 2024-09-17T02:42:59.863 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:59.863 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:42:59.863 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:42:59.863 INFO:teuthology.orchestra.run.smithi131.stdout: "87131ed3-d4e1-440a-b0c9-3ff18ff3d0d0": { 2024-09-17T02:42:59.864 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:42:59.654567+0000", 2024-09-17T02:42:59.864 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:42:59.864 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:42:59.864 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:42:59.864 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:42:59.864 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:42:59.864 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:59.864 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:42:59.864 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:42:59.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 -- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdcbc073200 msgr2=0x7fdcbc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdcbc073200 0x7fdcbc075710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40af000 tx=0x7fdcd401b3b0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 -- 172.21.15.131:0/3239931046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc06dea0 msgr2=0x7fdcdc14fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:42:59.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc06dea0 0x7fdcdc14fbd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdcd000b410 tx=0x7fdcd000b8d0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 -- 172.21.15.131:0/3239931046 shutdown_connections 2024-09-17T02:42:59.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdcbc073200 0x7fdcbc075710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdcdc155240 0x7fdcdc153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdcdc0cab60 0x7fdcdc1f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 --2- 172.21.15.131:0/3239931046 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdcdc06dea0 0x7fdcdc14fbd0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:42:59.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 -- 172.21.15.131:0/3239931046 >> 172.21.15.131:0/3239931046 conn(0x7fdcdc06eea0 msgr2=0x7fdcdc0b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:42:59.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 -- 172.21.15.131:0/3239931046 shutdown_connections 2024-09-17T02:42:59.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:42:59.855+0000 7fdce0f66640 1 -- 172.21.15.131:0/3239931046 wait complete. 2024-09-17T02:42:59.876 INFO:tasks.cephfs.filesystem:scrub status for tag:87131ed3-d4e1-440a-b0c9-3ff18ff3d0d0 - {'path': '/', 'tag': '87131ed3-d4e1-440a-b0c9-3ff18ff3d0d0', 'options': 'recursive,repair'} 2024-09-17T02:43:01.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:01.247+0000 7f8a3effd640 1 -- 172.21.15.131:0/2439293179 <== mon.0 v2:172.21.15.131:3300/0 7 ==== log(7 entries from seq 1 at 2024-09-17T02:42:59.660178+0000) ==== 1229+0+0 (secure 0 0 0) 0x7f8a44092920 con 0x7f8a500cf230 2024-09-17T02:43:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: cluster: 2024-09-17T02:43:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: id: f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:43:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: health: HEALTH_OK 2024-09-17T02:43:01.251 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: services: 2024-09-17T02:43:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-09-17T02:43:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: mgr: x(active, since 2m), standbys: y 2024-09-17T02:43:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: mds: 1/1 daemons up, 3 standby 2024-09-17T02:43:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-17T02:43:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: data: 2024-09-17T02:43:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: volumes: 1/1 healthy 2024-09-17T02:43:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: pools: 3 pools, 97 pgs 2024-09-17T02:43:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: objects: 24 objects, 579 KiB 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: pgs: 97 active+clean 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: io: 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: client: 85 B/s wr, 0 op/s rd, 0 op/s wr 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:42:59.660178+0000 mds.b [INF] scrub queued for path: / 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:42:59.660188+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:42:59.660199+0000 mds.b [INF] scrub summary: active paths [/] 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:42:59.664112+0000 mds.b [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-09-17T02:43:01.253 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:42:59.664160+0000 mds.b [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-09-17T02:43:01.254 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:42:59.667818+0000 mds.b [INF] scrub summary: idle+waiting paths [/] 2024-09-17T02:43:01.254 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:43:00.285434+0000 mds.b [INF] scrub summary: idle 2024-09-17T02:43:13.624 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:43:13.627 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:43:29.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:29.877 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-09-17T02:43:30.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.047+0000 7f04ff3a2640 1 -- 172.21.15.131:0/4197085165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f80cec00 msgr2=0x7f04f80cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.047+0000 7f04ff3a2640 1 --2- 172.21.15.131:0/4197085165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f80cec00 0x7f04f80cf010 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009c80 tx=0x7f04f001cd30 comp rx=0 tx=0).stop 2024-09-17T02:43:30.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.047+0000 7f04ff3a2640 1 -- 172.21.15.131:0/4197085165 shutdown_connections 2024-09-17T02:43:30.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.047+0000 7f04ff3a2640 1 --2- 172.21.15.131:0/4197085165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f80cec00 0x7f04f80cf010 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.047+0000 7f04ff3a2640 1 --2- 172.21.15.131:0/4197085165 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04f806e2c0 0x7f04f80c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.047+0000 7f04ff3a2640 1 -- 172.21.15.131:0/4197085165 >> 172.21.15.131:0/4197085165 conn(0x7f04f806eea0 msgr2=0x7f04f806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:30.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.047+0000 7f04ff3a2640 1 -- 172.21.15.131:0/4197085165 shutdown_connections 2024-09-17T02:43:30.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.047+0000 7f04ff3a2640 1 -- 172.21.15.131:0/4197085165 wait complete. 2024-09-17T02:43:30.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04ff3a2640 1 Processor -- start 2024-09-17T02:43:30.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04ff3a2640 1 -- start start 2024-09-17T02:43:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04ff3a2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04f806e2c0 0x7f04f8145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04ff3a2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04f80c4ac0 0x7f04f8152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04ff3a2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f80cec00 0x7f04f8152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04ff3a2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8169e70 con 0x7f04f80cec00 2024-09-17T02:43:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04ff3a2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8169cf0 con 0x7f04f806e2c0 2024-09-17T02:43:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04ff3a2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f04f8169ff0 con 0x7f04f80c4ac0 2024-09-17T02:43:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04fd93f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f80cec00 0x7f04f8152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04fd93f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f80cec00 0x7f04f8152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52790/0 (socket says 172.21.15.131:52790) 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04fd93f640 1 -- 172.21.15.131:0/2293857531 learned_addr learned my addr 172.21.15.131:0/2293857531 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04fd13e640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04f806e2c0 0x7f04f8145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04fc93d640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04f80c4ac0 0x7f04f8152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04fd93f640 1 -- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04f80c4ac0 msgr2=0x7f04f8152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04fd93f640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04f80c4ac0 0x7f04f8152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04fd93f640 1 -- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04f806e2c0 msgr2=0x7f04f8145140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04fd93f640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04f806e2c0 0x7f04f8145140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04fd93f640 1 -- 172.21.15.131:0/2293857531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f8145680 con 0x7f04f80cec00 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04fd93f640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f80cec00 0x7f04f8152d50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009bd0 tx=0x7f04f00093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04edffb640 1 -- 172.21.15.131:0/2293857531 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f00bc070 con 0x7f04f80cec00 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04edffb640 1 -- 172.21.15.131:0/2293857531 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04f00042f0 con 0x7f04f80cec00 2024-09-17T02:43:30.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04ff3a2640 1 -- 172.21.15.131:0/2293857531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f8149930 con 0x7f04f80cec00 2024-09-17T02:43:30.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04edffb640 1 -- 172.21.15.131:0/2293857531 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f00b2040 con 0x7f04f80cec00 2024-09-17T02:43:30.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.051+0000 7f04ff3a2640 1 -- 172.21.15.131:0/2293857531 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f8149eb0 con 0x7f04f80cec00 2024-09-17T02:43:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04edffb640 1 -- 172.21.15.131:0/2293857531 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f04f00bf070 con 0x7f04f80cec00 2024-09-17T02:43:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04edffb640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f04dc073200 0x7f04dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04edffb640 1 -- 172.21.15.131:0/2293857531 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f04f012e250 con 0x7f04f80cec00 2024-09-17T02:43:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd13e640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f04dc073200 0x7f04dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f050666d000 1 Processor -- start 2024-09-17T02:43:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f050666d000 1 -- start start 2024-09-17T02:43:30.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd13e640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f04dc073200 0x7f04dc075710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f04f4005fd0 tx=0x7f04f4005d10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f050666d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1ccd396d0 0x55e1ccea59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f050666d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1cc990890 0x55e1ccea5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f050666d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cca54aa0 0x55e1ccea6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f050666d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e1ccd44cd0 con 0x55e1ccd396d0 2024-09-17T02:43:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f050666d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e1ccc5a330 con 0x55e1cca54aa0 2024-09-17T02:43:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f050666d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55e1cccb7bb0 con 0x55e1cc990890 2024-09-17T02:43:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd13e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1ccd396d0 0x55e1ccea59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd13e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1ccd396d0 0x55e1ccea59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52792/0 (socket says 172.21.15.131:52792) 2024-09-17T02:43:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd13e640 1 -- 172.21.15.131:0/683114850 learned_addr learned my addr 172.21.15.131:0/683114850 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:30.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd13e640 1 -- 172.21.15.131:0/683114850 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1cc990890 msgr2=0x55e1ccea5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd93f640 1 --2- 172.21.15.131:0/683114850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cca54aa0 0x55e1ccea6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd13e640 1 --2- 172.21.15.131:0/683114850 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1cc990890 0x55e1ccea5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd13e640 1 -- 172.21.15.131:0/683114850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cca54aa0 msgr2=0x55e1ccea6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd13e640 1 --2- 172.21.15.131:0/683114850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cca54aa0 0x55e1ccea6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd13e640 1 -- 172.21.15.131:0/683114850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ccc5bc80 con 0x55e1ccd396d0 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04fd13e640 1 --2- 172.21.15.131:0/683114850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1ccd396d0 0x55e1ccea59f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f04f400eda0 tx=0x7f04f4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930b7469dde44704 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.055+0000 7f04e2ffd640 1 -- 172.21.15.131:0/683114850 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4021070 con 0x55e1ccd396d0 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04e2ffd640 1 -- 172.21.15.131:0/683114850 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04f401d740 con 0x55e1ccd396d0 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04e2ffd640 1 -- 172.21.15.131:0/683114850 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f401daf0 con 0x55e1ccd396d0 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 -- 172.21.15.131:0/683114850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1ccd396d0 msgr2=0x55e1ccea59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 --2- 172.21.15.131:0/683114850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1ccd396d0 0x55e1ccea59f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f04f400eda0 tx=0x7f04f4011000 comp rx=0 tx=0).stop 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 -- 172.21.15.131:0/683114850 shutdown_connections 2024-09-17T02:43:30.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 --2- 172.21.15.131:0/683114850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cca54aa0 0x55e1ccea6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 --2- 172.21.15.131:0/683114850 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1cc990890 0x55e1ccea5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 --2- 172.21.15.131:0/683114850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1ccd396d0 0x55e1ccea59f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 -- 172.21.15.131:0/683114850 >> 172.21.15.131:0/683114850 conn(0x55e1cc9645c0 msgr2=0x55e1cce62ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 -- 172.21.15.131:0/683114850 shutdown_connections 2024-09-17T02:43:30.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 -- 172.21.15.131:0/683114850 wait complete. 2024-09-17T02:43:30.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 -- start start 2024-09-17T02:43:30.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 Processor -- start 2024-09-17T02:43:30.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 0x55e1cce619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce61f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1cce685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e1cccaadf0 con 0x55e1cca54aa0 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e1ccca6fe0 con 0x55e1cc990890 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55e1ccc4c4d0 con 0x55e1ccd396d0 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f050666d000 10 client.0 _ll_register_callbacks cb 0x55e1ccc2a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fc93d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce61f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fc93d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce61f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52802/0 (socket says 172.21.15.131:52802) 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fc93d640 1 -- 172.21.15.131:0/3333829175 learned_addr learned my addr 172.21.15.131:0/3333829175 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fd13e640 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 0x55e1cce619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fd93f640 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1cce685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fc93d640 1 -- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 msgr2=0x55e1cce685f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fc93d640 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1cce685f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fc93d640 1 -- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 msgr2=0x55e1cce619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fc93d640 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 0x55e1cce619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fc93d640 1 -- 172.21.15.131:0/3333829175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1ccc45210 con 0x55e1cca54aa0 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fd93f640 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1cce685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fd13e640 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 0x55e1cce619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04fc93d640 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce61f10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f04f80b6130 tx=0x7f04e800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771c7dd882a3ec81 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.059+0000 7f04d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f04d7fff640 1 -- 172.21.15.131:0/3333829175 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8019060 con 0x55e1cca54aa0 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f04d7fff640 1 -- 172.21.15.131:0/3333829175 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f04e800edc0 con 0x55e1cca54aa0 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 -- 172.21.15.131:0/3333829175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1ccc3f350 con 0x55e1cca54aa0 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f04d7fff640 1 -- 172.21.15.131:0/3333829175 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04e80050d0 con 0x55e1cca54aa0 2024-09-17T02:43:30.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 10 client.5703 fetch_fsmap learned FSMap version 27 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 -- 172.21.15.131:0/3333829175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55e1ccc2bfc0 con 0x55e1cca54aa0 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f04d7fff640 1 -- 172.21.15.131:0/3333829175 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55e1ccc3f350 con 0x55e1cca54aa0 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f04d7fff640 1 -- 172.21.15.131:0/3333829175 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f04e8005270 con 0x55e1cca54aa0 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 10 client.5703 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 10 client.5703 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 20 client.5703 populate_metadata read hostname 'smithi131' 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55e1cc949930 0x55e1cceace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 4 client.5703 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 -- 172.21.15.131:0/3333829175 --> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1cccb9b00 con 0x55e1cc949930 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f04fd13e640 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55e1cc949930 0x55e1cceace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f04d7fff640 10 client.5703 ms_handle_connect on v2:172.21.15.189:6835/4030750882 2024-09-17T02:43:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f04fd13e640 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55e1cc949930 0x55e1cceace80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10515b14d9206576 server_cookie=ae62a201616c5900 in_seq=0 out_seq=0 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f04d7fff640 1 -- 172.21.15.131:0/3333829175 <== mds.0 v2:172.21.15.189:6835/4030750882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e1cccb9b00 con 0x55e1cc949930 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f04d7fff640 10 client.5703 handle_command_reply: tid=0 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 client.5703 shutdown 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 -- 172.21.15.131:0/3333829175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 msgr2=0x55e1cce61f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce61f10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f04f80b6130 tx=0x7f04e800e530 comp rx=0 tx=0).stop 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 -- 172.21.15.131:0/3333829175 shutdown_connections 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55e1cc949930 0x55e1cceace80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1cce685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce61f10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 --2- 172.21.15.131:0/3333829175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 0x55e1cce619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f04d7fff640 0 client.5703 ms_handle_reset on v2:172.21.15.189:6835/4030750882 2024-09-17T02:43:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 -- 172.21.15.131:0/3333829175 >> 172.21.15.131:0/3333829175 conn(0x55e1cc9645c0 msgr2=0x55e1cce5ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:30.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 -- 172.21.15.131:0/3333829175 shutdown_connections 2024-09-17T02:43:30.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.063+0000 7f050666d000 1 -- 172.21.15.131:0/3333829175 wait complete. 2024-09-17T02:43:30.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.067+0000 7f050666d000 20 client.5703 trim_cache size 0 max 16384 2024-09-17T02:43:30.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.083+0000 7f050666d000 1 Processor -- start 2024-09-17T02:43:30.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.083+0000 7f050666d000 1 -- start start 2024-09-17T02:43:30.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.083+0000 7f050666d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cc990890 0x55e1ccf171f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.083+0000 7f050666d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cca54aa0 0x55e1ccf17730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.083+0000 7f050666d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1ccf17c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.083+0000 7f050666d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8203520 con 0x55e1cc990890 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.083+0000 7f050666d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04f8202560 con 0x55e1cca54aa0 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.083+0000 7f050666d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f04f814d120 con 0x55e1ccd396d0 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04fd13e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cc990890 0x55e1ccf171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04fd13e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cc990890 0x55e1ccf171f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52814/0 (socket says 172.21.15.131:52814) 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04fd13e640 1 -- 172.21.15.131:0/3500053811 learned_addr learned my addr 172.21.15.131:0/3500053811 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04fd93f640 1 --2- 172.21.15.131:0/3500053811 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1ccf17c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04fd13e640 1 -- 172.21.15.131:0/3500053811 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 msgr2=0x55e1ccf17c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04fc93d640 1 --2- 172.21.15.131:0/3500053811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cca54aa0 0x55e1ccf17730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04fd13e640 1 --2- 172.21.15.131:0/3500053811 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1ccf17c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04fd13e640 1 -- 172.21.15.131:0/3500053811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cca54aa0 msgr2=0x55e1ccf17730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:30.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04fd13e640 1 --2- 172.21.15.131:0/3500053811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cca54aa0 0x55e1ccf17730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04fd13e640 1 -- 172.21.15.131:0/3500053811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ccd04680 con 0x55e1cc990890 2024-09-17T02:43:30.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04fd13e640 1 --2- 172.21.15.131:0/3500053811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cc990890 0x55e1ccf171f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55e1ccd5a760 tx=0x7f04f40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87332368bfe90bee server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04e27fc640 1 -- 172.21.15.131:0/3500053811 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4014040 con 0x55e1cc990890 2024-09-17T02:43:30.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04e27fc640 1 -- 172.21.15.131:0/3500053811 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04f4027660 con 0x55e1cc990890 2024-09-17T02:43:30.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f04e27fc640 1 -- 172.21.15.131:0/3500053811 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4021040 con 0x55e1cc990890 2024-09-17T02:43:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f050666d000 1 -- 172.21.15.131:0/3500053811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cc990890 msgr2=0x55e1ccf171f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f050666d000 1 --2- 172.21.15.131:0/3500053811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cc990890 0x55e1ccf171f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55e1ccd5a760 tx=0x7f04f40046d0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f050666d000 1 -- 172.21.15.131:0/3500053811 shutdown_connections 2024-09-17T02:43:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f050666d000 1 --2- 172.21.15.131:0/3500053811 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1ccf17c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f050666d000 1 --2- 172.21.15.131:0/3500053811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cca54aa0 0x55e1ccf17730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f050666d000 1 --2- 172.21.15.131:0/3500053811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cc990890 0x55e1ccf171f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f050666d000 1 -- 172.21.15.131:0/3500053811 >> 172.21.15.131:0/3500053811 conn(0x55e1cc9645c0 msgr2=0x55e1cce675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:30.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f050666d000 1 -- 172.21.15.131:0/3500053811 shutdown_connections 2024-09-17T02:43:30.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f050666d000 1 -- 172.21.15.131:0/3500053811 wait complete. 2024-09-17T02:43:30.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f050666d000 1 -- start start 2024-09-17T02:43:30.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.087+0000 7f050666d000 1 Processor -- start 2024-09-17T02:43:30.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 0x55e1ccf17820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce5c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1cce5ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e1cccaa100 con 0x55e1cca54aa0 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e1cce65d60 con 0x55e1cc990890 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55e1ccc3e300 con 0x55e1ccd396d0 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 10 client.0 _ll_register_callbacks cb 0x7f04e8007d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fc93d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce5c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fc93d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce5c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52822/0 (socket says 172.21.15.131:52822) 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fc93d640 1 -- 172.21.15.131:0/2109117612 learned_addr learned my addr 172.21.15.131:0/2109117612 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fd13e640 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 0x55e1ccf17820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fd93f640 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1cce5ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fc93d640 1 -- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 msgr2=0x55e1cce5ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fc93d640 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1cce5ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fc93d640 1 -- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 msgr2=0x55e1ccf17820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:30.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fc93d640 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 0x55e1ccf17820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fc93d640 1 -- 172.21.15.131:0/2109117612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1ccc710b0 con 0x55e1cca54aa0 2024-09-17T02:43:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fd13e640 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 0x55e1ccf17820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fd93f640 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1cce5ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fc93d640 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce5c990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f04e8001590 tx=0x7f04e8008fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a56d880ae0a347 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:43:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04d7fff640 1 -- 172.21.15.131:0/2109117612 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8005270 con 0x55e1cca54aa0 2024-09-17T02:43:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04d7fff640 1 -- 172.21.15.131:0/2109117612 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f04e8010050 con 0x55e1cca54aa0 2024-09-17T02:43:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 -- 172.21.15.131:0/2109117612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04f4027660 con 0x55e1cca54aa0 2024-09-17T02:43:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04d7fff640 1 -- 172.21.15.131:0/2109117612 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f04e8034050 con 0x55e1cca54aa0 2024-09-17T02:43:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 10 client.5709 fetch_fsmap learned FSMap version 27 2024-09-17T02:43:30.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 -- 172.21.15.131:0/2109117612 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) -- 0x55e1ccc3d610 con 0x55e1cca54aa0 2024-09-17T02:43:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04d7fff640 1 -- 172.21.15.131:0/2109117612 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04f4027660 con 0x55e1cca54aa0 2024-09-17T02:43:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04d7fff640 1 -- 172.21.15.131:0/2109117612 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 27) ==== 3502+0+0 (secure 0 0 0) 0x7f04e8037020 con 0x55e1cca54aa0 2024-09-17T02:43:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 10 client.5709 fetch_fsmap finished waiting for FSMap version 27 2024-09-17T02:43:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 10 client.5709 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.b 2024-09-17T02:43:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 20 client.5709 populate_metadata read hostname 'smithi131' 2024-09-17T02:43:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55e1cc937990 0x55e1cce5d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 4 client.5709 mds_command: new command op to 5463 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:43:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 -- 172.21.15.131:0/2109117612 --> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1cccb55b0 con 0x55e1cc937990 2024-09-17T02:43:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fd13e640 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55e1cc937990 0x55e1cce5d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04d7fff640 10 client.5709 ms_handle_connect on v2:172.21.15.189:6835/4030750882 2024-09-17T02:43:30.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04fd13e640 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55e1cc937990 0x55e1cce5d410 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e81f2caf0cec2af server_cookie=bc29bea8c0a4660d in_seq=0 out_seq=0 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04d7fff640 1 -- 172.21.15.131:0/2109117612 <== mds.0 v2:172.21.15.189:6835/4030750882 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55e1cccb55b0 con 0x55e1cc937990 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04d7fff640 10 client.5709 handle_command_reply: tid=0 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 client.5709 shutdown 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 -- 172.21.15.131:0/2109117612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 msgr2=0x55e1cce5c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce5c990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f04e8001590 tx=0x7f04e8008fa0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 -- 172.21.15.131:0/2109117612 shutdown_connections 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:6835/4030750882,v1:172.21.15.189:6837/4030750882] conn(0x55e1cc937990 0x55e1cce5d410 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1ccd396d0 0x55e1cce5ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1cca54aa0 0x55e1cce5c990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f04d7fff640 0 client.5709 ms_handle_reset on v2:172.21.15.189:6835/4030750882 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 --2- 172.21.15.131:0/2109117612 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e1cc990890 0x55e1ccf17820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 -- 172.21.15.131:0/2109117612 >> 172.21.15.131:0/2109117612 conn(0x55e1cc9645c0 msgr2=0x55e1cceb7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:30.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 -- 172.21.15.131:0/2109117612 shutdown_connections 2024-09-17T02:43:30.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 1 -- 172.21.15.131:0/2109117612 wait complete. 2024-09-17T02:43:30.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.091+0000 7f050666d000 20 client.5709 trim_cache size 0 max 16384 2024-09-17T02:43:30.100 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {}, 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: "87131ed3-d4e1-440a-b0c9-3ff18ff3d0d0": { 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:42:59.654567+0000", 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:30.101 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:30.102 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:43:30.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 -- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f04dc073200 msgr2=0x7f04dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f04dc073200 0x7f04dc075710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f04f4005fd0 tx=0x7f04f4005d10 comp rx=0 tx=0).stop 2024-09-17T02:43:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 -- 172.21.15.131:0/2293857531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f80cec00 msgr2=0x7f04f8152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f80cec00 0x7f04f8152d50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f04f0009bd0 tx=0x7f04f00093d0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 -- 172.21.15.131:0/2293857531 shutdown_connections 2024-09-17T02:43:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f04dc073200 0x7f04dc075710 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f04f80cec00 0x7f04f8152d50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04f80c4ac0 0x7f04f8152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 --2- 172.21.15.131:0/2293857531 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04f806e2c0 0x7f04f8145140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 -- 172.21.15.131:0/2293857531 >> 172.21.15.131:0/2293857531 conn(0x7f04f806eea0 msgr2=0x7f04f8153bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 -- 172.21.15.131:0/2293857531 shutdown_connections 2024-09-17T02:43:30.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.095+0000 7f04ff3a2640 1 -- 172.21.15.131:0/2293857531 wait complete. 2024-09-17T02:43:30.115 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-17T02:43:30.216 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:43:30.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.371+0000 7f0cfc979640 1 -- 172.21.15.131:0/689991548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf806e2c0 msgr2=0x7f0cf80c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.371+0000 7f0cfc979640 1 --2- 172.21.15.131:0/689991548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf806e2c0 0x7f0cf80c4580 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009840 tx=0x7f0ce801c7b0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.371+0000 7f0cfc979640 1 -- 172.21.15.131:0/689991548 shutdown_connections 2024-09-17T02:43:30.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.371+0000 7f0cfc979640 1 --2- 172.21.15.131:0/689991548 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0cf80c4ac0 0x7f0cf80c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.371+0000 7f0cfc979640 1 --2- 172.21.15.131:0/689991548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf806e2c0 0x7f0cf80c4580 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.371+0000 7f0cfc979640 1 -- 172.21.15.131:0/689991548 >> 172.21.15.131:0/689991548 conn(0x7f0cf806eea0 msgr2=0x7f0cf806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:30.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.371+0000 7f0cfc979640 1 -- 172.21.15.131:0/689991548 shutdown_connections 2024-09-17T02:43:30.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.371+0000 7f0cfc979640 1 -- 172.21.15.131:0/689991548 wait complete. 2024-09-17T02:43:30.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cfc979640 1 Processor -- start 2024-09-17T02:43:30.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cfc979640 1 -- start start 2024-09-17T02:43:30.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cfc979640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf806e2c0 0x7f0cf8149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cfc979640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0cf80c4ac0 0x7f0cf8155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cfc979640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf80cdbf0 0x7f0cf815aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cfc979640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf80cf0a0 con 0x7f0cf806e2c0 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cfc979640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf80cef20 con 0x7f0cf80cdbf0 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cfc979640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf80cf220 con 0x7f0cf80c4ac0 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cf659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf806e2c0 0x7f0cf8149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cf659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf806e2c0 0x7f0cf8149d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52830/0 (socket says 172.21.15.131:52830) 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cf659c640 1 -- 172.21.15.131:0/1718355610 learned_addr learned my addr 172.21.15.131:0/1718355610 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cf5d9b640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0cf80c4ac0 0x7f0cf8155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cf6d9d640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf80cdbf0 0x7f0cf815aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cf659c640 1 -- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0cf80c4ac0 msgr2=0x7f0cf8155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cf659c640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0cf80c4ac0 0x7f0cf8155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cf659c640 1 -- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf80cdbf0 msgr2=0x7f0cf815aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cf659c640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf80cdbf0 0x7f0cf815aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cf659c640 1 -- 172.21.15.131:0/1718355610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf80c8990 con 0x7f0cf806e2c0 2024-09-17T02:43:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.375+0000 7f0cf6d9d640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf80cdbf0 0x7f0cf815aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0cf5d9b640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0cf80c4ac0 0x7f0cf8155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0cf659c640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf806e2c0 0x7f0cf8149d40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8000c10 tx=0x7f0ce8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0ce6ffd640 1 -- 172.21.15.131:0/1718355610 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80ac030 con 0x7f0cf806e2c0 2024-09-17T02:43:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0ce6ffd640 1 -- 172.21.15.131:0/1718355610 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ce8005520 con 0x7f0cf806e2c0 2024-09-17T02:43:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0cfc979640 1 -- 172.21.15.131:0/1718355610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf80c7bf0 con 0x7f0cf806e2c0 2024-09-17T02:43:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0ce6ffd640 1 -- 172.21.15.131:0/1718355610 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80048d0 con 0x7f0cf806e2c0 2024-09-17T02:43:30.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0cfc979640 1 -- 172.21.15.131:0/1718355610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf80c83d0 con 0x7f0cf806e2c0 2024-09-17T02:43:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0cfc979640 1 -- 172.21.15.131:0/1718355610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf80c4580 con 0x7f0cf806e2c0 2024-09-17T02:43:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0ce6ffd640 1 -- 172.21.15.131:0/1718355610 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0ce80af070 con 0x7f0cf806e2c0 2024-09-17T02:43:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0ce6ffd640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0cd8073230 0x7f0cd8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0ce6ffd640 1 -- 172.21.15.131:0/1718355610 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f0ce812e8c0 con 0x7f0cf806e2c0 2024-09-17T02:43:30.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.379+0000 7f0cf5d9b640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0cd8073230 0x7f0cd8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.383+0000 7f0cf5d9b640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0cd8073230 0x7f0cd8075740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8150e70 tx=0x7f0cec005ca0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.383+0000 7f0ce6ffd640 1 -- 172.21.15.131:0/1718355610 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0ce8016430 con 0x7f0cf806e2c0 2024-09-17T02:43:30.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 -- 172.21.15.131:0/1718355610 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cf8155f40 con 0x7f0cf806e2c0 2024-09-17T02:43:30.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0ce6ffd640 1 -- 172.21.15.131:0/1718355610 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f0ce80f97d0 con 0x7f0cf806e2c0 2024-09-17T02:43:30.593 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:30.593 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"btime":"2024-09-17T02:42:47:892534+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/434043926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":434043926},{"type":"v1","addr":"172.21.15.189:6836","nonce":434043926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5445,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1496962214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1496962214},{"type":"v1","addr":"172.21.15.131:6833","nonce":1496962214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2196282015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2196282015},{"type":"v1","addr":"172.21.15.131:6835","nonce":2196282015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:42:44.218540+0000","modified":"2024-09-17T02:42:47.307497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/4030750882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":4030750882},{"type":"v1","addr":"172.21.15.189:6837","nonce":4030750882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5463,"qdb_cluster":[5463]},"id":3}]} 2024-09-17T02:43:30.594 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 -- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0cd8073230 msgr2=0x7f0cd8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0cd8073230 0x7f0cd8075740 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8150e70 tx=0x7f0cec005ca0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 -- 172.21.15.131:0/1718355610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf806e2c0 msgr2=0x7f0cf8149d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf806e2c0 0x7f0cf8149d40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8000c10 tx=0x7f0ce8005040 comp rx=0 tx=0).stop 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 -- 172.21.15.131:0/1718355610 shutdown_connections 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0cd8073230 0x7f0cd8075740 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0cf80cdbf0 0x7f0cf815aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0cf80c4ac0 0x7f0cf8155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 --2- 172.21.15.131:0/1718355610 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0cf806e2c0 0x7f0cf8149d40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 -- 172.21.15.131:0/1718355610 >> 172.21.15.131:0/1718355610 conn(0x7f0cf806eea0 msgr2=0x7f0cf80b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 -- 172.21.15.131:0/1718355610 shutdown_connections 2024-09-17T02:43:30.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.591+0000 7f0cfc979640 1 -- 172.21.15.131:0/1718355610 wait complete. 2024-09-17T02:43:30.616 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-09-17T02:43:30.800 INFO:teuthology.orchestra.run.smithi189.stdout:{"message":"","return_code":0} 2024-09-17T02:43:30.800 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T02:43:30.800 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-17T02:43:30.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa95f371080 1 Processor -- start 2024-09-17T02:43:30.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa95f371080 1 -- start start 2024-09-17T02:43:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa95f371080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619636ed9f0 0x5619637931c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa95f371080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5619636ede00 0x5619636ec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa95f371080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 0x56196378da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa95f371080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56196360fa00 con 0x5619637940d0 2024-09-17T02:43:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa95f371080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56196366df40 con 0x5619636ed9f0 2024-09-17T02:43:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa95f371080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x561963708720 con 0x5619636ede00 2024-09-17T02:43:30.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa959649640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 0x56196378da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa959649640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 0x56196378da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52834/0 (socket says 172.21.15.131:52834) 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa959649640 1 -- 172.21.15.131:0/2335715780 learned_addr learned my addr 172.21.15.131:0/2335715780 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa958e48640 1 --2- 172.21.15.131:0/2335715780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619636ed9f0 0x5619637931c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa958647640 1 --2- 172.21.15.131:0/2335715780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5619636ede00 0x5619636ec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa959649640 1 -- 172.21.15.131:0/2335715780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5619636ede00 msgr2=0x5619636ec840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa959649640 1 --2- 172.21.15.131:0/2335715780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5619636ede00 0x5619636ec840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa959649640 1 -- 172.21.15.131:0/2335715780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619636ed9f0 msgr2=0x5619637931c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa959649640 1 --2- 172.21.15.131:0/2335715780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619636ed9f0 0x5619637931c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.871+0000 7fa959649640 1 -- 172.21.15.131:0/2335715780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619635e8c10 con 0x5619637940d0 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa959649640 1 --2- 172.21.15.131:0/2335715780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 0x56196378da00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa94800a350 tx=0x7fa94800a810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806c5f2414053bae server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa957645640 1 -- 172.21.15.131:0/2335715780 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948010070 con 0x5619637940d0 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa957645640 1 -- 172.21.15.131:0/2335715780 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa94800c9f0 con 0x5619637940d0 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa957645640 1 -- 172.21.15.131:0/2335715780 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94800ce40 con 0x5619637940d0 2024-09-17T02:43:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 -- 172.21.15.131:0/2335715780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 msgr2=0x56196378da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 --2- 172.21.15.131:0/2335715780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 0x56196378da00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fa94800a350 tx=0x7fa94800a810 comp rx=0 tx=0).stop 2024-09-17T02:43:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 -- 172.21.15.131:0/2335715780 shutdown_connections 2024-09-17T02:43:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 --2- 172.21.15.131:0/2335715780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 0x56196378da00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 --2- 172.21.15.131:0/2335715780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5619636ede00 0x5619636ec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 --2- 172.21.15.131:0/2335715780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619636ed9f0 0x5619637931c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 -- 172.21.15.131:0/2335715780 >> 172.21.15.131:0/2335715780 conn(0x5619636add00 msgr2=0x5619636eea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 -- 172.21.15.131:0/2335715780 shutdown_connections 2024-09-17T02:43:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 -- 172.21.15.131:0/2335715780 wait complete. 2024-09-17T02:43:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 Processor -- start 2024-09-17T02:43:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 -- start start 2024-09-17T02:43:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619636ed9f0 0x5619638329f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5619636ede00 0x5619636ecd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 0x5619636ed290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x561963792540 con 0x5619637940d0 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5619637922a0 con 0x5619636ed9f0 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa95f371080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5619637927e0 con 0x5619636ede00 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa959649640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 0x5619636ed290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa959649640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 0x5619636ed290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52836/0 (socket says 172.21.15.131:52836) 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa959649640 1 -- 172.21.15.131:0/1985238760 learned_addr learned my addr 172.21.15.131:0/1985238760 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa958e48640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619636ed9f0 0x5619638329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa958647640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5619636ede00 0x5619636ecd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa959649640 1 -- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5619636ede00 msgr2=0x5619636ecd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa959649640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5619636ede00 0x5619636ecd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa959649640 1 -- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619636ed9f0 msgr2=0x5619638329f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa959649640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619636ed9f0 0x5619638329f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa959649640 1 -- 172.21.15.131:0/1985238760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561963700870 con 0x5619637940d0 2024-09-17T02:43:30.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa958647640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5619636ede00 0x5619636ecd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:43:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.875+0000 7fa958e48640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5619636ed9f0 0x5619638329f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:43:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa959649640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 0x5619636ed290 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa94800efd0 tx=0x7fa9480159f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa956643640 1 -- 172.21.15.131:0/1985238760 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948010040 con 0x5619637940d0 2024-09-17T02:43:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa956643640 1 -- 172.21.15.131:0/1985238760 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa94800c700 con 0x5619637940d0 2024-09-17T02:43:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa95f371080 1 -- 172.21.15.131:0/1985238760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561963700b10 con 0x5619637940d0 2024-09-17T02:43:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa956643640 1 -- 172.21.15.131:0/1985238760 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94800c9f0 con 0x5619637940d0 2024-09-17T02:43:30.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa95f371080 1 -- 172.21.15.131:0/1985238760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561963832550 con 0x5619637940d0 2024-09-17T02:43:30.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa956643640 1 -- 172.21.15.131:0/1985238760 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa948013070 con 0x5619637940d0 2024-09-17T02:43:30.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa956643640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa938071c10 0x7fa938074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa956643640 1 -- 172.21.15.131:0/1985238760 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fa948094470 con 0x5619637940d0 2024-09-17T02:43:30.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa95f371080 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x561963787f80 0x561963788390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:30.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa95f371080 1 -- 172.21.15.131:0/1985238760 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52) -- 0x5619637888d0 con 0x561963787f80 2024-09-17T02:43:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa958e48640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa938071c10 0x7fa938074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.879+0000 7fa958647640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x561963787f80 0x561963788390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa958e48640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa938071c10 0x7fa938074120 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa9500ad000 tx=0x7fa950019d60 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa958647640 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x561963787f80 0x561963788390 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa958647640 1 -- 172.21.15.131:0/1985238760 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fa94000b740 con 0x561963787f80 2024-09-17T02:43:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa95f371080 1 -- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x561963787f80 msgr2=0x561963788390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa95f371080 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x561963787f80 0x561963788390 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa95f371080 1 -- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa938071c10 msgr2=0x7fa938074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa95f371080 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa938071c10 0x7fa938074120 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa9500ad000 tx=0x7fa950019d60 comp rx=0 tx=0).stop 2024-09-17T02:43:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa95f371080 1 -- 172.21.15.131:0/1985238760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 msgr2=0x5619636ed290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa95f371080 1 --2- 172.21.15.131:0/1985238760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5619637940d0 0x5619636ed290 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fa94800efd0 tx=0x7fa9480159f0 comp rx=0 tx=0).stop 2024-09-17T02:43:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa959649640 1 -- 172.21.15.131:0/1985238760 reap_dead start 2024-09-17T02:43:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa95f371080 1 -- 172.21.15.131:0/1985238760 shutdown_connections 2024-09-17T02:43:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa95f371080 1 -- 172.21.15.131:0/1985238760 >> 172.21.15.131:0/1985238760 conn(0x5619636add00 msgr2=0x5619637971c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa95f371080 1 -- 172.21.15.131:0/1985238760 shutdown_connections 2024-09-17T02:43:30.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:30.883+0000 7fa95f371080 1 -- 172.21.15.131:0/1985238760 wait complete. 2024-09-17T02:43:30.891 INFO:teuthology.orchestra.run.smithi131.stdout:Yalphaparent_a 2024-09-17T02:43:30.891 DEBUG:teuthology.orchestra.run.smithi131:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-17T02:43:30.946 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:43:30.946 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 1099511627777, 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: "ancestors": [ 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: "dirino": 1099511627776, 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: "dname": "alpha", 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: "version": 4 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: "dirino": 1, 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: "dname": "parent_a", 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: "version": 7 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-17T02:43:30.947 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 8, 2024-09-17T02:43:30.948 INFO:teuthology.orchestra.run.smithi131.stdout: "old_pools": [] 2024-09-17T02:43:30.948 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:43:30.948 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:30.953 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:43:30.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:43:31.066 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:43:31.145 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:43:31.146 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:43:31.146 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:43:31.146 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:43:31.152 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:43:31.152 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:43:31.201 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:43:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.415+0000 7f142f840640 1 -- 172.21.15.131:0/4244889465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142815d6b0 msgr2=0x7f1428156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.415+0000 7f142f840640 1 --2- 172.21.15.131:0/4244889465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142815d6b0 0x7f1428156b00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1420009bf0 tx=0x7f142001cb10 comp rx=0 tx=0).stop 2024-09-17T02:43:31.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.415+0000 7f142f840640 1 -- 172.21.15.131:0/4244889465 shutdown_connections 2024-09-17T02:43:31.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.415+0000 7f142f840640 1 --2- 172.21.15.131:0/4244889465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142815d6b0 0x7f1428156b00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:31.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.415+0000 7f142f840640 1 -- 172.21.15.131:0/4244889465 >> 172.21.15.131:0/4244889465 conn(0x7f14280d2100 msgr2=0x7f14280d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:31.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.415+0000 7f142f840640 1 -- 172.21.15.131:0/4244889465 shutdown_connections 2024-09-17T02:43:31.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.415+0000 7f142f840640 1 -- 172.21.15.131:0/4244889465 wait complete. 2024-09-17T02:43:31.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.415+0000 7f142f840640 1 Processor -- start 2024-09-17T02:43:31.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.415+0000 7f142f840640 1 -- start start 2024-09-17T02:43:31.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142f840640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14281566f0 0x7f14281687f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:31.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142f840640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142815d6b0 0x7f1428168d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:31.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142f840640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14280c50a0 0x7f14280b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:31.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142f840640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1428071700 con 0x7f142815d6b0 2024-09-17T02:43:31.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142f840640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1428071580 con 0x7f14281566f0 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142f840640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f1428071880 con 0x7f14280c50a0 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142cddb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142815d6b0 0x7f1428168d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142cddb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142815d6b0 0x7f1428168d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50252/0 (socket says 172.21.15.131:50252) 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142cddb640 1 -- 172.21.15.131:0/3996909310 learned_addr learned my addr 172.21.15.131:0/3996909310 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142d5dc640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14281566f0 0x7f14281687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142dddd640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14280c50a0 0x7f14280b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142cddb640 1 -- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14280c50a0 msgr2=0x7f14280b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142cddb640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14280c50a0 0x7f14280b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142cddb640 1 -- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14281566f0 msgr2=0x7f14281687f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142cddb640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14281566f0 0x7f14281687f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142cddb640 1 -- 172.21.15.131:0/3996909310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14280b7470 con 0x7f142815d6b0 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142cddb640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142815d6b0 0x7f1428168d30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f141800b3e0 tx=0x7f141800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:31.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f141dffb640 1 -- 172.21.15.131:0/3996909310 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418015020 con 0x7f142815d6b0 2024-09-17T02:43:31.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f141dffb640 1 -- 172.21.15.131:0/3996909310 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1418011920 con 0x7f142815d6b0 2024-09-17T02:43:31.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f141dffb640 1 -- 172.21.15.131:0/3996909310 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418011db0 con 0x7f142815d6b0 2024-09-17T02:43:31.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142f840640 1 -- 172.21.15.131:0/3996909310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14281fa620 con 0x7f142815d6b0 2024-09-17T02:43:31.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142f840640 1 -- 172.21.15.131:0/3996909310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14280b79f0 con 0x7f142815d6b0 2024-09-17T02:43:31.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.419+0000 7f142f840640 1 -- 172.21.15.131:0/3996909310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1428156b00 con 0x7f142815d6b0 2024-09-17T02:43:31.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.423+0000 7f141dffb640 1 -- 172.21.15.131:0/3996909310 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1418018020 con 0x7f142815d6b0 2024-09-17T02:43:31.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.423+0000 7f141dffb640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f140c073230 0x7f140c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:31.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.423+0000 7f141dffb640 1 -- 172.21.15.131:0/3996909310 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f1418093fe0 con 0x7f142815d6b0 2024-09-17T02:43:31.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.423+0000 7f142d5dc640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f140c073230 0x7f140c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:31.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.423+0000 7f142d5dc640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f140c073230 0x7f140c075740 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f14240015f0 tx=0x7f1424009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:31.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.427+0000 7f141dffb640 1 -- 172.21.15.131:0/3996909310 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f141801f030 con 0x7f142815d6b0 2024-09-17T02:43:31.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 -- 172.21.15.131:0/3996909310 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f142815bf20 con 0x7f142815d6b0 2024-09-17T02:43:31.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f141dffb640 1 -- 172.21.15.131:0/3996909310 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) ==== 80+0+0 (secure 0 0 0) 0x7f141805ee70 con 0x7f142815d6b0 2024-09-17T02:43:31.613 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:43:31.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 -- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f140c073230 msgr2=0x7f140c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:31.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f140c073230 0x7f140c075740 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f14240015f0 tx=0x7f1424009040 comp rx=0 tx=0).stop 2024-09-17T02:43:31.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 -- 172.21.15.131:0/3996909310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142815d6b0 msgr2=0x7f1428168d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:31.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142815d6b0 0x7f1428168d30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f141800b3e0 tx=0x7f141800b8a0 comp rx=0 tx=0).stop 2024-09-17T02:43:31.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 -- 172.21.15.131:0/3996909310 shutdown_connections 2024-09-17T02:43:31.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f140c073230 0x7f140c075740 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:31.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14280c50a0 0x7f14280b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:31.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f142815d6b0 0x7f1428168d30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:31.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 --2- 172.21.15.131:0/3996909310 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14281566f0 0x7f14281687f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:31.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 -- 172.21.15.131:0/3996909310 >> 172.21.15.131:0/3996909310 conn(0x7f14280d2100 msgr2=0x7f14280cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:31.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 -- 172.21.15.131:0/3996909310 shutdown_connections 2024-09-17T02:43:31.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:31.611+0000 7f142f840640 1 -- 172.21.15.131:0/3996909310 wait complete. 2024-09-17T02:43:31.635 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:43:31.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:31.635 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:43:31.645 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[40405]: fuse finished with error 0 and tester_r 0 2024-09-17T02:43:31.647 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:43:31.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:31.647 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:43:31.809 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:43:31.809 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:43:31.809 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:43:31.817 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:43:31.817 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:43:31.817 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:43:31.817 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:43:31.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:43:31.857 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:43:31.857 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:43:31.857 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:43:31.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:43:31.865 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:43:32.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.055+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1148543079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7180cf310 msgr2=0x7fa71815ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.055+0000 7fa71dfd0640 1 --2- 172.21.15.131:0/1148543079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7180cf310 0x7fa71815ede0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa718072f20 tx=0x7fa70801c830 comp rx=0 tx=0).stop 2024-09-17T02:43:32.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.055+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1148543079 shutdown_connections 2024-09-17T02:43:32.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.055+0000 7fa71dfd0640 1 --2- 172.21.15.131:0/1148543079 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7180cf310 0x7fa71815ede0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.055+0000 7fa71dfd0640 1 --2- 172.21.15.131:0/1148543079 >> v2:172.21.15.189:3300/0 conn(0x7fa7180d01e0 0x7fa71815b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.055+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1148543079 >> 172.21.15.131:0/1148543079 conn(0x7fa7180d0740 msgr2=0x7fa7180d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:32.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.055+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1148543079 shutdown_connections 2024-09-17T02:43:32.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.055+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1148543079 wait complete. 2024-09-17T02:43:32.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa71dfd0640 1 Processor -- start 2024-09-17T02:43:32.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa71dfd0640 1 -- start start 2024-09-17T02:43:32.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa71dfd0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa71806fd90 0x7fa7181fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa71dfd0640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa7180cf310 0x7fa71814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa71dfd0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7180d01e0 0x7fa718154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa71dfd0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa718168ae0 con 0x7fa7180d01e0 2024-09-17T02:43:32.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa71dfd0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa718168960 con 0x7fa71806fd90 2024-09-17T02:43:32.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa71dfd0640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa718168c60 con 0x7fa7180cf310 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa717fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7180d01e0 0x7fa718154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa717fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7180d01e0 0x7fa718154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50268/0 (socket says 172.21.15.131:50268) 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa717fff640 1 -- 172.21.15.131:0/1469299704 learned_addr learned my addr 172.21.15.131:0/1469299704 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa716ffd640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa7180cf310 0x7fa71814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa7177fe640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa71806fd90 0x7fa7181fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa717fff640 1 -- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa7180cf310 msgr2=0x7fa71814dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa717fff640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa7180cf310 0x7fa71814dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa717fff640 1 -- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa71806fd90 msgr2=0x7fa7181fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa717fff640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa71806fd90 0x7fa7181fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa717fff640 1 -- 172.21.15.131:0/1469299704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa718152de0 con 0x7fa7180d01e0 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa717fff640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7180d01e0 0x7fa718154ae0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa708009850 tx=0x7fa7080a6620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa70ffff640 1 -- 172.21.15.131:0/1469299704 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708004450 con 0x7fa7180d01e0 2024-09-17T02:43:32.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa70ffff640 1 -- 172.21.15.131:0/1469299704 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7080bc070 con 0x7fa7180d01e0 2024-09-17T02:43:32.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1469299704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7181f9cf0 con 0x7fa7180d01e0 2024-09-17T02:43:32.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa70ffff640 1 -- 172.21.15.131:0/1469299704 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708005380 con 0x7fa7180d01e0 2024-09-17T02:43:32.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.059+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1469299704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7181fa2b0 con 0x7fa7180d01e0 2024-09-17T02:43:32.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.063+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1469299704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71815ede0 con 0x7fa7180d01e0 2024-09-17T02:43:32.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.063+0000 7fa70ffff640 1 -- 172.21.15.131:0/1469299704 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa7080046d0 con 0x7fa7180d01e0 2024-09-17T02:43:32.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.063+0000 7fa70ffff640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f8073200 0x7fa6f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.063+0000 7fa70ffff640 1 -- 172.21.15.131:0/1469299704 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7fa70812f2f0 con 0x7fa7180d01e0 2024-09-17T02:43:32.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.063+0000 7fa7177fe640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f8073200 0x7fa6f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.063+0000 7fa7177fe640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f8073200 0x7fa6f8075710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa710005c50 tx=0x7fa710005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:32.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.067+0000 7fa70ffff640 1 -- 172.21.15.131:0/1469299704 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa7080164b0 con 0x7fa7180d01e0 2024-09-17T02:43:32.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.275+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1469299704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa718152b50 con 0x7fa7180d01e0 2024-09-17T02:43:32.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.275+0000 7fa70ffff640 1 -- 172.21.15.131:0/1469299704 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7fa7080fa210 con 0x7fa7180d01e0 2024-09-17T02:43:32.279 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:32.279 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"btime":"2024-09-17T02:42:47:892534+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/434043926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":434043926},{"type":"v1","addr":"172.21.15.189:6836","nonce":434043926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5445,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1496962214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1496962214},{"type":"v1","addr":"172.21.15.131:6833","nonce":1496962214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2196282015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2196282015},{"type":"v1","addr":"172.21.15.131:6835","nonce":2196282015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:42:44.218540+0000","modified":"2024-09-17T02:42:47.307497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/4030750882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":4030750882},{"type":"v1","addr":"172.21.15.189:6837","nonce":4030750882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5463,"qdb_cluster":[5463]},"id":3}]} 2024-09-17T02:43:32.279 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-09-17T02:43:32.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f8073200 msgr2=0x7fa6f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f8073200 0x7fa6f8075710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa710005c50 tx=0x7fa710005be0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1469299704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7180d01e0 msgr2=0x7fa718154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7180d01e0 0x7fa718154ae0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fa708009850 tx=0x7fa7080a6620 comp rx=0 tx=0).stop 2024-09-17T02:43:32.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1469299704 shutdown_connections 2024-09-17T02:43:32.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f8073200 0x7fa6f8075710 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7180d01e0 0x7fa718154ae0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa7180cf310 0x7fa71814dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 --2- 172.21.15.131:0/1469299704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa71806fd90 0x7fa7181fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1469299704 >> 172.21.15.131:0/1469299704 conn(0x7fa7180d0740 msgr2=0x7fa7180d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:32.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1469299704 shutdown_connections 2024-09-17T02:43:32.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.279+0000 7fa71dfd0640 1 -- 172.21.15.131:0/1469299704 wait complete. 2024-09-17T02:43:32.302 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:43:32.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.451+0000 7f5e69051640 1 -- 172.21.15.131:0/941918492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e500062c0 msgr2=0x7f5e5000cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.451+0000 7f5e69051640 1 --2- 172.21.15.131:0/941918492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e500062c0 0x7f5e5000cae0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c006fd0 tx=0x7f5e5c01d260 comp rx=0 tx=0).stop 2024-09-17T02:43:32.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.451+0000 7f5e6359e640 1 -- 172.21.15.131:0/941918492 reap_dead start 2024-09-17T02:43:32.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.451+0000 7f5e69051640 1 -- 172.21.15.131:0/941918492 shutdown_connections 2024-09-17T02:43:32.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.451+0000 7f5e69051640 1 -- 172.21.15.131:0/941918492 >> 172.21.15.131:0/941918492 conn(0x7f5e6406eea0 msgr2=0x7f5e6406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:32.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.451+0000 7f5e69051640 1 -- 172.21.15.131:0/941918492 shutdown_connections 2024-09-17T02:43:32.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.451+0000 7f5e69051640 1 -- 172.21.15.131:0/941918492 wait complete. 2024-09-17T02:43:32.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.451+0000 7f5e69051640 1 Processor -- start 2024-09-17T02:43:32.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.451+0000 7f5e69051640 1 -- start start 2024-09-17T02:43:32.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e69051640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e500062c0 0x7f5e641484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e69051640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5e640c7e70 0x7f5e64148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e69051640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e641f6ae0 0x7f5e641f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e69051640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5e640cbf00 con 0x7f5e500062c0 2024-09-17T02:43:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e69051640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5e640cbd80 con 0x7f5e641f6ae0 2024-09-17T02:43:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e69051640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5e6406e200 con 0x7f5e640c7e70 2024-09-17T02:43:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e62d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e500062c0 0x7f5e641484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e62d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e500062c0 0x7f5e641484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50288/0 (socket says 172.21.15.131:50288) 2024-09-17T02:43:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e62d9d640 1 -- 172.21.15.131:0/2400691700 learned_addr learned my addr 172.21.15.131:0/2400691700 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e6359e640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e641f6ae0 0x7f5e641f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e5a59c640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5e640c7e70 0x7f5e64148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e62d9d640 1 -- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5e640c7e70 msgr2=0x7f5e64148a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e62d9d640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5e640c7e70 0x7f5e64148a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e62d9d640 1 -- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e641f6ae0 msgr2=0x7f5e641f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e62d9d640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e641f6ae0 0x7f5e641f6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e62d9d640 1 -- 172.21.15.131:0/2400691700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e640b1a20 con 0x7f5e500062c0 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e6359e640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e641f6ae0 0x7f5e641f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e62d9d640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e500062c0 0x7f5e641484d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400fa30 tx=0x7f5e5400fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e5bfff640 1 -- 172.21.15.131:0/2400691700 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5400bd70 con 0x7f5e500062c0 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e5bfff640 1 -- 172.21.15.131:0/2400691700 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e54005020 con 0x7f5e500062c0 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e5bfff640 1 -- 172.21.15.131:0/2400691700 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e54005490 con 0x7f5e500062c0 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e69051640 1 -- 172.21.15.131:0/2400691700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e641f7430 con 0x7f5e500062c0 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e69051640 1 -- 172.21.15.131:0/2400691700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e64158ef0 con 0x7f5e500062c0 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e69051640 1 -- 172.21.15.131:0/2400691700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e6406fda0 con 0x7f5e500062c0 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.455+0000 7f5e5bfff640 1 -- 172.21.15.131:0/2400691700 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5e54019030 con 0x7f5e500062c0 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.459+0000 7f5e5bfff640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5e44073200 0x7f5e44075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.459+0000 7f5e5bfff640 1 -- 172.21.15.131:0/2400691700 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f5e54094aa0 con 0x7f5e500062c0 2024-09-17T02:43:32.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.459+0000 7f5e5a59c640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5e44073200 0x7f5e44075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.459+0000 7f5e5a59c640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5e44073200 0x7f5e44075710 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005d10 tx=0x7f5e4c005c80 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:32.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.463+0000 7f5e5bfff640 1 -- 172.21.15.131:0/2400691700 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5e641f7430 con 0x7f5e500062c0 2024-09-17T02:43:32.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 -- 172.21.15.131:0/2400691700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5e6406e9d0 con 0x7f5e500062c0 2024-09-17T02:43:32.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e5bfff640 1 -- 172.21.15.131:0/2400691700 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f5e5405f400 con 0x7f5e500062c0 2024-09-17T02:43:32.657 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:32.657 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"btime":"2024-09-17T02:42:47:892534+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/434043926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":434043926},{"type":"v1","addr":"172.21.15.189:6836","nonce":434043926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5445,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1496962214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1496962214},{"type":"v1","addr":"172.21.15.131:6833","nonce":1496962214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2196282015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2196282015},{"type":"v1","addr":"172.21.15.131:6835","nonce":2196282015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:42:44.218540+0000","modified":"2024-09-17T02:42:47.307497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/4030750882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":4030750882},{"type":"v1","addr":"172.21.15.189:6837","nonce":4030750882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5463,"qdb_cluster":[5463]},"id":3}]} 2024-09-17T02:43:32.657 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-09-17T02:43:32.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 -- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5e44073200 msgr2=0x7f5e44075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5e44073200 0x7f5e44075710 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c005d10 tx=0x7f5e4c005c80 comp rx=0 tx=0).stop 2024-09-17T02:43:32.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 -- 172.21.15.131:0/2400691700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e500062c0 msgr2=0x7f5e641484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e500062c0 0x7f5e641484d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400fa30 tx=0x7f5e5400fef0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 -- 172.21.15.131:0/2400691700 shutdown_connections 2024-09-17T02:43:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5e44073200 0x7f5e44075710 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5e641f6ae0 0x7f5e641f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5e640c7e70 0x7f5e64148a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 --2- 172.21.15.131:0/2400691700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5e500062c0 0x7f5e641484d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 -- 172.21.15.131:0/2400691700 >> 172.21.15.131:0/2400691700 conn(0x7f5e6406eea0 msgr2=0x7f5e640b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 -- 172.21.15.131:0/2400691700 shutdown_connections 2024-09-17T02:43:32.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.655+0000 7f5e69051640 1 -- 172.21.15.131:0/2400691700 wait complete. 2024-09-17T02:43:32.678 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:43:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.815+0000 7f2313261640 1 -- 172.21.15.131:0/3985703247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c0d0b20 msgr2=0x7f230c071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.815+0000 7f2313261640 1 --2- 172.21.15.131:0/3985703247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c0d0b20 0x7f230c071210 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2300009900 tx=0x7f230001c700 comp rx=0 tx=0).stop 2024-09-17T02:43:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.815+0000 7f2313261640 1 -- 172.21.15.131:0/3985703247 shutdown_connections 2024-09-17T02:43:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.815+0000 7f2313261640 1 --2- 172.21.15.131:0/3985703247 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f230c071750 0x7f230c160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.815+0000 7f2313261640 1 --2- 172.21.15.131:0/3985703247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c0d0b20 0x7f230c071210 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.815+0000 7f2313261640 1 -- 172.21.15.131:0/3985703247 >> 172.21.15.131:0/3985703247 conn(0x7f230c0d1130 msgr2=0x7f230c0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:32.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.815+0000 7f2313261640 1 -- 172.21.15.131:0/3985703247 shutdown_connections 2024-09-17T02:43:32.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.815+0000 7f2313261640 1 -- 172.21.15.131:0/3985703247 wait complete. 2024-09-17T02:43:32.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2313261640 1 Processor -- start 2024-09-17T02:43:32.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2313261640 1 -- start start 2024-09-17T02:43:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2313261640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c071750 0x7f230c159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2313261640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f230c0d0710 0x7f230c1f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2313261640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f230c0d0b20 0x7f230c1fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2313261640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f230c169cb0 con 0x7f230c071750 2024-09-17T02:43:32.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2313261640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f230c0ce250 con 0x7f230c0d0710 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2313261640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f230c169e30 con 0x7f230c0d0b20 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2310ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c071750 0x7f230c159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2310ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c071750 0x7f230c159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50312/0 (socket says 172.21.15.131:50312) 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2310ffd640 1 -- 172.21.15.131:0/3890309571 learned_addr learned my addr 172.21.15.131:0/3890309571 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f230bfff640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f230c0d0710 0x7f230c1f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f23117fe640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f230c0d0b20 0x7f230c1fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2310ffd640 1 -- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f230c0d0b20 msgr2=0x7f230c1fa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2310ffd640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f230c0d0b20 0x7f230c1fa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2310ffd640 1 -- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f230c0d0710 msgr2=0x7f230c1f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2310ffd640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f230c0d0710 0x7f230c1f9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2310ffd640 1 -- 172.21.15.131:0/3890309571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c14b0f0 con 0x7f230c071750 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.819+0000 7f2310ffd640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c071750 0x7f230c159960 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f230400ba50 tx=0x7f230400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.823+0000 7f23097fa640 1 -- 172.21.15.131:0/3890309571 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304019070 con 0x7f230c071750 2024-09-17T02:43:32.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.823+0000 7f23097fa640 1 -- 172.21.15.131:0/3890309571 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f230400cd10 con 0x7f230c071750 2024-09-17T02:43:32.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.823+0000 7f2313261640 1 -- 172.21.15.131:0/3890309571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230c15a7b0 con 0x7f230c071750 2024-09-17T02:43:32.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.823+0000 7f23097fa640 1 -- 172.21.15.131:0/3890309571 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23040146a0 con 0x7f230c071750 2024-09-17T02:43:32.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.823+0000 7f2313261640 1 -- 172.21.15.131:0/3890309571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230c14ce60 con 0x7f230c071750 2024-09-17T02:43:32.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.823+0000 7f2313261640 1 -- 172.21.15.131:0/3890309571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f230c160500 con 0x7f230c071750 2024-09-17T02:43:32.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.823+0000 7f23097fa640 1 -- 172.21.15.131:0/3890309571 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f230401c070 con 0x7f230c071750 2024-09-17T02:43:32.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.823+0000 7f23097fa640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f22f8073200 0x7f22f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:32.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.823+0000 7f23097fa640 1 -- 172.21.15.131:0/3890309571 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f2304094050 con 0x7f230c071750 2024-09-17T02:43:32.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.823+0000 7f230bfff640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f22f8073200 0x7f22f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:32.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.823+0000 7f230bfff640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f22f8073200 0x7f22f8075710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f23000ae000 tx=0x7f230001af40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:32.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.827+0000 7f23097fa640 1 -- 172.21.15.131:0/3890309571 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f230405e920 con 0x7f230c071750 2024-09-17T02:43:32.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.995+0000 7f2313261640 1 -- 172.21.15.131:0/3890309571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f230c068e50 con 0x7f230c071750 2024-09-17T02:43:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f23097fa640 1 -- 172.21.15.131:0/3890309571 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16990 (secure 0 0 0) 0x7f2304063350 con 0x7f230c071750 2024-09-17T02:43:33.001 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:33.001 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":52,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:42:45.860734+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:42:41.917939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T02:42:43.186384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:43:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 -- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f22f8073200 msgr2=0x7f22f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f22f8073200 0x7f22f8075710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f23000ae000 tx=0x7f230001af40 comp rx=0 tx=0).stop 2024-09-17T02:43:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 -- 172.21.15.131:0/3890309571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c071750 msgr2=0x7f230c159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c071750 0x7f230c159960 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f230400ba50 tx=0x7f230400bf10 comp rx=0 tx=0).stop 2024-09-17T02:43:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 -- 172.21.15.131:0/3890309571 shutdown_connections 2024-09-17T02:43:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f22f8073200 0x7f22f8075710 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f230c0d0b20 0x7f230c1fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f230c0d0710 0x7f230c1f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 --2- 172.21.15.131:0/3890309571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f230c071750 0x7f230c159960 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 -- 172.21.15.131:0/3890309571 >> 172.21.15.131:0/3890309571 conn(0x7f230c0d1130 msgr2=0x7f230c06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:33.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 -- 172.21.15.131:0/3890309571 shutdown_connections 2024-09-17T02:43:33.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:32.999+0000 7f2313261640 1 -- 172.21.15.131:0/3890309571 wait complete. 2024-09-17T02:43:33.021 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T02:43:33.021 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:43:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.167+0000 7f7b4028c640 1 -- 172.21.15.131:0/3209220027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b3806e3e0 msgr2=0x7f7b3815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.167+0000 7f7b4028c640 1 --2- 172.21.15.131:0/3209220027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b3806e3e0 0x7f7b3815bea0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f7b380cca40 tx=0x7f7b3401cd80 comp rx=0 tx=0).stop 2024-09-17T02:43:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.167+0000 7f7b4028c640 1 -- 172.21.15.131:0/3209220027 shutdown_connections 2024-09-17T02:43:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.167+0000 7f7b4028c640 1 --2- 172.21.15.131:0/3209220027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b3806e3e0 0x7f7b3815bea0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.167+0000 7f7b4028c640 1 --2- 172.21.15.131:0/3209220027 >> v2:172.21.15.189:3300/0 conn(0x7f7b3815c3e0 0x7f7b380cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.167+0000 7f7b4028c640 1 --2- 172.21.15.131:0/3209220027 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7b380ced00 0x7f7b3806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.167+0000 7f7b4028c640 1 -- 172.21.15.131:0/3209220027 >> 172.21.15.131:0/3209220027 conn(0x7f7b3806eea0 msgr2=0x7f7b3806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.167+0000 7f7b4028c640 1 -- 172.21.15.131:0/3209220027 shutdown_connections 2024-09-17T02:43:33.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.167+0000 7f7b4028c640 1 -- 172.21.15.131:0/3209220027 wait complete. 2024-09-17T02:43:33.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b4028c640 1 Processor -- start 2024-09-17T02:43:33.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b4028c640 1 -- start start 2024-09-17T02:43:33.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b4028c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7b3806e3e0 0x7f7b381fa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b4028c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b380ced00 0x7f7b380b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b4028c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b3815c3e0 0x7f7b38148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b4028c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7b380caa00 con 0x7f7b3815c3e0 2024-09-17T02:43:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b4028c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7b380ca880 con 0x7f7b380ced00 2024-09-17T02:43:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b4028c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7b380cab80 con 0x7f7b3806e3e0 2024-09-17T02:43:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b3e829640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b3815c3e0 0x7f7b38148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b3e829640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b3815c3e0 0x7f7b38148980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50332/0 (socket says 172.21.15.131:50332) 2024-09-17T02:43:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b3e829640 1 -- 172.21.15.131:0/969042671 learned_addr learned my addr 172.21.15.131:0/969042671 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b3e028640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7b3806e3e0 0x7f7b381fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b3d827640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b380ced00 0x7f7b380b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b3e829640 1 -- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7b3806e3e0 msgr2=0x7f7b381fa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b3e829640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7b3806e3e0 0x7f7b381fa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b3e829640 1 -- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b380ced00 msgr2=0x7f7b380b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b3e829640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b380ced00 0x7f7b380b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b3e829640 1 -- 172.21.15.131:0/969042671 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b381fa8c0 con 0x7f7b3815c3e0 2024-09-17T02:43:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b3e829640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b3815c3e0 0x7f7b38148980 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800d690 tx=0x7f7b2800db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b2effd640 1 -- 172.21.15.131:0/969042671 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28017070 con 0x7f7b3815c3e0 2024-09-17T02:43:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b2effd640 1 -- 172.21.15.131:0/969042671 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b28005310 con 0x7f7b3815c3e0 2024-09-17T02:43:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b2effd640 1 -- 172.21.15.131:0/969042671 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28009820 con 0x7f7b3815c3e0 2024-09-17T02:43:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.171+0000 7f7b4028c640 1 -- 172.21.15.131:0/969042671 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3814f7c0 con 0x7f7b3815c3e0 2024-09-17T02:43:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.175+0000 7f7b4028c640 1 -- 172.21.15.131:0/969042671 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b38149260 con 0x7f7b3815c3e0 2024-09-17T02:43:33.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.175+0000 7f7b4028c640 1 -- 172.21.15.131:0/969042671 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b3806dea0 con 0x7f7b3815c3e0 2024-09-17T02:43:33.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.175+0000 7f7b2effd640 1 -- 172.21.15.131:0/969042671 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7b280099c0 con 0x7f7b3815c3e0 2024-09-17T02:43:33.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.175+0000 7f7b2effd640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b24073230 0x7f7b24075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.175+0000 7f7b2effd640 1 -- 172.21.15.131:0/969042671 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f7b28094450 con 0x7f7b3815c3e0 2024-09-17T02:43:33.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.175+0000 7f7b3e028640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b24073230 0x7f7b24075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.175+0000 7f7b3e028640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b24073230 0x7f7b24075740 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7b30005fd0 tx=0x7f7b30005d80 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.179+0000 7f7b2effd640 1 -- 172.21.15.131:0/969042671 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7b381fa8c0 con 0x7f7b3815c3e0 2024-09-17T02:43:33.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.395+0000 7f7b4028c640 1 -- 172.21.15.131:0/969042671 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b3815c1c0 con 0x7f7b3815c3e0 2024-09-17T02:43:33.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.395+0000 7f7b2effd640 1 -- 172.21.15.131:0/969042671 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7f7b2805f370 con 0x7f7b3815c3e0 2024-09-17T02:43:33.398 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:33.398 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"btime":"2024-09-17T02:42:47:892534+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/434043926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":434043926},{"type":"v1","addr":"172.21.15.189:6836","nonce":434043926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5445,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1496962214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1496962214},{"type":"v1","addr":"172.21.15.131:6833","nonce":1496962214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2196282015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2196282015},{"type":"v1","addr":"172.21.15.131:6835","nonce":2196282015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:42:44.218540+0000","modified":"2024-09-17T02:42:47.307497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/4030750882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":4030750882},{"type":"v1","addr":"172.21.15.189:6837","nonce":4030750882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5463,"qdb_cluster":[5463]},"id":3}]} 2024-09-17T02:43:33.398 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-09-17T02:43:33.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.395+0000 7f7b4028c640 1 -- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b24073230 msgr2=0x7f7b24075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.395+0000 7f7b4028c640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b24073230 0x7f7b24075740 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7b30005fd0 tx=0x7f7b30005d80 comp rx=0 tx=0).stop 2024-09-17T02:43:33.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.395+0000 7f7b4028c640 1 -- 172.21.15.131:0/969042671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b3815c3e0 msgr2=0x7f7b38148980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.395+0000 7f7b4028c640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b3815c3e0 0x7f7b38148980 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800d690 tx=0x7f7b2800db50 comp rx=0 tx=0).stop 2024-09-17T02:43:33.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.395+0000 7f7b4028c640 1 -- 172.21.15.131:0/969042671 shutdown_connections 2024-09-17T02:43:33.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.399+0000 7f7b4028c640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7b24073230 0x7f7b24075740 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.399+0000 7f7b4028c640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7b3815c3e0 0x7f7b38148980 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.399+0000 7f7b4028c640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7b380ced00 0x7f7b380b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.399+0000 7f7b4028c640 1 --2- 172.21.15.131:0/969042671 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7b3806e3e0 0x7f7b381fa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.399+0000 7f7b4028c640 1 -- 172.21.15.131:0/969042671 >> 172.21.15.131:0/969042671 conn(0x7f7b3806eea0 msgr2=0x7f7b3815ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:33.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.399+0000 7f7b4028c640 1 -- 172.21.15.131:0/969042671 shutdown_connections 2024-09-17T02:43:33.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.399+0000 7f7b4028c640 1 -- 172.21.15.131:0/969042671 wait complete. 2024-09-17T02:43:33.420 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:43:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.571+0000 7ffad220a640 1 -- 172.21.15.131:0/3306715871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc15d710 msgr2=0x7ffacc15e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.571+0000 7ffad220a640 1 --2- 172.21.15.131:0/3306715871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc15d710 0x7ffacc15e840 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009bc0 tx=0x7ffac401cae0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.571+0000 7ffad220a640 1 -- 172.21.15.131:0/3306715871 shutdown_connections 2024-09-17T02:43:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.571+0000 7ffad220a640 1 --2- 172.21.15.131:0/3306715871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc15d710 0x7ffacc15e840 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.571+0000 7ffad220a640 1 -- 172.21.15.131:0/3306715871 >> 172.21.15.131:0/3306715871 conn(0x7ffacc0b42f0 msgr2=0x7ffacc0d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.571+0000 7ffad220a640 1 -- 172.21.15.131:0/3306715871 shutdown_connections 2024-09-17T02:43:33.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.571+0000 7ffad220a640 1 -- 172.21.15.131:0/3306715871 wait complete. 2024-09-17T02:43:33.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.571+0000 7ffad220a640 1 Processor -- start 2024-09-17T02:43:33.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.571+0000 7ffad220a640 1 -- start start 2024-09-17T02:43:33.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffad220a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffacc0cfdb0 0x7ffacc142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffad220a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc15d710 0x7ffacc1f9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffad220a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffacc1f9c70 0x7ffacc1fa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffad220a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc16ad20 con 0x7ffacc15d710 2024-09-17T02:43:33.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffad220a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffacc16aba0 con 0x7ffacc1f9c70 2024-09-17T02:43:33.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffad220a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ffacc16aea0 con 0x7ffacc0cfdb0 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffacaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc15d710 0x7ffacc1f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffacaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc15d710 0x7ffacc1f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50352/0 (socket says 172.21.15.131:50352) 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffacaffd640 1 -- 172.21.15.131:0/2200526020 learned_addr learned my addr 172.21.15.131:0/2200526020 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffacbfff640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffacc1f9c70 0x7ffacc1fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffacb7fe640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffacc0cfdb0 0x7ffacc142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffacaffd640 1 -- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffacc0cfdb0 msgr2=0x7ffacc142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffacaffd640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffacc0cfdb0 0x7ffacc142600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffacaffd640 1 -- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffacc1f9c70 msgr2=0x7ffacc1fa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffacaffd640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffacc1f9c70 0x7ffacc1fa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffacaffd640 1 -- 172.21.15.131:0/2200526020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffacc0c9700 con 0x7ffacc15d710 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffacaffd640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc15d710 0x7ffacc1f9730 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c420 tx=0x7ffac000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffab3fff640 1 -- 172.21.15.131:0/2200526020 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0011790 con 0x7ffacc15d710 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffab3fff640 1 -- 172.21.15.131:0/2200526020 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffac0017070 con 0x7ffacc15d710 2024-09-17T02:43:33.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffad220a640 1 -- 172.21.15.131:0/2200526020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffacc0cbab0 con 0x7ffacc15d710 2024-09-17T02:43:33.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffab3fff640 1 -- 172.21.15.131:0/2200526020 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffac0011ce0 con 0x7ffacc15d710 2024-09-17T02:43:33.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffad220a640 1 -- 172.21.15.131:0/2200526020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffacc143d00 con 0x7ffacc15d710 2024-09-17T02:43:33.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffad220a640 1 -- 172.21.15.131:0/2200526020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffacc0d1370 con 0x7ffacc15d710 2024-09-17T02:43:33.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffab3fff640 1 -- 172.21.15.131:0/2200526020 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffac001c460 con 0x7ffacc15d710 2024-09-17T02:43:33.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.575+0000 7ffab3fff640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffab8073230 0x7ffab8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.579+0000 7ffab3fff640 1 -- 172.21.15.131:0/2200526020 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6130+0+0 (secure 0 0 0) 0x7ffac00941a0 con 0x7ffacc15d710 2024-09-17T02:43:33.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.579+0000 7ffacb7fe640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffab8073230 0x7ffab8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.579+0000 7ffacb7fe640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffab8073230 0x7ffab8075740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00b810 tx=0x7ffabc008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:33.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.583+0000 7ffab3fff640 1 -- 172.21.15.131:0/2200526020 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffac005f010 con 0x7ffacc15d710 2024-09-17T02:43:33.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.787+0000 7ffad220a640 1 -- 172.21.15.131:0/2200526020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffacc0b67f0 con 0x7ffacc15d710 2024-09-17T02:43:33.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.787+0000 7ffab3fff640 1 -- 172.21.15.131:0/2200526020 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5232 (secure 0 0 0) 0x7ffac005e910 con 0x7ffacc15d710 2024-09-17T02:43:33.790 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:33.790 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":27,"btime":"2024-09-17T02:42:47:892534+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/434043926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":434043926},{"type":"v1","addr":"172.21.15.189:6836","nonce":434043926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5445,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1496962214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1496962214},{"type":"v1","addr":"172.21.15.131:6833","nonce":1496962214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2196282015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2196282015},{"type":"v1","addr":"172.21.15.131:6835","nonce":2196282015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:42:44.218540+0000","modified":"2024-09-17T02:42:47.307497+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5463},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5463":{"gid":5463,"name":"b","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6837/4030750882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":4030750882},{"type":"v1","addr":"172.21.15.189:6837","nonce":4030750882}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5463,"qdb_cluster":[5463]},"id":3}]} 2024-09-17T02:43:33.791 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 27 2024-09-17T02:43:33.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 -- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffab8073230 msgr2=0x7ffab8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffab8073230 0x7ffab8075740 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ffabc00b810 tx=0x7ffabc008040 comp rx=0 tx=0).stop 2024-09-17T02:43:33.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 -- 172.21.15.131:0/2200526020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc15d710 msgr2=0x7ffacc1f9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc15d710 0x7ffacc1f9730 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ffac000c420 tx=0x7ffac000c8e0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 -- 172.21.15.131:0/2200526020 shutdown_connections 2024-09-17T02:43:33.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffab8073230 0x7ffab8075740 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffacc1f9c70 0x7ffacc1fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffacc15d710 0x7ffacc1f9730 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 --2- 172.21.15.131:0/2200526020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffacc0cfdb0 0x7ffacc142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 -- 172.21.15.131:0/2200526020 >> 172.21.15.131:0/2200526020 conn(0x7ffacc0b42f0 msgr2=0x7ffacc071ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:33.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 -- 172.21.15.131:0/2200526020 shutdown_connections 2024-09-17T02:43:33.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.791+0000 7ffad220a640 1 -- 172.21.15.131:0/2200526020 wait complete. 2024-09-17T02:43:33.813 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:43:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.959+0000 7fde27d05640 1 -- 172.21.15.131:0/475176340 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fde2015c3e0 msgr2=0x7fde200cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.959+0000 7fde27d05640 1 --2- 172.21.15.131:0/475176340 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fde2015c3e0 0x7fde200cbaf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fde18009bc0 tx=0x7fde1801cba0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.959+0000 7fde27d05640 1 -- 172.21.15.131:0/475176340 shutdown_connections 2024-09-17T02:43:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.959+0000 7fde27d05640 1 --2- 172.21.15.131:0/475176340 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fde2015c3e0 0x7fde200cbaf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.959+0000 7fde27d05640 1 --2- 172.21.15.131:0/475176340 >> v2:172.21.15.189:3300/0 conn(0x7fde2006e5b0 0x7fde2015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.959+0000 7fde27d05640 1 -- 172.21.15.131:0/475176340 >> 172.21.15.131:0/475176340 conn(0x7fde2006eea0 msgr2=0x7fde2006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.959+0000 7fde27d05640 1 -- 172.21.15.131:0/475176340 shutdown_connections 2024-09-17T02:43:33.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.959+0000 7fde27d05640 1 -- 172.21.15.131:0/475176340 wait complete. 2024-09-17T02:43:33.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde27d05640 1 Processor -- start 2024-09-17T02:43:33.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde27d05640 1 -- start start 2024-09-17T02:43:33.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde27d05640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde200ced00 0x7fde2014cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde27d05640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fde2015c3e0 0x7fde2014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde27d05640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde2015dca0 0x7fde201fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde27d05640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fde201687d0 con 0x7fde200ced00 2024-09-17T02:43:33.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde27d05640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fde20168650 con 0x7fde2015dca0 2024-09-17T02:43:33.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde27d05640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fde20168950 con 0x7fde2015c3e0 2024-09-17T02:43:33.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde25aa1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde200ced00 0x7fde2014cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde25aa1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde200ced00 0x7fde2014cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50360/0 (socket says 172.21.15.131:50360) 2024-09-17T02:43:33.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde25aa1640 1 -- 172.21.15.131:0/463974549 learned_addr learned my addr 172.21.15.131:0/463974549 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:33.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde262a2640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde2015dca0 0x7fde201fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde252a0640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fde2015c3e0 0x7fde2014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde25aa1640 1 -- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fde2015c3e0 msgr2=0x7fde2014ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde25aa1640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fde2015c3e0 0x7fde2014ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde25aa1640 1 -- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde2015dca0 msgr2=0x7fde201fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:33.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde25aa1640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde2015dca0 0x7fde201fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:33.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde25aa1640 1 -- 172.21.15.131:0/463974549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2015d8d0 con 0x7fde200ced00 2024-09-17T02:43:33.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.963+0000 7fde25aa1640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde200ced00 0x7fde2014cfa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00c3f0 tx=0x7fde1c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:33.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.967+0000 7fde167fc640 1 -- 172.21.15.131:0/463974549 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c017070 con 0x7fde200ced00 2024-09-17T02:43:33.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.967+0000 7fde167fc640 1 -- 172.21.15.131:0/463974549 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde1c0053c0 con 0x7fde200ced00 2024-09-17T02:43:33.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.967+0000 7fde27d05640 1 -- 172.21.15.131:0/463974549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde201fb850 con 0x7fde200ced00 2024-09-17T02:43:33.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.967+0000 7fde167fc640 1 -- 172.21.15.131:0/463974549 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c003860 con 0x7fde200ced00 2024-09-17T02:43:33.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.967+0000 7fde27d05640 1 -- 172.21.15.131:0/463974549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2014e4e0 con 0x7fde200ced00 2024-09-17T02:43:33.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.967+0000 7fde27d05640 1 -- 172.21.15.131:0/463974549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde201fba20 con 0x7fde200ced00 2024-09-17T02:43:33.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.967+0000 7fde167fc640 1 -- 172.21.15.131:0/463974549 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fde1c01a070 con 0x7fde200ced00 2024-09-17T02:43:33.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.967+0000 7fde167fc640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde04073200 0x7fde04075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:33.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.967+0000 7fde167fc640 1 -- 172.21.15.131:0/463974549 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6130+0+0 (secure 0 0 0) 0x7fde1c093670 con 0x7fde200ced00 2024-09-17T02:43:33.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.967+0000 7fde252a0640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde04073200 0x7fde04075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:33.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.967+0000 7fde252a0640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde04073200 0x7fde04075710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde20155c50 tx=0x7fde10009260 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:33.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:33.971+0000 7fde167fc640 1 -- 172.21.15.131:0/463974549 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fde1c05de90 con 0x7fde200ced00 2024-09-17T02:43:34.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.131+0000 7fde27d05640 1 -- 172.21.15.131:0/463974549 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde2015bea0 con 0x7fde200ced00 2024-09-17T02:43:34.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde167fc640 1 -- 172.21.15.131:0/463974549 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+17039 (secure 0 0 0) 0x7fde1c0628c0 con 0x7fde200ced00 2024-09-17T02:43:34.137 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:34.137 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":53,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:43:33.257043+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:42:41.917939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T02:42:43.186384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.3e","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:43:34.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 -- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde04073200 msgr2=0x7fde04075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:34.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde04073200 0x7fde04075710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde20155c50 tx=0x7fde10009260 comp rx=0 tx=0).stop 2024-09-17T02:43:34.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 -- 172.21.15.131:0/463974549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde200ced00 msgr2=0x7fde2014cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:34.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde200ced00 0x7fde2014cfa0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00c3f0 tx=0x7fde1c00c8b0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 -- 172.21.15.131:0/463974549 shutdown_connections 2024-09-17T02:43:34.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fde04073200 0x7fde04075710 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fde2015dca0 0x7fde201fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fde2015c3e0 0x7fde2014ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 --2- 172.21.15.131:0/463974549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fde200ced00 0x7fde2014cfa0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 -- 172.21.15.131:0/463974549 >> 172.21.15.131:0/463974549 conn(0x7fde2006eea0 msgr2=0x7fde200b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:34.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 -- 172.21.15.131:0/463974549 shutdown_connections 2024-09-17T02:43:34.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.135+0000 7fde27d05640 1 -- 172.21.15.131:0/463974549 wait complete. 2024-09-17T02:43:34.158 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:43:34.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.295+0000 7f84fe81e640 1 -- 172.21.15.131:0/567923276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84e4003a10 msgr2=0x7f84e4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:34.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.295+0000 7f84fe81e640 1 --2- 172.21.15.131:0/567923276 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84e4003a10 0x7f84e4003e80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f84e800ac70 tx=0x7f84e801e090 comp rx=0 tx=0).stop 2024-09-17T02:43:34.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.295+0000 7f84fcdbb640 1 -- 172.21.15.131:0/567923276 reap_dead start 2024-09-17T02:43:34.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 -- 172.21.15.131:0/567923276 shutdown_connections 2024-09-17T02:43:34.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 -- 172.21.15.131:0/567923276 >> 172.21.15.131:0/567923276 conn(0x7f84f806eea0 msgr2=0x7f84f806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:34.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 -- 172.21.15.131:0/567923276 shutdown_connections 2024-09-17T02:43:34.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 -- 172.21.15.131:0/567923276 wait complete. 2024-09-17T02:43:34.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 Processor -- start 2024-09-17T02:43:34.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 -- start start 2024-09-17T02:43:34.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84e4003a10 0x7f84f80c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:34.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84f815f300 0x7f84f81543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:34.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84f815f710 0x7f84f80c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:34.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80cbda0 con 0x7f84f815f710 2024-09-17T02:43:34.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80cbc20 con 0x7f84f815f300 2024-09-17T02:43:34.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f84f80cbf20 con 0x7f84e4003a10 2024-09-17T02:43:34.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fcdbb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84f815f710 0x7f84f80c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:34.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fcdbb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84f815f710 0x7f84f80c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50380/0 (socket says 172.21.15.131:50380) 2024-09-17T02:43:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fcdbb640 1 -- 172.21.15.131:0/881342006 learned_addr learned my addr 172.21.15.131:0/881342006 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84f7fff640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84e4003a10 0x7f84f80c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84ef7fe640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84f815f300 0x7f84f81543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fcdbb640 1 -- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84e4003a10 msgr2=0x7f84f80c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fcdbb640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84e4003a10 0x7f84f80c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fcdbb640 1 -- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84f815f300 msgr2=0x7f84f81543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fcdbb640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84f815f300 0x7f84f81543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fcdbb640 1 -- 172.21.15.131:0/881342006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f80c3bf0 con 0x7f84f815f710 2024-09-17T02:43:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84f7fff640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84e4003a10 0x7f84f80c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:34.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84ef7fe640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84f815f300 0x7f84f81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:34.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fcdbb640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84f815f710 0x7f84f80c4690 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f84f8072ad0 tx=0x7f84f000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:34.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84f57fa640 1 -- 172.21.15.131:0/881342006 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0004b30 con 0x7f84f815f710 2024-09-17T02:43:34.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84f57fa640 1 -- 172.21.15.131:0/881342006 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84f0004dc0 con 0x7f84f815f710 2024-09-17T02:43:34.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 -- 172.21.15.131:0/881342006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f815cfa0 con 0x7f84f815f710 2024-09-17T02:43:34.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84f57fa640 1 -- 172.21.15.131:0/881342006 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f000cd60 con 0x7f84f815f710 2024-09-17T02:43:34.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.299+0000 7f84fe81e640 1 -- 172.21.15.131:0/881342006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f815ae60 con 0x7f84f815f710 2024-09-17T02:43:34.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.303+0000 7f84fe81e640 1 -- 172.21.15.131:0/881342006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f80cab60 con 0x7f84f815f710 2024-09-17T02:43:34.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.303+0000 7f84f57fa640 1 -- 172.21.15.131:0/881342006 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f84f001b030 con 0x7f84f815f710 2024-09-17T02:43:34.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.303+0000 7f84f57fa640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84d8073200 0x7f84d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:34.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.303+0000 7f84f57fa640 1 -- 172.21.15.131:0/881342006 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6130+0+0 (secure 0 0 0) 0x7f84f0093e60 con 0x7f84f815f710 2024-09-17T02:43:34.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.303+0000 7f84f7fff640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84d8073200 0x7f84d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:34.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.303+0000 7f84f7fff640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84d8073200 0x7f84d8075710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f84e800ac70 tx=0x7f84e801ae30 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:34.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.307+0000 7f84f57fa640 1 -- 172.21.15.131:0/881342006 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f84f0022060 con 0x7f84f815f710 2024-09-17T02:43:34.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.499+0000 7f84fe81e640 1 -- 172.21.15.131:0/881342006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f84f806e9d0 con 0x7f84f815f710 2024-09-17T02:43:34.503 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:43:34.499+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:43:34.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.503+0000 7f84f57fa640 1 -- 172.21.15.131:0/881342006 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f84f005ed60 con 0x7f84f815f710 2024-09-17T02:43:34.506 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:43:34.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 -- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84d8073200 msgr2=0x7f84d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:34.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84d8073200 0x7f84d8075710 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f84e800ac70 tx=0x7f84e801ae30 comp rx=0 tx=0).stop 2024-09-17T02:43:34.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 -- 172.21.15.131:0/881342006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84f815f710 msgr2=0x7f84f80c4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:34.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84f815f710 0x7f84f80c4690 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f84f8072ad0 tx=0x7f84f000eec0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 -- 172.21.15.131:0/881342006 shutdown_connections 2024-09-17T02:43:34.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84d8073200 0x7f84d8075710 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84f815f710 0x7f84f80c4690 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84f815f300 0x7f84f81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 --2- 172.21.15.131:0/881342006 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84e4003a10 0x7f84f80c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 -- 172.21.15.131:0/881342006 >> 172.21.15.131:0/881342006 conn(0x7f84f806eea0 msgr2=0x7f84f80b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:34.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 -- 172.21.15.131:0/881342006 shutdown_connections 2024-09-17T02:43:34.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.507+0000 7f84fe81e640 1 -- 172.21.15.131:0/881342006 wait complete. 2024-09-17T02:43:34.523 INFO:tasks.ceph.mds.b.smithi189.stderr: -5955> 2024-09-17T02:42:46.778+0000 7f45b9496640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:43:34.528 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T02:43:34.531 INFO:tasks.ceph.mds.b.smithi189.stderr: -3693> 2024-09-17T02:42:59.662+0000 7f45b548e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 pv4 ap=4 DIRTYPARENT s=0 n(v0 1=1+0)->n(v0 rc2024-09-17T02:42:58.402946+0000 1=1+0) (iauth excl w=1) (ifile lock->xlock w=1) (iquiesce lock w=1) cr={5625=0-4194304@1} caps={5625=pAsLsXs/-@3},l=5625(-1) | ptrwaiter=1 request=1 lock=4 caps=1 dirtyparent=1 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x5569be6e4c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-17T02:43:34.621 INFO:tasks.ceph.mds.b.smithi189.stdout:starting mds.b at 2024-09-17T02:43:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.683+0000 7eff72476640 1 -- 172.21.15.131:0/2636738773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff6c0ced00 msgr2=0x7eff6c06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.683+0000 7eff72476640 1 --2- 172.21.15.131:0/2636738773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff6c0ced00 0x7eff6c06dea0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7eff54009840 tx=0x7eff5401c820 comp rx=0 tx=0).stop 2024-09-17T02:43:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.683+0000 7eff72476640 1 -- 172.21.15.131:0/2636738773 shutdown_connections 2024-09-17T02:43:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.683+0000 7eff72476640 1 --2- 172.21.15.131:0/2636738773 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7eff6c15c5b0 0x7eff6c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.683+0000 7eff72476640 1 --2- 172.21.15.131:0/2636738773 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff6c0ced00 0x7eff6c06dea0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.683+0000 7eff72476640 1 -- 172.21.15.131:0/2636738773 >> 172.21.15.131:0/2636738773 conn(0x7eff6c06eea0 msgr2=0x7eff6c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.683+0000 7eff72476640 1 -- 172.21.15.131:0/2636738773 shutdown_connections 2024-09-17T02:43:34.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.683+0000 7eff72476640 1 -- 172.21.15.131:0/2636738773 wait complete. 2024-09-17T02:43:34.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff72476640 1 Processor -- start 2024-09-17T02:43:34.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff72476640 1 -- start start 2024-09-17T02:43:34.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff72476640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff6c06e3e0 0x7eff6c150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff72476640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c0ced00 0x7eff6c1f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff72476640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7eff6c15c5b0 0x7eff6c1f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff72476640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c168770 con 0x7eff6c06e3e0 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff72476640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7eff6c1685f0 con 0x7eff6c0ced00 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff72476640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7eff6c1688f0 con 0x7eff6c15c5b0 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff6bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff6c06e3e0 0x7eff6c150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff6bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff6c06e3e0 0x7eff6c150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50388/0 (socket says 172.21.15.131:50388) 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff6bfff640 1 -- 172.21.15.131:0/3351681587 learned_addr learned my addr 172.21.15.131:0/3351681587 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff6b7fe640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c0ced00 0x7eff6c1f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff70a13640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7eff6c15c5b0 0x7eff6c1f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff6bfff640 1 -- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7eff6c15c5b0 msgr2=0x7eff6c1f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:34.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff6bfff640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7eff6c15c5b0 0x7eff6c1f6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff6bfff640 1 -- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c0ced00 msgr2=0x7eff6c1f5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:34.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff6bfff640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c0ced00 0x7eff6c1f5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:34.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff6bfff640 1 -- 172.21.15.131:0/3351681587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6c1facc0 con 0x7eff6c06e3e0 2024-09-17T02:43:34.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff6b7fe640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c0ced00 0x7eff6c1f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:34.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff6bfff640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff6c06e3e0 0x7eff6c150e90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7eff5401aed0 tx=0x7eff540a56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:34.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff68ff9640 1 -- 172.21.15.131:0/3351681587 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff540bb070 con 0x7eff6c06e3e0 2024-09-17T02:43:34.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff68ff9640 1 -- 172.21.15.131:0/3351681587 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff54005030 con 0x7eff6c06e3e0 2024-09-17T02:43:34.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff72476640 1 -- 172.21.15.131:0/3351681587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6c1fa140 con 0x7eff6c06e3e0 2024-09-17T02:43:34.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff68ff9640 1 -- 172.21.15.131:0/3351681587 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff540b2040 con 0x7eff6c06e3e0 2024-09-17T02:43:34.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff72476640 1 -- 172.21.15.131:0/3351681587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff6c1fa700 con 0x7eff6c06e3e0 2024-09-17T02:43:34.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.687+0000 7eff72476640 1 -- 172.21.15.131:0/3351681587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff6c06dea0 con 0x7eff6c06e3e0 2024-09-17T02:43:34.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.691+0000 7eff68ff9640 1 -- 172.21.15.131:0/3351681587 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7eff540be070 con 0x7eff6c06e3e0 2024-09-17T02:43:34.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.691+0000 7eff68ff9640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7eff4c073200 0x7eff4c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:34.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.691+0000 7eff68ff9640 1 -- 172.21.15.131:0/3351681587 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6216+0+0 (secure 0 0 0) 0x7eff5412dde0 con 0x7eff6c06e3e0 2024-09-17T02:43:34.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.691+0000 7eff6b7fe640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7eff4c073200 0x7eff4c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:34.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.691+0000 7eff6b7fe640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7eff4c073200 0x7eff4c075710 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff6c06a830 tx=0x7eff5c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:34.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.695+0000 7eff68ff9640 1 -- 172.21.15.131:0/3351681587 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7eff54016410 con 0x7eff6c06e3e0 2024-09-17T02:43:34.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:34.899+0000 7eff72476640 1 -- 172.21.15.131:0/3351681587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7eff6c0cafe0 con 0x7eff6c06e3e0 2024-09-17T02:43:35.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.519+0000 7eff68ff9640 1 -- 172.21.15.131:0/3351681587 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7eff540f8e00 con 0x7eff6c06e3e0 2024-09-17T02:43:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 -- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7eff4c073200 msgr2=0x7eff4c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7eff4c073200 0x7eff4c075710 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7eff6c06a830 tx=0x7eff5c009040 comp rx=0 tx=0).stop 2024-09-17T02:43:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 -- 172.21.15.131:0/3351681587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff6c06e3e0 msgr2=0x7eff6c150e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff6c06e3e0 0x7eff6c150e90 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7eff5401aed0 tx=0x7eff540a56e0 comp rx=0 tx=0).stop 2024-09-17T02:43:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 -- 172.21.15.131:0/3351681587 shutdown_connections 2024-09-17T02:43:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7eff4c073200 0x7eff4c075710 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7eff6c15c5b0 0x7eff6c1f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7eff6c0ced00 0x7eff6c1f5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 --2- 172.21.15.131:0/3351681587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7eff6c06e3e0 0x7eff6c150e90 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:35.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 -- 172.21.15.131:0/3351681587 >> 172.21.15.131:0/3351681587 conn(0x7eff6c06eea0 msgr2=0x7eff6c155630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:35.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 -- 172.21.15.131:0/3351681587 shutdown_connections 2024-09-17T02:43:35.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.523+0000 7eff72476640 1 -- 172.21.15.131:0/3351681587 wait complete. 2024-09-17T02:43:35.547 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T02:43:35.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.707+0000 7f6fd2987640 1 -- 172.21.15.131:0/1619983072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fcc0cbed0 msgr2=0x7f6fcc0cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:35.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.707+0000 7f6fd2987640 1 --2- 172.21.15.131:0/1619983072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fcc0cbed0 0x7f6fcc0cc2e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009bf0 tx=0x7f6fc401cb10 comp rx=0 tx=0).stop 2024-09-17T02:43:35.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.707+0000 7f6fd2987640 1 -- 172.21.15.131:0/1619983072 shutdown_connections 2024-09-17T02:43:35.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.707+0000 7f6fd2987640 1 --2- 172.21.15.131:0/1619983072 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fcc0cbed0 0x7f6fcc0cc2e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.707+0000 7f6fd2987640 1 --2- 172.21.15.131:0/1619983072 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6fcc0b2ef0 0x7f6fcc0cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.707+0000 7f6fd2987640 1 -- 172.21.15.131:0/1619983072 >> 172.21.15.131:0/1619983072 conn(0x7f6fcc0d05a0 msgr2=0x7f6fcc0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.707+0000 7f6fd2987640 1 -- 172.21.15.131:0/1619983072 shutdown_connections 2024-09-17T02:43:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.707+0000 7f6fd2987640 1 -- 172.21.15.131:0/1619983072 wait complete. 2024-09-17T02:43:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.707+0000 7f6fd2987640 1 Processor -- start 2024-09-17T02:43:35.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.707+0000 7f6fd2987640 1 -- start start 2024-09-17T02:43:35.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fd2987640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fcc06f700 0x7f6fcc1fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:35.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fd2987640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fcc0b2ef0 0x7f6fcc1419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:35.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fd2987640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6fcc0cbed0 0x7f6fcc1543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:35.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fd2987640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc073a80 con 0x7f6fcc0b2ef0 2024-09-17T02:43:35.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fd2987640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc073900 con 0x7f6fcc06f700 2024-09-17T02:43:35.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fd2987640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc073c00 con 0x7f6fcc0cbed0 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fcc0b2ef0 0x7f6fcc1419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fcc0b2ef0 0x7f6fcc1419a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50400/0 (socket says 172.21.15.131:50400) 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc37fe640 1 -- 172.21.15.131:0/1626506632 learned_addr learned my addr 172.21.15.131:0/1626506632 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fcbfff640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fcc06f700 0x7f6fcc1fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fd0f24640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6fcc0cbed0 0x7f6fcc1543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc37fe640 1 -- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6fcc0cbed0 msgr2=0x7f6fcc1543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc37fe640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6fcc0cbed0 0x7f6fcc1543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc37fe640 1 -- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fcc06f700 msgr2=0x7f6fcc1fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc37fe640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fcc06f700 0x7f6fcc1fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc37fe640 1 -- 172.21.15.131:0/1626506632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc14cf50 con 0x7f6fcc0b2ef0 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc37fe640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fcc0b2ef0 0x7f6fcc1419a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400c450 tx=0x7f6fb400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc97fa640 1 -- 172.21.15.131:0/1626506632 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40052d0 con 0x7f6fcc0b2ef0 2024-09-17T02:43:35.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc97fa640 1 -- 172.21.15.131:0/1626506632 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb4005470 con 0x7f6fcc0b2ef0 2024-09-17T02:43:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fd2987640 1 -- 172.21.15.131:0/1626506632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fcc150ba0 con 0x7f6fcc0b2ef0 2024-09-17T02:43:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc97fa640 1 -- 172.21.15.131:0/1626506632 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4012870 con 0x7f6fcc0b2ef0 2024-09-17T02:43:35.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fd2987640 1 -- 172.21.15.131:0/1626506632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fcc151160 con 0x7f6fcc0b2ef0 2024-09-17T02:43:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fd2987640 1 -- 172.21.15.131:0/1626506632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fcc0cf6d0 con 0x7f6fcc0b2ef0 2024-09-17T02:43:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc97fa640 1 -- 172.21.15.131:0/1626506632 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6fb4003600 con 0x7f6fcc0b2ef0 2024-09-17T02:43:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc97fa640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6fac073200 0x7f6fac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fc97fa640 1 -- 172.21.15.131:0/1626506632 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6216+0+0 (secure 0 0 0) 0x7f6fb4093e40 con 0x7f6fcc0b2ef0 2024-09-17T02:43:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.711+0000 7f6fcbfff640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6fac073200 0x7f6fac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:35.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.715+0000 7f6fcbfff640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6fac073200 0x7f6fac075710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00ab40 tx=0x7f6fbc00a370 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.719+0000 7f6fc97fa640 1 -- 172.21.15.131:0/1626506632 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6fcc14cf50 con 0x7f6fcc0b2ef0 2024-09-17T02:43:35.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:35.891+0000 7f6fd2987640 1 -- 172.21.15.131:0/1626506632 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6fcc0cdb00 con 0x7f6fcc0b2ef0 2024-09-17T02:43:36.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.519+0000 7f6fc97fa640 1 -- 172.21.15.131:0/1626506632 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) ==== 181+0+0 (secure 0 0 0) 0x7f6fb405ed80 con 0x7f6fcc0b2ef0 2024-09-17T02:43:36.522 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-09-17T02:43:36.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 -- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6fac073200 msgr2=0x7f6fac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:36.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6fac073200 0x7f6fac075710 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00ab40 tx=0x7f6fbc00a370 comp rx=0 tx=0).stop 2024-09-17T02:43:36.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 -- 172.21.15.131:0/1626506632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fcc0b2ef0 msgr2=0x7f6fcc1419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:36.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fcc0b2ef0 0x7f6fcc1419a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400c450 tx=0x7f6fb400c910 comp rx=0 tx=0).stop 2024-09-17T02:43:36.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 -- 172.21.15.131:0/1626506632 shutdown_connections 2024-09-17T02:43:36.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6fac073200 0x7f6fac075710 secure :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00ab40 tx=0x7f6fbc00a370 comp rx=0 tx=0).stop 2024-09-17T02:43:36.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6fcc0cbed0 0x7f6fcc1543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:36.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6fcc0b2ef0 0x7f6fcc1419a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:36.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 --2- 172.21.15.131:0/1626506632 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6fcc06f700 0x7f6fcc1fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:36.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 -- 172.21.15.131:0/1626506632 >> 172.21.15.131:0/1626506632 conn(0x7f6fcc0d05a0 msgr2=0x7f6fcc0b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:36.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 -- 172.21.15.131:0/1626506632 shutdown_connections 2024-09-17T02:43:36.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.523+0000 7f6fd2987640 1 -- 172.21.15.131:0/1626506632 wait complete. 2024-09-17T02:43:36.551 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T02:43:36.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 -- 172.21.15.131:0/3386196212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00c0003a40 msgr2=0x7f00c0003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:36.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 --2- 172.21.15.131:0/3386196212 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00c0003a40 0x7f00c0003eb0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f00d40b3ec0 tx=0x7f00c401cee0 comp rx=0 tx=0).stop 2024-09-17T02:43:36.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00d8f67640 1 -- 172.21.15.131:0/3386196212 reap_dead start 2024-09-17T02:43:36.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 -- 172.21.15.131:0/3386196212 shutdown_connections 2024-09-17T02:43:36.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 -- 172.21.15.131:0/3386196212 >> 172.21.15.131:0/3386196212 conn(0x7f00d40d05a0 msgr2=0x7f00d40d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:36.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 -- 172.21.15.131:0/3386196212 shutdown_connections 2024-09-17T02:43:36.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 -- 172.21.15.131:0/3386196212 wait complete. 2024-09-17T02:43:36.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 Processor -- start 2024-09-17T02:43:36.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 -- start start 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00c0003a40 0x7f00d4072c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00d4167500 0x7f00d415b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f00d41f9440 0x7f00d41663a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f00d4168a50 con 0x7f00d4167500 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f00d41688d0 con 0x7f00c0003a40 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00da9ca640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f00d4168bd0 con 0x7f00d41f9440 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00d4167500 0x7f00d415b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00cbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00d4167500 0x7f00d415b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50424/0 (socket says 172.21.15.131:50424) 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.715+0000 7f00cbfff640 1 -- 172.21.15.131:0/1014047146 learned_addr learned my addr 172.21.15.131:0/1014047146 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00d8f67640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f00d41f9440 0x7f00d41663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00d3fff640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00c0003a40 0x7f00d4072c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00cbfff640 1 -- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f00d41f9440 msgr2=0x7f00d41663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:36.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00cbfff640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f00d41f9440 0x7f00d41663a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:36.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00cbfff640 1 -- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00c0003a40 msgr2=0x7f00d4072c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:36.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00cbfff640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00c0003a40 0x7f00d4072c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:36.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00cbfff640 1 -- 172.21.15.131:0/1014047146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d4073140 con 0x7f00d4167500 2024-09-17T02:43:36.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00cbfff640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00d4167500 0x7f00d415b2e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f00bc011800 tx=0x7f00bc011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:36.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00d17fa640 1 -- 172.21.15.131:0/1014047146 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc004460 con 0x7f00d4167500 2024-09-17T02:43:36.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00d17fa640 1 -- 172.21.15.131:0/1014047146 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00bc016070 con 0x7f00d4167500 2024-09-17T02:43:36.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00d17fa640 1 -- 172.21.15.131:0/1014047146 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00bc00fba0 con 0x7f00d4167500 2024-09-17T02:43:36.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00da9ca640 1 -- 172.21.15.131:0/1014047146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d40c9830 con 0x7f00d4167500 2024-09-17T02:43:36.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00da9ca640 1 -- 172.21.15.131:0/1014047146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d40c9df0 con 0x7f00d4167500 2024-09-17T02:43:36.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00da9ca640 1 -- 172.21.15.131:0/1014047146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d40b41d0 con 0x7f00d4167500 2024-09-17T02:43:36.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00d17fa640 1 -- 172.21.15.131:0/1014047146 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f00bc018030 con 0x7f00d4167500 2024-09-17T02:43:36.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00d17fa640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f00b4073200 0x7f00b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:36.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00d17fa640 1 -- 172.21.15.131:0/1014047146 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5778+0+0 (secure 0 0 0) 0x7f00bc0949e0 con 0x7f00d4167500 2024-09-17T02:43:36.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.719+0000 7f00d3fff640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f00b4073200 0x7f00b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:36.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.723+0000 7f00d3fff640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f00b4073200 0x7f00b4075710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00c40ab000 tx=0x7f00c4019d20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:36.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.723+0000 7f00d17fa640 1 -- 172.21.15.131:0/1014047146 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f00bc05fa40 con 0x7f00d4167500 2024-09-17T02:43:36.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:36.887+0000 7f00da9ca640 1 -- 172.21.15.131:0/1014047146 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f00d41447f0 con 0x7f00d4167500 2024-09-17T02:43:37.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.523+0000 7f00d17fa640 1 -- 172.21.15.131:0/1014047146 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) ==== 169+0+0 (secure 0 0 0) 0x7f00bc05f340 con 0x7f00d4167500 2024-09-17T02:43:37.525 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-09-17T02:43:37.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 -- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f00b4073200 msgr2=0x7f00b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:37.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f00b4073200 0x7f00b4075710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f00c40ab000 tx=0x7f00c4019d20 comp rx=0 tx=0).stop 2024-09-17T02:43:37.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 -- 172.21.15.131:0/1014047146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00d4167500 msgr2=0x7f00d415b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:37.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00d4167500 0x7f00d415b2e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f00bc011800 tx=0x7f00bc011cc0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 -- 172.21.15.131:0/1014047146 shutdown_connections 2024-09-17T02:43:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f00b4073200 0x7f00b4075710 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f00d41f9440 0x7f00d41663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f00d4167500 0x7f00d415b2e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 --2- 172.21.15.131:0/1014047146 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f00c0003a40 0x7f00d4072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 -- 172.21.15.131:0/1014047146 >> 172.21.15.131:0/1014047146 conn(0x7f00d40d05a0 msgr2=0x7f00d41458c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 -- 172.21.15.131:0/1014047146 shutdown_connections 2024-09-17T02:43:37.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.531+0000 7f00da9ca640 1 -- 172.21.15.131:0/1014047146 wait complete. 2024-09-17T02:43:37.560 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T02:43:37.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.719+0000 7f61b5d04640 1 -- 172.21.15.131:0/2170337398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f619c005920 msgr2=0x7f619c005d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:37.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.719+0000 7f61b5d04640 1 --2- 172.21.15.131:0/2170337398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f619c005920 0x7f619c005d30 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f619800c290 tx=0x7f619801f700 comp rx=0 tx=0).stop 2024-09-17T02:43:37.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.719+0000 7f61affff640 1 -- 172.21.15.131:0/2170337398 reap_dead start 2024-09-17T02:43:37.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.719+0000 7f61b5d04640 1 -- 172.21.15.131:0/2170337398 shutdown_connections 2024-09-17T02:43:37.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.719+0000 7f61b5d04640 1 -- 172.21.15.131:0/2170337398 >> 172.21.15.131:0/2170337398 conn(0x7f61b006eea0 msgr2=0x7f61b006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:37.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.719+0000 7f61b5d04640 1 -- 172.21.15.131:0/2170337398 shutdown_connections 2024-09-17T02:43:37.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.719+0000 7f61b5d04640 1 -- 172.21.15.131:0/2170337398 wait complete. 2024-09-17T02:43:37.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.719+0000 7f61b5d04640 1 Processor -- start 2024-09-17T02:43:37.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.719+0000 7f61b5d04640 1 -- start start 2024-09-17T02:43:37.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61b5d04640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f619c005920 0x7f61b0155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:37.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61b5d04640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61b01f3110 0x7f61b01f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:37.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61b5d04640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61b00aea40 0x7f61b00aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:37.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61b5d04640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f61b006fd50 con 0x7f61b01f3110 2024-09-17T02:43:37.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61b5d04640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f61b006fbd0 con 0x7f61b00aea40 2024-09-17T02:43:37.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61b5d04640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f61b006fed0 con 0x7f619c005920 2024-09-17T02:43:37.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61a6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61b01f3110 0x7f61b01f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:37.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61a6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61b01f3110 0x7f61b01f3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50438/0 (socket says 172.21.15.131:50438) 2024-09-17T02:43:37.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61a6ffd640 1 -- 172.21.15.131:0/2956031477 learned_addr learned my addr 172.21.15.131:0/2956031477 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:37.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61affff640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61b00aea40 0x7f61b00aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:37.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61af7fe640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f619c005920 0x7f61b0155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61a6ffd640 1 -- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f619c005920 msgr2=0x7f61b0155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61a6ffd640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f619c005920 0x7f61b0155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61a6ffd640 1 -- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61b00aea40 msgr2=0x7f61b00aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61a6ffd640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61b00aea40 0x7f61b00aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61a6ffd640 1 -- 172.21.15.131:0/2956031477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b0156260 con 0x7f61b01f3110 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61affff640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61b00aea40 0x7f61b00aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61a6ffd640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61b01f3110 0x7f61b01f3520 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f619809c000 tx=0x7f6198001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61acff9640 1 -- 172.21.15.131:0/2956031477 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61980b3030 con 0x7f61b01f3110 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61acff9640 1 -- 172.21.15.131:0/2956031477 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61980c0070 con 0x7f61b01f3110 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61b5d04640 1 -- 172.21.15.131:0/2956031477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b0156540 con 0x7f61b01f3110 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61acff9640 1 -- 172.21.15.131:0/2956031477 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198003600 con 0x7f61b01f3110 2024-09-17T02:43:37.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61b5d04640 1 -- 172.21.15.131:0/2956031477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b0156b00 con 0x7f61b01f3110 2024-09-17T02:43:37.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61b5d04640 1 -- 172.21.15.131:0/2956031477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b00cab60 con 0x7f61b01f3110 2024-09-17T02:43:37.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61acff9640 1 -- 172.21.15.131:0/2956031477 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f61980037a0 con 0x7f61b01f3110 2024-09-17T02:43:37.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61acff9640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f61900731f0 0x7f6190075700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:37.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.723+0000 7f61acff9640 1 -- 172.21.15.131:0/2956031477 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f61981322a0 con 0x7f61b01f3110 2024-09-17T02:43:37.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.727+0000 7f61af7fe640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f61900731f0 0x7f6190075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:37.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.727+0000 7f61af7fe640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f61900731f0 0x7f6190075700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009950 tx=0x7f61a00091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:37.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.731+0000 7f61acff9640 1 -- 172.21.15.131:0/2956031477 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f61980194b0 con 0x7f61b01f3110 2024-09-17T02:43:37.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.919+0000 7f61b5d04640 1 -- 172.21.15.131:0/2956031477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f61b0155ff0 con 0x7f61b01f3110 2024-09-17T02:43:37.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.927+0000 7f61acff9640 1 -- 172.21.15.131:0/2956031477 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f61980fd430 con 0x7f61b01f3110 2024-09-17T02:43:37.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 -- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f61900731f0 msgr2=0x7f6190075700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:37.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f61900731f0 0x7f6190075700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f61a0009950 tx=0x7f61a00091e0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 -- 172.21.15.131:0/2956031477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61b01f3110 msgr2=0x7f61b01f3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:37.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61b01f3110 0x7f61b01f3520 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f619809c000 tx=0x7f6198001d40 comp rx=0 tx=0).stop 2024-09-17T02:43:37.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 -- 172.21.15.131:0/2956031477 shutdown_connections 2024-09-17T02:43:37.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f61900731f0 0x7f6190075700 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61b00aea40 0x7f61b00aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61b01f3110 0x7f61b01f3520 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 --2- 172.21.15.131:0/2956031477 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f619c005920 0x7f61b0155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:37.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 -- 172.21.15.131:0/2956031477 >> 172.21.15.131:0/2956031477 conn(0x7f61b006eea0 msgr2=0x7f61b00b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:37.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 -- 172.21.15.131:0/2956031477 shutdown_connections 2024-09-17T02:43:37.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:37.931+0000 7f61b5d04640 1 -- 172.21.15.131:0/2956031477 wait complete. 2024-09-17T02:43:37.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-09-17T02:43:38.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.103+0000 7feecf238640 1 -- 172.21.15.131:0/2079765069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec806e3e0 msgr2=0x7feec815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:38.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.103+0000 7feecf238640 1 --2- 172.21.15.131:0/2079765069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec806e3e0 0x7feec815bea0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7feebc009900 tx=0x7feebc01c700 comp rx=0 tx=0).stop 2024-09-17T02:43:38.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.103+0000 7feecf238640 1 -- 172.21.15.131:0/2079765069 shutdown_connections 2024-09-17T02:43:38.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.103+0000 7feecf238640 1 --2- 172.21.15.131:0/2079765069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec806e3e0 0x7feec815bea0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.103+0000 7feecf238640 1 --2- 172.21.15.131:0/2079765069 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feec80ced00 0x7feec806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.103+0000 7feecf238640 1 -- 172.21.15.131:0/2079765069 >> 172.21.15.131:0/2079765069 conn(0x7feec806eea0 msgr2=0x7feec806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:38.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.103+0000 7feecf238640 1 -- 172.21.15.131:0/2079765069 shutdown_connections 2024-09-17T02:43:38.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.103+0000 7feecf238640 1 -- 172.21.15.131:0/2079765069 wait complete. 2024-09-17T02:43:38.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feecf238640 1 Processor -- start 2024-09-17T02:43:38.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feecf238640 1 -- start start 2024-09-17T02:43:38.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feecf238640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec806e3e0 0x7feec80b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:38.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feecf238640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feec80ced00 0x7feec8148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:38.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feecf238640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feec815c3e0 0x7feec8154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:38.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feecf238640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feec8168b60 con 0x7feec806e3e0 2024-09-17T02:43:38.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feecf238640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feec81689e0 con 0x7feec80ced00 2024-09-17T02:43:38.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feecf238640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7feec8168ce0 con 0x7feec815c3e0 2024-09-17T02:43:38.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feeccfd4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec806e3e0 0x7feec80b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:38.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feeccfd4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec806e3e0 0x7feec80b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50450/0 (socket says 172.21.15.131:50450) 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feeccfd4640 1 -- 172.21.15.131:0/1112586756 learned_addr learned my addr 172.21.15.131:0/1112586756 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feec7fff640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feec80ced00 0x7feec8148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feecd7d5640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feec815c3e0 0x7feec8154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feeccfd4640 1 -- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feec815c3e0 msgr2=0x7feec8154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feeccfd4640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feec815c3e0 0x7feec8154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feeccfd4640 1 -- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feec80ced00 msgr2=0x7feec8148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feeccfd4640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feec80ced00 0x7feec8148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feeccfd4640 1 -- 172.21.15.131:0/1112586756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec80753b0 con 0x7feec806e3e0 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feeccfd4640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec806e3e0 0x7feec80b4b10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009860 tx=0x7feeb8009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feec57fa640 1 -- 172.21.15.131:0/1112586756 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8005020 con 0x7feec806e3e0 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feec57fa640 1 -- 172.21.15.131:0/1112586756 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeb80051c0 con 0x7feec806e3e0 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feecf238640 1 -- 172.21.15.131:0/1112586756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec81494a0 con 0x7feec806e3e0 2024-09-17T02:43:38.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feec57fa640 1 -- 172.21.15.131:0/1112586756 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb80055f0 con 0x7feec806e3e0 2024-09-17T02:43:38.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.107+0000 7feecf238640 1 -- 172.21.15.131:0/1112586756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec80b39b0 con 0x7feec806e3e0 2024-09-17T02:43:38.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.111+0000 7feecf238640 1 -- 172.21.15.131:0/1112586756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec815bea0 con 0x7feec806e3e0 2024-09-17T02:43:38.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.111+0000 7feec57fa640 1 -- 172.21.15.131:0/1112586756 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feeb80047b0 con 0x7feec806e3e0 2024-09-17T02:43:38.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.111+0000 7feec57fa640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feeb4073230 0x7feeb4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:38.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.111+0000 7feec57fa640 1 -- 172.21.15.131:0/1112586756 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7feeb8095b00 con 0x7feec806e3e0 2024-09-17T02:43:38.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.111+0000 7feec7fff640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feeb4073230 0x7feeb4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:38.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.111+0000 7feec7fff640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feeb4073230 0x7feeb4075740 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feebc0ae000 tx=0x7feebc01af40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:38.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.115+0000 7feec57fa640 1 -- 172.21.15.131:0/1112586756 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feeb8060630 con 0x7feec806e3e0 2024-09-17T02:43:38.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.351+0000 7feecf238640 1 -- 172.21.15.131:0/1112586756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7feec80b3760 con 0x7feec806e3e0 2024-09-17T02:43:38.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.539+0000 7feec57fa640 1 -- 172.21.15.131:0/1112586756 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v153) ==== 149+0+0 (secure 0 0 0) 0x7feeb8095e20 con 0x7feec806e3e0 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 -- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feeb4073230 msgr2=0x7feeb4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feeb4073230 0x7feeb4075740 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feebc0ae000 tx=0x7feebc01af40 comp rx=0 tx=0).stop 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 -- 172.21.15.131:0/1112586756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec806e3e0 msgr2=0x7feec80b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec806e3e0 0x7feec80b4b10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7feeb8009860 tx=0x7feeb8009d20 comp rx=0 tx=0).stop 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 -- 172.21.15.131:0/1112586756 shutdown_connections 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feeb4073230 0x7feeb4075740 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feec815c3e0 0x7feec8154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feec80ced00 0x7feec8148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 --2- 172.21.15.131:0/1112586756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feec806e3e0 0x7feec80b4b10 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 -- 172.21.15.131:0/1112586756 >> 172.21.15.131:0/1112586756 conn(0x7feec806eea0 msgr2=0x7feec80b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 -- 172.21.15.131:0/1112586756 shutdown_connections 2024-09-17T02:43:38.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.543+0000 7feecf238640 1 -- 172.21.15.131:0/1112586756 wait complete. 2024-09-17T02:43:38.572 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-09-17T02:43:38.572 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-09-17T02:43:38.572 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-17T02:43:38.573 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:43:38.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.727+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/1961967792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef0004810 msgr2=0x7fdef0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:38.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.727+0000 7fdf0e4fd640 1 --2- 172.21.15.131:0/1961967792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef0004810 0x7fdef0004c20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00c260 tx=0x7fdefc01f680 comp rx=0 tx=0).stop 2024-09-17T02:43:38.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.727+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/1961967792 shutdown_connections 2024-09-17T02:43:38.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.727+0000 7fdf0e4fd640 1 --2- 172.21.15.131:0/1961967792 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef0004810 0x7fdef0004c20 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.727+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/1961967792 >> 172.21.15.131:0/1961967792 conn(0x7fdf080d13d0 msgr2=0x7fdf080734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:38.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.727+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/1961967792 shutdown_connections 2024-09-17T02:43:38.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.727+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/1961967792 wait complete. 2024-09-17T02:43:38.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf0e4fd640 1 Processor -- start 2024-09-17T02:43:38.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf0e4fd640 1 -- start start 2024-09-17T02:43:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf0e4fd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef0004810 0x7fdf080b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf0e4fd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf0814c340 0x7fdf08155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf0e4fd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf080cb820 0x7fdf0814b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf0e4fd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08151ed0 con 0x7fdef0004810 2024-09-17T02:43:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf0e4fd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf08151d50 con 0x7fdf0814c340 2024-09-17T02:43:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf0e4fd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdf08152050 con 0x7fdf080cb820 2024-09-17T02:43:38.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf07fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef0004810 0x7fdf080b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf07fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef0004810 0x7fdf080b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50474/0 (socket says 172.21.15.131:50474) 2024-09-17T02:43:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf07fff640 1 -- 172.21.15.131:0/116790282 learned_addr learned my addr 172.21.15.131:0/116790282 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf077fe640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf0814c340 0x7fdf08155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf0ca9a640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf080cb820 0x7fdf0814b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:38.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf07fff640 1 -- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf080cb820 msgr2=0x7fdf0814b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:38.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf07fff640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf080cb820 0x7fdf0814b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf07fff640 1 -- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf0814c340 msgr2=0x7fdf08155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:38.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf07fff640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf0814c340 0x7fdf08155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf07fff640 1 -- 172.21.15.131:0/116790282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf080b2160 con 0x7fdef0004810 2024-09-17T02:43:38.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.731+0000 7fdf077fe640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf0814c340 0x7fdf08155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:43:38.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf07fff640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef0004810 0x7fdf080b1c20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdef800d550 tx=0x7fdef800da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:38.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf04ff9640 1 -- 172.21.15.131:0/116790282 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8016020 con 0x7fdef0004810 2024-09-17T02:43:38.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf04ff9640 1 -- 172.21.15.131:0/116790282 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdef8012630 con 0x7fdef0004810 2024-09-17T02:43:38.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/116790282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf0814bdf0 con 0x7fdef0004810 2024-09-17T02:43:38.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf04ff9640 1 -- 172.21.15.131:0/116790282 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdef8012a00 con 0x7fdef0004810 2024-09-17T02:43:38.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/116790282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf08155d60 con 0x7fdef0004810 2024-09-17T02:43:38.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/116790282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf08071090 con 0x7fdef0004810 2024-09-17T02:43:38.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf04ff9640 1 -- 172.21.15.131:0/116790282 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdef8019020 con 0x7fdef0004810 2024-09-17T02:43:38.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf04ff9640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdef4073230 0x7fdef4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:38.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf04ff9640 1 -- 172.21.15.131:0/116790282 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fdef8093b70 con 0x7fdef0004810 2024-09-17T02:43:38.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf077fe640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdef4073230 0x7fdef4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:38.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.735+0000 7fdf077fe640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdef4073230 0x7fdef4075740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0b0000 tx=0x7fdefc01c4c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:38.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.739+0000 7fdf04ff9640 1 -- 172.21.15.131:0/116790282 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdef8020030 con 0x7fdef0004810 2024-09-17T02:43:38.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.911+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/116790282 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf08069c70 con 0x7fdef0004810 2024-09-17T02:43:38.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.911+0000 7fdf04ff9640 1 -- 172.21.15.131:0/116790282 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+13355 (secure 0 0 0) 0x7fdef805ed90 con 0x7fdef0004810 2024-09-17T02:43:38.916 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:38.916 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":57,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:43:37.524753+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.189:6837/4030750882":"2024-09-18T02:43:34.506065+0000","172.21.15.189:6835/4030750882":"2024-09-18T02:43:34.506065+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:43:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdef4073230 msgr2=0x7fdef4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdef4073230 0x7fdef4075740 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdefc0b0000 tx=0x7fdefc01c4c0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/116790282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef0004810 msgr2=0x7fdf080b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef0004810 0x7fdf080b1c20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdef800d550 tx=0x7fdef800da10 comp rx=0 tx=0).stop 2024-09-17T02:43:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/116790282 shutdown_connections 2024-09-17T02:43:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdef4073230 0x7fdef4075740 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf080cb820 0x7fdf0814b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf0814c340 0x7fdf08155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 --2- 172.21.15.131:0/116790282 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdef0004810 0x7fdf080b1c20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/116790282 >> 172.21.15.131:0/116790282 conn(0x7fdf080d13d0 msgr2=0x7fdf08151260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/116790282 shutdown_connections 2024-09-17T02:43:38.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:38.915+0000 7fdf0e4fd640 1 -- 172.21.15.131:0/116790282 wait complete. 2024-09-17T02:43:38.938 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T02:40:41.163614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T02:43:38.939 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T02:43:39.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.091+0000 7f5a12324640 1 -- 172.21.15.131:0/1411894498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c0ced00 msgr2=0x7f5a0c06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:39.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.091+0000 7f5a12324640 1 --2- 172.21.15.131:0/1411894498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c0ced00 0x7f5a0c06dea0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0cca40 tx=0x7f5a0401cd80 comp rx=0 tx=0).stop 2024-09-17T02:43:39.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.091+0000 7f5a12324640 1 -- 172.21.15.131:0/1411894498 shutdown_connections 2024-09-17T02:43:39.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.091+0000 7f5a12324640 1 --2- 172.21.15.131:0/1411894498 >> v2:172.21.15.189:3300/0 conn(0x7f5a0c15c3e0 0x7f5a0c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.091+0000 7f5a12324640 1 --2- 172.21.15.131:0/1411894498 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a0c06e3e0 0x7f5a0c15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.091+0000 7f5a12324640 1 --2- 172.21.15.131:0/1411894498 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c0ced00 0x7f5a0c06dea0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.091+0000 7f5a12324640 1 -- 172.21.15.131:0/1411894498 >> 172.21.15.131:0/1411894498 conn(0x7f5a0c06eea0 msgr2=0x7f5a0c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:39.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.091+0000 7f5a12324640 1 -- 172.21.15.131:0/1411894498 shutdown_connections 2024-09-17T02:43:39.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.091+0000 7f5a12324640 1 -- 172.21.15.131:0/1411894498 wait complete. 2024-09-17T02:43:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a12324640 1 Processor -- start 2024-09-17T02:43:39.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a12324640 1 -- start start 2024-09-17T02:43:39.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a12324640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a0c06e3e0 0x7f5a0c14ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:39.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a12324640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c0ced00 0x7f5a0c154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:39.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a12324640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a0c15c3e0 0x7f5a0c155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:39.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a12324640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c0caa00 con 0x7f5a0c0ced00 2024-09-17T02:43:39.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a12324640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a0c0ca880 con 0x7f5a0c15c3e0 2024-09-17T02:43:39.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a12324640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5a0c0cab80 con 0x7f5a0c06e3e0 2024-09-17T02:43:39.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a0b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c0ced00 0x7f5a0c154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:39.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a0b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c0ced00 0x7f5a0c154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50486/0 (socket says 172.21.15.131:50486) 2024-09-17T02:43:39.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a0b7fe640 1 -- 172.21.15.131:0/325058172 learned_addr learned my addr 172.21.15.131:0/325058172 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:39.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a108c1640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a0c15c3e0 0x7f5a0c155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:39.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a0bfff640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a0c06e3e0 0x7f5a0c14ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a0b7fe640 1 -- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a0c06e3e0 msgr2=0x7f5a0c14ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a0b7fe640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a0c06e3e0 0x7f5a0c14ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a0b7fe640 1 -- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a0c15c3e0 msgr2=0x7f5a0c155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a0b7fe640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a0c15c3e0 0x7f5a0c155520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a0b7fe640 1 -- 172.21.15.131:0/325058172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0c14a7f0 con 0x7f5a0c0ced00 2024-09-17T02:43:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a0b7fe640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c0ced00 0x7f5a0c154fe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f59f400d690 tx=0x7f59f400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.095+0000 7f5a08ff9640 1 -- 172.21.15.131:0/325058172 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f4017070 con 0x7f5a0c0ced00 2024-09-17T02:43:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.099+0000 7f5a08ff9640 1 -- 172.21.15.131:0/325058172 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59f4005310 con 0x7f5a0c0ced00 2024-09-17T02:43:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.099+0000 7f5a12324640 1 -- 172.21.15.131:0/325058172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0c149750 con 0x7f5a0c0ced00 2024-09-17T02:43:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.099+0000 7f5a08ff9640 1 -- 172.21.15.131:0/325058172 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f4009820 con 0x7f5a0c0ced00 2024-09-17T02:43:39.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.099+0000 7f5a12324640 1 -- 172.21.15.131:0/325058172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a0c149d10 con 0x7f5a0c0ced00 2024-09-17T02:43:39.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.099+0000 7f5a12324640 1 -- 172.21.15.131:0/325058172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a0c0cb150 con 0x7f5a0c0ced00 2024-09-17T02:43:39.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.099+0000 7f5a08ff9640 1 -- 172.21.15.131:0/325058172 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f59f40099c0 con 0x7f5a0c0ced00 2024-09-17T02:43:39.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.099+0000 7f5a08ff9640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59ec073200 0x7f59ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:39.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.099+0000 7f5a08ff9640 1 -- 172.21.15.131:0/325058172 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f59f4094190 con 0x7f5a0c0ced00 2024-09-17T02:43:39.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.099+0000 7f5a0bfff640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59ec073200 0x7f59ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:39.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.099+0000 7f5a0bfff640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59ec073200 0x7f59ec075710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5a040af000 tx=0x7f5a0401b5c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:39.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.103+0000 7f5a08ff9640 1 -- 172.21.15.131:0/325058172 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f59f405f320 con 0x7f5a0c0ced00 2024-09-17T02:43:39.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a12324640 1 -- 172.21.15.131:0/325058172 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5a0c06f350 con 0x7f5a0c0ced00 2024-09-17T02:43:39.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a08ff9640 1 -- 172.21.15.131:0/325058172 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) ==== 93+0+10 (secure 0 0 0) 0x7f59f405ec20 con 0x7f5a0c0ced00 2024-09-17T02:43:39.277 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-09-17T02:43:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a12324640 1 -- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59ec073200 msgr2=0x7f59ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a12324640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59ec073200 0x7f59ec075710 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5a040af000 tx=0x7f5a0401b5c0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a12324640 1 -- 172.21.15.131:0/325058172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c0ced00 msgr2=0x7f5a0c154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a12324640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c0ced00 0x7f5a0c154fe0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f59f400d690 tx=0x7f59f400db50 comp rx=0 tx=0).stop 2024-09-17T02:43:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a12324640 1 -- 172.21.15.131:0/325058172 shutdown_connections 2024-09-17T02:43:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a12324640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59ec073200 0x7f59ec075710 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a12324640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a0c15c3e0 0x7f5a0c155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a12324640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a0c0ced00 0x7f5a0c154fe0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a12324640 1 --2- 172.21.15.131:0/325058172 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a0c06e3e0 0x7f5a0c14ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.275+0000 7f5a12324640 1 -- 172.21.15.131:0/325058172 >> 172.21.15.131:0/325058172 conn(0x7f5a0c06eea0 msgr2=0x7f5a0c15e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:39.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.279+0000 7f5a12324640 1 -- 172.21.15.131:0/325058172 shutdown_connections 2024-09-17T02:43:39.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.279+0000 7f5a12324640 1 -- 172.21.15.131:0/325058172 wait complete. 2024-09-17T02:43:39.299 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-09-17T02:43:39.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.451+0000 7f485e505640 1 -- 172.21.15.131:0/2537605128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4840003a60 msgr2=0x7f4840003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:39.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.451+0000 7f485e505640 1 --2- 172.21.15.131:0/2537605128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4840003a60 0x7f4840003ed0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4848009b60 tx=0x7f484801d210 comp rx=0 tx=0).stop 2024-09-17T02:43:39.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.451+0000 7f485e505640 1 -- 172.21.15.131:0/2537605128 shutdown_connections 2024-09-17T02:43:39.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.451+0000 7f485e505640 1 --2- 172.21.15.131:0/2537605128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4840003a60 0x7f4840003ed0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.451+0000 7f485e505640 1 -- 172.21.15.131:0/2537605128 >> 172.21.15.131:0/2537605128 conn(0x7f485806fbe0 msgr2=0x7f485806fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:39.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.451+0000 7f485e505640 1 -- 172.21.15.131:0/2537605128 shutdown_connections 2024-09-17T02:43:39.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.451+0000 7f485e505640 1 -- 172.21.15.131:0/2537605128 wait complete. 2024-09-17T02:43:39.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f485e505640 1 Processor -- start 2024-09-17T02:43:39.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f485e505640 1 -- start start 2024-09-17T02:43:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f485e505640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4840003a60 0x7f48580c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f485e505640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48581563c0 0x7f48581f6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f485e505640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48581f6bd0 0x7f48580c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f485e505640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4858153470 con 0x7f48581563c0 2024-09-17T02:43:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f485e505640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48581532f0 con 0x7f48581f6bd0 2024-09-17T02:43:39.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f485e505640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f48581535f0 con 0x7f4840003a60 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f48577fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48581563c0 0x7f48581f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f48577fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48581563c0 0x7f48581f6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50498/0 (socket says 172.21.15.131:50498) 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f48577fe640 1 -- 172.21.15.131:0/1648696261 learned_addr learned my addr 172.21.15.131:0/1648696261 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f4857fff640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4840003a60 0x7f48580c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f485caa2640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48581f6bd0 0x7f48580c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f48577fe640 1 -- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4840003a60 msgr2=0x7f48580c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f48577fe640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4840003a60 0x7f48580c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f48577fe640 1 -- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48581f6bd0 msgr2=0x7f48580c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f48577fe640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48581f6bd0 0x7f48580c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f48577fe640 1 -- 172.21.15.131:0/1648696261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48580c9290 con 0x7f48581563c0 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f485caa2640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48581f6bd0 0x7f48580c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:39.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f48577fe640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48581563c0 0x7f48581f6690 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f484c00d780 tx=0x7f484c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:39.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f4854ff9640 1 -- 172.21.15.131:0/1648696261 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c018070 con 0x7f48581563c0 2024-09-17T02:43:39.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.455+0000 7f4854ff9640 1 -- 172.21.15.131:0/1648696261 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f484c0050d0 con 0x7f48581563c0 2024-09-17T02:43:39.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.459+0000 7f485e505640 1 -- 172.21.15.131:0/1648696261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48581f7430 con 0x7f48581563c0 2024-09-17T02:43:39.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.459+0000 7f4854ff9640 1 -- 172.21.15.131:0/1648696261 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484c00b6c0 con 0x7f48581563c0 2024-09-17T02:43:39.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.459+0000 7f485e505640 1 -- 172.21.15.131:0/1648696261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48580c9830 con 0x7f48581563c0 2024-09-17T02:43:39.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.459+0000 7f485e505640 1 -- 172.21.15.131:0/1648696261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48581f71a0 con 0x7f48581563c0 2024-09-17T02:43:39.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.459+0000 7f4854ff9640 1 -- 172.21.15.131:0/1648696261 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f484c01b070 con 0x7f48581563c0 2024-09-17T02:43:39.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.459+0000 7f4854ff9640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4844073230 0x7f4844075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:39.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.459+0000 7f4854ff9640 1 -- 172.21.15.131:0/1648696261 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7f484c0936d0 con 0x7f48581563c0 2024-09-17T02:43:39.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.459+0000 7f4857fff640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4844073230 0x7f4844075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:39.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.459+0000 7f4857fff640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4844073230 0x7f4844075740 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f48480ae000 tx=0x7f484801a010 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:39.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.463+0000 7f4854ff9640 1 -- 172.21.15.131:0/1648696261 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f484c05e210 con 0x7f48581563c0 2024-09-17T02:43:39.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:39.683+0000 7f485e505640 1 -- 172.21.15.131:0/1648696261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f485815d280 con 0x7f48581563c0 2024-09-17T02:43:40.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.555+0000 7f4854ff9640 1 -- 172.21.15.131:0/1648696261 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v155) ==== 172+0+0 (secure 0 0 0) 0x7f484c062c40 con 0x7f48581563c0 2024-09-17T02:43:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.555+0000 7f485e505640 1 -- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4844073230 msgr2=0x7f4844075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.555+0000 7f485e505640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4844073230 0x7f4844075740 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f48480ae000 tx=0x7f484801a010 comp rx=0 tx=0).stop 2024-09-17T02:43:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.559+0000 7f485e505640 1 -- 172.21.15.131:0/1648696261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48581563c0 msgr2=0x7f48581f6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.559+0000 7f485e505640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48581563c0 0x7f48581f6690 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f484c00d780 tx=0x7f484c00dc40 comp rx=0 tx=0).stop 2024-09-17T02:43:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.559+0000 7f485e505640 1 -- 172.21.15.131:0/1648696261 shutdown_connections 2024-09-17T02:43:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.559+0000 7f485e505640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4844073230 0x7f4844075740 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.559+0000 7f485e505640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48581f6bd0 0x7f48580c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.559+0000 7f485e505640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48581563c0 0x7f48581f6690 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.559+0000 7f485e505640 1 --2- 172.21.15.131:0/1648696261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4840003a60 0x7f48580c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:40.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.559+0000 7f485e505640 1 -- 172.21.15.131:0/1648696261 >> 172.21.15.131:0/1648696261 conn(0x7f485806fbe0 msgr2=0x7f48580b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:40.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.559+0000 7f485e505640 1 -- 172.21.15.131:0/1648696261 shutdown_connections 2024-09-17T02:43:40.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.559+0000 7f485e505640 1 -- 172.21.15.131:0/1648696261 wait complete. 2024-09-17T02:43:40.583 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T02:43:40.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.735+0000 7fb217a9f640 1 -- 172.21.15.131:0/3045873555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1fc004d50 msgr2=0x7fb1fc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:40.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.735+0000 7fb217a9f640 1 --2- 172.21.15.131:0/3045873555 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1fc004d50 0x7fb1fc005200 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb210072ad0 tx=0x7fb20801d260 comp rx=0 tx=0).stop 2024-09-17T02:43:40.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.735+0000 7fb21603c640 1 -- 172.21.15.131:0/3045873555 reap_dead start 2024-09-17T02:43:40.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.735+0000 7fb217a9f640 1 -- 172.21.15.131:0/3045873555 shutdown_connections 2024-09-17T02:43:40.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.735+0000 7fb217a9f640 1 -- 172.21.15.131:0/3045873555 >> 172.21.15.131:0/3045873555 conn(0x7fb21006eea0 msgr2=0x7fb21006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:40.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.735+0000 7fb217a9f640 1 -- 172.21.15.131:0/3045873555 shutdown_connections 2024-09-17T02:43:40.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.735+0000 7fb217a9f640 1 -- 172.21.15.131:0/3045873555 wait complete. 2024-09-17T02:43:40.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb217a9f640 1 Processor -- start 2024-09-17T02:43:40.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb217a9f640 1 -- start start 2024-09-17T02:43:40.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb217a9f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1fc004d50 0x7fb2101584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb217a9f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb2101f3350 0x7fb2100aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb217a9f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb210158c30 0x7fb210159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb217a9f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb21006fd50 con 0x7fb1fc004d50 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb217a9f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb21006fbd0 con 0x7fb210158c30 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb217a9f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb21006fed0 con 0x7fb2101f3350 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21583b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1fc004d50 0x7fb2101584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21583b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1fc004d50 0x7fb2101584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50518/0 (socket says 172.21.15.131:50518) 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21583b640 1 -- 172.21.15.131:0/3964821068 learned_addr learned my addr 172.21.15.131:0/3964821068 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21503a640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb2101f3350 0x7fb2100aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21603c640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb210158c30 0x7fb210159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21583b640 1 -- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb2101f3350 msgr2=0x7fb2100aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21583b640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb2101f3350 0x7fb2100aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21583b640 1 -- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb210158c30 msgr2=0x7fb210159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21583b640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb210158c30 0x7fb210159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21583b640 1 -- 172.21.15.131:0/3964821068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2101f3e40 con 0x7fb1fc004d50 2024-09-17T02:43:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21603c640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb210158c30 0x7fb210159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb21583b640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1fc004d50 0x7fb2101584e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb20c011a50 tx=0x7fb20c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb2067fc640 1 -- 172.21.15.131:0/3964821068 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c01f070 con 0x7fb1fc004d50 2024-09-17T02:43:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb2067fc640 1 -- 172.21.15.131:0/3964821068 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb20c00fe80 con 0x7fb1fc004d50 2024-09-17T02:43:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb217a9f640 1 -- 172.21.15.131:0/3964821068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2101f4120 con 0x7fb1fc004d50 2024-09-17T02:43:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb2067fc640 1 -- 172.21.15.131:0/3964821068 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20c012690 con 0x7fb1fc004d50 2024-09-17T02:43:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb217a9f640 1 -- 172.21.15.131:0/3964821068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2101595f0 con 0x7fb1fc004d50 2024-09-17T02:43:40.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.739+0000 7fb217a9f640 1 -- 172.21.15.131:0/3964821068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb210073de0 con 0x7fb1fc004d50 2024-09-17T02:43:40.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.743+0000 7fb2067fc640 1 -- 172.21.15.131:0/3964821068 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb20c022070 con 0x7fb1fc004d50 2024-09-17T02:43:40.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.743+0000 7fb2067fc640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb1f4073200 0x7fb1f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:40.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.743+0000 7fb2067fc640 1 -- 172.21.15.131:0/3964821068 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fb20c094b20 con 0x7fb1fc004d50 2024-09-17T02:43:40.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.743+0000 7fb21503a640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb1f4073200 0x7fb1f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:40.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.743+0000 7fb21503a640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb1f4073200 0x7fb1f4075710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb210071480 tx=0x7fb200006b70 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.747+0000 7fb2067fc640 1 -- 172.21.15.131:0/3964821068 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb20c05f6f0 con 0x7fb1fc004d50 2024-09-17T02:43:40.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.915+0000 7fb217a9f640 1 -- 172.21.15.131:0/3964821068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fb21006e9d0 con 0x7fb1fc004d50 2024-09-17T02:43:40.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb2067fc640 1 -- 172.21.15.131:0/3964821068 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+2294 (secure 0 0 0) 0x7fb20c064120 con 0x7fb1fc004d50 2024-09-17T02:43:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T02:43:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T02:43:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced client_mount_timeout 30 2024-09-17T02:43:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_client 20/20 2024-09-17T02:43:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_mgr 20/20 2024-09-17T02:43:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_ms 1/1 2024-09-17T02:43:40.920 INFO:teuthology.orchestra.run.smithi131.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3855.598857 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3598.912856 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3977.024117 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3823.557728 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4643.710284 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4452.598518 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6659.946233 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4539.778076 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds 20/20 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_ms 2/2 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_frag true 2024-09-17T02:43:40.921 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_scatterstat true 2024-09-17T02:43:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T02:43:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_verify_scatter true 2024-09-17T02:43:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T02:43:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T02:43:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T02:43:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced client_mount_timeout 600 2024-09-17T02:43:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_client 20/20 2024-09-17T02:43:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_ms 1/1 2024-09-17T02:43:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T02:43:40.922 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T02:43:40.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 -- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb1f4073200 msgr2=0x7fb1f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:40.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb1f4073200 0x7fb1f4075710 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb210071480 tx=0x7fb200006b70 comp rx=0 tx=0).stop 2024-09-17T02:43:40.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 -- 172.21.15.131:0/3964821068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1fc004d50 msgr2=0x7fb2101584e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:40.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1fc004d50 0x7fb2101584e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb20c011a50 tx=0x7fb20c011f10 comp rx=0 tx=0).stop 2024-09-17T02:43:40.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 -- 172.21.15.131:0/3964821068 shutdown_connections 2024-09-17T02:43:40.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb1f4073200 0x7fb1f4075710 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:40.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb210158c30 0x7fb210159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:40.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb2101f3350 0x7fb2100aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:40.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 --2- 172.21.15.131:0/3964821068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1fc004d50 0x7fb2101584e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:40.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 -- 172.21.15.131:0/3964821068 >> 172.21.15.131:0/3964821068 conn(0x7fb21006eea0 msgr2=0x7fb2100b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:40.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 -- 172.21.15.131:0/3964821068 shutdown_connections 2024-09-17T02:43:40.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:40.919+0000 7fb217a9f640 1 -- 172.21.15.131:0/3964821068 wait complete. 2024-09-17T02:43:40.941 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T02:43:41.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.083+0000 7fd027ffc640 1 -- 172.21.15.131:0/3906232806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd020164310 msgr2=0x7fd020166730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:41.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.083+0000 7fd027ffc640 1 --2- 172.21.15.131:0/3906232806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd020164310 0x7fd020166730 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd0200c9330 tx=0x7fd01801cd50 comp rx=0 tx=0).stop 2024-09-17T02:43:41.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.083+0000 7fd027ffc640 1 -- 172.21.15.131:0/3906232806 shutdown_connections 2024-09-17T02:43:41.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.083+0000 7fd027ffc640 1 --2- 172.21.15.131:0/3906232806 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd020164310 0x7fd020166730 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.083+0000 7fd027ffc640 1 --2- 172.21.15.131:0/3906232806 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd0200b48b0 0x7fd0200cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.083+0000 7fd027ffc640 1 -- 172.21.15.131:0/3906232806 >> 172.21.15.131:0/3906232806 conn(0x7fd0200cf150 msgr2=0x7fd0200cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:41.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.083+0000 7fd027ffc640 1 -- 172.21.15.131:0/3906232806 shutdown_connections 2024-09-17T02:43:41.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.083+0000 7fd027ffc640 1 -- 172.21.15.131:0/3906232806 wait complete. 2024-09-17T02:43:41.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.083+0000 7fd027ffc640 1 Processor -- start 2024-09-17T02:43:41.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.083+0000 7fd027ffc640 1 -- start start 2024-09-17T02:43:41.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd027ffc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd020156bd0 0x7fd020153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:41.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd027ffc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd020164310 0x7fd020153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:41.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd027ffc640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd020151160 0x7fd020151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:41.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd027ffc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd020169870 con 0x7fd020156bd0 2024-09-17T02:43:41.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd027ffc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd0201696f0 con 0x7fd020164310 2024-09-17T02:43:41.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd027ffc640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd0201699f0 con 0x7fd020151160 2024-09-17T02:43:41.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd025d98640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd020156bd0 0x7fd020153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:41.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd025d98640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd020156bd0 0x7fd020153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50532/0 (socket says 172.21.15.131:50532) 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd025d98640 1 -- 172.21.15.131:0/3606962403 learned_addr learned my addr 172.21.15.131:0/3606962403 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd026599640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd020151160 0x7fd020151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd025597640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd020164310 0x7fd020153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd025d98640 1 -- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd020151160 msgr2=0x7fd020151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd025d98640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd020151160 0x7fd020151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd025d98640 1 -- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd020164310 msgr2=0x7fd020153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd025d98640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd020164310 0x7fd020153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd025d98640 1 -- 172.21.15.131:0/3606962403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd020154430 con 0x7fd020156bd0 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd025597640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd020164310 0x7fd020153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd025d98640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd020156bd0 0x7fd020153540 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd01000b410 tx=0x7fd01000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd0167fc640 1 -- 172.21.15.131:0/3606962403 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010015030 con 0x7fd020156bd0 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd0167fc640 1 -- 172.21.15.131:0/3606962403 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd010004020 con 0x7fd020156bd0 2024-09-17T02:43:41.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd027ffc640 1 -- 172.21.15.131:0/3606962403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0201640e0 con 0x7fd020156bd0 2024-09-17T02:43:41.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd0167fc640 1 -- 172.21.15.131:0/3606962403 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd010004470 con 0x7fd020156bd0 2024-09-17T02:43:41.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd027ffc640 1 -- 172.21.15.131:0/3606962403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0201607c0 con 0x7fd020156bd0 2024-09-17T02:43:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd027ffc640 1 -- 172.21.15.131:0/3606962403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0200cca70 con 0x7fd020156bd0 2024-09-17T02:43:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.087+0000 7fd0167fc640 1 -- 172.21.15.131:0/3606962403 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd010018070 con 0x7fd020156bd0 2024-09-17T02:43:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.091+0000 7fd0167fc640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd004073230 0x7fd004075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.091+0000 7fd0167fc640 1 -- 172.21.15.131:0/3606962403 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fd010093c30 con 0x7fd020156bd0 2024-09-17T02:43:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.091+0000 7fd025597640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd004073230 0x7fd004075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.091+0000 7fd025597640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd004073230 0x7fd004075740 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd01c005fd0 tx=0x7fd01c005de0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.095+0000 7fd0167fc640 1 -- 172.21.15.131:0/3606962403 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd01005ee50 con 0x7fd020156bd0 2024-09-17T02:43:41.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.255+0000 7fd027ffc640 1 -- 172.21.15.131:0/3606962403 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd02006a170 con 0x7fd020156bd0 2024-09-17T02:43:41.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd0167fc640 1 -- 172.21.15.131:0/3606962403 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fd01005e750 con 0x7fd020156bd0 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 -- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd004073230 msgr2=0x7fd004075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd004073230 0x7fd004075740 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd01c005fd0 tx=0x7fd01c005de0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 -- 172.21.15.131:0/3606962403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd020156bd0 msgr2=0x7fd020153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd020156bd0 0x7fd020153540 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd01000b410 tx=0x7fd01000b8d0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 -- 172.21.15.131:0/3606962403 shutdown_connections 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd004073230 0x7fd004075740 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd020151160 0x7fd020151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd020164310 0x7fd020153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 --2- 172.21.15.131:0/3606962403 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd020156bd0 0x7fd020153540 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 -- 172.21.15.131:0/3606962403 >> 172.21.15.131:0/3606962403 conn(0x7fd0200cf150 msgr2=0x7fd0200718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 -- 172.21.15.131:0/3606962403 shutdown_connections 2024-09-17T02:43:41.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.259+0000 7fd027ffc640 1 -- 172.21.15.131:0/3606962403 wait complete. 2024-09-17T02:43:41.282 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:43:41.282 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:43:41.357 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:43:41.413 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:43:41.414 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:43:41.414 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:43:41.419 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:43:41.420 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:43:41.420 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:43:41.426 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:43:41.426 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:43:41.426 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:43:41.468 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:43:41.469 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:43:41.469 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:43:41.474 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:43:41.474 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T02:43:41.474 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:43:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.667+0000 7fe09ffb9640 1 -- 172.21.15.131:0/663409656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0840041d0 msgr2=0x7fe084004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.667+0000 7fe09ffb9640 1 --2- 172.21.15.131:0/663409656 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0840041d0 0x7fe084004810 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fe088006fb0 tx=0x7fe08801cec0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.667+0000 7fe09e556640 1 -- 172.21.15.131:0/663409656 reap_dead start 2024-09-17T02:43:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.667+0000 7fe09ffb9640 1 -- 172.21.15.131:0/663409656 shutdown_connections 2024-09-17T02:43:41.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.667+0000 7fe09ffb9640 1 -- 172.21.15.131:0/663409656 >> 172.21.15.131:0/663409656 conn(0x7fe09806eea0 msgr2=0x7fe09806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:41.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.667+0000 7fe09ffb9640 1 -- 172.21.15.131:0/663409656 shutdown_connections 2024-09-17T02:43:41.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.667+0000 7fe09ffb9640 1 -- 172.21.15.131:0/663409656 wait complete. 2024-09-17T02:43:41.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09ffb9640 1 Processor -- start 2024-09-17T02:43:41.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09ffb9640 1 -- start start 2024-09-17T02:43:41.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09ffb9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0840041d0 0x7fe09814ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:41.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09ffb9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0981fae90 0x7fe0980c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:41.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09ffb9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe098154620 0x7fe098154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:41.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09ffb9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe098168c60 con 0x7fe0840041d0 2024-09-17T02:43:41.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09ffb9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe098168ae0 con 0x7fe098154620 2024-09-17T02:43:41.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09ffb9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe098168de0 con 0x7fe0981fae90 2024-09-17T02:43:41.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09dd55640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0840041d0 0x7fe09814ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:41.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09dd55640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0840041d0 0x7fe09814ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55672/0 (socket says 172.21.15.131:55672) 2024-09-17T02:43:41.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09dd55640 1 -- 172.21.15.131:0/768725089 learned_addr learned my addr 172.21.15.131:0/768725089 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:41.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09e556640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe098154620 0x7fe098154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09d554640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0981fae90 0x7fe0980c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09dd55640 1 -- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0981fae90 msgr2=0x7fe0980c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09dd55640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0981fae90 0x7fe0980c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09dd55640 1 -- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe098154620 msgr2=0x7fe098154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09dd55640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe098154620 0x7fe098154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09dd55640 1 -- 172.21.15.131:0/768725089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe098154f70 con 0x7fe0840041d0 2024-09-17T02:43:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09d554640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0981fae90 0x7fe0980c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09dd55640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0840041d0 0x7fe09814ad50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe094003e50 tx=0x7fe094001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe08e7fc640 1 -- 172.21.15.131:0/768725089 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094015070 con 0x7fe0840041d0 2024-09-17T02:43:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe08e7fc640 1 -- 172.21.15.131:0/768725089 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe094011900 con 0x7fe0840041d0 2024-09-17T02:43:41.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe08e7fc640 1 -- 172.21.15.131:0/768725089 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe094011d50 con 0x7fe0840041d0 2024-09-17T02:43:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09ffb9640 1 -- 172.21.15.131:0/768725089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0980c8030 con 0x7fe0840041d0 2024-09-17T02:43:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.671+0000 7fe09ffb9640 1 -- 172.21.15.131:0/768725089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0981fa690 con 0x7fe0840041d0 2024-09-17T02:43:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.675+0000 7fe09ffb9640 1 -- 172.21.15.131:0/768725089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0980c4580 con 0x7fe0840041d0 2024-09-17T02:43:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.675+0000 7fe08e7fc640 1 -- 172.21.15.131:0/768725089 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe094018070 con 0x7fe0840041d0 2024-09-17T02:43:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.675+0000 7fe08e7fc640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe07c073200 0x7fe07c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.675+0000 7fe08e7fc640 1 -- 172.21.15.131:0/768725089 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fe094094db0 con 0x7fe0840041d0 2024-09-17T02:43:41.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.675+0000 7fe09d554640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe07c073200 0x7fe07c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:41.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.675+0000 7fe09d554640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe07c073200 0x7fe07c075710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0880ac000 tx=0x7fe088019d00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:41.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.679+0000 7fe08e7fc640 1 -- 172.21.15.131:0/768725089 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe098154f70 con 0x7fe0840041d0 2024-09-17T02:43:41.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.871+0000 7fe09ffb9640 1 -- 172.21.15.131:0/768725089 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe098073310 con 0x7fe0840041d0 2024-09-17T02:43:41.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.871+0000 7fe08e7fc640 1 -- 172.21.15.131:0/768725089 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3920 (secure 0 0 0) 0x7fe094060090 con 0x7fe0840041d0 2024-09-17T02:43:41.875 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:41.875 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":29,"btime":"2024-09-17T02:43:35:509578+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4949,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4279195239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6835","nonce":4279195239},{"type":"v1","addr":"172.21.15.189:6837","nonce":4279195239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6836/434043926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":434043926},{"type":"v1","addr":"172.21.15.189:6836","nonce":434043926}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5445,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/1496962214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1496962214},{"type":"v1","addr":"172.21.15.131:6833","nonce":1496962214}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2196282015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2196282015},{"type":"v1","addr":"172.21.15.131:6835","nonce":2196282015}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[]} 2024-09-17T02:43:41.875 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 29 2024-09-17T02:43:41.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 -- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe07c073200 msgr2=0x7fe07c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:41.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe07c073200 0x7fe07c075710 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe0880ac000 tx=0x7fe088019d00 comp rx=0 tx=0).stop 2024-09-17T02:43:41.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 -- 172.21.15.131:0/768725089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0840041d0 msgr2=0x7fe09814ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:41.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0840041d0 0x7fe09814ad50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe094003e50 tx=0x7fe094001810 comp rx=0 tx=0).stop 2024-09-17T02:43:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 -- 172.21.15.131:0/768725089 shutdown_connections 2024-09-17T02:43:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe07c073200 0x7fe07c075710 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe098154620 0x7fe098154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe0981fae90 0x7fe0980c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 --2- 172.21.15.131:0/768725089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0840041d0 0x7fe09814ad50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 -- 172.21.15.131:0/768725089 >> 172.21.15.131:0/768725089 conn(0x7fe09806eea0 msgr2=0x7fe0980b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 -- 172.21.15.131:0/768725089 shutdown_connections 2024-09-17T02:43:41.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:41.875+0000 7fe09ffb9640 1 -- 172.21.15.131:0/768725089 wait complete. 2024-09-17T02:43:41.896 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T02:43:41.896 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T02:43:41.896 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T02:43:41.896 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:43:41.896 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T02:43:41.896 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T02:43:41.897 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T02:43:41.897 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:43:41.897 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T02:43:41.897 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T02:43:41.897 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T02:43:41.897 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:43:41.897 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T02:43:41.897 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T02:43:41.898 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T02:43:41.898 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:43:41.910 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T02:43:41.910 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T02:43:41.910 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T02:43:41.911 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T02:43:41.914 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T02:43:41.914 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T02:43:41.915 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T02:43:41.915 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T02:43:41.953 INFO:tasks.ceph.mds.c:Started 2024-09-17T02:43:41.954 DEBUG:teuthology.parallel:result is None 2024-09-17T02:43:41.954 INFO:tasks.ceph.mds.a:Started 2024-09-17T02:43:41.955 DEBUG:teuthology.parallel:result is None 2024-09-17T02:43:41.956 INFO:tasks.ceph.mds.d:Started 2024-09-17T02:43:41.960 DEBUG:teuthology.parallel:result is None 2024-09-17T02:43:41.961 INFO:tasks.ceph.mds.b:Started 2024-09-17T02:43:41.961 DEBUG:teuthology.parallel:result is None 2024-09-17T02:43:41.961 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T02:43:42.041 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:43:42.041 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:43:42.058 INFO:tasks.ceph.mds.d.smithi189.stdout:starting mds.d at 2024-09-17T02:43:42.060 INFO:tasks.ceph.mds.b.smithi189.stdout:starting mds.b at 2024-09-17T02:43:42.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 -- 172.21.15.131:0/2486968980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1700cf710 msgr2=0x7fb17015f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:42.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 --2- 172.21.15.131:0/2486968980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1700cf710 0x7fb17015f290 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb164009bc0 tx=0x7fb16401cac0 comp rx=0 tx=0).stop 2024-09-17T02:43:42.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 -- 172.21.15.131:0/2486968980 shutdown_connections 2024-09-17T02:43:42.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 --2- 172.21.15.131:0/2486968980 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1700cf710 0x7fb17015f290 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:42.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 -- 172.21.15.131:0/2486968980 >> 172.21.15.131:0/2486968980 conn(0x7fb1700d1b10 msgr2=0x7fb1700d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:42.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 -- 172.21.15.131:0/2486968980 shutdown_connections 2024-09-17T02:43:42.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 -- 172.21.15.131:0/2486968980 wait complete. 2024-09-17T02:43:42.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 Processor -- start 2024-09-17T02:43:42.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 -- start start 2024-09-17T02:43:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1700cf710 0x7fb1700cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1700d3470 0x7fb170150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb17015f7d0 0x7fb1701611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb17016cfd0 con 0x7fb1700cf710 2024-09-17T02:43:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb17015fdf0 con 0x7fb1700d3470 2024-09-17T02:43:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb176868640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb17016d150 con 0x7fb17015f7d0 2024-09-17T02:43:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb16ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1700cf710 0x7fb1700cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb16ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1700cf710 0x7fb1700cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55712/0 (socket says 172.21.15.131:55712) 2024-09-17T02:43:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb16ffff640 1 -- 172.21.15.131:0/2861023728 learned_addr learned my addr 172.21.15.131:0/2861023728 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb16f7fe640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1700d3470 0x7fb170150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:42.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.107+0000 7fb174e05640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb17015f7d0 0x7fb1701611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16ffff640 1 -- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb17015f7d0 msgr2=0x7fb1701611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16ffff640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb17015f7d0 0x7fb1701611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16ffff640 1 -- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1700d3470 msgr2=0x7fb170150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16ffff640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1700d3470 0x7fb170150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16ffff640 1 -- 172.21.15.131:0/2861023728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1700cc880 con 0x7fb1700cf710 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb174e05640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb17015f7d0 0x7fb1701611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16ffff640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1700cf710 0x7fb1700cd640 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb1640196d0 tx=0x7fb164009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16cff9640 1 -- 172.21.15.131:0/2861023728 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640b2040 con 0x7fb1700cf710 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16cff9640 1 -- 172.21.15.131:0/2861023728 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb164004040 con 0x7fb1700cf710 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb176868640 1 -- 172.21.15.131:0/2861023728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1700ccb60 con 0x7fb1700cf710 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16cff9640 1 -- 172.21.15.131:0/2861023728 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1640043f0 con 0x7fb1700cf710 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb176868640 1 -- 172.21.15.131:0/2861023728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1700cd120 con 0x7fb1700cf710 2024-09-17T02:43:42.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb176868640 1 -- 172.21.15.131:0/2861023728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb17015f290 con 0x7fb1700cf710 2024-09-17T02:43:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16cff9640 1 -- 172.21.15.131:0/2861023728 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb1640bd060 con 0x7fb1700cf710 2024-09-17T02:43:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16cff9640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb15c073230 0x7fb15c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16cff9640 1 -- 172.21.15.131:0/2861023728 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5335+0+0 (secure 0 0 0) 0x7fb16412da70 con 0x7fb1700cf710 2024-09-17T02:43:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16f7fe640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb15c073230 0x7fb15c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:42.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.111+0000 7fb16f7fe640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb15c073230 0x7fb15c075740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb170150180 tx=0x7fb160009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:42.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.115+0000 7fb16cff9640 1 -- 172.21.15.131:0/2861023728 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb1640f1db0 con 0x7fb1700cf710 2024-09-17T02:43:42.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.295+0000 7fb176868640 1 -- 172.21.15.131:0/2861023728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fb170069550 con 0x7fb1700cf710 2024-09-17T02:43:42.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.587+0000 7fb16cff9640 1 -- 172.21.15.131:0/2861023728 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) ==== 97+0+0 (secure 0 0 0) 0x7fb1640166b0 con 0x7fb1700cf710 2024-09-17T02:43:42.590 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-09-17T02:43:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 -- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb15c073230 msgr2=0x7fb15c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb15c073230 0x7fb15c075740 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb170150180 tx=0x7fb160009040 comp rx=0 tx=0).stop 2024-09-17T02:43:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 -- 172.21.15.131:0/2861023728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1700cf710 msgr2=0x7fb1700cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1700cf710 0x7fb1700cd640 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb1640196d0 tx=0x7fb164009310 comp rx=0 tx=0).stop 2024-09-17T02:43:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 -- 172.21.15.131:0/2861023728 shutdown_connections 2024-09-17T02:43:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb15c073230 0x7fb15c075740 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb17015f7d0 0x7fb1701611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb1700d3470 0x7fb170150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 --2- 172.21.15.131:0/2861023728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1700cf710 0x7fb1700cd640 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 -- 172.21.15.131:0/2861023728 >> 172.21.15.131:0/2861023728 conn(0x7fb1700d1b10 msgr2=0x7fb1700b7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 -- 172.21.15.131:0/2861023728 shutdown_connections 2024-09-17T02:43:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.591+0000 7fb176868640 1 -- 172.21.15.131:0/2861023728 wait complete. 2024-09-17T02:43:42.616 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T02:43:42.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.771+0000 7f322e7e5640 1 -- 172.21.15.131:0/2178529732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3210004d50 msgr2=0x7f3210005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:42.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.771+0000 7f322e7e5640 1 --2- 172.21.15.131:0/2178529732 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3210004d50 0x7f3210005200 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f32280c8090 tx=0x7f322001cad0 comp rx=0 tx=0).stop 2024-09-17T02:43:42.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.771+0000 7f322cd82640 1 -- 172.21.15.131:0/2178529732 reap_dead start 2024-09-17T02:43:42.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.771+0000 7f322e7e5640 1 -- 172.21.15.131:0/2178529732 shutdown_connections 2024-09-17T02:43:42.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.771+0000 7f322e7e5640 1 -- 172.21.15.131:0/2178529732 >> 172.21.15.131:0/2178529732 conn(0x7f322806fbe0 msgr2=0x7f322806fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:42.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.771+0000 7f322e7e5640 1 -- 172.21.15.131:0/2178529732 shutdown_connections 2024-09-17T02:43:42.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.771+0000 7f322e7e5640 1 -- 172.21.15.131:0/2178529732 wait complete. 2024-09-17T02:43:42.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.771+0000 7f322e7e5640 1 Processor -- start 2024-09-17T02:43:42.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.771+0000 7f322e7e5640 1 -- start start 2024-09-17T02:43:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f322e7e5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3210004d50 0x7f32280bc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f322e7e5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f322814e7b0 0x7f322814ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f322e7e5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32281f63e0 0x7f32281f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f322e7e5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f322806db20 con 0x7f322814e7b0 2024-09-17T02:43:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f322e7e5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f32281537e0 con 0x7f3210004d50 2024-09-17T02:43:42.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f322e7e5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f322806dca0 con 0x7f32281f63e0 2024-09-17T02:43:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f32277fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f322814e7b0 0x7f322814ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f32277fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f322814e7b0 0x7f322814ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55736/0 (socket says 172.21.15.131:55736) 2024-09-17T02:43:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f32277fe640 1 -- 172.21.15.131:0/3228239519 learned_addr learned my addr 172.21.15.131:0/3228239519 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f3227fff640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3210004d50 0x7f32280bc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f322cd82640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32281f63e0 0x7f32281f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f32277fe640 1 -- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32281f63e0 msgr2=0x7f32281f67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f32277fe640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32281f63e0 0x7f32281f67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:42.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f32277fe640 1 -- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3210004d50 msgr2=0x7f32280bc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f32277fe640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3210004d50 0x7f32280bc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f32277fe640 1 -- 172.21.15.131:0/3228239519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32281f6d30 con 0x7f322814e7b0 2024-09-17T02:43:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f32277fe640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f322814e7b0 0x7f322814ebc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f321c00e360 tx=0x7f321c00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f3224ff9640 1 -- 172.21.15.131:0/3228239519 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c012850 con 0x7f322814e7b0 2024-09-17T02:43:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f3224ff9640 1 -- 172.21.15.131:0/3228239519 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f321c012ae0 con 0x7f322814e7b0 2024-09-17T02:43:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f322e7e5640 1 -- 172.21.15.131:0/3228239519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f322814d710 con 0x7f322814e7b0 2024-09-17T02:43:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f3224ff9640 1 -- 172.21.15.131:0/3228239519 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c01a400 con 0x7f322814e7b0 2024-09-17T02:43:42.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f322e7e5640 1 -- 172.21.15.131:0/3228239519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f322814dcd0 con 0x7f322814e7b0 2024-09-17T02:43:42.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f322e7e5640 1 -- 172.21.15.131:0/3228239519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3228152be0 con 0x7f322814e7b0 2024-09-17T02:43:42.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.775+0000 7f3224ff9640 1 -- 172.21.15.131:0/3228239519 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f321c00d030 con 0x7f322814e7b0 2024-09-17T02:43:42.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.779+0000 7f3224ff9640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3214073200 0x7f3214075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:42.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.779+0000 7f3224ff9640 1 -- 172.21.15.131:0/3228239519 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5249+0+0 (secure 0 0 0) 0x7f321c0974b0 con 0x7f322814e7b0 2024-09-17T02:43:42.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.779+0000 7f3227fff640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3214073200 0x7f3214075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:42.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.779+0000 7f3227fff640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3214073200 0x7f3214075710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3218009ab0 tx=0x7f32180091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:42.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.783+0000 7f3224ff9640 1 -- 172.21.15.131:0/3228239519 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f32281f6d30 con 0x7f322814e7b0 2024-09-17T02:43:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.995+0000 7f322e7e5640 1 -- 172.21.15.131:0/3228239519 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f322806d1e0 con 0x7f322814e7b0 2024-09-17T02:43:42.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f3224ff9640 1 -- 172.21.15.131:0/3228239519 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v30) ==== 80+0+5304 (secure 0 0 0) 0x7f321c016030 con 0x7f322814e7b0 2024-09-17T02:43:43.000 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:43.000 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:43:43.000 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-09-17T02:43:43.000 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.000 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-09-17T02:43:43.000 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm4U1cHhAATu86VrwT2vMjL/cW2SICMw==", 2024-09-17T02:43:43.000 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.000 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:43:43.000 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:43:43.000 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmVpUSKxAA4l5zGHXg0VRasiyBI/j4ig==", 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.001 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmwHuCJRAAlOj+za8ZmqMI55PsShjKDw==", 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.002 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm9f+9MhAAS8z9jZFYon3nrmu9/4JMPg==", 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD16+hmV9xkBxAAT+IDf2J4awJfqYcx2xDoNA==", 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.003 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD46+hmOhJsGBAAdlA4BSdOPk8BySUTfo1ghg==", 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.004 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD76+hm3mSoKBAAQPoGFy816yWZQW6vPIK6bA==", 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-09-17T02:43:43.005 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD+6+hmpZaPNxAAp0EV6bn5SugDMwA4RnnJwg==", 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAF7OhmgDfVAxAALACK3bk+PK3UhG20b498HQ==", 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.006 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAI7Ohm6w44FBAA90GkelUAbiNjAwn+6IJWcg==", 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.007 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAL7Ohm6S6TIxAArISJW//rmEONG2bC/sfkXA==", 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAO7OhmICwTMxAArAr7DhcO8A0FDSu7VEBehw==", 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:43:43.008 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA==", 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.009 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmduVoBRAAwr50jSXw6sCyo+5t+3iT7A==", 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmBBCDABAAMw9581QyaOxJk5fs35o+ng==", 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:43:43.010 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmgznRCRAA1Z8d32ADoSG+7RHtdPGmTQ==", 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:43:43.011 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmpUfvABAAzclxwjR4E+UxvtkbwVqDXw==", 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.012 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmnWmJGBAA6fzewrm8ELqUAz/MczTOGA==", 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmrTq3ExAAuR8+2bItlpHqqFzlvPxASA==", 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:43:43.013 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:43:43.014 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:43:43.014 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:43:43.014 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.014 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.014 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-09-17T02:43:43.014 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 -- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3214073200 msgr2=0x7f3214075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3214073200 0x7f3214075710 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3218009ab0 tx=0x7f32180091e0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 -- 172.21.15.131:0/3228239519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f322814e7b0 msgr2=0x7f322814ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f322814e7b0 0x7f322814ebc0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f321c00e360 tx=0x7f321c00e820 comp rx=0 tx=0).stop 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 -- 172.21.15.131:0/3228239519 shutdown_connections 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3214073200 0x7f3214075710 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f32281f63e0 0x7f32281f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f322814e7b0 0x7f322814ebc0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 --2- 172.21.15.131:0/3228239519 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3210004d50 0x7f32280bc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 -- 172.21.15.131:0/3228239519 >> 172.21.15.131:0/3228239519 conn(0x7f322806fbe0 msgr2=0x7f32280b3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 -- 172.21.15.131:0/3228239519 shutdown_connections 2024-09-17T02:43:43.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:42.999+0000 7f322e7e5640 1 -- 172.21.15.131:0/3228239519 wait complete. 2024-09-17T02:43:43.020 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T02:43:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 -- 172.21.15.131:0/4131770203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f237c15cfa0 msgr2=0x7f237c15d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 --2- 172.21.15.131:0/4131770203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f237c15cfa0 0x7f237c15d3b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2374009840 tx=0x7f237401c750 comp rx=0 tx=0).stop 2024-09-17T02:43:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 -- 172.21.15.131:0/4131770203 shutdown_connections 2024-09-17T02:43:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 --2- 172.21.15.131:0/4131770203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f237c15cfa0 0x7f237c15d3b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 --2- 172.21.15.131:0/4131770203 >> v2:172.21.15.189:3300/0 conn(0x7f237c1566f0 0x7f237c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 -- 172.21.15.131:0/4131770203 >> 172.21.15.131:0/4131770203 conn(0x7f237c0d2100 msgr2=0x7f237c0d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 -- 172.21.15.131:0/4131770203 shutdown_connections 2024-09-17T02:43:43.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 -- 172.21.15.131:0/4131770203 wait complete. 2024-09-17T02:43:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 Processor -- start 2024-09-17T02:43:43.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 -- start start 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f237c0cc750 0x7f237c1f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.159+0000 7f238363e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f237c1566f0 0x7f237c0c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f238363e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f237c15cfa0 0x7f237c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f238363e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f237c0716e0 con 0x7f237c1566f0 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f238363e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f237c071560 con 0x7f237c0cc750 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f238363e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f237c071860 con 0x7f237c15cfa0 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2380bd9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f237c1566f0 0x7f237c0c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2380bd9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f237c1566f0 0x7f237c0c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55762/0 (socket says 172.21.15.131:55762) 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2380bd9640 1 -- 172.21.15.131:0/1747596433 learned_addr learned my addr 172.21.15.131:0/1747596433 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f23813da640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f237c0cc750 0x7f237c1f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2380bd9640 1 -- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f237c15cfa0 msgr2=0x7f237c0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:43.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2380bd9640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f237c15cfa0 0x7f237c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2380bd9640 1 -- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f237c0cc750 msgr2=0x7f237c1f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2380bd9640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f237c0cc750 0x7f237c1f0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2380bd9640 1 -- 172.21.15.131:0/1747596433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c071cc0 con 0x7f237c1566f0 2024-09-17T02:43:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2380bd9640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f237c1566f0 0x7f237c0c9f40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c400 tx=0x7f236c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2371ffb640 1 -- 172.21.15.131:0/1747596433 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c011790 con 0x7f237c1566f0 2024-09-17T02:43:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2371ffb640 1 -- 172.21.15.131:0/1747596433 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f236c017070 con 0x7f237c1566f0 2024-09-17T02:43:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f238363e640 1 -- 172.21.15.131:0/1747596433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237c1f0ff0 con 0x7f237c1566f0 2024-09-17T02:43:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2371ffb640 1 -- 172.21.15.131:0/1747596433 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c011cc0 con 0x7f237c1566f0 2024-09-17T02:43:43.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f238363e640 1 -- 172.21.15.131:0/1747596433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237c0c9180 con 0x7f237c1566f0 2024-09-17T02:43:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f238363e640 1 -- 172.21.15.131:0/1747596433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237c0c5660 con 0x7f237c1566f0 2024-09-17T02:43:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2371ffb640 1 -- 172.21.15.131:0/1747596433 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f236c01c460 con 0x7f237c1566f0 2024-09-17T02:43:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2371ffb640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2360073230 0x7f2360075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f2371ffb640 1 -- 172.21.15.131:0/1747596433 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5249+0+0 (secure 0 0 0) 0x7f236c093e30 con 0x7f237c1566f0 2024-09-17T02:43:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.163+0000 7f23813da640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2360073230 0x7f2360075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:43.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.167+0000 7f23813da640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2360073230 0x7f2360075740 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2378005c50 tx=0x7f2378005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:43.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.167+0000 7f2371ffb640 1 -- 172.21.15.131:0/1747596433 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f237c1f0ff0 con 0x7f237c1566f0 2024-09-17T02:43:43.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.339+0000 7f238363e640 1 -- 172.21.15.131:0/1747596433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f237c15d3b0 con 0x7f237c1566f0 2024-09-17T02:43:43.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f2371ffb640 1 -- 172.21.15.131:0/1747596433 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) ==== 84+0+65 (secure 0 0 0) 0x7f236c05f010 con 0x7f237c1566f0 2024-09-17T02:43:43.344 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:43.344 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:43:43.344 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:43:43.344 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-09-17T02:43:43.344 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-09-17T02:43:43.344 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:43.344 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:43:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 -- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2360073230 msgr2=0x7f2360075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2360073230 0x7f2360075740 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2378005c50 tx=0x7f2378005be0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 -- 172.21.15.131:0/1747596433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f237c1566f0 msgr2=0x7f237c0c9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f237c1566f0 0x7f237c0c9f40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c400 tx=0x7f236c00c8c0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 -- 172.21.15.131:0/1747596433 shutdown_connections 2024-09-17T02:43:43.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2360073230 0x7f2360075740 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f237c15cfa0 0x7f237c0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f237c1566f0 0x7f237c0c9f40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 --2- 172.21.15.131:0/1747596433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f237c0cc750 0x7f237c1f0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 -- 172.21.15.131:0/1747596433 >> 172.21.15.131:0/1747596433 conn(0x7f237c0d2100 msgr2=0x7f237c152980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:43.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 -- 172.21.15.131:0/1747596433 shutdown_connections 2024-09-17T02:43:43.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.343+0000 7f238363e640 1 -- 172.21.15.131:0/1747596433 wait complete. 2024-09-17T02:43:43.366 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T02:43:43.366 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T02:43:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.523+0000 7f8c99a72640 1 -- 172.21.15.131:0/7507709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c9406f460 msgr2=0x7f8c9406f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:43.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.523+0000 7f8c99a72640 1 --2- 172.21.15.131:0/7507709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c9406f460 0x7f8c9406f870 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8c94072a00 tx=0x7f8c8c01c830 comp rx=0 tx=0).stop 2024-09-17T02:43:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.523+0000 7f8c99a72640 1 -- 172.21.15.131:0/7507709 shutdown_connections 2024-09-17T02:43:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.523+0000 7f8c99a72640 1 --2- 172.21.15.131:0/7507709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c9406f460 0x7f8c9406f870 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.523+0000 7f8c99a72640 1 --2- 172.21.15.131:0/7507709 >> v2:172.21.15.189:3300/0 conn(0x7f8c941606d0 0x7f8c940cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.523+0000 7f8c99a72640 1 -- 172.21.15.131:0/7507709 >> 172.21.15.131:0/7507709 conn(0x7f8c940d05a0 msgr2=0x7f8c940d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.523+0000 7f8c99a72640 1 -- 172.21.15.131:0/7507709 shutdown_connections 2024-09-17T02:43:43.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.523+0000 7f8c99a72640 1 -- 172.21.15.131:0/7507709 wait complete. 2024-09-17T02:43:43.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c99a72640 1 Processor -- start 2024-09-17T02:43:43.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c99a72640 1 -- start start 2024-09-17T02:43:43.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c99a72640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c9406f460 0x7f8c94156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:43.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c99a72640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c940cc870 0x7f8c94159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c99a72640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94138f40 0x7f8c941598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c99a72640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8c94169c90 con 0x7f8c940cc870 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c99a72640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8c94169b10 con 0x7f8c94138f40 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c99a72640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8c94169e10 con 0x7f8c9406f460 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c92ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c940cc870 0x7f8c94159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c92ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c940cc870 0x7f8c94159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55790/0 (socket says 172.21.15.131:55790) 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c92ffd640 1 -- 172.21.15.131:0/166308983 learned_addr learned my addr 172.21.15.131:0/166308983 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c937fe640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c9406f460 0x7f8c94156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c93fff640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94138f40 0x7f8c941598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c92ffd640 1 -- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c9406f460 msgr2=0x7f8c94156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c92ffd640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c9406f460 0x7f8c94156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c92ffd640 1 -- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94138f40 msgr2=0x7f8c941598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:43.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c92ffd640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94138f40 0x7f8c941598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:43.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c92ffd640 1 -- 172.21.15.131:0/166308983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c94154fe0 con 0x7f8c940cc870 2024-09-17T02:43:43.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c92ffd640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c940cc870 0x7f8c94159370 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800d720 tx=0x7f8c8800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:43.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c7bfff640 1 -- 172.21.15.131:0/166308983 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c88017070 con 0x7f8c940cc870 2024-09-17T02:43:43.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c7bfff640 1 -- 172.21.15.131:0/166308983 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c88005310 con 0x7f8c940cc870 2024-09-17T02:43:43.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c99a72640 1 -- 172.21.15.131:0/166308983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c94157670 con 0x7f8c940cc870 2024-09-17T02:43:43.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c7bfff640 1 -- 172.21.15.131:0/166308983 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c88009820 con 0x7f8c940cc870 2024-09-17T02:43:43.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.527+0000 7f8c99a72640 1 -- 172.21.15.131:0/166308983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c94157c30 con 0x7f8c940cc870 2024-09-17T02:43:43.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.531+0000 7f8c99a72640 1 -- 172.21.15.131:0/166308983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c94157e00 con 0x7f8c940cc870 2024-09-17T02:43:43.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.531+0000 7f8c7bfff640 1 -- 172.21.15.131:0/166308983 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8c880099c0 con 0x7f8c940cc870 2024-09-17T02:43:43.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.531+0000 7f8c7bfff640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c80073230 0x7f8c80075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:43.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.531+0000 7f8c7bfff640 1 -- 172.21.15.131:0/166308983 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5249+0+0 (secure 0 0 0) 0x7f8c880940d0 con 0x7f8c940cc870 2024-09-17T02:43:43.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.531+0000 7f8c937fe640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c80073230 0x7f8c80075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:43.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.531+0000 7f8c937fe640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c80073230 0x7f8c80075740 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009f20 tx=0x7f8c840094d0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:43.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.535+0000 7f8c7bfff640 1 -- 172.21.15.131:0/166308983 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8c8805f340 con 0x7f8c940cc870 2024-09-17T02:43:43.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:43.703+0000 7f8c99a72640 1 -- 172.21.15.131:0/166308983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8c94157400 con 0x7f8c940cc870 2024-09-17T02:43:43.881 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:43:43.884 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:43:44.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.615+0000 7f8c7bfff640 1 -- 172.21.15.131:0/166308983 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) ==== 138+0+0 (secure 0 0 0) 0x7f8c8805ec40 con 0x7f8c940cc870 2024-09-17T02:43:44.620 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-09-17T02:43:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 -- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c80073230 msgr2=0x7f8c80075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c80073230 0x7f8c80075740 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009f20 tx=0x7f8c840094d0 comp rx=0 tx=0).stop 2024-09-17T02:43:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 -- 172.21.15.131:0/166308983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c940cc870 msgr2=0x7f8c94159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:44.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c940cc870 0x7f8c94159370 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800d720 tx=0x7f8c8800dbe0 comp rx=0 tx=0).stop 2024-09-17T02:43:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 -- 172.21.15.131:0/166308983 shutdown_connections 2024-09-17T02:43:44.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c80073230 0x7f8c80075740 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:44.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c94138f40 0x7f8c941598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:44.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c940cc870 0x7f8c94159370 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:44.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 --2- 172.21.15.131:0/166308983 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c9406f460 0x7f8c94156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:44.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 -- 172.21.15.131:0/166308983 >> 172.21.15.131:0/166308983 conn(0x7f8c940d05a0 msgr2=0x7f8c9413f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:44.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 -- 172.21.15.131:0/166308983 shutdown_connections 2024-09-17T02:43:44.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.623+0000 7f8c99a72640 1 -- 172.21.15.131:0/166308983 wait complete. 2024-09-17T02:43:44.656 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T02:43:44.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.811+0000 7f8a049ed640 1 -- 172.21.15.131:0/2535090107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a0006f6c0 msgr2=0x7f8a000b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:44.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.811+0000 7f8a049ed640 1 --2- 172.21.15.131:0/2535090107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a0006f6c0 0x7f8a000b4130 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8a00073190 tx=0x7f89e801c740 comp rx=0 tx=0).stop 2024-09-17T02:43:44.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.811+0000 7f8a049ed640 1 -- 172.21.15.131:0/2535090107 shutdown_connections 2024-09-17T02:43:44.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.811+0000 7f8a049ed640 1 --2- 172.21.15.131:0/2535090107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a0006f6c0 0x7f8a000b4130 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:44.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.811+0000 7f8a049ed640 1 -- 172.21.15.131:0/2535090107 >> 172.21.15.131:0/2535090107 conn(0x7f8a00073ca0 msgr2=0x7f8a000740b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:44.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.811+0000 7f8a049ed640 1 -- 172.21.15.131:0/2535090107 shutdown_connections 2024-09-17T02:43:44.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.811+0000 7f8a049ed640 1 -- 172.21.15.131:0/2535090107 wait complete. 2024-09-17T02:43:44.814 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.811+0000 7f8a049ed640 1 Processor -- start 2024-09-17T02:43:44.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.811+0000 7f8a049ed640 1 -- start start 2024-09-17T02:43:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f8a049ed640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8a0006f6c0 0x7f8a001518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f8a049ed640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a00070ce0 0x7f8a001f9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:44.817 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f8a049ed640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000b4670 0x7f8a001f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:44.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f8a049ed640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8a00169870 con 0x7f8a000b4670 2024-09-17T02:43:44.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f8a049ed640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8a001696f0 con 0x7f8a00070ce0 2024-09-17T02:43:44.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f8a049ed640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8a001699f0 con 0x7f8a0006f6c0 2024-09-17T02:43:44.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89fed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000b4670 0x7f8a001f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:44.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89fed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000b4670 0x7f8a001f9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55802/0 (socket says 172.21.15.131:55802) 2024-09-17T02:43:44.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89fed9d640 1 -- 172.21.15.131:0/1678296518 learned_addr learned my addr 172.21.15.131:0/1678296518 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:44.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89f5d9b640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a00070ce0 0x7f8a001f9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:44.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89fed9d640 1 -- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8a0006f6c0 msgr2=0x7f8a001518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89fe59c640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8a0006f6c0 0x7f8a001518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89fed9d640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8a0006f6c0 0x7f8a001518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89fed9d640 1 -- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a00070ce0 msgr2=0x7f8a001f9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89fed9d640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a00070ce0 0x7f8a001f9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89fed9d640 1 -- 172.21.15.131:0/1678296518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0015e240 con 0x7f8a000b4670 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89f5d9b640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a00070ce0 0x7f8a001f9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89fed9d640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000b4670 0x7f8a001f9f90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f89f800c760 tx=0x7f89f800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89f77fe640 1 -- 172.21.15.131:0/1678296518 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8003db0 con 0x7f8a000b4670 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89f77fe640 1 -- 172.21.15.131:0/1678296518 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89f800ce10 con 0x7f8a000b4670 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89f77fe640 1 -- 172.21.15.131:0/1678296518 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f80052d0 con 0x7f8a000b4670 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f8a049ed640 1 -- 172.21.15.131:0/1678296518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0015b8f0 con 0x7f8a000b4670 2024-09-17T02:43:44.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f8a049ed640 1 -- 172.21.15.131:0/1678296518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0015dcd0 con 0x7f8a000b4670 2024-09-17T02:43:44.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f8a049ed640 1 -- 172.21.15.131:0/1678296518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a000b4130 con 0x7f8a000b4670 2024-09-17T02:43:44.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89f77fe640 1 -- 172.21.15.131:0/1678296518 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f89f8005470 con 0x7f8a000b4670 2024-09-17T02:43:44.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89f77fe640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89e0073200 0x7f89e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:44.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.815+0000 7f89f77fe640 1 -- 172.21.15.131:0/1678296518 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5619+0+0 (secure 0 0 0) 0x7f89f80948f0 con 0x7f8a000b4670 2024-09-17T02:43:44.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.819+0000 7f89fe59c640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89e0073200 0x7f89e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:44.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.819+0000 7f89fe59c640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89e0073200 0x7f89e0075710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f89f000a750 tx=0x7f89f0008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:44.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.819+0000 7f89f77fe640 1 -- 172.21.15.131:0/1678296518 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f89f805fad0 con 0x7f8a000b4670 2024-09-17T02:43:44.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:44.979+0000 7f8a049ed640 1 -- 172.21.15.131:0/1678296518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8a0006f4a0 con 0x7f8a000b4670 2024-09-17T02:43:45.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.631+0000 7f89f77fe640 1 -- 172.21.15.131:0/1678296518 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) ==== 170+0+0 (secure 0 0 0) 0x7f89f805f3d0 con 0x7f8a000b4670 2024-09-17T02:43:45.633 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-09-17T02:43:45.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 -- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89e0073200 msgr2=0x7f89e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:45.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89e0073200 0x7f89e0075710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f89f000a750 tx=0x7f89f0008040 comp rx=0 tx=0).stop 2024-09-17T02:43:45.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 -- 172.21.15.131:0/1678296518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000b4670 msgr2=0x7f8a001f9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:45.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000b4670 0x7f8a001f9f90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f89f800c760 tx=0x7f89f800cc20 comp rx=0 tx=0).stop 2024-09-17T02:43:45.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 -- 172.21.15.131:0/1678296518 shutdown_connections 2024-09-17T02:43:45.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89e0073200 0x7f89e0075710 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:45.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8a000b4670 0x7f8a001f9f90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:45.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8a00070ce0 0x7f8a001f9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:45.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 --2- 172.21.15.131:0/1678296518 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8a0006f6c0 0x7f8a001518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:45.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 -- 172.21.15.131:0/1678296518 >> 172.21.15.131:0/1678296518 conn(0x7f8a00073ca0 msgr2=0x7f8a000710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:45.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 -- 172.21.15.131:0/1678296518 shutdown_connections 2024-09-17T02:43:45.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.635+0000 7f8a049ed640 1 -- 172.21.15.131:0/1678296518 wait complete. 2024-09-17T02:43:45.681 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T02:43:45.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.835+0000 7fb49ac44640 1 -- 172.21.15.131:0/1394661650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4940ced00 msgr2=0x7fb49406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:45.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.835+0000 7fb49ac44640 1 --2- 172.21.15.131:0/1394661650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4940ced00 0x7fb49406dea0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb488009840 tx=0x7fb48801c640 comp rx=0 tx=0).stop 2024-09-17T02:43:45.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.835+0000 7fb49ac44640 1 -- 172.21.15.131:0/1394661650 shutdown_connections 2024-09-17T02:43:45.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.835+0000 7fb49ac44640 1 --2- 172.21.15.131:0/1394661650 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4940ced00 0x7fb49406dea0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:45.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.835+0000 7fb49ac44640 1 -- 172.21.15.131:0/1394661650 >> 172.21.15.131:0/1394661650 conn(0x7fb49406eea0 msgr2=0x7fb49406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:45.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.835+0000 7fb49ac44640 1 -- 172.21.15.131:0/1394661650 shutdown_connections 2024-09-17T02:43:45.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.835+0000 7fb49ac44640 1 -- 172.21.15.131:0/1394661650 wait complete. 2024-09-17T02:43:45.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.835+0000 7fb49ac44640 1 Processor -- start 2024-09-17T02:43:45.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.835+0000 7fb49ac44640 1 -- start start 2024-09-17T02:43:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb49ac44640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49406e3e0 0x7fb4940b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb49ac44640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4940ced00 0x7fb49413c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb49ac44640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb49415c5b0 0x7fb4941514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:45.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb49ac44640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb494169e30 con 0x7fb4940ced00 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb49ac44640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb494169cb0 con 0x7fb49406e3e0 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb49ac44640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb494169fb0 con 0x7fb49415c5b0 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb493fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4940ced00 0x7fb49413c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb493fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4940ced00 0x7fb49413c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55822/0 (socket says 172.21.15.131:55822) 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb493fff640 1 -- 172.21.15.131:0/3436602020 learned_addr learned my addr 172.21.15.131:0/3436602020 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb4991e1640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb49415c5b0 0x7fb4941514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb493fff640 1 -- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb49415c5b0 msgr2=0x7fb4941514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb493fff640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb49415c5b0 0x7fb4941514b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb4989e0640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49406e3e0 0x7fb4940b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb493fff640 1 -- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49406e3e0 msgr2=0x7fb4940b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb493fff640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49406e3e0 0x7fb4940b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:45.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb493fff640 1 -- 172.21.15.131:0/3436602020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4941f9e00 con 0x7fb4940ced00 2024-09-17T02:43:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb493fff640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4940ced00 0x7fb49413c1b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb48000c3f0 tx=0x7fb48000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb4917fa640 1 -- 172.21.15.131:0/3436602020 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480017070 con 0x7fb4940ced00 2024-09-17T02:43:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb4917fa640 1 -- 172.21.15.131:0/3436602020 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb480003600 con 0x7fb4940ced00 2024-09-17T02:43:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb49ac44640 1 -- 172.21.15.131:0/3436602020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4941fa0e0 con 0x7fb4940ced00 2024-09-17T02:43:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb4917fa640 1 -- 172.21.15.131:0/3436602020 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480003a50 con 0x7fb4940ced00 2024-09-17T02:43:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb49ac44640 1 -- 172.21.15.131:0/3436602020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4941fa6a0 con 0x7fb4940ced00 2024-09-17T02:43:45.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb49ac44640 1 -- 172.21.15.131:0/3436602020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb494075b80 con 0x7fb4940ced00 2024-09-17T02:43:45.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.839+0000 7fb4917fa640 1 -- 172.21.15.131:0/3436602020 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb48001a070 con 0x7fb4940ced00 2024-09-17T02:43:45.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.843+0000 7fb4917fa640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb478073200 0x7fb478075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:45.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.843+0000 7fb4917fa640 1 -- 172.21.15.131:0/3436602020 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6001+0+0 (secure 0 0 0) 0x7fb480093970 con 0x7fb4940ced00 2024-09-17T02:43:45.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.843+0000 7fb4989e0640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb478073200 0x7fb478075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:45.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.843+0000 7fb4989e0640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb478073200 0x7fb478075710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb4880af000 tx=0x7fb48801ae80 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:45.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:45.847+0000 7fb4917fa640 1 -- 172.21.15.131:0/3436602020 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb480021020 con 0x7fb4940ced00 2024-09-17T02:43:46.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.055+0000 7fb49ac44640 1 -- 172.21.15.131:0/3436602020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb49406dea0 con 0x7fb4940ced00 2024-09-17T02:43:46.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.683+0000 7fb4917fa640 1 -- 172.21.15.131:0/3436602020 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v32) ==== 173+0+0 (secure 0 0 0) 0x7fb49406e7f0 con 0x7fb4940ced00 2024-09-17T02:43:46.688 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 9 and data pool 10 2024-09-17T02:43:46.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.695+0000 7fb49ac44640 1 -- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb478073200 msgr2=0x7fb478075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:46.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.695+0000 7fb49ac44640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb478073200 0x7fb478075710 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb4880af000 tx=0x7fb48801ae80 comp rx=0 tx=0).stop 2024-09-17T02:43:46.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.695+0000 7fb49ac44640 1 -- 172.21.15.131:0/3436602020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4940ced00 msgr2=0x7fb49413c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:46.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.695+0000 7fb49ac44640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4940ced00 0x7fb49413c1b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb48000c3f0 tx=0x7fb48000c8b0 comp rx=0 tx=0).stop 2024-09-17T02:43:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.699+0000 7fb49ac44640 1 -- 172.21.15.131:0/3436602020 shutdown_connections 2024-09-17T02:43:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.699+0000 7fb49ac44640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb478073200 0x7fb478075710 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.699+0000 7fb49ac44640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb49415c5b0 0x7fb4941514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.699+0000 7fb49ac44640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb4940ced00 0x7fb49413c1b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.699+0000 7fb49ac44640 1 --2- 172.21.15.131:0/3436602020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb49406e3e0 0x7fb4940b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.699+0000 7fb49ac44640 1 -- 172.21.15.131:0/3436602020 >> 172.21.15.131:0/3436602020 conn(0x7fb49406eea0 msgr2=0x7fb4940b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.699+0000 7fb49ac44640 1 -- 172.21.15.131:0/3436602020 shutdown_connections 2024-09-17T02:43:46.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.699+0000 7fb49ac44640 1 -- 172.21.15.131:0/3436602020 wait complete. 2024-09-17T02:43:46.733 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:43:46.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.883+0000 7f043b5fc640 1 -- 172.21.15.131:0/3173652141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043406e2c0 msgr2=0x7f04340c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:46.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.883+0000 7f043b5fc640 1 --2- 172.21.15.131:0/3173652141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043406e2c0 0x7f04340c4580 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f0424009c80 tx=0x7f042401cd10 comp rx=0 tx=0).stop 2024-09-17T02:43:46.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.883+0000 7f043b5fc640 1 -- 172.21.15.131:0/3173652141 shutdown_connections 2024-09-17T02:43:46.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.883+0000 7f043b5fc640 1 --2- 172.21.15.131:0/3173652141 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04340c4ac0 0x7f04340c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:46.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.883+0000 7f043b5fc640 1 --2- 172.21.15.131:0/3173652141 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043406e2c0 0x7f04340c4580 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:46.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.883+0000 7f043b5fc640 1 -- 172.21.15.131:0/3173652141 >> 172.21.15.131:0/3173652141 conn(0x7f043406eea0 msgr2=0x7f043406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:46.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.883+0000 7f043b5fc640 1 -- 172.21.15.131:0/3173652141 shutdown_connections 2024-09-17T02:43:46.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.883+0000 7f043b5fc640 1 -- 172.21.15.131:0/3173652141 wait complete. 2024-09-17T02:43:46.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.883+0000 7f043b5fc640 1 Processor -- start 2024-09-17T02:43:46.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f043b5fc640 1 -- start start 2024-09-17T02:43:46.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f043b5fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043406e2c0 0x7f043414a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:46.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f043b5fc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04340c4ac0 0x7f043415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:46.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f043b5fc640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04340cdbf0 0x7f0434153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:46.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f043b5fc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f04340cf2b0 con 0x7f043406e2c0 2024-09-17T02:43:46.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f043b5fc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f04340cf130 con 0x7f04340c4ac0 2024-09-17T02:43:46.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f043b5fc640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f04340cf430 con 0x7f04340cdbf0 2024-09-17T02:43:46.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0439398640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043406e2c0 0x7f043414a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:46.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0439398640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043406e2c0 0x7f043414a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55846/0 (socket says 172.21.15.131:55846) 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0439398640 1 -- 172.21.15.131:0/3599778717 learned_addr learned my addr 172.21.15.131:0/3599778717 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0439b99640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04340cdbf0 0x7f0434153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0438b97640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04340c4ac0 0x7f043415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0439398640 1 -- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04340cdbf0 msgr2=0x7f0434153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0439398640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04340cdbf0 0x7f0434153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0439398640 1 -- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04340c4ac0 msgr2=0x7f043415a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0439398640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04340c4ac0 0x7f043415a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0439398640 1 -- 172.21.15.131:0/3599778717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f043414ba20 con 0x7f043406e2c0 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0439b99640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04340cdbf0 0x7f0434153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0439398640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043406e2c0 0x7f043414a090 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0424009bd0 tx=0x7f04240a5ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0429ffb640 1 -- 172.21.15.131:0/3599778717 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04240bc070 con 0x7f043406e2c0 2024-09-17T02:43:46.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0429ffb640 1 -- 172.21.15.131:0/3599778717 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0424004430 con 0x7f043406e2c0 2024-09-17T02:43:46.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f0429ffb640 1 -- 172.21.15.131:0/3599778717 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04240b2040 con 0x7f043406e2c0 2024-09-17T02:43:46.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f043b5fc640 1 -- 172.21.15.131:0/3599778717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f043414a980 con 0x7f043406e2c0 2024-09-17T02:43:46.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f043b5fc640 1 -- 172.21.15.131:0/3599778717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f043414af40 con 0x7f043406e2c0 2024-09-17T02:43:46.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.887+0000 7f043b5fc640 1 -- 172.21.15.131:0/3599778717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0434073310 con 0x7f043406e2c0 2024-09-17T02:43:46.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.891+0000 7f0429ffb640 1 -- 172.21.15.131:0/3599778717 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f04240ad440 con 0x7f043406e2c0 2024-09-17T02:43:46.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.891+0000 7f0429ffb640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0420073230 0x7f0420075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:46.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.891+0000 7f0429ffb640 1 -- 172.21.15.131:0/3599778717 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f042412e4c0 con 0x7f043406e2c0 2024-09-17T02:43:46.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.891+0000 7f0438b97640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0420073230 0x7f0420075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:46.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.891+0000 7f0438b97640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0420073230 0x7f0420075740 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0434157da0 tx=0x7f042c005c50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:46.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:46.895+0000 7f0429ffb640 1 -- 172.21.15.131:0/3599778717 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f04240a5e20 con 0x7f043406e2c0 2024-09-17T02:43:47.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.063+0000 7f043b5fc640 1 -- 172.21.15.131:0/3599778717 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f04340c4560 con 0x7f043406e2c0 2024-09-17T02:43:47.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f0429ffb640 1 -- 172.21.15.131:0/3599778717 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+17053 (secure 0 0 0) 0x7f0424016990 con 0x7f043406e2c0 2024-09-17T02:43:47.069 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:47.069 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":64,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:43:46.061642+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:43:43.717619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-17T02:43:44.992949+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:43:47.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 -- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0420073230 msgr2=0x7f0420075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:47.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0420073230 0x7f0420075740 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0434157da0 tx=0x7f042c005c50 comp rx=0 tx=0).stop 2024-09-17T02:43:47.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 -- 172.21.15.131:0/3599778717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043406e2c0 msgr2=0x7f043414a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:47.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043406e2c0 0x7f043414a090 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0424009bd0 tx=0x7f04240a5ae0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 -- 172.21.15.131:0/3599778717 shutdown_connections 2024-09-17T02:43:47.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0420073230 0x7f0420075740 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f04340cdbf0 0x7f0434153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f04340c4ac0 0x7f043415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 --2- 172.21.15.131:0/3599778717 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f043406e2c0 0x7f043414a090 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 -- 172.21.15.131:0/3599778717 >> 172.21.15.131:0/3599778717 conn(0x7f043406eea0 msgr2=0x7f0434152840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:47.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 -- 172.21.15.131:0/3599778717 shutdown_connections 2024-09-17T02:43:47.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.067+0000 7f043b5fc640 1 -- 172.21.15.131:0/3599778717 wait complete. 2024-09-17T02:43:47.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:43:47.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.243+0000 7f27f705c640 1 -- 172.21.15.131:0/27048695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27f00ced00 msgr2=0x7f27f006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.243+0000 7f27f705c640 1 --2- 172.21.15.131:0/27048695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27f00ced00 0x7f27f006dea0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f27f00b61b0 tx=0x7f27e001c8c0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.243+0000 7f27f705c640 1 -- 172.21.15.131:0/27048695 shutdown_connections 2024-09-17T02:43:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.243+0000 7f27f705c640 1 --2- 172.21.15.131:0/27048695 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27f015c3e0 0x7f27f00cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.243+0000 7f27f705c640 1 --2- 172.21.15.131:0/27048695 >> v2:172.21.15.189:3300/0 conn(0x7f27f006e3e0 0x7f27f015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.243+0000 7f27f705c640 1 --2- 172.21.15.131:0/27048695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27f00ced00 0x7f27f006dea0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.243+0000 7f27f705c640 1 -- 172.21.15.131:0/27048695 >> 172.21.15.131:0/27048695 conn(0x7f27f006eea0 msgr2=0x7f27f006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.243+0000 7f27f705c640 1 -- 172.21.15.131:0/27048695 shutdown_connections 2024-09-17T02:43:47.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.243+0000 7f27f705c640 1 -- 172.21.15.131:0/27048695 wait complete. 2024-09-17T02:43:47.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f705c640 1 Processor -- start 2024-09-17T02:43:47.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f705c640 1 -- start start 2024-09-17T02:43:47.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f705c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27f006e3e0 0x7f27f0153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:47.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f705c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27f015c3e0 0x7f27f00b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:47.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f705c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f01545c0 0x7f27f01549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:47.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f705c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f27f00caba0 con 0x7f27f015c3e0 2024-09-17T02:43:47.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f705c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f27f00caa20 con 0x7f27f01545c0 2024-09-17T02:43:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f705c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f27f00cad20 con 0x7f27f006e3e0 2024-09-17T02:43:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27effff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27f015c3e0 0x7f27f00b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27effff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27f015c3e0 0x7f27f00b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55860/0 (socket says 172.21.15.131:55860) 2024-09-17T02:43:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27effff640 1 -- 172.21.15.131:0/787693259 learned_addr learned my addr 172.21.15.131:0/787693259 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f4df8640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27f006e3e0 0x7f27f0153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f55f9640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f01545c0 0x7f27f01549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27effff640 1 -- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27f006e3e0 msgr2=0x7f27f0153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27effff640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27f006e3e0 0x7f27f0153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27effff640 1 -- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f01545c0 msgr2=0x7f27f01549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:47.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27effff640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f01545c0 0x7f27f01549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27effff640 1 -- 172.21.15.131:0/787693259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f0154f10 con 0x7f27f015c3e0 2024-09-17T02:43:47.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f4df8640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27f006e3e0 0x7f27f0153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:47.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27effff640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27f015c3e0 0x7f27f00b2ef0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005950 tx=0x7f27e400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:47.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27ed7fa640 1 -- 172.21.15.131:0/787693259 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4007bb0 con 0x7f27f015c3e0 2024-09-17T02:43:47.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27ed7fa640 1 -- 172.21.15.131:0/787693259 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27e4007e40 con 0x7f27f015c3e0 2024-09-17T02:43:47.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f705c640 1 -- 172.21.15.131:0/787693259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f00b4e10 con 0x7f27f015c3e0 2024-09-17T02:43:47.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27ed7fa640 1 -- 172.21.15.131:0/787693259 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e400f690 con 0x7f27f015c3e0 2024-09-17T02:43:47.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f705c640 1 -- 172.21.15.131:0/787693259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f00b53d0 con 0x7f27f015c3e0 2024-09-17T02:43:47.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.247+0000 7f27f705c640 1 -- 172.21.15.131:0/787693259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f00cb2a0 con 0x7f27f015c3e0 2024-09-17T02:43:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.251+0000 7f27ed7fa640 1 -- 172.21.15.131:0/787693259 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f27e4018400 con 0x7f27f015c3e0 2024-09-17T02:43:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.251+0000 7f27ed7fa640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f27dc073200 0x7f27dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:47.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.251+0000 7f27ed7fa640 1 -- 172.21.15.131:0/787693259 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7f27e4095150 con 0x7f27f015c3e0 2024-09-17T02:43:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.251+0000 7f27f4df8640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f27dc073200 0x7f27dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:47.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.251+0000 7f27f4df8640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f27dc073200 0x7f27dc075710 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f27e00af000 tx=0x7f27e001b100 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:47.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.255+0000 7f27ed7fa640 1 -- 172.21.15.131:0/787693259 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f27e4026020 con 0x7f27f015c3e0 2024-09-17T02:43:47.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.423+0000 7f27f705c640 1 -- 172.21.15.131:0/787693259 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27f00cbc20 con 0x7f27f015c3e0 2024-09-17T02:43:47.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27ed7fa640 1 -- 172.21.15.131:0/787693259 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) ==== 74+0+17053 (secure 0 0 0) 0x7f27f0154f10 con 0x7f27f015c3e0 2024-09-17T02:43:47.428 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:47.428 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":64,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:43:46.061642+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:43:43.717619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-17T02:43:44.992949+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:43:47.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 -- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f27dc073200 msgr2=0x7f27dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:47.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f27dc073200 0x7f27dc075710 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f27e00af000 tx=0x7f27e001b100 comp rx=0 tx=0).stop 2024-09-17T02:43:47.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 -- 172.21.15.131:0/787693259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27f015c3e0 msgr2=0x7f27f00b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:47.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27f015c3e0 0x7f27f00b2ef0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005950 tx=0x7f27e400cdd0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 -- 172.21.15.131:0/787693259 shutdown_connections 2024-09-17T02:43:47.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f27dc073200 0x7f27dc075710 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f27f01545c0 0x7f27f01549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f27f015c3e0 0x7f27f00b2ef0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 --2- 172.21.15.131:0/787693259 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f27f006e3e0 0x7f27f0153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 -- 172.21.15.131:0/787693259 >> 172.21.15.131:0/787693259 conn(0x7f27f006eea0 msgr2=0x7f27f0155650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:47.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 -- 172.21.15.131:0/787693259 shutdown_connections 2024-09-17T02:43:47.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.427+0000 7f27f705c640 1 -- 172.21.15.131:0/787693259 wait complete. 2024-09-17T02:43:47.452 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T02:43:47.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.607+0000 7fb05e522640 1 -- 172.21.15.131:0/733768290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb058070c90 msgr2=0x7fb0580710a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:47.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.607+0000 7fb05e522640 1 --2- 172.21.15.131:0/733768290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb058070c90 0x7fb0580710a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb0580d18b0 tx=0x7fb04801cac0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.607+0000 7fb05e522640 1 -- 172.21.15.131:0/733768290 shutdown_connections 2024-09-17T02:43:47.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.607+0000 7fb05e522640 1 --2- 172.21.15.131:0/733768290 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb058070c90 0x7fb0580710a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.607+0000 7fb05e522640 1 --2- 172.21.15.131:0/733768290 >> v2:172.21.15.189:3300/0 conn(0x7fb05815b280 0x7fb0580cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.607+0000 7fb05e522640 1 -- 172.21.15.131:0/733768290 >> 172.21.15.131:0/733768290 conn(0x7fb0580cd6a0 msgr2=0x7fb0580cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:47.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.607+0000 7fb05e522640 1 -- 172.21.15.131:0/733768290 shutdown_connections 2024-09-17T02:43:47.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.607+0000 7fb05e522640 1 -- 172.21.15.131:0/733768290 wait complete. 2024-09-17T02:43:47.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05e522640 1 Processor -- start 2024-09-17T02:43:47.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05e522640 1 -- start start 2024-09-17T02:43:47.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05e522640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb058070c90 0x7fb058159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05e522640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0580cf580 0x7fb058159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05e522640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb05815b280 0x7fb0581f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05e522640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb058168a70 con 0x7fb05815b280 2024-09-17T02:43:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05e522640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb0581688f0 con 0x7fb0580cf580 2024-09-17T02:43:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05e522640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb058168bf0 con 0x7fb058070c90 2024-09-17T02:43:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05cabf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb05815b280 0x7fb0581f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05cabf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb05815b280 0x7fb0581f9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55866/0 (socket says 172.21.15.131:55866) 2024-09-17T02:43:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05cabf640 1 -- 172.21.15.131:0/1868933621 learned_addr learned my addr 172.21.15.131:0/1868933621 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb057fff640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb058070c90 0x7fb058159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05cabf640 1 -- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb058070c90 msgr2=0x7fb058159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb0577fe640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0580cf580 0x7fb058159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:47.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05cabf640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb058070c90 0x7fb058159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05cabf640 1 -- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0580cf580 msgr2=0x7fb058159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:47.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05cabf640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0580cf580 0x7fb058159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:47.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05cabf640 1 -- 172.21.15.131:0/1868933621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0581f6890 con 0x7fb05815b280 2024-09-17T02:43:47.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05cabf640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb05815b280 0x7fb0581f9a70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb0480986d0 tx=0x7fb048005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:47.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb054ff9640 1 -- 172.21.15.131:0/1868933621 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0480b2040 con 0x7fb05815b280 2024-09-17T02:43:47.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb054ff9640 1 -- 172.21.15.131:0/1868933621 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0480bb070 con 0x7fb05815b280 2024-09-17T02:43:47.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05e522640 1 -- 172.21.15.131:0/1868933621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0581f6b70 con 0x7fb05815b280 2024-09-17T02:43:47.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb054ff9640 1 -- 172.21.15.131:0/1868933621 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048004330 con 0x7fb05815b280 2024-09-17T02:43:47.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05e522640 1 -- 172.21.15.131:0/1868933621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0581f7130 con 0x7fb05815b280 2024-09-17T02:43:47.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.611+0000 7fb05e522640 1 -- 172.21.15.131:0/1868933621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0580710a0 con 0x7fb05815b280 2024-09-17T02:43:47.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.615+0000 7fb054ff9640 1 -- 172.21.15.131:0/1868933621 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0480044d0 con 0x7fb05815b280 2024-09-17T02:43:47.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.615+0000 7fb054ff9640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb044073200 0x7fb044075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:47.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.615+0000 7fb054ff9640 1 -- 172.21.15.131:0/1868933621 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6101+0+0 (secure 0 0 0) 0x7fb04812f850 con 0x7fb05815b280 2024-09-17T02:43:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.615+0000 7fb057fff640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb044073200 0x7fb044075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.615+0000 7fb057fff640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb044073200 0x7fb044075710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb050009a10 tx=0x7fb0500091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:47.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.619+0000 7fb054ff9640 1 -- 172.21.15.131:0/1868933621 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb048016770 con 0x7fb05815b280 2024-09-17T02:43:47.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:47.835+0000 7fb05e522640 1 -- 172.21.15.131:0/1868933621 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb0581577c0 con 0x7fb05815b280 2024-09-17T02:43:48.603 INFO:tasks.ceph.mds.d.smithi189.stderr:2024-09-17T02:43:48.601+0000 7fbd29b6b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:43:48.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.695+0000 7fb054ff9640 1 -- 172.21.15.131:0/1868933621 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7fb0480fa770 con 0x7fb05815b280 2024-09-17T02:43:48.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 -- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb044073200 msgr2=0x7fb044075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:48.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb044073200 0x7fb044075710 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb050009a10 tx=0x7fb0500091e0 comp rx=0 tx=0).stop 2024-09-17T02:43:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 -- 172.21.15.131:0/1868933621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb05815b280 msgr2=0x7fb0581f9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb05815b280 0x7fb0581f9a70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb0480986d0 tx=0x7fb048005a80 comp rx=0 tx=0).stop 2024-09-17T02:43:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 -- 172.21.15.131:0/1868933621 shutdown_connections 2024-09-17T02:43:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb044073200 0x7fb044075710 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb05815b280 0x7fb0581f9a70 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0580cf580 0x7fb058159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 --2- 172.21.15.131:0/1868933621 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb058070c90 0x7fb058159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 -- 172.21.15.131:0/1868933621 >> 172.21.15.131:0/1868933621 conn(0x7fb0580cd6a0 msgr2=0x7fb0580b5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 -- 172.21.15.131:0/1868933621 shutdown_connections 2024-09-17T02:43:48.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.699+0000 7fb05e522640 1 -- 172.21.15.131:0/1868933621 wait complete. 2024-09-17T02:43:48.728 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T02:43:48.728 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T02:43:48.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.887+0000 7fb1394a1640 1 -- 172.21.15.131:0/3956768915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb11c005230 msgr2=0x7fb11c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:48.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.887+0000 7fb1394a1640 1 --2- 172.21.15.131:0/3956768915 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb11c005230 0x7fb11c00b9d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb134145bb0 tx=0x7fb12c01d1f0 comp rx=0 tx=0).stop 2024-09-17T02:43:48.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.887+0000 7fb1337fe640 1 -- 172.21.15.131:0/3956768915 reap_dead start 2024-09-17T02:43:48.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.887+0000 7fb1394a1640 1 -- 172.21.15.131:0/3956768915 shutdown_connections 2024-09-17T02:43:48.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.887+0000 7fb1394a1640 1 -- 172.21.15.131:0/3956768915 >> 172.21.15.131:0/3956768915 conn(0x7fb1340d2130 msgr2=0x7fb1340d2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:48.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.887+0000 7fb1394a1640 1 -- 172.21.15.131:0/3956768915 shutdown_connections 2024-09-17T02:43:48.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.887+0000 7fb1394a1640 1 -- 172.21.15.131:0/3956768915 wait complete. 2024-09-17T02:43:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1394a1640 1 Processor -- start 2024-09-17T02:43:48.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1394a1640 1 -- start start 2024-09-17T02:43:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1394a1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb11c005230 0x7fb1341f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1394a1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1341f6ce0 0x7fb134155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1394a1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb1341f33e0 0x7fb134155da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1394a1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb1340ccf20 con 0x7fb1341f6ce0 2024-09-17T02:43:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1394a1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb1340ccda0 con 0x7fb11c005230 2024-09-17T02:43:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1394a1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb134143ca0 con 0x7fb1341f33e0 2024-09-17T02:43:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1327fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1341f6ce0 0x7fb134155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1327fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1341f6ce0 0x7fb134155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55878/0 (socket says 172.21.15.131:55878) 2024-09-17T02:43:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1327fc640 1 -- 172.21.15.131:0/3967359553 learned_addr learned my addr 172.21.15.131:0/3967359553 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:48.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb132ffd640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb11c005230 0x7fb1341f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1337fe640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb1341f33e0 0x7fb134155da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1327fc640 1 -- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb1341f33e0 msgr2=0x7fb134155da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1327fc640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb1341f33e0 0x7fb134155da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1327fc640 1 -- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb11c005230 msgr2=0x7fb1341f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1327fc640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb11c005230 0x7fb1341f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1327fc640 1 -- 172.21.15.131:0/3967359553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb134158980 con 0x7fb1341f6ce0 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb132ffd640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb11c005230 0x7fb1341f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1327fc640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1341f6ce0 0x7fb134155860 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb128005bd0 tx=0x7fb12800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb11b7fe640 1 -- 172.21.15.131:0/3967359553 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128004230 con 0x7fb1341f6ce0 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb11b7fe640 1 -- 172.21.15.131:0/3967359553 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1280044e0 con 0x7fb1341f6ce0 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1394a1640 1 -- 172.21.15.131:0/3967359553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb134158c60 con 0x7fb1341f6ce0 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb11b7fe640 1 -- 172.21.15.131:0/3967359553 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128009950 con 0x7fb1341f6ce0 2024-09-17T02:43:48.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.891+0000 7fb1394a1640 1 -- 172.21.15.131:0/3967359553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb134159220 con 0x7fb1341f6ce0 2024-09-17T02:43:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.895+0000 7fb1394a1640 1 -- 172.21.15.131:0/3967359553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb134143760 con 0x7fb1341f6ce0 2024-09-17T02:43:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.895+0000 7fb11b7fe640 1 -- 172.21.15.131:0/3967359553 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb128009af0 con 0x7fb1341f6ce0 2024-09-17T02:43:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.895+0000 7fb11b7fe640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb120073230 0x7fb120075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.895+0000 7fb11b7fe640 1 -- 172.21.15.131:0/3967359553 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fb128094530 con 0x7fb1341f6ce0 2024-09-17T02:43:48.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.895+0000 7fb132ffd640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb120073230 0x7fb120075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:48.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.895+0000 7fb132ffd640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb120073230 0x7fb120075740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb13415feb0 tx=0x7fb124001380 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:48.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:48.899+0000 7fb11b7fe640 1 -- 172.21.15.131:0/3967359553 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb128017020 con 0x7fb1341f6ce0 2024-09-17T02:43:49.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.103+0000 7fb1394a1640 1 -- 172.21.15.131:0/3967359553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb134145d30 con 0x7fb1341f6ce0 2024-09-17T02:43:49.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb11b7fe640 1 -- 172.21.15.131:0/3967359553 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fb12805f3c0 con 0x7fb1341f6ce0 2024-09-17T02:43:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 -- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb120073230 msgr2=0x7fb120075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb120073230 0x7fb120075740 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb13415feb0 tx=0x7fb124001380 comp rx=0 tx=0).stop 2024-09-17T02:43:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 -- 172.21.15.131:0/3967359553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1341f6ce0 msgr2=0x7fb134155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1341f6ce0 0x7fb134155860 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb128005bd0 tx=0x7fb12800da90 comp rx=0 tx=0).stop 2024-09-17T02:43:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 -- 172.21.15.131:0/3967359553 shutdown_connections 2024-09-17T02:43:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb120073230 0x7fb120075740 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:49.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb1341f33e0 0x7fb134155da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:49.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb1341f6ce0 0x7fb134155860 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:49.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 --2- 172.21.15.131:0/3967359553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb11c005230 0x7fb1341f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:49.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 -- 172.21.15.131:0/3967359553 >> 172.21.15.131:0/3967359553 conn(0x7fb1340d2130 msgr2=0x7fb13415d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:49.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 -- 172.21.15.131:0/3967359553 shutdown_connections 2024-09-17T02:43:49.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.711+0000 7fb1394a1640 1 -- 172.21.15.131:0/3967359553 wait complete. 2024-09-17T02:43:49.741 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T02:43:49.741 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:43:49.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.899+0000 7f1903c70640 1 -- 172.21.15.131:0/3583791166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18fc15b2f0 msgr2=0x7f18fc0cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:49.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.899+0000 7f1903c70640 1 --2- 172.21.15.131:0/3583791166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18fc15b2f0 0x7f18fc0cae70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009900 tx=0x7f18f401c8b0 comp rx=0 tx=0).stop 2024-09-17T02:43:49.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.899+0000 7f1903c70640 1 -- 172.21.15.131:0/3583791166 shutdown_connections 2024-09-17T02:43:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.899+0000 7f1903c70640 1 --2- 172.21.15.131:0/3583791166 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18fc070c90 0x7f18fc0710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.899+0000 7f1903c70640 1 --2- 172.21.15.131:0/3583791166 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18fc15b2f0 0x7f18fc0cae70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.899+0000 7f1903c70640 1 -- 172.21.15.131:0/3583791166 >> 172.21.15.131:0/3583791166 conn(0x7f18fc0cd6a0 msgr2=0x7f18fc0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.899+0000 7f1903c70640 1 -- 172.21.15.131:0/3583791166 shutdown_connections 2024-09-17T02:43:49.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.899+0000 7f1903c70640 1 -- 172.21.15.131:0/3583791166 wait complete. 2024-09-17T02:43:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.899+0000 7f1903c70640 1 Processor -- start 2024-09-17T02:43:49.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.899+0000 7f1903c70640 1 -- start start 2024-09-17T02:43:49.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1903c70640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18fc070c90 0x7f18fc0c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:49.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1903c70640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18fc0cf4e0 0x7f18fc0c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:49.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1903c70640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18fc15b2f0 0x7f18fc15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:49.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1903c70640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc168b00 con 0x7f18fc070c90 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1903c70640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18fc071bb0 con 0x7f18fc15b2f0 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1903c70640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f18fc168c80 con 0x7f18fc0cf4e0 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1901a0c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18fc070c90 0x7f18fc0c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1901a0c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18fc070c90 0x7f18fc0c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55902/0 (socket says 172.21.15.131:55902) 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1901a0c640 1 -- 172.21.15.131:0/282419611 learned_addr learned my addr 172.21.15.131:0/282419611 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f190120b640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18fc0cf4e0 0x7f18fc0c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f190220d640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18fc15b2f0 0x7f18fc15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1901a0c640 1 -- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18fc0cf4e0 msgr2=0x7f18fc0c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1901a0c640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18fc0cf4e0 0x7f18fc0c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1901a0c640 1 -- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18fc15b2f0 msgr2=0x7f18fc15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1901a0c640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18fc15b2f0 0x7f18fc15dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1901a0c640 1 -- 172.21.15.131:0/282419611 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18fc1fa450 con 0x7f18fc070c90 2024-09-17T02:43:49.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1901a0c640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18fc070c90 0x7f18fc0c9f60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009850 tx=0x7f18f4005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f18fa7fc640 1 -- 172.21.15.131:0/282419611 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f40b2040 con 0x7f18fc070c90 2024-09-17T02:43:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f18fa7fc640 1 -- 172.21.15.131:0/282419611 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18f4005310 con 0x7f18fc070c90 2024-09-17T02:43:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1903c70640 1 -- 172.21.15.131:0/282419611 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18fc1fa730 con 0x7f18fc070c90 2024-09-17T02:43:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f18fa7fc640 1 -- 172.21.15.131:0/282419611 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f40046d0 con 0x7f18fc070c90 2024-09-17T02:43:49.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1903c70640 1 -- 172.21.15.131:0/282419611 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18fc1facf0 con 0x7f18fc070c90 2024-09-17T02:43:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f1903c70640 1 -- 172.21.15.131:0/282419611 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18fc0710a0 con 0x7f18fc070c90 2024-09-17T02:43:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.903+0000 7f18fa7fc640 1 -- 172.21.15.131:0/282419611 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f18f4004890 con 0x7f18fc070c90 2024-09-17T02:43:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.907+0000 7f18fa7fc640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18e8073200 0x7f18e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.907+0000 7f18fa7fc640 1 -- 172.21.15.131:0/282419611 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f18f412e700 con 0x7f18fc070c90 2024-09-17T02:43:49.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.907+0000 7f190120b640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18e8073200 0x7f18e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:49.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.907+0000 7f190120b640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18e8073200 0x7f18e8075710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0afe70 tx=0x7f18f0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:49.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:49.907+0000 7f18fa7fc640 1 -- 172.21.15.131:0/282419611 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f18f40bd070 con 0x7f18fc070c90 2024-09-17T02:43:50.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 -- 172.21.15.131:0/282419611 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18fc0b6770 con 0x7f18fc070c90 2024-09-17T02:43:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f18fa7fc640 1 -- 172.21.15.131:0/282419611 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5233 (secure 0 0 0) 0x7f18f40164d0 con 0x7f18fc070c90 2024-09-17T02:43:50.117 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:50.117 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":36,"btime":"2024-09-17T02:43:49:702553+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3554877374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3554877374},{"type":"v1","addr":"172.21.15.189:6837","nonce":3554877374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1670590113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1670590113},{"type":"v1","addr":"172.21.15.131:6834","nonce":1670590113}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1046159039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1046159039},{"type":"v1","addr":"172.21.15.131:6835","nonce":1046159039}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:43:46.061550+0000","modified":"2024-09-17T02:43:49.109393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5871":{"gid":5871,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6835/730392436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":730392436},{"type":"v1","addr":"172.21.15.189:6835","nonce":730392436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5871,"qdb_cluster":[5871]},"id":4}]} 2024-09-17T02:43:50.117 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 36 2024-09-17T02:43:50.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 -- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18e8073200 msgr2=0x7f18e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18e8073200 0x7f18e8075710 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0afe70 tx=0x7f18f0009040 comp rx=0 tx=0).stop 2024-09-17T02:43:50.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 -- 172.21.15.131:0/282419611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18fc070c90 msgr2=0x7f18fc0c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18fc070c90 0x7f18fc0c9f60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f18f4009850 tx=0x7f18f4005e00 comp rx=0 tx=0).stop 2024-09-17T02:43:50.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 -- 172.21.15.131:0/282419611 shutdown_connections 2024-09-17T02:43:50.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18e8073200 0x7f18e8075710 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18fc15b2f0 0x7f18fc15dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18fc0cf4e0 0x7f18fc0c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 --2- 172.21.15.131:0/282419611 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18fc070c90 0x7f18fc0c9f60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 -- 172.21.15.131:0/282419611 >> 172.21.15.131:0/282419611 conn(0x7f18fc0cd6a0 msgr2=0x7f18fc0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:50.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 -- 172.21.15.131:0/282419611 shutdown_connections 2024-09-17T02:43:50.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.115+0000 7f1903c70640 1 -- 172.21.15.131:0/282419611 wait complete. 2024-09-17T02:43:50.139 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:43:50.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.291+0000 7f289e153640 1 -- 172.21.15.131:0/1361613023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898156bd0 msgr2=0x7f28980cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.291+0000 7f289e153640 1 --2- 172.21.15.131:0/1361613023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898156bd0 0x7f28980cca70 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f28980d22a0 tx=0x7f288c01c8c0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.291+0000 7f289e153640 1 -- 172.21.15.131:0/1361613023 shutdown_connections 2024-09-17T02:43:50.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.291+0000 7f289e153640 1 --2- 172.21.15.131:0/1361613023 >> v2:172.21.15.189:3300/0 conn(0x7f28980b41a0 0x7f28980b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.291+0000 7f289e153640 1 --2- 172.21.15.131:0/1361613023 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2898156bd0 0x7f28980cca70 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.291+0000 7f289e153640 1 -- 172.21.15.131:0/1361613023 >> 172.21.15.131:0/1361613023 conn(0x7f28980cf150 msgr2=0x7f28980cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:50.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.291+0000 7f289e153640 1 -- 172.21.15.131:0/1361613023 shutdown_connections 2024-09-17T02:43:50.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.291+0000 7f289e153640 1 -- 172.21.15.131:0/1361613023 wait complete. 2024-09-17T02:43:50.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f289e153640 1 Processor -- start 2024-09-17T02:43:50.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f289e153640 1 -- start start 2024-09-17T02:43:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f289e153640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28980b41a0 0x7f2898140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f289e153640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2898156bd0 0x7f28981553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f289e153640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28981601a0 0x7f28980b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f289e153640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2898169d60 con 0x7f28980b41a0 2024-09-17T02:43:50.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f289e153640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f28980b2bc0 con 0x7f28981601a0 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f289e153640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2898169ee0 con 0x7f2898156bd0 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f28977fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28980b41a0 0x7f2898140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f28977fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28980b41a0 0x7f2898140e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55922/0 (socket says 172.21.15.131:55922) 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f28977fe640 1 -- 172.21.15.131:0/2942898016 learned_addr learned my addr 172.21.15.131:0/2942898016 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f2897fff640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28981601a0 0x7f28980b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f2896ffd640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2898156bd0 0x7f28981553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f28977fe640 1 -- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2898156bd0 msgr2=0x7f28981553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f28977fe640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2898156bd0 0x7f28981553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f28977fe640 1 -- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28981601a0 msgr2=0x7f28980b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f28977fe640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28981601a0 0x7f28980b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f28977fe640 1 -- 172.21.15.131:0/2942898016 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2898144e90 con 0x7f28980b41a0 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f28977fe640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28980b41a0 0x7f2898140e40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f288c0194d0 tx=0x7f288c0a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f287ffff640 1 -- 172.21.15.131:0/2942898016 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c0bc070 con 0x7f28980b41a0 2024-09-17T02:43:50.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f287ffff640 1 -- 172.21.15.131:0/2942898016 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f288c0043b0 con 0x7f28980b41a0 2024-09-17T02:43:50.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f289e153640 1 -- 172.21.15.131:0/2942898016 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2898145170 con 0x7f28980b41a0 2024-09-17T02:43:50.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f287ffff640 1 -- 172.21.15.131:0/2942898016 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c0b2040 con 0x7f28980b41a0 2024-09-17T02:43:50.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.295+0000 7f289e153640 1 -- 172.21.15.131:0/2942898016 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2898145730 con 0x7f28980b41a0 2024-09-17T02:43:50.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.299+0000 7f289e153640 1 -- 172.21.15.131:0/2942898016 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28980b45b0 con 0x7f28980b41a0 2024-09-17T02:43:50.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.299+0000 7f287ffff640 1 -- 172.21.15.131:0/2942898016 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f288c005290 con 0x7f28980b41a0 2024-09-17T02:43:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.299+0000 7f287ffff640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2884073200 0x7f2884075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.299+0000 7f287ffff640 1 -- 172.21.15.131:0/2942898016 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f288c12daa0 con 0x7f28980b41a0 2024-09-17T02:43:50.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.299+0000 7f2896ffd640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2884073200 0x7f2884075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:50.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.299+0000 7f2896ffd640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2884073200 0x7f2884075710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f28981462b0 tx=0x7f28880091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:50.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.303+0000 7f287ffff640 1 -- 172.21.15.131:0/2942898016 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f288c0164b0 con 0x7f28980b41a0 2024-09-17T02:43:50.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.475+0000 7f289e153640 1 -- 172.21.15.131:0/2942898016 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28980d00b0 con 0x7f28980b41a0 2024-09-17T02:43:50.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.475+0000 7f287ffff640 1 -- 172.21.15.131:0/2942898016 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+17040 (secure 0 0 0) 0x7f288c0f8a50 con 0x7f28980b41a0 2024-09-17T02:43:50.478 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:50.478 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":65,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:43:47.639182+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:43:43.717619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-17T02:43:44.992949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:43:50.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 -- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2884073200 msgr2=0x7f2884075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2884073200 0x7f2884075710 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f28981462b0 tx=0x7f28880091e0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 -- 172.21.15.131:0/2942898016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28980b41a0 msgr2=0x7f2898140e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28980b41a0 0x7f2898140e40 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f288c0194d0 tx=0x7f288c0a56f0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 -- 172.21.15.131:0/2942898016 shutdown_connections 2024-09-17T02:43:50.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2884073200 0x7f2884075710 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f28981601a0 0x7f28980b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2898156bd0 0x7f28981553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 --2- 172.21.15.131:0/2942898016 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f28980b41a0 0x7f2898140e40 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 -- 172.21.15.131:0/2942898016 >> 172.21.15.131:0/2942898016 conn(0x7f28980cf150 msgr2=0x7f28980c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:50.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 -- 172.21.15.131:0/2942898016 shutdown_connections 2024-09-17T02:43:50.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.479+0000 7f289e153640 1 -- 172.21.15.131:0/2942898016 wait complete. 2024-09-17T02:43:50.500 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T02:43:50.500 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:43:50.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.639+0000 7fba35941640 1 -- 172.21.15.131:0/2087155661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba300c4c90 msgr2=0x7fba300c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.639+0000 7fba35941640 1 --2- 172.21.15.131:0/2087155661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba300c4c90 0x7fba300c50a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fba28009840 tx=0x7fba2801c790 comp rx=0 tx=0).stop 2024-09-17T02:43:50.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.639+0000 7fba35941640 1 -- 172.21.15.131:0/2087155661 shutdown_connections 2024-09-17T02:43:50.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.639+0000 7fba35941640 1 --2- 172.21.15.131:0/2087155661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba300c4c90 0x7fba300c50a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.639+0000 7fba35941640 1 --2- 172.21.15.131:0/2087155661 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba300cec00 0x7fba300cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.639+0000 7fba35941640 1 -- 172.21.15.131:0/2087155661 >> 172.21.15.131:0/2087155661 conn(0x7fba3006eea0 msgr2=0x7fba3006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:50.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.639+0000 7fba35941640 1 -- 172.21.15.131:0/2087155661 shutdown_connections 2024-09-17T02:43:50.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.639+0000 7fba35941640 1 -- 172.21.15.131:0/2087155661 wait complete. 2024-09-17T02:43:50.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.639+0000 7fba35941640 1 Processor -- start 2024-09-17T02:43:50.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.639+0000 7fba35941640 1 -- start start 2024-09-17T02:43:50.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba35941640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba3006e2c0 0x7fba301fa820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:50.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba35941640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba300c4c90 0x7fba3014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:50.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba35941640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba300cec00 0x7fba300c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:50.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba35941640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fba30169e70 con 0x7fba3006e2c0 2024-09-17T02:43:50.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba35941640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fba30169cf0 con 0x7fba300cec00 2024-09-17T02:43:50.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba35941640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fba30169ff0 con 0x7fba300c4c90 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba3006e2c0 0x7fba301fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba3006e2c0 0x7fba301fa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55934/0 (socket says 172.21.15.131:55934) 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2effd640 1 -- 172.21.15.131:0/2983786801 learned_addr learned my addr 172.21.15.131:0/2983786801 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2f7fe640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba300cec00 0x7fba300c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2effd640 1 -- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba300c4c90 msgr2=0x7fba3014ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2e7fc640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba300c4c90 0x7fba3014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2effd640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba300c4c90 0x7fba3014ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2effd640 1 -- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba300cec00 msgr2=0x7fba300c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2effd640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba300cec00 0x7fba300c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2effd640 1 -- 172.21.15.131:0/2983786801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba301508e0 con 0x7fba3006e2c0 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2f7fe640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba300cec00 0x7fba300c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2effd640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba3006e2c0 0x7fba301fa820 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fba1800c400 tx=0x7fba1800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba27fff640 1 -- 172.21.15.131:0/2983786801 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18017070 con 0x7fba3006e2c0 2024-09-17T02:43:50.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba27fff640 1 -- 172.21.15.131:0/2983786801 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba18003600 con 0x7fba3006e2c0 2024-09-17T02:43:50.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba35941640 1 -- 172.21.15.131:0/2983786801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3014f840 con 0x7fba3006e2c0 2024-09-17T02:43:50.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba27fff640 1 -- 172.21.15.131:0/2983786801 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18003a50 con 0x7fba3006e2c0 2024-09-17T02:43:50.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba35941640 1 -- 172.21.15.131:0/2983786801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3014fe00 con 0x7fba3006e2c0 2024-09-17T02:43:50.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba35941640 1 -- 172.21.15.131:0/2983786801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba300cf010 con 0x7fba3006e2c0 2024-09-17T02:43:50.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba27fff640 1 -- 172.21.15.131:0/2983786801 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fba1801a070 con 0x7fba3006e2c0 2024-09-17T02:43:50.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba27fff640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba10073230 0x7fba10075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:50.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba27fff640 1 -- 172.21.15.131:0/2983786801 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fba18093610 con 0x7fba3006e2c0 2024-09-17T02:43:50.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.643+0000 7fba2e7fc640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba10073230 0x7fba10075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:50.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.647+0000 7fba2e7fc640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba10073230 0x7fba10075740 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fba301544a0 tx=0x7fba20008230 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:50.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.651+0000 7fba27fff640 1 -- 172.21.15.131:0/2983786801 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fba1805e5c0 con 0x7fba3006e2c0 2024-09-17T02:43:50.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.811+0000 7fba35941640 1 -- 172.21.15.131:0/2983786801 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba300b19d0 con 0x7fba10073230 2024-09-17T02:43:50.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.827+0000 7fba27fff640 1 -- 172.21.15.131:0/2983786801 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247322 (secure 0 0 0) 0x7fba300b19d0 con 0x7fba10073230 2024-09-17T02:43:50.830 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:50.832 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:43:50.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 -- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba10073230 msgr2=0x7fba10075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba10073230 0x7fba10075740 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fba301544a0 tx=0x7fba20008230 comp rx=0 tx=0).stop 2024-09-17T02:43:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 -- 172.21.15.131:0/2983786801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba3006e2c0 msgr2=0x7fba301fa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba3006e2c0 0x7fba301fa820 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fba1800c400 tx=0x7fba1800c8c0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 -- 172.21.15.131:0/2983786801 shutdown_connections 2024-09-17T02:43:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba10073230 0x7fba10075740 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba300cec00 0x7fba300c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba300c4c90 0x7fba3014ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 --2- 172.21.15.131:0/2983786801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba3006e2c0 0x7fba301fa820 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 -- 172.21.15.131:0/2983786801 >> 172.21.15.131:0/2983786801 conn(0x7fba3006eea0 msgr2=0x7fba300b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 -- 172.21.15.131:0/2983786801 shutdown_connections 2024-09-17T02:43:50.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:50.831+0000 7fba35941640 1 -- 172.21.15.131:0/2983786801 wait complete. 2024-09-17T02:43:50.851 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-17T02:43:49.185955+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248276,"kb_used_data":7056,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726444,"statfs":{"total":773094113280,"available":772839878656,"internally_reserved":0,"allocated":7225344,"data_stored":3863394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.557537"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652370+0000","last_change":"2024-09-17T02:43:46.680494+0000","last_active":"2024-09-17T02:43:47.652370+0000","last_peered":"2024-09-17T02:43:47.652370+0000","last_clean":"2024-09-17T02:43:47.652370+0000","last_became_active":"2024-09-17T02:43:46.679436+0000","last_became_peered":"2024-09-17T02:43:46.679436+0000","last_unstale":"2024-09-17T02:43:47.652370+0000","last_undegraded":"2024-09-17T02:43:47.652370+0000","last_fullsized":"2024-09-17T02:43:47.652370+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:18.520380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.670539+0000","last_change":"2024-09-17T02:43:45.670539+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.670539+0000","last_undegraded":"2024-09-17T02:43:45.670539+0000","last_fullsized":"2024-09-17T02:43:45.670539+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652810+0000","last_change":"2024-09-17T02:43:45.652810+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652810+0000","last_undegraded":"2024-09-17T02:43:45.652810+0000","last_fullsized":"2024-09-17T02:43:45.652810+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.677660+0000","last_change":"2024-09-17T02:43:45.677660+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.677660+0000","last_undegraded":"2024-09-17T02:43:45.677660+0000","last_fullsized":"2024-09-17T02:43:45.677660+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653624+0000","last_change":"2024-09-17T02:43:45.653624+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653624+0000","last_undegraded":"2024-09-17T02:43:45.653624+0000","last_fullsized":"2024-09-17T02:43:45.653624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.679025+0000","last_change":"2024-09-17T02:43:45.679025+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.679025+0000","last_undegraded":"2024-09-17T02:43:45.679025+0000","last_fullsized":"2024-09-17T02:43:45.679025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649842+0000","last_change":"2024-09-17T02:43:46.678701+0000","last_active":"2024-09-17T02:43:47.649842+0000","last_peered":"2024-09-17T02:43:47.649842+0000","last_clean":"2024-09-17T02:43:47.649842+0000","last_became_active":"2024-09-17T02:43:46.674482+0000","last_became_peered":"2024-09-17T02:43:46.674482+0000","last_unstale":"2024-09-17T02:43:47.649842+0000","last_undegraded":"2024-09-17T02:43:47.649842+0000","last_fullsized":"2024-09-17T02:43:47.649842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:30.882039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651787+0000","last_change":"2024-09-17T02:43:45.651787+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651787+0000","last_undegraded":"2024-09-17T02:43:45.651787+0000","last_fullsized":"2024-09-17T02:43:45.651787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.664804+0000","last_change":"2024-09-17T02:43:45.664804+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.664804+0000","last_undegraded":"2024-09-17T02:43:45.664804+0000","last_fullsized":"2024-09-17T02:43:45.664804+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646728+0000","last_change":"2024-09-17T02:43:45.646728+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646728+0000","last_undegraded":"2024-09-17T02:43:45.646728+0000","last_fullsized":"2024-09-17T02:43:45.646728+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.665120+0000","last_change":"2024-09-17T02:43:45.665120+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.665120+0000","last_undegraded":"2024-09-17T02:43:45.665120+0000","last_fullsized":"2024-09-17T02:43:45.665120+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.665103+0000","last_change":"2024-09-17T02:43:45.665103+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.665103+0000","last_undegraded":"2024-09-17T02:43:45.665103+0000","last_fullsized":"2024-09-17T02:43:45.665103+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.655366+0000","last_change":"2024-09-17T02:43:45.655366+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.655366+0000","last_undegraded":"2024-09-17T02:43:45.655366+0000","last_fullsized":"2024-09-17T02:43:45.655366+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654842+0000","last_change":"2024-09-17T02:43:45.654842+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654842+0000","last_undegraded":"2024-09-17T02:43:45.654842+0000","last_fullsized":"2024-09-17T02:43:45.654842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.648693+0000","last_change":"2024-09-17T02:43:45.648693+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.648693+0000","last_undegraded":"2024-09-17T02:43:45.648693+0000","last_fullsized":"2024-09-17T02:43:45.648693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.650433+0000","last_change":"2024-09-17T02:43:45.650433+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.650433+0000","last_undegraded":"2024-09-17T02:43:45.650433+0000","last_fullsized":"2024-09-17T02:43:45.650433+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.658134+0000","last_change":"2024-09-17T02:43:45.658134+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.658134+0000","last_undegraded":"2024-09-17T02:43:45.658134+0000","last_fullsized":"2024-09-17T02:43:45.658134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.662566+0000","last_change":"2024-09-17T02:43:45.662566+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.662566+0000","last_undegraded":"2024-09-17T02:43:45.662566+0000","last_fullsized":"2024-09-17T02:43:45.662566+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.656117+0000","last_change":"2024-09-17T02:43:45.656117+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.656117+0000","last_undegraded":"2024-09-17T02:43:45.656117+0000","last_fullsized":"2024-09-17T02:43:45.656117+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.655100+0000","last_change":"2024-09-17T02:43:45.655100+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.655100+0000","last_undegraded":"2024-09-17T02:43:45.655100+0000","last_fullsized":"2024-09-17T02:43:45.655100+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654859+0000","last_change":"2024-09-17T02:43:45.654859+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654859+0000","last_undegraded":"2024-09-17T02:43:45.654859+0000","last_fullsized":"2024-09-17T02:43:45.654859+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.669183+0000","last_change":"2024-09-17T02:43:45.669183+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.669183+0000","last_undegraded":"2024-09-17T02:43:45.669183+0000","last_fullsized":"2024-09-17T02:43:45.669183+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.648191+0000","last_change":"2024-09-17T02:43:45.648191+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.648191+0000","last_undegraded":"2024-09-17T02:43:45.648191+0000","last_fullsized":"2024-09-17T02:43:45.648191+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651233+0000","last_change":"2024-09-17T02:43:45.651233+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651233+0000","last_undegraded":"2024-09-17T02:43:45.651233+0000","last_fullsized":"2024-09-17T02:43:45.651233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654320+0000","last_change":"2024-09-17T02:43:45.654320+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654320+0000","last_undegraded":"2024-09-17T02:43:45.654320+0000","last_fullsized":"2024-09-17T02:43:45.654320+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652203+0000","last_change":"2024-09-17T02:43:45.652203+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652203+0000","last_undegraded":"2024-09-17T02:43:45.652203+0000","last_fullsized":"2024-09-17T02:43:45.652203+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654755+0000","last_change":"2024-09-17T02:43:45.654755+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654755+0000","last_undegraded":"2024-09-17T02:43:45.654755+0000","last_fullsized":"2024-09-17T02:43:45.654755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652393+0000","last_change":"2024-09-17T02:43:46.713535+0000","last_active":"2024-09-17T02:43:47.652393+0000","last_peered":"2024-09-17T02:43:47.652393+0000","last_clean":"2024-09-17T02:43:47.652393+0000","last_became_active":"2024-09-17T02:43:46.712681+0000","last_became_peered":"2024-09-17T02:43:46.712681+0000","last_unstale":"2024-09-17T02:43:47.652393+0000","last_undegraded":"2024-09-17T02:43:47.652393+0000","last_fullsized":"2024-09-17T02:43:47.652393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:21.464657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.663865+0000","last_change":"2024-09-17T02:43:45.663865+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.663865+0000","last_undegraded":"2024-09-17T02:43:45.663865+0000","last_fullsized":"2024-09-17T02:43:45.663865+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.655569+0000","last_change":"2024-09-17T02:43:45.655569+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.655569+0000","last_undegraded":"2024-09-17T02:43:45.655569+0000","last_fullsized":"2024-09-17T02:43:45.655569+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.650778+0000","last_change":"2024-09-17T02:43:45.650778+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.650778+0000","last_undegraded":"2024-09-17T02:43:45.650778+0000","last_fullsized":"2024-09-17T02:43:45.650778+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649781+0000","last_change":"2024-09-17T02:43:46.673631+0000","last_active":"2024-09-17T02:43:47.649781+0000","last_peered":"2024-09-17T02:43:47.649781+0000","last_clean":"2024-09-17T02:43:47.649781+0000","last_became_active":"2024-09-17T02:43:46.672779+0000","last_became_peered":"2024-09-17T02:43:46.672779+0000","last_unstale":"2024-09-17T02:43:47.649781+0000","last_undegraded":"2024-09-17T02:43:47.649781+0000","last_fullsized":"2024-09-17T02:43:47.649781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:04.209405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.663306+0000","last_change":"2024-09-17T02:43:45.663306+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.663306+0000","last_undegraded":"2024-09-17T02:43:45.663306+0000","last_fullsized":"2024-09-17T02:43:45.663306+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":19,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652160+0000","last_change":"2024-09-17T02:43:45.678361+0000","last_active":"2024-09-17T02:43:47.652160+0000","last_peered":"2024-09-17T02:43:47.652160+0000","last_clean":"2024-09-17T02:43:47.652160+0000","last_became_active":"2024-09-17T02:43:45.677598+0000","last_became_peered":"2024-09-17T02:43:45.677598+0000","last_unstale":"2024-09-17T02:43:47.652160+0000","last_undegraded":"2024-09-17T02:43:47.652160+0000","last_fullsized":"2024-09-17T02:43:47.652160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:08.498525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651601+0000","last_change":"2024-09-17T02:43:45.651601+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651601+0000","last_undegraded":"2024-09-17T02:43:45.651601+0000","last_fullsized":"2024-09-17T02:43:45.651601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651944+0000","last_change":"2024-09-17T02:43:45.660427+0000","last_active":"2024-09-17T02:43:47.651944+0000","last_peered":"2024-09-17T02:43:47.651944+0000","last_clean":"2024-09-17T02:43:47.651944+0000","last_became_active":"2024-09-17T02:43:45.659575+0000","last_became_peered":"2024-09-17T02:43:45.659575+0000","last_unstale":"2024-09-17T02:43:47.651944+0000","last_undegraded":"2024-09-17T02:43:47.651944+0000","last_fullsized":"2024-09-17T02:43:47.651944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:35.225547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.669291+0000","last_change":"2024-09-17T02:43:45.669291+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.669291+0000","last_undegraded":"2024-09-17T02:43:45.669291+0000","last_fullsized":"2024-09-17T02:43:45.669291+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652828+0000","last_change":"2024-09-17T02:43:45.665351+0000","last_active":"2024-09-17T02:43:47.652828+0000","last_peered":"2024-09-17T02:43:47.652828+0000","last_clean":"2024-09-17T02:43:47.652828+0000","last_became_active":"2024-09-17T02:43:45.664498+0000","last_became_peered":"2024-09-17T02:43:45.664498+0000","last_unstale":"2024-09-17T02:43:47.652828+0000","last_undegraded":"2024-09-17T02:43:47.652828+0000","last_fullsized":"2024-09-17T02:43:47.652828+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:35.011302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.650166+0000","last_change":"2024-09-17T02:43:45.650166+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.650166+0000","last_undegraded":"2024-09-17T02:43:45.650166+0000","last_fullsized":"2024-09-17T02:43:45.650166+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.693843+0000","last_change":"2024-09-17T02:43:45.690176+0000","last_active":"2024-09-17T02:43:45.693843+0000","last_peered":"2024-09-17T02:43:45.693843+0000","last_clean":"2024-09-17T02:43:45.693843+0000","last_became_active":"2024-09-17T02:43:45.689355+0000","last_became_peered":"2024-09-17T02:43:45.689355+0000","last_unstale":"2024-09-17T02:43:45.693843+0000","last_undegraded":"2024-09-17T02:43:45.693843+0000","last_fullsized":"2024-09-17T02:43:45.693843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:40.666325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653697+0000","last_change":"2024-09-17T02:43:45.653697+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653697+0000","last_undegraded":"2024-09-17T02:43:45.653697+0000","last_fullsized":"2024-09-17T02:43:45.653697+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.673171+0000","last_change":"2024-09-17T02:43:45.666157+0000","last_active":"2024-09-17T02:43:45.673171+0000","last_peered":"2024-09-17T02:43:45.673171+0000","last_clean":"2024-09-17T02:43:45.673171+0000","last_became_active":"2024-09-17T02:43:45.665300+0000","last_became_peered":"2024-09-17T02:43:45.665300+0000","last_unstale":"2024-09-17T02:43:45.673171+0000","last_undegraded":"2024-09-17T02:43:45.673171+0000","last_fullsized":"2024-09-17T02:43:45.673171+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:17.776424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651046+0000","last_change":"2024-09-17T02:43:46.697615+0000","last_active":"2024-09-17T02:43:47.651046+0000","last_peered":"2024-09-17T02:43:47.651046+0000","last_clean":"2024-09-17T02:43:47.651046+0000","last_became_active":"2024-09-17T02:43:46.697182+0000","last_became_peered":"2024-09-17T02:43:46.697182+0000","last_unstale":"2024-09-17T02:43:47.651046+0000","last_undegraded":"2024-09-17T02:43:47.651046+0000","last_fullsized":"2024-09-17T02:43:47.651046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:48:05.414527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691924+0000","last_change":"2024-09-17T02:43:45.687236+0000","last_active":"2024-09-17T02:43:45.691924+0000","last_peered":"2024-09-17T02:43:45.691924+0000","last_clean":"2024-09-17T02:43:45.691924+0000","last_became_active":"2024-09-17T02:43:45.685471+0000","last_became_peered":"2024-09-17T02:43:45.685471+0000","last_unstale":"2024-09-17T02:43:45.691924+0000","last_undegraded":"2024-09-17T02:43:45.691924+0000","last_fullsized":"2024-09-17T02:43:45.691924+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:39.927955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646618+0000","last_change":"2024-09-17T02:43:45.646618+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646618+0000","last_undegraded":"2024-09-17T02:43:45.646618+0000","last_fullsized":"2024-09-17T02:43:45.646618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651487+0000","last_change":"2024-09-17T02:43:45.661349+0000","last_active":"2024-09-17T02:43:47.651487+0000","last_peered":"2024-09-17T02:43:47.651487+0000","last_clean":"2024-09-17T02:43:47.651487+0000","last_became_active":"2024-09-17T02:43:45.660480+0000","last_became_peered":"2024-09-17T02:43:45.660480+0000","last_unstale":"2024-09-17T02:43:47.651487+0000","last_undegraded":"2024-09-17T02:43:47.651487+0000","last_fullsized":"2024-09-17T02:43:47.651487+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:22.763370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.656839+0000","last_change":"2024-09-17T02:43:45.656839+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.656839+0000","last_undegraded":"2024-09-17T02:43:45.656839+0000","last_fullsized":"2024-09-17T02:43:45.656839+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.677373+0000","last_change":"2024-09-17T02:43:45.655934+0000","last_active":"2024-09-17T02:43:45.677373+0000","last_peered":"2024-09-17T02:43:45.677373+0000","last_clean":"2024-09-17T02:43:45.677373+0000","last_became_active":"2024-09-17T02:43:45.649802+0000","last_became_peered":"2024-09-17T02:43:45.649802+0000","last_unstale":"2024-09-17T02:43:45.677373+0000","last_undegraded":"2024-09-17T02:43:45.677373+0000","last_fullsized":"2024-09-17T02:43:45.677373+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:14:58.433368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652445+0000","last_change":"2024-09-17T02:43:46.677885+0000","last_active":"2024-09-17T02:43:47.652445+0000","last_peered":"2024-09-17T02:43:47.652445+0000","last_clean":"2024-09-17T02:43:47.652445+0000","last_became_active":"2024-09-17T02:43:46.677091+0000","last_became_peered":"2024-09-17T02:43:46.677091+0000","last_unstale":"2024-09-17T02:43:47.652445+0000","last_undegraded":"2024-09-17T02:43:47.652445+0000","last_fullsized":"2024-09-17T02:43:47.652445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:59.608768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.689872+0000","last_change":"2024-09-17T02:43:45.684007+0000","last_active":"2024-09-17T02:43:45.689872+0000","last_peered":"2024-09-17T02:43:45.689872+0000","last_clean":"2024-09-17T02:43:45.689872+0000","last_became_active":"2024-09-17T02:43:45.683525+0000","last_became_peered":"2024-09-17T02:43:45.683525+0000","last_unstale":"2024-09-17T02:43:45.689872+0000","last_undegraded":"2024-09-17T02:43:45.689872+0000","last_fullsized":"2024-09-17T02:43:45.689872+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:49:54.474525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.657805+0000","last_change":"2024-09-17T02:43:45.657805+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.657805+0000","last_undegraded":"2024-09-17T02:43:45.657805+0000","last_fullsized":"2024-09-17T02:43:45.657805+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.667876+0000","last_change":"2024-09-17T02:43:45.667054+0000","last_active":"2024-09-17T02:43:45.667876+0000","last_peered":"2024-09-17T02:43:45.667876+0000","last_clean":"2024-09-17T02:43:45.667876+0000","last_became_active":"2024-09-17T02:43:45.666184+0000","last_became_peered":"2024-09-17T02:43:45.666184+0000","last_unstale":"2024-09-17T02:43:45.667876+0000","last_undegraded":"2024-09-17T02:43:45.667876+0000","last_fullsized":"2024-09-17T02:43:45.667876+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:33.813934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654576+0000","last_change":"2024-09-17T02:43:45.654576+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654576+0000","last_undegraded":"2024-09-17T02:43:45.654576+0000","last_fullsized":"2024-09-17T02:43:45.654576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.687040+0000","last_change":"2024-09-17T02:43:45.679687+0000","last_active":"2024-09-17T02:43:45.687040+0000","last_peered":"2024-09-17T02:43:45.687040+0000","last_clean":"2024-09-17T02:43:45.687040+0000","last_became_active":"2024-09-17T02:43:45.679104+0000","last_became_peered":"2024-09-17T02:43:45.679104+0000","last_unstale":"2024-09-17T02:43:45.687040+0000","last_undegraded":"2024-09-17T02:43:45.687040+0000","last_fullsized":"2024-09-17T02:43:45.687040+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:30.134634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653809+0000","last_change":"2024-09-17T02:43:45.653809+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653809+0000","last_undegraded":"2024-09-17T02:43:45.653809+0000","last_fullsized":"2024-09-17T02:43:45.653809+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.694452+0000","last_change":"2024-09-17T02:43:45.692080+0000","last_active":"2024-09-17T02:43:45.694452+0000","last_peered":"2024-09-17T02:43:45.694452+0000","last_clean":"2024-09-17T02:43:45.694452+0000","last_became_active":"2024-09-17T02:43:45.690832+0000","last_became_peered":"2024-09-17T02:43:45.690832+0000","last_unstale":"2024-09-17T02:43:45.694452+0000","last_undegraded":"2024-09-17T02:43:45.694452+0000","last_fullsized":"2024-09-17T02:43:45.694452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:00.297892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646761+0000","last_change":"2024-09-17T02:43:45.646761+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646761+0000","last_undegraded":"2024-09-17T02:43:45.646761+0000","last_fullsized":"2024-09-17T02:43:45.646761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.677271+0000","last_change":"2024-09-17T02:43:45.651787+0000","last_active":"2024-09-17T02:43:45.677271+0000","last_peered":"2024-09-17T02:43:45.677271+0000","last_clean":"2024-09-17T02:43:45.677271+0000","last_became_active":"2024-09-17T02:43:45.649506+0000","last_became_peered":"2024-09-17T02:43:45.649506+0000","last_unstale":"2024-09-17T02:43:45.677271+0000","last_undegraded":"2024-09-17T02:43:45.677271+0000","last_fullsized":"2024-09-17T02:43:45.677271+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:04.257415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.648211+0000","last_change":"2024-09-17T02:43:45.648211+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.648211+0000","last_undegraded":"2024-09-17T02:43:45.648211+0000","last_fullsized":"2024-09-17T02:43:45.648211+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.664021+0000","last_change":"2024-09-17T02:43:45.661781+0000","last_active":"2024-09-17T02:43:45.664021+0000","last_peered":"2024-09-17T02:43:45.664021+0000","last_clean":"2024-09-17T02:43:45.664021+0000","last_became_active":"2024-09-17T02:43:45.661416+0000","last_became_peered":"2024-09-17T02:43:45.661416+0000","last_unstale":"2024-09-17T02:43:45.664021+0000","last_undegraded":"2024-09-17T02:43:45.664021+0000","last_fullsized":"2024-09-17T02:43:45.664021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:57.140547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.645997+0000","last_change":"2024-09-17T02:43:45.645997+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.645997+0000","last_undegraded":"2024-09-17T02:43:45.645997+0000","last_fullsized":"2024-09-17T02:43:45.645997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.670177+0000","last_change":"2024-09-17T02:43:45.662327+0000","last_active":"2024-09-17T02:43:45.670177+0000","last_peered":"2024-09-17T02:43:45.670177+0000","last_clean":"2024-09-17T02:43:45.670177+0000","last_became_active":"2024-09-17T02:43:45.661481+0000","last_became_peered":"2024-09-17T02:43:45.661481+0000","last_unstale":"2024-09-17T02:43:45.670177+0000","last_undegraded":"2024-09-17T02:43:45.670177+0000","last_fullsized":"2024-09-17T02:43:45.670177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:59.435650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651896+0000","last_change":"2024-09-17T02:43:46.676027+0000","last_active":"2024-09-17T02:43:47.651896+0000","last_peered":"2024-09-17T02:43:47.651896+0000","last_clean":"2024-09-17T02:43:47.651896+0000","last_became_active":"2024-09-17T02:43:46.675175+0000","last_became_peered":"2024-09-17T02:43:46.675175+0000","last_unstale":"2024-09-17T02:43:47.651896+0000","last_undegraded":"2024-09-17T02:43:47.651896+0000","last_fullsized":"2024-09-17T02:43:47.651896+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:54.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.657384+0000","last_change":"2024-09-17T02:43:45.650314+0000","last_active":"2024-09-17T02:43:45.657384+0000","last_peered":"2024-09-17T02:43:45.657384+0000","last_clean":"2024-09-17T02:43:45.657384+0000","last_became_active":"2024-09-17T02:43:45.649675+0000","last_became_peered":"2024-09-17T02:43:45.649675+0000","last_unstale":"2024-09-17T02:43:45.657384+0000","last_undegraded":"2024-09-17T02:43:45.657384+0000","last_fullsized":"2024-09-17T02:43:45.657384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:53.908737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651132+0000","last_change":"2024-09-17T02:43:45.651132+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651132+0000","last_undegraded":"2024-09-17T02:43:45.651132+0000","last_fullsized":"2024-09-17T02:43:45.651132+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.687271+0000","last_change":"2024-09-17T02:43:45.683317+0000","last_active":"2024-09-17T02:43:45.687271+0000","last_peered":"2024-09-17T02:43:45.687271+0000","last_clean":"2024-09-17T02:43:45.687271+0000","last_became_active":"2024-09-17T02:43:45.682686+0000","last_became_peered":"2024-09-17T02:43:45.682686+0000","last_unstale":"2024-09-17T02:43:45.687271+0000","last_undegraded":"2024-09-17T02:43:45.687271+0000","last_fullsized":"2024-09-17T02:43:45.687271+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:22.524365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652497+0000","last_change":"2024-09-17T02:43:45.652497+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652497+0000","last_undegraded":"2024-09-17T02:43:45.652497+0000","last_fullsized":"2024-09-17T02:43:45.652497+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.681461+0000","last_change":"2024-09-17T02:43:45.675971+0000","last_active":"2024-09-17T02:43:45.681461+0000","last_peered":"2024-09-17T02:43:45.681461+0000","last_clean":"2024-09-17T02:43:45.681461+0000","last_became_active":"2024-09-17T02:43:45.665279+0000","last_became_peered":"2024-09-17T02:43:45.665279+0000","last_unstale":"2024-09-17T02:43:45.681461+0000","last_undegraded":"2024-09-17T02:43:45.681461+0000","last_fullsized":"2024-09-17T02:43:45.681461+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:18.856376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653757+0000","last_change":"2024-09-17T02:43:45.653757+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653757+0000","last_undegraded":"2024-09-17T02:43:45.653757+0000","last_fullsized":"2024-09-17T02:43:45.653757+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.673123+0000","last_change":"2024-09-17T02:43:45.669829+0000","last_active":"2024-09-17T02:43:45.673123+0000","last_peered":"2024-09-17T02:43:45.673123+0000","last_clean":"2024-09-17T02:43:45.673123+0000","last_became_active":"2024-09-17T02:43:45.669491+0000","last_became_peered":"2024-09-17T02:43:45.669491+0000","last_unstale":"2024-09-17T02:43:45.673123+0000","last_undegraded":"2024-09-17T02:43:45.673123+0000","last_fullsized":"2024-09-17T02:43:45.673123+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:33.777991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652610+0000","last_change":"2024-09-17T02:43:45.652610+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652610+0000","last_undegraded":"2024-09-17T02:43:45.652610+0000","last_fullsized":"2024-09-17T02:43:45.652610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.681565+0000","last_change":"2024-09-17T02:43:45.674406+0000","last_active":"2024-09-17T02:43:45.681565+0000","last_peered":"2024-09-17T02:43:45.681565+0000","last_clean":"2024-09-17T02:43:45.681565+0000","last_became_active":"2024-09-17T02:43:45.671107+0000","last_became_peered":"2024-09-17T02:43:45.671107+0000","last_unstale":"2024-09-17T02:43:45.681565+0000","last_undegraded":"2024-09-17T02:43:45.681565+0000","last_fullsized":"2024-09-17T02:43:45.681565+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:52:10.241429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.675296+0000","last_change":"2024-09-17T02:43:45.675296+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.675296+0000","last_undegraded":"2024-09-17T02:43:45.675296+0000","last_fullsized":"2024-09-17T02:43:45.675296+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.683619+0000","last_change":"2024-09-17T02:43:45.680606+0000","last_active":"2024-09-17T02:43:45.683619+0000","last_peered":"2024-09-17T02:43:45.683619+0000","last_clean":"2024-09-17T02:43:45.683619+0000","last_became_active":"2024-09-17T02:43:45.679640+0000","last_became_peered":"2024-09-17T02:43:45.679640+0000","last_unstale":"2024-09-17T02:43:45.683619+0000","last_undegraded":"2024-09-17T02:43:45.683619+0000","last_fullsized":"2024-09-17T02:43:45.683619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:24.412203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652569+0000","last_change":"2024-09-17T02:43:45.652569+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652569+0000","last_undegraded":"2024-09-17T02:43:45.652569+0000","last_fullsized":"2024-09-17T02:43:45.652569+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.667071+0000","last_change":"2024-09-17T02:43:45.664123+0000","last_active":"2024-09-17T02:43:45.667071+0000","last_peered":"2024-09-17T02:43:45.667071+0000","last_clean":"2024-09-17T02:43:45.667071+0000","last_became_active":"2024-09-17T02:43:45.663776+0000","last_became_peered":"2024-09-17T02:43:45.663776+0000","last_unstale":"2024-09-17T02:43:45.667071+0000","last_undegraded":"2024-09-17T02:43:45.667071+0000","last_fullsized":"2024-09-17T02:43:45.667071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:21.659777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.650406+0000","last_change":"2024-09-17T02:43:45.650406+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.650406+0000","last_undegraded":"2024-09-17T02:43:45.650406+0000","last_fullsized":"2024-09-17T02:43:45.650406+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691634+0000","last_change":"2024-09-17T02:43:45.684484+0000","last_active":"2024-09-17T02:43:45.691634+0000","last_peered":"2024-09-17T02:43:45.691634+0000","last_clean":"2024-09-17T02:43:45.691634+0000","last_became_active":"2024-09-17T02:43:45.683639+0000","last_became_peered":"2024-09-17T02:43:45.683639+0000","last_unstale":"2024-09-17T02:43:45.691634+0000","last_undegraded":"2024-09-17T02:43:45.691634+0000","last_fullsized":"2024-09-17T02:43:45.691634+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:34.106052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653653+0000","last_change":"2024-09-17T02:43:45.653653+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653653+0000","last_undegraded":"2024-09-17T02:43:45.653653+0000","last_fullsized":"2024-09-17T02:43:45.653653+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.662980+0000","last_change":"2024-09-17T02:43:45.647880+0000","last_active":"2024-09-17T02:43:45.662980+0000","last_peered":"2024-09-17T02:43:45.662980+0000","last_clean":"2024-09-17T02:43:45.662980+0000","last_became_active":"2024-09-17T02:43:45.647466+0000","last_became_peered":"2024-09-17T02:43:45.647466+0000","last_unstale":"2024-09-17T02:43:45.662980+0000","last_undegraded":"2024-09-17T02:43:45.662980+0000","last_fullsized":"2024-09-17T02:43:45.662980+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:13.407925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.656466+0000","last_change":"2024-09-17T02:43:45.656466+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.656466+0000","last_undegraded":"2024-09-17T02:43:45.656466+0000","last_fullsized":"2024-09-17T02:43:45.656466+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.681493+0000","last_change":"2024-09-17T02:43:45.676827+0000","last_active":"2024-09-17T02:43:45.681493+0000","last_peered":"2024-09-17T02:43:45.681493+0000","last_clean":"2024-09-17T02:43:45.681493+0000","last_became_active":"2024-09-17T02:43:45.665485+0000","last_became_peered":"2024-09-17T02:43:45.665485+0000","last_unstale":"2024-09-17T02:43:45.681493+0000","last_undegraded":"2024-09-17T02:43:45.681493+0000","last_fullsized":"2024-09-17T02:43:45.681493+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:00:53.864665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.649260+0000","last_change":"2024-09-17T02:43:45.649260+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.649260+0000","last_undegraded":"2024-09-17T02:43:45.649260+0000","last_fullsized":"2024-09-17T02:43:45.649260+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691707+0000","last_change":"2024-09-17T02:43:45.685852+0000","last_active":"2024-09-17T02:43:45.691707+0000","last_peered":"2024-09-17T02:43:45.691707+0000","last_clean":"2024-09-17T02:43:45.691707+0000","last_became_active":"2024-09-17T02:43:45.683659+0000","last_became_peered":"2024-09-17T02:43:45.683659+0000","last_unstale":"2024-09-17T02:43:45.691707+0000","last_undegraded":"2024-09-17T02:43:45.691707+0000","last_fullsized":"2024-09-17T02:43:45.691707+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:51.724553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651328+0000","last_change":"2024-09-17T02:43:45.651328+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651328+0000","last_undegraded":"2024-09-17T02:43:45.651328+0000","last_fullsized":"2024-09-17T02:43:45.651328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.681643+0000","last_change":"2024-09-17T02:43:45.675150+0000","last_active":"2024-09-17T02:43:45.681643+0000","last_peered":"2024-09-17T02:43:45.681643+0000","last_clean":"2024-09-17T02:43:45.681643+0000","last_became_active":"2024-09-17T02:43:45.671999+0000","last_became_peered":"2024-09-17T02:43:45.671999+0000","last_unstale":"2024-09-17T02:43:45.681643+0000","last_undegraded":"2024-09-17T02:43:45.681643+0000","last_fullsized":"2024-09-17T02:43:45.681643+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:33.813934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.650468+0000","last_change":"2024-09-17T02:43:45.650468+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.650468+0000","last_undegraded":"2024-09-17T02:43:45.650468+0000","last_fullsized":"2024-09-17T02:43:45.650468+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.684341+0000","last_change":"2024-09-17T02:43:45.682922+0000","last_active":"2024-09-17T02:43:45.684341+0000","last_peered":"2024-09-17T02:43:45.684341+0000","last_clean":"2024-09-17T02:43:45.684341+0000","last_became_active":"2024-09-17T02:43:45.682556+0000","last_became_peered":"2024-09-17T02:43:45.682556+0000","last_unstale":"2024-09-17T02:43:45.684341+0000","last_undegraded":"2024-09-17T02:43:45.684341+0000","last_fullsized":"2024-09-17T02:43:45.684341+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:17.815894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651770+0000","last_change":"2024-09-17T02:43:45.651770+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651770+0000","last_undegraded":"2024-09-17T02:43:45.651770+0000","last_fullsized":"2024-09-17T02:43:45.651770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652374+0000","last_change":"2024-09-17T02:43:45.666290+0000","last_active":"2024-09-17T02:43:47.652374+0000","last_peered":"2024-09-17T02:43:47.652374+0000","last_clean":"2024-09-17T02:43:47.652374+0000","last_became_active":"2024-09-17T02:43:45.665410+0000","last_became_peered":"2024-09-17T02:43:45.665410+0000","last_unstale":"2024-09-17T02:43:47.652374+0000","last_undegraded":"2024-09-17T02:43:47.652374+0000","last_fullsized":"2024-09-17T02:43:47.652374+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:33.788490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.677792+0000","last_change":"2024-09-17T02:43:45.677792+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.677792+0000","last_undegraded":"2024-09-17T02:43:45.677792+0000","last_fullsized":"2024-09-17T02:43:45.677792+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.681740+0000","last_change":"2024-09-17T02:43:45.672483+0000","last_active":"2024-09-17T02:43:45.681740+0000","last_peered":"2024-09-17T02:43:45.681740+0000","last_clean":"2024-09-17T02:43:45.681740+0000","last_became_active":"2024-09-17T02:43:45.669710+0000","last_became_peered":"2024-09-17T02:43:45.669710+0000","last_unstale":"2024-09-17T02:43:45.681740+0000","last_undegraded":"2024-09-17T02:43:45.681740+0000","last_fullsized":"2024-09-17T02:43:45.681740+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:13:13.039770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.655418+0000","last_change":"2024-09-17T02:43:45.655418+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.655418+0000","last_undegraded":"2024-09-17T02:43:45.655418+0000","last_fullsized":"2024-09-17T02:43:45.655418+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.693830+0000","last_change":"2024-09-17T02:43:45.690426+0000","last_active":"2024-09-17T02:43:45.693830+0000","last_peered":"2024-09-17T02:43:45.693830+0000","last_clean":"2024-09-17T02:43:45.693830+0000","last_became_active":"2024-09-17T02:43:45.689221+0000","last_became_peered":"2024-09-17T02:43:45.689221+0000","last_unstale":"2024-09-17T02:43:45.693830+0000","last_undegraded":"2024-09-17T02:43:45.693830+0000","last_fullsized":"2024-09-17T02:43:45.693830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:16.815190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.647781+0000","last_change":"2024-09-17T02:43:45.647781+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.647781+0000","last_undegraded":"2024-09-17T02:43:45.647781+0000","last_fullsized":"2024-09-17T02:43:45.647781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691983+0000","last_change":"2024-09-17T02:43:45.688114+0000","last_active":"2024-09-17T02:43:45.691983+0000","last_peered":"2024-09-17T02:43:45.691983+0000","last_clean":"2024-09-17T02:43:45.691983+0000","last_became_active":"2024-09-17T02:43:45.687238+0000","last_became_peered":"2024-09-17T02:43:45.687238+0000","last_unstale":"2024-09-17T02:43:45.691983+0000","last_undegraded":"2024-09-17T02:43:45.691983+0000","last_fullsized":"2024-09-17T02:43:45.691983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:07.931860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":187,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.645912+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:43:45.645912+0000","last_peered":"2024-09-17T02:43:45.645912+0000","last_clean":"2024-09-17T02:43:45.645912+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:43:45.645912+0000","last_undegraded":"2024-09-17T02:43:45.645912+0000","last_fullsized":"2024-09-17T02:43:45.645912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":753664,"data_stored":335817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1335296,"data_stored":919518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509482,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31924,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339916,"statfs":{"total":96636764160,"available":96604073984,"internally_reserved":0,"allocated":753664,"data_stored":335817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":741376,"data_stored":329150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":753664,"data_stored":335817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1335296,"data_stored":919518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31932,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339908,"statfs":{"total":96636764160,"available":96604065792,"internally_reserved":0,"allocated":753664,"data_stored":335817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":798720,"data_stored":351940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:43:50.855 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:43:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.039+0000 7f2826744640 1 -- 172.21.15.131:0/1997577167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282006dea0 msgr2=0x7f282006e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.039+0000 7f2826744640 1 --2- 172.21.15.131:0/1997577167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282006dea0 0x7f282006e2b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2808009840 tx=0x7f280801c810 comp rx=0 tx=0).stop 2024-09-17T02:43:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.039+0000 7f2826744640 1 -- 172.21.15.131:0/1997577167 shutdown_connections 2024-09-17T02:43:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.039+0000 7f2826744640 1 --2- 172.21.15.131:0/1997577167 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2820073800 0x7f2820073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.039+0000 7f2826744640 1 --2- 172.21.15.131:0/1997577167 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282006dea0 0x7f282006e2b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.039+0000 7f2826744640 1 -- 172.21.15.131:0/1997577167 >> 172.21.15.131:0/1997577167 conn(0x7f282006eea0 msgr2=0x7f282006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.039+0000 7f2826744640 1 -- 172.21.15.131:0/1997577167 shutdown_connections 2024-09-17T02:43:51.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.039+0000 7f2826744640 1 -- 172.21.15.131:0/1997577167 wait complete. 2024-09-17T02:43:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.039+0000 7f2826744640 1 Processor -- start 2024-09-17T02:43:51.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.039+0000 7f2826744640 1 -- start start 2024-09-17T02:43:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f2826744640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282006dea0 0x7f282014df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f2826744640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2820073800 0x7f282014c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f2826744640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2820155040 0x7f282014af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f2826744640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2820165f90 con 0x7f282006dea0 2024-09-17T02:43:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f2826744640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2820165e10 con 0x7f2820073800 2024-09-17T02:43:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f2826744640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2820166110 con 0x7f2820155040 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282006dea0 0x7f282014df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282006dea0 0x7f282014df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55962/0 (socket says 172.21.15.131:55962) 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281ffff640 1 -- 172.21.15.131:0/1317826383 learned_addr learned my addr 172.21.15.131:0/1317826383 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f2824ce1640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2820155040 0x7f282014af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281f7fe640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2820073800 0x7f282014c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281ffff640 1 -- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2820155040 msgr2=0x7f282014af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281ffff640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2820155040 0x7f282014af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281ffff640 1 -- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2820073800 msgr2=0x7f282014c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281ffff640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2820073800 0x7f282014c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281ffff640 1 -- 172.21.15.131:0/1317826383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f282014a940 con 0x7f282006dea0 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f2824ce1640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2820155040 0x7f282014af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281ffff640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282006dea0 0x7f282014df30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f281000cb40 tx=0x7f281000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281cff9640 1 -- 172.21.15.131:0/1317826383 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810016020 con 0x7f282006dea0 2024-09-17T02:43:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281cff9640 1 -- 172.21.15.131:0/1317826383 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f281000ab40 con 0x7f282006dea0 2024-09-17T02:43:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f281cff9640 1 -- 172.21.15.131:0/1317826383 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810012420 con 0x7f282006dea0 2024-09-17T02:43:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f2826744640 1 -- 172.21.15.131:0/1317826383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f282015d350 con 0x7f282006dea0 2024-09-17T02:43:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f2826744640 1 -- 172.21.15.131:0/1317826383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f282015d910 con 0x7f282006dea0 2024-09-17T02:43:51.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.043+0000 7f2826744640 1 -- 172.21.15.131:0/1317826383 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f282006e2b0 con 0x7f282006dea0 2024-09-17T02:43:51.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.047+0000 7f281cff9640 1 -- 172.21.15.131:0/1317826383 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2810019020 con 0x7f282006dea0 2024-09-17T02:43:51.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.047+0000 7f281cff9640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2800073200 0x7f2800075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.047+0000 7f281cff9640 1 -- 172.21.15.131:0/1317826383 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f2810094020 con 0x7f282006dea0 2024-09-17T02:43:51.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.047+0000 7f281f7fe640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2800073200 0x7f2800075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:51.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.047+0000 7f281f7fe640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2800073200 0x7f2800075710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28080af000 tx=0x7f280801b050 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.051+0000 7f281cff9640 1 -- 172.21.15.131:0/1317826383 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2810020070 con 0x7f282006dea0 2024-09-17T02:43:51.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.215+0000 7f2826744640 1 -- 172.21.15.131:0/1317826383 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f282015d100 con 0x7f2800073200 2024-09-17T02:43:51.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.227+0000 7f281cff9640 1 -- 172.21.15.131:0/1317826383 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249217 (secure 0 0 0) 0x7f282015d100 con 0x7f2800073200 2024-09-17T02:43:51.234 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:51.235 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:43:51.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 -- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2800073200 msgr2=0x7f2800075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2800073200 0x7f2800075710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f28080af000 tx=0x7f280801b050 comp rx=0 tx=0).stop 2024-09-17T02:43:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 -- 172.21.15.131:0/1317826383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282006dea0 msgr2=0x7f282014df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282006dea0 0x7f282014df30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f281000cb40 tx=0x7f281000cc30 comp rx=0 tx=0).stop 2024-09-17T02:43:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 -- 172.21.15.131:0/1317826383 shutdown_connections 2024-09-17T02:43:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2800073200 0x7f2800075710 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2820155040 0x7f282014af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2820073800 0x7f282014c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 --2- 172.21.15.131:0/1317826383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f282006dea0 0x7f282014df30 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 -- 172.21.15.131:0/1317826383 >> 172.21.15.131:0/1317826383 conn(0x7f282006eea0 msgr2=0x7f2820166fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 -- 172.21.15.131:0/1317826383 shutdown_connections 2024-09-17T02:43:51.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.235+0000 7f2826744640 1 -- 172.21.15.131:0/1317826383 wait complete. 2024-09-17T02:43:51.255 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-09-17T02:43:51.186677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591006,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":119,"num_write_kb":1637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252524,"kb_used_data":7140,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722196,"statfs":{"total":773094113280,"available":772835528704,"internally_reserved":0,"allocated":7311360,"data_stored":3905609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002002"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652370+0000","last_change":"2024-09-17T02:43:46.680494+0000","last_active":"2024-09-17T02:43:47.652370+0000","last_peered":"2024-09-17T02:43:47.652370+0000","last_clean":"2024-09-17T02:43:47.652370+0000","last_became_active":"2024-09-17T02:43:46.679436+0000","last_became_peered":"2024-09-17T02:43:46.679436+0000","last_unstale":"2024-09-17T02:43:47.652370+0000","last_undegraded":"2024-09-17T02:43:47.652370+0000","last_fullsized":"2024-09-17T02:43:47.652370+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:18.520380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.670539+0000","last_change":"2024-09-17T02:43:45.670539+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.670539+0000","last_undegraded":"2024-09-17T02:43:45.670539+0000","last_fullsized":"2024-09-17T02:43:45.670539+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652810+0000","last_change":"2024-09-17T02:43:45.652810+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652810+0000","last_undegraded":"2024-09-17T02:43:45.652810+0000","last_fullsized":"2024-09-17T02:43:45.652810+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701851+0000","last_change":"2024-09-17T02:43:46.708732+0000","last_active":"2024-09-17T02:43:47.701851+0000","last_peered":"2024-09-17T02:43:47.701851+0000","last_clean":"2024-09-17T02:43:47.701851+0000","last_became_active":"2024-09-17T02:43:46.706193+0000","last_became_peered":"2024-09-17T02:43:46.706193+0000","last_unstale":"2024-09-17T02:43:47.701851+0000","last_undegraded":"2024-09-17T02:43:47.701851+0000","last_fullsized":"2024-09-17T02:43:47.701851+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:27.991604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653624+0000","last_change":"2024-09-17T02:43:45.653624+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653624+0000","last_undegraded":"2024-09-17T02:43:45.653624+0000","last_fullsized":"2024-09-17T02:43:45.653624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.679025+0000","last_change":"2024-09-17T02:43:45.679025+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.679025+0000","last_undegraded":"2024-09-17T02:43:45.679025+0000","last_fullsized":"2024-09-17T02:43:45.679025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649842+0000","last_change":"2024-09-17T02:43:46.678701+0000","last_active":"2024-09-17T02:43:47.649842+0000","last_peered":"2024-09-17T02:43:47.649842+0000","last_clean":"2024-09-17T02:43:47.649842+0000","last_became_active":"2024-09-17T02:43:46.674482+0000","last_became_peered":"2024-09-17T02:43:46.674482+0000","last_unstale":"2024-09-17T02:43:47.649842+0000","last_undegraded":"2024-09-17T02:43:47.649842+0000","last_fullsized":"2024-09-17T02:43:47.649842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:30.882039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651787+0000","last_change":"2024-09-17T02:43:45.651787+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651787+0000","last_undegraded":"2024-09-17T02:43:45.651787+0000","last_fullsized":"2024-09-17T02:43:45.651787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702086+0000","last_change":"2024-09-17T02:43:46.695684+0000","last_active":"2024-09-17T02:43:47.702086+0000","last_peered":"2024-09-17T02:43:47.702086+0000","last_clean":"2024-09-17T02:43:47.702086+0000","last_became_active":"2024-09-17T02:43:46.694860+0000","last_became_peered":"2024-09-17T02:43:46.694860+0000","last_unstale":"2024-09-17T02:43:47.702086+0000","last_undegraded":"2024-09-17T02:43:47.702086+0000","last_fullsized":"2024-09-17T02:43:47.702086+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:53.983748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646728+0000","last_change":"2024-09-17T02:43:45.646728+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646728+0000","last_undegraded":"2024-09-17T02:43:45.646728+0000","last_fullsized":"2024-09-17T02:43:45.646728+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702375+0000","last_change":"2024-09-17T02:43:46.701807+0000","last_active":"2024-09-17T02:43:47.702375+0000","last_peered":"2024-09-17T02:43:47.702375+0000","last_clean":"2024-09-17T02:43:47.702375+0000","last_became_active":"2024-09-17T02:43:46.696272+0000","last_became_peered":"2024-09-17T02:43:46.696272+0000","last_unstale":"2024-09-17T02:43:47.702375+0000","last_undegraded":"2024-09-17T02:43:47.702375+0000","last_fullsized":"2024-09-17T02:43:47.702375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:47.850100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702109+0000","last_change":"2024-09-17T02:43:46.707994+0000","last_active":"2024-09-17T02:43:47.702109+0000","last_peered":"2024-09-17T02:43:47.702109+0000","last_clean":"2024-09-17T02:43:47.702109+0000","last_became_active":"2024-09-17T02:43:46.701425+0000","last_became_peered":"2024-09-17T02:43:46.701425+0000","last_unstale":"2024-09-17T02:43:47.702109+0000","last_undegraded":"2024-09-17T02:43:47.702109+0000","last_fullsized":"2024-09-17T02:43:47.702109+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:42.080955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.655366+0000","last_change":"2024-09-17T02:43:45.655366+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.655366+0000","last_undegraded":"2024-09-17T02:43:45.655366+0000","last_fullsized":"2024-09-17T02:43:45.655366+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654842+0000","last_change":"2024-09-17T02:43:45.654842+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654842+0000","last_undegraded":"2024-09-17T02:43:45.654842+0000","last_fullsized":"2024-09-17T02:43:45.654842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.648693+0000","last_change":"2024-09-17T02:43:45.648693+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.648693+0000","last_undegraded":"2024-09-17T02:43:45.648693+0000","last_fullsized":"2024-09-17T02:43:45.648693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707657+0000","last_change":"2024-09-17T02:43:46.708131+0000","last_active":"2024-09-17T02:43:47.707657+0000","last_peered":"2024-09-17T02:43:47.707657+0000","last_clean":"2024-09-17T02:43:47.707657+0000","last_became_active":"2024-09-17T02:43:46.707665+0000","last_became_peered":"2024-09-17T02:43:46.707665+0000","last_unstale":"2024-09-17T02:43:47.707657+0000","last_undegraded":"2024-09-17T02:43:47.707657+0000","last_fullsized":"2024-09-17T02:43:47.707657+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:48:16.207446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.650043+0000","last_change":"2024-09-17T02:43:46.702269+0000","last_active":"2024-09-17T02:43:47.650043+0000","last_peered":"2024-09-17T02:43:47.650043+0000","last_clean":"2024-09-17T02:43:47.650043+0000","last_became_active":"2024-09-17T02:43:46.701883+0000","last_became_peered":"2024-09-17T02:43:46.701883+0000","last_unstale":"2024-09-17T02:43:47.650043+0000","last_undegraded":"2024-09-17T02:43:47.650043+0000","last_fullsized":"2024-09-17T02:43:47.650043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:22.975866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702426+0000","last_change":"2024-09-17T02:43:46.693189+0000","last_active":"2024-09-17T02:43:47.702426+0000","last_peered":"2024-09-17T02:43:47.702426+0000","last_clean":"2024-09-17T02:43:47.702426+0000","last_became_active":"2024-09-17T02:43:46.691164+0000","last_became_peered":"2024-09-17T02:43:46.691164+0000","last_unstale":"2024-09-17T02:43:47.702426+0000","last_undegraded":"2024-09-17T02:43:47.702426+0000","last_fullsized":"2024-09-17T02:43:47.702426+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:52.980125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702146+0000","last_change":"2024-09-17T02:43:46.692645+0000","last_active":"2024-09-17T02:43:47.702146+0000","last_peered":"2024-09-17T02:43:47.702146+0000","last_clean":"2024-09-17T02:43:47.702146+0000","last_became_active":"2024-09-17T02:43:46.691861+0000","last_became_peered":"2024-09-17T02:43:46.691861+0000","last_unstale":"2024-09-17T02:43:47.702146+0000","last_undegraded":"2024-09-17T02:43:47.702146+0000","last_fullsized":"2024-09-17T02:43:47.702146+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:03.405001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647781+0000","last_change":"2024-09-17T02:43:46.684848+0000","last_active":"2024-09-17T02:43:47.647781+0000","last_peered":"2024-09-17T02:43:47.647781+0000","last_clean":"2024-09-17T02:43:47.647781+0000","last_became_active":"2024-09-17T02:43:46.683919+0000","last_became_peered":"2024-09-17T02:43:46.683919+0000","last_unstale":"2024-09-17T02:43:47.647781+0000","last_undegraded":"2024-09-17T02:43:47.647781+0000","last_fullsized":"2024-09-17T02:43:47.647781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:58.516266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654859+0000","last_change":"2024-09-17T02:43:45.654859+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654859+0000","last_undegraded":"2024-09-17T02:43:45.654859+0000","last_fullsized":"2024-09-17T02:43:45.654859+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.669183+0000","last_change":"2024-09-17T02:43:45.669183+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.669183+0000","last_undegraded":"2024-09-17T02:43:45.669183+0000","last_fullsized":"2024-09-17T02:43:45.669183+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707376+0000","last_change":"2024-09-17T02:43:46.708531+0000","last_active":"2024-09-17T02:43:47.707376+0000","last_peered":"2024-09-17T02:43:47.707376+0000","last_clean":"2024-09-17T02:43:47.707376+0000","last_became_active":"2024-09-17T02:43:46.708224+0000","last_became_peered":"2024-09-17T02:43:46.708224+0000","last_unstale":"2024-09-17T02:43:47.707376+0000","last_undegraded":"2024-09-17T02:43:47.707376+0000","last_fullsized":"2024-09-17T02:43:47.707376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:37.303173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651233+0000","last_change":"2024-09-17T02:43:45.651233+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651233+0000","last_undegraded":"2024-09-17T02:43:45.651233+0000","last_fullsized":"2024-09-17T02:43:45.651233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654320+0000","last_change":"2024-09-17T02:43:45.654320+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654320+0000","last_undegraded":"2024-09-17T02:43:45.654320+0000","last_fullsized":"2024-09-17T02:43:45.654320+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652203+0000","last_change":"2024-09-17T02:43:45.652203+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652203+0000","last_undegraded":"2024-09-17T02:43:45.652203+0000","last_fullsized":"2024-09-17T02:43:45.652203+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654755+0000","last_change":"2024-09-17T02:43:45.654755+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654755+0000","last_undegraded":"2024-09-17T02:43:45.654755+0000","last_fullsized":"2024-09-17T02:43:45.654755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652393+0000","last_change":"2024-09-17T02:43:46.713535+0000","last_active":"2024-09-17T02:43:47.652393+0000","last_peered":"2024-09-17T02:43:47.652393+0000","last_clean":"2024-09-17T02:43:47.652393+0000","last_became_active":"2024-09-17T02:43:46.712681+0000","last_became_peered":"2024-09-17T02:43:46.712681+0000","last_unstale":"2024-09-17T02:43:47.652393+0000","last_undegraded":"2024-09-17T02:43:47.652393+0000","last_fullsized":"2024-09-17T02:43:47.652393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:21.464657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701453+0000","last_change":"2024-09-17T02:43:46.706426+0000","last_active":"2024-09-17T02:43:47.701453+0000","last_peered":"2024-09-17T02:43:47.701453+0000","last_clean":"2024-09-17T02:43:47.701453+0000","last_became_active":"2024-09-17T02:43:46.706046+0000","last_became_peered":"2024-09-17T02:43:46.706046+0000","last_unstale":"2024-09-17T02:43:47.701453+0000","last_undegraded":"2024-09-17T02:43:47.701453+0000","last_fullsized":"2024-09-17T02:43:47.701453+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:58.594878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700841+0000","last_change":"2024-09-17T02:43:46.702605+0000","last_active":"2024-09-17T02:43:47.700841+0000","last_peered":"2024-09-17T02:43:47.700841+0000","last_clean":"2024-09-17T02:43:47.700841+0000","last_became_active":"2024-09-17T02:43:46.702182+0000","last_became_peered":"2024-09-17T02:43:46.702182+0000","last_unstale":"2024-09-17T02:43:47.700841+0000","last_undegraded":"2024-09-17T02:43:47.700841+0000","last_fullsized":"2024-09-17T02:43:47.700841+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:17.795947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.650778+0000","last_change":"2024-09-17T02:43:45.650778+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.650778+0000","last_undegraded":"2024-09-17T02:43:45.650778+0000","last_fullsized":"2024-09-17T02:43:45.650778+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649781+0000","last_change":"2024-09-17T02:43:46.673631+0000","last_active":"2024-09-17T02:43:47.649781+0000","last_peered":"2024-09-17T02:43:47.649781+0000","last_clean":"2024-09-17T02:43:47.649781+0000","last_became_active":"2024-09-17T02:43:46.672779+0000","last_became_peered":"2024-09-17T02:43:46.672779+0000","last_unstale":"2024-09-17T02:43:47.649781+0000","last_undegraded":"2024-09-17T02:43:47.649781+0000","last_fullsized":"2024-09-17T02:43:47.649781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:04.209405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702521+0000","last_change":"2024-09-17T02:43:46.710347+0000","last_active":"2024-09-17T02:43:47.702521+0000","last_peered":"2024-09-17T02:43:47.702521+0000","last_clean":"2024-09-17T02:43:47.702521+0000","last_became_active":"2024-09-17T02:43:46.708415+0000","last_became_peered":"2024-09-17T02:43:46.708415+0000","last_unstale":"2024-09-17T02:43:47.702521+0000","last_undegraded":"2024-09-17T02:43:47.702521+0000","last_fullsized":"2024-09-17T02:43:47.702521+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:23.928100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":19,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652160+0000","last_change":"2024-09-17T02:43:45.678361+0000","last_active":"2024-09-17T02:43:47.652160+0000","last_peered":"2024-09-17T02:43:47.652160+0000","last_clean":"2024-09-17T02:43:47.652160+0000","last_became_active":"2024-09-17T02:43:45.677598+0000","last_became_peered":"2024-09-17T02:43:45.677598+0000","last_unstale":"2024-09-17T02:43:47.652160+0000","last_undegraded":"2024-09-17T02:43:47.652160+0000","last_fullsized":"2024-09-17T02:43:47.652160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:08.498525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651601+0000","last_change":"2024-09-17T02:43:45.651601+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651601+0000","last_undegraded":"2024-09-17T02:43:45.651601+0000","last_fullsized":"2024-09-17T02:43:45.651601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651944+0000","last_change":"2024-09-17T02:43:45.660427+0000","last_active":"2024-09-17T02:43:47.651944+0000","last_peered":"2024-09-17T02:43:47.651944+0000","last_clean":"2024-09-17T02:43:47.651944+0000","last_became_active":"2024-09-17T02:43:45.659575+0000","last_became_peered":"2024-09-17T02:43:45.659575+0000","last_unstale":"2024-09-17T02:43:47.651944+0000","last_undegraded":"2024-09-17T02:43:47.651944+0000","last_fullsized":"2024-09-17T02:43:47.651944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:35.225547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.669291+0000","last_change":"2024-09-17T02:43:45.669291+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.669291+0000","last_undegraded":"2024-09-17T02:43:45.669291+0000","last_fullsized":"2024-09-17T02:43:45.669291+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652828+0000","last_change":"2024-09-17T02:43:45.665351+0000","last_active":"2024-09-17T02:43:47.652828+0000","last_peered":"2024-09-17T02:43:47.652828+0000","last_clean":"2024-09-17T02:43:47.652828+0000","last_became_active":"2024-09-17T02:43:45.664498+0000","last_became_peered":"2024-09-17T02:43:45.664498+0000","last_unstale":"2024-09-17T02:43:47.652828+0000","last_undegraded":"2024-09-17T02:43:47.652828+0000","last_fullsized":"2024-09-17T02:43:47.652828+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:35.011302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703229+0000","last_change":"2024-09-17T02:43:46.694855+0000","last_active":"2024-09-17T02:43:47.703229+0000","last_peered":"2024-09-17T02:43:47.703229+0000","last_clean":"2024-09-17T02:43:47.703229+0000","last_became_active":"2024-09-17T02:43:46.692123+0000","last_became_peered":"2024-09-17T02:43:46.692123+0000","last_unstale":"2024-09-17T02:43:47.703229+0000","last_undegraded":"2024-09-17T02:43:47.703229+0000","last_fullsized":"2024-09-17T02:43:47.703229+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:38.594076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.693843+0000","last_change":"2024-09-17T02:43:45.690176+0000","last_active":"2024-09-17T02:43:45.693843+0000","last_peered":"2024-09-17T02:43:45.693843+0000","last_clean":"2024-09-17T02:43:45.693843+0000","last_became_active":"2024-09-17T02:43:45.689355+0000","last_became_peered":"2024-09-17T02:43:45.689355+0000","last_unstale":"2024-09-17T02:43:45.693843+0000","last_undegraded":"2024-09-17T02:43:45.693843+0000","last_fullsized":"2024-09-17T02:43:45.693843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:40.666325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.646844+0000","last_change":"2024-09-17T02:43:46.678861+0000","last_active":"2024-09-17T02:43:47.646844+0000","last_peered":"2024-09-17T02:43:47.646844+0000","last_clean":"2024-09-17T02:43:47.646844+0000","last_became_active":"2024-09-17T02:43:46.677801+0000","last_became_peered":"2024-09-17T02:43:46.677801+0000","last_unstale":"2024-09-17T02:43:47.646844+0000","last_undegraded":"2024-09-17T02:43:47.646844+0000","last_fullsized":"2024-09-17T02:43:47.646844+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:31.162139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647995+0000","last_change":"2024-09-17T02:43:45.666157+0000","last_active":"2024-09-17T02:43:47.647995+0000","last_peered":"2024-09-17T02:43:47.647995+0000","last_clean":"2024-09-17T02:43:47.647995+0000","last_became_active":"2024-09-17T02:43:45.665300+0000","last_became_peered":"2024-09-17T02:43:45.665300+0000","last_unstale":"2024-09-17T02:43:47.647995+0000","last_undegraded":"2024-09-17T02:43:47.647995+0000","last_fullsized":"2024-09-17T02:43:47.647995+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:41.076834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651046+0000","last_change":"2024-09-17T02:43:46.697615+0000","last_active":"2024-09-17T02:43:47.651046+0000","last_peered":"2024-09-17T02:43:47.651046+0000","last_clean":"2024-09-17T02:43:47.651046+0000","last_became_active":"2024-09-17T02:43:46.697182+0000","last_became_peered":"2024-09-17T02:43:46.697182+0000","last_unstale":"2024-09-17T02:43:47.651046+0000","last_undegraded":"2024-09-17T02:43:47.651046+0000","last_fullsized":"2024-09-17T02:43:47.651046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:48:05.414527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691924+0000","last_change":"2024-09-17T02:43:45.687236+0000","last_active":"2024-09-17T02:43:45.691924+0000","last_peered":"2024-09-17T02:43:45.691924+0000","last_clean":"2024-09-17T02:43:45.691924+0000","last_became_active":"2024-09-17T02:43:45.685471+0000","last_became_peered":"2024-09-17T02:43:45.685471+0000","last_unstale":"2024-09-17T02:43:45.691924+0000","last_undegraded":"2024-09-17T02:43:45.691924+0000","last_fullsized":"2024-09-17T02:43:45.691924+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:39.927955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646618+0000","last_change":"2024-09-17T02:43:45.646618+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646618+0000","last_undegraded":"2024-09-17T02:43:45.646618+0000","last_fullsized":"2024-09-17T02:43:45.646618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651487+0000","last_change":"2024-09-17T02:43:45.661349+0000","last_active":"2024-09-17T02:43:47.651487+0000","last_peered":"2024-09-17T02:43:47.651487+0000","last_clean":"2024-09-17T02:43:47.651487+0000","last_became_active":"2024-09-17T02:43:45.660480+0000","last_became_peered":"2024-09-17T02:43:45.660480+0000","last_unstale":"2024-09-17T02:43:47.651487+0000","last_undegraded":"2024-09-17T02:43:47.651487+0000","last_fullsized":"2024-09-17T02:43:47.651487+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:22.763370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647445+0000","last_change":"2024-09-17T02:43:46.694871+0000","last_active":"2024-09-17T02:43:47.647445+0000","last_peered":"2024-09-17T02:43:47.647445+0000","last_clean":"2024-09-17T02:43:47.647445+0000","last_became_active":"2024-09-17T02:43:46.694495+0000","last_became_peered":"2024-09-17T02:43:46.694495+0000","last_unstale":"2024-09-17T02:43:47.647445+0000","last_undegraded":"2024-09-17T02:43:47.647445+0000","last_fullsized":"2024-09-17T02:43:47.647445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:01.223420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700581+0000","last_change":"2024-09-17T02:43:45.655934+0000","last_active":"2024-09-17T02:43:47.700581+0000","last_peered":"2024-09-17T02:43:47.700581+0000","last_clean":"2024-09-17T02:43:47.700581+0000","last_became_active":"2024-09-17T02:43:45.649802+0000","last_became_peered":"2024-09-17T02:43:45.649802+0000","last_unstale":"2024-09-17T02:43:47.700581+0000","last_undegraded":"2024-09-17T02:43:47.700581+0000","last_fullsized":"2024-09-17T02:43:47.700581+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:24.816574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652445+0000","last_change":"2024-09-17T02:43:46.677885+0000","last_active":"2024-09-17T02:43:47.652445+0000","last_peered":"2024-09-17T02:43:47.652445+0000","last_clean":"2024-09-17T02:43:47.652445+0000","last_became_active":"2024-09-17T02:43:46.677091+0000","last_became_peered":"2024-09-17T02:43:46.677091+0000","last_unstale":"2024-09-17T02:43:47.652445+0000","last_undegraded":"2024-09-17T02:43:47.652445+0000","last_fullsized":"2024-09-17T02:43:47.652445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:59.608768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703726+0000","last_change":"2024-09-17T02:43:45.684007+0000","last_active":"2024-09-17T02:43:47.703726+0000","last_peered":"2024-09-17T02:43:47.703726+0000","last_clean":"2024-09-17T02:43:47.703726+0000","last_became_active":"2024-09-17T02:43:45.683525+0000","last_became_peered":"2024-09-17T02:43:45.683525+0000","last_unstale":"2024-09-17T02:43:47.703726+0000","last_undegraded":"2024-09-17T02:43:47.703726+0000","last_fullsized":"2024-09-17T02:43:47.703726+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:41.128475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647829+0000","last_change":"2024-09-17T02:43:46.668355+0000","last_active":"2024-09-17T02:43:47.647829+0000","last_peered":"2024-09-17T02:43:47.647829+0000","last_clean":"2024-09-17T02:43:47.647829+0000","last_became_active":"2024-09-17T02:43:46.667493+0000","last_became_peered":"2024-09-17T02:43:46.667493+0000","last_unstale":"2024-09-17T02:43:47.647829+0000","last_undegraded":"2024-09-17T02:43:47.647829+0000","last_fullsized":"2024-09-17T02:43:47.647829+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:11.487744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.667876+0000","last_change":"2024-09-17T02:43:45.667054+0000","last_active":"2024-09-17T02:43:45.667876+0000","last_peered":"2024-09-17T02:43:45.667876+0000","last_clean":"2024-09-17T02:43:45.667876+0000","last_became_active":"2024-09-17T02:43:45.666184+0000","last_became_peered":"2024-09-17T02:43:45.666184+0000","last_unstale":"2024-09-17T02:43:45.667876+0000","last_undegraded":"2024-09-17T02:43:45.667876+0000","last_fullsized":"2024-09-17T02:43:45.667876+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:33.813934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654576+0000","last_change":"2024-09-17T02:43:45.654576+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654576+0000","last_undegraded":"2024-09-17T02:43:45.654576+0000","last_fullsized":"2024-09-17T02:43:45.654576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.687040+0000","last_change":"2024-09-17T02:43:45.679687+0000","last_active":"2024-09-17T02:43:45.687040+0000","last_peered":"2024-09-17T02:43:45.687040+0000","last_clean":"2024-09-17T02:43:45.687040+0000","last_became_active":"2024-09-17T02:43:45.679104+0000","last_became_peered":"2024-09-17T02:43:45.679104+0000","last_unstale":"2024-09-17T02:43:45.687040+0000","last_undegraded":"2024-09-17T02:43:45.687040+0000","last_fullsized":"2024-09-17T02:43:45.687040+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:30.134634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653809+0000","last_change":"2024-09-17T02:43:45.653809+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653809+0000","last_undegraded":"2024-09-17T02:43:45.653809+0000","last_fullsized":"2024-09-17T02:43:45.653809+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.694452+0000","last_change":"2024-09-17T02:43:45.692080+0000","last_active":"2024-09-17T02:43:45.694452+0000","last_peered":"2024-09-17T02:43:45.694452+0000","last_clean":"2024-09-17T02:43:45.694452+0000","last_became_active":"2024-09-17T02:43:45.690832+0000","last_became_peered":"2024-09-17T02:43:45.690832+0000","last_unstale":"2024-09-17T02:43:45.694452+0000","last_undegraded":"2024-09-17T02:43:45.694452+0000","last_fullsized":"2024-09-17T02:43:45.694452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:00.297892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646761+0000","last_change":"2024-09-17T02:43:45.646761+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646761+0000","last_undegraded":"2024-09-17T02:43:45.646761+0000","last_fullsized":"2024-09-17T02:43:45.646761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701443+0000","last_change":"2024-09-17T02:43:45.651787+0000","last_active":"2024-09-17T02:43:47.701443+0000","last_peered":"2024-09-17T02:43:47.701443+0000","last_clean":"2024-09-17T02:43:47.701443+0000","last_became_active":"2024-09-17T02:43:45.649506+0000","last_became_peered":"2024-09-17T02:43:45.649506+0000","last_unstale":"2024-09-17T02:43:47.701443+0000","last_undegraded":"2024-09-17T02:43:47.701443+0000","last_fullsized":"2024-09-17T02:43:47.701443+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:39.203265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.648211+0000","last_change":"2024-09-17T02:43:45.648211+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.648211+0000","last_undegraded":"2024-09-17T02:43:45.648211+0000","last_fullsized":"2024-09-17T02:43:45.648211+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.664021+0000","last_change":"2024-09-17T02:43:45.661781+0000","last_active":"2024-09-17T02:43:45.664021+0000","last_peered":"2024-09-17T02:43:45.664021+0000","last_clean":"2024-09-17T02:43:45.664021+0000","last_became_active":"2024-09-17T02:43:45.661416+0000","last_became_peered":"2024-09-17T02:43:45.661416+0000","last_unstale":"2024-09-17T02:43:45.664021+0000","last_undegraded":"2024-09-17T02:43:45.664021+0000","last_fullsized":"2024-09-17T02:43:45.664021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:57.140547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.645997+0000","last_change":"2024-09-17T02:43:45.645997+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.645997+0000","last_undegraded":"2024-09-17T02:43:45.645997+0000","last_fullsized":"2024-09-17T02:43:45.645997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.670177+0000","last_change":"2024-09-17T02:43:45.662327+0000","last_active":"2024-09-17T02:43:45.670177+0000","last_peered":"2024-09-17T02:43:45.670177+0000","last_clean":"2024-09-17T02:43:45.670177+0000","last_became_active":"2024-09-17T02:43:45.661481+0000","last_became_peered":"2024-09-17T02:43:45.661481+0000","last_unstale":"2024-09-17T02:43:45.670177+0000","last_undegraded":"2024-09-17T02:43:45.670177+0000","last_fullsized":"2024-09-17T02:43:45.670177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:59.435650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651896+0000","last_change":"2024-09-17T02:43:46.676027+0000","last_active":"2024-09-17T02:43:47.651896+0000","last_peered":"2024-09-17T02:43:47.651896+0000","last_clean":"2024-09-17T02:43:47.651896+0000","last_became_active":"2024-09-17T02:43:46.675175+0000","last_became_peered":"2024-09-17T02:43:46.675175+0000","last_unstale":"2024-09-17T02:43:47.651896+0000","last_undegraded":"2024-09-17T02:43:47.651896+0000","last_fullsized":"2024-09-17T02:43:47.651896+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:54.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.657384+0000","last_change":"2024-09-17T02:43:45.650314+0000","last_active":"2024-09-17T02:43:45.657384+0000","last_peered":"2024-09-17T02:43:45.657384+0000","last_clean":"2024-09-17T02:43:45.657384+0000","last_became_active":"2024-09-17T02:43:45.649675+0000","last_became_peered":"2024-09-17T02:43:45.649675+0000","last_unstale":"2024-09-17T02:43:45.657384+0000","last_undegraded":"2024-09-17T02:43:45.657384+0000","last_fullsized":"2024-09-17T02:43:45.657384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:53.908737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702763+0000","last_change":"2024-09-17T02:43:46.707370+0000","last_active":"2024-09-17T02:43:47.702763+0000","last_peered":"2024-09-17T02:43:47.702763+0000","last_clean":"2024-09-17T02:43:47.702763+0000","last_became_active":"2024-09-17T02:43:46.697031+0000","last_became_peered":"2024-09-17T02:43:46.697031+0000","last_unstale":"2024-09-17T02:43:47.702763+0000","last_undegraded":"2024-09-17T02:43:47.702763+0000","last_fullsized":"2024-09-17T02:43:47.702763+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:53.627837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703754+0000","last_change":"2024-09-17T02:43:45.683317+0000","last_active":"2024-09-17T02:43:47.703754+0000","last_peered":"2024-09-17T02:43:47.703754+0000","last_clean":"2024-09-17T02:43:47.703754+0000","last_became_active":"2024-09-17T02:43:45.682686+0000","last_became_peered":"2024-09-17T02:43:45.682686+0000","last_unstale":"2024-09-17T02:43:47.703754+0000","last_undegraded":"2024-09-17T02:43:47.703754+0000","last_fullsized":"2024-09-17T02:43:47.703754+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:45.603856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652497+0000","last_change":"2024-09-17T02:43:45.652497+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652497+0000","last_undegraded":"2024-09-17T02:43:45.652497+0000","last_fullsized":"2024-09-17T02:43:45.652497+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702089+0000","last_change":"2024-09-17T02:43:45.675971+0000","last_active":"2024-09-17T02:43:47.702089+0000","last_peered":"2024-09-17T02:43:47.702089+0000","last_clean":"2024-09-17T02:43:47.702089+0000","last_became_active":"2024-09-17T02:43:45.665279+0000","last_became_peered":"2024-09-17T02:43:45.665279+0000","last_unstale":"2024-09-17T02:43:47.702089+0000","last_undegraded":"2024-09-17T02:43:47.702089+0000","last_fullsized":"2024-09-17T02:43:47.702089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:02.887437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653757+0000","last_change":"2024-09-17T02:43:45.653757+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653757+0000","last_undegraded":"2024-09-17T02:43:45.653757+0000","last_fullsized":"2024-09-17T02:43:45.653757+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.673123+0000","last_change":"2024-09-17T02:43:45.669829+0000","last_active":"2024-09-17T02:43:45.673123+0000","last_peered":"2024-09-17T02:43:45.673123+0000","last_clean":"2024-09-17T02:43:45.673123+0000","last_became_active":"2024-09-17T02:43:45.669491+0000","last_became_peered":"2024-09-17T02:43:45.669491+0000","last_unstale":"2024-09-17T02:43:45.673123+0000","last_undegraded":"2024-09-17T02:43:45.673123+0000","last_fullsized":"2024-09-17T02:43:45.673123+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:33.777991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652610+0000","last_change":"2024-09-17T02:43:45.652610+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652610+0000","last_undegraded":"2024-09-17T02:43:45.652610+0000","last_fullsized":"2024-09-17T02:43:45.652610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707366+0000","last_change":"2024-09-17T02:43:45.674406+0000","last_active":"2024-09-17T02:43:47.707366+0000","last_peered":"2024-09-17T02:43:47.707366+0000","last_clean":"2024-09-17T02:43:47.707366+0000","last_became_active":"2024-09-17T02:43:45.671107+0000","last_became_peered":"2024-09-17T02:43:45.671107+0000","last_unstale":"2024-09-17T02:43:47.707366+0000","last_undegraded":"2024-09-17T02:43:47.707366+0000","last_fullsized":"2024-09-17T02:43:47.707366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:13.407925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.675296+0000","last_change":"2024-09-17T02:43:45.675296+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.675296+0000","last_undegraded":"2024-09-17T02:43:45.675296+0000","last_fullsized":"2024-09-17T02:43:45.675296+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.683619+0000","last_change":"2024-09-17T02:43:45.680606+0000","last_active":"2024-09-17T02:43:45.683619+0000","last_peered":"2024-09-17T02:43:45.683619+0000","last_clean":"2024-09-17T02:43:45.683619+0000","last_became_active":"2024-09-17T02:43:45.679640+0000","last_became_peered":"2024-09-17T02:43:45.679640+0000","last_unstale":"2024-09-17T02:43:45.683619+0000","last_undegraded":"2024-09-17T02:43:45.683619+0000","last_fullsized":"2024-09-17T02:43:45.683619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:24.412203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652569+0000","last_change":"2024-09-17T02:43:45.652569+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652569+0000","last_undegraded":"2024-09-17T02:43:45.652569+0000","last_fullsized":"2024-09-17T02:43:45.652569+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.667071+0000","last_change":"2024-09-17T02:43:45.664123+0000","last_active":"2024-09-17T02:43:45.667071+0000","last_peered":"2024-09-17T02:43:45.667071+0000","last_clean":"2024-09-17T02:43:45.667071+0000","last_became_active":"2024-09-17T02:43:45.663776+0000","last_became_peered":"2024-09-17T02:43:45.663776+0000","last_unstale":"2024-09-17T02:43:45.667071+0000","last_undegraded":"2024-09-17T02:43:45.667071+0000","last_fullsized":"2024-09-17T02:43:45.667071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:21.659777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700904+0000","last_change":"2024-09-17T02:43:46.693857+0000","last_active":"2024-09-17T02:43:47.700904+0000","last_peered":"2024-09-17T02:43:47.700904+0000","last_clean":"2024-09-17T02:43:47.700904+0000","last_became_active":"2024-09-17T02:43:46.692225+0000","last_became_peered":"2024-09-17T02:43:46.692225+0000","last_unstale":"2024-09-17T02:43:47.700904+0000","last_undegraded":"2024-09-17T02:43:47.700904+0000","last_fullsized":"2024-09-17T02:43:47.700904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:58:56.128166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691634+0000","last_change":"2024-09-17T02:43:45.684484+0000","last_active":"2024-09-17T02:43:45.691634+0000","last_peered":"2024-09-17T02:43:45.691634+0000","last_clean":"2024-09-17T02:43:45.691634+0000","last_became_active":"2024-09-17T02:43:45.683639+0000","last_became_peered":"2024-09-17T02:43:45.683639+0000","last_unstale":"2024-09-17T02:43:45.691634+0000","last_undegraded":"2024-09-17T02:43:45.691634+0000","last_fullsized":"2024-09-17T02:43:45.691634+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:34.106052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.648456+0000","last_change":"2024-09-17T02:43:46.674643+0000","last_active":"2024-09-17T02:43:47.648456+0000","last_peered":"2024-09-17T02:43:47.648456+0000","last_clean":"2024-09-17T02:43:47.648456+0000","last_became_active":"2024-09-17T02:43:46.673681+0000","last_became_peered":"2024-09-17T02:43:46.673681+0000","last_unstale":"2024-09-17T02:43:47.648456+0000","last_undegraded":"2024-09-17T02:43:47.648456+0000","last_fullsized":"2024-09-17T02:43:47.648456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:09.748085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703860+0000","last_change":"2024-09-17T02:43:45.647880+0000","last_active":"2024-09-17T02:43:47.703860+0000","last_peered":"2024-09-17T02:43:47.703860+0000","last_clean":"2024-09-17T02:43:47.703860+0000","last_became_active":"2024-09-17T02:43:45.647466+0000","last_became_peered":"2024-09-17T02:43:45.647466+0000","last_unstale":"2024-09-17T02:43:47.703860+0000","last_undegraded":"2024-09-17T02:43:47.703860+0000","last_fullsized":"2024-09-17T02:43:47.703860+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:19.936068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649200+0000","last_change":"2024-09-17T02:43:46.674511+0000","last_active":"2024-09-17T02:43:47.649200+0000","last_peered":"2024-09-17T02:43:47.649200+0000","last_clean":"2024-09-17T02:43:47.649200+0000","last_became_active":"2024-09-17T02:43:46.673628+0000","last_became_peered":"2024-09-17T02:43:46.673628+0000","last_unstale":"2024-09-17T02:43:47.649200+0000","last_undegraded":"2024-09-17T02:43:47.649200+0000","last_fullsized":"2024-09-17T02:43:47.649200+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:44.828816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"65'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702986+0000","last_change":"2024-09-17T02:43:45.676827+0000","last_active":"2024-09-17T02:43:47.702986+0000","last_peered":"2024-09-17T02:43:47.702986+0000","last_clean":"2024-09-17T02:43:47.702986+0000","last_became_active":"2024-09-17T02:43:45.665485+0000","last_became_peered":"2024-09-17T02:43:45.665485+0000","last_unstale":"2024-09-17T02:43:47.702986+0000","last_undegraded":"2024-09-17T02:43:47.702986+0000","last_fullsized":"2024-09-17T02:43:47.702986+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:17.346724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.649260+0000","last_change":"2024-09-17T02:43:45.649260+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.649260+0000","last_undegraded":"2024-09-17T02:43:45.649260+0000","last_fullsized":"2024-09-17T02:43:45.649260+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691707+0000","last_change":"2024-09-17T02:43:45.685852+0000","last_active":"2024-09-17T02:43:45.691707+0000","last_peered":"2024-09-17T02:43:45.691707+0000","last_clean":"2024-09-17T02:43:45.691707+0000","last_became_active":"2024-09-17T02:43:45.683659+0000","last_became_peered":"2024-09-17T02:43:45.683659+0000","last_unstale":"2024-09-17T02:43:45.691707+0000","last_undegraded":"2024-09-17T02:43:45.691707+0000","last_fullsized":"2024-09-17T02:43:45.691707+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:51.724553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651328+0000","last_change":"2024-09-17T02:43:45.651328+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651328+0000","last_undegraded":"2024-09-17T02:43:45.651328+0000","last_fullsized":"2024-09-17T02:43:45.651328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707587+0000","last_change":"2024-09-17T02:43:45.675150+0000","last_active":"2024-09-17T02:43:47.707587+0000","last_peered":"2024-09-17T02:43:47.707587+0000","last_clean":"2024-09-17T02:43:47.707587+0000","last_became_active":"2024-09-17T02:43:45.671999+0000","last_became_peered":"2024-09-17T02:43:45.671999+0000","last_unstale":"2024-09-17T02:43:47.707587+0000","last_undegraded":"2024-09-17T02:43:47.707587+0000","last_fullsized":"2024-09-17T02:43:47.707587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:17.815894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700564+0000","last_change":"2024-09-17T02:43:46.698516+0000","last_active":"2024-09-17T02:43:47.700564+0000","last_peered":"2024-09-17T02:43:47.700564+0000","last_clean":"2024-09-17T02:43:47.700564+0000","last_became_active":"2024-09-17T02:43:46.696239+0000","last_became_peered":"2024-09-17T02:43:46.696239+0000","last_unstale":"2024-09-17T02:43:47.700564+0000","last_undegraded":"2024-09-17T02:43:47.700564+0000","last_fullsized":"2024-09-17T02:43:47.700564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:46.407009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.684341+0000","last_change":"2024-09-17T02:43:45.682922+0000","last_active":"2024-09-17T02:43:45.684341+0000","last_peered":"2024-09-17T02:43:45.684341+0000","last_clean":"2024-09-17T02:43:45.684341+0000","last_became_active":"2024-09-17T02:43:45.682556+0000","last_became_peered":"2024-09-17T02:43:45.682556+0000","last_unstale":"2024-09-17T02:43:45.684341+0000","last_undegraded":"2024-09-17T02:43:45.684341+0000","last_fullsized":"2024-09-17T02:43:45.684341+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:17.815894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651770+0000","last_change":"2024-09-17T02:43:45.651770+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651770+0000","last_undegraded":"2024-09-17T02:43:45.651770+0000","last_fullsized":"2024-09-17T02:43:45.651770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652374+0000","last_change":"2024-09-17T02:43:45.666290+0000","last_active":"2024-09-17T02:43:47.652374+0000","last_peered":"2024-09-17T02:43:47.652374+0000","last_clean":"2024-09-17T02:43:47.652374+0000","last_became_active":"2024-09-17T02:43:45.665410+0000","last_became_peered":"2024-09-17T02:43:45.665410+0000","last_unstale":"2024-09-17T02:43:47.652374+0000","last_undegraded":"2024-09-17T02:43:47.652374+0000","last_fullsized":"2024-09-17T02:43:47.652374+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:33.788490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.677792+0000","last_change":"2024-09-17T02:43:45.677792+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.677792+0000","last_undegraded":"2024-09-17T02:43:45.677792+0000","last_fullsized":"2024-09-17T02:43:45.677792+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707446+0000","last_change":"2024-09-17T02:43:45.672483+0000","last_active":"2024-09-17T02:43:47.707446+0000","last_peered":"2024-09-17T02:43:47.707446+0000","last_clean":"2024-09-17T02:43:47.707446+0000","last_became_active":"2024-09-17T02:43:45.669710+0000","last_became_peered":"2024-09-17T02:43:45.669710+0000","last_unstale":"2024-09-17T02:43:47.707446+0000","last_undegraded":"2024-09-17T02:43:47.707446+0000","last_fullsized":"2024-09-17T02:43:47.707446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:20.160389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647737+0000","last_change":"2024-09-17T02:43:46.697145+0000","last_active":"2024-09-17T02:43:47.647737+0000","last_peered":"2024-09-17T02:43:47.647737+0000","last_clean":"2024-09-17T02:43:47.647737+0000","last_became_active":"2024-09-17T02:43:46.696508+0000","last_became_peered":"2024-09-17T02:43:46.696508+0000","last_unstale":"2024-09-17T02:43:47.647737+0000","last_undegraded":"2024-09-17T02:43:47.647737+0000","last_fullsized":"2024-09-17T02:43:47.647737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:45.286919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.693830+0000","last_change":"2024-09-17T02:43:45.690426+0000","last_active":"2024-09-17T02:43:45.693830+0000","last_peered":"2024-09-17T02:43:45.693830+0000","last_clean":"2024-09-17T02:43:45.693830+0000","last_became_active":"2024-09-17T02:43:45.689221+0000","last_became_peered":"2024-09-17T02:43:45.689221+0000","last_unstale":"2024-09-17T02:43:45.693830+0000","last_undegraded":"2024-09-17T02:43:45.693830+0000","last_fullsized":"2024-09-17T02:43:45.693830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:16.815190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.647781+0000","last_change":"2024-09-17T02:43:45.647781+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.647781+0000","last_undegraded":"2024-09-17T02:43:45.647781+0000","last_fullsized":"2024-09-17T02:43:45.647781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691983+0000","last_change":"2024-09-17T02:43:45.688114+0000","last_active":"2024-09-17T02:43:45.691983+0000","last_peered":"2024-09-17T02:43:45.691983+0000","last_clean":"2024-09-17T02:43:45.691983+0000","last_became_active":"2024-09-17T02:43:45.687238+0000","last_became_peered":"2024-09-17T02:43:45.687238+0000","last_unstale":"2024-09-17T02:43:45.691983+0000","last_undegraded":"2024-09-17T02:43:45.691983+0000","last_fullsized":"2024-09-17T02:43:45.691983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:07.931860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":187,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.645912+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:43:45.645912+0000","last_peered":"2024-09-17T02:43:45.645912+0000","last_clean":"2024-09-17T02:43:45.645912+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:43:45.645912+0000","last_undegraded":"2024-09-17T02:43:45.645912+0000","last_fullsized":"2024-09-17T02:43:45.645912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":778240,"data_stored":349676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1335296,"data_stored":919518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":782336,"data_stored":350258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":741376,"data_stored":329150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":786432,"data_stored":349732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1335296,"data_stored":919518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31932,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339908,"statfs":{"total":96636764160,"available":96604065792,"internally_reserved":0,"allocated":753664,"data_stored":335817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":798720,"data_stored":351940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:43:51.260 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:43:51.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.435+0000 7fa629ea6640 1 -- 172.21.15.131:0/842902623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa624155670 msgr2=0x7fa62415ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.435+0000 7fa629ea6640 1 --2- 172.21.15.131:0/842902623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa624155670 0x7fa62415ecd0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa618009c80 tx=0x7fa61801cd10 comp rx=0 tx=0).stop 2024-09-17T02:43:51.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.435+0000 7fa629ea6640 1 -- 172.21.15.131:0/842902623 shutdown_connections 2024-09-17T02:43:51.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.435+0000 7fa629ea6640 1 --2- 172.21.15.131:0/842902623 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa624155670 0x7fa62415ecd0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.435+0000 7fa629ea6640 1 --2- 172.21.15.131:0/842902623 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa624070340 0x7fa624155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.435+0000 7fa629ea6640 1 -- 172.21.15.131:0/842902623 >> 172.21.15.131:0/842902623 conn(0x7fa62406ce90 msgr2=0x7fa62406d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:51.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.435+0000 7fa629ea6640 1 -- 172.21.15.131:0/842902623 shutdown_connections 2024-09-17T02:43:51.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.435+0000 7fa629ea6640 1 -- 172.21.15.131:0/842902623 wait complete. 2024-09-17T02:43:51.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.435+0000 7fa629ea6640 1 Processor -- start 2024-09-17T02:43:51.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.435+0000 7fa629ea6640 1 -- start start 2024-09-17T02:43:51.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa629ea6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa624070340 0x7fa624148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa629ea6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa624155670 0x7fa6240cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa629ea6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa62415f210 0x7fa6240cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa629ea6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6240c67f0 con 0x7fa624155670 2024-09-17T02:43:51.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa629ea6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa6240c6670 con 0x7fa62415f210 2024-09-17T02:43:51.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa629ea6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa6240c6970 con 0x7fa624070340 2024-09-17T02:43:51.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa623fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa624155670 0x7fa6240cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa623fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa624155670 0x7fa6240cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51306/0 (socket says 172.21.15.131:51306) 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa623fff640 1 -- 172.21.15.131:0/3215473854 learned_addr learned my addr 172.21.15.131:0/3215473854 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa623fff640 1 -- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa624070340 msgr2=0x7fa624148ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa628ea4640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa624070340 0x7fa624148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa6296a5640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa62415f210 0x7fa6240cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa623fff640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa624070340 0x7fa624148ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa623fff640 1 -- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa62415f210 msgr2=0x7fa6240cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa623fff640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa62415f210 0x7fa6240cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa623fff640 1 -- 172.21.15.131:0/3215473854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa624149300 con 0x7fa624155670 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa6296a5640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa62415f210 0x7fa6240cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa623fff640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa624155670 0x7fa6240cb650 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa618000f90 tx=0x7fa6180093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa6217fa640 1 -- 172.21.15.131:0/3215473854 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6180bb070 con 0x7fa624155670 2024-09-17T02:43:51.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa6217fa640 1 -- 172.21.15.131:0/3215473854 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa618004d10 con 0x7fa624155670 2024-09-17T02:43:51.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa629ea6640 1 -- 172.21.15.131:0/3215473854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa624073f50 con 0x7fa624155670 2024-09-17T02:43:51.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa6217fa640 1 -- 172.21.15.131:0/3215473854 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6180b2040 con 0x7fa624155670 2024-09-17T02:43:51.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa629ea6640 1 -- 172.21.15.131:0/3215473854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62414eab0 con 0x7fa624155670 2024-09-17T02:43:51.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa629ea6640 1 -- 172.21.15.131:0/3215473854 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6241494d0 con 0x7fa624155670 2024-09-17T02:43:51.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa6217fa640 1 -- 172.21.15.131:0/3215473854 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa6180be070 con 0x7fa624155670 2024-09-17T02:43:51.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa6217fa640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa610073200 0x7fa610075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.439+0000 7fa6217fa640 1 -- 172.21.15.131:0/3215473854 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fa61812e0f0 con 0x7fa624155670 2024-09-17T02:43:51.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.443+0000 7fa628ea4640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa610073200 0x7fa610075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:51.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.443+0000 7fa628ea4640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa610073200 0x7fa610075710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa61400a790 tx=0x7fa614008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:51.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.447+0000 7fa6217fa640 1 -- 172.21.15.131:0/3215473854 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa6241494d0 con 0x7fa624155670 2024-09-17T02:43:51.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.599+0000 7fa629ea6640 1 -- 172.21.15.131:0/3215473854 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6240cf7d0 con 0x7fa610073200 2024-09-17T02:43:51.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.611+0000 7fa6217fa640 1 -- 172.21.15.131:0/3215473854 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249217 (secure 0 0 0) 0x7fa6240cf7d0 con 0x7fa610073200 2024-09-17T02:43:51.614 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:51.617 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:43:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 -- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa610073200 msgr2=0x7fa610075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa610073200 0x7fa610075710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa61400a790 tx=0x7fa614008040 comp rx=0 tx=0).stop 2024-09-17T02:43:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 -- 172.21.15.131:0/3215473854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa624155670 msgr2=0x7fa6240cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa624155670 0x7fa6240cb650 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa618000f90 tx=0x7fa6180093f0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 -- 172.21.15.131:0/3215473854 shutdown_connections 2024-09-17T02:43:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa610073200 0x7fa610075710 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa62415f210 0x7fa6240cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa624155670 0x7fa6240cb650 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 --2- 172.21.15.131:0/3215473854 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa624070340 0x7fa624148ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 -- 172.21.15.131:0/3215473854 >> 172.21.15.131:0/3215473854 conn(0x7fa62406ce90 msgr2=0x7fa62415e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:51.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 -- 172.21.15.131:0/3215473854 shutdown_connections 2024-09-17T02:43:51.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.615+0000 7fa629ea6640 1 -- 172.21.15.131:0/3215473854 wait complete. 2024-09-17T02:43:51.638 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-09-17T02:43:51.186677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591006,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":119,"num_write_kb":1637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252524,"kb_used_data":7140,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722196,"statfs":{"total":773094113280,"available":772835528704,"internally_reserved":0,"allocated":7311360,"data_stored":3905609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002002"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652370+0000","last_change":"2024-09-17T02:43:46.680494+0000","last_active":"2024-09-17T02:43:47.652370+0000","last_peered":"2024-09-17T02:43:47.652370+0000","last_clean":"2024-09-17T02:43:47.652370+0000","last_became_active":"2024-09-17T02:43:46.679436+0000","last_became_peered":"2024-09-17T02:43:46.679436+0000","last_unstale":"2024-09-17T02:43:47.652370+0000","last_undegraded":"2024-09-17T02:43:47.652370+0000","last_fullsized":"2024-09-17T02:43:47.652370+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:18.520380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.670539+0000","last_change":"2024-09-17T02:43:45.670539+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.670539+0000","last_undegraded":"2024-09-17T02:43:45.670539+0000","last_fullsized":"2024-09-17T02:43:45.670539+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652810+0000","last_change":"2024-09-17T02:43:45.652810+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652810+0000","last_undegraded":"2024-09-17T02:43:45.652810+0000","last_fullsized":"2024-09-17T02:43:45.652810+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701851+0000","last_change":"2024-09-17T02:43:46.708732+0000","last_active":"2024-09-17T02:43:47.701851+0000","last_peered":"2024-09-17T02:43:47.701851+0000","last_clean":"2024-09-17T02:43:47.701851+0000","last_became_active":"2024-09-17T02:43:46.706193+0000","last_became_peered":"2024-09-17T02:43:46.706193+0000","last_unstale":"2024-09-17T02:43:47.701851+0000","last_undegraded":"2024-09-17T02:43:47.701851+0000","last_fullsized":"2024-09-17T02:43:47.701851+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:27.991604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653624+0000","last_change":"2024-09-17T02:43:45.653624+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653624+0000","last_undegraded":"2024-09-17T02:43:45.653624+0000","last_fullsized":"2024-09-17T02:43:45.653624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.679025+0000","last_change":"2024-09-17T02:43:45.679025+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.679025+0000","last_undegraded":"2024-09-17T02:43:45.679025+0000","last_fullsized":"2024-09-17T02:43:45.679025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649842+0000","last_change":"2024-09-17T02:43:46.678701+0000","last_active":"2024-09-17T02:43:47.649842+0000","last_peered":"2024-09-17T02:43:47.649842+0000","last_clean":"2024-09-17T02:43:47.649842+0000","last_became_active":"2024-09-17T02:43:46.674482+0000","last_became_peered":"2024-09-17T02:43:46.674482+0000","last_unstale":"2024-09-17T02:43:47.649842+0000","last_undegraded":"2024-09-17T02:43:47.649842+0000","last_fullsized":"2024-09-17T02:43:47.649842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:30.882039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651787+0000","last_change":"2024-09-17T02:43:45.651787+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651787+0000","last_undegraded":"2024-09-17T02:43:45.651787+0000","last_fullsized":"2024-09-17T02:43:45.651787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702086+0000","last_change":"2024-09-17T02:43:46.695684+0000","last_active":"2024-09-17T02:43:47.702086+0000","last_peered":"2024-09-17T02:43:47.702086+0000","last_clean":"2024-09-17T02:43:47.702086+0000","last_became_active":"2024-09-17T02:43:46.694860+0000","last_became_peered":"2024-09-17T02:43:46.694860+0000","last_unstale":"2024-09-17T02:43:47.702086+0000","last_undegraded":"2024-09-17T02:43:47.702086+0000","last_fullsized":"2024-09-17T02:43:47.702086+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:53.983748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646728+0000","last_change":"2024-09-17T02:43:45.646728+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646728+0000","last_undegraded":"2024-09-17T02:43:45.646728+0000","last_fullsized":"2024-09-17T02:43:45.646728+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702375+0000","last_change":"2024-09-17T02:43:46.701807+0000","last_active":"2024-09-17T02:43:47.702375+0000","last_peered":"2024-09-17T02:43:47.702375+0000","last_clean":"2024-09-17T02:43:47.702375+0000","last_became_active":"2024-09-17T02:43:46.696272+0000","last_became_peered":"2024-09-17T02:43:46.696272+0000","last_unstale":"2024-09-17T02:43:47.702375+0000","last_undegraded":"2024-09-17T02:43:47.702375+0000","last_fullsized":"2024-09-17T02:43:47.702375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:47.850100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702109+0000","last_change":"2024-09-17T02:43:46.707994+0000","last_active":"2024-09-17T02:43:47.702109+0000","last_peered":"2024-09-17T02:43:47.702109+0000","last_clean":"2024-09-17T02:43:47.702109+0000","last_became_active":"2024-09-17T02:43:46.701425+0000","last_became_peered":"2024-09-17T02:43:46.701425+0000","last_unstale":"2024-09-17T02:43:47.702109+0000","last_undegraded":"2024-09-17T02:43:47.702109+0000","last_fullsized":"2024-09-17T02:43:47.702109+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:42.080955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.655366+0000","last_change":"2024-09-17T02:43:45.655366+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.655366+0000","last_undegraded":"2024-09-17T02:43:45.655366+0000","last_fullsized":"2024-09-17T02:43:45.655366+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654842+0000","last_change":"2024-09-17T02:43:45.654842+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654842+0000","last_undegraded":"2024-09-17T02:43:45.654842+0000","last_fullsized":"2024-09-17T02:43:45.654842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.648693+0000","last_change":"2024-09-17T02:43:45.648693+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.648693+0000","last_undegraded":"2024-09-17T02:43:45.648693+0000","last_fullsized":"2024-09-17T02:43:45.648693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707657+0000","last_change":"2024-09-17T02:43:46.708131+0000","last_active":"2024-09-17T02:43:47.707657+0000","last_peered":"2024-09-17T02:43:47.707657+0000","last_clean":"2024-09-17T02:43:47.707657+0000","last_became_active":"2024-09-17T02:43:46.707665+0000","last_became_peered":"2024-09-17T02:43:46.707665+0000","last_unstale":"2024-09-17T02:43:47.707657+0000","last_undegraded":"2024-09-17T02:43:47.707657+0000","last_fullsized":"2024-09-17T02:43:47.707657+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:48:16.207446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.650043+0000","last_change":"2024-09-17T02:43:46.702269+0000","last_active":"2024-09-17T02:43:47.650043+0000","last_peered":"2024-09-17T02:43:47.650043+0000","last_clean":"2024-09-17T02:43:47.650043+0000","last_became_active":"2024-09-17T02:43:46.701883+0000","last_became_peered":"2024-09-17T02:43:46.701883+0000","last_unstale":"2024-09-17T02:43:47.650043+0000","last_undegraded":"2024-09-17T02:43:47.650043+0000","last_fullsized":"2024-09-17T02:43:47.650043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:22.975866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702426+0000","last_change":"2024-09-17T02:43:46.693189+0000","last_active":"2024-09-17T02:43:47.702426+0000","last_peered":"2024-09-17T02:43:47.702426+0000","last_clean":"2024-09-17T02:43:47.702426+0000","last_became_active":"2024-09-17T02:43:46.691164+0000","last_became_peered":"2024-09-17T02:43:46.691164+0000","last_unstale":"2024-09-17T02:43:47.702426+0000","last_undegraded":"2024-09-17T02:43:47.702426+0000","last_fullsized":"2024-09-17T02:43:47.702426+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:52.980125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702146+0000","last_change":"2024-09-17T02:43:46.692645+0000","last_active":"2024-09-17T02:43:47.702146+0000","last_peered":"2024-09-17T02:43:47.702146+0000","last_clean":"2024-09-17T02:43:47.702146+0000","last_became_active":"2024-09-17T02:43:46.691861+0000","last_became_peered":"2024-09-17T02:43:46.691861+0000","last_unstale":"2024-09-17T02:43:47.702146+0000","last_undegraded":"2024-09-17T02:43:47.702146+0000","last_fullsized":"2024-09-17T02:43:47.702146+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:03.405001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647781+0000","last_change":"2024-09-17T02:43:46.684848+0000","last_active":"2024-09-17T02:43:47.647781+0000","last_peered":"2024-09-17T02:43:47.647781+0000","last_clean":"2024-09-17T02:43:47.647781+0000","last_became_active":"2024-09-17T02:43:46.683919+0000","last_became_peered":"2024-09-17T02:43:46.683919+0000","last_unstale":"2024-09-17T02:43:47.647781+0000","last_undegraded":"2024-09-17T02:43:47.647781+0000","last_fullsized":"2024-09-17T02:43:47.647781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:58.516266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654859+0000","last_change":"2024-09-17T02:43:45.654859+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654859+0000","last_undegraded":"2024-09-17T02:43:45.654859+0000","last_fullsized":"2024-09-17T02:43:45.654859+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.669183+0000","last_change":"2024-09-17T02:43:45.669183+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.669183+0000","last_undegraded":"2024-09-17T02:43:45.669183+0000","last_fullsized":"2024-09-17T02:43:45.669183+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707376+0000","last_change":"2024-09-17T02:43:46.708531+0000","last_active":"2024-09-17T02:43:47.707376+0000","last_peered":"2024-09-17T02:43:47.707376+0000","last_clean":"2024-09-17T02:43:47.707376+0000","last_became_active":"2024-09-17T02:43:46.708224+0000","last_became_peered":"2024-09-17T02:43:46.708224+0000","last_unstale":"2024-09-17T02:43:47.707376+0000","last_undegraded":"2024-09-17T02:43:47.707376+0000","last_fullsized":"2024-09-17T02:43:47.707376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:37.303173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651233+0000","last_change":"2024-09-17T02:43:45.651233+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651233+0000","last_undegraded":"2024-09-17T02:43:45.651233+0000","last_fullsized":"2024-09-17T02:43:45.651233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654320+0000","last_change":"2024-09-17T02:43:45.654320+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654320+0000","last_undegraded":"2024-09-17T02:43:45.654320+0000","last_fullsized":"2024-09-17T02:43:45.654320+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652203+0000","last_change":"2024-09-17T02:43:45.652203+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652203+0000","last_undegraded":"2024-09-17T02:43:45.652203+0000","last_fullsized":"2024-09-17T02:43:45.652203+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654755+0000","last_change":"2024-09-17T02:43:45.654755+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654755+0000","last_undegraded":"2024-09-17T02:43:45.654755+0000","last_fullsized":"2024-09-17T02:43:45.654755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652393+0000","last_change":"2024-09-17T02:43:46.713535+0000","last_active":"2024-09-17T02:43:47.652393+0000","last_peered":"2024-09-17T02:43:47.652393+0000","last_clean":"2024-09-17T02:43:47.652393+0000","last_became_active":"2024-09-17T02:43:46.712681+0000","last_became_peered":"2024-09-17T02:43:46.712681+0000","last_unstale":"2024-09-17T02:43:47.652393+0000","last_undegraded":"2024-09-17T02:43:47.652393+0000","last_fullsized":"2024-09-17T02:43:47.652393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:21.464657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701453+0000","last_change":"2024-09-17T02:43:46.706426+0000","last_active":"2024-09-17T02:43:47.701453+0000","last_peered":"2024-09-17T02:43:47.701453+0000","last_clean":"2024-09-17T02:43:47.701453+0000","last_became_active":"2024-09-17T02:43:46.706046+0000","last_became_peered":"2024-09-17T02:43:46.706046+0000","last_unstale":"2024-09-17T02:43:47.701453+0000","last_undegraded":"2024-09-17T02:43:47.701453+0000","last_fullsized":"2024-09-17T02:43:47.701453+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:58.594878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700841+0000","last_change":"2024-09-17T02:43:46.702605+0000","last_active":"2024-09-17T02:43:47.700841+0000","last_peered":"2024-09-17T02:43:47.700841+0000","last_clean":"2024-09-17T02:43:47.700841+0000","last_became_active":"2024-09-17T02:43:46.702182+0000","last_became_peered":"2024-09-17T02:43:46.702182+0000","last_unstale":"2024-09-17T02:43:47.700841+0000","last_undegraded":"2024-09-17T02:43:47.700841+0000","last_fullsized":"2024-09-17T02:43:47.700841+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:17.795947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.650778+0000","last_change":"2024-09-17T02:43:45.650778+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.650778+0000","last_undegraded":"2024-09-17T02:43:45.650778+0000","last_fullsized":"2024-09-17T02:43:45.650778+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649781+0000","last_change":"2024-09-17T02:43:46.673631+0000","last_active":"2024-09-17T02:43:47.649781+0000","last_peered":"2024-09-17T02:43:47.649781+0000","last_clean":"2024-09-17T02:43:47.649781+0000","last_became_active":"2024-09-17T02:43:46.672779+0000","last_became_peered":"2024-09-17T02:43:46.672779+0000","last_unstale":"2024-09-17T02:43:47.649781+0000","last_undegraded":"2024-09-17T02:43:47.649781+0000","last_fullsized":"2024-09-17T02:43:47.649781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:04.209405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702521+0000","last_change":"2024-09-17T02:43:46.710347+0000","last_active":"2024-09-17T02:43:47.702521+0000","last_peered":"2024-09-17T02:43:47.702521+0000","last_clean":"2024-09-17T02:43:47.702521+0000","last_became_active":"2024-09-17T02:43:46.708415+0000","last_became_peered":"2024-09-17T02:43:46.708415+0000","last_unstale":"2024-09-17T02:43:47.702521+0000","last_undegraded":"2024-09-17T02:43:47.702521+0000","last_fullsized":"2024-09-17T02:43:47.702521+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:23.928100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":19,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652160+0000","last_change":"2024-09-17T02:43:45.678361+0000","last_active":"2024-09-17T02:43:47.652160+0000","last_peered":"2024-09-17T02:43:47.652160+0000","last_clean":"2024-09-17T02:43:47.652160+0000","last_became_active":"2024-09-17T02:43:45.677598+0000","last_became_peered":"2024-09-17T02:43:45.677598+0000","last_unstale":"2024-09-17T02:43:47.652160+0000","last_undegraded":"2024-09-17T02:43:47.652160+0000","last_fullsized":"2024-09-17T02:43:47.652160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:08.498525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651601+0000","last_change":"2024-09-17T02:43:45.651601+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651601+0000","last_undegraded":"2024-09-17T02:43:45.651601+0000","last_fullsized":"2024-09-17T02:43:45.651601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651944+0000","last_change":"2024-09-17T02:43:45.660427+0000","last_active":"2024-09-17T02:43:47.651944+0000","last_peered":"2024-09-17T02:43:47.651944+0000","last_clean":"2024-09-17T02:43:47.651944+0000","last_became_active":"2024-09-17T02:43:45.659575+0000","last_became_peered":"2024-09-17T02:43:45.659575+0000","last_unstale":"2024-09-17T02:43:47.651944+0000","last_undegraded":"2024-09-17T02:43:47.651944+0000","last_fullsized":"2024-09-17T02:43:47.651944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:35.225547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.669291+0000","last_change":"2024-09-17T02:43:45.669291+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.669291+0000","last_undegraded":"2024-09-17T02:43:45.669291+0000","last_fullsized":"2024-09-17T02:43:45.669291+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652828+0000","last_change":"2024-09-17T02:43:45.665351+0000","last_active":"2024-09-17T02:43:47.652828+0000","last_peered":"2024-09-17T02:43:47.652828+0000","last_clean":"2024-09-17T02:43:47.652828+0000","last_became_active":"2024-09-17T02:43:45.664498+0000","last_became_peered":"2024-09-17T02:43:45.664498+0000","last_unstale":"2024-09-17T02:43:47.652828+0000","last_undegraded":"2024-09-17T02:43:47.652828+0000","last_fullsized":"2024-09-17T02:43:47.652828+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:35.011302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703229+0000","last_change":"2024-09-17T02:43:46.694855+0000","last_active":"2024-09-17T02:43:47.703229+0000","last_peered":"2024-09-17T02:43:47.703229+0000","last_clean":"2024-09-17T02:43:47.703229+0000","last_became_active":"2024-09-17T02:43:46.692123+0000","last_became_peered":"2024-09-17T02:43:46.692123+0000","last_unstale":"2024-09-17T02:43:47.703229+0000","last_undegraded":"2024-09-17T02:43:47.703229+0000","last_fullsized":"2024-09-17T02:43:47.703229+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:38.594076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.693843+0000","last_change":"2024-09-17T02:43:45.690176+0000","last_active":"2024-09-17T02:43:45.693843+0000","last_peered":"2024-09-17T02:43:45.693843+0000","last_clean":"2024-09-17T02:43:45.693843+0000","last_became_active":"2024-09-17T02:43:45.689355+0000","last_became_peered":"2024-09-17T02:43:45.689355+0000","last_unstale":"2024-09-17T02:43:45.693843+0000","last_undegraded":"2024-09-17T02:43:45.693843+0000","last_fullsized":"2024-09-17T02:43:45.693843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:40.666325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.646844+0000","last_change":"2024-09-17T02:43:46.678861+0000","last_active":"2024-09-17T02:43:47.646844+0000","last_peered":"2024-09-17T02:43:47.646844+0000","last_clean":"2024-09-17T02:43:47.646844+0000","last_became_active":"2024-09-17T02:43:46.677801+0000","last_became_peered":"2024-09-17T02:43:46.677801+0000","last_unstale":"2024-09-17T02:43:47.646844+0000","last_undegraded":"2024-09-17T02:43:47.646844+0000","last_fullsized":"2024-09-17T02:43:47.646844+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:31.162139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647995+0000","last_change":"2024-09-17T02:43:45.666157+0000","last_active":"2024-09-17T02:43:47.647995+0000","last_peered":"2024-09-17T02:43:47.647995+0000","last_clean":"2024-09-17T02:43:47.647995+0000","last_became_active":"2024-09-17T02:43:45.665300+0000","last_became_peered":"2024-09-17T02:43:45.665300+0000","last_unstale":"2024-09-17T02:43:47.647995+0000","last_undegraded":"2024-09-17T02:43:47.647995+0000","last_fullsized":"2024-09-17T02:43:47.647995+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:41.076834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651046+0000","last_change":"2024-09-17T02:43:46.697615+0000","last_active":"2024-09-17T02:43:47.651046+0000","last_peered":"2024-09-17T02:43:47.651046+0000","last_clean":"2024-09-17T02:43:47.651046+0000","last_became_active":"2024-09-17T02:43:46.697182+0000","last_became_peered":"2024-09-17T02:43:46.697182+0000","last_unstale":"2024-09-17T02:43:47.651046+0000","last_undegraded":"2024-09-17T02:43:47.651046+0000","last_fullsized":"2024-09-17T02:43:47.651046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:48:05.414527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691924+0000","last_change":"2024-09-17T02:43:45.687236+0000","last_active":"2024-09-17T02:43:45.691924+0000","last_peered":"2024-09-17T02:43:45.691924+0000","last_clean":"2024-09-17T02:43:45.691924+0000","last_became_active":"2024-09-17T02:43:45.685471+0000","last_became_peered":"2024-09-17T02:43:45.685471+0000","last_unstale":"2024-09-17T02:43:45.691924+0000","last_undegraded":"2024-09-17T02:43:45.691924+0000","last_fullsized":"2024-09-17T02:43:45.691924+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:39.927955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646618+0000","last_change":"2024-09-17T02:43:45.646618+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646618+0000","last_undegraded":"2024-09-17T02:43:45.646618+0000","last_fullsized":"2024-09-17T02:43:45.646618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651487+0000","last_change":"2024-09-17T02:43:45.661349+0000","last_active":"2024-09-17T02:43:47.651487+0000","last_peered":"2024-09-17T02:43:47.651487+0000","last_clean":"2024-09-17T02:43:47.651487+0000","last_became_active":"2024-09-17T02:43:45.660480+0000","last_became_peered":"2024-09-17T02:43:45.660480+0000","last_unstale":"2024-09-17T02:43:47.651487+0000","last_undegraded":"2024-09-17T02:43:47.651487+0000","last_fullsized":"2024-09-17T02:43:47.651487+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:22.763370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647445+0000","last_change":"2024-09-17T02:43:46.694871+0000","last_active":"2024-09-17T02:43:47.647445+0000","last_peered":"2024-09-17T02:43:47.647445+0000","last_clean":"2024-09-17T02:43:47.647445+0000","last_became_active":"2024-09-17T02:43:46.694495+0000","last_became_peered":"2024-09-17T02:43:46.694495+0000","last_unstale":"2024-09-17T02:43:47.647445+0000","last_undegraded":"2024-09-17T02:43:47.647445+0000","last_fullsized":"2024-09-17T02:43:47.647445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:01.223420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700581+0000","last_change":"2024-09-17T02:43:45.655934+0000","last_active":"2024-09-17T02:43:47.700581+0000","last_peered":"2024-09-17T02:43:47.700581+0000","last_clean":"2024-09-17T02:43:47.700581+0000","last_became_active":"2024-09-17T02:43:45.649802+0000","last_became_peered":"2024-09-17T02:43:45.649802+0000","last_unstale":"2024-09-17T02:43:47.700581+0000","last_undegraded":"2024-09-17T02:43:47.700581+0000","last_fullsized":"2024-09-17T02:43:47.700581+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:24.816574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652445+0000","last_change":"2024-09-17T02:43:46.677885+0000","last_active":"2024-09-17T02:43:47.652445+0000","last_peered":"2024-09-17T02:43:47.652445+0000","last_clean":"2024-09-17T02:43:47.652445+0000","last_became_active":"2024-09-17T02:43:46.677091+0000","last_became_peered":"2024-09-17T02:43:46.677091+0000","last_unstale":"2024-09-17T02:43:47.652445+0000","last_undegraded":"2024-09-17T02:43:47.652445+0000","last_fullsized":"2024-09-17T02:43:47.652445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:59.608768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703726+0000","last_change":"2024-09-17T02:43:45.684007+0000","last_active":"2024-09-17T02:43:47.703726+0000","last_peered":"2024-09-17T02:43:47.703726+0000","last_clean":"2024-09-17T02:43:47.703726+0000","last_became_active":"2024-09-17T02:43:45.683525+0000","last_became_peered":"2024-09-17T02:43:45.683525+0000","last_unstale":"2024-09-17T02:43:47.703726+0000","last_undegraded":"2024-09-17T02:43:47.703726+0000","last_fullsized":"2024-09-17T02:43:47.703726+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:41.128475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647829+0000","last_change":"2024-09-17T02:43:46.668355+0000","last_active":"2024-09-17T02:43:47.647829+0000","last_peered":"2024-09-17T02:43:47.647829+0000","last_clean":"2024-09-17T02:43:47.647829+0000","last_became_active":"2024-09-17T02:43:46.667493+0000","last_became_peered":"2024-09-17T02:43:46.667493+0000","last_unstale":"2024-09-17T02:43:47.647829+0000","last_undegraded":"2024-09-17T02:43:47.647829+0000","last_fullsized":"2024-09-17T02:43:47.647829+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:11.487744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.667876+0000","last_change":"2024-09-17T02:43:45.667054+0000","last_active":"2024-09-17T02:43:45.667876+0000","last_peered":"2024-09-17T02:43:45.667876+0000","last_clean":"2024-09-17T02:43:45.667876+0000","last_became_active":"2024-09-17T02:43:45.666184+0000","last_became_peered":"2024-09-17T02:43:45.666184+0000","last_unstale":"2024-09-17T02:43:45.667876+0000","last_undegraded":"2024-09-17T02:43:45.667876+0000","last_fullsized":"2024-09-17T02:43:45.667876+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:33.813934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654576+0000","last_change":"2024-09-17T02:43:45.654576+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654576+0000","last_undegraded":"2024-09-17T02:43:45.654576+0000","last_fullsized":"2024-09-17T02:43:45.654576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.687040+0000","last_change":"2024-09-17T02:43:45.679687+0000","last_active":"2024-09-17T02:43:45.687040+0000","last_peered":"2024-09-17T02:43:45.687040+0000","last_clean":"2024-09-17T02:43:45.687040+0000","last_became_active":"2024-09-17T02:43:45.679104+0000","last_became_peered":"2024-09-17T02:43:45.679104+0000","last_unstale":"2024-09-17T02:43:45.687040+0000","last_undegraded":"2024-09-17T02:43:45.687040+0000","last_fullsized":"2024-09-17T02:43:45.687040+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:30.134634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653809+0000","last_change":"2024-09-17T02:43:45.653809+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653809+0000","last_undegraded":"2024-09-17T02:43:45.653809+0000","last_fullsized":"2024-09-17T02:43:45.653809+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.694452+0000","last_change":"2024-09-17T02:43:45.692080+0000","last_active":"2024-09-17T02:43:45.694452+0000","last_peered":"2024-09-17T02:43:45.694452+0000","last_clean":"2024-09-17T02:43:45.694452+0000","last_became_active":"2024-09-17T02:43:45.690832+0000","last_became_peered":"2024-09-17T02:43:45.690832+0000","last_unstale":"2024-09-17T02:43:45.694452+0000","last_undegraded":"2024-09-17T02:43:45.694452+0000","last_fullsized":"2024-09-17T02:43:45.694452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:00.297892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646761+0000","last_change":"2024-09-17T02:43:45.646761+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646761+0000","last_undegraded":"2024-09-17T02:43:45.646761+0000","last_fullsized":"2024-09-17T02:43:45.646761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701443+0000","last_change":"2024-09-17T02:43:45.651787+0000","last_active":"2024-09-17T02:43:47.701443+0000","last_peered":"2024-09-17T02:43:47.701443+0000","last_clean":"2024-09-17T02:43:47.701443+0000","last_became_active":"2024-09-17T02:43:45.649506+0000","last_became_peered":"2024-09-17T02:43:45.649506+0000","last_unstale":"2024-09-17T02:43:47.701443+0000","last_undegraded":"2024-09-17T02:43:47.701443+0000","last_fullsized":"2024-09-17T02:43:47.701443+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:39.203265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.648211+0000","last_change":"2024-09-17T02:43:45.648211+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.648211+0000","last_undegraded":"2024-09-17T02:43:45.648211+0000","last_fullsized":"2024-09-17T02:43:45.648211+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.664021+0000","last_change":"2024-09-17T02:43:45.661781+0000","last_active":"2024-09-17T02:43:45.664021+0000","last_peered":"2024-09-17T02:43:45.664021+0000","last_clean":"2024-09-17T02:43:45.664021+0000","last_became_active":"2024-09-17T02:43:45.661416+0000","last_became_peered":"2024-09-17T02:43:45.661416+0000","last_unstale":"2024-09-17T02:43:45.664021+0000","last_undegraded":"2024-09-17T02:43:45.664021+0000","last_fullsized":"2024-09-17T02:43:45.664021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:57.140547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.645997+0000","last_change":"2024-09-17T02:43:45.645997+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.645997+0000","last_undegraded":"2024-09-17T02:43:45.645997+0000","last_fullsized":"2024-09-17T02:43:45.645997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.670177+0000","last_change":"2024-09-17T02:43:45.662327+0000","last_active":"2024-09-17T02:43:45.670177+0000","last_peered":"2024-09-17T02:43:45.670177+0000","last_clean":"2024-09-17T02:43:45.670177+0000","last_became_active":"2024-09-17T02:43:45.661481+0000","last_became_peered":"2024-09-17T02:43:45.661481+0000","last_unstale":"2024-09-17T02:43:45.670177+0000","last_undegraded":"2024-09-17T02:43:45.670177+0000","last_fullsized":"2024-09-17T02:43:45.670177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:59.435650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651896+0000","last_change":"2024-09-17T02:43:46.676027+0000","last_active":"2024-09-17T02:43:47.651896+0000","last_peered":"2024-09-17T02:43:47.651896+0000","last_clean":"2024-09-17T02:43:47.651896+0000","last_became_active":"2024-09-17T02:43:46.675175+0000","last_became_peered":"2024-09-17T02:43:46.675175+0000","last_unstale":"2024-09-17T02:43:47.651896+0000","last_undegraded":"2024-09-17T02:43:47.651896+0000","last_fullsized":"2024-09-17T02:43:47.651896+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:54.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.657384+0000","last_change":"2024-09-17T02:43:45.650314+0000","last_active":"2024-09-17T02:43:45.657384+0000","last_peered":"2024-09-17T02:43:45.657384+0000","last_clean":"2024-09-17T02:43:45.657384+0000","last_became_active":"2024-09-17T02:43:45.649675+0000","last_became_peered":"2024-09-17T02:43:45.649675+0000","last_unstale":"2024-09-17T02:43:45.657384+0000","last_undegraded":"2024-09-17T02:43:45.657384+0000","last_fullsized":"2024-09-17T02:43:45.657384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:53.908737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702763+0000","last_change":"2024-09-17T02:43:46.707370+0000","last_active":"2024-09-17T02:43:47.702763+0000","last_peered":"2024-09-17T02:43:47.702763+0000","last_clean":"2024-09-17T02:43:47.702763+0000","last_became_active":"2024-09-17T02:43:46.697031+0000","last_became_peered":"2024-09-17T02:43:46.697031+0000","last_unstale":"2024-09-17T02:43:47.702763+0000","last_undegraded":"2024-09-17T02:43:47.702763+0000","last_fullsized":"2024-09-17T02:43:47.702763+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:53.627837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703754+0000","last_change":"2024-09-17T02:43:45.683317+0000","last_active":"2024-09-17T02:43:47.703754+0000","last_peered":"2024-09-17T02:43:47.703754+0000","last_clean":"2024-09-17T02:43:47.703754+0000","last_became_active":"2024-09-17T02:43:45.682686+0000","last_became_peered":"2024-09-17T02:43:45.682686+0000","last_unstale":"2024-09-17T02:43:47.703754+0000","last_undegraded":"2024-09-17T02:43:47.703754+0000","last_fullsized":"2024-09-17T02:43:47.703754+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:45.603856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652497+0000","last_change":"2024-09-17T02:43:45.652497+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652497+0000","last_undegraded":"2024-09-17T02:43:45.652497+0000","last_fullsized":"2024-09-17T02:43:45.652497+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702089+0000","last_change":"2024-09-17T02:43:45.675971+0000","last_active":"2024-09-17T02:43:47.702089+0000","last_peered":"2024-09-17T02:43:47.702089+0000","last_clean":"2024-09-17T02:43:47.702089+0000","last_became_active":"2024-09-17T02:43:45.665279+0000","last_became_peered":"2024-09-17T02:43:45.665279+0000","last_unstale":"2024-09-17T02:43:47.702089+0000","last_undegraded":"2024-09-17T02:43:47.702089+0000","last_fullsized":"2024-09-17T02:43:47.702089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:02.887437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653757+0000","last_change":"2024-09-17T02:43:45.653757+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653757+0000","last_undegraded":"2024-09-17T02:43:45.653757+0000","last_fullsized":"2024-09-17T02:43:45.653757+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.673123+0000","last_change":"2024-09-17T02:43:45.669829+0000","last_active":"2024-09-17T02:43:45.673123+0000","last_peered":"2024-09-17T02:43:45.673123+0000","last_clean":"2024-09-17T02:43:45.673123+0000","last_became_active":"2024-09-17T02:43:45.669491+0000","last_became_peered":"2024-09-17T02:43:45.669491+0000","last_unstale":"2024-09-17T02:43:45.673123+0000","last_undegraded":"2024-09-17T02:43:45.673123+0000","last_fullsized":"2024-09-17T02:43:45.673123+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:33.777991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652610+0000","last_change":"2024-09-17T02:43:45.652610+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652610+0000","last_undegraded":"2024-09-17T02:43:45.652610+0000","last_fullsized":"2024-09-17T02:43:45.652610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707366+0000","last_change":"2024-09-17T02:43:45.674406+0000","last_active":"2024-09-17T02:43:47.707366+0000","last_peered":"2024-09-17T02:43:47.707366+0000","last_clean":"2024-09-17T02:43:47.707366+0000","last_became_active":"2024-09-17T02:43:45.671107+0000","last_became_peered":"2024-09-17T02:43:45.671107+0000","last_unstale":"2024-09-17T02:43:47.707366+0000","last_undegraded":"2024-09-17T02:43:47.707366+0000","last_fullsized":"2024-09-17T02:43:47.707366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:13.407925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.675296+0000","last_change":"2024-09-17T02:43:45.675296+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.675296+0000","last_undegraded":"2024-09-17T02:43:45.675296+0000","last_fullsized":"2024-09-17T02:43:45.675296+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.683619+0000","last_change":"2024-09-17T02:43:45.680606+0000","last_active":"2024-09-17T02:43:45.683619+0000","last_peered":"2024-09-17T02:43:45.683619+0000","last_clean":"2024-09-17T02:43:45.683619+0000","last_became_active":"2024-09-17T02:43:45.679640+0000","last_became_peered":"2024-09-17T02:43:45.679640+0000","last_unstale":"2024-09-17T02:43:45.683619+0000","last_undegraded":"2024-09-17T02:43:45.683619+0000","last_fullsized":"2024-09-17T02:43:45.683619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:24.412203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652569+0000","last_change":"2024-09-17T02:43:45.652569+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652569+0000","last_undegraded":"2024-09-17T02:43:45.652569+0000","last_fullsized":"2024-09-17T02:43:45.652569+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.667071+0000","last_change":"2024-09-17T02:43:45.664123+0000","last_active":"2024-09-17T02:43:45.667071+0000","last_peered":"2024-09-17T02:43:45.667071+0000","last_clean":"2024-09-17T02:43:45.667071+0000","last_became_active":"2024-09-17T02:43:45.663776+0000","last_became_peered":"2024-09-17T02:43:45.663776+0000","last_unstale":"2024-09-17T02:43:45.667071+0000","last_undegraded":"2024-09-17T02:43:45.667071+0000","last_fullsized":"2024-09-17T02:43:45.667071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:21.659777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700904+0000","last_change":"2024-09-17T02:43:46.693857+0000","last_active":"2024-09-17T02:43:47.700904+0000","last_peered":"2024-09-17T02:43:47.700904+0000","last_clean":"2024-09-17T02:43:47.700904+0000","last_became_active":"2024-09-17T02:43:46.692225+0000","last_became_peered":"2024-09-17T02:43:46.692225+0000","last_unstale":"2024-09-17T02:43:47.700904+0000","last_undegraded":"2024-09-17T02:43:47.700904+0000","last_fullsized":"2024-09-17T02:43:47.700904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:58:56.128166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691634+0000","last_change":"2024-09-17T02:43:45.684484+0000","last_active":"2024-09-17T02:43:45.691634+0000","last_peered":"2024-09-17T02:43:45.691634+0000","last_clean":"2024-09-17T02:43:45.691634+0000","last_became_active":"2024-09-17T02:43:45.683639+0000","last_became_peered":"2024-09-17T02:43:45.683639+0000","last_unstale":"2024-09-17T02:43:45.691634+0000","last_undegraded":"2024-09-17T02:43:45.691634+0000","last_fullsized":"2024-09-17T02:43:45.691634+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:34.106052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.648456+0000","last_change":"2024-09-17T02:43:46.674643+0000","last_active":"2024-09-17T02:43:47.648456+0000","last_peered":"2024-09-17T02:43:47.648456+0000","last_clean":"2024-09-17T02:43:47.648456+0000","last_became_active":"2024-09-17T02:43:46.673681+0000","last_became_peered":"2024-09-17T02:43:46.673681+0000","last_unstale":"2024-09-17T02:43:47.648456+0000","last_undegraded":"2024-09-17T02:43:47.648456+0000","last_fullsized":"2024-09-17T02:43:47.648456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:09.748085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703860+0000","last_change":"2024-09-17T02:43:45.647880+0000","last_active":"2024-09-17T02:43:47.703860+0000","last_peered":"2024-09-17T02:43:47.703860+0000","last_clean":"2024-09-17T02:43:47.703860+0000","last_became_active":"2024-09-17T02:43:45.647466+0000","last_became_peered":"2024-09-17T02:43:45.647466+0000","last_unstale":"2024-09-17T02:43:47.703860+0000","last_undegraded":"2024-09-17T02:43:47.703860+0000","last_fullsized":"2024-09-17T02:43:47.703860+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:19.936068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649200+0000","last_change":"2024-09-17T02:43:46.674511+0000","last_active":"2024-09-17T02:43:47.649200+0000","last_peered":"2024-09-17T02:43:47.649200+0000","last_clean":"2024-09-17T02:43:47.649200+0000","last_became_active":"2024-09-17T02:43:46.673628+0000","last_became_peered":"2024-09-17T02:43:46.673628+0000","last_unstale":"2024-09-17T02:43:47.649200+0000","last_undegraded":"2024-09-17T02:43:47.649200+0000","last_fullsized":"2024-09-17T02:43:47.649200+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:44.828816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"65'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702986+0000","last_change":"2024-09-17T02:43:45.676827+0000","last_active":"2024-09-17T02:43:47.702986+0000","last_peered":"2024-09-17T02:43:47.702986+0000","last_clean":"2024-09-17T02:43:47.702986+0000","last_became_active":"2024-09-17T02:43:45.665485+0000","last_became_peered":"2024-09-17T02:43:45.665485+0000","last_unstale":"2024-09-17T02:43:47.702986+0000","last_undegraded":"2024-09-17T02:43:47.702986+0000","last_fullsized":"2024-09-17T02:43:47.702986+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:17.346724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.649260+0000","last_change":"2024-09-17T02:43:45.649260+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.649260+0000","last_undegraded":"2024-09-17T02:43:45.649260+0000","last_fullsized":"2024-09-17T02:43:45.649260+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691707+0000","last_change":"2024-09-17T02:43:45.685852+0000","last_active":"2024-09-17T02:43:45.691707+0000","last_peered":"2024-09-17T02:43:45.691707+0000","last_clean":"2024-09-17T02:43:45.691707+0000","last_became_active":"2024-09-17T02:43:45.683659+0000","last_became_peered":"2024-09-17T02:43:45.683659+0000","last_unstale":"2024-09-17T02:43:45.691707+0000","last_undegraded":"2024-09-17T02:43:45.691707+0000","last_fullsized":"2024-09-17T02:43:45.691707+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:51.724553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651328+0000","last_change":"2024-09-17T02:43:45.651328+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651328+0000","last_undegraded":"2024-09-17T02:43:45.651328+0000","last_fullsized":"2024-09-17T02:43:45.651328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707587+0000","last_change":"2024-09-17T02:43:45.675150+0000","last_active":"2024-09-17T02:43:47.707587+0000","last_peered":"2024-09-17T02:43:47.707587+0000","last_clean":"2024-09-17T02:43:47.707587+0000","last_became_active":"2024-09-17T02:43:45.671999+0000","last_became_peered":"2024-09-17T02:43:45.671999+0000","last_unstale":"2024-09-17T02:43:47.707587+0000","last_undegraded":"2024-09-17T02:43:47.707587+0000","last_fullsized":"2024-09-17T02:43:47.707587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:17.815894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700564+0000","last_change":"2024-09-17T02:43:46.698516+0000","last_active":"2024-09-17T02:43:47.700564+0000","last_peered":"2024-09-17T02:43:47.700564+0000","last_clean":"2024-09-17T02:43:47.700564+0000","last_became_active":"2024-09-17T02:43:46.696239+0000","last_became_peered":"2024-09-17T02:43:46.696239+0000","last_unstale":"2024-09-17T02:43:47.700564+0000","last_undegraded":"2024-09-17T02:43:47.700564+0000","last_fullsized":"2024-09-17T02:43:47.700564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:46.407009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.684341+0000","last_change":"2024-09-17T02:43:45.682922+0000","last_active":"2024-09-17T02:43:45.684341+0000","last_peered":"2024-09-17T02:43:45.684341+0000","last_clean":"2024-09-17T02:43:45.684341+0000","last_became_active":"2024-09-17T02:43:45.682556+0000","last_became_peered":"2024-09-17T02:43:45.682556+0000","last_unstale":"2024-09-17T02:43:45.684341+0000","last_undegraded":"2024-09-17T02:43:45.684341+0000","last_fullsized":"2024-09-17T02:43:45.684341+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:17.815894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651770+0000","last_change":"2024-09-17T02:43:45.651770+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651770+0000","last_undegraded":"2024-09-17T02:43:45.651770+0000","last_fullsized":"2024-09-17T02:43:45.651770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652374+0000","last_change":"2024-09-17T02:43:45.666290+0000","last_active":"2024-09-17T02:43:47.652374+0000","last_peered":"2024-09-17T02:43:47.652374+0000","last_clean":"2024-09-17T02:43:47.652374+0000","last_became_active":"2024-09-17T02:43:45.665410+0000","last_became_peered":"2024-09-17T02:43:45.665410+0000","last_unstale":"2024-09-17T02:43:47.652374+0000","last_undegraded":"2024-09-17T02:43:47.652374+0000","last_fullsized":"2024-09-17T02:43:47.652374+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:33.788490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.677792+0000","last_change":"2024-09-17T02:43:45.677792+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.677792+0000","last_undegraded":"2024-09-17T02:43:45.677792+0000","last_fullsized":"2024-09-17T02:43:45.677792+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707446+0000","last_change":"2024-09-17T02:43:45.672483+0000","last_active":"2024-09-17T02:43:47.707446+0000","last_peered":"2024-09-17T02:43:47.707446+0000","last_clean":"2024-09-17T02:43:47.707446+0000","last_became_active":"2024-09-17T02:43:45.669710+0000","last_became_peered":"2024-09-17T02:43:45.669710+0000","last_unstale":"2024-09-17T02:43:47.707446+0000","last_undegraded":"2024-09-17T02:43:47.707446+0000","last_fullsized":"2024-09-17T02:43:47.707446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:20.160389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647737+0000","last_change":"2024-09-17T02:43:46.697145+0000","last_active":"2024-09-17T02:43:47.647737+0000","last_peered":"2024-09-17T02:43:47.647737+0000","last_clean":"2024-09-17T02:43:47.647737+0000","last_became_active":"2024-09-17T02:43:46.696508+0000","last_became_peered":"2024-09-17T02:43:46.696508+0000","last_unstale":"2024-09-17T02:43:47.647737+0000","last_undegraded":"2024-09-17T02:43:47.647737+0000","last_fullsized":"2024-09-17T02:43:47.647737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:45.286919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.693830+0000","last_change":"2024-09-17T02:43:45.690426+0000","last_active":"2024-09-17T02:43:45.693830+0000","last_peered":"2024-09-17T02:43:45.693830+0000","last_clean":"2024-09-17T02:43:45.693830+0000","last_became_active":"2024-09-17T02:43:45.689221+0000","last_became_peered":"2024-09-17T02:43:45.689221+0000","last_unstale":"2024-09-17T02:43:45.693830+0000","last_undegraded":"2024-09-17T02:43:45.693830+0000","last_fullsized":"2024-09-17T02:43:45.693830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:16.815190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.647781+0000","last_change":"2024-09-17T02:43:45.647781+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.647781+0000","last_undegraded":"2024-09-17T02:43:45.647781+0000","last_fullsized":"2024-09-17T02:43:45.647781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691983+0000","last_change":"2024-09-17T02:43:45.688114+0000","last_active":"2024-09-17T02:43:45.691983+0000","last_peered":"2024-09-17T02:43:45.691983+0000","last_clean":"2024-09-17T02:43:45.691983+0000","last_became_active":"2024-09-17T02:43:45.687238+0000","last_became_peered":"2024-09-17T02:43:45.687238+0000","last_unstale":"2024-09-17T02:43:45.691983+0000","last_undegraded":"2024-09-17T02:43:45.691983+0000","last_fullsized":"2024-09-17T02:43:45.691983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:07.931860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":187,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.645912+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:43:45.645912+0000","last_peered":"2024-09-17T02:43:45.645912+0000","last_clean":"2024-09-17T02:43:45.645912+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:43:45.645912+0000","last_undegraded":"2024-09-17T02:43:45.645912+0000","last_fullsized":"2024-09-17T02:43:45.645912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":778240,"data_stored":349676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1335296,"data_stored":919518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":782336,"data_stored":350258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":741376,"data_stored":329150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":786432,"data_stored":349732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1335296,"data_stored":919518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31932,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339908,"statfs":{"total":96636764160,"available":96604065792,"internally_reserved":0,"allocated":753664,"data_stored":335817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":798720,"data_stored":351940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:43:51.641 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3e is not active+clean 2024-09-17T02:43:51.642 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.670539+0000', 'last_change': '2024-09-17T02:43:45.670539+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.670539+0000', 'last_undegraded': '2024-09-17T02:43:45.670539+0000', 'last_fullsized': '2024-09-17T02:43:45.670539+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T02:43:51.642 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3d is not active+clean 2024-09-17T02:43:51.642 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.652810+0000', 'last_change': '2024-09-17T02:43:45.652810+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.652810+0000', 'last_undegraded': '2024-09-17T02:43:45.652810+0000', 'last_fullsized': '2024-09-17T02:43:45.652810+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:43:51.642 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3b is not active+clean 2024-09-17T02:43:51.642 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.653624+0000', 'last_change': '2024-09-17T02:43:45.653624+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.653624+0000', 'last_undegraded': '2024-09-17T02:43:45.653624+0000', 'last_fullsized': '2024-09-17T02:43:45.653624+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:43:51.642 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3a is not active+clean 2024-09-17T02:43:51.643 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.679025+0000', 'last_change': '2024-09-17T02:43:45.679025+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.679025+0000', 'last_undegraded': '2024-09-17T02:43:45.679025+0000', 'last_fullsized': '2024-09-17T02:43:45.679025+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T02:43:51.643 INFO:tasks.ceph.ceph_manager.ceph:PG 10.38 is not active+clean 2024-09-17T02:43:51.643 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.651787+0000', 'last_change': '2024-09-17T02:43:45.651787+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.651787+0000', 'last_undegraded': '2024-09-17T02:43:45.651787+0000', 'last_fullsized': '2024-09-17T02:43:45.651787+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:43:51.643 INFO:tasks.ceph.ceph_manager.ceph:PG 10.36 is not active+clean 2024-09-17T02:43:51.643 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.646728+0000', 'last_change': '2024-09-17T02:43:45.646728+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.646728+0000', 'last_undegraded': '2024-09-17T02:43:45.646728+0000', 'last_fullsized': '2024-09-17T02:43:45.646728+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:43:51.643 INFO:tasks.ceph.ceph_manager.ceph:PG 10.33 is not active+clean 2024-09-17T02:43:51.643 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.655366+0000', 'last_change': '2024-09-17T02:43:45.655366+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.655366+0000', 'last_undegraded': '2024-09-17T02:43:45.655366+0000', 'last_fullsized': '2024-09-17T02:43:45.655366+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:43:51.643 INFO:tasks.ceph.ceph_manager.ceph:PG 10.32 is not active+clean 2024-09-17T02:43:51.644 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.654842+0000', 'last_change': '2024-09-17T02:43:45.654842+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.654842+0000', 'last_undegraded': '2024-09-17T02:43:45.654842+0000', 'last_fullsized': '2024-09-17T02:43:45.654842+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:43:51.644 INFO:tasks.ceph.ceph_manager.ceph:PG 10.31 is not active+clean 2024-09-17T02:43:51.644 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.648693+0000', 'last_change': '2024-09-17T02:43:45.648693+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.648693+0000', 'last_undegraded': '2024-09-17T02:43:45.648693+0000', 'last_fullsized': '2024-09-17T02:43:45.648693+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:43:51.644 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2b is not active+clean 2024-09-17T02:43:51.644 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.654859+0000', 'last_change': '2024-09-17T02:43:45.654859+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.654859+0000', 'last_undegraded': '2024-09-17T02:43:45.654859+0000', 'last_fullsized': '2024-09-17T02:43:45.654859+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:43:51.644 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2a is not active+clean 2024-09-17T02:43:51.644 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.669183+0000', 'last_change': '2024-09-17T02:43:45.669183+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.669183+0000', 'last_undegraded': '2024-09-17T02:43:45.669183+0000', 'last_fullsized': '2024-09-17T02:43:45.669183+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T02:43:51.644 INFO:tasks.ceph.ceph_manager.ceph:PG 10.28 is not active+clean 2024-09-17T02:43:51.644 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.651233+0000', 'last_change': '2024-09-17T02:43:45.651233+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.651233+0000', 'last_undegraded': '2024-09-17T02:43:45.651233+0000', 'last_fullsized': '2024-09-17T02:43:45.651233+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:43:51.644 INFO:tasks.ceph.ceph_manager.ceph:PG 10.27 is not active+clean 2024-09-17T02:43:51.644 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.654320+0000', 'last_change': '2024-09-17T02:43:45.654320+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.654320+0000', 'last_undegraded': '2024-09-17T02:43:45.654320+0000', 'last_fullsized': '2024-09-17T02:43:45.654320+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:43:51.645 INFO:tasks.ceph.ceph_manager.ceph:PG 10.26 is not active+clean 2024-09-17T02:43:51.645 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.652203+0000', 'last_change': '2024-09-17T02:43:45.652203+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.652203+0000', 'last_undegraded': '2024-09-17T02:43:45.652203+0000', 'last_fullsized': '2024-09-17T02:43:45.652203+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:43:51.645 INFO:tasks.ceph.ceph_manager.ceph:PG 10.25 is not active+clean 2024-09-17T02:43:51.645 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.654755+0000', 'last_change': '2024-09-17T02:43:45.654755+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.654755+0000', 'last_undegraded': '2024-09-17T02:43:45.654755+0000', 'last_fullsized': '2024-09-17T02:43:45.654755+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:43:51.645 INFO:tasks.ceph.ceph_manager.ceph:PG 10.21 is not active+clean 2024-09-17T02:43:51.645 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.650778+0000', 'last_change': '2024-09-17T02:43:45.650778+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.650778+0000', 'last_undegraded': '2024-09-17T02:43:45.650778+0000', 'last_fullsized': '2024-09-17T02:43:45.650778+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:43:51.645 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1d is not active+clean 2024-09-17T02:43:51.645 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.651601+0000', 'last_change': '2024-09-17T02:43:45.651601+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.651601+0000', 'last_undegraded': '2024-09-17T02:43:45.651601+0000', 'last_fullsized': '2024-09-17T02:43:45.651601+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:43:51.645 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1e is not active+clean 2024-09-17T02:43:51.646 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.669291+0000', 'last_change': '2024-09-17T02:43:45.669291+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.669291+0000', 'last_undegraded': '2024-09-17T02:43:45.669291+0000', 'last_fullsized': '2024-09-17T02:43:45.669291+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T02:43:51.646 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1a is not active+clean 2024-09-17T02:43:51.646 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.646618+0000', 'last_change': '2024-09-17T02:43:45.646618+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.646618+0000', 'last_undegraded': '2024-09-17T02:43:45.646618+0000', 'last_fullsized': '2024-09-17T02:43:45.646618+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:43:51.646 INFO:tasks.ceph.ceph_manager.ceph:PG 10.16 is not active+clean 2024-09-17T02:43:51.646 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.654576+0000', 'last_change': '2024-09-17T02:43:45.654576+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.654576+0000', 'last_undegraded': '2024-09-17T02:43:45.654576+0000', 'last_fullsized': '2024-09-17T02:43:45.654576+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:43:51.646 INFO:tasks.ceph.ceph_manager.ceph:PG 10.17 is not active+clean 2024-09-17T02:43:51.646 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.653809+0000', 'last_change': '2024-09-17T02:43:45.653809+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.653809+0000', 'last_undegraded': '2024-09-17T02:43:45.653809+0000', 'last_fullsized': '2024-09-17T02:43:45.653809+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:43:51.646 INFO:tasks.ceph.ceph_manager.ceph:PG 10.10 is not active+clean 2024-09-17T02:43:51.646 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.646761+0000', 'last_change': '2024-09-17T02:43:45.646761+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.646761+0000', 'last_undegraded': '2024-09-17T02:43:45.646761+0000', 'last_fullsized': '2024-09-17T02:43:45.646761+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:43:51.646 INFO:tasks.ceph.ceph_manager.ceph:PG 10.11 is not active+clean 2024-09-17T02:43:51.646 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.648211+0000', 'last_change': '2024-09-17T02:43:45.648211+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.648211+0000', 'last_undegraded': '2024-09-17T02:43:45.648211+0000', 'last_fullsized': '2024-09-17T02:43:45.648211+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:43:51.647 INFO:tasks.ceph.ceph_manager.ceph:PG 10.12 is not active+clean 2024-09-17T02:43:51.647 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.645997+0000', 'last_change': '2024-09-17T02:43:45.645997+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.645997+0000', 'last_undegraded': '2024-09-17T02:43:45.645997+0000', 'last_fullsized': '2024-09-17T02:43:45.645997+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:43:51.647 INFO:tasks.ceph.ceph_manager.ceph:PG 10.d is not active+clean 2024-09-17T02:43:51.647 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.652497+0000', 'last_change': '2024-09-17T02:43:45.652497+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.652497+0000', 'last_undegraded': '2024-09-17T02:43:45.652497+0000', 'last_fullsized': '2024-09-17T02:43:45.652497+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:43:51.647 INFO:tasks.ceph.ceph_manager.ceph:PG 10.e is not active+clean 2024-09-17T02:43:51.647 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.653757+0000', 'last_change': '2024-09-17T02:43:45.653757+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.653757+0000', 'last_undegraded': '2024-09-17T02:43:45.653757+0000', 'last_fullsized': '2024-09-17T02:43:45.653757+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:43:51.647 INFO:tasks.ceph.ceph_manager.ceph:PG 10.f is not active+clean 2024-09-17T02:43:51.647 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.652610+0000', 'last_change': '2024-09-17T02:43:45.652610+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.652610+0000', 'last_undegraded': '2024-09-17T02:43:45.652610+0000', 'last_fullsized': '2024-09-17T02:43:45.652610+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:43:51.647 INFO:tasks.ceph.ceph_manager.ceph:PG 10.8 is not active+clean 2024-09-17T02:43:51.647 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.675296+0000', 'last_change': '2024-09-17T02:43:45.675296+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.675296+0000', 'last_undegraded': '2024-09-17T02:43:45.675296+0000', 'last_fullsized': '2024-09-17T02:43:45.675296+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T02:43:51.648 INFO:tasks.ceph.ceph_manager.ceph:PG 10.9 is not active+clean 2024-09-17T02:43:51.648 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.652569+0000', 'last_change': '2024-09-17T02:43:45.652569+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.652569+0000', 'last_undegraded': '2024-09-17T02:43:45.652569+0000', 'last_fullsized': '2024-09-17T02:43:45.652569+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:43:51.648 INFO:tasks.ceph.ceph_manager.ceph:PG 10.6 is not active+clean 2024-09-17T02:43:51.648 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.649260+0000', 'last_change': '2024-09-17T02:43:45.649260+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.649260+0000', 'last_undegraded': '2024-09-17T02:43:45.649260+0000', 'last_fullsized': '2024-09-17T02:43:45.649260+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:43:51.648 INFO:tasks.ceph.ceph_manager.ceph:PG 10.7 is not active+clean 2024-09-17T02:43:51.648 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.651328+0000', 'last_change': '2024-09-17T02:43:45.651328+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.651328+0000', 'last_undegraded': '2024-09-17T02:43:45.651328+0000', 'last_fullsized': '2024-09-17T02:43:45.651328+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T02:43:51.648 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1 is not active+clean 2024-09-17T02:43:51.648 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.651770+0000', 'last_change': '2024-09-17T02:43:45.651770+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.651770+0000', 'last_undegraded': '2024-09-17T02:43:45.651770+0000', 'last_fullsized': '2024-09-17T02:43:45.651770+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T02:43:51.648 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2 is not active+clean 2024-09-17T02:43:51.648 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.677792+0000', 'last_change': '2024-09-17T02:43:45.677792+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.677792+0000', 'last_undegraded': '2024-09-17T02:43:45.677792+0000', 'last_fullsized': '2024-09-17T02:43:45.677792+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T02:43:51.649 INFO:tasks.ceph.ceph_manager.ceph:PG 10.b is not active+clean 2024-09-17T02:43:51.649 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:43:45.647781+0000', 'last_change': '2024-09-17T02:43:45.647781+0000', 'last_active': '2024-09-17T02:43:45.623379+0000', 'last_peered': '2024-09-17T02:43:45.623379+0000', 'last_clean': '2024-09-17T02:43:45.623379+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:43:45.647781+0000', 'last_undegraded': '2024-09-17T02:43:45.647781+0000', 'last_fullsized': '2024-09-17T02:43:45.647781+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'last_clean_scrub_stamp': '2024-09-17T02:43:45.623379+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T02:43:51.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T02:43:51.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.815+0000 7fe21c07e640 1 -- 172.21.15.131:0/1975512376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe21415c5b0 msgr2=0x7fe2140cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.815+0000 7fe21c07e640 1 --2- 172.21.15.131:0/1975512376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe21415c5b0 0x7fe2140cbaf0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009bc0 tx=0x7fe20c01cba0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.818 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.815+0000 7fe21c07e640 1 -- 172.21.15.131:0/1975512376 shutdown_connections 2024-09-17T02:43:51.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.815+0000 7fe21c07e640 1 --2- 172.21.15.131:0/1975512376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe21415c5b0 0x7fe2140cbaf0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.815+0000 7fe21c07e640 1 -- 172.21.15.131:0/1975512376 >> 172.21.15.131:0/1975512376 conn(0x7fe21406eea0 msgr2=0x7fe21406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:51.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.815+0000 7fe21c07e640 1 -- 172.21.15.131:0/1975512376 shutdown_connections 2024-09-17T02:43:51.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.815+0000 7fe21c07e640 1 -- 172.21.15.131:0/1975512376 wait complete. 2024-09-17T02:43:51.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.815+0000 7fe21c07e640 1 Processor -- start 2024-09-17T02:43:51.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.815+0000 7fe21c07e640 1 -- start start 2024-09-17T02:43:51.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21c07e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2140ced00 0x7fe214148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21c07e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe21415c5b0 0x7fe21414a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21c07e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe21414f8e0 0x7fe2140b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21c07e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe214169ad0 con 0x7fe21414f8e0 2024-09-17T02:43:51.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21c07e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe214169950 con 0x7fe21415c5b0 2024-09-17T02:43:51.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21c07e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe214169c50 con 0x7fe2140ced00 2024-09-17T02:43:51.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21a61b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe21414f8e0 0x7fe2140b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:51.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21a61b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe21414f8e0 0x7fe2140b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51318/0 (socket says 172.21.15.131:51318) 2024-09-17T02:43:51.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21a61b640 1 -- 172.21.15.131:0/3842451939 learned_addr learned my addr 172.21.15.131:0/3842451939 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:51.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21a61b640 1 -- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2140ced00 msgr2=0x7fe214148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe219e1a640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2140ced00 0x7fe214148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:51.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21a61b640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2140ced00 0x7fe214148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21a61b640 1 -- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe21415c5b0 msgr2=0x7fe21414a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:51.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21a61b640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe21415c5b0 0x7fe21414a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:51.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21a61b640 1 -- 172.21.15.131:0/3842451939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2141504c0 con 0x7fe21414f8e0 2024-09-17T02:43:51.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21a61b640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe21414f8e0 0x7fe2140b0420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe20c01a7d0 tx=0x7fe20c0a59d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:51.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe20a7fc640 1 -- 172.21.15.131:0/3842451939 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0bb070 con 0x7fe21414f8e0 2024-09-17T02:43:51.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe20a7fc640 1 -- 172.21.15.131:0/3842451939 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe20c004d40 con 0x7fe21414f8e0 2024-09-17T02:43:51.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe20a7fc640 1 -- 172.21.15.131:0/3842451939 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0b2040 con 0x7fe21414f8e0 2024-09-17T02:43:51.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21c07e640 1 -- 172.21.15.131:0/3842451939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21414ffc0 con 0x7fe21414f8e0 2024-09-17T02:43:51.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21c07e640 1 -- 172.21.15.131:0/3842451939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe214149650 con 0x7fe21414f8e0 2024-09-17T02:43:51.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe21c07e640 1 -- 172.21.15.131:0/3842451939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe214075b80 con 0x7fe21414f8e0 2024-09-17T02:43:51.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.819+0000 7fe20a7fc640 1 -- 172.21.15.131:0/3842451939 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe20c0be070 con 0x7fe21414f8e0 2024-09-17T02:43:51.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.823+0000 7fe20a7fc640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe1f8073230 0x7fe1f8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:51.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.823+0000 7fe20a7fc640 1 -- 172.21.15.131:0/3842451939 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fe20c12e400 con 0x7fe21414f8e0 2024-09-17T02:43:51.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.823+0000 7fe219e1a640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe1f8073230 0x7fe1f8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:51.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.823+0000 7fe219e1a640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe1f8073230 0x7fe1f8075740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe21000a790 tx=0x7fe210008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:51.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:51.823+0000 7fe20a7fc640 1 -- 172.21.15.131:0/3842451939 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe20c016790 con 0x7fe21414f8e0 2024-09-17T02:43:52.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.047+0000 7fe21c07e640 1 -- 172.21.15.131:0/3842451939 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe2140cbaf0 con 0x7fe21414f8e0 2024-09-17T02:43:52.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.047+0000 7fe20a7fc640 1 -- 172.21.15.131:0/3842451939 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1431 (secure 0 0 0) 0x7fe20c0f9490 con 0x7fe21414f8e0 2024-09-17T02:43:52.051 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:52.052 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":205,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":8,"num_up_osds":8,"osd_up_since":1726540840,"num_in_osds":8,"osd_in_since":1726540830,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":63},{"state_name":"creating+peering","count":34}],"num_pgs":97,"num_pools":3,"num_objects":11,"data_bytes":591006,"bytes_used":258584576,"bytes_avail":772835528704,"bytes_total":773094113280,"inactive_pgs_ratio":0.3505154550075531,"write_bytes_sec":1535,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":36,"btime":"2024-09-17T02:43:49:702553+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":5871}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-17T02:43:43.184158+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T02:43:52.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 -- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe1f8073230 msgr2=0x7fe1f8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:52.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe1f8073230 0x7fe1f8075740 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe21000a790 tx=0x7fe210008040 comp rx=0 tx=0).stop 2024-09-17T02:43:52.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 -- 172.21.15.131:0/3842451939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe21414f8e0 msgr2=0x7fe2140b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:52.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe21414f8e0 0x7fe2140b0420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fe20c01a7d0 tx=0x7fe20c0a59d0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 -- 172.21.15.131:0/3842451939 shutdown_connections 2024-09-17T02:43:52.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe1f8073230 0x7fe1f8075740 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe21414f8e0 0x7fe2140b0420 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe21415c5b0 0x7fe21414a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 --2- 172.21.15.131:0/3842451939 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2140ced00 0x7fe214148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 -- 172.21.15.131:0/3842451939 >> 172.21.15.131:0/3842451939 conn(0x7fe21406eea0 msgr2=0x7fe2140cb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:52.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 -- 172.21.15.131:0/3842451939 shutdown_connections 2024-09-17T02:43:52.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.051+0000 7fe21c07e640 1 -- 172.21.15.131:0/3842451939 wait complete. 2024-09-17T02:43:52.072 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T02:43:52.072 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:43:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.215+0000 7f139420a640 1 -- 172.21.15.131:0/1639743484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f138c1606a0 msgr2=0x7f138c152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.215+0000 7f139420a640 1 --2- 172.21.15.131:0/1639743484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f138c1606a0 0x7f138c152be0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1384009bc0 tx=0x7f138401cae0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.215+0000 7f139420a640 1 -- 172.21.15.131:0/1639743484 shutdown_connections 2024-09-17T02:43:52.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.215+0000 7f139420a640 1 --2- 172.21.15.131:0/1639743484 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f138c1606a0 0x7f138c152be0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.215+0000 7f139420a640 1 --2- 172.21.15.131:0/1639743484 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f138c0cce40 0x7f138c06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.215+0000 7f139420a640 1 -- 172.21.15.131:0/1639743484 >> 172.21.15.131:0/1639743484 conn(0x7f138c06fbe0 msgr2=0x7f138c06fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.215+0000 7f139420a640 1 -- 172.21.15.131:0/1639743484 shutdown_connections 2024-09-17T02:43:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.215+0000 7f139420a640 1 -- 172.21.15.131:0/1639743484 wait complete. 2024-09-17T02:43:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f139420a640 1 Processor -- start 2024-09-17T02:43:52.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f139420a640 1 -- start start 2024-09-17T02:43:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f139420a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f138c0cce40 0x7f138c14b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f139420a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f138c0cd250 0x7f138c0d0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f139420a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f138c0b4f50 0x7f138c0b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f139420a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f138c0cc180 con 0x7f138c0cce40 2024-09-17T02:43:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f139420a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f138c0cc000 con 0x7f138c0cd250 2024-09-17T02:43:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f139420a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f138c0cc300 con 0x7f138c0b4f50 2024-09-17T02:43:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f1391fa6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f138c0cce40 0x7f138c14b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f1391fa6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f138c0cce40 0x7f138c14b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51324/0 (socket says 172.21.15.131:51324) 2024-09-17T02:43:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f1391fa6640 1 -- 172.21.15.131:0/2696786469 learned_addr learned my addr 172.21.15.131:0/2696786469 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:52.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f13927a7640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f138c0b4f50 0x7f138c0b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f13917a5640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f138c0cd250 0x7f138c0d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f1391fa6640 1 -- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f138c0b4f50 msgr2=0x7f138c0b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f1391fa6640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f138c0b4f50 0x7f138c0b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f1391fa6640 1 -- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f138c0cd250 msgr2=0x7f138c0d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f1391fa6640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f138c0cd250 0x7f138c0d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f1391fa6640 1 -- 172.21.15.131:0/2696786469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138c0b58a0 con 0x7f138c0cce40 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f1391fa6640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f138c0cce40 0x7f138c14b060 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f137c00e7a0 tx=0x7f137c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f13827fc640 1 -- 172.21.15.131:0/2696786469 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c00ccc0 con 0x7f138c0cce40 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f139420a640 1 -- 172.21.15.131:0/2696786469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138c1525b0 con 0x7f138c0cce40 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f13827fc640 1 -- 172.21.15.131:0/2696786469 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f137c010430 con 0x7f138c0cce40 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f13827fc640 1 -- 172.21.15.131:0/2696786469 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c010860 con 0x7f138c0cce40 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.219+0000 7f139420a640 1 -- 172.21.15.131:0/2696786469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f138c152b70 con 0x7f138c0cce40 2024-09-17T02:43:52.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.223+0000 7f139420a640 1 -- 172.21.15.131:0/2696786469 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138c06db20 con 0x7f138c0cce40 2024-09-17T02:43:52.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.223+0000 7f13827fc640 1 -- 172.21.15.131:0/2696786469 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f137c010a00 con 0x7f138c0cce40 2024-09-17T02:43:52.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.223+0000 7f13827fc640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1370073230 0x7f1370075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:52.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.223+0000 7f13827fc640 1 -- 172.21.15.131:0/2696786469 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f137c094dd0 con 0x7f138c0cce40 2024-09-17T02:43:52.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.223+0000 7f13917a5640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1370073230 0x7f1370075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:52.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.223+0000 7f13917a5640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1370073230 0x7f1370075740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f138c151680 tx=0x7f1388009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:52.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.227+0000 7f13827fc640 1 -- 172.21.15.131:0/2696786469 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f138c0b58a0 con 0x7f138c0cce40 2024-09-17T02:43:52.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.387+0000 7f139420a640 1 -- 172.21.15.131:0/2696786469 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f138c14b5a0 con 0x7f1370073230 2024-09-17T02:43:52.400 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.399+0000 7f13827fc640 1 -- 172.21.15.131:0/2696786469 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249217 (secure 0 0 0) 0x7f1388014420 con 0x7f1370073230 2024-09-17T02:43:52.403 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:52.405 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:43:52.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 -- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1370073230 msgr2=0x7f1370075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:52.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1370073230 0x7f1370075740 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f138c151680 tx=0x7f1388009040 comp rx=0 tx=0).stop 2024-09-17T02:43:52.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 -- 172.21.15.131:0/2696786469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f138c0cce40 msgr2=0x7f138c14b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:52.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f138c0cce40 0x7f138c14b060 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f137c00e7a0 tx=0x7f137c00ec60 comp rx=0 tx=0).stop 2024-09-17T02:43:52.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 -- 172.21.15.131:0/2696786469 shutdown_connections 2024-09-17T02:43:52.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1370073230 0x7f1370075740 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f138c0b4f50 0x7f138c0b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f138c0cd250 0x7f138c0d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 --2- 172.21.15.131:0/2696786469 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f138c0cce40 0x7f138c14b060 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:52.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 -- 172.21.15.131:0/2696786469 >> 172.21.15.131:0/2696786469 conn(0x7f138c06fbe0 msgr2=0x7f138c0b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:52.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 -- 172.21.15.131:0/2696786469 shutdown_connections 2024-09-17T02:43:52.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:52.403+0000 7f139420a640 1 -- 172.21.15.131:0/2696786469 wait complete. 2024-09-17T02:43:52.423 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-09-17T02:43:51.186677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591006,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":119,"num_write_kb":1637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":109,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252524,"kb_used_data":7140,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722196,"statfs":{"total":773094113280,"available":772835528704,"internally_reserved":0,"allocated":7311360,"data_stored":3905609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002002"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652370+0000","last_change":"2024-09-17T02:43:46.680494+0000","last_active":"2024-09-17T02:43:47.652370+0000","last_peered":"2024-09-17T02:43:47.652370+0000","last_clean":"2024-09-17T02:43:47.652370+0000","last_became_active":"2024-09-17T02:43:46.679436+0000","last_became_peered":"2024-09-17T02:43:46.679436+0000","last_unstale":"2024-09-17T02:43:47.652370+0000","last_undegraded":"2024-09-17T02:43:47.652370+0000","last_fullsized":"2024-09-17T02:43:47.652370+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:18.520380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.670539+0000","last_change":"2024-09-17T02:43:45.670539+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.670539+0000","last_undegraded":"2024-09-17T02:43:45.670539+0000","last_fullsized":"2024-09-17T02:43:45.670539+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652810+0000","last_change":"2024-09-17T02:43:45.652810+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652810+0000","last_undegraded":"2024-09-17T02:43:45.652810+0000","last_fullsized":"2024-09-17T02:43:45.652810+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701851+0000","last_change":"2024-09-17T02:43:46.708732+0000","last_active":"2024-09-17T02:43:47.701851+0000","last_peered":"2024-09-17T02:43:47.701851+0000","last_clean":"2024-09-17T02:43:47.701851+0000","last_became_active":"2024-09-17T02:43:46.706193+0000","last_became_peered":"2024-09-17T02:43:46.706193+0000","last_unstale":"2024-09-17T02:43:47.701851+0000","last_undegraded":"2024-09-17T02:43:47.701851+0000","last_fullsized":"2024-09-17T02:43:47.701851+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:27.991604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653624+0000","last_change":"2024-09-17T02:43:45.653624+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653624+0000","last_undegraded":"2024-09-17T02:43:45.653624+0000","last_fullsized":"2024-09-17T02:43:45.653624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.679025+0000","last_change":"2024-09-17T02:43:45.679025+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.679025+0000","last_undegraded":"2024-09-17T02:43:45.679025+0000","last_fullsized":"2024-09-17T02:43:45.679025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649842+0000","last_change":"2024-09-17T02:43:46.678701+0000","last_active":"2024-09-17T02:43:47.649842+0000","last_peered":"2024-09-17T02:43:47.649842+0000","last_clean":"2024-09-17T02:43:47.649842+0000","last_became_active":"2024-09-17T02:43:46.674482+0000","last_became_peered":"2024-09-17T02:43:46.674482+0000","last_unstale":"2024-09-17T02:43:47.649842+0000","last_undegraded":"2024-09-17T02:43:47.649842+0000","last_fullsized":"2024-09-17T02:43:47.649842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:30.882039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651787+0000","last_change":"2024-09-17T02:43:45.651787+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651787+0000","last_undegraded":"2024-09-17T02:43:45.651787+0000","last_fullsized":"2024-09-17T02:43:45.651787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702086+0000","last_change":"2024-09-17T02:43:46.695684+0000","last_active":"2024-09-17T02:43:47.702086+0000","last_peered":"2024-09-17T02:43:47.702086+0000","last_clean":"2024-09-17T02:43:47.702086+0000","last_became_active":"2024-09-17T02:43:46.694860+0000","last_became_peered":"2024-09-17T02:43:46.694860+0000","last_unstale":"2024-09-17T02:43:47.702086+0000","last_undegraded":"2024-09-17T02:43:47.702086+0000","last_fullsized":"2024-09-17T02:43:47.702086+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:53.983748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646728+0000","last_change":"2024-09-17T02:43:45.646728+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646728+0000","last_undegraded":"2024-09-17T02:43:45.646728+0000","last_fullsized":"2024-09-17T02:43:45.646728+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702375+0000","last_change":"2024-09-17T02:43:46.701807+0000","last_active":"2024-09-17T02:43:47.702375+0000","last_peered":"2024-09-17T02:43:47.702375+0000","last_clean":"2024-09-17T02:43:47.702375+0000","last_became_active":"2024-09-17T02:43:46.696272+0000","last_became_peered":"2024-09-17T02:43:46.696272+0000","last_unstale":"2024-09-17T02:43:47.702375+0000","last_undegraded":"2024-09-17T02:43:47.702375+0000","last_fullsized":"2024-09-17T02:43:47.702375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:47.850100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702109+0000","last_change":"2024-09-17T02:43:46.707994+0000","last_active":"2024-09-17T02:43:47.702109+0000","last_peered":"2024-09-17T02:43:47.702109+0000","last_clean":"2024-09-17T02:43:47.702109+0000","last_became_active":"2024-09-17T02:43:46.701425+0000","last_became_peered":"2024-09-17T02:43:46.701425+0000","last_unstale":"2024-09-17T02:43:47.702109+0000","last_undegraded":"2024-09-17T02:43:47.702109+0000","last_fullsized":"2024-09-17T02:43:47.702109+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:42.080955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.655366+0000","last_change":"2024-09-17T02:43:45.655366+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.655366+0000","last_undegraded":"2024-09-17T02:43:45.655366+0000","last_fullsized":"2024-09-17T02:43:45.655366+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654842+0000","last_change":"2024-09-17T02:43:45.654842+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654842+0000","last_undegraded":"2024-09-17T02:43:45.654842+0000","last_fullsized":"2024-09-17T02:43:45.654842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.648693+0000","last_change":"2024-09-17T02:43:45.648693+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.648693+0000","last_undegraded":"2024-09-17T02:43:45.648693+0000","last_fullsized":"2024-09-17T02:43:45.648693+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707657+0000","last_change":"2024-09-17T02:43:46.708131+0000","last_active":"2024-09-17T02:43:47.707657+0000","last_peered":"2024-09-17T02:43:47.707657+0000","last_clean":"2024-09-17T02:43:47.707657+0000","last_became_active":"2024-09-17T02:43:46.707665+0000","last_became_peered":"2024-09-17T02:43:46.707665+0000","last_unstale":"2024-09-17T02:43:47.707657+0000","last_undegraded":"2024-09-17T02:43:47.707657+0000","last_fullsized":"2024-09-17T02:43:47.707657+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:48:16.207446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.650043+0000","last_change":"2024-09-17T02:43:46.702269+0000","last_active":"2024-09-17T02:43:47.650043+0000","last_peered":"2024-09-17T02:43:47.650043+0000","last_clean":"2024-09-17T02:43:47.650043+0000","last_became_active":"2024-09-17T02:43:46.701883+0000","last_became_peered":"2024-09-17T02:43:46.701883+0000","last_unstale":"2024-09-17T02:43:47.650043+0000","last_undegraded":"2024-09-17T02:43:47.650043+0000","last_fullsized":"2024-09-17T02:43:47.650043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:22.975866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702426+0000","last_change":"2024-09-17T02:43:46.693189+0000","last_active":"2024-09-17T02:43:47.702426+0000","last_peered":"2024-09-17T02:43:47.702426+0000","last_clean":"2024-09-17T02:43:47.702426+0000","last_became_active":"2024-09-17T02:43:46.691164+0000","last_became_peered":"2024-09-17T02:43:46.691164+0000","last_unstale":"2024-09-17T02:43:47.702426+0000","last_undegraded":"2024-09-17T02:43:47.702426+0000","last_fullsized":"2024-09-17T02:43:47.702426+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:52.980125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702146+0000","last_change":"2024-09-17T02:43:46.692645+0000","last_active":"2024-09-17T02:43:47.702146+0000","last_peered":"2024-09-17T02:43:47.702146+0000","last_clean":"2024-09-17T02:43:47.702146+0000","last_became_active":"2024-09-17T02:43:46.691861+0000","last_became_peered":"2024-09-17T02:43:46.691861+0000","last_unstale":"2024-09-17T02:43:47.702146+0000","last_undegraded":"2024-09-17T02:43:47.702146+0000","last_fullsized":"2024-09-17T02:43:47.702146+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:03.405001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647781+0000","last_change":"2024-09-17T02:43:46.684848+0000","last_active":"2024-09-17T02:43:47.647781+0000","last_peered":"2024-09-17T02:43:47.647781+0000","last_clean":"2024-09-17T02:43:47.647781+0000","last_became_active":"2024-09-17T02:43:46.683919+0000","last_became_peered":"2024-09-17T02:43:46.683919+0000","last_unstale":"2024-09-17T02:43:47.647781+0000","last_undegraded":"2024-09-17T02:43:47.647781+0000","last_fullsized":"2024-09-17T02:43:47.647781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:58.516266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654859+0000","last_change":"2024-09-17T02:43:45.654859+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654859+0000","last_undegraded":"2024-09-17T02:43:45.654859+0000","last_fullsized":"2024-09-17T02:43:45.654859+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.669183+0000","last_change":"2024-09-17T02:43:45.669183+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.669183+0000","last_undegraded":"2024-09-17T02:43:45.669183+0000","last_fullsized":"2024-09-17T02:43:45.669183+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707376+0000","last_change":"2024-09-17T02:43:46.708531+0000","last_active":"2024-09-17T02:43:47.707376+0000","last_peered":"2024-09-17T02:43:47.707376+0000","last_clean":"2024-09-17T02:43:47.707376+0000","last_became_active":"2024-09-17T02:43:46.708224+0000","last_became_peered":"2024-09-17T02:43:46.708224+0000","last_unstale":"2024-09-17T02:43:47.707376+0000","last_undegraded":"2024-09-17T02:43:47.707376+0000","last_fullsized":"2024-09-17T02:43:47.707376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:37.303173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651233+0000","last_change":"2024-09-17T02:43:45.651233+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651233+0000","last_undegraded":"2024-09-17T02:43:45.651233+0000","last_fullsized":"2024-09-17T02:43:45.651233+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654320+0000","last_change":"2024-09-17T02:43:45.654320+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654320+0000","last_undegraded":"2024-09-17T02:43:45.654320+0000","last_fullsized":"2024-09-17T02:43:45.654320+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652203+0000","last_change":"2024-09-17T02:43:45.652203+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652203+0000","last_undegraded":"2024-09-17T02:43:45.652203+0000","last_fullsized":"2024-09-17T02:43:45.652203+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654755+0000","last_change":"2024-09-17T02:43:45.654755+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654755+0000","last_undegraded":"2024-09-17T02:43:45.654755+0000","last_fullsized":"2024-09-17T02:43:45.654755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652393+0000","last_change":"2024-09-17T02:43:46.713535+0000","last_active":"2024-09-17T02:43:47.652393+0000","last_peered":"2024-09-17T02:43:47.652393+0000","last_clean":"2024-09-17T02:43:47.652393+0000","last_became_active":"2024-09-17T02:43:46.712681+0000","last_became_peered":"2024-09-17T02:43:46.712681+0000","last_unstale":"2024-09-17T02:43:47.652393+0000","last_undegraded":"2024-09-17T02:43:47.652393+0000","last_fullsized":"2024-09-17T02:43:47.652393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:21.464657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701453+0000","last_change":"2024-09-17T02:43:46.706426+0000","last_active":"2024-09-17T02:43:47.701453+0000","last_peered":"2024-09-17T02:43:47.701453+0000","last_clean":"2024-09-17T02:43:47.701453+0000","last_became_active":"2024-09-17T02:43:46.706046+0000","last_became_peered":"2024-09-17T02:43:46.706046+0000","last_unstale":"2024-09-17T02:43:47.701453+0000","last_undegraded":"2024-09-17T02:43:47.701453+0000","last_fullsized":"2024-09-17T02:43:47.701453+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:58.594878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700841+0000","last_change":"2024-09-17T02:43:46.702605+0000","last_active":"2024-09-17T02:43:47.700841+0000","last_peered":"2024-09-17T02:43:47.700841+0000","last_clean":"2024-09-17T02:43:47.700841+0000","last_became_active":"2024-09-17T02:43:46.702182+0000","last_became_peered":"2024-09-17T02:43:46.702182+0000","last_unstale":"2024-09-17T02:43:47.700841+0000","last_undegraded":"2024-09-17T02:43:47.700841+0000","last_fullsized":"2024-09-17T02:43:47.700841+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:17.795947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.650778+0000","last_change":"2024-09-17T02:43:45.650778+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.650778+0000","last_undegraded":"2024-09-17T02:43:45.650778+0000","last_fullsized":"2024-09-17T02:43:45.650778+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649781+0000","last_change":"2024-09-17T02:43:46.673631+0000","last_active":"2024-09-17T02:43:47.649781+0000","last_peered":"2024-09-17T02:43:47.649781+0000","last_clean":"2024-09-17T02:43:47.649781+0000","last_became_active":"2024-09-17T02:43:46.672779+0000","last_became_peered":"2024-09-17T02:43:46.672779+0000","last_unstale":"2024-09-17T02:43:47.649781+0000","last_undegraded":"2024-09-17T02:43:47.649781+0000","last_fullsized":"2024-09-17T02:43:47.649781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:04.209405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702521+0000","last_change":"2024-09-17T02:43:46.710347+0000","last_active":"2024-09-17T02:43:47.702521+0000","last_peered":"2024-09-17T02:43:47.702521+0000","last_clean":"2024-09-17T02:43:47.702521+0000","last_became_active":"2024-09-17T02:43:46.708415+0000","last_became_peered":"2024-09-17T02:43:46.708415+0000","last_unstale":"2024-09-17T02:43:47.702521+0000","last_undegraded":"2024-09-17T02:43:47.702521+0000","last_fullsized":"2024-09-17T02:43:47.702521+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:23.928100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":19,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652160+0000","last_change":"2024-09-17T02:43:45.678361+0000","last_active":"2024-09-17T02:43:47.652160+0000","last_peered":"2024-09-17T02:43:47.652160+0000","last_clean":"2024-09-17T02:43:47.652160+0000","last_became_active":"2024-09-17T02:43:45.677598+0000","last_became_peered":"2024-09-17T02:43:45.677598+0000","last_unstale":"2024-09-17T02:43:47.652160+0000","last_undegraded":"2024-09-17T02:43:47.652160+0000","last_fullsized":"2024-09-17T02:43:47.652160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:08.498525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651601+0000","last_change":"2024-09-17T02:43:45.651601+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651601+0000","last_undegraded":"2024-09-17T02:43:45.651601+0000","last_fullsized":"2024-09-17T02:43:45.651601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651944+0000","last_change":"2024-09-17T02:43:45.660427+0000","last_active":"2024-09-17T02:43:47.651944+0000","last_peered":"2024-09-17T02:43:47.651944+0000","last_clean":"2024-09-17T02:43:47.651944+0000","last_became_active":"2024-09-17T02:43:45.659575+0000","last_became_peered":"2024-09-17T02:43:45.659575+0000","last_unstale":"2024-09-17T02:43:47.651944+0000","last_undegraded":"2024-09-17T02:43:47.651944+0000","last_fullsized":"2024-09-17T02:43:47.651944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:35.225547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.669291+0000","last_change":"2024-09-17T02:43:45.669291+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.669291+0000","last_undegraded":"2024-09-17T02:43:45.669291+0000","last_fullsized":"2024-09-17T02:43:45.669291+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652828+0000","last_change":"2024-09-17T02:43:45.665351+0000","last_active":"2024-09-17T02:43:47.652828+0000","last_peered":"2024-09-17T02:43:47.652828+0000","last_clean":"2024-09-17T02:43:47.652828+0000","last_became_active":"2024-09-17T02:43:45.664498+0000","last_became_peered":"2024-09-17T02:43:45.664498+0000","last_unstale":"2024-09-17T02:43:47.652828+0000","last_undegraded":"2024-09-17T02:43:47.652828+0000","last_fullsized":"2024-09-17T02:43:47.652828+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:35.011302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703229+0000","last_change":"2024-09-17T02:43:46.694855+0000","last_active":"2024-09-17T02:43:47.703229+0000","last_peered":"2024-09-17T02:43:47.703229+0000","last_clean":"2024-09-17T02:43:47.703229+0000","last_became_active":"2024-09-17T02:43:46.692123+0000","last_became_peered":"2024-09-17T02:43:46.692123+0000","last_unstale":"2024-09-17T02:43:47.703229+0000","last_undegraded":"2024-09-17T02:43:47.703229+0000","last_fullsized":"2024-09-17T02:43:47.703229+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:38.594076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.693843+0000","last_change":"2024-09-17T02:43:45.690176+0000","last_active":"2024-09-17T02:43:45.693843+0000","last_peered":"2024-09-17T02:43:45.693843+0000","last_clean":"2024-09-17T02:43:45.693843+0000","last_became_active":"2024-09-17T02:43:45.689355+0000","last_became_peered":"2024-09-17T02:43:45.689355+0000","last_unstale":"2024-09-17T02:43:45.693843+0000","last_undegraded":"2024-09-17T02:43:45.693843+0000","last_fullsized":"2024-09-17T02:43:45.693843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:40.666325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.646844+0000","last_change":"2024-09-17T02:43:46.678861+0000","last_active":"2024-09-17T02:43:47.646844+0000","last_peered":"2024-09-17T02:43:47.646844+0000","last_clean":"2024-09-17T02:43:47.646844+0000","last_became_active":"2024-09-17T02:43:46.677801+0000","last_became_peered":"2024-09-17T02:43:46.677801+0000","last_unstale":"2024-09-17T02:43:47.646844+0000","last_undegraded":"2024-09-17T02:43:47.646844+0000","last_fullsized":"2024-09-17T02:43:47.646844+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:31.162139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647995+0000","last_change":"2024-09-17T02:43:45.666157+0000","last_active":"2024-09-17T02:43:47.647995+0000","last_peered":"2024-09-17T02:43:47.647995+0000","last_clean":"2024-09-17T02:43:47.647995+0000","last_became_active":"2024-09-17T02:43:45.665300+0000","last_became_peered":"2024-09-17T02:43:45.665300+0000","last_unstale":"2024-09-17T02:43:47.647995+0000","last_undegraded":"2024-09-17T02:43:47.647995+0000","last_fullsized":"2024-09-17T02:43:47.647995+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:41.076834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651046+0000","last_change":"2024-09-17T02:43:46.697615+0000","last_active":"2024-09-17T02:43:47.651046+0000","last_peered":"2024-09-17T02:43:47.651046+0000","last_clean":"2024-09-17T02:43:47.651046+0000","last_became_active":"2024-09-17T02:43:46.697182+0000","last_became_peered":"2024-09-17T02:43:46.697182+0000","last_unstale":"2024-09-17T02:43:47.651046+0000","last_undegraded":"2024-09-17T02:43:47.651046+0000","last_fullsized":"2024-09-17T02:43:47.651046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:48:05.414527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691924+0000","last_change":"2024-09-17T02:43:45.687236+0000","last_active":"2024-09-17T02:43:45.691924+0000","last_peered":"2024-09-17T02:43:45.691924+0000","last_clean":"2024-09-17T02:43:45.691924+0000","last_became_active":"2024-09-17T02:43:45.685471+0000","last_became_peered":"2024-09-17T02:43:45.685471+0000","last_unstale":"2024-09-17T02:43:45.691924+0000","last_undegraded":"2024-09-17T02:43:45.691924+0000","last_fullsized":"2024-09-17T02:43:45.691924+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:39.927955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646618+0000","last_change":"2024-09-17T02:43:45.646618+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646618+0000","last_undegraded":"2024-09-17T02:43:45.646618+0000","last_fullsized":"2024-09-17T02:43:45.646618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651487+0000","last_change":"2024-09-17T02:43:45.661349+0000","last_active":"2024-09-17T02:43:47.651487+0000","last_peered":"2024-09-17T02:43:47.651487+0000","last_clean":"2024-09-17T02:43:47.651487+0000","last_became_active":"2024-09-17T02:43:45.660480+0000","last_became_peered":"2024-09-17T02:43:45.660480+0000","last_unstale":"2024-09-17T02:43:47.651487+0000","last_undegraded":"2024-09-17T02:43:47.651487+0000","last_fullsized":"2024-09-17T02:43:47.651487+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:22.763370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647445+0000","last_change":"2024-09-17T02:43:46.694871+0000","last_active":"2024-09-17T02:43:47.647445+0000","last_peered":"2024-09-17T02:43:47.647445+0000","last_clean":"2024-09-17T02:43:47.647445+0000","last_became_active":"2024-09-17T02:43:46.694495+0000","last_became_peered":"2024-09-17T02:43:46.694495+0000","last_unstale":"2024-09-17T02:43:47.647445+0000","last_undegraded":"2024-09-17T02:43:47.647445+0000","last_fullsized":"2024-09-17T02:43:47.647445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:01.223420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700581+0000","last_change":"2024-09-17T02:43:45.655934+0000","last_active":"2024-09-17T02:43:47.700581+0000","last_peered":"2024-09-17T02:43:47.700581+0000","last_clean":"2024-09-17T02:43:47.700581+0000","last_became_active":"2024-09-17T02:43:45.649802+0000","last_became_peered":"2024-09-17T02:43:45.649802+0000","last_unstale":"2024-09-17T02:43:47.700581+0000","last_undegraded":"2024-09-17T02:43:47.700581+0000","last_fullsized":"2024-09-17T02:43:47.700581+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:24.816574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652445+0000","last_change":"2024-09-17T02:43:46.677885+0000","last_active":"2024-09-17T02:43:47.652445+0000","last_peered":"2024-09-17T02:43:47.652445+0000","last_clean":"2024-09-17T02:43:47.652445+0000","last_became_active":"2024-09-17T02:43:46.677091+0000","last_became_peered":"2024-09-17T02:43:46.677091+0000","last_unstale":"2024-09-17T02:43:47.652445+0000","last_undegraded":"2024-09-17T02:43:47.652445+0000","last_fullsized":"2024-09-17T02:43:47.652445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:59.608768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703726+0000","last_change":"2024-09-17T02:43:45.684007+0000","last_active":"2024-09-17T02:43:47.703726+0000","last_peered":"2024-09-17T02:43:47.703726+0000","last_clean":"2024-09-17T02:43:47.703726+0000","last_became_active":"2024-09-17T02:43:45.683525+0000","last_became_peered":"2024-09-17T02:43:45.683525+0000","last_unstale":"2024-09-17T02:43:47.703726+0000","last_undegraded":"2024-09-17T02:43:47.703726+0000","last_fullsized":"2024-09-17T02:43:47.703726+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:41.128475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647829+0000","last_change":"2024-09-17T02:43:46.668355+0000","last_active":"2024-09-17T02:43:47.647829+0000","last_peered":"2024-09-17T02:43:47.647829+0000","last_clean":"2024-09-17T02:43:47.647829+0000","last_became_active":"2024-09-17T02:43:46.667493+0000","last_became_peered":"2024-09-17T02:43:46.667493+0000","last_unstale":"2024-09-17T02:43:47.647829+0000","last_undegraded":"2024-09-17T02:43:47.647829+0000","last_fullsized":"2024-09-17T02:43:47.647829+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:11.487744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.667876+0000","last_change":"2024-09-17T02:43:45.667054+0000","last_active":"2024-09-17T02:43:45.667876+0000","last_peered":"2024-09-17T02:43:45.667876+0000","last_clean":"2024-09-17T02:43:45.667876+0000","last_became_active":"2024-09-17T02:43:45.666184+0000","last_became_peered":"2024-09-17T02:43:45.666184+0000","last_unstale":"2024-09-17T02:43:45.667876+0000","last_undegraded":"2024-09-17T02:43:45.667876+0000","last_fullsized":"2024-09-17T02:43:45.667876+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:33.813934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.654576+0000","last_change":"2024-09-17T02:43:45.654576+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.654576+0000","last_undegraded":"2024-09-17T02:43:45.654576+0000","last_fullsized":"2024-09-17T02:43:45.654576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.687040+0000","last_change":"2024-09-17T02:43:45.679687+0000","last_active":"2024-09-17T02:43:45.687040+0000","last_peered":"2024-09-17T02:43:45.687040+0000","last_clean":"2024-09-17T02:43:45.687040+0000","last_became_active":"2024-09-17T02:43:45.679104+0000","last_became_peered":"2024-09-17T02:43:45.679104+0000","last_unstale":"2024-09-17T02:43:45.687040+0000","last_undegraded":"2024-09-17T02:43:45.687040+0000","last_fullsized":"2024-09-17T02:43:45.687040+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:30.134634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653809+0000","last_change":"2024-09-17T02:43:45.653809+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653809+0000","last_undegraded":"2024-09-17T02:43:45.653809+0000","last_fullsized":"2024-09-17T02:43:45.653809+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.694452+0000","last_change":"2024-09-17T02:43:45.692080+0000","last_active":"2024-09-17T02:43:45.694452+0000","last_peered":"2024-09-17T02:43:45.694452+0000","last_clean":"2024-09-17T02:43:45.694452+0000","last_became_active":"2024-09-17T02:43:45.690832+0000","last_became_peered":"2024-09-17T02:43:45.690832+0000","last_unstale":"2024-09-17T02:43:45.694452+0000","last_undegraded":"2024-09-17T02:43:45.694452+0000","last_fullsized":"2024-09-17T02:43:45.694452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:00.297892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.646761+0000","last_change":"2024-09-17T02:43:45.646761+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.646761+0000","last_undegraded":"2024-09-17T02:43:45.646761+0000","last_fullsized":"2024-09-17T02:43:45.646761+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701443+0000","last_change":"2024-09-17T02:43:45.651787+0000","last_active":"2024-09-17T02:43:47.701443+0000","last_peered":"2024-09-17T02:43:47.701443+0000","last_clean":"2024-09-17T02:43:47.701443+0000","last_became_active":"2024-09-17T02:43:45.649506+0000","last_became_peered":"2024-09-17T02:43:45.649506+0000","last_unstale":"2024-09-17T02:43:47.701443+0000","last_undegraded":"2024-09-17T02:43:47.701443+0000","last_fullsized":"2024-09-17T02:43:47.701443+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:39.203265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.648211+0000","last_change":"2024-09-17T02:43:45.648211+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.648211+0000","last_undegraded":"2024-09-17T02:43:45.648211+0000","last_fullsized":"2024-09-17T02:43:45.648211+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.664021+0000","last_change":"2024-09-17T02:43:45.661781+0000","last_active":"2024-09-17T02:43:45.664021+0000","last_peered":"2024-09-17T02:43:45.664021+0000","last_clean":"2024-09-17T02:43:45.664021+0000","last_became_active":"2024-09-17T02:43:45.661416+0000","last_became_peered":"2024-09-17T02:43:45.661416+0000","last_unstale":"2024-09-17T02:43:45.664021+0000","last_undegraded":"2024-09-17T02:43:45.664021+0000","last_fullsized":"2024-09-17T02:43:45.664021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:47:57.140547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.645997+0000","last_change":"2024-09-17T02:43:45.645997+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.645997+0000","last_undegraded":"2024-09-17T02:43:45.645997+0000","last_fullsized":"2024-09-17T02:43:45.645997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.670177+0000","last_change":"2024-09-17T02:43:45.662327+0000","last_active":"2024-09-17T02:43:45.670177+0000","last_peered":"2024-09-17T02:43:45.670177+0000","last_clean":"2024-09-17T02:43:45.670177+0000","last_became_active":"2024-09-17T02:43:45.661481+0000","last_became_peered":"2024-09-17T02:43:45.661481+0000","last_unstale":"2024-09-17T02:43:45.670177+0000","last_undegraded":"2024-09-17T02:43:45.670177+0000","last_fullsized":"2024-09-17T02:43:45.670177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:59.435650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651896+0000","last_change":"2024-09-17T02:43:46.676027+0000","last_active":"2024-09-17T02:43:47.651896+0000","last_peered":"2024-09-17T02:43:47.651896+0000","last_clean":"2024-09-17T02:43:47.651896+0000","last_became_active":"2024-09-17T02:43:46.675175+0000","last_became_peered":"2024-09-17T02:43:46.675175+0000","last_unstale":"2024-09-17T02:43:47.651896+0000","last_undegraded":"2024-09-17T02:43:47.651896+0000","last_fullsized":"2024-09-17T02:43:47.651896+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:54.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.657384+0000","last_change":"2024-09-17T02:43:45.650314+0000","last_active":"2024-09-17T02:43:45.657384+0000","last_peered":"2024-09-17T02:43:45.657384+0000","last_clean":"2024-09-17T02:43:45.657384+0000","last_became_active":"2024-09-17T02:43:45.649675+0000","last_became_peered":"2024-09-17T02:43:45.649675+0000","last_unstale":"2024-09-17T02:43:45.657384+0000","last_undegraded":"2024-09-17T02:43:45.657384+0000","last_fullsized":"2024-09-17T02:43:45.657384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:53.908737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702763+0000","last_change":"2024-09-17T02:43:46.707370+0000","last_active":"2024-09-17T02:43:47.702763+0000","last_peered":"2024-09-17T02:43:47.702763+0000","last_clean":"2024-09-17T02:43:47.702763+0000","last_became_active":"2024-09-17T02:43:46.697031+0000","last_became_peered":"2024-09-17T02:43:46.697031+0000","last_unstale":"2024-09-17T02:43:47.702763+0000","last_undegraded":"2024-09-17T02:43:47.702763+0000","last_fullsized":"2024-09-17T02:43:47.702763+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:53.627837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703754+0000","last_change":"2024-09-17T02:43:45.683317+0000","last_active":"2024-09-17T02:43:47.703754+0000","last_peered":"2024-09-17T02:43:47.703754+0000","last_clean":"2024-09-17T02:43:47.703754+0000","last_became_active":"2024-09-17T02:43:45.682686+0000","last_became_peered":"2024-09-17T02:43:45.682686+0000","last_unstale":"2024-09-17T02:43:47.703754+0000","last_undegraded":"2024-09-17T02:43:47.703754+0000","last_fullsized":"2024-09-17T02:43:47.703754+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:45.603856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652497+0000","last_change":"2024-09-17T02:43:45.652497+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652497+0000","last_undegraded":"2024-09-17T02:43:45.652497+0000","last_fullsized":"2024-09-17T02:43:45.652497+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702089+0000","last_change":"2024-09-17T02:43:45.675971+0000","last_active":"2024-09-17T02:43:47.702089+0000","last_peered":"2024-09-17T02:43:47.702089+0000","last_clean":"2024-09-17T02:43:47.702089+0000","last_became_active":"2024-09-17T02:43:45.665279+0000","last_became_peered":"2024-09-17T02:43:45.665279+0000","last_unstale":"2024-09-17T02:43:47.702089+0000","last_undegraded":"2024-09-17T02:43:47.702089+0000","last_fullsized":"2024-09-17T02:43:47.702089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:02.887437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.653757+0000","last_change":"2024-09-17T02:43:45.653757+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.653757+0000","last_undegraded":"2024-09-17T02:43:45.653757+0000","last_fullsized":"2024-09-17T02:43:45.653757+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.673123+0000","last_change":"2024-09-17T02:43:45.669829+0000","last_active":"2024-09-17T02:43:45.673123+0000","last_peered":"2024-09-17T02:43:45.673123+0000","last_clean":"2024-09-17T02:43:45.673123+0000","last_became_active":"2024-09-17T02:43:45.669491+0000","last_became_peered":"2024-09-17T02:43:45.669491+0000","last_unstale":"2024-09-17T02:43:45.673123+0000","last_undegraded":"2024-09-17T02:43:45.673123+0000","last_fullsized":"2024-09-17T02:43:45.673123+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:33.777991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652610+0000","last_change":"2024-09-17T02:43:45.652610+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652610+0000","last_undegraded":"2024-09-17T02:43:45.652610+0000","last_fullsized":"2024-09-17T02:43:45.652610+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707366+0000","last_change":"2024-09-17T02:43:45.674406+0000","last_active":"2024-09-17T02:43:47.707366+0000","last_peered":"2024-09-17T02:43:47.707366+0000","last_clean":"2024-09-17T02:43:47.707366+0000","last_became_active":"2024-09-17T02:43:45.671107+0000","last_became_peered":"2024-09-17T02:43:45.671107+0000","last_unstale":"2024-09-17T02:43:47.707366+0000","last_undegraded":"2024-09-17T02:43:47.707366+0000","last_fullsized":"2024-09-17T02:43:47.707366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:13.407925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.675296+0000","last_change":"2024-09-17T02:43:45.675296+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.675296+0000","last_undegraded":"2024-09-17T02:43:45.675296+0000","last_fullsized":"2024-09-17T02:43:45.675296+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.683619+0000","last_change":"2024-09-17T02:43:45.680606+0000","last_active":"2024-09-17T02:43:45.683619+0000","last_peered":"2024-09-17T02:43:45.683619+0000","last_clean":"2024-09-17T02:43:45.683619+0000","last_became_active":"2024-09-17T02:43:45.679640+0000","last_became_peered":"2024-09-17T02:43:45.679640+0000","last_unstale":"2024-09-17T02:43:45.683619+0000","last_undegraded":"2024-09-17T02:43:45.683619+0000","last_fullsized":"2024-09-17T02:43:45.683619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:24.412203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.652569+0000","last_change":"2024-09-17T02:43:45.652569+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.652569+0000","last_undegraded":"2024-09-17T02:43:45.652569+0000","last_fullsized":"2024-09-17T02:43:45.652569+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.667071+0000","last_change":"2024-09-17T02:43:45.664123+0000","last_active":"2024-09-17T02:43:45.667071+0000","last_peered":"2024-09-17T02:43:45.667071+0000","last_clean":"2024-09-17T02:43:45.667071+0000","last_became_active":"2024-09-17T02:43:45.663776+0000","last_became_peered":"2024-09-17T02:43:45.663776+0000","last_unstale":"2024-09-17T02:43:45.667071+0000","last_undegraded":"2024-09-17T02:43:45.667071+0000","last_fullsized":"2024-09-17T02:43:45.667071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:21.659777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700904+0000","last_change":"2024-09-17T02:43:46.693857+0000","last_active":"2024-09-17T02:43:47.700904+0000","last_peered":"2024-09-17T02:43:47.700904+0000","last_clean":"2024-09-17T02:43:47.700904+0000","last_became_active":"2024-09-17T02:43:46.692225+0000","last_became_peered":"2024-09-17T02:43:46.692225+0000","last_unstale":"2024-09-17T02:43:47.700904+0000","last_undegraded":"2024-09-17T02:43:47.700904+0000","last_fullsized":"2024-09-17T02:43:47.700904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:58:56.128166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691634+0000","last_change":"2024-09-17T02:43:45.684484+0000","last_active":"2024-09-17T02:43:45.691634+0000","last_peered":"2024-09-17T02:43:45.691634+0000","last_clean":"2024-09-17T02:43:45.691634+0000","last_became_active":"2024-09-17T02:43:45.683639+0000","last_became_peered":"2024-09-17T02:43:45.683639+0000","last_unstale":"2024-09-17T02:43:45.691634+0000","last_undegraded":"2024-09-17T02:43:45.691634+0000","last_fullsized":"2024-09-17T02:43:45.691634+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:52:34.106052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.648456+0000","last_change":"2024-09-17T02:43:46.674643+0000","last_active":"2024-09-17T02:43:47.648456+0000","last_peered":"2024-09-17T02:43:47.648456+0000","last_clean":"2024-09-17T02:43:47.648456+0000","last_became_active":"2024-09-17T02:43:46.673681+0000","last_became_peered":"2024-09-17T02:43:46.673681+0000","last_unstale":"2024-09-17T02:43:47.648456+0000","last_undegraded":"2024-09-17T02:43:47.648456+0000","last_fullsized":"2024-09-17T02:43:47.648456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:09.748085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703860+0000","last_change":"2024-09-17T02:43:45.647880+0000","last_active":"2024-09-17T02:43:47.703860+0000","last_peered":"2024-09-17T02:43:47.703860+0000","last_clean":"2024-09-17T02:43:47.703860+0000","last_became_active":"2024-09-17T02:43:45.647466+0000","last_became_peered":"2024-09-17T02:43:45.647466+0000","last_unstale":"2024-09-17T02:43:47.703860+0000","last_undegraded":"2024-09-17T02:43:47.703860+0000","last_fullsized":"2024-09-17T02:43:47.703860+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:19.936068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649200+0000","last_change":"2024-09-17T02:43:46.674511+0000","last_active":"2024-09-17T02:43:47.649200+0000","last_peered":"2024-09-17T02:43:47.649200+0000","last_clean":"2024-09-17T02:43:47.649200+0000","last_became_active":"2024-09-17T02:43:46.673628+0000","last_became_peered":"2024-09-17T02:43:46.673628+0000","last_unstale":"2024-09-17T02:43:47.649200+0000","last_undegraded":"2024-09-17T02:43:47.649200+0000","last_fullsized":"2024-09-17T02:43:47.649200+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:44.828816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"65'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702986+0000","last_change":"2024-09-17T02:43:45.676827+0000","last_active":"2024-09-17T02:43:47.702986+0000","last_peered":"2024-09-17T02:43:47.702986+0000","last_clean":"2024-09-17T02:43:47.702986+0000","last_became_active":"2024-09-17T02:43:45.665485+0000","last_became_peered":"2024-09-17T02:43:45.665485+0000","last_unstale":"2024-09-17T02:43:47.702986+0000","last_undegraded":"2024-09-17T02:43:47.702986+0000","last_fullsized":"2024-09-17T02:43:47.702986+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:17.346724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.649260+0000","last_change":"2024-09-17T02:43:45.649260+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.649260+0000","last_undegraded":"2024-09-17T02:43:45.649260+0000","last_fullsized":"2024-09-17T02:43:45.649260+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691707+0000","last_change":"2024-09-17T02:43:45.685852+0000","last_active":"2024-09-17T02:43:45.691707+0000","last_peered":"2024-09-17T02:43:45.691707+0000","last_clean":"2024-09-17T02:43:45.691707+0000","last_became_active":"2024-09-17T02:43:45.683659+0000","last_became_peered":"2024-09-17T02:43:45.683659+0000","last_unstale":"2024-09-17T02:43:45.691707+0000","last_undegraded":"2024-09-17T02:43:45.691707+0000","last_fullsized":"2024-09-17T02:43:45.691707+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:51.724553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651328+0000","last_change":"2024-09-17T02:43:45.651328+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651328+0000","last_undegraded":"2024-09-17T02:43:45.651328+0000","last_fullsized":"2024-09-17T02:43:45.651328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707587+0000","last_change":"2024-09-17T02:43:45.675150+0000","last_active":"2024-09-17T02:43:47.707587+0000","last_peered":"2024-09-17T02:43:47.707587+0000","last_clean":"2024-09-17T02:43:47.707587+0000","last_became_active":"2024-09-17T02:43:45.671999+0000","last_became_peered":"2024-09-17T02:43:45.671999+0000","last_unstale":"2024-09-17T02:43:47.707587+0000","last_undegraded":"2024-09-17T02:43:47.707587+0000","last_fullsized":"2024-09-17T02:43:47.707587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:17.815894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700564+0000","last_change":"2024-09-17T02:43:46.698516+0000","last_active":"2024-09-17T02:43:47.700564+0000","last_peered":"2024-09-17T02:43:47.700564+0000","last_clean":"2024-09-17T02:43:47.700564+0000","last_became_active":"2024-09-17T02:43:46.696239+0000","last_became_peered":"2024-09-17T02:43:46.696239+0000","last_unstale":"2024-09-17T02:43:47.700564+0000","last_undegraded":"2024-09-17T02:43:47.700564+0000","last_fullsized":"2024-09-17T02:43:47.700564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:46.407009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.684341+0000","last_change":"2024-09-17T02:43:45.682922+0000","last_active":"2024-09-17T02:43:45.684341+0000","last_peered":"2024-09-17T02:43:45.684341+0000","last_clean":"2024-09-17T02:43:45.684341+0000","last_became_active":"2024-09-17T02:43:45.682556+0000","last_became_peered":"2024-09-17T02:43:45.682556+0000","last_unstale":"2024-09-17T02:43:45.684341+0000","last_undegraded":"2024-09-17T02:43:45.684341+0000","last_fullsized":"2024-09-17T02:43:45.684341+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:17.815894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.651770+0000","last_change":"2024-09-17T02:43:45.651770+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.651770+0000","last_undegraded":"2024-09-17T02:43:45.651770+0000","last_fullsized":"2024-09-17T02:43:45.651770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652374+0000","last_change":"2024-09-17T02:43:45.666290+0000","last_active":"2024-09-17T02:43:47.652374+0000","last_peered":"2024-09-17T02:43:47.652374+0000","last_clean":"2024-09-17T02:43:47.652374+0000","last_became_active":"2024-09-17T02:43:45.665410+0000","last_became_peered":"2024-09-17T02:43:45.665410+0000","last_unstale":"2024-09-17T02:43:47.652374+0000","last_undegraded":"2024-09-17T02:43:47.652374+0000","last_fullsized":"2024-09-17T02:43:47.652374+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:33.788490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.677792+0000","last_change":"2024-09-17T02:43:45.677792+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.677792+0000","last_undegraded":"2024-09-17T02:43:45.677792+0000","last_fullsized":"2024-09-17T02:43:45.677792+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707446+0000","last_change":"2024-09-17T02:43:45.672483+0000","last_active":"2024-09-17T02:43:47.707446+0000","last_peered":"2024-09-17T02:43:47.707446+0000","last_clean":"2024-09-17T02:43:47.707446+0000","last_became_active":"2024-09-17T02:43:45.669710+0000","last_became_peered":"2024-09-17T02:43:45.669710+0000","last_unstale":"2024-09-17T02:43:47.707446+0000","last_undegraded":"2024-09-17T02:43:47.707446+0000","last_fullsized":"2024-09-17T02:43:47.707446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:20.160389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647737+0000","last_change":"2024-09-17T02:43:46.697145+0000","last_active":"2024-09-17T02:43:47.647737+0000","last_peered":"2024-09-17T02:43:47.647737+0000","last_clean":"2024-09-17T02:43:47.647737+0000","last_became_active":"2024-09-17T02:43:46.696508+0000","last_became_peered":"2024-09-17T02:43:46.696508+0000","last_unstale":"2024-09-17T02:43:47.647737+0000","last_undegraded":"2024-09-17T02:43:47.647737+0000","last_fullsized":"2024-09-17T02:43:47.647737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:45.286919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.693830+0000","last_change":"2024-09-17T02:43:45.690426+0000","last_active":"2024-09-17T02:43:45.693830+0000","last_peered":"2024-09-17T02:43:45.693830+0000","last_clean":"2024-09-17T02:43:45.693830+0000","last_became_active":"2024-09-17T02:43:45.689221+0000","last_became_peered":"2024-09-17T02:43:45.689221+0000","last_unstale":"2024-09-17T02:43:45.693830+0000","last_undegraded":"2024-09-17T02:43:45.693830+0000","last_fullsized":"2024-09-17T02:43:45.693830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:16.815190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2024-09-17T02:43:45.647781+0000","last_change":"2024-09-17T02:43:45.647781+0000","last_active":"2024-09-17T02:43:45.623379+0000","last_peered":"2024-09-17T02:43:45.623379+0000","last_clean":"2024-09-17T02:43:45.623379+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:43:45.647781+0000","last_undegraded":"2024-09-17T02:43:45.647781+0000","last_fullsized":"2024-09-17T02:43:45.647781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.691983+0000","last_change":"2024-09-17T02:43:45.688114+0000","last_active":"2024-09-17T02:43:45.691983+0000","last_peered":"2024-09-17T02:43:45.691983+0000","last_clean":"2024-09-17T02:43:45.691983+0000","last_became_active":"2024-09-17T02:43:45.687238+0000","last_became_peered":"2024-09-17T02:43:45.687238+0000","last_unstale":"2024-09-17T02:43:45.691983+0000","last_undegraded":"2024-09-17T02:43:45.691983+0000","last_fullsized":"2024-09-17T02:43:45.691983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:07.931860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":187,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-17T02:43:45.645912+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:43:45.645912+0000","last_peered":"2024-09-17T02:43:45.645912+0000","last_clean":"2024-09-17T02:43:45.645912+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:43:45.645912+0000","last_undegraded":"2024-09-17T02:43:45.645912+0000","last_fullsized":"2024-09-17T02:43:45.645912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":778240,"data_stored":349676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1335296,"data_stored":919518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":782336,"data_stored":350258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":741376,"data_stored":329150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":786432,"data_stored":349732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1335296,"data_stored":919518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31932,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339908,"statfs":{"total":96636764160,"available":96604065792,"internally_reserved":0,"allocated":753664,"data_stored":335817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":798720,"data_stored":351940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:43:55.428 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:43:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.591+0000 7f54b8c31640 1 -- 172.21.15.131:0/3536352585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54b40c4c90 msgr2=0x7f54b40c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.591+0000 7f54b8c31640 1 --2- 172.21.15.131:0/3536352585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54b40c4c90 0x7f54b40c50a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f54b4074bb0 tx=0x7f54ac01c770 comp rx=0 tx=0).stop 2024-09-17T02:43:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.591+0000 7f54b8c31640 1 -- 172.21.15.131:0/3536352585 shutdown_connections 2024-09-17T02:43:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.591+0000 7f54b8c31640 1 --2- 172.21.15.131:0/3536352585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54b40c4c90 0x7f54b40c50a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.591+0000 7f54b8c31640 1 --2- 172.21.15.131:0/3536352585 >> v2:172.21.15.189:3300/0 conn(0x7f54b40cec00 0x7f54b40cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.591+0000 7f54b8c31640 1 -- 172.21.15.131:0/3536352585 >> 172.21.15.131:0/3536352585 conn(0x7f54b406eea0 msgr2=0x7f54b406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:55.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.591+0000 7f54b8c31640 1 -- 172.21.15.131:0/3536352585 shutdown_connections 2024-09-17T02:43:55.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.591+0000 7f54b8c31640 1 -- 172.21.15.131:0/3536352585 wait complete. 2024-09-17T02:43:55.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.591+0000 7f54b8c31640 1 Processor -- start 2024-09-17T02:43:55.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.591+0000 7f54b8c31640 1 -- start start 2024-09-17T02:43:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b8c31640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54b406e2c0 0x7f54b414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b8c31640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54b40c4c90 0x7f54b40c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b8c31640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54b40cec00 0x7f54b40c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b8c31640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4169c60 con 0x7f54b406e2c0 2024-09-17T02:43:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b8c31640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f54b4169ae0 con 0x7f54b40cec00 2024-09-17T02:43:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b8c31640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f54b4169de0 con 0x7f54b40c4c90 2024-09-17T02:43:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54b406e2c0 0x7f54b414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54b406e2c0 0x7f54b414eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51356/0 (socket says 172.21.15.131:51356) 2024-09-17T02:43:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b259c640 1 -- 172.21.15.131:0/2399485840 learned_addr learned my addr 172.21.15.131:0/2399485840 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b2d9d640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54b40cec00 0x7f54b40c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b1d9b640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54b40c4c90 0x7f54b40c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b259c640 1 -- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54b40c4c90 msgr2=0x7f54b40c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b259c640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54b40c4c90 0x7f54b40c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b259c640 1 -- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54b40cec00 msgr2=0x7f54b40c5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b259c640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54b40cec00 0x7f54b40c5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b259c640 1 -- 172.21.15.131:0/2399485840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b414f3e0 con 0x7f54b406e2c0 2024-09-17T02:43:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b259c640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54b406e2c0 0x7f54b414eea0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f549c00c400 tx=0x7f549c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54aaffd640 1 -- 172.21.15.131:0/2399485840 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c017070 con 0x7f54b406e2c0 2024-09-17T02:43:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54aaffd640 1 -- 172.21.15.131:0/2399485840 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f549c003600 con 0x7f54b406e2c0 2024-09-17T02:43:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54aaffd640 1 -- 172.21.15.131:0/2399485840 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c003a50 con 0x7f54b406e2c0 2024-09-17T02:43:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b8c31640 1 -- 172.21.15.131:0/2399485840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b415f450 con 0x7f54b406e2c0 2024-09-17T02:43:55.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b8c31640 1 -- 172.21.15.131:0/2399485840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b415fa10 con 0x7f54b406e2c0 2024-09-17T02:43:55.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.595+0000 7f54b8c31640 1 -- 172.21.15.131:0/2399485840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b40cf010 con 0x7f54b406e2c0 2024-09-17T02:43:55.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.599+0000 7f54aaffd640 1 -- 172.21.15.131:0/2399485840 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f549c01a070 con 0x7f54b406e2c0 2024-09-17T02:43:55.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.599+0000 7f54aaffd640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5494073200 0x7f5494075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:55.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.599+0000 7f54aaffd640 1 -- 172.21.15.131:0/2399485840 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f549c0936f0 con 0x7f54b406e2c0 2024-09-17T02:43:55.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.599+0000 7f54b1d9b640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5494073200 0x7f5494075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:55.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.599+0000 7f54b1d9b640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5494073200 0x7f5494075710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005c50 tx=0x7f54a4005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:55.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.603+0000 7f54aaffd640 1 -- 172.21.15.131:0/2399485840 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f549c05e580 con 0x7f54b406e2c0 2024-09-17T02:43:55.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.771+0000 7f54b8c31640 1 -- 172.21.15.131:0/2399485840 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54b415f1a0 con 0x7f5494073200 2024-09-17T02:43:55.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.783+0000 7f54aaffd640 1 -- 172.21.15.131:0/2399485840 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252005 (secure 0 0 0) 0x7f54b415f1a0 con 0x7f5494073200 2024-09-17T02:43:55.787 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:55.789 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:43:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 -- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5494073200 msgr2=0x7f5494075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5494073200 0x7f5494075710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f54a4005c50 tx=0x7f54a4005be0 comp rx=0 tx=0).stop 2024-09-17T02:43:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 -- 172.21.15.131:0/2399485840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54b406e2c0 msgr2=0x7f54b414eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54b406e2c0 0x7f54b414eea0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f549c00c400 tx=0x7f549c00c8c0 comp rx=0 tx=0).stop 2024-09-17T02:43:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 -- 172.21.15.131:0/2399485840 shutdown_connections 2024-09-17T02:43:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5494073200 0x7f5494075710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f54b40cec00 0x7f54b40c5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f54b40c4c90 0x7f54b40c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 --2- 172.21.15.131:0/2399485840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f54b406e2c0 0x7f54b414eea0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:55.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 -- 172.21.15.131:0/2399485840 >> 172.21.15.131:0/2399485840 conn(0x7f54b406eea0 msgr2=0x7f54b4159810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:55.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 -- 172.21.15.131:0/2399485840 shutdown_connections 2024-09-17T02:43:55.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:55.787+0000 7f54b8c31640 1 -- 172.21.15.131:0/2399485840 wait complete. 2024-09-17T02:43:55.809 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-17T02:43:55.188597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":256856,"kb_used_data":7304,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717864,"statfs":{"total":773094113280,"available":772831092736,"internally_reserved":0,"allocated":7479296,"data_stored":3982964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.545940"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652370+0000","last_change":"2024-09-17T02:43:46.680494+0000","last_active":"2024-09-17T02:43:47.652370+0000","last_peered":"2024-09-17T02:43:47.652370+0000","last_clean":"2024-09-17T02:43:47.652370+0000","last_became_active":"2024-09-17T02:43:46.679436+0000","last_became_peered":"2024-09-17T02:43:46.679436+0000","last_unstale":"2024-09-17T02:43:47.652370+0000","last_undegraded":"2024-09-17T02:43:47.652370+0000","last_fullsized":"2024-09-17T02:43:47.652370+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:18.520380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986695+0000","last_change":"2024-09-17T02:43:46.692792+0000","last_active":"2024-09-17T02:43:47.986695+0000","last_peered":"2024-09-17T02:43:47.986695+0000","last_clean":"2024-09-17T02:43:47.986695+0000","last_became_active":"2024-09-17T02:43:46.692355+0000","last_became_peered":"2024-09-17T02:43:46.692355+0000","last_unstale":"2024-09-17T02:43:47.986695+0000","last_undegraded":"2024-09-17T02:43:47.986695+0000","last_fullsized":"2024-09-17T02:43:47.986695+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:42.080955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986865+0000","last_change":"2024-09-17T02:43:46.669168+0000","last_active":"2024-09-17T02:43:47.986865+0000","last_peered":"2024-09-17T02:43:47.986865+0000","last_clean":"2024-09-17T02:43:47.986865+0000","last_became_active":"2024-09-17T02:43:46.668233+0000","last_became_peered":"2024-09-17T02:43:46.668233+0000","last_unstale":"2024-09-17T02:43:47.986865+0000","last_undegraded":"2024-09-17T02:43:47.986865+0000","last_fullsized":"2024-09-17T02:43:47.986865+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:11:21.891298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701851+0000","last_change":"2024-09-17T02:43:46.708732+0000","last_active":"2024-09-17T02:43:47.701851+0000","last_peered":"2024-09-17T02:43:47.701851+0000","last_clean":"2024-09-17T02:43:47.701851+0000","last_became_active":"2024-09-17T02:43:46.706193+0000","last_became_peered":"2024-09-17T02:43:46.706193+0000","last_unstale":"2024-09-17T02:43:47.701851+0000","last_undegraded":"2024-09-17T02:43:47.701851+0000","last_fullsized":"2024-09-17T02:43:47.701851+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:27.991604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.987698+0000","last_change":"2024-09-17T02:43:46.698216+0000","last_active":"2024-09-17T02:43:47.987698+0000","last_peered":"2024-09-17T02:43:47.987698+0000","last_clean":"2024-09-17T02:43:47.987698+0000","last_became_active":"2024-09-17T02:43:46.697857+0000","last_became_peered":"2024-09-17T02:43:46.697857+0000","last_unstale":"2024-09-17T02:43:47.987698+0000","last_undegraded":"2024-09-17T02:43:47.987698+0000","last_fullsized":"2024-09-17T02:43:47.987698+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:06:38.102820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986173+0000","last_change":"2024-09-17T02:43:46.699860+0000","last_active":"2024-09-17T02:43:47.986173+0000","last_peered":"2024-09-17T02:43:47.986173+0000","last_clean":"2024-09-17T02:43:47.986173+0000","last_became_active":"2024-09-17T02:43:46.699460+0000","last_became_peered":"2024-09-17T02:43:46.699460+0000","last_unstale":"2024-09-17T02:43:47.986173+0000","last_undegraded":"2024-09-17T02:43:47.986173+0000","last_fullsized":"2024-09-17T02:43:47.986173+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:27.716342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649842+0000","last_change":"2024-09-17T02:43:46.678701+0000","last_active":"2024-09-17T02:43:47.649842+0000","last_peered":"2024-09-17T02:43:47.649842+0000","last_clean":"2024-09-17T02:43:47.649842+0000","last_became_active":"2024-09-17T02:43:46.674482+0000","last_became_peered":"2024-09-17T02:43:46.674482+0000","last_unstale":"2024-09-17T02:43:47.649842+0000","last_undegraded":"2024-09-17T02:43:47.649842+0000","last_fullsized":"2024-09-17T02:43:47.649842+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:30.882039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.985836+0000","last_change":"2024-09-17T02:43:46.699949+0000","last_active":"2024-09-17T02:43:47.985836+0000","last_peered":"2024-09-17T02:43:47.985836+0000","last_clean":"2024-09-17T02:43:47.985836+0000","last_became_active":"2024-09-17T02:43:46.699332+0000","last_became_peered":"2024-09-17T02:43:46.699332+0000","last_unstale":"2024-09-17T02:43:47.985836+0000","last_undegraded":"2024-09-17T02:43:47.985836+0000","last_fullsized":"2024-09-17T02:43:47.985836+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:55:13.358267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702086+0000","last_change":"2024-09-17T02:43:46.695684+0000","last_active":"2024-09-17T02:43:47.702086+0000","last_peered":"2024-09-17T02:43:47.702086+0000","last_clean":"2024-09-17T02:43:47.702086+0000","last_became_active":"2024-09-17T02:43:46.694860+0000","last_became_peered":"2024-09-17T02:43:46.694860+0000","last_unstale":"2024-09-17T02:43:47.702086+0000","last_undegraded":"2024-09-17T02:43:47.702086+0000","last_fullsized":"2024-09-17T02:43:47.702086+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:25:53.983748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.988636+0000","last_change":"2024-09-17T02:43:46.678967+0000","last_active":"2024-09-17T02:43:47.988636+0000","last_peered":"2024-09-17T02:43:47.988636+0000","last_clean":"2024-09-17T02:43:47.988636+0000","last_became_active":"2024-09-17T02:43:46.674598+0000","last_became_peered":"2024-09-17T02:43:46.674598+0000","last_unstale":"2024-09-17T02:43:47.988636+0000","last_undegraded":"2024-09-17T02:43:47.988636+0000","last_fullsized":"2024-09-17T02:43:47.988636+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:32:30.126731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702375+0000","last_change":"2024-09-17T02:43:46.701807+0000","last_active":"2024-09-17T02:43:47.702375+0000","last_peered":"2024-09-17T02:43:47.702375+0000","last_clean":"2024-09-17T02:43:47.702375+0000","last_became_active":"2024-09-17T02:43:46.696272+0000","last_became_peered":"2024-09-17T02:43:46.696272+0000","last_unstale":"2024-09-17T02:43:47.702375+0000","last_undegraded":"2024-09-17T02:43:47.702375+0000","last_fullsized":"2024-09-17T02:43:47.702375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:47.850100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702109+0000","last_change":"2024-09-17T02:43:46.707994+0000","last_active":"2024-09-17T02:43:47.702109+0000","last_peered":"2024-09-17T02:43:47.702109+0000","last_clean":"2024-09-17T02:43:47.702109+0000","last_became_active":"2024-09-17T02:43:46.701425+0000","last_became_peered":"2024-09-17T02:43:46.701425+0000","last_unstale":"2024-09-17T02:43:47.702109+0000","last_undegraded":"2024-09-17T02:43:47.702109+0000","last_fullsized":"2024-09-17T02:43:47.702109+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:42.080955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.643465+0000","last_change":"2024-09-17T02:43:46.698051+0000","last_active":"2024-09-17T02:43:47.643465+0000","last_peered":"2024-09-17T02:43:47.643465+0000","last_clean":"2024-09-17T02:43:47.643465+0000","last_became_active":"2024-09-17T02:43:46.697470+0000","last_became_peered":"2024-09-17T02:43:46.697470+0000","last_unstale":"2024-09-17T02:43:47.643465+0000","last_undegraded":"2024-09-17T02:43:47.643465+0000","last_fullsized":"2024-09-17T02:43:47.643465+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:41.670445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.643553+0000","last_change":"2024-09-17T02:43:46.676986+0000","last_active":"2024-09-17T02:43:47.643553+0000","last_peered":"2024-09-17T02:43:47.643553+0000","last_clean":"2024-09-17T02:43:47.643553+0000","last_became_active":"2024-09-17T02:43:46.676060+0000","last_became_peered":"2024-09-17T02:43:46.676060+0000","last_unstale":"2024-09-17T02:43:47.643553+0000","last_undegraded":"2024-09-17T02:43:47.643553+0000","last_fullsized":"2024-09-17T02:43:47.643553+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:01.302012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.985273+0000","last_change":"2024-09-17T02:43:46.674818+0000","last_active":"2024-09-17T02:43:47.985273+0000","last_peered":"2024-09-17T02:43:47.985273+0000","last_clean":"2024-09-17T02:43:47.985273+0000","last_became_active":"2024-09-17T02:43:46.673848+0000","last_became_peered":"2024-09-17T02:43:46.673848+0000","last_unstale":"2024-09-17T02:43:47.985273+0000","last_undegraded":"2024-09-17T02:43:47.985273+0000","last_fullsized":"2024-09-17T02:43:47.985273+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:25.820695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707657+0000","last_change":"2024-09-17T02:43:46.708131+0000","last_active":"2024-09-17T02:43:47.707657+0000","last_peered":"2024-09-17T02:43:47.707657+0000","last_clean":"2024-09-17T02:43:47.707657+0000","last_became_active":"2024-09-17T02:43:46.707665+0000","last_became_peered":"2024-09-17T02:43:46.707665+0000","last_unstale":"2024-09-17T02:43:47.707657+0000","last_undegraded":"2024-09-17T02:43:47.707657+0000","last_fullsized":"2024-09-17T02:43:47.707657+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:48:16.207446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.650043+0000","last_change":"2024-09-17T02:43:46.702269+0000","last_active":"2024-09-17T02:43:47.650043+0000","last_peered":"2024-09-17T02:43:47.650043+0000","last_clean":"2024-09-17T02:43:47.650043+0000","last_became_active":"2024-09-17T02:43:46.701883+0000","last_became_peered":"2024-09-17T02:43:46.701883+0000","last_unstale":"2024-09-17T02:43:47.650043+0000","last_undegraded":"2024-09-17T02:43:47.650043+0000","last_fullsized":"2024-09-17T02:43:47.650043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:22.975866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702426+0000","last_change":"2024-09-17T02:43:46.693189+0000","last_active":"2024-09-17T02:43:47.702426+0000","last_peered":"2024-09-17T02:43:47.702426+0000","last_clean":"2024-09-17T02:43:47.702426+0000","last_became_active":"2024-09-17T02:43:46.691164+0000","last_became_peered":"2024-09-17T02:43:46.691164+0000","last_unstale":"2024-09-17T02:43:47.702426+0000","last_undegraded":"2024-09-17T02:43:47.702426+0000","last_fullsized":"2024-09-17T02:43:47.702426+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:52.980125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702146+0000","last_change":"2024-09-17T02:43:46.692645+0000","last_active":"2024-09-17T02:43:47.702146+0000","last_peered":"2024-09-17T02:43:47.702146+0000","last_clean":"2024-09-17T02:43:47.702146+0000","last_became_active":"2024-09-17T02:43:46.691861+0000","last_became_peered":"2024-09-17T02:43:46.691861+0000","last_unstale":"2024-09-17T02:43:47.702146+0000","last_undegraded":"2024-09-17T02:43:47.702146+0000","last_fullsized":"2024-09-17T02:43:47.702146+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:03.405001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647781+0000","last_change":"2024-09-17T02:43:46.684848+0000","last_active":"2024-09-17T02:43:47.647781+0000","last_peered":"2024-09-17T02:43:47.647781+0000","last_clean":"2024-09-17T02:43:47.647781+0000","last_became_active":"2024-09-17T02:43:46.683919+0000","last_became_peered":"2024-09-17T02:43:46.683919+0000","last_unstale":"2024-09-17T02:43:47.647781+0000","last_undegraded":"2024-09-17T02:43:47.647781+0000","last_fullsized":"2024-09-17T02:43:47.647781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:58.516266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.643484+0000","last_change":"2024-09-17T02:43:46.688265+0000","last_active":"2024-09-17T02:43:47.643484+0000","last_peered":"2024-09-17T02:43:47.643484+0000","last_clean":"2024-09-17T02:43:47.643484+0000","last_became_active":"2024-09-17T02:43:46.687374+0000","last_became_peered":"2024-09-17T02:43:46.687374+0000","last_unstale":"2024-09-17T02:43:47.643484+0000","last_undegraded":"2024-09-17T02:43:47.643484+0000","last_fullsized":"2024-09-17T02:43:47.643484+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:01:17.819311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.987073+0000","last_change":"2024-09-17T02:43:46.699575+0000","last_active":"2024-09-17T02:43:47.987073+0000","last_peered":"2024-09-17T02:43:47.987073+0000","last_clean":"2024-09-17T02:43:47.987073+0000","last_became_active":"2024-09-17T02:43:46.699248+0000","last_became_peered":"2024-09-17T02:43:46.699248+0000","last_unstale":"2024-09-17T02:43:47.987073+0000","last_undegraded":"2024-09-17T02:43:47.987073+0000","last_fullsized":"2024-09-17T02:43:47.987073+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:52.980125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707376+0000","last_change":"2024-09-17T02:43:46.708531+0000","last_active":"2024-09-17T02:43:47.707376+0000","last_peered":"2024-09-17T02:43:47.707376+0000","last_clean":"2024-09-17T02:43:47.707376+0000","last_became_active":"2024-09-17T02:43:46.708224+0000","last_became_peered":"2024-09-17T02:43:46.708224+0000","last_unstale":"2024-09-17T02:43:47.707376+0000","last_undegraded":"2024-09-17T02:43:47.707376+0000","last_fullsized":"2024-09-17T02:43:47.707376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:19:37.303173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986484+0000","last_change":"2024-09-17T02:43:46.693777+0000","last_active":"2024-09-17T02:43:47.986484+0000","last_peered":"2024-09-17T02:43:47.986484+0000","last_clean":"2024-09-17T02:43:47.986484+0000","last_became_active":"2024-09-17T02:43:46.693400+0000","last_became_peered":"2024-09-17T02:43:46.693400+0000","last_unstale":"2024-09-17T02:43:47.986484+0000","last_undegraded":"2024-09-17T02:43:47.986484+0000","last_fullsized":"2024-09-17T02:43:47.986484+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:22:25.233450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.643938+0000","last_change":"2024-09-17T02:43:46.694506+0000","last_active":"2024-09-17T02:43:47.643938+0000","last_peered":"2024-09-17T02:43:47.643938+0000","last_clean":"2024-09-17T02:43:47.643938+0000","last_became_active":"2024-09-17T02:43:46.688542+0000","last_became_peered":"2024-09-17T02:43:46.688542+0000","last_unstale":"2024-09-17T02:43:47.643938+0000","last_undegraded":"2024-09-17T02:43:47.643938+0000","last_fullsized":"2024-09-17T02:43:47.643938+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:51.392525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.987328+0000","last_change":"2024-09-17T02:43:46.657540+0000","last_active":"2024-09-17T02:43:47.987328+0000","last_peered":"2024-09-17T02:43:47.987328+0000","last_clean":"2024-09-17T02:43:47.987328+0000","last_became_active":"2024-09-17T02:43:46.656685+0000","last_became_peered":"2024-09-17T02:43:46.656685+0000","last_unstale":"2024-09-17T02:43:47.987328+0000","last_undegraded":"2024-09-17T02:43:47.987328+0000","last_fullsized":"2024-09-17T02:43:47.987328+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:38:09.940437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.643816+0000","last_change":"2024-09-17T02:43:46.685286+0000","last_active":"2024-09-17T02:43:47.643816+0000","last_peered":"2024-09-17T02:43:47.643816+0000","last_clean":"2024-09-17T02:43:47.643816+0000","last_became_active":"2024-09-17T02:43:46.684501+0000","last_became_peered":"2024-09-17T02:43:46.684501+0000","last_unstale":"2024-09-17T02:43:47.643816+0000","last_undegraded":"2024-09-17T02:43:47.643816+0000","last_fullsized":"2024-09-17T02:43:47.643816+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:20.940189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652393+0000","last_change":"2024-09-17T02:43:46.713535+0000","last_active":"2024-09-17T02:43:47.652393+0000","last_peered":"2024-09-17T02:43:47.652393+0000","last_clean":"2024-09-17T02:43:47.652393+0000","last_became_active":"2024-09-17T02:43:46.712681+0000","last_became_peered":"2024-09-17T02:43:46.712681+0000","last_unstale":"2024-09-17T02:43:47.652393+0000","last_undegraded":"2024-09-17T02:43:47.652393+0000","last_fullsized":"2024-09-17T02:43:47.652393+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:21.464657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701453+0000","last_change":"2024-09-17T02:43:46.706426+0000","last_active":"2024-09-17T02:43:47.701453+0000","last_peered":"2024-09-17T02:43:47.701453+0000","last_clean":"2024-09-17T02:43:47.701453+0000","last_became_active":"2024-09-17T02:43:46.706046+0000","last_became_peered":"2024-09-17T02:43:46.706046+0000","last_unstale":"2024-09-17T02:43:47.701453+0000","last_undegraded":"2024-09-17T02:43:47.701453+0000","last_fullsized":"2024-09-17T02:43:47.701453+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:40:58.594878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700841+0000","last_change":"2024-09-17T02:43:46.702605+0000","last_active":"2024-09-17T02:43:47.700841+0000","last_peered":"2024-09-17T02:43:47.700841+0000","last_clean":"2024-09-17T02:43:47.700841+0000","last_became_active":"2024-09-17T02:43:46.702182+0000","last_became_peered":"2024-09-17T02:43:46.702182+0000","last_unstale":"2024-09-17T02:43:47.700841+0000","last_undegraded":"2024-09-17T02:43:47.700841+0000","last_fullsized":"2024-09-17T02:43:47.700841+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:17.795947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.985849+0000","last_change":"2024-09-17T02:43:46.678388+0000","last_active":"2024-09-17T02:43:47.985849+0000","last_peered":"2024-09-17T02:43:47.985849+0000","last_clean":"2024-09-17T02:43:47.985849+0000","last_became_active":"2024-09-17T02:43:46.677507+0000","last_became_peered":"2024-09-17T02:43:46.677507+0000","last_unstale":"2024-09-17T02:43:47.985849+0000","last_undegraded":"2024-09-17T02:43:47.985849+0000","last_fullsized":"2024-09-17T02:43:47.985849+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:34:43.476989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649781+0000","last_change":"2024-09-17T02:43:46.673631+0000","last_active":"2024-09-17T02:43:47.649781+0000","last_peered":"2024-09-17T02:43:47.649781+0000","last_clean":"2024-09-17T02:43:47.649781+0000","last_became_active":"2024-09-17T02:43:46.672779+0000","last_became_peered":"2024-09-17T02:43:46.672779+0000","last_unstale":"2024-09-17T02:43:47.649781+0000","last_undegraded":"2024-09-17T02:43:47.649781+0000","last_fullsized":"2024-09-17T02:43:47.649781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:04.209405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702521+0000","last_change":"2024-09-17T02:43:46.710347+0000","last_active":"2024-09-17T02:43:47.702521+0000","last_peered":"2024-09-17T02:43:47.702521+0000","last_clean":"2024-09-17T02:43:47.702521+0000","last_became_active":"2024-09-17T02:43:46.708415+0000","last_became_peered":"2024-09-17T02:43:46.708415+0000","last_unstale":"2024-09-17T02:43:47.702521+0000","last_undegraded":"2024-09-17T02:43:47.702521+0000","last_fullsized":"2024-09-17T02:43:47.702521+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:23.928100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":19,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652160+0000","last_change":"2024-09-17T02:43:45.678361+0000","last_active":"2024-09-17T02:43:47.652160+0000","last_peered":"2024-09-17T02:43:47.652160+0000","last_clean":"2024-09-17T02:43:47.652160+0000","last_became_active":"2024-09-17T02:43:45.677598+0000","last_became_peered":"2024-09-17T02:43:45.677598+0000","last_unstale":"2024-09-17T02:43:47.652160+0000","last_undegraded":"2024-09-17T02:43:47.652160+0000","last_fullsized":"2024-09-17T02:43:47.652160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:08.498525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.987035+0000","last_change":"2024-09-17T02:43:46.698444+0000","last_active":"2024-09-17T02:43:47.987035+0000","last_peered":"2024-09-17T02:43:47.987035+0000","last_clean":"2024-09-17T02:43:47.987035+0000","last_became_active":"2024-09-17T02:43:46.697653+0000","last_became_peered":"2024-09-17T02:43:46.697653+0000","last_unstale":"2024-09-17T02:43:47.987035+0000","last_undegraded":"2024-09-17T02:43:47.987035+0000","last_fullsized":"2024-09-17T02:43:47.987035+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:06.612395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651944+0000","last_change":"2024-09-17T02:43:45.660427+0000","last_active":"2024-09-17T02:43:47.651944+0000","last_peered":"2024-09-17T02:43:47.651944+0000","last_clean":"2024-09-17T02:43:47.651944+0000","last_became_active":"2024-09-17T02:43:45.659575+0000","last_became_peered":"2024-09-17T02:43:45.659575+0000","last_unstale":"2024-09-17T02:43:47.651944+0000","last_undegraded":"2024-09-17T02:43:47.651944+0000","last_fullsized":"2024-09-17T02:43:47.651944+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:35.225547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.987055+0000","last_change":"2024-09-17T02:43:46.680246+0000","last_active":"2024-09-17T02:43:47.987055+0000","last_peered":"2024-09-17T02:43:47.987055+0000","last_clean":"2024-09-17T02:43:47.987055+0000","last_became_active":"2024-09-17T02:43:46.679204+0000","last_became_peered":"2024-09-17T02:43:46.679204+0000","last_unstale":"2024-09-17T02:43:47.987055+0000","last_undegraded":"2024-09-17T02:43:47.987055+0000","last_fullsized":"2024-09-17T02:43:47.987055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:01:03.424029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652828+0000","last_change":"2024-09-17T02:43:45.665351+0000","last_active":"2024-09-17T02:43:47.652828+0000","last_peered":"2024-09-17T02:43:47.652828+0000","last_clean":"2024-09-17T02:43:47.652828+0000","last_became_active":"2024-09-17T02:43:45.664498+0000","last_became_peered":"2024-09-17T02:43:45.664498+0000","last_unstale":"2024-09-17T02:43:47.652828+0000","last_undegraded":"2024-09-17T02:43:47.652828+0000","last_fullsized":"2024-09-17T02:43:47.652828+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:55:35.011302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703229+0000","last_change":"2024-09-17T02:43:46.694855+0000","last_active":"2024-09-17T02:43:47.703229+0000","last_peered":"2024-09-17T02:43:47.703229+0000","last_clean":"2024-09-17T02:43:47.703229+0000","last_became_active":"2024-09-17T02:43:46.692123+0000","last_became_peered":"2024-09-17T02:43:46.692123+0000","last_unstale":"2024-09-17T02:43:47.703229+0000","last_undegraded":"2024-09-17T02:43:47.703229+0000","last_fullsized":"2024-09-17T02:43:47.703229+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:38.594076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986580+0000","last_change":"2024-09-17T02:43:45.690176+0000","last_active":"2024-09-17T02:43:47.986580+0000","last_peered":"2024-09-17T02:43:47.986580+0000","last_clean":"2024-09-17T02:43:47.986580+0000","last_became_active":"2024-09-17T02:43:45.689355+0000","last_became_peered":"2024-09-17T02:43:45.689355+0000","last_unstale":"2024-09-17T02:43:47.986580+0000","last_undegraded":"2024-09-17T02:43:47.986580+0000","last_fullsized":"2024-09-17T02:43:47.986580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:15.567234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.646844+0000","last_change":"2024-09-17T02:43:46.678861+0000","last_active":"2024-09-17T02:43:47.646844+0000","last_peered":"2024-09-17T02:43:47.646844+0000","last_clean":"2024-09-17T02:43:47.646844+0000","last_became_active":"2024-09-17T02:43:46.677801+0000","last_became_peered":"2024-09-17T02:43:46.677801+0000","last_unstale":"2024-09-17T02:43:47.646844+0000","last_undegraded":"2024-09-17T02:43:47.646844+0000","last_fullsized":"2024-09-17T02:43:47.646844+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:17:31.162139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647995+0000","last_change":"2024-09-17T02:43:45.666157+0000","last_active":"2024-09-17T02:43:47.647995+0000","last_peered":"2024-09-17T02:43:47.647995+0000","last_clean":"2024-09-17T02:43:47.647995+0000","last_became_active":"2024-09-17T02:43:45.665300+0000","last_became_peered":"2024-09-17T02:43:45.665300+0000","last_unstale":"2024-09-17T02:43:47.647995+0000","last_undegraded":"2024-09-17T02:43:47.647995+0000","last_fullsized":"2024-09-17T02:43:47.647995+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:58:41.076834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651046+0000","last_change":"2024-09-17T02:43:46.697615+0000","last_active":"2024-09-17T02:43:47.651046+0000","last_peered":"2024-09-17T02:43:47.651046+0000","last_clean":"2024-09-17T02:43:47.651046+0000","last_became_active":"2024-09-17T02:43:46.697182+0000","last_became_peered":"2024-09-17T02:43:46.697182+0000","last_unstale":"2024-09-17T02:43:47.651046+0000","last_undegraded":"2024-09-17T02:43:47.651046+0000","last_fullsized":"2024-09-17T02:43:47.651046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:48:05.414527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1a","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986979+0000","last_change":"2024-09-17T02:43:45.687236+0000","last_active":"2024-09-17T02:43:47.986979+0000","last_peered":"2024-09-17T02:43:47.986979+0000","last_clean":"2024-09-17T02:43:47.986979+0000","last_became_active":"2024-09-17T02:43:45.685471+0000","last_became_peered":"2024-09-17T02:43:45.685471+0000","last_unstale":"2024-09-17T02:43:47.986979+0000","last_undegraded":"2024-09-17T02:43:47.986979+0000","last_fullsized":"2024-09-17T02:43:47.986979+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:14.696208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986800+0000","last_change":"2024-09-17T02:43:46.685638+0000","last_active":"2024-09-17T02:43:47.986800+0000","last_peered":"2024-09-17T02:43:47.986800+0000","last_clean":"2024-09-17T02:43:47.986800+0000","last_became_active":"2024-09-17T02:43:46.683740+0000","last_became_peered":"2024-09-17T02:43:46.683740+0000","last_unstale":"2024-09-17T02:43:47.986800+0000","last_undegraded":"2024-09-17T02:43:47.986800+0000","last_fullsized":"2024-09-17T02:43:47.986800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:03.530685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651487+0000","last_change":"2024-09-17T02:43:45.661349+0000","last_active":"2024-09-17T02:43:47.651487+0000","last_peered":"2024-09-17T02:43:47.651487+0000","last_clean":"2024-09-17T02:43:47.651487+0000","last_became_active":"2024-09-17T02:43:45.660480+0000","last_became_peered":"2024-09-17T02:43:45.660480+0000","last_unstale":"2024-09-17T02:43:47.651487+0000","last_undegraded":"2024-09-17T02:43:47.651487+0000","last_fullsized":"2024-09-17T02:43:47.651487+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:22:22.763370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647445+0000","last_change":"2024-09-17T02:43:46.694871+0000","last_active":"2024-09-17T02:43:47.647445+0000","last_peered":"2024-09-17T02:43:47.647445+0000","last_clean":"2024-09-17T02:43:47.647445+0000","last_became_active":"2024-09-17T02:43:46.694495+0000","last_became_peered":"2024-09-17T02:43:46.694495+0000","last_unstale":"2024-09-17T02:43:47.647445+0000","last_undegraded":"2024-09-17T02:43:47.647445+0000","last_fullsized":"2024-09-17T02:43:47.647445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:01.223420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700581+0000","last_change":"2024-09-17T02:43:45.655934+0000","last_active":"2024-09-17T02:43:47.700581+0000","last_peered":"2024-09-17T02:43:47.700581+0000","last_clean":"2024-09-17T02:43:47.700581+0000","last_became_active":"2024-09-17T02:43:45.649802+0000","last_became_peered":"2024-09-17T02:43:45.649802+0000","last_unstale":"2024-09-17T02:43:47.700581+0000","last_undegraded":"2024-09-17T02:43:47.700581+0000","last_fullsized":"2024-09-17T02:43:47.700581+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:24.816574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652445+0000","last_change":"2024-09-17T02:43:46.677885+0000","last_active":"2024-09-17T02:43:47.652445+0000","last_peered":"2024-09-17T02:43:47.652445+0000","last_clean":"2024-09-17T02:43:47.652445+0000","last_became_active":"2024-09-17T02:43:46.677091+0000","last_became_peered":"2024-09-17T02:43:46.677091+0000","last_unstale":"2024-09-17T02:43:47.652445+0000","last_undegraded":"2024-09-17T02:43:47.652445+0000","last_fullsized":"2024-09-17T02:43:47.652445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:50:59.608768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703726+0000","last_change":"2024-09-17T02:43:45.684007+0000","last_active":"2024-09-17T02:43:47.703726+0000","last_peered":"2024-09-17T02:43:47.703726+0000","last_clean":"2024-09-17T02:43:47.703726+0000","last_became_active":"2024-09-17T02:43:45.683525+0000","last_became_peered":"2024-09-17T02:43:45.683525+0000","last_unstale":"2024-09-17T02:43:47.703726+0000","last_undegraded":"2024-09-17T02:43:47.703726+0000","last_fullsized":"2024-09-17T02:43:47.703726+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:41.128475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647829+0000","last_change":"2024-09-17T02:43:46.668355+0000","last_active":"2024-09-17T02:43:47.647829+0000","last_peered":"2024-09-17T02:43:47.647829+0000","last_clean":"2024-09-17T02:43:47.647829+0000","last_became_active":"2024-09-17T02:43:46.667493+0000","last_became_peered":"2024-09-17T02:43:46.667493+0000","last_unstale":"2024-09-17T02:43:47.647829+0000","last_undegraded":"2024-09-17T02:43:47.647829+0000","last_fullsized":"2024-09-17T02:43:47.647829+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:11.487744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.644122+0000","last_change":"2024-09-17T02:43:45.667054+0000","last_active":"2024-09-17T02:43:47.644122+0000","last_peered":"2024-09-17T02:43:47.644122+0000","last_clean":"2024-09-17T02:43:47.644122+0000","last_became_active":"2024-09-17T02:43:45.666184+0000","last_became_peered":"2024-09-17T02:43:45.666184+0000","last_unstale":"2024-09-17T02:43:47.644122+0000","last_undegraded":"2024-09-17T02:43:47.644122+0000","last_fullsized":"2024-09-17T02:43:47.644122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:13.407925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.643818+0000","last_change":"2024-09-17T02:43:46.699804+0000","last_active":"2024-09-17T02:43:47.643818+0000","last_peered":"2024-09-17T02:43:47.643818+0000","last_clean":"2024-09-17T02:43:47.643818+0000","last_became_active":"2024-09-17T02:43:46.698917+0000","last_became_peered":"2024-09-17T02:43:46.698917+0000","last_unstale":"2024-09-17T02:43:47.643818+0000","last_undegraded":"2024-09-17T02:43:47.643818+0000","last_fullsized":"2024-09-17T02:43:47.643818+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:45.898641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"64'3","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.987393+0000","last_change":"2024-09-17T02:43:45.679687+0000","last_active":"2024-09-17T02:43:47.987393+0000","last_peered":"2024-09-17T02:43:47.987393+0000","last_clean":"2024-09-17T02:43:47.987393+0000","last_became_active":"2024-09-17T02:43:45.679104+0000","last_became_peered":"2024-09-17T02:43:45.679104+0000","last_unstale":"2024-09-17T02:43:47.987393+0000","last_undegraded":"2024-09-17T02:43:47.987393+0000","last_fullsized":"2024-09-17T02:43:47.987393+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:26.375740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.988205+0000","last_change":"2024-09-17T02:43:46.680203+0000","last_active":"2024-09-17T02:43:47.988205+0000","last_peered":"2024-09-17T02:43:47.988205+0000","last_clean":"2024-09-17T02:43:47.988205+0000","last_became_active":"2024-09-17T02:43:46.679326+0000","last_became_peered":"2024-09-17T02:43:46.679326+0000","last_unstale":"2024-09-17T02:43:47.988205+0000","last_undegraded":"2024-09-17T02:43:47.988205+0000","last_fullsized":"2024-09-17T02:43:47.988205+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:41.459370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"64'6","reported_seq":21,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986906+0000","last_change":"2024-09-17T02:43:45.692080+0000","last_active":"2024-09-17T02:43:47.986906+0000","last_peered":"2024-09-17T02:43:47.986906+0000","last_clean":"2024-09-17T02:43:47.986906+0000","last_became_active":"2024-09-17T02:43:45.690832+0000","last_became_peered":"2024-09-17T02:43:45.690832+0000","last_unstale":"2024-09-17T02:43:47.986906+0000","last_undegraded":"2024-09-17T02:43:47.986906+0000","last_fullsized":"2024-09-17T02:43:47.986906+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:21:06.722764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.985034+0000","last_change":"2024-09-17T02:43:46.698748+0000","last_active":"2024-09-17T02:43:47.985034+0000","last_peered":"2024-09-17T02:43:47.985034+0000","last_clean":"2024-09-17T02:43:47.985034+0000","last_became_active":"2024-09-17T02:43:46.698239+0000","last_became_peered":"2024-09-17T02:43:46.698239+0000","last_unstale":"2024-09-17T02:43:47.985034+0000","last_undegraded":"2024-09-17T02:43:47.985034+0000","last_fullsized":"2024-09-17T02:43:47.985034+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:19:24.052633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.701443+0000","last_change":"2024-09-17T02:43:45.651787+0000","last_active":"2024-09-17T02:43:47.701443+0000","last_peered":"2024-09-17T02:43:47.701443+0000","last_clean":"2024-09-17T02:43:47.701443+0000","last_became_active":"2024-09-17T02:43:45.649506+0000","last_became_peered":"2024-09-17T02:43:45.649506+0000","last_unstale":"2024-09-17T02:43:47.701443+0000","last_undegraded":"2024-09-17T02:43:47.701443+0000","last_fullsized":"2024-09-17T02:43:47.701443+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:39.203265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.985403+0000","last_change":"2024-09-17T02:43:46.670429+0000","last_active":"2024-09-17T02:43:47.985403+0000","last_peered":"2024-09-17T02:43:47.985403+0000","last_clean":"2024-09-17T02:43:47.985403+0000","last_became_active":"2024-09-17T02:43:46.669550+0000","last_became_peered":"2024-09-17T02:43:46.669550+0000","last_unstale":"2024-09-17T02:43:47.985403+0000","last_undegraded":"2024-09-17T02:43:47.985403+0000","last_fullsized":"2024-09-17T02:43:47.985403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:27:40.207386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986335+0000","last_change":"2024-09-17T02:43:45.661781+0000","last_active":"2024-09-17T02:43:47.986335+0000","last_peered":"2024-09-17T02:43:47.986335+0000","last_clean":"2024-09-17T02:43:47.986335+0000","last_became_active":"2024-09-17T02:43:45.661416+0000","last_became_peered":"2024-09-17T02:43:45.661416+0000","last_unstale":"2024-09-17T02:43:47.986335+0000","last_undegraded":"2024-09-17T02:43:47.986335+0000","last_fullsized":"2024-09-17T02:43:47.986335+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:12:32.311858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.985619+0000","last_change":"2024-09-17T02:43:46.684730+0000","last_active":"2024-09-17T02:43:47.985619+0000","last_peered":"2024-09-17T02:43:47.985619+0000","last_clean":"2024-09-17T02:43:47.985619+0000","last_became_active":"2024-09-17T02:43:46.682627+0000","last_became_peered":"2024-09-17T02:43:46.682627+0000","last_unstale":"2024-09-17T02:43:47.985619+0000","last_undegraded":"2024-09-17T02:43:47.985619+0000","last_fullsized":"2024-09-17T02:43:47.985619+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:03.891557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986529+0000","last_change":"2024-09-17T02:43:45.662327+0000","last_active":"2024-09-17T02:43:47.986529+0000","last_peered":"2024-09-17T02:43:47.986529+0000","last_clean":"2024-09-17T02:43:47.986529+0000","last_became_active":"2024-09-17T02:43:45.661481+0000","last_became_peered":"2024-09-17T02:43:45.661481+0000","last_unstale":"2024-09-17T02:43:47.986529+0000","last_undegraded":"2024-09-17T02:43:47.986529+0000","last_fullsized":"2024-09-17T02:43:47.986529+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:57.512145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.651896+0000","last_change":"2024-09-17T02:43:46.676027+0000","last_active":"2024-09-17T02:43:47.651896+0000","last_peered":"2024-09-17T02:43:47.651896+0000","last_clean":"2024-09-17T02:43:47.651896+0000","last_became_active":"2024-09-17T02:43:46.675175+0000","last_became_peered":"2024-09-17T02:43:46.675175+0000","last_unstale":"2024-09-17T02:43:47.651896+0000","last_undegraded":"2024-09-17T02:43:47.651896+0000","last_fullsized":"2024-09-17T02:43:47.651896+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:01:54.280430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"64'8","reported_seq":23,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.984842+0000","last_change":"2024-09-17T02:43:45.650314+0000","last_active":"2024-09-17T02:43:47.984842+0000","last_peered":"2024-09-17T02:43:47.984842+0000","last_clean":"2024-09-17T02:43:47.984842+0000","last_became_active":"2024-09-17T02:43:45.649675+0000","last_became_peered":"2024-09-17T02:43:45.649675+0000","last_unstale":"2024-09-17T02:43:47.984842+0000","last_undegraded":"2024-09-17T02:43:47.984842+0000","last_fullsized":"2024-09-17T02:43:47.984842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:03.788695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702763+0000","last_change":"2024-09-17T02:43:46.707370+0000","last_active":"2024-09-17T02:43:47.702763+0000","last_peered":"2024-09-17T02:43:47.702763+0000","last_clean":"2024-09-17T02:43:47.702763+0000","last_became_active":"2024-09-17T02:43:46.697031+0000","last_became_peered":"2024-09-17T02:43:46.697031+0000","last_unstale":"2024-09-17T02:43:47.702763+0000","last_undegraded":"2024-09-17T02:43:47.702763+0000","last_fullsized":"2024-09-17T02:43:47.702763+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:53.627837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703754+0000","last_change":"2024-09-17T02:43:45.683317+0000","last_active":"2024-09-17T02:43:47.703754+0000","last_peered":"2024-09-17T02:43:47.703754+0000","last_clean":"2024-09-17T02:43:47.703754+0000","last_became_active":"2024-09-17T02:43:45.682686+0000","last_became_peered":"2024-09-17T02:43:45.682686+0000","last_unstale":"2024-09-17T02:43:47.703754+0000","last_undegraded":"2024-09-17T02:43:47.703754+0000","last_fullsized":"2024-09-17T02:43:47.703754+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:14:45.603856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.643816+0000","last_change":"2024-09-17T02:43:46.676172+0000","last_active":"2024-09-17T02:43:47.643816+0000","last_peered":"2024-09-17T02:43:47.643816+0000","last_clean":"2024-09-17T02:43:47.643816+0000","last_became_active":"2024-09-17T02:43:46.673709+0000","last_became_peered":"2024-09-17T02:43:46.673709+0000","last_unstale":"2024-09-17T02:43:47.643816+0000","last_undegraded":"2024-09-17T02:43:47.643816+0000","last_fullsized":"2024-09-17T02:43:47.643816+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:20.534905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702089+0000","last_change":"2024-09-17T02:43:45.675971+0000","last_active":"2024-09-17T02:43:47.702089+0000","last_peered":"2024-09-17T02:43:47.702089+0000","last_clean":"2024-09-17T02:43:47.702089+0000","last_became_active":"2024-09-17T02:43:45.665279+0000","last_became_peered":"2024-09-17T02:43:45.665279+0000","last_unstale":"2024-09-17T02:43:47.702089+0000","last_undegraded":"2024-09-17T02:43:47.702089+0000","last_fullsized":"2024-09-17T02:43:47.702089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:02.887437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.644020+0000","last_change":"2024-09-17T02:43:46.693861+0000","last_active":"2024-09-17T02:43:47.644020+0000","last_peered":"2024-09-17T02:43:47.644020+0000","last_clean":"2024-09-17T02:43:47.644020+0000","last_became_active":"2024-09-17T02:43:46.688553+0000","last_became_peered":"2024-09-17T02:43:46.688553+0000","last_unstale":"2024-09-17T02:43:47.644020+0000","last_undegraded":"2024-09-17T02:43:47.644020+0000","last_fullsized":"2024-09-17T02:43:47.644020+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:25.647747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.d","version":"64'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.987558+0000","last_change":"2024-09-17T02:43:45.669829+0000","last_active":"2024-09-17T02:43:47.987558+0000","last_peered":"2024-09-17T02:43:47.987558+0000","last_clean":"2024-09-17T02:43:47.987558+0000","last_became_active":"2024-09-17T02:43:45.669491+0000","last_became_peered":"2024-09-17T02:43:45.669491+0000","last_unstale":"2024-09-17T02:43:47.987558+0000","last_undegraded":"2024-09-17T02:43:47.987558+0000","last_fullsized":"2024-09-17T02:43:47.987558+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:35.464346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.988583+0000","last_change":"2024-09-17T02:43:46.671220+0000","last_active":"2024-09-17T02:43:47.988583+0000","last_peered":"2024-09-17T02:43:47.988583+0000","last_clean":"2024-09-17T02:43:47.988583+0000","last_became_active":"2024-09-17T02:43:46.670289+0000","last_became_peered":"2024-09-17T02:43:46.670289+0000","last_unstale":"2024-09-17T02:43:47.988583+0000","last_undegraded":"2024-09-17T02:43:47.988583+0000","last_fullsized":"2024-09-17T02:43:47.988583+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:55.469678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707366+0000","last_change":"2024-09-17T02:43:45.674406+0000","last_active":"2024-09-17T02:43:47.707366+0000","last_peered":"2024-09-17T02:43:47.707366+0000","last_clean":"2024-09-17T02:43:47.707366+0000","last_became_active":"2024-09-17T02:43:45.671107+0000","last_became_peered":"2024-09-17T02:43:45.671107+0000","last_unstale":"2024-09-17T02:43:47.707366+0000","last_undegraded":"2024-09-17T02:43:47.707366+0000","last_fullsized":"2024-09-17T02:43:47.707366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:13.407925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986196+0000","last_change":"2024-09-17T02:43:46.690095+0000","last_active":"2024-09-17T02:43:47.986196+0000","last_peered":"2024-09-17T02:43:47.986196+0000","last_clean":"2024-09-17T02:43:47.986196+0000","last_became_active":"2024-09-17T02:43:46.689128+0000","last_became_peered":"2024-09-17T02:43:46.689128+0000","last_unstale":"2024-09-17T02:43:47.986196+0000","last_undegraded":"2024-09-17T02:43:47.986196+0000","last_fullsized":"2024-09-17T02:43:47.986196+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:41:18.780544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.b","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986419+0000","last_change":"2024-09-17T02:43:45.680606+0000","last_active":"2024-09-17T02:43:47.986419+0000","last_peered":"2024-09-17T02:43:47.986419+0000","last_clean":"2024-09-17T02:43:47.986419+0000","last_became_active":"2024-09-17T02:43:45.679640+0000","last_became_peered":"2024-09-17T02:43:45.679640+0000","last_unstale":"2024-09-17T02:43:47.986419+0000","last_undegraded":"2024-09-17T02:43:47.986419+0000","last_fullsized":"2024-09-17T02:43:47.986419+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:19.822462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.643260+0000","last_change":"2024-09-17T02:43:46.682953+0000","last_active":"2024-09-17T02:43:47.643260+0000","last_peered":"2024-09-17T02:43:47.643260+0000","last_clean":"2024-09-17T02:43:47.643260+0000","last_became_active":"2024-09-17T02:43:46.671608+0000","last_became_peered":"2024-09-17T02:43:46.671608+0000","last_unstale":"2024-09-17T02:43:47.643260+0000","last_undegraded":"2024-09-17T02:43:47.643260+0000","last_fullsized":"2024-09-17T02:43:47.643260+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:56:40.932076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.987023+0000","last_change":"2024-09-17T02:43:45.664123+0000","last_active":"2024-09-17T02:43:47.987023+0000","last_peered":"2024-09-17T02:43:47.987023+0000","last_clean":"2024-09-17T02:43:47.987023+0000","last_became_active":"2024-09-17T02:43:45.663776+0000","last_became_peered":"2024-09-17T02:43:45.663776+0000","last_unstale":"2024-09-17T02:43:47.987023+0000","last_undegraded":"2024-09-17T02:43:47.987023+0000","last_fullsized":"2024-09-17T02:43:47.987023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:07.938859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700904+0000","last_change":"2024-09-17T02:43:46.693857+0000","last_active":"2024-09-17T02:43:47.700904+0000","last_peered":"2024-09-17T02:43:47.700904+0000","last_clean":"2024-09-17T02:43:47.700904+0000","last_became_active":"2024-09-17T02:43:46.692225+0000","last_became_peered":"2024-09-17T02:43:46.692225+0000","last_unstale":"2024-09-17T02:43:47.700904+0000","last_undegraded":"2024-09-17T02:43:47.700904+0000","last_fullsized":"2024-09-17T02:43:47.700904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:58:56.128166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.985853+0000","last_change":"2024-09-17T02:43:45.684484+0000","last_active":"2024-09-17T02:43:47.985853+0000","last_peered":"2024-09-17T02:43:47.985853+0000","last_clean":"2024-09-17T02:43:47.985853+0000","last_became_active":"2024-09-17T02:43:45.683639+0000","last_became_peered":"2024-09-17T02:43:45.683639+0000","last_unstale":"2024-09-17T02:43:47.985853+0000","last_undegraded":"2024-09-17T02:43:47.985853+0000","last_fullsized":"2024-09-17T02:43:47.985853+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:44:44.894520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.648456+0000","last_change":"2024-09-17T02:43:46.674643+0000","last_active":"2024-09-17T02:43:47.648456+0000","last_peered":"2024-09-17T02:43:47.648456+0000","last_clean":"2024-09-17T02:43:47.648456+0000","last_became_active":"2024-09-17T02:43:46.673681+0000","last_became_peered":"2024-09-17T02:43:46.673681+0000","last_unstale":"2024-09-17T02:43:47.648456+0000","last_undegraded":"2024-09-17T02:43:47.648456+0000","last_fullsized":"2024-09-17T02:43:47.648456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:14:09.748085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.703860+0000","last_change":"2024-09-17T02:43:45.647880+0000","last_active":"2024-09-17T02:43:47.703860+0000","last_peered":"2024-09-17T02:43:47.703860+0000","last_clean":"2024-09-17T02:43:47.703860+0000","last_became_active":"2024-09-17T02:43:45.647466+0000","last_became_peered":"2024-09-17T02:43:45.647466+0000","last_unstale":"2024-09-17T02:43:47.703860+0000","last_undegraded":"2024-09-17T02:43:47.703860+0000","last_fullsized":"2024-09-17T02:43:47.703860+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:30:19.936068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.649200+0000","last_change":"2024-09-17T02:43:46.674511+0000","last_active":"2024-09-17T02:43:47.649200+0000","last_peered":"2024-09-17T02:43:47.649200+0000","last_clean":"2024-09-17T02:43:47.649200+0000","last_became_active":"2024-09-17T02:43:46.673628+0000","last_became_peered":"2024-09-17T02:43:46.673628+0000","last_unstale":"2024-09-17T02:43:47.649200+0000","last_undegraded":"2024-09-17T02:43:47.649200+0000","last_fullsized":"2024-09-17T02:43:47.649200+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:44.828816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"65'1","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.702986+0000","last_change":"2024-09-17T02:43:45.676827+0000","last_active":"2024-09-17T02:43:47.702986+0000","last_peered":"2024-09-17T02:43:47.702986+0000","last_clean":"2024-09-17T02:43:47.702986+0000","last_became_active":"2024-09-17T02:43:45.665485+0000","last_became_peered":"2024-09-17T02:43:45.665485+0000","last_unstale":"2024-09-17T02:43:47.702986+0000","last_undegraded":"2024-09-17T02:43:47.702986+0000","last_fullsized":"2024-09-17T02:43:47.702986+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:17.346724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.988019+0000","last_change":"2024-09-17T02:43:46.676583+0000","last_active":"2024-09-17T02:43:47.988019+0000","last_peered":"2024-09-17T02:43:47.988019+0000","last_clean":"2024-09-17T02:43:47.988019+0000","last_became_active":"2024-09-17T02:43:46.675742+0000","last_became_peered":"2024-09-17T02:43:46.675742+0000","last_unstale":"2024-09-17T02:43:47.988019+0000","last_undegraded":"2024-09-17T02:43:47.988019+0000","last_fullsized":"2024-09-17T02:43:47.988019+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:28.674095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986742+0000","last_change":"2024-09-17T02:43:45.685852+0000","last_active":"2024-09-17T02:43:47.986742+0000","last_peered":"2024-09-17T02:43:47.986742+0000","last_clean":"2024-09-17T02:43:47.986742+0000","last_became_active":"2024-09-17T02:43:45.683659+0000","last_became_peered":"2024-09-17T02:43:45.683659+0000","last_unstale":"2024-09-17T02:43:47.986742+0000","last_undegraded":"2024-09-17T02:43:47.986742+0000","last_fullsized":"2024-09-17T02:43:47.986742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:24.643627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.988627+0000","last_change":"2024-09-17T02:43:46.676488+0000","last_active":"2024-09-17T02:43:47.988627+0000","last_peered":"2024-09-17T02:43:47.988627+0000","last_clean":"2024-09-17T02:43:47.988627+0000","last_became_active":"2024-09-17T02:43:46.673426+0000","last_became_peered":"2024-09-17T02:43:46.673426+0000","last_unstale":"2024-09-17T02:43:47.988627+0000","last_undegraded":"2024-09-17T02:43:47.988627+0000","last_fullsized":"2024-09-17T02:43:47.988627+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:18:33.047658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707587+0000","last_change":"2024-09-17T02:43:45.675150+0000","last_active":"2024-09-17T02:43:47.707587+0000","last_peered":"2024-09-17T02:43:47.707587+0000","last_clean":"2024-09-17T02:43:47.707587+0000","last_became_active":"2024-09-17T02:43:45.671999+0000","last_became_peered":"2024-09-17T02:43:45.671999+0000","last_unstale":"2024-09-17T02:43:47.707587+0000","last_undegraded":"2024-09-17T02:43:47.707587+0000","last_fullsized":"2024-09-17T02:43:47.707587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:17.815894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.700564+0000","last_change":"2024-09-17T02:43:46.698516+0000","last_active":"2024-09-17T02:43:47.700564+0000","last_peered":"2024-09-17T02:43:47.700564+0000","last_clean":"2024-09-17T02:43:47.700564+0000","last_became_active":"2024-09-17T02:43:46.696239+0000","last_became_peered":"2024-09-17T02:43:46.696239+0000","last_unstale":"2024-09-17T02:43:47.700564+0000","last_undegraded":"2024-09-17T02:43:47.700564+0000","last_fullsized":"2024-09-17T02:43:47.700564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:46.407009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.643675+0000","last_change":"2024-09-17T02:43:45.682922+0000","last_active":"2024-09-17T02:43:47.643675+0000","last_peered":"2024-09-17T02:43:47.643675+0000","last_clean":"2024-09-17T02:43:47.643675+0000","last_became_active":"2024-09-17T02:43:45.682556+0000","last_became_peered":"2024-09-17T02:43:45.682556+0000","last_unstale":"2024-09-17T02:43:47.643675+0000","last_undegraded":"2024-09-17T02:43:47.643675+0000","last_fullsized":"2024-09-17T02:43:47.643675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:20.160389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.643304+0000","last_change":"2024-09-17T02:43:46.696657+0000","last_active":"2024-09-17T02:43:47.643304+0000","last_peered":"2024-09-17T02:43:47.643304+0000","last_clean":"2024-09-17T02:43:47.643304+0000","last_became_active":"2024-09-17T02:43:46.695382+0000","last_became_peered":"2024-09-17T02:43:46.695382+0000","last_unstale":"2024-09-17T02:43:47.643304+0000","last_undegraded":"2024-09-17T02:43:47.643304+0000","last_fullsized":"2024-09-17T02:43:47.643304+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:18.350845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.652374+0000","last_change":"2024-09-17T02:43:45.666290+0000","last_active":"2024-09-17T02:43:47.652374+0000","last_peered":"2024-09-17T02:43:47.652374+0000","last_clean":"2024-09-17T02:43:47.652374+0000","last_became_active":"2024-09-17T02:43:45.665410+0000","last_became_peered":"2024-09-17T02:43:45.665410+0000","last_unstale":"2024-09-17T02:43:47.652374+0000","last_undegraded":"2024-09-17T02:43:47.652374+0000","last_fullsized":"2024-09-17T02:43:47.652374+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:33.788490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986391+0000","last_change":"2024-09-17T02:43:46.692989+0000","last_active":"2024-09-17T02:43:47.986391+0000","last_peered":"2024-09-17T02:43:47.986391+0000","last_clean":"2024-09-17T02:43:47.986391+0000","last_became_active":"2024-09-17T02:43:46.692113+0000","last_became_peered":"2024-09-17T02:43:46.692113+0000","last_unstale":"2024-09-17T02:43:47.986391+0000","last_undegraded":"2024-09-17T02:43:47.986391+0000","last_fullsized":"2024-09-17T02:43:47.986391+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:26:27.991604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.707446+0000","last_change":"2024-09-17T02:43:45.672483+0000","last_active":"2024-09-17T02:43:47.707446+0000","last_peered":"2024-09-17T02:43:47.707446+0000","last_clean":"2024-09-17T02:43:47.707446+0000","last_became_active":"2024-09-17T02:43:45.669710+0000","last_became_peered":"2024-09-17T02:43:45.669710+0000","last_unstale":"2024-09-17T02:43:47.707446+0000","last_undegraded":"2024-09-17T02:43:47.707446+0000","last_fullsized":"2024-09-17T02:43:47.707446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:20.160389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.647737+0000","last_change":"2024-09-17T02:43:46.697145+0000","last_active":"2024-09-17T02:43:47.647737+0000","last_peered":"2024-09-17T02:43:47.647737+0000","last_clean":"2024-09-17T02:43:47.647737+0000","last_became_active":"2024-09-17T02:43:46.696508+0000","last_became_peered":"2024-09-17T02:43:46.696508+0000","last_unstale":"2024-09-17T02:43:47.647737+0000","last_undegraded":"2024-09-17T02:43:47.647737+0000","last_fullsized":"2024-09-17T02:43:47.647737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:49:45.286919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.986013+0000","last_change":"2024-09-17T02:43:45.690426+0000","last_active":"2024-09-17T02:43:47.986013+0000","last_peered":"2024-09-17T02:43:47.986013+0000","last_clean":"2024-09-17T02:43:47.986013+0000","last_became_active":"2024-09-17T02:43:45.689221+0000","last_became_peered":"2024-09-17T02:43:45.689221+0000","last_unstale":"2024-09-17T02:43:47.986013+0000","last_undegraded":"2024-09-17T02:43:47.986013+0000","last_fullsized":"2024-09-17T02:43:47.986013+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:35:19.530784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.987478+0000","last_change":"2024-09-17T02:43:46.667348+0000","last_active":"2024-09-17T02:43:47.987478+0000","last_peered":"2024-09-17T02:43:47.987478+0000","last_clean":"2024-09-17T02:43:47.987478+0000","last_became_active":"2024-09-17T02:43:46.666476+0000","last_became_peered":"2024-09-17T02:43:46.666476+0000","last_unstale":"2024-09-17T02:43:47.987478+0000","last_undegraded":"2024-09-17T02:43:47.987478+0000","last_fullsized":"2024-09-17T02:43:47.987478+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:45.623379+0000","last_clean_scrub_stamp":"2024-09-17T02:43:45.623379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:18.350845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.987012+0000","last_change":"2024-09-17T02:43:45.688114+0000","last_active":"2024-09-17T02:43:47.987012+0000","last_peered":"2024-09-17T02:43:47.987012+0000","last_clean":"2024-09-17T02:43:47.987012+0000","last_became_active":"2024-09-17T02:43:45.687238+0000","last_became_peered":"2024-09-17T02:43:45.687238+0000","last_unstale":"2024-09-17T02:43:47.987012+0000","last_undegraded":"2024-09-17T02:43:47.987012+0000","last_fullsized":"2024-09-17T02:43:47.987012+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:43:44.619259+0000","last_clean_scrub_stamp":"2024-09-17T02:43:44.619259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:51:50.388404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":191,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-17T02:43:47.985257+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:43:47.985257+0000","last_peered":"2024-09-17T02:43:47.985257+0000","last_clean":"2024-09-17T02:43:47.985257+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:43:47.985257+0000","last_undegraded":"2024-09-17T02:43:47.985257+0000","last_fullsized":"2024-09-17T02:43:47.985257+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":778240,"data_stored":349676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1392640,"data_stored":941768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":782336,"data_stored":350258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":786432,"data_stored":349836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":786432,"data_stored":349732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32540,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339300,"statfs":{"total":96636764160,"available":96603443200,"internally_reserved":0,"allocated":1376256,"data_stored":940078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":778240,"data_stored":349676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":798720,"data_stored":351940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:43:55.813 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T02:43:55.814 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T02:43:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.019+0000 7f38e31b4640 1 -- 172.21.15.131:0/2306033697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc06ecd0 msgr2=0x7f38dc071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.019+0000 7f38e31b4640 1 --2- 172.21.15.131:0/2306033697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc06ecd0 0x7f38dc071210 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0ce820 tx=0x7f38d001c7a0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.019+0000 7f38e31b4640 1 -- 172.21.15.131:0/2306033697 shutdown_connections 2024-09-17T02:43:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.019+0000 7f38e31b4640 1 --2- 172.21.15.131:0/2306033697 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f38dc15bee0 0x7f38dc0b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.019+0000 7f38e31b4640 1 --2- 172.21.15.131:0/2306033697 >> v2:172.21.15.189:3300/0 conn(0x7f38dc071750 0x7f38dc15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.019+0000 7f38e31b4640 1 --2- 172.21.15.131:0/2306033697 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc06ecd0 0x7f38dc071210 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.019+0000 7f38e31b4640 1 -- 172.21.15.131:0/2306033697 >> 172.21.15.131:0/2306033697 conn(0x7f38dc0d1120 msgr2=0x7f38dc0d1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.019+0000 7f38e31b4640 1 -- 172.21.15.131:0/2306033697 shutdown_connections 2024-09-17T02:43:56.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.019+0000 7f38e31b4640 1 -- 172.21.15.131:0/2306033697 wait complete. 2024-09-17T02:43:56.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e31b4640 1 Processor -- start 2024-09-17T02:43:56.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e31b4640 1 -- start start 2024-09-17T02:43:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e31b4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc06ecd0 0x7f38dc14bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e31b4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f38dc071750 0x7f38dc1643b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e31b4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38dc15bee0 0x7f38dc1648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e31b4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc15f6c0 con 0x7f38dc06ecd0 2024-09-17T02:43:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e31b4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc0b4bd0 con 0x7f38dc15bee0 2024-09-17T02:43:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e31b4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f38dc15f840 con 0x7f38dc071750 2024-09-17T02:43:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e0f50640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc06ecd0 0x7f38dc14bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:56.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e0f50640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc06ecd0 0x7f38dc14bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51380/0 (socket says 172.21.15.131:51380) 2024-09-17T02:43:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e0f50640 1 -- 172.21.15.131:0/2797376814 learned_addr learned my addr 172.21.15.131:0/2797376814 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38dbfff640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f38dc071750 0x7f38dc1643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e1751640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38dc15bee0 0x7f38dc1648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e0f50640 1 -- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f38dc071750 msgr2=0x7f38dc1643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e0f50640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f38dc071750 0x7f38dc1643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e0f50640 1 -- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38dc15bee0 msgr2=0x7f38dc1648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e0f50640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38dc15bee0 0x7f38dc1648f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e0f50640 1 -- 172.21.15.131:0/2797376814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc155260 con 0x7f38dc06ecd0 2024-09-17T02:43:56.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38dbfff640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f38dc071750 0x7f38dc1643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:43:56.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e0f50640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc06ecd0 0x7f38dc14bd70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f38d00b0040 tx=0x7f38d00a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:56.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38d97fa640 1 -- 172.21.15.131:0/2797376814 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d00b2040 con 0x7f38dc06ecd0 2024-09-17T02:43:56.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38d97fa640 1 -- 172.21.15.131:0/2797376814 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38d00044b0 con 0x7f38dc06ecd0 2024-09-17T02:43:56.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e31b4640 1 -- 172.21.15.131:0/2797376814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc14c2b0 con 0x7f38dc06ecd0 2024-09-17T02:43:56.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38d97fa640 1 -- 172.21.15.131:0/2797376814 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d0003860 con 0x7f38dc06ecd0 2024-09-17T02:43:56.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.023+0000 7f38e31b4640 1 -- 172.21.15.131:0/2797376814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc1fc890 con 0x7f38dc06ecd0 2024-09-17T02:43:56.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.027+0000 7f38e31b4640 1 -- 172.21.15.131:0/2797376814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38dc15fc90 con 0x7f38dc06ecd0 2024-09-17T02:43:56.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.027+0000 7f38d97fa640 1 -- 172.21.15.131:0/2797376814 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f38d0003a00 con 0x7f38dc06ecd0 2024-09-17T02:43:56.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.027+0000 7f38d97fa640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f38c0073200 0x7f38c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:56.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.027+0000 7f38d97fa640 1 -- 172.21.15.131:0/2797376814 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f38d012e260 con 0x7f38dc06ecd0 2024-09-17T02:43:56.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.027+0000 7f38dbfff640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f38c0073200 0x7f38c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:56.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.027+0000 7f38dbfff640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f38c0073200 0x7f38c0075710 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f38c8005fd0 tx=0x7f38c8005e40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:56.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.031+0000 7f38d97fa640 1 -- 172.21.15.131:0/2797376814 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f38d00bd060 con 0x7f38dc06ecd0 2024-09-17T02:43:56.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.259+0000 7f38e31b4640 1 -- 172.21.15.131:0/2797376814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f38dc0d00e0 con 0x7f38dc06ecd0 2024-09-17T02:43:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.263+0000 7f38d97fa640 1 -- 172.21.15.131:0/2797376814 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v31) ==== 190+0+147 (secure 0 0 0) 0x7f38d00164b0 con 0x7f38dc06ecd0 2024-09-17T02:43:56.267 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-09-17T02:43:56.268 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA== 2024-09-17T02:43:56.268 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-09-17T02:43:56.268 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-09-17T02:43:56.268 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T02:43:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-09-17T02:43:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 -- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f38c0073200 msgr2=0x7f38c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f38c0073200 0x7f38c0075710 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f38c8005fd0 tx=0x7f38c8005e40 comp rx=0 tx=0).stop 2024-09-17T02:43:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 -- 172.21.15.131:0/2797376814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc06ecd0 msgr2=0x7f38dc14bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc06ecd0 0x7f38dc14bd70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f38d00b0040 tx=0x7f38d00a57d0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 -- 172.21.15.131:0/2797376814 shutdown_connections 2024-09-17T02:43:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f38c0073200 0x7f38c0075710 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f38dc15bee0 0x7f38dc1648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f38dc071750 0x7f38dc1643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 --2- 172.21.15.131:0/2797376814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f38dc06ecd0 0x7f38dc14bd70 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 -- 172.21.15.131:0/2797376814 >> 172.21.15.131:0/2797376814 conn(0x7f38dc0d1120 msgr2=0x7f38dc0d0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 -- 172.21.15.131:0/2797376814 shutdown_connections 2024-09-17T02:43:56.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.267+0000 7f38e31b4640 1 -- 172.21.15.131:0/2797376814 wait complete. 2024-09-17T02:43:56.287 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:43:56.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.443+0000 7f112c7e4640 1 -- 172.21.15.131:0/4284389117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1110003a40 msgr2=0x7f1110003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:56.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.443+0000 7f112c7e4640 1 --2- 172.21.15.131:0/4284389117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1110003a40 0x7f1110003eb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f1120009b60 tx=0x7f112001d290 comp rx=0 tx=0).stop 2024-09-17T02:43:56.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.443+0000 7f112ad81640 1 -- 172.21.15.131:0/4284389117 reap_dead start 2024-09-17T02:43:56.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.443+0000 7f112c7e4640 1 -- 172.21.15.131:0/4284389117 shutdown_connections 2024-09-17T02:43:56.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.443+0000 7f112c7e4640 1 -- 172.21.15.131:0/4284389117 >> 172.21.15.131:0/4284389117 conn(0x7f11240cf110 msgr2=0x7f11240cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:56.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.443+0000 7f112c7e4640 1 -- 172.21.15.131:0/4284389117 shutdown_connections 2024-09-17T02:43:56.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.443+0000 7f112c7e4640 1 -- 172.21.15.131:0/4284389117 wait complete. 2024-09-17T02:43:56.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.443+0000 7f112c7e4640 1 Processor -- start 2024-09-17T02:43:56.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.443+0000 7f112c7e4640 1 -- start start 2024-09-17T02:43:56.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f112c7e4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1110003a40 0x7f11241f6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:56.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f112c7e4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124154260 0x7f11240ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:56.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f112c7e4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f11240caea0 0x7f11240c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:56.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f112c7e4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1124153060 con 0x7f1124154260 2024-09-17T02:43:56.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f112c7e4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f11240b4ab0 con 0x7f1110003a40 2024-09-17T02:43:56.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f112c7e4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f11241531e0 con 0x7f11240caea0 2024-09-17T02:43:56.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f1129d7f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124154260 0x7f11240ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f1129d7f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124154260 0x7f11240ca960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51396/0 (socket says 172.21.15.131:51396) 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f1129d7f640 1 -- 172.21.15.131:0/2345943448 learned_addr learned my addr 172.21.15.131:0/2345943448 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f112a580640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1110003a40 0x7f11241f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f1129d7f640 1 -- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f11240caea0 msgr2=0x7f11240c6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f112ad81640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f11240caea0 0x7f11240c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f1129d7f640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f11240caea0 0x7f11240c6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f1129d7f640 1 -- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1110003a40 msgr2=0x7f11241f6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f1129d7f640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1110003a40 0x7f11241f6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f1129d7f640 1 -- 172.21.15.131:0/2345943448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11240c6d40 con 0x7f1124154260 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f1129d7f640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124154260 0x7f11240ca960 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f111400d6f0 tx=0x7f111400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f111affd640 1 -- 172.21.15.131:0/2345943448 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11140060d0 con 0x7f1124154260 2024-09-17T02:43:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f111affd640 1 -- 172.21.15.131:0/2345943448 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1114015020 con 0x7f1124154260 2024-09-17T02:43:56.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f111affd640 1 -- 172.21.15.131:0/2345943448 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111400b830 con 0x7f1124154260 2024-09-17T02:43:56.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f112c7e4640 1 -- 172.21.15.131:0/2345943448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11240cb2b0 con 0x7f1124154260 2024-09-17T02:43:56.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f112c7e4640 1 -- 172.21.15.131:0/2345943448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1124156090 con 0x7f1124154260 2024-09-17T02:43:56.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f112c7e4640 1 -- 172.21.15.131:0/2345943448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1124153740 con 0x7f1124154260 2024-09-17T02:43:56.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.447+0000 7f111affd640 1 -- 172.21.15.131:0/2345943448 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f111400b9d0 con 0x7f1124154260 2024-09-17T02:43:56.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.451+0000 7f111affd640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1108073200 0x7f1108075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:56.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.451+0000 7f111affd640 1 -- 172.21.15.131:0/2345943448 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f1114094a10 con 0x7f1124154260 2024-09-17T02:43:56.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.451+0000 7f112a580640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1108073200 0x7f1108075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:56.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.451+0000 7f112a580640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1108073200 0x7f1108075710 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f11200ae000 tx=0x7f112001a030 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.455+0000 7f111affd640 1 -- 172.21.15.131:0/2345943448 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f111401f020 con 0x7f1124154260 2024-09-17T02:43:56.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.643+0000 7f112c7e4640 1 -- 172.21.15.131:0/2345943448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f112406a170 con 0x7f1124154260 2024-09-17T02:43:56.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.643+0000 7f111affd640 1 -- 172.21.15.131:0/2345943448 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5233 (secure 0 0 0) 0x7f11240cb2b0 con 0x7f1124154260 2024-09-17T02:43:56.647 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:43:56.647 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":36,"btime":"2024-09-17T02:43:49:702553+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3554877374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3554877374},{"type":"v1","addr":"172.21.15.189:6837","nonce":3554877374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1670590113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1670590113},{"type":"v1","addr":"172.21.15.131:6834","nonce":1670590113}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1046159039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1046159039},{"type":"v1","addr":"172.21.15.131:6835","nonce":1046159039}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:43:46.061550+0000","modified":"2024-09-17T02:43:49.109393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5871":{"gid":5871,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6835/730392436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":730392436},{"type":"v1","addr":"172.21.15.189:6835","nonce":730392436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5871,"qdb_cluster":[5871]},"id":4}]} 2024-09-17T02:43:56.647 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 36 2024-09-17T02:43:56.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 -- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1108073200 msgr2=0x7f1108075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:56.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1108073200 0x7f1108075710 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f11200ae000 tx=0x7f112001a030 comp rx=0 tx=0).stop 2024-09-17T02:43:56.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 -- 172.21.15.131:0/2345943448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124154260 msgr2=0x7f11240ca960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:56.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124154260 0x7f11240ca960 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f111400d6f0 tx=0x7f111400dbb0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 -- 172.21.15.131:0/2345943448 shutdown_connections 2024-09-17T02:43:56.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1108073200 0x7f1108075710 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f11240caea0 0x7f11240c6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1124154260 0x7f11240ca960 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 --2- 172.21.15.131:0/2345943448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1110003a40 0x7f11241f6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:56.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 -- 172.21.15.131:0/2345943448 >> 172.21.15.131:0/2345943448 conn(0x7f11240cf110 msgr2=0x7f11240d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:56.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 -- 172.21.15.131:0/2345943448 shutdown_connections 2024-09-17T02:43:56.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:56.647+0000 7f112c7e4640 1 -- 172.21.15.131:0/2345943448 wait complete. 2024-09-17T02:43:57.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:43:46.061550+0000', 'modified': '2024-09-17T02:43:49.109393+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5871}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5871': {'gid': 5871, 'name': 'd', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.189:6835/730392436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.189:6834', 'nonce': 730392436}, {'type': 'v1', 'addr': '172.21.15.189:6835', 'nonce': 730392436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5871, 'qdb_cluster': [5871]} 2024-09-17T02:43:57.073 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:43:57.073 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-17T02:43:57.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.239+0000 7f937ae88640 1 -- 172.21.15.131:0/1517561985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cd470 msgr2=0x7f937406efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.239+0000 7f937ae88640 1 --2- 172.21.15.131:0/1517561985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cd470 0x7f937406efb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f93740d0d50 tx=0x7f936c01cd60 comp rx=0 tx=0).stop 2024-09-17T02:43:57.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.239+0000 7f937ae88640 1 -- 172.21.15.131:0/1517561985 shutdown_connections 2024-09-17T02:43:57.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.239+0000 7f937ae88640 1 --2- 172.21.15.131:0/1517561985 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f937415bdb0 0x7f9374151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.239+0000 7f937ae88640 1 --2- 172.21.15.131:0/1517561985 >> v2:172.21.15.189:3300/0 conn(0x7f937406f4f0 0x7f937415b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.239+0000 7f937ae88640 1 --2- 172.21.15.131:0/1517561985 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93740cd470 0x7f937406efb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.239+0000 7f937ae88640 1 -- 172.21.15.131:0/1517561985 >> 172.21.15.131:0/1517561985 conn(0x7f9374073d10 msgr2=0x7f9374074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:57.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.239+0000 7f937ae88640 1 -- 172.21.15.131:0/1517561985 shutdown_connections 2024-09-17T02:43:57.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.239+0000 7f937ae88640 1 -- 172.21.15.131:0/1517561985 wait complete. 2024-09-17T02:43:57.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f937ae88640 1 Processor -- start 2024-09-17T02:43:57.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f937ae88640 1 -- start start 2024-09-17T02:43:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f937ae88640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f937406f4f0 0x7f93740c5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f937ae88640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93740cd470 0x7f937414b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f937ae88640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f937415bdb0 0x7f9374071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f937ae88640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f93740cc280 con 0x7f937415bdb0 2024-09-17T02:43:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f937ae88640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93740cc100 con 0x7f93740cd470 2024-09-17T02:43:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f937ae88640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f93740cc400 con 0x7f937406f4f0 2024-09-17T02:43:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f9379425640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f937415bdb0 0x7f9374071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f9379425640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f937415bdb0 0x7f9374071420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51414/0 (socket says 172.21.15.131:51414) 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f9379425640 1 -- 172.21.15.131:0/2395836992 learned_addr learned my addr 172.21.15.131:0/2395836992 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f9378c24640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f937406f4f0 0x7f93740c5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f9373fff640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93740cd470 0x7f937414b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f9379425640 1 -- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f937406f4f0 msgr2=0x7f93740c5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f9379425640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f937406f4f0 0x7f93740c5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f9379425640 1 -- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93740cd470 msgr2=0x7f937414b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f9379425640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93740cd470 0x7f937414b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f9379425640 1 -- 172.21.15.131:0/2395836992 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93740c61b0 con 0x7f937415bdb0 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f9379425640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f937415bdb0 0x7f9374071420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f936400d640 tx=0x7f936400db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f93717fa640 1 -- 172.21.15.131:0/2395836992 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364017070 con 0x7f937415bdb0 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f93717fa640 1 -- 172.21.15.131:0/2395836992 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9364005310 con 0x7f937415bdb0 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f937ae88640 1 -- 172.21.15.131:0/2395836992 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93740c6490 con 0x7f937415bdb0 2024-09-17T02:43:57.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f937ae88640 1 -- 172.21.15.131:0/2395836992 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937414be30 con 0x7f937415bdb0 2024-09-17T02:43:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.243+0000 7f93717fa640 1 -- 172.21.15.131:0/2395836992 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93640047b0 con 0x7f937415bdb0 2024-09-17T02:43:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.247+0000 7f93717fa640 1 -- 172.21.15.131:0/2395836992 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9364004950 con 0x7f937415bdb0 2024-09-17T02:43:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.247+0000 7f93717fa640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9360073230 0x7f9360075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.247+0000 7f93717fa640 1 -- 172.21.15.131:0/2395836992 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f9364094430 con 0x7f937415bdb0 2024-09-17T02:43:57.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.247+0000 7f9378c24640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9360073230 0x7f9360075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.247+0000 7f9378c24640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9360073230 0x7f9360075740 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f936c0af000 tx=0x7f936c01b5a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:57.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.247+0000 7f9382153000 1 Processor -- start 2024-09-17T02:43:57.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.247+0000 7f9382153000 1 -- start start 2024-09-17T02:43:57.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df8378d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df6e4950 0x55f8df837e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df838350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f8df70a410 con 0x55f8df2f4460 2024-09-17T02:43:57.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f8df647b40 con 0x55f8df6cdca0 2024-09-17T02:43:57.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f8df636f70 con 0x55f8df6e4950 2024-09-17T02:43:57.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9378c24640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df8378d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9378c24640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df8378d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51420/0 (socket says 172.21.15.131:51420) 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9378c24640 1 -- 172.21.15.131:0/3509034197 learned_addr learned my addr 172.21.15.131:0/3509034197 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9373fff640 1 --2- 172.21.15.131:0/3509034197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df6e4950 0x55f8df837e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9379425640 1 --2- 172.21.15.131:0/3509034197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df838350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9378c24640 1 -- 172.21.15.131:0/3509034197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df6e4950 msgr2=0x55f8df837e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9378c24640 1 --2- 172.21.15.131:0/3509034197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df6e4950 0x55f8df837e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9378c24640 1 -- 172.21.15.131:0/3509034197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 msgr2=0x55f8df838350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9378c24640 1 --2- 172.21.15.131:0/3509034197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df838350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9378c24640 1 -- 172.21.15.131:0/3509034197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8df5fe5b0 con 0x55f8df2f4460 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9373fff640 1 --2- 172.21.15.131:0/3509034197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df6e4950 0x55f8df837e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9378c24640 1 --2- 172.21.15.131:0/3509034197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df8378d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f936c098820 tx=0x7f936c0b6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1502fad49a0d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f935a7fc640 1 -- 172.21.15.131:0/3509034197 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c0b2040 con 0x55f8df2f4460 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f935a7fc640 1 -- 172.21.15.131:0/3509034197 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f936c0c6070 con 0x55f8df2f4460 2024-09-17T02:43:57.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f935a7fc640 1 -- 172.21.15.131:0/3509034197 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c004da0 con 0x55f8df2f4460 2024-09-17T02:43:57.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 -- 172.21.15.131:0/3509034197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 msgr2=0x55f8df8378d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 --2- 172.21.15.131:0/3509034197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df8378d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f936c098820 tx=0x7f936c0b6000 comp rx=0 tx=0).stop 2024-09-17T02:43:57.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 -- 172.21.15.131:0/3509034197 shutdown_connections 2024-09-17T02:43:57.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 --2- 172.21.15.131:0/3509034197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df838350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 --2- 172.21.15.131:0/3509034197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df6e4950 0x55f8df837e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 --2- 172.21.15.131:0/3509034197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df8378d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 -- 172.21.15.131:0/3509034197 >> 172.21.15.131:0/3509034197 conn(0x55f8df7bad90 msgr2=0x55f8df7ea690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:57.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 -- 172.21.15.131:0/3509034197 shutdown_connections 2024-09-17T02:43:57.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 -- 172.21.15.131:0/3509034197 wait complete. 2024-09-17T02:43:57.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 -- start start 2024-09-17T02:43:57.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.251+0000 7f9382153000 1 Processor -- start 2024-09-17T02:43:57.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df823000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df823540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df6e4950 0x55f8df823a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f8df5d15f0 con 0x55f8df2f4460 2024-09-17T02:43:57.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f8df63ad80 con 0x55f8df6cdca0 2024-09-17T02:43:57.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f8df5dd520 con 0x55f8df6e4950 2024-09-17T02:43:57.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9378c24640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df823000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 10 client.0 _ll_register_callbacks cb 0x55f8df5e1830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:43:57.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9378c24640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df823000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51436/0 (socket says 172.21.15.131:51436) 2024-09-17T02:43:57.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9378c24640 1 -- 172.21.15.131:0/3268937988 learned_addr learned my addr 172.21.15.131:0/3268937988 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:57.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9379425640 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df6e4950 0x55f8df823a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9373fff640 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df823540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9378c24640 1 -- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df6e4950 msgr2=0x55f8df823a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:57.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9378c24640 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df6e4950 0x55f8df823a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9378c24640 1 -- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 msgr2=0x55f8df823540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:57.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9378c24640 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df823540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9378c24640 1 -- 172.21.15.131:0/3268937988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8df5d5190 con 0x55f8df2f4460 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9378c24640 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df823000 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f936c098820 tx=0x7f936c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbccaf7806f8f51b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f93537fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f93537fe640 1 -- 172.21.15.131:0/3268937988 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c0b2040 con 0x55f8df2f4460 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f93537fe640 1 -- 172.21.15.131:0/3268937988 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f936c004240 con 0x55f8df2f4460 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 1 -- 172.21.15.131:0/3268937988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f8df5cf110 con 0x55f8df2f4460 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f93537fe640 1 -- 172.21.15.131:0/3268937988 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f936c0db050 con 0x55f8df2f4460 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 10 client.6027 fetch_fsmap learned FSMap version 36 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 1 -- 172.21.15.131:0/3268937988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55f8df6bbf10 con 0x55f8df2f4460 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f93537fe640 1 -- 172.21.15.131:0/3268937988 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f8df5cf110 con 0x55f8df2f4460 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f93537fe640 1 -- 172.21.15.131:0/3268937988 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f936c0de020 con 0x55f8df2f4460 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 10 client.6027 fetch_fsmap finished waiting for FSMap version 36 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 10 client.6027 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T02:43:57.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 20 client.6027 populate_metadata read hostname 'smithi131' 2024-09-17T02:43:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55f8df2d97d0 0x55f8df848e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 4 client.6027 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:43:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9382153000 1 -- 172.21.15.131:0/3268937988 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8df649a90 con 0x55f8df2d97d0 2024-09-17T02:43:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9373fff640 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55f8df2d97d0 0x55f8df848e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f93537fe640 10 client.6027 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:43:57.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.255+0000 7f9373fff640 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55f8df2d97d0 0x55f8df848e90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fcaa706640cfda4 server_cookie=53b27fdc0907e60b in_seq=0 out_seq=0 2024-09-17T02:43:57.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f93537fe640 1 -- 172.21.15.131:0/3268937988 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8df649a90 con 0x55f8df2d97d0 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f93537fe640 10 client.6027 handle_command_reply: tid=0 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 1 client.6027 shutdown 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 1 -- 172.21.15.131:0/3268937988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 msgr2=0x55f8df823000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df823000 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f936c098820 tx=0x7f936c005d90 comp rx=0 tx=0).stop 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 1 -- 172.21.15.131:0/3268937988 shutdown_connections 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55f8df2d97d0 0x55f8df848e90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df6e4950 0x55f8df823a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df823540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 1 --2- 172.21.15.131:0/3268937988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df2f4460 0x55f8df823000 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f93537fe640 0 client.6027 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 1 -- 172.21.15.131:0/3268937988 >> 172.21.15.131:0/3268937988 conn(0x55f8df7bad90 msgr2=0x55f8df7f1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:57.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 1 -- 172.21.15.131:0/3268937988 shutdown_connections 2024-09-17T02:43:57.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 1 -- 172.21.15.131:0/3268937988 wait complete. 2024-09-17T02:43:57.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.259+0000 7f9382153000 20 client.6027 trim_cache size 0 max 16384 2024-09-17T02:43:57.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 Processor -- start 2024-09-17T02:43:57.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 -- start start 2024-09-17T02:43:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df2f4460 0x55f8df84e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df84ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df848e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f935400cea0 con 0x55f8df6e4950 2024-09-17T02:43:57.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f935400be80 con 0x55f8df6cdca0 2024-09-17T02:43:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f935400b140 con 0x55f8df2f4460 2024-09-17T02:43:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9379425640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df848e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9379425640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df848e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51452/0 (socket says 172.21.15.131:51452) 2024-09-17T02:43:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9379425640 1 -- 172.21.15.131:0/1269291255 learned_addr learned my addr 172.21.15.131:0/1269291255 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9378c24640 1 --2- 172.21.15.131:0/1269291255 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df2f4460 0x55f8df84e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9379425640 1 -- 172.21.15.131:0/1269291255 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df2f4460 msgr2=0x55f8df84e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9379425640 1 --2- 172.21.15.131:0/1269291255 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df2f4460 0x55f8df84e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9373fff640 1 --2- 172.21.15.131:0/1269291255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df84ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9379425640 1 -- 172.21.15.131:0/1269291255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 msgr2=0x55f8df84ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9379425640 1 --2- 172.21.15.131:0/1269291255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df84ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9379425640 1 -- 172.21.15.131:0/1269291255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8df7f70b0 con 0x55f8df6e4950 2024-09-17T02:43:57.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9379425640 1 --2- 172.21.15.131:0/1269291255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df848e90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f936405be30 tx=0x7f9364005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d6d8d481116dc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:57.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9359ffb640 1 -- 172.21.15.131:0/1269291255 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364020040 con 0x55f8df6e4950 2024-09-17T02:43:57.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9359ffb640 1 -- 172.21.15.131:0/1269291255 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f936405ebc0 con 0x55f8df6e4950 2024-09-17T02:43:57.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9359ffb640 1 -- 172.21.15.131:0/1269291255 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93640636a0 con 0x55f8df6e4950 2024-09-17T02:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 -- 172.21.15.131:0/1269291255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 msgr2=0x55f8df848e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 --2- 172.21.15.131:0/1269291255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df848e90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f936405be30 tx=0x7f9364005760 comp rx=0 tx=0).stop 2024-09-17T02:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 -- 172.21.15.131:0/1269291255 shutdown_connections 2024-09-17T02:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 --2- 172.21.15.131:0/1269291255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df848e90 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 --2- 172.21.15.131:0/1269291255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df84ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 --2- 172.21.15.131:0/1269291255 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df2f4460 0x55f8df84e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 -- 172.21.15.131:0/1269291255 >> 172.21.15.131:0/1269291255 conn(0x55f8df7bad90 msgr2=0x55f8df84e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 -- 172.21.15.131:0/1269291255 shutdown_connections 2024-09-17T02:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.279+0000 7f9382153000 1 -- 172.21.15.131:0/1269291255 wait complete. 2024-09-17T02:43:57.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 -- start start 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 Processor -- start 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df2f4460 0x55f8df84bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df84c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df8693b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f8df63ad80 con 0x55f8df6e4950 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f935400b7b0 con 0x55f8df6cdca0 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f8df5d15f0 con 0x55f8df2f4460 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 10 client.0 _ll_register_callbacks cb 0x7f936c009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9379425640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df8693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9379425640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df8693b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51456/0 (socket says 172.21.15.131:51456) 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9379425640 1 -- 172.21.15.131:0/2967724780 learned_addr learned my addr 172.21.15.131:0/2967724780 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9378c24640 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df2f4460 0x55f8df84bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9373fff640 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df84c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9379425640 1 -- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df2f4460 msgr2=0x55f8df84bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:57.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9379425640 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df2f4460 0x55f8df84bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9379425640 1 -- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 msgr2=0x55f8df84c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:43:57.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9379425640 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df84c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9379425640 1 -- 172.21.15.131:0/2967724780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8df692db0 con 0x55f8df6e4950 2024-09-17T02:43:57.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9379425640 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df8693b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f936405be30 tx=0x7f936405b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c55b3a0cbb31752 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:57.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f93537fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:43:57.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f93537fe640 1 -- 172.21.15.131:0/2967724780 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9364020040 con 0x55f8df6e4950 2024-09-17T02:43:57.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f93537fe640 1 -- 172.21.15.131:0/2967724780 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f9364084180 con 0x55f8df6e4950 2024-09-17T02:43:57.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 -- 172.21.15.131:0/2967724780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936405ebc0 con 0x55f8df6e4950 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f93537fe640 1 -- 172.21.15.131:0/2967724780 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9364084380 con 0x55f8df6e4950 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 10 client.6033 fetch_fsmap learned FSMap version 36 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 -- 172.21.15.131:0/2967724780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55f8df5fdab0 con 0x55f8df6e4950 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f93537fe640 1 -- 172.21.15.131:0/2967724780 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f936405ebc0 con 0x55f8df6e4950 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f93537fe640 1 -- 172.21.15.131:0/2967724780 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f93640631e0 con 0x55f8df6e4950 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 10 client.6033 fetch_fsmap finished waiting for FSMap version 36 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 10 client.6033 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 20 client.6033 populate_metadata read hostname 'smithi131' 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55f8df2c7830 0x55f8df8698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 4 client.6033 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9382153000 1 -- 172.21.15.131:0/2967724780 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "status"}) -- 0x55f8df645540 con 0x55f8df2c7830 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.283+0000 7f9378c24640 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55f8df2c7830 0x55f8df8698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f93537fe640 10 client.6033 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:43:57.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9378c24640 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55f8df2c7830 0x55f8df8698f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=776269a65e848055 server_cookie=727854d4aa3383f in_seq=0 out_seq=0 2024-09-17T02:43:57.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f93537fe640 1 -- 172.21.15.131:0/2967724780 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55f8df645540 con 0x55f8df2c7830 2024-09-17T02:43:57.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f93537fe640 10 client.6033 handle_command_reply: tid=0 2024-09-17T02:43:57.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 1 client.6033 shutdown 2024-09-17T02:43:57.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 1 -- 172.21.15.131:0/2967724780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 msgr2=0x55f8df8693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df8693b0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f936405be30 tx=0x7f936405b060 comp rx=0 tx=0).stop 2024-09-17T02:43:57.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 1 -- 172.21.15.131:0/2967724780 shutdown_connections 2024-09-17T02:43:57.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f8df6e4950 0x55f8df8693b0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f8df6cdca0 0x55f8df84c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55f8df2c7830 0x55f8df8698f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f93537fe640 0 client.6033 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:43:57.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 1 --2- 172.21.15.131:0/2967724780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f8df2f4460 0x55f8df84bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 1 -- 172.21.15.131:0/2967724780 >> 172.21.15.131:0/2967724780 conn(0x55f8df7bad90 msgr2=0x55f8df836fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:57.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 1 -- 172.21.15.131:0/2967724780 shutdown_connections 2024-09-17T02:43:57.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 1 -- 172.21.15.131:0/2967724780 wait complete. 2024-09-17T02:43:57.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f9382153000 20 client.6033 trim_cache size 0 max 16384 2024-09-17T02:43:57.296 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:43:57.296 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:43:57.296 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:43:57.296 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 5871, 2024-09-17T02:43:57.296 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:43:57.297 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:43:57.297 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:43:57.297 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 10.600098951, 2024-09-17T02:43:57.297 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 36, 2024-09-17T02:43:57.297 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 65, 2024-09-17T02:43:57.298 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 65, 2024-09-17T02:43:57.298 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 15.228708615 2024-09-17T02:43:57.298 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:43:57.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 -- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9360073230 msgr2=0x7f9360075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9360073230 0x7f9360075740 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f936c0af000 tx=0x7f936c01b5a0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 -- 172.21.15.131:0/2395836992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f937415bdb0 msgr2=0x7f9374071420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f937415bdb0 0x7f9374071420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f936400d640 tx=0x7f936400db00 comp rx=0 tx=0).stop 2024-09-17T02:43:57.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 -- 172.21.15.131:0/2395836992 shutdown_connections 2024-09-17T02:43:57.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9360073230 0x7f9360075740 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f937415bdb0 0x7f9374071420 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93740cd470 0x7f937414b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 --2- 172.21.15.131:0/2395836992 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f937406f4f0 0x7f93740c5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 -- 172.21.15.131:0/2395836992 >> 172.21.15.131:0/2395836992 conn(0x7f9374073d10 msgr2=0x7f93740b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:57.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 -- 172.21.15.131:0/2395836992 shutdown_connections 2024-09-17T02:43:57.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.287+0000 7f937ae88640 1 -- 172.21.15.131:0/2395836992 wait complete. 2024-09-17T02:43:57.310 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:43:57.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.455+0000 7ff132817640 1 -- 172.21.15.131:0/4039482031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff118003fe0 msgr2=0x7ff118004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.455+0000 7ff132817640 1 --2- 172.21.15.131:0/4039482031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff118003fe0 0x7ff118004810 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ff114009b40 tx=0x7ff11401d230 comp rx=0 tx=0).stop 2024-09-17T02:43:57.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.455+0000 7ff130db4640 1 -- 172.21.15.131:0/4039482031 reap_dead start 2024-09-17T02:43:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.455+0000 7ff132817640 1 -- 172.21.15.131:0/4039482031 shutdown_connections 2024-09-17T02:43:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.455+0000 7ff132817640 1 -- 172.21.15.131:0/4039482031 >> 172.21.15.131:0/4039482031 conn(0x7ff12c0734f0 msgr2=0x7ff12c073900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.455+0000 7ff132817640 1 -- 172.21.15.131:0/4039482031 shutdown_connections 2024-09-17T02:43:57.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.455+0000 7ff132817640 1 -- 172.21.15.131:0/4039482031 wait complete. 2024-09-17T02:43:57.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff132817640 1 Processor -- start 2024-09-17T02:43:57.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff132817640 1 -- start start 2024-09-17T02:43:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff132817640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff118003fe0 0x7ff12c0cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff132817640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff12c1f8d70 0x7ff12c1f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff132817640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff12c1f96c0 0x7ff12c0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff132817640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c0cacb0 con 0x7ff12c1f8d70 2024-09-17T02:43:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff132817640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c0c6010 con 0x7ff118003fe0 2024-09-17T02:43:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff132817640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c0cae30 con 0x7ff12c1f96c0 2024-09-17T02:43:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff123fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff12c1f8d70 0x7ff12c1f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff123fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff12c1f8d70 0x7ff12c1f9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51472/0 (socket says 172.21.15.131:51472) 2024-09-17T02:43:57.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff123fff640 1 -- 172.21.15.131:0/2674830358 learned_addr learned my addr 172.21.15.131:0/2674830358 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:43:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff12bfff640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff118003fe0 0x7ff12c0cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff130db4640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff12c1f96c0 0x7ff12c0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff123fff640 1 -- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff12c1f96c0 msgr2=0x7ff12c0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff123fff640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff12c1f96c0 0x7ff12c0c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff123fff640 1 -- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff118003fe0 msgr2=0x7ff12c0cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff123fff640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff118003fe0 0x7ff12c0cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff123fff640 1 -- 172.21.15.131:0/2674830358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0aeab0 con 0x7ff12c1f8d70 2024-09-17T02:43:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff123fff640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff12c1f8d70 0x7ff12c1f9180 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff1140ae040 tx=0x7ff114005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff1297fa640 1 -- 172.21.15.131:0/2674830358 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1140bb030 con 0x7ff12c1f8d70 2024-09-17T02:43:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff1297fa640 1 -- 172.21.15.131:0/2674830358 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff114004900 con 0x7ff12c1f8d70 2024-09-17T02:43:57.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff132817640 1 -- 172.21.15.131:0/2674830358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c0aed90 con 0x7ff12c1f8d70 2024-09-17T02:43:57.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff1297fa640 1 -- 172.21.15.131:0/2674830358 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114004d10 con 0x7ff12c1f8d70 2024-09-17T02:43:57.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.459+0000 7ff132817640 1 -- 172.21.15.131:0/2674830358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c0c8c70 con 0x7ff12c1f8d70 2024-09-17T02:43:57.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.463+0000 7ff132817640 1 -- 172.21.15.131:0/2674830358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12c0c8780 con 0x7ff12c1f8d70 2024-09-17T02:43:57.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.463+0000 7ff1297fa640 1 -- 172.21.15.131:0/2674830358 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff1140be030 con 0x7ff12c1f8d70 2024-09-17T02:43:57.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.463+0000 7ff1297fa640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff10c073200 0x7ff10c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:43:57.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.463+0000 7ff1297fa640 1 -- 172.21.15.131:0/2674830358 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7ff11412f8d0 con 0x7ff12c1f8d70 2024-09-17T02:43:57.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.463+0000 7ff12bfff640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff10c073200 0x7ff10c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:43:57.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.463+0000 7ff12bfff640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff10c073200 0x7ff10c075710 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009b30 tx=0x7ff11c0091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:43:57.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.467+0000 7ff1297fa640 1 -- 172.21.15.131:0/2674830358 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff114016f80 con 0x7ff12c1f8d70 2024-09-17T02:43:57.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.667+0000 7ff132817640 1 -- 172.21.15.131:0/2674830358 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff12c0cb390 con 0x7ff12c1f8d70 2024-09-17T02:43:57.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.667+0000 7ff1297fa640 1 -- 172.21.15.131:0/2674830358 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7ff1140fa880 con 0x7ff12c1f8d70 2024-09-17T02:43:57.669 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:43:57.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.667+0000 7ff132817640 1 -- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff10c073200 msgr2=0x7ff10c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.667+0000 7ff132817640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff10c073200 0x7ff10c075710 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009b30 tx=0x7ff11c0091e0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.667+0000 7ff132817640 1 -- 172.21.15.131:0/2674830358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff12c1f8d70 msgr2=0x7ff12c1f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:43:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.667+0000 7ff132817640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff12c1f8d70 0x7ff12c1f9180 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ff1140ae040 tx=0x7ff114005020 comp rx=0 tx=0).stop 2024-09-17T02:43:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.671+0000 7ff132817640 1 -- 172.21.15.131:0/2674830358 shutdown_connections 2024-09-17T02:43:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.671+0000 7ff132817640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff10c073200 0x7ff10c075710 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.671+0000 7ff132817640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff12c1f96c0 0x7ff12c0c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.671+0000 7ff132817640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff12c1f8d70 0x7ff12c1f9180 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.671+0000 7ff132817640 1 --2- 172.21.15.131:0/2674830358 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff118003fe0 0x7ff12c0cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:43:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.671+0000 7ff132817640 1 -- 172.21.15.131:0/2674830358 >> 172.21.15.131:0/2674830358 conn(0x7ff12c0734f0 msgr2=0x7ff12c0c61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:43:57.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.671+0000 7ff132817640 1 -- 172.21.15.131:0/2674830358 shutdown_connections 2024-09-17T02:43:57.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57.671+0000 7ff132817640 1 -- 172.21.15.131:0/2674830358 wait complete. 2024-09-17T02:43:57.690 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:43:57.690 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:43:57.690 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:43:57.691 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:43:57.691 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:43:57.691 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:43:57.691 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:43:57.691 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:43:57.691 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:43:57.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:57.691 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:43:57.697 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:43:57.697 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:43:57.697 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:43:57.697 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:43:57.697 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:43:57.697 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:43:57.697 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:43:57.697 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:43:57.697 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42268sec preferred_lft 42268sec 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:43:57.698 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:43:57.699 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:43:57.700 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:43:57.700 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:43:57.700 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:43:57.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:57.700 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:43:57.749 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:43:57.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:57.750 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:43:57.800 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:43:57.800 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:43:57.800 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:43:57.800 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:43:57.800 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:43:57.800 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:43:57.800 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:43:57.800 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:43:57.801 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:43:57.801 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:43:57.801 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:43:57.801 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:43:57.801 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:43:57.801 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:43:57.801 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:43:57.801 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:43:57.801 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:43:57.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:43:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:43:57.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:57.917 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:43:57.964 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:43:57.964 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:43:57.964 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:43:57.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:57.965 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:43:58.020 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:43:58.021 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:43:58.021 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:43:58.068 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:43:58.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:43:58.069 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:43:58.116 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:43:58.174 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:43:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:43:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:43:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:43:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:43:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:43:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:43:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:43:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:43:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:43:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:43:58.223 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:43:58.224 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:43:58.225 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:43:58.225 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:43:58.225 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:43:58.225 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:43:58.225 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:43:58.225 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:43:58.225 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:43:58.225 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:43:58.225 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:43:58.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:58.226 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:43:58.273 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:43:58.274 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:43:58.317 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:43:58.333 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:43:58.380 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:43:58.381 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:43:58.381 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:43:58.381 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:43:58.381 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:43:58.381 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:43:58.381 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:43:58.381 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:43:58.381 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:43:58.381 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:43:58.382 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:43:58.383 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:43:58.383 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:43:58.383 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:43:58.383 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:43:58.383 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:43:58.383 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:43:58.383 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:43:58.383 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:43:58.383 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:43:58.383 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:43:58.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:58.384 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:43:58.427 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:43:58.423+0000 7f042fc02580 -1 init, newargv = 0x7f0420001f10 newargc=15 2024-09-17T02:43:58.427 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[41958]: starting ceph client 2024-09-17T02:43:58.435 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[41958]: starting fuse 2024-09-17T02:43:59.533 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:43:59.549 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:43:59.597 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:43:59.597 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:43:59.597 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:43:59.598 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:43:59.599 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:43:59.600 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:43:59.600 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:43:59.600 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:43:59.600 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:43:59.600 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:43:59.600 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:43:59.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:59.604 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:43:59.648 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:43:59.649 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:43:59.649 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:43:59.649 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:43:59.649 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:43:59.649 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:43:59.649 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:43:59.649 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:43:59.649 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:43:59.649 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:43:59.649 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:43:59.649 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:43:59.650 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:43:59.651 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:43:59.651 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:43:59.771 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.41958.asok 2024-09-17T02:43:59.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:43:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:43:59.782 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41958.asok 2024-09-17T02:43:59.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:59.783 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41958.asok status 2024-09-17T02:43:59.963 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:43:59.963 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:43:59.963 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:43:59.963 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:43:59.963 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:43:59.963 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:43:59.963 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:43:59.963 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "41958", 2024-09-17T02:43:59.963 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:43:59.963 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6048, 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 6048 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3668983343 2024-09-17T02:43:59.964 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3668983343 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.6048 192.168.0.1:0/3668983343", 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/3668983343", 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 36, 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 65, 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:43:59.965 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:43:59.966 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:43:59.966 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:43:59.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:59.979 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:43:59.979 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:43:59.985 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:43:59.985 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:43:59.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:43:59.985 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:44:00.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:44:00.042 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-09-17T02:44:00.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:44:00.102 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-09-17T02:44:00.156 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:44:00.156 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:44:00.156 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:44:00.156 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:44:00.156 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-09-17T02:44:00.156 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:44:00.267 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627777 2024-09-17T02:44:00.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:44:00.277 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:44:00.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.479+0000 7fc5fd878640 1 -- 172.21.15.131:0/1492707232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f80706c0 msgr2=0x7fc5f8070ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:00.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.479+0000 7fc5fd878640 1 --2- 172.21.15.131:0/1492707232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f80706c0 0x7fc5f8070ad0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8073d30 tx=0x7fc5e801c890 comp rx=0 tx=0).stop 2024-09-17T02:44:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.479+0000 7fc5fd878640 1 -- 172.21.15.131:0/1492707232 shutdown_connections 2024-09-17T02:44:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.479+0000 7fc5fd878640 1 --2- 172.21.15.131:0/1492707232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f80706c0 0x7fc5f8070ad0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.479+0000 7fc5fd878640 1 --2- 172.21.15.131:0/1492707232 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5f80cca90 0x7fc5f80ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.479+0000 7fc5fd878640 1 --2- 172.21.15.131:0/1492707232 >> v2:172.21.15.189:3300/0 conn(0x7fc5f815c740 0x7fc5f80d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.479+0000 7fc5fd878640 1 -- 172.21.15.131:0/1492707232 >> 172.21.15.131:0/1492707232 conn(0x7fc5f80cf0b0 msgr2=0x7fc5f80cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.479+0000 7fc5fd878640 1 -- 172.21.15.131:0/1492707232 shutdown_connections 2024-09-17T02:44:00.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.479+0000 7fc5fd878640 1 -- 172.21.15.131:0/1492707232 wait complete. 2024-09-17T02:44:00.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.479+0000 7fc5fd878640 1 Processor -- start 2024-09-17T02:44:00.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.479+0000 7fc5fd878640 1 -- start start 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5fd878640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5f80706c0 0x7fc5f814f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5fd878640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5f80cca90 0x7fc5f8156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5fd878640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f815c740 0x7fc5f8156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5fd878640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f80b66b0 con 0x7fc5f815c740 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5fd878640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc5f80b6530 con 0x7fc5f80706c0 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5fd878640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc5f80b6830 con 0x7fc5f80cca90 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5f77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f815c740 0x7fc5f8156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5f77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f815c740 0x7fc5f8156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51488/0 (socket says 172.21.15.131:51488) 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5f77fe640 1 -- 172.21.15.131:0/3102221121 learned_addr learned my addr 172.21.15.131:0/3102221121 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5f6ffd640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5f80706c0 0x7fc5f814f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5f67fc640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5f80cca90 0x7fc5f8156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:00.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5f77fe640 1 -- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5f80cca90 msgr2=0x7fc5f8156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5f77fe640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5f80cca90 0x7fc5f8156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5f77fe640 1 -- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5f80706c0 msgr2=0x7fc5f814f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5f77fe640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5f80706c0 0x7fc5f814f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5f77fe640 1 -- 172.21.15.131:0/3102221121 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5f81f9b60 con 0x7fc5f815c740 2024-09-17T02:44:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5f77fe640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f815c740 0x7fc5f8156d70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80b0040 tx=0x7fc5e80a5890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5dffff640 1 -- 172.21.15.131:0/3102221121 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80b2040 con 0x7fc5f815c740 2024-09-17T02:44:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5dffff640 1 -- 172.21.15.131:0/3102221121 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5e8005030 con 0x7fc5f815c740 2024-09-17T02:44:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5dffff640 1 -- 172.21.15.131:0/3102221121 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8005440 con 0x7fc5f815c740 2024-09-17T02:44:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5fd878640 1 -- 172.21.15.131:0/3102221121 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5f81f9e40 con 0x7fc5f815c740 2024-09-17T02:44:00.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5fd878640 1 -- 172.21.15.131:0/3102221121 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5f81fa400 con 0x7fc5f815c740 2024-09-17T02:44:00.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5fd878640 1 -- 172.21.15.131:0/3102221121 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5f80ccea0 con 0x7fc5f815c740 2024-09-17T02:44:00.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.483+0000 7fc5dffff640 1 -- 172.21.15.131:0/3102221121 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc5e80046d0 con 0x7fc5f815c740 2024-09-17T02:44:00.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.487+0000 7fc5dffff640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5e4073230 0x7fc5e4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.487+0000 7fc5dffff640 1 -- 172.21.15.131:0/3102221121 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fc5e812e1a0 con 0x7fc5f815c740 2024-09-17T02:44:00.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.487+0000 7fc5f6ffd640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5e4073230 0x7fc5e4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:00.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.487+0000 7fc5f6ffd640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5e4073230 0x7fc5e4075740 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009e50 tx=0x7fc5f00095e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:00.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.491+0000 7fc5dffff640 1 -- 172.21.15.131:0/3102221121 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc5e80164b0 con 0x7fc5f815c740 2024-09-17T02:44:00.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.683+0000 7fc5fd878640 1 -- 172.21.15.131:0/3102221121 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5f81f7470 con 0x7fc5f815c740 2024-09-17T02:44:00.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5dffff640 1 -- 172.21.15.131:0/3102221121 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5233 (secure 0 0 0) 0x7fc5e80f9110 con 0x7fc5f815c740 2024-09-17T02:44:00.688 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:44:00.688 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":36,"btime":"2024-09-17T02:43:49:702553+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3554877374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3554877374},{"type":"v1","addr":"172.21.15.189:6837","nonce":3554877374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1670590113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1670590113},{"type":"v1","addr":"172.21.15.131:6834","nonce":1670590113}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1046159039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1046159039},{"type":"v1","addr":"172.21.15.131:6835","nonce":1046159039}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:43:46.061550+0000","modified":"2024-09-17T02:43:49.109393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5871":{"gid":5871,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.189:6835/730392436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":730392436},{"type":"v1","addr":"172.21.15.189:6835","nonce":730392436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5871,"qdb_cluster":[5871]},"id":4}]} 2024-09-17T02:44:00.688 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 36 2024-09-17T02:44:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 -- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5e4073230 msgr2=0x7fc5e4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5e4073230 0x7fc5e4075740 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009e50 tx=0x7fc5f00095e0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 -- 172.21.15.131:0/3102221121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f815c740 msgr2=0x7fc5f8156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f815c740 0x7fc5f8156d70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80b0040 tx=0x7fc5e80a5890 comp rx=0 tx=0).stop 2024-09-17T02:44:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 -- 172.21.15.131:0/3102221121 shutdown_connections 2024-09-17T02:44:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5e4073230 0x7fc5e4075740 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5f815c740 0x7fc5f8156d70 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5f80cca90 0x7fc5f8156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 --2- 172.21.15.131:0/3102221121 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5f80706c0 0x7fc5f814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 -- 172.21.15.131:0/3102221121 >> 172.21.15.131:0/3102221121 conn(0x7fc5f80cf0b0 msgr2=0x7fc5f80b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:00.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 -- 172.21.15.131:0/3102221121 shutdown_connections 2024-09-17T02:44:00.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.687+0000 7fc5fd878640 1 -- 172.21.15.131:0/3102221121 wait complete. 2024-09-17T02:44:00.708 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-09-17T02:44:00.902 INFO:teuthology.orchestra.run.smithi189.stdout:{"message":"","return_code":0} 2024-09-17T02:44:00.902 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T02:44:00.902 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-17T02:44:00.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 Processor -- start 2024-09-17T02:44:00.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 -- start start 2024-09-17T02:44:00.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 0x55fb86615130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb86585770 0x55fb8660fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fb86612540 0x55fb86610090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fb8648aa00 con 0x55fb8656ae90 2024-09-17T02:44:00.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fb864e8f40 con 0x55fb86585770 2024-09-17T02:44:00.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fb8656a7a0 con 0x55fb86612540 2024-09-17T02:44:00.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afd7f1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 0x55fb86615130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:00.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afd7f1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 0x55fb86615130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51494/0 (socket says 172.21.15.131:51494) 2024-09-17T02:44:00.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afd7f1640 1 -- 172.21.15.131:0/3130186559 learned_addr learned my addr 172.21.15.131:0/3130186559 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:00.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afd7f1640 1 -- 172.21.15.131:0/3130186559 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fb86612540 msgr2=0x55fb86610090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:00.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afdff2640 1 --2- 172.21.15.131:0/3130186559 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fb86612540 0x55fb86610090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:00.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afcff0640 1 --2- 172.21.15.131:0/3130186559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb86585770 0x55fb8660fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:00.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afd7f1640 1 --2- 172.21.15.131:0/3130186559 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fb86612540 0x55fb86610090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afd7f1640 1 -- 172.21.15.131:0/3130186559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb86585770 msgr2=0x55fb8660fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:00.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afd7f1640 1 --2- 172.21.15.131:0/3130186559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb86585770 0x55fb8660fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afd7f1640 1 -- 172.21.15.131:0/3130186559 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb86463c10 con 0x55fb8656ae90 2024-09-17T02:44:00.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afdff2640 1 --2- 172.21.15.131:0/3130186559 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fb86612540 0x55fb86610090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4afd7f1640 1 --2- 172.21.15.131:0/3130186559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 0x55fb86615130 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4af800b200 tx=0x7f4af800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec5bc73f49a3386 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4af77fe640 1 -- 172.21.15.131:0/3130186559 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8016070 con 0x55fb8656ae90 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4af77fe640 1 -- 172.21.15.131:0/3130186559 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4af800d4c0 con 0x55fb8656ae90 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4af77fe640 1 -- 172.21.15.131:0/3130186559 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af800d750 con 0x55fb8656ae90 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 -- 172.21.15.131:0/3130186559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 msgr2=0x55fb86615130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 --2- 172.21.15.131:0/3130186559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 0x55fb86615130 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4af800b200 tx=0x7f4af800b6c0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 -- 172.21.15.131:0/3130186559 shutdown_connections 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 --2- 172.21.15.131:0/3130186559 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fb86612540 0x55fb86610090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 --2- 172.21.15.131:0/3130186559 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb86585770 0x55fb8660fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 --2- 172.21.15.131:0/3130186559 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 0x55fb86615130 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 -- 172.21.15.131:0/3130186559 >> 172.21.15.131:0/3130186559 conn(0x55fb86528d00 msgr2=0x55fb865682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 -- 172.21.15.131:0/3130186559 shutdown_connections 2024-09-17T02:44:00.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 -- 172.21.15.131:0/3130186559 wait complete. 2024-09-17T02:44:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.971+0000 7f4b03d1a080 1 Processor -- start 2024-09-17T02:44:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4b03d1a080 1 -- start start 2024-09-17T02:44:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4b03d1a080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 0x55fb86610ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4b03d1a080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fb86585770 0x55fb8660ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4b03d1a080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb86612540 0x55fb8660f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4b03d1a080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fb8660bd40 con 0x55fb8656ae90 2024-09-17T02:44:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4b03d1a080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fb8660bfe0 con 0x55fb86612540 2024-09-17T02:44:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4b03d1a080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fb8660e4f0 con 0x55fb86585770 2024-09-17T02:44:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4afd7f1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 0x55fb86610ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4afd7f1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 0x55fb86610ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51496/0 (socket says 172.21.15.131:51496) 2024-09-17T02:44:00.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4afd7f1640 1 -- 172.21.15.131:0/4150112524 learned_addr learned my addr 172.21.15.131:0/4150112524 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:00.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4afd7f1640 1 -- 172.21.15.131:0/4150112524 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fb86585770 msgr2=0x55fb8660ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:00.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4afd7f1640 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fb86585770 0x55fb8660ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4afd7f1640 1 -- 172.21.15.131:0/4150112524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb86612540 msgr2=0x55fb8660f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:00.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4afd7f1640 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fb86612540 0x55fb8660f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4afd7f1640 1 -- 172.21.15.131:0/4150112524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb86609b20 con 0x55fb8656ae90 2024-09-17T02:44:00.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4afd7f1640 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 0x55fb86610ba0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4af8000f70 tx=0x7f4af8010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:00.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4af67fc640 1 -- 172.21.15.131:0/4150112524 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af8016040 con 0x55fb8656ae90 2024-09-17T02:44:00.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4af67fc640 1 -- 172.21.15.131:0/4150112524 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4af8020070 con 0x55fb8656ae90 2024-09-17T02:44:00.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4b03d1a080 1 -- 172.21.15.131:0/4150112524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fb86609dc0 con 0x55fb8656ae90 2024-09-17T02:44:00.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4af67fc640 1 -- 172.21.15.131:0/4150112524 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af800d6e0 con 0x55fb8656ae90 2024-09-17T02:44:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4b03d1a080 1 -- 172.21.15.131:0/4150112524 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb866110e0 con 0x55fb8656ae90 2024-09-17T02:44:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.975+0000 7f4af67fc640 1 -- 172.21.15.131:0/4150112524 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4af800d880 con 0x55fb8656ae90 2024-09-17T02:44:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4af67fc640 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4ae0071be0 0x7f4ae00740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4af67fc640 1 -- 172.21.15.131:0/4150112524 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f4af8093e30 con 0x55fb8656ae90 2024-09-17T02:44:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55fb8657ff90 0x55fb865803a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 -- 172.21.15.131:0/4150112524 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65) -- 0x55fb865808e0 con 0x55fb8657ff90 2024-09-17T02:44:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4afcff0640 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4ae0071be0 0x7f4ae00740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:00.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4afdff2640 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55fb8657ff90 0x55fb865803a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4afdff2640 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55fb8657ff90 0x55fb865803a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:00.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4afdff2640 1 -- 172.21.15.131:0/4150112524 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7f4af0009430 con 0x55fb8657ff90 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4afcff0640 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4ae0071be0 0x7f4ae00740f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80ac000 tx=0x7f4ae8018240 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 -- 172.21.15.131:0/4150112524 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55fb8657ff90 msgr2=0x55fb865803a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55fb8657ff90 0x55fb865803a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 -- 172.21.15.131:0/4150112524 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4ae0071be0 msgr2=0x7f4ae00740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4ae0071be0 0x7f4ae00740f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80ac000 tx=0x7f4ae8018240 comp rx=0 tx=0).stop 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 -- 172.21.15.131:0/4150112524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 msgr2=0x55fb86610ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 --2- 172.21.15.131:0/4150112524 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fb8656ae90 0x55fb86610ba0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4af8000f70 tx=0x7f4af8010f20 comp rx=0 tx=0).stop 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4afdff2640 1 -- 172.21.15.131:0/4150112524 reap_dead start 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 -- 172.21.15.131:0/4150112524 shutdown_connections 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 -- 172.21.15.131:0/4150112524 >> 172.21.15.131:0/4150112524 conn(0x55fb86528d00 msgr2=0x55fb865682e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 -- 172.21.15.131:0/4150112524 shutdown_connections 2024-09-17T02:44:00.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:00.979+0000 7f4b03d1a080 1 -- 172.21.15.131:0/4150112524 wait complete. 2024-09-17T02:44:00.985 INFO:teuthology.orchestra.run.smithi131.stdout:Xfiledir_test 2024-09-17T02:44:00.986 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:44:00.987 DEBUG:teuthology.orchestra.run.smithi131:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-17T02:44:01.039 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:44:01.039 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 1099511627777, 2024-09-17T02:44:01.039 INFO:teuthology.orchestra.run.smithi131.stdout: "ancestors": [ 2024-09-17T02:44:01.039 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:44:01.039 INFO:teuthology.orchestra.run.smithi131.stdout: "dirino": 1099511627776, 2024-09-17T02:44:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: "dname": "file", 2024-09-17T02:44:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: "version": 2 2024-09-17T02:44:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:44:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:44:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: "dirino": 1, 2024-09-17T02:44:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: "dname": "dir_test", 2024-09-17T02:44:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: "version": 4 2024-09-17T02:44:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:44:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-17T02:44:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: "pool": 10, 2024-09-17T02:44:01.040 INFO:teuthology.orchestra.run.smithi131.stdout: "old_pools": [] 2024-09-17T02:44:01.041 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:44:01.041 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:44:01.044 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-09-17T02:44:01.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab6f7e080 1 Processor -- start 2024-09-17T02:44:01.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab6f7e080 1 -- start start 2024-09-17T02:44:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab6f7e080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b4ce45fc60 0x55b4ce3d5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab6f7e080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ce3d4110 0x55b4ce2feac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab6f7e080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 0x55b4ce461370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab6f7e080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b4ce24fa00 con 0x55b4ce2ff000 2024-09-17T02:44:01.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab6f7e080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b4ce2adf40 con 0x55b4ce3d4110 2024-09-17T02:44:01.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab6f7e080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b4ce349f90 con 0x55b4ce45fc60 2024-09-17T02:44:01.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab1256640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 0x55b4ce461370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab1256640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 0x55b4ce461370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57424/0 (socket says 172.21.15.131:57424) 2024-09-17T02:44:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab1256640 1 -- 172.21.15.131:0/3362660352 learned_addr learned my addr 172.21.15.131:0/3362660352 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab0a55640 1 --2- 172.21.15.131:0/3362660352 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b4ce45fc60 0x55b4ce3d5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab0254640 1 --2- 172.21.15.131:0/3362660352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ce3d4110 0x55b4ce2feac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab1256640 1 -- 172.21.15.131:0/3362660352 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b4ce45fc60 msgr2=0x55b4ce3d5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab1256640 1 --2- 172.21.15.131:0/3362660352 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b4ce45fc60 0x55b4ce3d5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab1256640 1 -- 172.21.15.131:0/3362660352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ce3d4110 msgr2=0x55b4ce2feac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab1256640 1 --2- 172.21.15.131:0/3362660352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ce3d4110 0x55b4ce2feac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab1256640 1 -- 172.21.15.131:0/3362660352 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ce228c10 con 0x55b4ce2ff000 2024-09-17T02:44:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcab1256640 1 --2- 172.21.15.131:0/3362660352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 0x55b4ce461370 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000b5e0 tx=0x7fca9c004130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bb7dc0af8ee35e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcaaf252640 1 -- 172.21.15.131:0/3362660352 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00a57d0 con 0x55b4ce2ff000 2024-09-17T02:44:01.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcaaf252640 1 -- 172.21.15.131:0/3362660352 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaa00b5070 con 0x55b4ce2ff000 2024-09-17T02:44:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.155+0000 7fcaaf252640 1 -- 172.21.15.131:0/3362660352 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa00a5ce0 con 0x55b4ce2ff000 2024-09-17T02:44:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 -- 172.21.15.131:0/3362660352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 msgr2=0x55b4ce461370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 --2- 172.21.15.131:0/3362660352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 0x55b4ce461370 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000b5e0 tx=0x7fca9c004130 comp rx=0 tx=0).stop 2024-09-17T02:44:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 -- 172.21.15.131:0/3362660352 shutdown_connections 2024-09-17T02:44:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 --2- 172.21.15.131:0/3362660352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 0x55b4ce461370 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 --2- 172.21.15.131:0/3362660352 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ce3d4110 0x55b4ce2feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 --2- 172.21.15.131:0/3362660352 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b4ce45fc60 0x55b4ce3d5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 -- 172.21.15.131:0/3362660352 >> 172.21.15.131:0/3362660352 conn(0x55b4ce2edd00 msgr2=0x55b4ce2ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 -- 172.21.15.131:0/3362660352 shutdown_connections 2024-09-17T02:44:01.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 -- 172.21.15.131:0/3362660352 wait complete. 2024-09-17T02:44:01.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 Processor -- start 2024-09-17T02:44:01.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 -- start start 2024-09-17T02:44:01.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 0x55b4ce3d2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b4ce3d4110 0x55b4ce3bdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ce45fc60 0x55b4ce3c0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b4ce3d3f20 con 0x55b4ce2ff000 2024-09-17T02:44:01.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b4ce3d50c0 con 0x55b4ce45fc60 2024-09-17T02:44:01.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b4ce3c0230 con 0x55b4ce3d4110 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab0a55640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 0x55b4ce3d2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab0a55640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 0x55b4ce3d2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57432/0 (socket says 172.21.15.131:57432) 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab0a55640 1 -- 172.21.15.131:0/1638100843 learned_addr learned my addr 172.21.15.131:0/1638100843 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab0a55640 1 -- 172.21.15.131:0/1638100843 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b4ce3d4110 msgr2=0x55b4ce3bdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab1256640 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ce45fc60 0x55b4ce3c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab0a55640 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b4ce3d4110 0x55b4ce3bdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab0a55640 1 -- 172.21.15.131:0/1638100843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ce45fc60 msgr2=0x55b4ce3c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab0a55640 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b4ce45fc60 0x55b4ce3c0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab0a55640 1 -- 172.21.15.131:0/1638100843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4ce3c11e0 con 0x55b4ce2ff000 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab0a55640 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 0x55b4ce3d2350 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800a650 tx=0x7fcaa800ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcaae250640 1 -- 172.21.15.131:0/1638100843 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa801b070 con 0x55b4ce2ff000 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcaae250640 1 -- 172.21.15.131:0/1638100843 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaa80099b0 con 0x55b4ce2ff000 2024-09-17T02:44:01.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 -- 172.21.15.131:0/1638100843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b4ce3bff90 con 0x55b4ce2ff000 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcaae250640 1 -- 172.21.15.131:0/1638100843 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa800d2d0 con 0x55b4ce2ff000 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.159+0000 7fcab6f7e080 1 -- 172.21.15.131:0/1638100843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4ce3d28d0 con 0x55b4ce2ff000 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.163+0000 7fcaae250640 1 -- 172.21.15.131:0/1638100843 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcaa801e070 con 0x55b4ce2ff000 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.163+0000 7fcaae250640 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fca90071be0 0x7fca900740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.163+0000 7fcaae250640 1 -- 172.21.15.131:0/1638100843 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fcaa80939d0 con 0x55b4ce2ff000 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.163+0000 7fcab6f7e080 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55b4ce478e70 0x55b4ce47b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.163+0000 7fcab6f7e080 1 -- 172.21.15.131:0/1638100843 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65) -- 0x55b4ce3c28d0 con 0x55b4ce478e70 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.163+0000 7fcab0254640 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fca90071be0 0x7fca900740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.163+0000 7fcab1256640 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55b4ce478e70 0x55b4ce47b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.163+0000 7fcab1256640 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55b4ce478e70 0x55b4ce47b290 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.163+0000 7fcab0254640 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fca90071be0 0x7fca900740f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fca98001df0 tx=0x7fca98001d20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab1256640 1 -- 172.21.15.131:0/1638100843 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v65'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcaa00ad020 con 0x55b4ce478e70 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab6f7e080 1 -- 172.21.15.131:0/1638100843 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55b4ce478e70 msgr2=0x55b4ce47b290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab6f7e080 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55b4ce478e70 0x55b4ce47b290 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab6f7e080 1 -- 172.21.15.131:0/1638100843 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fca90071be0 msgr2=0x7fca900740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab6f7e080 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fca90071be0 0x7fca900740f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fca98001df0 tx=0x7fca98001d20 comp rx=0 tx=0).stop 2024-09-17T02:44:01.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab6f7e080 1 -- 172.21.15.131:0/1638100843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 msgr2=0x55b4ce3d2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab6f7e080 1 --2- 172.21.15.131:0/1638100843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b4ce2ff000 0x55b4ce3d2350 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800a650 tx=0x7fcaa800ab10 comp rx=0 tx=0).stop 2024-09-17T02:44:01.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab1256640 1 -- 172.21.15.131:0/1638100843 reap_dead start 2024-09-17T02:44:01.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab6f7e080 1 -- 172.21.15.131:0/1638100843 shutdown_connections 2024-09-17T02:44:01.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab6f7e080 1 -- 172.21.15.131:0/1638100843 >> 172.21.15.131:0/1638100843 conn(0x55b4ce2edd00 msgr2=0x55b4ce3d5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab6f7e080 1 -- 172.21.15.131:0/1638100843 shutdown_connections 2024-09-17T02:44:01.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.167+0000 7fcab6f7e080 1 -- 172.21.15.131:0/1638100843 wait complete. 2024-09-17T02:44:01.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:44:01.176 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-09-17T02:44:01.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.327+0000 7ff36a233640 1 -- 172.21.15.131:0/4238217515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3640cd880 msgr2=0x7ff364074460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.327+0000 7ff36a233640 1 --2- 172.21.15.131:0/4238217515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3640cd880 0x7ff364074460 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff36415a5c0 tx=0x7ff35401c6e0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.327+0000 7ff36a233640 1 -- 172.21.15.131:0/4238217515 shutdown_connections 2024-09-17T02:44:01.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.327+0000 7ff36a233640 1 --2- 172.21.15.131:0/4238217515 >> v2:172.21.15.189:3300/0 conn(0x7ff3640cdc90 0x7ff3640749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.327+0000 7ff36a233640 1 --2- 172.21.15.131:0/4238217515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3640cd880 0x7ff364074460 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.327+0000 7ff36a233640 1 -- 172.21.15.131:0/4238217515 >> 172.21.15.131:0/4238217515 conn(0x7ff364070bb0 msgr2=0x7ff364073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.327+0000 7ff36a233640 1 -- 172.21.15.131:0/4238217515 shutdown_connections 2024-09-17T02:44:01.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.327+0000 7ff36a233640 1 -- 172.21.15.131:0/4238217515 wait complete. 2024-09-17T02:44:01.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff36a233640 1 Processor -- start 2024-09-17T02:44:01.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff36a233640 1 -- start start 2024-09-17T02:44:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff36a233640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3640cd880 0x7ff3640b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff36a233640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3640cdc90 0x7ff3641668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff36a233640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff364160070 0x7ff364157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff36a233640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff364169da0 con 0x7ff3640cdc90 2024-09-17T02:44:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff36a233640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff364169c20 con 0x7ff3640cd880 2024-09-17T02:44:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff36a233640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff364169f20 con 0x7ff364160070 2024-09-17T02:44:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3640cdc90 0x7ff3641668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3640cdc90 0x7ff3641668d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57444/0 (socket says 172.21.15.131:57444) 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff362ffd640 1 -- 172.21.15.131:0/1476163307 learned_addr learned my addr 172.21.15.131:0/1476163307 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff363fff640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff364160070 0x7ff364157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff3637fe640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3640cd880 0x7ff3640b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff362ffd640 1 -- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff364160070 msgr2=0x7ff364157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff362ffd640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff364160070 0x7ff364157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff362ffd640 1 -- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3640cd880 msgr2=0x7ff3640b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff362ffd640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3640cd880 0x7ff3640b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff362ffd640 1 -- 172.21.15.131:0/1476163307 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3640b4d00 con 0x7ff3640cdc90 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff363fff640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff364160070 0x7ff364157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff3637fe640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3640cd880 0x7ff3640b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff362ffd640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3640cdc90 0x7ff3641668d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff35800c970 tx=0x7ff35800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff34bfff640 1 -- 172.21.15.131:0/1476163307 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358014070 con 0x7ff3640cdc90 2024-09-17T02:44:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff34bfff640 1 -- 172.21.15.131:0/1476163307 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff358007740 con 0x7ff3640cdc90 2024-09-17T02:44:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff36a233640 1 -- 172.21.15.131:0/1476163307 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3640b4fe0 con 0x7ff3640cdc90 2024-09-17T02:44:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff34bfff640 1 -- 172.21.15.131:0/1476163307 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358007b90 con 0x7ff3640cdc90 2024-09-17T02:44:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.331+0000 7ff36a233640 1 -- 172.21.15.131:0/1476163307 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3641663b0 con 0x7ff3640cdc90 2024-09-17T02:44:01.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff34bfff640 1 -- 172.21.15.131:0/1476163307 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff358017070 con 0x7ff3640cdc90 2024-09-17T02:44:01.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff34bfff640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff350073230 0x7ff350075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff34bfff640 1 -- 172.21.15.131:0/1476163307 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7ff358094550 con 0x7ff3640cdc90 2024-09-17T02:44:01.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3637fe640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff350073230 0x7ff350075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3714fe000 1 Processor -- start 2024-09-17T02:44:01.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3714fe000 1 -- start start 2024-09-17T02:44:01.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3637fe640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff350073230 0x7ff350075740 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff3540ae000 tx=0x7ff35401af20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 0x556425d37db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bbedb0 0x556425d38830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3714fe000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556425aec460 con 0x556425bc5140 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3714fe000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556425b49ce0 con 0x556425bbedb0 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3714fe000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x556425b39110 con 0x556425bd8a50 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d382f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57454/0 (socket says 172.21.15.131:57454) 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff362ffd640 1 -- 172.21.15.131:0/814539577 learned_addr learned my addr 172.21.15.131:0/814539577 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff363fff640 1 --2- 172.21.15.131:0/814539577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bbedb0 0x556425d38830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3637fe640 1 --2- 172.21.15.131:0/814539577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 0x556425d37db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff362ffd640 1 -- 172.21.15.131:0/814539577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 msgr2=0x556425d37db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff362ffd640 1 --2- 172.21.15.131:0/814539577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 0x556425d37db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff362ffd640 1 -- 172.21.15.131:0/814539577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bbedb0 msgr2=0x556425d38830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff362ffd640 1 --2- 172.21.15.131:0/814539577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bbedb0 0x556425d38830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff362ffd640 1 -- 172.21.15.131:0/814539577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556425ad7340 con 0x556425bc5140 2024-09-17T02:44:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff363fff640 1 --2- 172.21.15.131:0/814539577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bbedb0 0x556425d38830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff362ffd640 1 --2- 172.21.15.131:0/814539577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d382f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff358092890 tx=0x7ff35805e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2789009d08eb303c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3497fa640 1 -- 172.21.15.131:0/814539577 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35801e030 con 0x556425bc5140 2024-09-17T02:44:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3497fa640 1 -- 172.21.15.131:0/814539577 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3580638e0 con 0x556425bc5140 2024-09-17T02:44:01.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.335+0000 7ff3497fa640 1 -- 172.21.15.131:0/814539577 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff358068700 con 0x556425bc5140 2024-09-17T02:44:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 -- 172.21.15.131:0/814539577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 msgr2=0x556425d382f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 --2- 172.21.15.131:0/814539577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d382f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff358092890 tx=0x7ff35805e910 comp rx=0 tx=0).stop 2024-09-17T02:44:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 -- 172.21.15.131:0/814539577 shutdown_connections 2024-09-17T02:44:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 --2- 172.21.15.131:0/814539577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bbedb0 0x556425d38830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 --2- 172.21.15.131:0/814539577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d382f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 --2- 172.21.15.131:0/814539577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 0x556425d37db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 -- 172.21.15.131:0/814539577 >> 172.21.15.131:0/814539577 conn(0x5564257f7590 msgr2=0x556425cecb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 -- 172.21.15.131:0/814539577 shutdown_connections 2024-09-17T02:44:01.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 -- 172.21.15.131:0/814539577 wait complete. 2024-09-17T02:44:01.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 -- start start 2024-09-17T02:44:01.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 Processor -- start 2024-09-17T02:44:01.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bbedb0 0x556425cf3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bc5140 0x556425cf4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 0x556425d23680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556425ade600 con 0x556425bbedb0 2024-09-17T02:44:01.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556425b3cf20 con 0x556425bc5140 2024-09-17T02:44:01.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5564257fe810 con 0x556425bd8a50 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3714fe000 10 client.0 _ll_register_callbacks cb 0x556425cd9cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3637fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bbedb0 0x556425cf3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3637fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bbedb0 0x556425cf3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57466/0 (socket says 172.21.15.131:57466) 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3637fe640 1 -- 172.21.15.131:0/2021247193 learned_addr learned my addr 172.21.15.131:0/2021247193 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff363fff640 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 0x556425d23680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff362ffd640 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bc5140 0x556425cf4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3637fe640 1 -- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 msgr2=0x556425d23680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3637fe640 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 0x556425d23680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3637fe640 1 -- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bc5140 msgr2=0x556425cf4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3637fe640 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bc5140 0x556425cf4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3637fe640 1 -- 172.21.15.131:0/2021247193 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556425abe1f0 con 0x556425bbedb0 2024-09-17T02:44:01.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3637fe640 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bbedb0 0x556425cf3c40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff35401cbc0 tx=0x7ff354009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f7ca376e1d9614 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.339+0000 7ff3427fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3427fc640 1 -- 172.21.15.131:0/2021247193 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3540ac040 con 0x556425bbedb0 2024-09-17T02:44:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3427fc640 1 -- 172.21.15.131:0/2021247193 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7ff3540a6b50 con 0x556425bbedb0 2024-09-17T02:44:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 -- 172.21.15.131:0/2021247193 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556425ad1480 con 0x556425bbedb0 2024-09-17T02:44:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3427fc640 1 -- 172.21.15.131:0/2021247193 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff3540a6d50 con 0x556425bbedb0 2024-09-17T02:44:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 10 client.6087 fetch_fsmap learned FSMap version 36 2024-09-17T02:44:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 -- 172.21.15.131:0/2021247193 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x556425bbe1f0 con 0x556425bbedb0 2024-09-17T02:44:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3427fc640 1 -- 172.21.15.131:0/2021247193 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556425ad1480 con 0x556425bbedb0 2024-09-17T02:44:01.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3427fc640 1 -- 172.21.15.131:0/2021247193 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7ff3540046d0 con 0x556425bbedb0 2024-09-17T02:44:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 10 client.6087 fetch_fsmap finished waiting for FSMap version 36 2024-09-17T02:44:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 10 client.6087 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:44:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 20 client.6087 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5564257dbac0 0x556425d24090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 4 client.6087 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:44:01.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 -- 172.21.15.131:0/2021247193 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556425b4bc30 con 0x5564257dbac0 2024-09-17T02:44:01.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff362ffd640 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5564257dbac0 0x556425d24090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3427fc640 10 client.6087 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:01.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff362ffd640 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5564257dbac0 0x556425d24090 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de5d299ff33e5cd4 server_cookie=6a2e7051f4e822dd in_seq=0 out_seq=0 2024-09-17T02:44:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3427fc640 1 -- 172.21.15.131:0/2021247193 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556425b4bc30 con 0x5564257dbac0 2024-09-17T02:44:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3427fc640 10 client.6087 handle_command_reply: tid=0 2024-09-17T02:44:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 client.6087 shutdown 2024-09-17T02:44:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 -- 172.21.15.131:0/2021247193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bbedb0 msgr2=0x556425cf3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bbedb0 0x556425cf3c40 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff35401cbc0 tx=0x7ff354009260 comp rx=0 tx=0).stop 2024-09-17T02:44:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 -- 172.21.15.131:0/2021247193 shutdown_connections 2024-09-17T02:44:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5564257dbac0 0x556425d24090 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 0x556425d23680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bc5140 0x556425cf4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 --2- 172.21.15.131:0/2021247193 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bbedb0 0x556425cf3c40 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3427fc640 0 client.6087 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:01.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.343+0000 7ff3714fe000 1 -- 172.21.15.131:0/2021247193 >> 172.21.15.131:0/2021247193 conn(0x5564257f7590 msgr2=0x556425ced2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.347+0000 7ff3714fe000 1 -- 172.21.15.131:0/2021247193 shutdown_connections 2024-09-17T02:44:01.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.347+0000 7ff3714fe000 1 -- 172.21.15.131:0/2021247193 wait complete. 2024-09-17T02:44:01.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.347+0000 7ff3714fe000 20 client.6087 trim_cache size 0 max 16384 2024-09-17T02:44:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.363+0000 7ff3714fe000 1 Processor -- start 2024-09-17T02:44:01.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.363+0000 7ff3714fe000 1 -- start start 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.363+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bbedb0 0x556425d4f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.363+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d4ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.363+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 0x556425cec150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.363+0000 7ff3714fe000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff3642059c0 con 0x556425bc5140 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.363+0000 7ff3714fe000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff364204a40 con 0x556425bbedb0 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.363+0000 7ff3714fe000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff364203e70 con 0x556425bd8a50 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.363+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d4ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.363+0000 7ff362ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d4ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57482/0 (socket says 172.21.15.131:57482) 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.363+0000 7ff362ffd640 1 -- 172.21.15.131:0/4180116197 learned_addr learned my addr 172.21.15.131:0/4180116197 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff362ffd640 1 -- 172.21.15.131:0/4180116197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 msgr2=0x556425cec150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff362ffd640 1 --2- 172.21.15.131:0/4180116197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 0x556425cec150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff362ffd640 1 -- 172.21.15.131:0/4180116197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bbedb0 msgr2=0x556425d4f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff362ffd640 1 --2- 172.21.15.131:0/4180116197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bbedb0 0x556425d4f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff362ffd640 1 -- 172.21.15.131:0/4180116197 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556425b954f0 con 0x556425bc5140 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff362ffd640 1 --2- 172.21.15.131:0/4180116197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d4ff30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff35805ed90 tx=0x7ff35805e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd5473f5920c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff348ff9640 1 -- 172.21.15.131:0/4180116197 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35801e070 con 0x556425bc5140 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff348ff9640 1 -- 172.21.15.131:0/4180116197 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff35804c020 con 0x556425bc5140 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff348ff9640 1 -- 172.21.15.131:0/4180116197 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35805b070 con 0x556425bc5140 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 -- 172.21.15.131:0/4180116197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 msgr2=0x556425d4ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 --2- 172.21.15.131:0/4180116197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d4ff30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff35805ed90 tx=0x7ff35805e910 comp rx=0 tx=0).stop 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 -- 172.21.15.131:0/4180116197 shutdown_connections 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 --2- 172.21.15.131:0/4180116197 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bd8a50 0x556425cec150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 --2- 172.21.15.131:0/4180116197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bc5140 0x556425d4ff30 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 --2- 172.21.15.131:0/4180116197 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bbedb0 0x556425d4f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 -- 172.21.15.131:0/4180116197 >> 172.21.15.131:0/4180116197 conn(0x5564257f7590 msgr2=0x556425d37070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 -- 172.21.15.131:0/4180116197 shutdown_connections 2024-09-17T02:44:01.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 -- 172.21.15.131:0/4180116197 wait complete. 2024-09-17T02:44:01.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 -- start start 2024-09-17T02:44:01.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 Processor -- start 2024-09-17T02:44:01.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bbedb0 0x556425db5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bc5140 0x556425db6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bd8a50 0x556425db6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556425b04df0 con 0x556425bd8a50 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556425b3c230 con 0x556425bc5140 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x556425b41550 con 0x556425bbedb0 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff3714fe000 10 client.0 _ll_register_callbacks cb 0x7ff354009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff363fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bd8a50 0x556425db6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff363fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bd8a50 0x556425db6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57490/0 (socket says 172.21.15.131:57490) 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff363fff640 1 -- 172.21.15.131:0/3344973652 learned_addr learned my addr 172.21.15.131:0/3344973652 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff363fff640 1 -- 172.21.15.131:0/3344973652 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bbedb0 msgr2=0x556425db5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff363fff640 1 --2- 172.21.15.131:0/3344973652 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bbedb0 0x556425db5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff363fff640 1 -- 172.21.15.131:0/3344973652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bc5140 msgr2=0x556425db6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.367+0000 7ff363fff640 1 --2- 172.21.15.131:0/3344973652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bc5140 0x556425db6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff363fff640 1 -- 172.21.15.131:0/3344973652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556425abea10 con 0x556425bd8a50 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff363fff640 1 --2- 172.21.15.131:0/3344973652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bd8a50 0x556425db6670 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00fa10 tx=0x7ff35c00fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38e83253451933f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3427fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3427fc640 1 -- 172.21.15.131:0/3344973652 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff35c00dc10 con 0x556425bd8a50 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3427fc640 1 -- 172.21.15.131:0/3344973652 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7ff35c00ddb0 con 0x556425bd8a50 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3714fe000 1 -- 172.21.15.131:0/3344973652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff35804c020 con 0x556425bd8a50 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3427fc640 1 -- 172.21.15.131:0/3344973652 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff35c010400 con 0x556425bd8a50 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3714fe000 10 client.6093 fetch_fsmap learned FSMap version 36 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3714fe000 1 -- 172.21.15.131:0/3344973652 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x556425acf740 con 0x556425bd8a50 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3427fc640 1 -- 172.21.15.131:0/3344973652 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff35804c020 con 0x556425bd8a50 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3427fc640 1 -- 172.21.15.131:0/3344973652 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7ff35c021050 con 0x556425bd8a50 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3714fe000 10 client.6093 fetch_fsmap finished waiting for FSMap version 36 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3714fe000 10 client.6093 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3714fe000 20 client.6093 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3714fe000 1 --2- 172.21.15.131:0/3344973652 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5564257c9b20 0x556425cefb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3714fe000 4 client.6093 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-09-17T02:44:01.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3714fe000 1 -- 172.21.15.131:0/3344973652 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x556425b476e0 con 0x5564257c9b20 2024-09-17T02:44:01.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3637fe640 1 --2- 172.21.15.131:0/3344973652 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5564257c9b20 0x556425cefb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3427fc640 10 client.6093 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:01.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.371+0000 7ff3637fe640 1 --2- 172.21.15.131:0/3344973652 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5564257c9b20 0x556425cefb80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe0a3e3cacdc051 server_cookie=889fdc134bd5ae2d in_seq=0 out_seq=0 2024-09-17T02:44:01.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3427fc640 1 -- 172.21.15.131:0/3344973652 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556425b476e0 con 0x5564257c9b20 2024-09-17T02:44:01.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3427fc640 10 client.6093 handle_command_reply: tid=0 2024-09-17T02:44:01.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 1 client.6093 shutdown 2024-09-17T02:44:01.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 1 -- 172.21.15.131:0/3344973652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bd8a50 msgr2=0x556425db6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 1 --2- 172.21.15.131:0/3344973652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bd8a50 0x556425db6670 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff35c00fa10 tx=0x7ff35c00fed0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 1 -- 172.21.15.131:0/3344973652 shutdown_connections 2024-09-17T02:44:01.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 1 --2- 172.21.15.131:0/3344973652 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556425bd8a50 0x556425db6670 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 1 --2- 172.21.15.131:0/3344973652 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556425bc5140 0x556425db6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 1 --2- 172.21.15.131:0/3344973652 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5564257c9b20 0x556425cefb80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 1 --2- 172.21.15.131:0/3344973652 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556425bbedb0 0x556425db5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3427fc640 0 client.6093 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:01.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 1 -- 172.21.15.131:0/3344973652 >> 172.21.15.131:0/3344973652 conn(0x5564257f7590 msgr2=0x556425d4e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 1 -- 172.21.15.131:0/3344973652 shutdown_connections 2024-09-17T02:44:01.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 1 -- 172.21.15.131:0/3344973652 wait complete. 2024-09-17T02:44:01.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff3714fe000 20 client.6093 trim_cache size 0 max 16384 2024-09-17T02:44:01.379 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:44:01.379 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-09-17T02:44:01.379 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "7c955166-b5f6-4490-8c58-d278c543ef77", 2024-09-17T02:44:01.380 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-09-17T02:44:01.380 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:44:01.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 -- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff350073230 msgr2=0x7ff350075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff350073230 0x7ff350075740 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff3540ae000 tx=0x7ff35401af20 comp rx=0 tx=0).stop 2024-09-17T02:44:01.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 -- 172.21.15.131:0/1476163307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3640cdc90 msgr2=0x7ff3641668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3640cdc90 0x7ff3641668d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff35800c970 tx=0x7ff35800ce30 comp rx=0 tx=0).stop 2024-09-17T02:44:01.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 -- 172.21.15.131:0/1476163307 shutdown_connections 2024-09-17T02:44:01.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff350073230 0x7ff350075740 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff364160070 0x7ff364157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3640cdc90 0x7ff3641668d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 --2- 172.21.15.131:0/1476163307 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff3640cd880 0x7ff3640b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 -- 172.21.15.131:0/1476163307 >> 172.21.15.131:0/1476163307 conn(0x7ff364070bb0 msgr2=0x7ff36415e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 -- 172.21.15.131:0/1476163307 shutdown_connections 2024-09-17T02:44:01.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.375+0000 7ff36a233640 1 -- 172.21.15.131:0/1476163307 wait complete. 2024-09-17T02:44:01.382 INFO:tasks.ceph.mds.d.smithi189.stderr:2024-09-17T02:44:01.377+0000 7fbd25b63640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 pv4 ap=4 s=0 n(v0 1=1+0)->n(v0 rc2024-09-17T02:44:00.158448+0000 1=1+0) (iauth excl w=1) (ifile lock->xlock w=1) (iquiesce lock w=1) cr={6048=0-4194304@1} caps={6048=pAsLsXs/-@3},l=6048(-1) | ptrwaiter=1 request=1 lock=4 caps=1 dirtyparent=0 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x55b59e570580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-17T02:44:01.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:44:01.395 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-17T02:44:01.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 -- 172.21.15.131:0/1072141673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faecc004b70 msgr2=0x7faecc004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 --2- 172.21.15.131:0/1072141673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faecc004b70 0x7faecc004fe0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7faed000c470 tx=0x7faed001cf10 comp rx=0 tx=0).stop 2024-09-17T02:44:01.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 -- 172.21.15.131:0/1072141673 shutdown_connections 2024-09-17T02:44:01.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 --2- 172.21.15.131:0/1072141673 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faecc004b70 0x7faecc004fe0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 -- 172.21.15.131:0/1072141673 >> 172.21.15.131:0/1072141673 conn(0x7faedc06eea0 msgr2=0x7faedc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 -- 172.21.15.131:0/1072141673 shutdown_connections 2024-09-17T02:44:01.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 -- 172.21.15.131:0/1072141673 wait complete. 2024-09-17T02:44:01.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 Processor -- start 2024-09-17T02:44:01.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 -- start start 2024-09-17T02:44:01.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faecc004b70 0x7faedc1fcd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc153170 0x7faedc159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faedc153580 0x7faedc15a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faedc15e7d0 con 0x7faedc153170 2024-09-17T02:44:01.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faedc073c60 con 0x7faedc153580 2024-09-17T02:44:01.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.547+0000 7faee30f2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7faedc15e950 con 0x7faecc004b70 2024-09-17T02:44:01.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faedbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc153170 0x7faedc159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faedbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc153170 0x7faedc159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57508/0 (socket says 172.21.15.131:57508) 2024-09-17T02:44:01.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faedbfff640 1 -- 172.21.15.131:0/1476206993 learned_addr learned my addr 172.21.15.131:0/1476206993 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faee0e8e640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faecc004b70 0x7faedc1fcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faee168f640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faedc153580 0x7faedc15a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faedbfff640 1 -- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faecc004b70 msgr2=0x7faedc1fcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faedbfff640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faecc004b70 0x7faedc1fcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faedbfff640 1 -- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faedc153580 msgr2=0x7faedc15a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faedbfff640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faedc153580 0x7faedc15a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faedbfff640 1 -- 172.21.15.131:0/1476206993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc1fd250 con 0x7faedc153170 2024-09-17T02:44:01.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faedbfff640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc153170 0x7faedc159c70 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7faec8011a80 tx=0x7faec8011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faed97fa640 1 -- 172.21.15.131:0/1476206993 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec800fe10 con 0x7faedc153170 2024-09-17T02:44:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faed97fa640 1 -- 172.21.15.131:0/1476206993 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faec8016070 con 0x7faedc153170 2024-09-17T02:44:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faed97fa640 1 -- 172.21.15.131:0/1476206993 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec8012780 con 0x7faedc153170 2024-09-17T02:44:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faee30f2640 1 -- 172.21.15.131:0/1476206993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faedc1f9120 con 0x7faedc153170 2024-09-17T02:44:01.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faee30f2640 1 -- 172.21.15.131:0/1476206993 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faedc1f96e0 con 0x7faedc153170 2024-09-17T02:44:01.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faed97fa640 1 -- 172.21.15.131:0/1476206993 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faec8018030 con 0x7faedc153170 2024-09-17T02:44:01.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faed97fa640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec0073200 0x7faec0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faed97fa640 1 -- 172.21.15.131:0/1476206993 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7faec8094bb0 con 0x7faedc153170 2024-09-17T02:44:01.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.551+0000 7faee0e8e640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec0073200 0x7faec0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 Processor -- start 2024-09-17T02:44:01.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 -- start start 2024-09-17T02:44:01.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faee0e8e640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec0073200 0x7faec0075710 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faed00aa000 tx=0x7faed0019d50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754f7b6d0 0x5607550e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754bd2890 0x5607550e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754c96aa0 0x5607550e8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560754f86cd0 con 0x560754bd2890 2024-09-17T02:44:01.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560754e9c330 con 0x560754c96aa0 2024-09-17T02:44:01.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x560754ef9bb0 con 0x560754f7b6d0 2024-09-17T02:44:01.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faedbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754bd2890 0x5607550e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faedbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754bd2890 0x5607550e7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57524/0 (socket says 172.21.15.131:57524) 2024-09-17T02:44:01.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faedbfff640 1 -- 172.21.15.131:0/2197898043 learned_addr learned my addr 172.21.15.131:0/2197898043 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faee168f640 1 --2- 172.21.15.131:0/2197898043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754c96aa0 0x5607550e8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faee0e8e640 1 --2- 172.21.15.131:0/2197898043 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754f7b6d0 0x5607550e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faedbfff640 1 -- 172.21.15.131:0/2197898043 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754f7b6d0 msgr2=0x5607550e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faedbfff640 1 --2- 172.21.15.131:0/2197898043 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754f7b6d0 0x5607550e79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faedbfff640 1 -- 172.21.15.131:0/2197898043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754c96aa0 msgr2=0x5607550e8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faedbfff640 1 --2- 172.21.15.131:0/2197898043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754c96aa0 0x5607550e8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faedbfff640 1 -- 172.21.15.131:0/2197898043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560754e9dc80 con 0x560754bd2890 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faee168f640 1 --2- 172.21.15.131:0/2197898043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754c96aa0 0x5607550e8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faedbfff640 1 --2- 172.21.15.131:0/2197898043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754bd2890 0x5607550e7f30 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7faec805f0d0 tx=0x7faec805eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7b4f45a9399368 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faec67fc640 1 -- 172.21.15.131:0/2197898043 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec805f3d0 con 0x560754bd2890 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faec67fc640 1 -- 172.21.15.131:0/2197898043 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faec8063eb0 con 0x560754bd2890 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faec67fc640 1 -- 172.21.15.131:0/2197898043 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec805e850 con 0x560754bd2890 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 -- 172.21.15.131:0/2197898043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754bd2890 msgr2=0x5607550e7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 --2- 172.21.15.131:0/2197898043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754bd2890 0x5607550e7f30 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7faec805f0d0 tx=0x7faec805eee0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 -- 172.21.15.131:0/2197898043 shutdown_connections 2024-09-17T02:44:01.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 --2- 172.21.15.131:0/2197898043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754c96aa0 0x5607550e8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 --2- 172.21.15.131:0/2197898043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754bd2890 0x5607550e7f30 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 --2- 172.21.15.131:0/2197898043 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754f7b6d0 0x5607550e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 -- 172.21.15.131:0/2197898043 >> 172.21.15.131:0/2197898043 conn(0x560754ba65c0 msgr2=0x5607550a4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 -- 172.21.15.131:0/2197898043 shutdown_connections 2024-09-17T02:44:01.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.555+0000 7faeea3bd000 1 -- 172.21.15.131:0/2197898043 wait complete. 2024-09-17T02:44:01.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 -- start start 2024-09-17T02:44:01.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 Processor -- start 2024-09-17T02:44:01.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754bd2890 0x5607550a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754c96aa0 0x5607550a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x5607550aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560754eecdf0 con 0x560754f7b6d0 2024-09-17T02:44:01.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560754ee8fe0 con 0x560754bd2890 2024-09-17T02:44:01.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x560754e8e4d0 con 0x560754c96aa0 2024-09-17T02:44:01.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 10 client.0 _ll_register_callbacks cb 0x560754e6c7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:01.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee168f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x5607550aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee168f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x5607550aa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57526/0 (socket says 172.21.15.131:57526) 2024-09-17T02:44:01.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee168f640 1 -- 172.21.15.131:0/3043389439 learned_addr learned my addr 172.21.15.131:0/3043389439 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee0e8e640 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754bd2890 0x5607550a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faedbfff640 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754c96aa0 0x5607550a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee168f640 1 -- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754c96aa0 msgr2=0x5607550a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee168f640 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754c96aa0 0x5607550a3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee168f640 1 -- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754bd2890 msgr2=0x5607550a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee168f640 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754bd2890 0x5607550a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee168f640 1 -- 172.21.15.131:0/3043389439 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560754e87210 con 0x560754f7b6d0 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faedbfff640 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754c96aa0 0x5607550a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee168f640 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x5607550aa5f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7faedc072250 tx=0x7faed4006850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f65daa0f2536076 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faebb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faebb7fe640 1 -- 172.21.15.131:0/3043389439 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4003b90 con 0x560754f7b6d0 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faebb7fe640 1 -- 172.21.15.131:0/3043389439 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7faed4003d30 con 0x560754f7b6d0 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 -- 172.21.15.131:0/3043389439 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560754e81350 con 0x560754f7b6d0 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faebb7fe640 1 -- 172.21.15.131:0/3043389439 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7faed4001b60 con 0x560754f7b6d0 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 10 client.6105 fetch_fsmap learned FSMap version 36 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 -- 172.21.15.131:0/3043389439 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x560754e6dfc0 con 0x560754f7b6d0 2024-09-17T02:44:01.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faebb7fe640 1 -- 172.21.15.131:0/3043389439 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560754e81350 con 0x560754f7b6d0 2024-09-17T02:44:01.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faebb7fe640 1 -- 172.21.15.131:0/3043389439 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7faed4001df0 con 0x560754f7b6d0 2024-09-17T02:44:01.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 10 client.6105 fetch_fsmap finished waiting for FSMap version 36 2024-09-17T02:44:01.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 10 client.6105 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:44:01.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 20 client.6105 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x560754b8b930 0x5607550eee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 4 client.6105 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:44:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faeea3bd000 1 -- 172.21.15.131:0/3043389439 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560754efbb00 con 0x560754b8b930 2024-09-17T02:44:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee0e8e640 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x560754b8b930 0x5607550eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faebb7fe640 10 client.6105 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:01.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.559+0000 7faee0e8e640 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x560754b8b930 0x5607550eee80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12a920d5f0b154a7 server_cookie=22b21f2e2d72342f in_seq=0 out_seq=0 2024-09-17T02:44:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faebb7fe640 1 -- 172.21.15.131:0/3043389439 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560754efbb00 con 0x560754b8b930 2024-09-17T02:44:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faebb7fe640 10 client.6105 handle_command_reply: tid=0 2024-09-17T02:44:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 1 client.6105 shutdown 2024-09-17T02:44:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 1 -- 172.21.15.131:0/3043389439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 msgr2=0x5607550aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x5607550aa5f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7faedc072250 tx=0x7faed4006850 comp rx=0 tx=0).stop 2024-09-17T02:44:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 1 -- 172.21.15.131:0/3043389439 shutdown_connections 2024-09-17T02:44:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x5607550aa5f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x560754b8b930 0x5607550eee80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754c96aa0 0x5607550a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3043389439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754bd2890 0x5607550a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faebb7fe640 0 client.6105 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 1 -- 172.21.15.131:0/3043389439 >> 172.21.15.131:0/3043389439 conn(0x560754ba65c0 msgr2=0x56075509cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 1 -- 172.21.15.131:0/3043389439 shutdown_connections 2024-09-17T02:44:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 1 -- 172.21.15.131:0/3043389439 wait complete. 2024-09-17T02:44:01.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.563+0000 7faeea3bd000 20 client.6105 trim_cache size 0 max 16384 2024-09-17T02:44:01.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.579+0000 7faeea3bd000 1 Processor -- start 2024-09-17T02:44:01.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.579+0000 7faeea3bd000 1 -- start start 2024-09-17T02:44:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754bd2890 0x5607551591f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754c96aa0 0x560755159730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x560755159c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5607550a2af0 con 0x560754f7b6d0 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faedc2062f0 con 0x560754c96aa0 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7faedc205330 con 0x560754bd2890 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faee168f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x560755159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faee168f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x560755159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57530/0 (socket says 172.21.15.131:57530) 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faee168f640 1 -- 172.21.15.131:0/663353147 learned_addr learned my addr 172.21.15.131:0/663353147 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faedbfff640 1 --2- 172.21.15.131:0/663353147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754c96aa0 0x560755159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faee168f640 1 -- 172.21.15.131:0/663353147 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754bd2890 msgr2=0x5607551591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faee168f640 1 --2- 172.21.15.131:0/663353147 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754bd2890 0x5607551591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faee168f640 1 -- 172.21.15.131:0/663353147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754c96aa0 msgr2=0x560755159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faee168f640 1 --2- 172.21.15.131:0/663353147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754c96aa0 0x560755159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faee168f640 1 -- 172.21.15.131:0/663353147 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560754f46680 con 0x560754f7b6d0 2024-09-17T02:44:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faee168f640 1 --2- 172.21.15.131:0/663353147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x560755159c70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7faed40041f0 tx=0x7faed4004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217c2515b4dd5e5c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faec5ffb640 1 -- 172.21.15.131:0/663353147 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed402e070 con 0x560754f7b6d0 2024-09-17T02:44:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faec5ffb640 1 -- 172.21.15.131:0/663353147 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faed4017020 con 0x560754f7b6d0 2024-09-17T02:44:01.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faec5ffb640 1 -- 172.21.15.131:0/663353147 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4016030 con 0x560754f7b6d0 2024-09-17T02:44:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 -- 172.21.15.131:0/663353147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 msgr2=0x560755159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 --2- 172.21.15.131:0/663353147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x560755159c70 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7faed40041f0 tx=0x7faed4004220 comp rx=0 tx=0).stop 2024-09-17T02:44:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 -- 172.21.15.131:0/663353147 shutdown_connections 2024-09-17T02:44:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 --2- 172.21.15.131:0/663353147 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754f7b6d0 0x560755159c70 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 --2- 172.21.15.131:0/663353147 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754c96aa0 0x560755159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 --2- 172.21.15.131:0/663353147 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754bd2890 0x5607551591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 -- 172.21.15.131:0/663353147 >> 172.21.15.131:0/663353147 conn(0x560754ba65c0 msgr2=0x5607550a95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 -- 172.21.15.131:0/663353147 shutdown_connections 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 -- 172.21.15.131:0/663353147 wait complete. 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 -- start start 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.583+0000 7faeea3bd000 1 Processor -- start 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754bd2890 0x56075515d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754c96aa0 0x56075515db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754f7b6d0 0x56075515e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faedc204ce0 con 0x560754c96aa0 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faedc204690 con 0x560754bd2890 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x560754bae680 con 0x560754f7b6d0 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 10 client.0 _ll_register_callbacks cb 0x7faed40049f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:01.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faedbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754c96aa0 0x56075515db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faedbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754c96aa0 0x56075515db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:57538/0 (socket says 172.21.15.131:57538) 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faedbfff640 1 -- 172.21.15.131:0/3939327494 learned_addr learned my addr 172.21.15.131:0/3939327494 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faee0e8e640 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754bd2890 0x56075515d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faee168f640 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754f7b6d0 0x56075515e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faedbfff640 1 -- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754f7b6d0 msgr2=0x56075515e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faedbfff640 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754f7b6d0 0x56075515e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faedbfff640 1 -- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754bd2890 msgr2=0x56075515d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faedbfff640 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754bd2890 0x56075515d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faedbfff640 1 -- 172.21.15.131:0/3939327494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560754eb30b0 con 0x560754c96aa0 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faee0e8e640 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754bd2890 0x56075515d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faedbfff640 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754c96aa0 0x56075515db90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7faec8005ec0 tx=0x7faec805e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a759d80f598ada2f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faebb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:01.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faebb7fe640 1 -- 172.21.15.131:0/3939327494 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faec807e060 con 0x560754c96aa0 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faebb7fe640 1 -- 172.21.15.131:0/3939327494 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7faec805af20 con 0x560754c96aa0 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 -- 172.21.15.131:0/3939327494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faed4017020 con 0x560754c96aa0 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faebb7fe640 1 -- 172.21.15.131:0/3939327494 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7faec8079070 con 0x560754c96aa0 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 10 client.6111 fetch_fsmap learned FSMap version 36 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 -- 172.21.15.131:0/3939327494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x560754e7f610 con 0x560754c96aa0 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faebb7fe640 1 -- 172.21.15.131:0/3939327494 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faed4017020 con 0x560754c96aa0 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faebb7fe640 1 -- 172.21.15.131:0/3939327494 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7faec805e0a0 con 0x560754c96aa0 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 10 client.6111 fetch_fsmap finished waiting for FSMap version 36 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 10 client.6111 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 20 client.6111 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x560754b79990 0x56075509ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 4 client.6111 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 -- 172.21.15.131:0/3939327494 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560754ef75b0 con 0x560754b79990 2024-09-17T02:44:01.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faee0e8e640 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x560754b79990 0x56075509ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:01.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faebb7fe640 10 client.6111 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:01.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faee0e8e640 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x560754b79990 0x56075509ee10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2adc13720191a437 server_cookie=1ac063bc2ea6a715 in_seq=0 out_seq=0 2024-09-17T02:44:01.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faebb7fe640 1 -- 172.21.15.131:0/3939327494 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+564 (crc 0 0 0) 0x560754ef75b0 con 0x560754b79990 2024-09-17T02:44:01.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faebb7fe640 10 client.6111 handle_command_reply: tid=0 2024-09-17T02:44:01.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 client.6111 shutdown 2024-09-17T02:44:01.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 -- 172.21.15.131:0/3939327494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754c96aa0 msgr2=0x56075515db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754c96aa0 0x56075515db90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7faec8005ec0 tx=0x7faec805e840 comp rx=0 tx=0).stop 2024-09-17T02:44:01.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 -- 172.21.15.131:0/3939327494 shutdown_connections 2024-09-17T02:44:01.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x560754b79990 0x56075509ee10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560754f7b6d0 0x56075515e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560754c96aa0 0x56075515db90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faebb7fe640 0 client.6111 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 --2- 172.21.15.131:0/3939327494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560754bd2890 0x56075515d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 -- 172.21.15.131:0/3939327494 >> 172.21.15.131:0/3939327494 conn(0x560754ba65c0 msgr2=0x5607550f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 -- 172.21.15.131:0/3939327494 shutdown_connections 2024-09-17T02:44:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 1 -- 172.21.15.131:0/3939327494 wait complete. 2024-09-17T02:44:01.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.587+0000 7faeea3bd000 20 client.6111 trim_cache size 0 max 16384 2024-09-17T02:44:01.596 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: "7c955166-b5f6-4490-8c58-d278c543ef77": { 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "7c955166-b5f6-4490-8c58-d278c543ef77", 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive" 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: "7c955166-b5f6-4490-8c58-d278c543ef77": { 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:44:01.373757+0000", 2024-09-17T02:44:01.597 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:44:01.598 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:44:01.598 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:44:01.598 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:44:01.598 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:44:01.598 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:44:01.598 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:44:01.598 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:44:01.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 -- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec0073200 msgr2=0x7faec0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec0073200 0x7faec0075710 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faed00aa000 tx=0x7faed0019d50 comp rx=0 tx=0).stop 2024-09-17T02:44:01.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 -- 172.21.15.131:0/1476206993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc153170 msgr2=0x7faedc159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:01.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc153170 0x7faedc159c70 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7faec8011a80 tx=0x7faec8011f40 comp rx=0 tx=0).stop 2024-09-17T02:44:01.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 -- 172.21.15.131:0/1476206993 shutdown_connections 2024-09-17T02:44:01.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faec0073200 0x7faec0075710 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faedc153580 0x7faedc15a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faedc153170 0x7faedc159c70 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 --2- 172.21.15.131:0/1476206993 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faecc004b70 0x7faedc1fcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:01.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 -- 172.21.15.131:0/1476206993 >> 172.21.15.131:0/1476206993 conn(0x7faedc06eea0 msgr2=0x7faedc1540d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:01.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 -- 172.21.15.131:0/1476206993 shutdown_connections 2024-09-17T02:44:01.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:01.591+0000 7faee30f2640 1 -- 172.21.15.131:0/1476206993 wait complete. 2024-09-17T02:44:01.610 INFO:tasks.cephfs.filesystem:scrub status for tag:7c955166-b5f6-4490-8c58-d278c543ef77 - {'path': '/', 'tag': '7c955166-b5f6-4490-8c58-d278c543ef77', 'options': 'recursive'} 2024-09-17T02:44:03.288 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:44:03.287+0000 7f3a39041640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-17T02:44:14.089 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:44:14.092 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:44:31.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:44:31.611 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-17T02:44:31.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 -- 172.21.15.131:0/427677509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f598c003a10 msgr2=0x7f598c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:31.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 --2- 172.21.15.131:0/427677509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f598c003a10 0x7f598c003e60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f599400c2e0 tx=0x7f599401ced0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 -- 172.21.15.131:0/427677509 shutdown_connections 2024-09-17T02:44:31.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 --2- 172.21.15.131:0/427677509 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f598c003a10 0x7f598c003e60 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 -- 172.21.15.131:0/427677509 >> 172.21.15.131:0/427677509 conn(0x7f59a40cf0b0 msgr2=0x7f59a40cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:31.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 -- 172.21.15.131:0/427677509 shutdown_connections 2024-09-17T02:44:31.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 -- 172.21.15.131:0/427677509 wait complete. 2024-09-17T02:44:31.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 Processor -- start 2024-09-17T02:44:31.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 -- start start 2024-09-17T02:44:31.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59a414a380 0x7f59a41f9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a41fbdd0 0x7f59a41fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a40b6df0 0x7f59a40b7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59a40713a0 con 0x7f59a40b6df0 2024-09-17T02:44:31.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4071220 con 0x7f59a41fbdd0 2024-09-17T02:44:31.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a8b40640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f59a4071520 con 0x7f59a414a380 2024-09-17T02:44:31.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a40b6df0 0x7f59a40b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a40b6df0 0x7f59a40b7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60654/0 (socket says 172.21.15.131:60654) 2024-09-17T02:44:31.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.787+0000 7f59a2d9d640 1 -- 172.21.15.131:0/3686581442 learned_addr learned my addr 172.21.15.131:0/3686581442 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:31.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a1d9b640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a41fbdd0 0x7f59a41fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a259c640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59a414a380 0x7f59a41f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a2d9d640 1 -- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59a414a380 msgr2=0x7f59a41f9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59a414a380 0x7f59a41f9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a2d9d640 1 -- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a41fbdd0 msgr2=0x7f59a41fc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a41fbdd0 0x7f59a41fc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a2d9d640 1 -- 172.21.15.131:0/3686581442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a40695a0 con 0x7f59a40b6df0 2024-09-17T02:44:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a259c640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59a414a380 0x7f59a41f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a1d9b640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a41fbdd0 0x7f59a41fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a40b6df0 0x7f59a40b7200 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f59a40ce1d0 tx=0x7f599c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f5992ffd640 1 -- 172.21.15.131:0/3686581442 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c004d30 con 0x7f59a40b6df0 2024-09-17T02:44:31.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f5992ffd640 1 -- 172.21.15.131:0/3686581442 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f599c00ea30 con 0x7f59a40b6df0 2024-09-17T02:44:31.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a8b40640 1 -- 172.21.15.131:0/3686581442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a40b7740 con 0x7f59a40b6df0 2024-09-17T02:44:31.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f5992ffd640 1 -- 172.21.15.131:0/3686581442 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c00ee60 con 0x7f59a40b6df0 2024-09-17T02:44:31.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a8b40640 1 -- 172.21.15.131:0/3686581442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a41fcc20 con 0x7f59a40b6df0 2024-09-17T02:44:31.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f5992ffd640 1 -- 172.21.15.131:0/3686581442 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f599c01a070 con 0x7f59a40b6df0 2024-09-17T02:44:31.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f5992ffd640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5984073230 0x7f5984075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f5992ffd640 1 -- 172.21.15.131:0/3686581442 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f599c094c30 con 0x7f59a40b6df0 2024-09-17T02:44:31.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59a259c640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5984073230 0x7f5984075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59afe0b000 1 Processor -- start 2024-09-17T02:44:31.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.791+0000 7f59afe0b000 1 -- start start 2024-09-17T02:44:31.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59a259c640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5984073230 0x7f5984075740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f59940ac000 tx=0x7f5994019d10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:31.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2a096d0 0x55b7a2b759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2660890 0x55b7a2b75f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b76470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b7a2a14cd0 con 0x55b7a2724aa0 2024-09-17T02:44:31.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b7a292a330 con 0x55b7a2a096d0 2024-09-17T02:44:31.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b7a2987bb0 con 0x55b7a2660890 2024-09-17T02:44:31.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b76470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b76470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60666/0 (socket says 172.21.15.131:60666) 2024-09-17T02:44:31.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59a2d9d640 1 -- 172.21.15.131:0/2389120490 learned_addr learned my addr 172.21.15.131:0/2389120490 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:31.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59a2d9d640 1 -- 172.21.15.131:0/2389120490 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2660890 msgr2=0x55b7a2b75f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:31.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/2389120490 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2660890 0x55b7a2b75f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59a2d9d640 1 -- 172.21.15.131:0/2389120490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2a096d0 msgr2=0x55b7a2b759f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/2389120490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2a096d0 0x55b7a2b759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59a2d9d640 1 -- 172.21.15.131:0/2389120490 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a292bc80 con 0x55b7a2724aa0 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/2389120490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b76470 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f599c009fd0 tx=0x7f599c05f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb41ec015ce29ca server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f5983fff640 1 -- 172.21.15.131:0/2389120490 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c021070 con 0x55b7a2724aa0 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f5983fff640 1 -- 172.21.15.131:0/2389120490 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f599c05b880 con 0x55b7a2724aa0 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f5983fff640 1 -- 172.21.15.131:0/2389120490 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c05ead0 con 0x55b7a2724aa0 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 -- 172.21.15.131:0/2389120490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 msgr2=0x55b7a2b76470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 --2- 172.21.15.131:0/2389120490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b76470 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f599c009fd0 tx=0x7f599c05f610 comp rx=0 tx=0).stop 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 -- 172.21.15.131:0/2389120490 shutdown_connections 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 --2- 172.21.15.131:0/2389120490 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b76470 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 --2- 172.21.15.131:0/2389120490 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2660890 0x55b7a2b75f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 --2- 172.21.15.131:0/2389120490 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2a096d0 0x55b7a2b759f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 -- 172.21.15.131:0/2389120490 >> 172.21.15.131:0/2389120490 conn(0x55b7a26345c0 msgr2=0x55b7a2b32ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 -- 172.21.15.131:0/2389120490 shutdown_connections 2024-09-17T02:44:31.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 -- 172.21.15.131:0/2389120490 wait complete. 2024-09-17T02:44:31.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.795+0000 7f59afe0b000 1 -- start start 2024-09-17T02:44:31.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 1 Processor -- start 2024-09-17T02:44:31.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2660890 0x55b7a2b31a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b31f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2a096d0 0x55b7a2b385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b7a297adf0 con 0x55b7a2724aa0 2024-09-17T02:44:31.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b7a2976fe0 con 0x55b7a2660890 2024-09-17T02:44:31.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b7a291c4d0 con 0x55b7a2a096d0 2024-09-17T02:44:31.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 10 client.0 _ll_register_callbacks cb 0x55b7a28fa7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:31.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a1d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b31f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a1d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b31f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60682/0 (socket says 172.21.15.131:60682) 2024-09-17T02:44:31.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a1d9b640 1 -- 172.21.15.131:0/1163811145 learned_addr learned my addr 172.21.15.131:0/1163811145 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:31.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2a096d0 0x55b7a2b385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a259c640 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2660890 0x55b7a2b31a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a1d9b640 1 -- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2a096d0 msgr2=0x55b7a2b385f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:31.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a1d9b640 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2a096d0 0x55b7a2b385f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a1d9b640 1 -- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2660890 msgr2=0x55b7a2b31a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:31.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a1d9b640 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2660890 0x55b7a2b31a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a1d9b640 1 -- 172.21.15.131:0/1163811145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a2915210 con 0x55b7a2724aa0 2024-09-17T02:44:31.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a1d9b640 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b31f40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f59980038f0 tx=0x7f5998003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eef4540019b7e89 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:31.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59817fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:31.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59817fa640 1 -- 172.21.15.131:0/1163811145 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998015020 con 0x55b7a2724aa0 2024-09-17T02:44:31.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59817fa640 1 -- 172.21.15.131:0/1163811145 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f599800ecd0 con 0x55b7a2724aa0 2024-09-17T02:44:31.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 1 -- 172.21.15.131:0/1163811145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b7a290f350 con 0x55b7a2724aa0 2024-09-17T02:44:31.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59817fa640 1 -- 172.21.15.131:0/1163811145 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f599801f050 con 0x55b7a2724aa0 2024-09-17T02:44:31.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 10 client.6123 fetch_fsmap learned FSMap version 37 2024-09-17T02:44:31.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 1 -- 172.21.15.131:0/1163811145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55b7a28fbfc0 con 0x55b7a2724aa0 2024-09-17T02:44:31.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59817fa640 1 -- 172.21.15.131:0/1163811145 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b7a290f350 con 0x55b7a2724aa0 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59817fa640 1 -- 172.21.15.131:0/1163811145 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7f5998022020 con 0x55b7a2724aa0 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 10 client.6123 fetch_fsmap finished waiting for FSMap version 37 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 10 client.6123 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 20 client.6123 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55b7a2619930 0x55b7a2b7ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 4 client.6123 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59afe0b000 1 -- 172.21.15.131:0/1163811145 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7a2989b00 con 0x55b7a2619930 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a259c640 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55b7a2619930 0x55b7a2b7ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59817fa640 10 client.6123 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.799+0000 7f59a259c640 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55b7a2619930 0x55b7a2b7ce80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66dc169d5b329222 server_cookie=366c59f52d5b5c97 in_seq=0 out_seq=0 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59817fa640 1 -- 172.21.15.131:0/1163811145 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b7a2989b00 con 0x55b7a2619930 2024-09-17T02:44:31.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59817fa640 10 client.6123 handle_command_reply: tid=0 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 1 client.6123 shutdown 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 1 -- 172.21.15.131:0/1163811145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 msgr2=0x55b7a2b31f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b31f40 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f59980038f0 tx=0x7f5998003db0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 1 -- 172.21.15.131:0/1163811145 shutdown_connections 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55b7a2619930 0x55b7a2b7ce80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2a096d0 0x55b7a2b385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59817fa640 0 client.6123 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2b31f40 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1163811145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2660890 0x55b7a2b31a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 1 -- 172.21.15.131:0/1163811145 >> 172.21.15.131:0/1163811145 conn(0x55b7a26345c0 msgr2=0x55b7a2b2ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 1 -- 172.21.15.131:0/1163811145 shutdown_connections 2024-09-17T02:44:31.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 1 -- 172.21.15.131:0/1163811145 wait complete. 2024-09-17T02:44:31.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.803+0000 7f59afe0b000 20 client.6123 trim_cache size 0 max 16384 2024-09-17T02:44:31.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 Processor -- start 2024-09-17T02:44:31.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 -- start start 2024-09-17T02:44:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2619930 0x55b7a2be71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2660890 0x55b7a2be7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2be7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4206580 con 0x55b7a2724aa0 2024-09-17T02:44:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59a42055c0 con 0x55b7a2619930 2024-09-17T02:44:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f59a4204a70 con 0x55b7a2660890 2024-09-17T02:44:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2be7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2be7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60694/0 (socket says 172.21.15.131:60694) 2024-09-17T02:44:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59a2d9d640 1 -- 172.21.15.131:0/3998126721 learned_addr learned my addr 172.21.15.131:0/3998126721 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:31.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59a2d9d640 1 -- 172.21.15.131:0/3998126721 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2660890 msgr2=0x55b7a2be7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59a259c640 1 --2- 172.21.15.131:0/3998126721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2619930 0x55b7a2be71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/3998126721 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2660890 0x55b7a2be7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59a2d9d640 1 -- 172.21.15.131:0/3998126721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2619930 msgr2=0x55b7a2be71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/3998126721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2619930 0x55b7a2be71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59a2d9d640 1 -- 172.21.15.131:0/3998126721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7a29d4680 con 0x55b7a2724aa0 2024-09-17T02:44:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/3998126721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2be7c70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f599c093030 tx=0x7f599c092f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17649e39400d82c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59837fe640 1 -- 172.21.15.131:0/3998126721 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c021070 con 0x55b7a2724aa0 2024-09-17T02:44:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59837fe640 1 -- 172.21.15.131:0/3998126721 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f599c05eb60 con 0x55b7a2724aa0 2024-09-17T02:44:31.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59837fe640 1 -- 172.21.15.131:0/3998126721 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c05b1c0 con 0x55b7a2724aa0 2024-09-17T02:44:31.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 -- 172.21.15.131:0/3998126721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 msgr2=0x55b7a2be7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:31.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 --2- 172.21.15.131:0/3998126721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2be7c70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f599c093030 tx=0x7f599c092f30 comp rx=0 tx=0).stop 2024-09-17T02:44:31.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 -- 172.21.15.131:0/3998126721 shutdown_connections 2024-09-17T02:44:31.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 --2- 172.21.15.131:0/3998126721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2be7c70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 --2- 172.21.15.131:0/3998126721 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2660890 0x55b7a2be7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 --2- 172.21.15.131:0/3998126721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2619930 0x55b7a2be71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 -- 172.21.15.131:0/3998126721 >> 172.21.15.131:0/3998126721 conn(0x55b7a26345c0 msgr2=0x55b7a2b375a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:31.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 -- 172.21.15.131:0/3998126721 shutdown_connections 2024-09-17T02:44:31.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.823+0000 7f59afe0b000 1 -- 172.21.15.131:0/3998126721 wait complete. 2024-09-17T02:44:31.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 -- start start 2024-09-17T02:44:31.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 Processor -- start 2024-09-17T02:44:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2619930 0x55b7a2beb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2660890 0x55b7a2beb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2bebec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b7a2971ee0 con 0x55b7a2724aa0 2024-09-17T02:44:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b7a293f610 con 0x55b7a2660890 2024-09-17T02:44:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b7a297a100 con 0x55b7a2619930 2024-09-17T02:44:31.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 10 client.0 _ll_register_callbacks cb 0x7f5998006ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2bebec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2bebec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60702/0 (socket says 172.21.15.131:60702) 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a2d9d640 1 -- 172.21.15.131:0/1474389433 learned_addr learned my addr 172.21.15.131:0/1474389433 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a1d9b640 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2660890 0x55b7a2beb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a259c640 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2619930 0x55b7a2beb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a2d9d640 1 -- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2619930 msgr2=0x55b7a2beb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2619930 0x55b7a2beb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a2d9d640 1 -- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2660890 msgr2=0x55b7a2beb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2660890 0x55b7a2beb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a2d9d640 1 -- 172.21.15.131:0/1474389433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7a29410b0 con 0x55b7a2724aa0 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a2d9d640 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2bebec0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f599c05f310 tx=0x7f599c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15287db0d6063f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59817fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:31.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59817fa640 1 -- 172.21.15.131:0/1474389433 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c021070 con 0x55b7a2724aa0 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59817fa640 1 -- 172.21.15.131:0/1474389433 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f599c05b170 con 0x55b7a2724aa0 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 -- 172.21.15.131:0/1474389433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f599c05eb60 con 0x55b7a2724aa0 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59817fa640 1 -- 172.21.15.131:0/1474389433 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f599c05e2e0 con 0x55b7a2724aa0 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 10 client.6129 fetch_fsmap learned FSMap version 37 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 -- 172.21.15.131:0/1474389433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55b7a290d610 con 0x55b7a2724aa0 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59817fa640 1 -- 172.21.15.131:0/1474389433 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f599c05eb60 con 0x55b7a2724aa0 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59817fa640 1 -- 172.21.15.131:0/1474389433 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7f599c00f040 con 0x55b7a2724aa0 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 10 client.6129 fetch_fsmap finished waiting for FSMap version 37 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 10 client.6129 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 20 client.6129 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55b7a2632fe0 0x55b7a2b2ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 4 client.6129 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59afe0b000 1 -- 172.21.15.131:0/1474389433 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b7a29855b0 con 0x55b7a2632fe0 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.827+0000 7f59a259c640 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55b7a2632fe0 0x55b7a2b2ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:31.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59817fa640 10 client.6129 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a259c640 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55b7a2632fe0 0x55b7a2b2ce10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=997f81b51e3dc7f6 server_cookie=d69ebbcb43fed7f6 in_seq=0 out_seq=0 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59817fa640 1 -- 172.21.15.131:0/1474389433 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55b7a29855b0 con 0x55b7a2632fe0 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59817fa640 10 client.6129 handle_command_reply: tid=0 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 1 client.6129 shutdown 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 1 -- 172.21.15.131:0/1474389433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 msgr2=0x55b7a2bebec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2bebec0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f599c05f310 tx=0x7f599c009f70 comp rx=0 tx=0).stop 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 1 -- 172.21.15.131:0/1474389433 shutdown_connections 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b7a2724aa0 0x55b7a2bebec0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b7a2660890 0x55b7a2beb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55b7a2632fe0 0x55b7a2b2ce10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 1 --2- 172.21.15.131:0/1474389433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b7a2619930 0x55b7a2beb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59817fa640 0 client.6129 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:31.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 1 -- 172.21.15.131:0/1474389433 >> 172.21.15.131:0/1474389433 conn(0x55b7a26345c0 msgr2=0x55b7a2b87d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:31.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 1 -- 172.21.15.131:0/1474389433 shutdown_connections 2024-09-17T02:44:31.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 1 -- 172.21.15.131:0/1474389433 wait complete. 2024-09-17T02:44:31.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59afe0b000 20 client.6129 trim_cache size 0 max 16384 2024-09-17T02:44:31.833 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:44:31.833 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-09-17T02:44:31.833 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {}, 2024-09-17T02:44:31.833 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:44:31.833 INFO:teuthology.orchestra.run.smithi131.stdout: "7c955166-b5f6-4490-8c58-d278c543ef77": { 2024-09-17T02:44:31.833 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:44:01.373757+0000", 2024-09-17T02:44:31.833 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:44:31.834 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:44:31.834 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:44:31.834 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:44:31.834 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:44:31.834 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:44:31.834 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:44:31.834 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:44:31.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 -- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5984073230 msgr2=0x7f5984075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:31.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5984073230 0x7f5984075740 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f59940ac000 tx=0x7f5994019d10 comp rx=0 tx=0).stop 2024-09-17T02:44:31.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 -- 172.21.15.131:0/3686581442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a40b6df0 msgr2=0x7f59a40b7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:31.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a40b6df0 0x7f59a40b7200 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f59a40ce1d0 tx=0x7f599c010da0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 -- 172.21.15.131:0/3686581442 shutdown_connections 2024-09-17T02:44:31.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5984073230 0x7f5984075740 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59a40b6df0 0x7f59a40b7200 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59a41fbdd0 0x7f59a41fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 --2- 172.21.15.131:0/3686581442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59a414a380 0x7f59a41f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 -- 172.21.15.131:0/3686581442 >> 172.21.15.131:0/3686581442 conn(0x7f59a40cf0b0 msgr2=0x7f59a40d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:31.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 -- 172.21.15.131:0/3686581442 shutdown_connections 2024-09-17T02:44:31.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.831+0000 7f59a8b40640 1 -- 172.21.15.131:0/3686581442 wait complete. 2024-09-17T02:44:31.848 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-17T02:44:31.848 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:44:31.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.999+0000 7fa4a1e45640 1 -- 172.21.15.131:0/1803720781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49c15ba70 msgr2=0x7fa49c071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:31.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.999+0000 7fa4a1e45640 1 --2- 172.21.15.131:0/1803720781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49c15ba70 0x7fa49c071210 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa49c072bd0 tx=0x7fa48401c800 comp rx=0 tx=0).stop 2024-09-17T02:44:31.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.999+0000 7fa4a1e45640 1 -- 172.21.15.131:0/1803720781 shutdown_connections 2024-09-17T02:44:31.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.999+0000 7fa4a1e45640 1 --2- 172.21.15.131:0/1803720781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49c15ba70 0x7fa49c071210 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.999+0000 7fa4a1e45640 1 --2- 172.21.15.131:0/1803720781 >> v2:172.21.15.189:3300/0 conn(0x7fa49c152550 0x7fa49c06e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:31.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.999+0000 7fa4a1e45640 1 -- 172.21.15.131:0/1803720781 >> 172.21.15.131:0/1803720781 conn(0x7fa49c0d1130 msgr2=0x7fa49c0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:31.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.999+0000 7fa4a1e45640 1 -- 172.21.15.131:0/1803720781 shutdown_connections 2024-09-17T02:44:31.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.999+0000 7fa4a1e45640 1 -- 172.21.15.131:0/1803720781 wait complete. 2024-09-17T02:44:31.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.999+0000 7fa4a1e45640 1 Processor -- start 2024-09-17T02:44:31.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:31.999+0000 7fa4a1e45640 1 -- start start 2024-09-17T02:44:32.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa4a1e45640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa49c152550 0x7fa49c0aa1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa4a1e45640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49c15ba70 0x7fa49c1fa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa4a1e45640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c15be80 0x7fa49c1fab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa4a1e45640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c1690b0 con 0x7fa49c15ba70 2024-09-17T02:44:32.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa4a1e45640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c168f30 con 0x7fa49c15be80 2024-09-17T02:44:32.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa4a1e45640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa49c169230 con 0x7fa49c152550 2024-09-17T02:44:32.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49c15ba70 0x7fa49c1fa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49c15ba70 0x7fa49c1fa610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60710/0 (socket says 172.21.15.131:60710) 2024-09-17T02:44:32.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49affd640 1 -- 172.21.15.131:0/3362365460 learned_addr learned my addr 172.21.15.131:0/3362365460 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49bfff640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c15be80 0x7fa49c1fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49b7fe640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa49c152550 0x7fa49c0aa1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49affd640 1 -- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa49c152550 msgr2=0x7fa49c0aa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49affd640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa49c152550 0x7fa49c0aa1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49affd640 1 -- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c15be80 msgr2=0x7fa49c1fab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49affd640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c15be80 0x7fa49c1fab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49affd640 1 -- 172.21.15.131:0/3362365460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49c1f7020 con 0x7fa49c15ba70 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49bfff640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c15be80 0x7fa49c1fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa49affd640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49c15ba70 0x7fa49c1fa610 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa4840b0040 tx=0x7fa484005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa493fff640 1 -- 172.21.15.131:0/3362365460 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840b2040 con 0x7fa49c15ba70 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa493fff640 1 -- 172.21.15.131:0/3362365460 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4840bd070 con 0x7fa49c15ba70 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa4a1e45640 1 -- 172.21.15.131:0/3362365460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49c1fb320 con 0x7fa49c15ba70 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa493fff640 1 -- 172.21.15.131:0/3362365460 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484003600 con 0x7fa49c15ba70 2024-09-17T02:44:32.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa4a1e45640 1 -- 172.21.15.131:0/3362365460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49c1f7580 con 0x7fa49c15ba70 2024-09-17T02:44:32.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa4a1e45640 1 -- 172.21.15.131:0/3362365460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa49c1fb090 con 0x7fa49c15ba70 2024-09-17T02:44:32.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.003+0000 7fa493fff640 1 -- 172.21.15.131:0/3362365460 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa4840037a0 con 0x7fa49c15ba70 2024-09-17T02:44:32.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.007+0000 7fa493fff640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa47c073200 0x7fa47c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.007+0000 7fa493fff640 1 -- 172.21.15.131:0/3362365460 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fa48412eef0 con 0x7fa49c15ba70 2024-09-17T02:44:32.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.007+0000 7fa49b7fe640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa47c073200 0x7fa47c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.007+0000 7fa49b7fe640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa47c073200 0x7fa47c075710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005c50 tx=0x7fa48c005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.011+0000 7fa493fff640 1 -- 172.21.15.131:0/3362365460 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa49c1f7020 con 0x7fa49c15ba70 2024-09-17T02:44:32.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 -- 172.21.15.131:0/3362365460 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa49c06e840 con 0x7fa49c15ba70 2024-09-17T02:44:32.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa493fff640 1 -- 172.21.15.131:0/3362365460 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5233 (secure 0 0 0) 0x7fa4840164b0 con 0x7fa49c15ba70 2024-09-17T02:44:32.212 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:44:32.212 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":37,"btime":"2024-09-17T02:44:03:291432+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3554877374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3554877374},{"type":"v1","addr":"172.21.15.189:6837","nonce":3554877374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1670590113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1670590113},{"type":"v1","addr":"172.21.15.131:6834","nonce":1670590113}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1046159039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1046159039},{"type":"v1","addr":"172.21.15.131:6835","nonce":1046159039}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:43:46.061550+0000","modified":"2024-09-17T02:44:02.721760+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5871":{"gid":5871,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":7,"addr":"172.21.15.189:6835/730392436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":730392436},{"type":"v1","addr":"172.21.15.189:6835","nonce":730392436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5871,"qdb_cluster":[5871]},"id":4}]} 2024-09-17T02:44:32.213 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 37 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 -- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa47c073200 msgr2=0x7fa47c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa47c073200 0x7fa47c075710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa48c005c50 tx=0x7fa48c005be0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 -- 172.21.15.131:0/3362365460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49c15ba70 msgr2=0x7fa49c1fa610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49c15ba70 0x7fa49c1fa610 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa4840b0040 tx=0x7fa484005e70 comp rx=0 tx=0).stop 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 -- 172.21.15.131:0/3362365460 shutdown_connections 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa47c073200 0x7fa47c075710 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa49c15be80 0x7fa49c1fab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa49c15ba70 0x7fa49c1fa610 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 --2- 172.21.15.131:0/3362365460 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa49c152550 0x7fa49c0aa1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 -- 172.21.15.131:0/3362365460 >> 172.21.15.131:0/3362365460 conn(0x7fa49c0d1130 msgr2=0x7fa49c0b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 -- 172.21.15.131:0/3362365460 shutdown_connections 2024-09-17T02:44:32.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.215+0000 7fa4a1e45640 1 -- 172.21.15.131:0/3362365460 wait complete. 2024-09-17T02:44:32.234 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-09-17T02:44:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.379+0000 7f0f437dc640 1 -- 172.21.15.131:0/1628916639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3c06e3e0 msgr2=0x7f0f3c15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.379+0000 7f0f437dc640 1 --2- 172.21.15.131:0/1628916639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3c06e3e0 0x7f0f3c15bea0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c06a1c0 tx=0x7f0f2c01c800 comp rx=0 tx=0).stop 2024-09-17T02:44:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.379+0000 7f0f437dc640 1 -- 172.21.15.131:0/1628916639 shutdown_connections 2024-09-17T02:44:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.379+0000 7f0f437dc640 1 --2- 172.21.15.131:0/1628916639 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3c06e3e0 0x7f0f3c15bea0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.379+0000 7f0f437dc640 1 --2- 172.21.15.131:0/1628916639 >> v2:172.21.15.189:3300/0 conn(0x7f0f3c0ced00 0x7f0f3c06dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.379+0000 7f0f437dc640 1 -- 172.21.15.131:0/1628916639 >> 172.21.15.131:0/1628916639 conn(0x7f0f3c06eea0 msgr2=0x7f0f3c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.379+0000 7f0f437dc640 1 -- 172.21.15.131:0/1628916639 shutdown_connections 2024-09-17T02:44:32.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.379+0000 7f0f437dc640 1 -- 172.21.15.131:0/1628916639 wait complete. 2024-09-17T02:44:32.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.379+0000 7f0f437dc640 1 Processor -- start 2024-09-17T02:44:32.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f437dc640 1 -- start start 2024-09-17T02:44:32.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f437dc640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0f3c06e3e0 0x7f0f3c14b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f437dc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3c0ced00 0x7f0f3c0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f437dc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c15c3e0 0x7f0f3c150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f437dc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c168bf0 con 0x7f0f3c0ced00 2024-09-17T02:44:32.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f437dc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c0cf310 con 0x7f0f3c15c3e0 2024-09-17T02:44:32.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f437dc640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0f3c168d70 con 0x7f0f3c06e3e0 2024-09-17T02:44:32.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f40d77640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3c0ced00 0x7f0f3c0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f40d77640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3c0ced00 0x7f0f3c0b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60722/0 (socket says 172.21.15.131:60722) 2024-09-17T02:44:32.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f40d77640 1 -- 172.21.15.131:0/1231860325 learned_addr learned my addr 172.21.15.131:0/1231860325 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:32.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f41d79640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c15c3e0 0x7f0f3c150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f41578640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0f3c06e3e0 0x7f0f3c14b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f40d77640 1 -- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0f3c06e3e0 msgr2=0x7f0f3c14b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f40d77640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0f3c06e3e0 0x7f0f3c14b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f40d77640 1 -- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c15c3e0 msgr2=0x7f0f3c150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f40d77640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c15c3e0 0x7f0f3c150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f40d77640 1 -- 172.21.15.131:0/1231860325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c0b4510 con 0x7f0f3c0ced00 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f40d77640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3c0ced00 0x7f0f3c0b3890 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009790 tx=0x7f0f2c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f31ffb640 1 -- 172.21.15.131:0/1231860325 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0bb070 con 0x7f0f3c0ced00 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f31ffb640 1 -- 172.21.15.131:0/1231860325 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f2c005030 con 0x7f0f3c0ced00 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f437dc640 1 -- 172.21.15.131:0/1231860325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c1f9e00 con 0x7f0f3c0ced00 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f31ffb640 1 -- 172.21.15.131:0/1231860325 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0b2040 con 0x7f0f3c0ced00 2024-09-17T02:44:32.383 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.383+0000 7f0f437dc640 1 -- 172.21.15.131:0/1231860325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c1fa3c0 con 0x7f0f3c0ced00 2024-09-17T02:44:32.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f31ffb640 1 -- 172.21.15.131:0/1231860325 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0f2c0be070 con 0x7f0f3c0ced00 2024-09-17T02:44:32.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f31ffb640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0f20073230 0x7f0f20075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f31ffb640 1 -- 172.21.15.131:0/1231860325 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f0f2c12df50 con 0x7f0f3c0ced00 2024-09-17T02:44:32.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f41578640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0f20073230 0x7f0f20075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f4aaa7000 1 Processor -- start 2024-09-17T02:44:32.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f4aaa7000 1 -- start start 2024-09-17T02:44:32.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f41578640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0f20073230 0x7f0f20075740 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005fd0 tx=0x7f0f38005d10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124d4b60 0x56201261a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124ca1c0 0x56201261a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201261ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5620124b8d00 con 0x5620124b3d90 2024-09-17T02:44:32.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56201242cc90 con 0x5620124d4b60 2024-09-17T02:44:32.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56201241c0c0 con 0x5620124ca1c0 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f41d79640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201261ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f41d79640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201261ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60730/0 (socket says 172.21.15.131:60730) 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f41d79640 1 -- 172.21.15.131:0/1408043846 learned_addr learned my addr 172.21.15.131:0/1408043846 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f40d77640 1 --2- 172.21.15.131:0/1408043846 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124ca1c0 0x56201261a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f41578640 1 --2- 172.21.15.131:0/1408043846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124d4b60 0x56201261a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f41d79640 1 -- 172.21.15.131:0/1408043846 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124ca1c0 msgr2=0x56201261a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f41d79640 1 --2- 172.21.15.131:0/1408043846 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124ca1c0 0x56201261a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f41d79640 1 -- 172.21.15.131:0/1408043846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124d4b60 msgr2=0x56201261a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f41d79640 1 --2- 172.21.15.131:0/1408043846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124d4b60 0x56201261a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.387+0000 7f0f41d79640 1 -- 172.21.15.131:0/1408043846 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620123f7290 con 0x5620124b3d90 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f41d79640 1 --2- 172.21.15.131:0/1408043846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201261ad60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0f34001f80 tx=0x7f0f34002070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233502bb4db92277 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f26ffd640 1 -- 172.21.15.131:0/1408043846 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3400ea00 con 0x5620124b3d90 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f26ffd640 1 -- 172.21.15.131:0/1408043846 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f3400eba0 con 0x5620124b3d90 2024-09-17T02:44:32.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f26ffd640 1 -- 172.21.15.131:0/1408043846 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f340135e0 con 0x5620124b3d90 2024-09-17T02:44:32.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/1408043846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 msgr2=0x56201261ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/1408043846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201261ad60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0f34001f80 tx=0x7f0f34002070 comp rx=0 tx=0).stop 2024-09-17T02:44:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/1408043846 shutdown_connections 2024-09-17T02:44:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/1408043846 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201261ad60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/1408043846 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124ca1c0 0x56201261a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/1408043846 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124d4b60 0x56201261a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/1408043846 >> 172.21.15.131:0/1408043846 conn(0x5620120d9760 msgr2=0x5620125fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/1408043846 shutdown_connections 2024-09-17T02:44:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/1408043846 wait complete. 2024-09-17T02:44:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 -- start start 2024-09-17T02:44:32.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 Processor -- start 2024-09-17T02:44:32.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x5620125d60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124ca1c0 0x5620125d6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124d4b60 0x5620125fc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5620123b6940 con 0x5620124b3d90 2024-09-17T02:44:32.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56201241fed0 con 0x5620124ca1c0 2024-09-17T02:44:32.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5620120e1820 con 0x5620124d4b60 2024-09-17T02:44:32.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f4aaa7000 10 client.0 _ll_register_callbacks cb 0x5620123fa7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:32.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f41578640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x5620125d60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f41578640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x5620125d60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60732/0 (socket says 172.21.15.131:60732) 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f41578640 1 -- 172.21.15.131:0/2141293053 learned_addr learned my addr 172.21.15.131:0/2141293053 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f40d77640 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124ca1c0 0x5620125d6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f41578640 1 -- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124d4b60 msgr2=0x5620125fc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f41d79640 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124d4b60 0x5620125fc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f41578640 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124d4b60 0x5620125fc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f41578640 1 -- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124ca1c0 msgr2=0x5620125d6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f41578640 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124ca1c0 0x5620125d6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f41578640 1 -- 172.21.15.131:0/2141293053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620123cf550 con 0x5620124b3d90 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f40d77640 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124ca1c0 0x5620125d6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f41578640 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x5620125d60d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0f380109b0 tx=0x7f0f380046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206b93506b7ec0d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.391+0000 7f0f1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f1bfff640 1 -- 172.21.15.131:0/2141293053 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800fd50 con 0x5620124b3d90 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f1bfff640 1 -- 172.21.15.131:0/2141293053 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f0f3801e430 con 0x5620124b3d90 2024-09-17T02:44:32.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/2141293053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5620123b4400 con 0x5620124b3d90 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f1bfff640 1 -- 172.21.15.131:0/2141293053 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0f3802e050 con 0x5620124b3d90 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 10 client.6153 fetch_fsmap learned FSMap version 37 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/2141293053 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x5620123ba480 con 0x5620124b3d90 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f1bfff640 1 -- 172.21.15.131:0/2141293053 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5620123b4400 con 0x5620124b3d90 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f1bfff640 1 -- 172.21.15.131:0/2141293053 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7f0f3801e630 con 0x5620124b3d90 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 10 client.6153 fetch_fsmap finished waiting for FSMap version 37 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 10 client.6153 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 20 client.6153 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5620120bead0 0x56201262b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 4 client.6153 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/2141293053 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56201242ebe0 con 0x5620120bead0 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f40d77640 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5620120bead0 0x56201262b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f1bfff640 10 client.6153 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:32.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f40d77640 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5620120bead0 0x56201262b8a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10162e408014b24c server_cookie=e6f3cf548595eaf6 in_seq=0 out_seq=0 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f1bfff640 1 -- 172.21.15.131:0/2141293053 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56201242ebe0 con 0x5620120bead0 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f1bfff640 10 client.6153 handle_command_reply: tid=0 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 client.6153 shutdown 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/2141293053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 msgr2=0x5620125d60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x5620125d60d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0f380109b0 tx=0x7f0f380046d0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/2141293053 shutdown_connections 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5620120bead0 0x56201262b8a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124d4b60 0x5620125fc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124ca1c0 0x5620125d6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f1bfff640 0 client.6153 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/2141293053 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x5620125d60d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/2141293053 >> 172.21.15.131:0/2141293053 conn(0x5620120d9760 msgr2=0x5620125ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/2141293053 shutdown_connections 2024-09-17T02:44:32.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/2141293053 wait complete. 2024-09-17T02:44:32.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.395+0000 7f0f4aaa7000 20 client.6153 trim_cache size 0 max 16384 2024-09-17T02:44:32.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f4aaa7000 1 Processor -- start 2024-09-17T02:44:32.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f4aaa7000 1 -- start start 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124b3d90 0x5620125d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124ca1c0 0x5620125d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124d4b60 0x56201262b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5620124dbfc0 con 0x5620124ca1c0 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5620123c7300 con 0x5620124b3d90 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56201242ba60 con 0x5620124d4b60 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f40d77640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124ca1c0 0x5620125d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f40d77640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124ca1c0 0x5620125d1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60742/0 (socket says 172.21.15.131:60742) 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f40d77640 1 -- 172.21.15.131:0/3586293709 learned_addr learned my addr 172.21.15.131:0/3586293709 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f40d77640 1 -- 172.21.15.131:0/3586293709 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124d4b60 msgr2=0x56201262b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f40d77640 1 --2- 172.21.15.131:0/3586293709 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124d4b60 0x56201262b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f40d77640 1 -- 172.21.15.131:0/3586293709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124b3d90 msgr2=0x5620125d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f40d77640 1 --2- 172.21.15.131:0/3586293709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124b3d90 0x5620125d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f40d77640 1 -- 172.21.15.131:0/3586293709 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620124a1d10 con 0x5620124ca1c0 2024-09-17T02:44:32.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f40d77640 1 --2- 172.21.15.131:0/3586293709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124ca1c0 0x5620125d1be0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c01cd70 tx=0x7f0f2c11a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c33367f5bed5fb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f267fc640 1 -- 172.21.15.131:0/3586293709 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0a58a0 con 0x5620124ca1c0 2024-09-17T02:44:32.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f267fc640 1 -- 172.21.15.131:0/3586293709 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f2c0f8280 con 0x5620124ca1c0 2024-09-17T02:44:32.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.411+0000 7f0f267fc640 1 -- 172.21.15.131:0/3586293709 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0f8e70 con 0x5620124ca1c0 2024-09-17T02:44:32.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/3586293709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124ca1c0 msgr2=0x5620125d1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/3586293709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124ca1c0 0x5620125d1be0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c01cd70 tx=0x7f0f2c11a040 comp rx=0 tx=0).stop 2024-09-17T02:44:32.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/3586293709 shutdown_connections 2024-09-17T02:44:32.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/3586293709 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124d4b60 0x56201262b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/3586293709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124ca1c0 0x5620125d1be0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/3586293709 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124b3d90 0x5620125d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/3586293709 >> 172.21.15.131:0/3586293709 conn(0x5620120d9760 msgr2=0x5620125d4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/3586293709 shutdown_connections 2024-09-17T02:44:32.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/3586293709 wait complete. 2024-09-17T02:44:32.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 -- start start 2024-09-17T02:44:32.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 Processor -- start 2024-09-17T02:44:32.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201262e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124ca1c0 0x56201262ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124d4b60 0x56201264bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5620123e8300 con 0x5620124b3d90 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5620123d9670 con 0x5620124d4b60 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56201241f1e0 con 0x5620124ca1c0 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 10 client.0 _ll_register_callbacks cb 0x7f0f38014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f41578640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201262e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f41578640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201262e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60756/0 (socket says 172.21.15.131:60756) 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f41578640 1 -- 172.21.15.131:0/827383756 learned_addr learned my addr 172.21.15.131:0/827383756 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f41d79640 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124d4b60 0x56201264bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f40d77640 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124ca1c0 0x56201262ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f41578640 1 -- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124ca1c0 msgr2=0x56201262ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f41578640 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124ca1c0 0x56201262ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f41578640 1 -- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124d4b60 msgr2=0x56201264bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f41578640 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124d4b60 0x56201264bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f41578640 1 -- 172.21.15.131:0/827383756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620123d2e50 con 0x5620124b3d90 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f41d79640 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124d4b60 0x56201264bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f41578640 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201262e8a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005790 tx=0x7f0f3801ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad5169c6c5e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f1bfff640 1 -- 172.21.15.131:0/827383756 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38036020 con 0x5620124b3d90 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f1bfff640 1 -- 172.21.15.131:0/827383756 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f0f3801e430 con 0x5620124b3d90 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/827383756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f2c0f8280 con 0x5620124b3d90 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f1bfff640 1 -- 172.21.15.131:0/827383756 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0f3801e630 con 0x5620124b3d90 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 10 client.6159 fetch_fsmap learned FSMap version 37 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.415+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/827383756 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x5620123c12d0 con 0x5620124b3d90 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f1bfff640 1 -- 172.21.15.131:0/827383756 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f2c0f8280 con 0x5620124b3d90 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f1bfff640 1 -- 172.21.15.131:0/827383756 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7f0f38032430 con 0x5620124b3d90 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 10 client.6159 fetch_fsmap finished waiting for FSMap version 37 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 10 client.6159 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T02:44:32.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 20 client.6159 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5620120acb30 0x56201268fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 4 client.6159 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/827383756 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x56201242a690 con 0x5620120acb30 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f40d77640 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5620120acb30 0x56201268fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f1bfff640 10 client.6159 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f40d77640 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5620120acb30 0x56201268fc40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696363475289bbde server_cookie=20316200936d39f7 in_seq=0 out_seq=0 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f1bfff640 1 -- 172.21.15.131:0/827383756 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x56201242a690 con 0x5620120acb30 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f1bfff640 10 client.6159 handle_command_reply: tid=0 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 client.6159 shutdown 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/827383756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 msgr2=0x56201262e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201262e8a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005790 tx=0x7f0f3801ede0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/827383756 shutdown_connections 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5620124d4b60 0x56201264bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5620120acb30 0x56201268fc40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5620124ca1c0 0x56201262ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 --2- 172.21.15.131:0/827383756 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5620124b3d90 0x56201262e8a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f1bfff640 0 client.6159 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:32.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/827383756 >> 172.21.15.131:0/827383756 conn(0x5620120d9760 msgr2=0x5620125d5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/827383756 shutdown_connections 2024-09-17T02:44:32.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 1 -- 172.21.15.131:0/827383756 wait complete. 2024-09-17T02:44:32.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f4aaa7000 20 client.6159 trim_cache size 0 max 16384 2024-09-17T02:44:32.421 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:44:32.421 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:44:32.421 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:44:32.422 INFO:teuthology.orchestra.run.smithi131.stdout: "damage_type": "backtrace", 2024-09-17T02:44:32.422 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 1864273176, 2024-09-17T02:44:32.422 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 1099511627777, 2024-09-17T02:44:32.422 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir_test/file" 2024-09-17T02:44:32.422 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:44:32.422 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:44:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 -- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0f20073230 msgr2=0x7f0f20075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0f20073230 0x7f0f20075740 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005fd0 tx=0x7f0f38005d10 comp rx=0 tx=0).stop 2024-09-17T02:44:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 -- 172.21.15.131:0/1231860325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3c0ced00 msgr2=0x7f0f3c0b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3c0ced00 0x7f0f3c0b3890 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c009790 tx=0x7f0f2c004040 comp rx=0 tx=0).stop 2024-09-17T02:44:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 -- 172.21.15.131:0/1231860325 shutdown_connections 2024-09-17T02:44:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0f20073230 0x7f0f20075740 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0f3c15c3e0 0x7f0f3c150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0f3c0ced00 0x7f0f3c0b3890 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 --2- 172.21.15.131:0/1231860325 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0f3c06e3e0 0x7f0f3c14b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 -- 172.21.15.131:0/1231860325 >> 172.21.15.131:0/1231860325 conn(0x7f0f3c06eea0 msgr2=0x7f0f3c0cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 -- 172.21.15.131:0/1231860325 shutdown_connections 2024-09-17T02:44:32.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.419+0000 7f0f437dc640 1 -- 172.21.15.131:0/1231860325 wait complete. 2024-09-17T02:44:32.435 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T02:44:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 -- 172.21.15.131:0/298964194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f814006e5b0 msgr2=0x7f814015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 --2- 172.21.15.131:0/298964194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f814006e5b0 0x7f814015bea0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f81400cfe00 tx=0x7f813001c800 comp rx=0 tx=0).stop 2024-09-17T02:44:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 -- 172.21.15.131:0/298964194 shutdown_connections 2024-09-17T02:44:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 --2- 172.21.15.131:0/298964194 >> v2:172.21.15.189:3300/0 conn(0x7f814015c3e0 0x7f81400cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 --2- 172.21.15.131:0/298964194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f814006e5b0 0x7f814015bea0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 -- 172.21.15.131:0/298964194 >> 172.21.15.131:0/298964194 conn(0x7f814006eea0 msgr2=0x7f814006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 -- 172.21.15.131:0/298964194 shutdown_connections 2024-09-17T02:44:32.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 -- 172.21.15.131:0/298964194 wait complete. 2024-09-17T02:44:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 Processor -- start 2024-09-17T02:44:32.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 -- start start 2024-09-17T02:44:32.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81400ced00 0x7f81401f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f814015c3e0 0x7f8140155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f81401534e0 0x7f81400b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f81401687a0 con 0x7f81400ced00 2024-09-17T02:44:32.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8140168620 con 0x7f814015c3e0 2024-09-17T02:44:32.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8147d23640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8140168920 con 0x7f81401534e0 2024-09-17T02:44:32.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8145abf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81400ced00 0x7f81401f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8145abf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81400ced00 0x7f81401f5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60774/0 (socket says 172.21.15.131:60774) 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8145abf640 1 -- 172.21.15.131:0/1333453733 learned_addr learned my addr 172.21.15.131:0/1333453733 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f81452be640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f814015c3e0 0x7f8140155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f81462c0640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f81401534e0 0x7f81400b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.575+0000 7f8145abf640 1 -- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f81401534e0 msgr2=0x7f81400b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f8145abf640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f81401534e0 0x7f81400b52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f8145abf640 1 -- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f814015c3e0 msgr2=0x7f8140155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f8145abf640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f814015c3e0 0x7f8140155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f8145abf640 1 -- 172.21.15.131:0/1333453733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8140165a00 con 0x7f81400ced00 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f81452be640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f814015c3e0 0x7f8140155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f81462c0640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f81401534e0 0x7f81400b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f8145abf640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81400ced00 0x7f81401f5f60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f813800c430 tx=0x7f813800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f81367fc640 1 -- 172.21.15.131:0/1333453733 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138011790 con 0x7f81400ced00 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f81367fc640 1 -- 172.21.15.131:0/1333453733 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8138017070 con 0x7f81400ced00 2024-09-17T02:44:32.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f81367fc640 1 -- 172.21.15.131:0/1333453733 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138011ce0 con 0x7f81400ced00 2024-09-17T02:44:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f8147d23640 1 -- 172.21.15.131:0/1333453733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8140165ca0 con 0x7f81400ced00 2024-09-17T02:44:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f8147d23640 1 -- 172.21.15.131:0/1333453733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81401f6790 con 0x7f81400ced00 2024-09-17T02:44:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f8147d23640 1 -- 172.21.15.131:0/1333453733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814015bea0 con 0x7f81400ced00 2024-09-17T02:44:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f81367fc640 1 -- 172.21.15.131:0/1333453733 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f813801c460 con 0x7f81400ced00 2024-09-17T02:44:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f81367fc640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8124073230 0x7f8124075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f81367fc640 1 -- 172.21.15.131:0/1333453733 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f81380940f0 con 0x7f81400ced00 2024-09-17T02:44:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f81452be640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8124073230 0x7f8124075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.579+0000 7f81452be640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8124073230 0x7f8124075740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f81300af000 tx=0x7f813001b040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.583+0000 7f81367fc640 1 -- 172.21.15.131:0/1333453733 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f813805f0d0 con 0x7f81400ced00 2024-09-17T02:44:32.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.787+0000 7f8147d23640 1 -- 172.21.15.131:0/1333453733 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f81401f69c0 con 0x7f81400ced00 2024-09-17T02:44:32.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.787+0000 7f81367fc640 1 -- 172.21.15.131:0/1333453733 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f813805e9d0 con 0x7f81400ced00 2024-09-17T02:44:32.785 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:44:32.785 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-09-17T02:44:32.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.787+0000 7f8147d23640 1 -- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8124073230 msgr2=0x7f8124075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.787+0000 7f8147d23640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8124073230 0x7f8124075740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f81300af000 tx=0x7f813001b040 comp rx=0 tx=0).stop 2024-09-17T02:44:32.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.787+0000 7f8147d23640 1 -- 172.21.15.131:0/1333453733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81400ced00 msgr2=0x7f81401f5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.787+0000 7f8147d23640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81400ced00 0x7f81401f5f60 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f813800c430 tx=0x7f813800c8f0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.787+0000 7f8147d23640 1 -- 172.21.15.131:0/1333453733 shutdown_connections 2024-09-17T02:44:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.787+0000 7f8147d23640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8124073230 0x7f8124075740 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.787+0000 7f8147d23640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f81401534e0 0x7f81400b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.787+0000 7f8147d23640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f814015c3e0 0x7f8140155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.787+0000 7f8147d23640 1 --2- 172.21.15.131:0/1333453733 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f81400ced00 0x7f81401f5f60 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.791+0000 7f8147d23640 1 -- 172.21.15.131:0/1333453733 >> 172.21.15.131:0/1333453733 conn(0x7f814006eea0 msgr2=0x7f81400745b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.791+0000 7f8147d23640 1 -- 172.21.15.131:0/1333453733 shutdown_connections 2024-09-17T02:44:32.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.791+0000 7f8147d23640 1 -- 172.21.15.131:0/1333453733 wait complete. 2024-09-17T02:44:32.805 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T02:44:32.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:44:32.805 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-09-17T02:44:32.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.959+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/113853089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf10004810 msgr2=0x7fdf10004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.959+0000 7fdf2bbe3640 1 --2- 172.21.15.131:0/113853089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf10004810 0x7fdf10004c20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fdf240b4830 tx=0x7fdf1401c670 comp rx=0 tx=0).stop 2024-09-17T02:44:32.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.959+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/113853089 shutdown_connections 2024-09-17T02:44:32.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.959+0000 7fdf2bbe3640 1 --2- 172.21.15.131:0/113853089 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf10004810 0x7fdf10004c20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.959+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/113853089 >> 172.21.15.131:0/113853089 conn(0x7fdf240d05a0 msgr2=0x7fdf240d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.959+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/113853089 shutdown_connections 2024-09-17T02:44:32.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.959+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/113853089 wait complete. 2024-09-17T02:44:32.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.959+0000 7fdf2bbe3640 1 Processor -- start 2024-09-17T02:44:32.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.959+0000 7fdf2bbe3640 1 -- start start 2024-09-17T02:44:32.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2bbe3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf10004810 0x7fdf2414b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2bbe3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf2415a8e0 0x7fdf24154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2bbe3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf240aed50 0x7fdf241551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2bbe3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2415c160 con 0x7fdf240aed50 2024-09-17T02:44:32.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2bbe3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2415bfe0 con 0x7fdf2415a8e0 2024-09-17T02:44:32.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2bbe3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdf2415c2e0 con 0x7fdf10004810 2024-09-17T02:44:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2a180640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf240aed50 0x7fdf241551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2a180640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf240aed50 0x7fdf241551d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60792/0 (socket says 172.21.15.131:60792) 2024-09-17T02:44:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2a180640 1 -- 172.21.15.131:0/2569028024 learned_addr learned my addr 172.21.15.131:0/2569028024 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2997f640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf10004810 0x7fdf2414b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2917e640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf2415a8e0 0x7fdf24154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2a180640 1 -- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf10004810 msgr2=0x7fdf2414b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2a180640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf10004810 0x7fdf2414b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2a180640 1 -- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf2415a8e0 msgr2=0x7fdf24154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2a180640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf2415a8e0 0x7fdf24154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2a180640 1 -- 172.21.15.131:0/2569028024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf24159380 con 0x7fdf240aed50 2024-09-17T02:44:32.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2a180640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf240aed50 0x7fdf241551d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdf241600d0 tx=0x7fdf1c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf1a7fc640 1 -- 172.21.15.131:0/2569028024 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c018070 con 0x7fdf240aed50 2024-09-17T02:44:32.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf1a7fc640 1 -- 172.21.15.131:0/2569028024 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf1c003b90 con 0x7fdf240aed50 2024-09-17T02:44:32.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf1a7fc640 1 -- 172.21.15.131:0/2569028024 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c005030 con 0x7fdf240aed50 2024-09-17T02:44:32.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/2569028024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf240aeb10 con 0x7fdf240aed50 2024-09-17T02:44:32.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.963+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/2569028024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf240adac0 con 0x7fdf240aed50 2024-09-17T02:44:32.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf1a7fc640 1 -- 172.21.15.131:0/2569028024 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdf1c01b070 con 0x7fdf240aed50 2024-09-17T02:44:32.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf1a7fc640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf08073200 0x7fdf08075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf1a7fc640 1 -- 172.21.15.131:0/2569028024 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fdf1c093970 con 0x7fdf240aed50 2024-09-17T02:44:32.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2997f640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf08073200 0x7fdf08075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf32eae000 1 Processor -- start 2024-09-17T02:44:32.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf32eae000 1 -- start start 2024-09-17T02:44:32.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2997f640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf08073200 0x7fdf08075710 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdf200045b0 tx=0x7fdf2000d410 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a8749eac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749eb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf32eae000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a8747fd8b0 con 0x55a874871ef0 2024-09-17T02:44:32.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf32eae000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a8747ecce0 con 0x55a874884e20 2024-09-17T02:44:32.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf32eae000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55a8747f0af0 con 0x55a8748716e0 2024-09-17T02:44:32.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2a180640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2a180640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749eb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60808/0 (socket says 172.21.15.131:60808) 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2a180640 1 -- 172.21.15.131:0/2545032214 learned_addr learned my addr 172.21.15.131:0/2545032214 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2997f640 1 --2- 172.21.15.131:0/2545032214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a8749eac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2917e640 1 --2- 172.21.15.131:0/2545032214 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749eb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2a180640 1 -- 172.21.15.131:0/2545032214 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 msgr2=0x55a8749eb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2a180640 1 --2- 172.21.15.131:0/2545032214 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749eb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2a180640 1 -- 172.21.15.131:0/2545032214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 msgr2=0x55a8749eac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2a180640 1 --2- 172.21.15.131:0/2545032214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a8749eac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2a180640 1 -- 172.21.15.131:0/2545032214 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8747a04e0 con 0x55a874871ef0 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf2a180640 1 --2- 172.21.15.131:0/2545032214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749eb6e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c05e110 tx=0x7fdf1c05e140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fe70b7cf3d850f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf077fe640 1 -- 172.21.15.131:0/2545032214 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c05e240 con 0x55a874871ef0 2024-09-17T02:44:32.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf077fe640 1 -- 172.21.15.131:0/2545032214 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf1c062c70 con 0x55a874871ef0 2024-09-17T02:44:32.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.967+0000 7fdf077fe640 1 -- 172.21.15.131:0/2545032214 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c067b20 con 0x55a874871ef0 2024-09-17T02:44:32.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 -- 172.21.15.131:0/2545032214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 msgr2=0x55a8749eb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 --2- 172.21.15.131:0/2545032214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749eb6e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c05e110 tx=0x7fdf1c05e140 comp rx=0 tx=0).stop 2024-09-17T02:44:32.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 -- 172.21.15.131:0/2545032214 shutdown_connections 2024-09-17T02:44:32.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 --2- 172.21.15.131:0/2545032214 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749eb6e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 --2- 172.21.15.131:0/2545032214 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749eb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 --2- 172.21.15.131:0/2545032214 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a8749eac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 -- 172.21.15.131:0/2545032214 >> 172.21.15.131:0/2545032214 conn(0x55a8748a8980 msgr2=0x55a87499fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 -- 172.21.15.131:0/2545032214 shutdown_connections 2024-09-17T02:44:32.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 -- 172.21.15.131:0/2545032214 wait complete. 2024-09-17T02:44:32.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 -- start start 2024-09-17T02:44:32.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 Processor -- start 2024-09-17T02:44:32.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749f10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a8749f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a8747aca20 con 0x55a874871ef0 2024-09-17T02:44:32.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a874796cc0 con 0x55a874884e20 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55a8747a4f70 con 0x55a8748716e0 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 10 client.0 _ll_register_callbacks cb 0x55a87498cb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2917e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2917e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60818/0 (socket says 172.21.15.131:60818) 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2917e640 1 -- 172.21.15.131:0/1550760059 learned_addr learned my addr 172.21.15.131:0/1550760059 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2a180640 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a8749f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2997f640 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749f10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2917e640 1 -- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 msgr2=0x55a8749f10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2917e640 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749f10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2917e640 1 -- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 msgr2=0x55a8749f1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2917e640 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a8749f1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2917e640 1 -- 172.21.15.131:0/1550760059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a874795ea0 con 0x55a874871ef0 2024-09-17T02:44:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2a180640 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a8749f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2997f640 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749f10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf2917e640 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f1620 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdf140a9040 tx=0x7fdf14006620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5740cee2e29b56b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1550760059 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf14004630 con 0x55a874871ef0 2024-09-17T02:44:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 -- 172.21.15.131:0/1550760059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a8747853d0 con 0x55a874871ef0 2024-09-17T02:44:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1550760059 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fdf140047d0 con 0x55a874871ef0 2024-09-17T02:44:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1550760059 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdf140c3050 con 0x55a874871ef0 2024-09-17T02:44:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 10 client.6180 fetch_fsmap learned FSMap version 37 2024-09-17T02:44:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.971+0000 7fdf32eae000 1 -- 172.21.15.131:0/1550760059 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55a874792500 con 0x55a874871ef0 2024-09-17T02:44:32.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1550760059 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a8747853d0 con 0x55a874871ef0 2024-09-17T02:44:32.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1550760059 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7fdf140c6020 con 0x55a874871ef0 2024-09-17T02:44:32.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 10 client.6180 fetch_fsmap finished waiting for FSMap version 37 2024-09-17T02:44:32.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 10 client.6180 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:44:32.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 20 client.6180 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:32.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55a87448f860 0x55a8749fb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 4 client.6180 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:44:32.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 -- 172.21.15.131:0/1550760059 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8747ff800 con 0x55a87448f860 2024-09-17T02:44:32.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf2997f640 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55a87448f860 0x55a8749fb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf04ff9640 10 client.6180 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:32.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf2997f640 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55a87448f860 0x55a8749fb4c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33df739e56c23fd3 server_cookie=2f1ad0d8569fdbb3 in_seq=0 out_seq=0 2024-09-17T02:44:32.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1550760059 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a8747ff800 con 0x55a87448f860 2024-09-17T02:44:32.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf04ff9640 10 client.6180 handle_command_reply: tid=0 2024-09-17T02:44:32.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 client.6180 shutdown 2024-09-17T02:44:32.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 -- 172.21.15.131:0/1550760059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 msgr2=0x55a8749f1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:32.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f1620 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdf140a9040 tx=0x7fdf14006620 comp rx=0 tx=0).stop 2024-09-17T02:44:32.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 -- 172.21.15.131:0/1550760059 shutdown_connections 2024-09-17T02:44:32.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a8749f1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f1620 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55a87448f860 0x55a8749fb4c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1550760059 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749f10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf04ff9640 0 client.6180 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:32.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 -- 172.21.15.131:0/1550760059 >> 172.21.15.131:0/1550760059 conn(0x55a8748a8980 msgr2=0x55a8749a0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:32.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 -- 172.21.15.131:0/1550760059 shutdown_connections 2024-09-17T02:44:32.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 1 -- 172.21.15.131:0/1550760059 wait complete. 2024-09-17T02:44:32.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.975+0000 7fdf32eae000 20 client.6180 trim_cache size 0 max 16384 2024-09-17T02:44:32.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf32eae000 1 Processor -- start 2024-09-17T02:44:32.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf32eae000 1 -- start start 2024-09-17T02:44:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a874a5b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874871ef0 0x55a874a5bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874884e20 0x55a874a5c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf32eae000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdf24204dd0 con 0x55a874884e20 2024-09-17T02:44:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf32eae000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdf24204130 con 0x55a874871ef0 2024-09-17T02:44:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf32eae000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdf24204780 con 0x55a8748716e0 2024-09-17T02:44:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf2a180640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874884e20 0x55a874a5c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf2a180640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874884e20 0x55a874a5c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60820/0 (socket says 172.21.15.131:60820) 2024-09-17T02:44:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf2a180640 1 -- 172.21.15.131:0/1013248321 learned_addr learned my addr 172.21.15.131:0/1013248321 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf2a180640 1 -- 172.21.15.131:0/1013248321 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 msgr2=0x55a874a5b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf2997f640 1 --2- 172.21.15.131:0/1013248321 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a874a5b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:32.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf2a180640 1 --2- 172.21.15.131:0/1013248321 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a874a5b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf2a180640 1 -- 172.21.15.131:0/1013248321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874871ef0 msgr2=0x55a874a5bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:32.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf2a180640 1 --2- 172.21.15.131:0/1013248321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874871ef0 0x55a874a5bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:32.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf2a180640 1 -- 172.21.15.131:0/1013248321 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8747badf0 con 0x55a874884e20 2024-09-17T02:44:32.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf2a180640 1 --2- 172.21.15.131:0/1013248321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874884e20 0x55a874a5c2a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c091d60 tx=0x7fdf1c091be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7099280cec02bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:32.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf06ffd640 1 -- 172.21.15.131:0/1013248321 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c07d070 con 0x55a874884e20 2024-09-17T02:44:32.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf06ffd640 1 -- 172.21.15.131:0/1013248321 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf1c067b20 con 0x55a874884e20 2024-09-17T02:44:32.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.995+0000 7fdf06ffd640 1 -- 172.21.15.131:0/1013248321 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c05a4b0 con 0x55a874884e20 2024-09-17T02:44:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 -- 172.21.15.131:0/1013248321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874884e20 msgr2=0x55a874a5c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1013248321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874884e20 0x55a874a5c2a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c091d60 tx=0x7fdf1c091be0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 -- 172.21.15.131:0/1013248321 shutdown_connections 2024-09-17T02:44:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1013248321 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874884e20 0x55a874a5c2a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1013248321 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874871ef0 0x55a874a5bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1013248321 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a874a5b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 -- 172.21.15.131:0/1013248321 >> 172.21.15.131:0/1013248321 conn(0x55a8748a8980 msgr2=0x55a87499f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 -- 172.21.15.131:0/1013248321 shutdown_connections 2024-09-17T02:44:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 -- 172.21.15.131:0/1013248321 wait complete. 2024-09-17T02:44:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 -- start start 2024-09-17T02:44:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 Processor -- start 2024-09-17T02:44:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749f7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a874a5bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a8747b9ea0 con 0x55a874871ef0 2024-09-17T02:44:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a8747fc680 con 0x55a874884e20 2024-09-17T02:44:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55a8747efe00 con 0x55a8748716e0 2024-09-17T02:44:33.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 10 client.0 _ll_register_callbacks cb 0x7fdf10004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf2917e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf2917e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60826/0 (socket says 172.21.15.131:60826) 2024-09-17T02:44:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf2917e640 1 -- 172.21.15.131:0/1114865109 learned_addr learned my addr 172.21.15.131:0/1114865109 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf2997f640 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749f7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf2917e640 1 -- 172.21.15.131:0/1114865109 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 msgr2=0x55a8749f7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf2917e640 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749f7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf2917e640 1 -- 172.21.15.131:0/1114865109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 msgr2=0x55a874a5bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf2917e640 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a874a5bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf2917e640 1 -- 172.21.15.131:0/1114865109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a87478b290 con 0x55a874871ef0 2024-09-17T02:44:33.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf2917e640 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f7a70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdf1401c4a0 tx=0x7fdf14004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fbdd314afe97e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1114865109 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf140b6040 con 0x55a874871ef0 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1114865109 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7fdf14006530 con 0x55a874871ef0 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 -- 172.21.15.131:0/1114865109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf1c067b20 con 0x55a874871ef0 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1114865109 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fdf140a7430 con 0x55a874871ef0 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 10 client.6186 fetch_fsmap learned FSMap version 37 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf32eae000 1 -- 172.21.15.131:0/1114865109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55a8744a3150 con 0x55a874871ef0 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:32.999+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1114865109 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf1c067b20 con 0x55a874871ef0 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1114865109 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7fdf140cf050 con 0x55a874871ef0 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 10 client.6186 fetch_fsmap finished waiting for FSMap version 37 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 10 client.6186 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:44:33.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 20 client.6186 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:33.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55a87447d8c0 0x55a874a1b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 4 client.6186 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-09-17T02:44:33.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 -- 172.21.15.131:0/1114865109 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x55a8747fb2b0 con 0x55a87447d8c0 2024-09-17T02:44:33.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf2997f640 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55a87447d8c0 0x55a874a1b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf04ff9640 10 client.6186 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:33.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf2997f640 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55a87447d8c0 0x55a874a1b9e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84eb2698c0e593d server_cookie=b11f5eedc0a7fdac in_seq=0 out_seq=0 2024-09-17T02:44:33.005 INFO:tasks.ceph.mds.d.smithi189.stderr:2024-09-17T02:44:33.001+0000 7fbd25b63640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-09-17T02:44:00.158448+0000 1=1+0) caps={6048=pLsXs/-@4} | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55b59e570580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-17T02:44:33.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf04ff9640 1 -- 172.21.15.131:0/1114865109 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a8747fb2b0 con 0x55a87447d8c0 2024-09-17T02:44:33.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf04ff9640 10 client.6186 handle_command_reply: tid=0 2024-09-17T02:44:33.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 client.6186 shutdown 2024-09-17T02:44:33.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 -- 172.21.15.131:0/1114865109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 msgr2=0x55a8749f7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f7a70 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdf1401c4a0 tx=0x7fdf14004a70 comp rx=0 tx=0).stop 2024-09-17T02:44:33.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 -- 172.21.15.131:0/1114865109 shutdown_connections 2024-09-17T02:44:33.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a874884e20 0x55a874a5bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a874871ef0 0x55a8749f7a70 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55a87447d8c0 0x55a874a1b9e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 --2- 172.21.15.131:0/1114865109 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a8748716e0 0x55a8749f7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf04ff9640 0 client.6186 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:33.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 -- 172.21.15.131:0/1114865109 >> 172.21.15.131:0/1114865109 conn(0x55a8748a8980 msgr2=0x55a8749a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:33.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 -- 172.21.15.131:0/1114865109 shutdown_connections 2024-09-17T02:44:33.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.003+0000 7fdf32eae000 1 -- 172.21.15.131:0/1114865109 wait complete. 2024-09-17T02:44:33.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf32eae000 20 client.6186 trim_cache size 0 max 16384 2024-09-17T02:44:33.007 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:44:33.007 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-09-17T02:44:33.008 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "3099326f-5d6e-47b0-bf0f-8774042d29fb", 2024-09-17T02:44:33.008 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-09-17T02:44:33.008 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:44:33.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf08073200 msgr2=0x7fdf08075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf08073200 0x7fdf08075710 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdf200045b0 tx=0x7fdf2000d410 comp rx=0 tx=0).stop 2024-09-17T02:44:33.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/2569028024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf240aed50 msgr2=0x7fdf241551d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf240aed50 0x7fdf241551d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fdf241600d0 tx=0x7fdf1c00dcd0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/2569028024 shutdown_connections 2024-09-17T02:44:33.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdf08073200 0x7fdf08075710 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdf240aed50 0x7fdf241551d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdf2415a8e0 0x7fdf24154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 --2- 172.21.15.131:0/2569028024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdf10004810 0x7fdf2414b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/2569028024 >> 172.21.15.131:0/2569028024 conn(0x7fdf240d05a0 msgr2=0x7fdf24073f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/2569028024 shutdown_connections 2024-09-17T02:44:33.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.007+0000 7fdf2bbe3640 1 -- 172.21.15.131:0/2569028024 wait complete. 2024-09-17T02:44:33.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:44:33.024 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-17T02:44:33.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.175+0000 7f90967f5640 1 -- 172.21.15.131:0/786340156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90900ce470 msgr2=0x7f90900ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.175+0000 7f90967f5640 1 --2- 172.21.15.131:0/786340156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90900ce470 0x7f90900ce880 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f909015f2e0 tx=0x7f908801ccd0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.175+0000 7f90967f5640 1 -- 172.21.15.131:0/786340156 shutdown_connections 2024-09-17T02:44:33.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.175+0000 7f90967f5640 1 --2- 172.21.15.131:0/786340156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90900ce470 0x7f90900ce880 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.175+0000 7f90967f5640 1 --2- 172.21.15.131:0/786340156 >> v2:172.21.15.189:3300/0 conn(0x7f90900cac80 0x7f90900cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.175+0000 7f90967f5640 1 -- 172.21.15.131:0/786340156 >> 172.21.15.131:0/786340156 conn(0x7f909006eea0 msgr2=0x7f909006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:33.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.175+0000 7f90967f5640 1 -- 172.21.15.131:0/786340156 shutdown_connections 2024-09-17T02:44:33.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.175+0000 7f90967f5640 1 -- 172.21.15.131:0/786340156 wait complete. 2024-09-17T02:44:33.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.175+0000 7f90967f5640 1 Processor -- start 2024-09-17T02:44:33.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.175+0000 7f90967f5640 1 -- start start 2024-09-17T02:44:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f90967f5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f909006dea0 0x7f90900c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f90967f5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90900cac80 0x7f90900c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f90967f5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90900ce470 0x7f90900c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f90967f5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f909016c010 con 0x7f90900ce470 2024-09-17T02:44:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f90967f5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f909016be90 con 0x7f90900cac80 2024-09-17T02:44:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f90967f5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f909016c190 con 0x7f909006dea0 2024-09-17T02:44:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f9094d92640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90900ce470 0x7f90900c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f9094d92640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90900ce470 0x7f90900c8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60856/0 (socket says 172.21.15.131:60856) 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f9094d92640 1 -- 172.21.15.131:0/1278166941 learned_addr learned my addr 172.21.15.131:0/1278166941 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f9087fff640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90900cac80 0x7f90900c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f9094d92640 1 -- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f909006dea0 msgr2=0x7f90900c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f908ffff640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f909006dea0 0x7f90900c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f9094d92640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f909006dea0 0x7f90900c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f9094d92640 1 -- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90900cac80 msgr2=0x7f90900c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f9094d92640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90900cac80 0x7f90900c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f9094d92640 1 -- 172.21.15.131:0/1278166941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9090149050 con 0x7f90900ce470 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f9087fff640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90900cac80 0x7f90900c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f9094d92640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90900ce470 0x7f90900c8a50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f90880198e0 tx=0x7f9088005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f908d7fa640 1 -- 172.21.15.131:0/1278166941 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90880b2040 con 0x7f90900ce470 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f908d7fa640 1 -- 172.21.15.131:0/1278166941 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9088004240 con 0x7f90900ce470 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f90967f5640 1 -- 172.21.15.131:0/1278166941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9090149330 con 0x7f90900ce470 2024-09-17T02:44:33.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f908d7fa640 1 -- 172.21.15.131:0/1278166941 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088004650 con 0x7f90900ce470 2024-09-17T02:44:33.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f90967f5640 1 -- 172.21.15.131:0/1278166941 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90901498f0 con 0x7f90900ce470 2024-09-17T02:44:33.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.179+0000 7f908d7fa640 1 -- 172.21.15.131:0/1278166941 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f90880ad440 con 0x7f90900ce470 2024-09-17T02:44:33.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f908d7fa640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9074073230 0x7f9074075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f908d7fa640 1 -- 172.21.15.131:0/1278166941 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f908812ebe0 con 0x7f90900ce470 2024-09-17T02:44:33.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f908ffff640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9074073230 0x7f9074075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f909dac0000 1 Processor -- start 2024-09-17T02:44:33.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f909dac0000 1 -- start start 2024-09-17T02:44:33.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f908ffff640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9074073230 0x7f9074075740 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9080001570 tx=0x7f9080009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:33.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f909dac0000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b038196d0 0x556b039859f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f909dac0000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b03985f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f909dac0000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b03534aa0 0x556b03986470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f909dac0000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556b03824cd0 con 0x556b03470890 2024-09-17T02:44:33.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f909dac0000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556b0373a330 con 0x556b038196d0 2024-09-17T02:44:33.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f909dac0000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x556b03797bb0 con 0x556b03534aa0 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9087fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b03985f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9087fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b03985f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60872/0 (socket says 172.21.15.131:60872) 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9087fff640 1 -- 172.21.15.131:0/2261011068 learned_addr learned my addr 172.21.15.131:0/2261011068 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f908ffff640 1 --2- 172.21.15.131:0/2261011068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b038196d0 0x556b039859f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9094d92640 1 --2- 172.21.15.131:0/2261011068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b03534aa0 0x556b03986470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9087fff640 1 -- 172.21.15.131:0/2261011068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b03534aa0 msgr2=0x556b03986470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9087fff640 1 --2- 172.21.15.131:0/2261011068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b03534aa0 0x556b03986470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9087fff640 1 -- 172.21.15.131:0/2261011068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b038196d0 msgr2=0x556b039859f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9087fff640 1 --2- 172.21.15.131:0/2261011068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b038196d0 0x556b039859f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9087fff640 1 -- 172.21.15.131:0/2261011068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b0373bc80 con 0x556b03470890 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f908ffff640 1 --2- 172.21.15.131:0/2261011068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b038196d0 0x556b039859f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9087fff640 1 --2- 172.21.15.131:0/2261011068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b03985f30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9078009b40 tx=0x7f907800e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1531f7fab280c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9085ffb640 1 -- 172.21.15.131:0/2261011068 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078016070 con 0x556b03470890 2024-09-17T02:44:33.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9085ffb640 1 -- 172.21.15.131:0/2261011068 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f907800ee10 con 0x556b03470890 2024-09-17T02:44:33.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f9085ffb640 1 -- 172.21.15.131:0/2261011068 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078005280 con 0x556b03470890 2024-09-17T02:44:33.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f909dac0000 1 -- 172.21.15.131:0/2261011068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 msgr2=0x556b03985f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f909dac0000 1 --2- 172.21.15.131:0/2261011068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b03985f30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f9078009b40 tx=0x7f907800e2f0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.183+0000 7f909dac0000 1 -- 172.21.15.131:0/2261011068 shutdown_connections 2024-09-17T02:44:33.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 --2- 172.21.15.131:0/2261011068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b03534aa0 0x556b03986470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 --2- 172.21.15.131:0/2261011068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b03985f30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 --2- 172.21.15.131:0/2261011068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b038196d0 0x556b039859f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 -- 172.21.15.131:0/2261011068 >> 172.21.15.131:0/2261011068 conn(0x556b034445c0 msgr2=0x556b03942ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:33.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 -- 172.21.15.131:0/2261011068 shutdown_connections 2024-09-17T02:44:33.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 -- 172.21.15.131:0/2261011068 wait complete. 2024-09-17T02:44:33.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 -- start start 2024-09-17T02:44:33.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 Processor -- start 2024-09-17T02:44:33.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03470890 0x556b039419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b03534aa0 0x556b03941ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b038196d0 0x556b039485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556b0378adf0 con 0x556b038196d0 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556b03786fe0 con 0x556b03470890 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x556b0372c4d0 con 0x556b03534aa0 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f9094d92640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b038196d0 0x556b039485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 10 client.0 _ll_register_callbacks cb 0x556b0370a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f9094d92640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b038196d0 0x556b039485f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60882/0 (socket says 172.21.15.131:60882) 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f9094d92640 1 -- 172.21.15.131:0/2552491223 learned_addr learned my addr 172.21.15.131:0/2552491223 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f908ffff640 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03470890 0x556b039419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f9087fff640 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b03534aa0 0x556b03941ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f9094d92640 1 -- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b03534aa0 msgr2=0x556b03941ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f9094d92640 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b03534aa0 0x556b03941ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f9094d92640 1 -- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03470890 msgr2=0x556b039419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f9094d92640 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03470890 0x556b039419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f9094d92640 1 -- 172.21.15.131:0/2552491223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b03725210 con 0x556b038196d0 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f908ffff640 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03470890 0x556b039419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f9094d92640 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b038196d0 0x556b039485f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f90880f94d0 tx=0x7f90880f8cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491ab895ce2f6f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f907effd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f907effd640 1 -- 172.21.15.131:0/2552491223 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90880fdef0 con 0x556b038196d0 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f907effd640 1 -- 172.21.15.131:0/2552491223 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f9088102910 con 0x556b038196d0 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 -- 172.21.15.131:0/2552491223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b0371f350 con 0x556b038196d0 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f907effd640 1 -- 172.21.15.131:0/2552491223 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9088102da0 con 0x556b038196d0 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 10 client.6201 fetch_fsmap learned FSMap version 37 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 -- 172.21.15.131:0/2552491223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x556b0370bfc0 con 0x556b038196d0 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f907effd640 1 -- 172.21.15.131:0/2552491223 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556b0371f350 con 0x556b038196d0 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f907effd640 1 -- 172.21.15.131:0/2552491223 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7f90880b5070 con 0x556b038196d0 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 10 client.6201 fetch_fsmap finished waiting for FSMap version 37 2024-09-17T02:44:33.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 10 client.6201 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:44:33.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 20 client.6201 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:33.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x556b03429930 0x556b0398ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 4 client.6201 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:44:33.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.187+0000 7f909dac0000 1 -- 172.21.15.131:0/2552491223 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b03799b00 con 0x556b03429930 2024-09-17T02:44:33.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f908ffff640 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x556b03429930 0x556b0398ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f907effd640 10 client.6201 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:33.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f908ffff640 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x556b03429930 0x556b0398ce80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff375c1b1a268a60 server_cookie=dc35c459ab084263 in_seq=0 out_seq=0 2024-09-17T02:44:33.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f907effd640 1 -- 172.21.15.131:0/2552491223 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b03799b00 con 0x556b03429930 2024-09-17T02:44:33.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f907effd640 10 client.6201 handle_command_reply: tid=0 2024-09-17T02:44:33.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 1 client.6201 shutdown 2024-09-17T02:44:33.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 1 -- 172.21.15.131:0/2552491223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b038196d0 msgr2=0x556b039485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b038196d0 0x556b039485f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f90880f94d0 tx=0x7f90880f8cb0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 1 -- 172.21.15.131:0/2552491223 shutdown_connections 2024-09-17T02:44:33.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b038196d0 0x556b039485f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x556b03429930 0x556b0398ce80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b03534aa0 0x556b03941ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 1 --2- 172.21.15.131:0/2552491223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03470890 0x556b039419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f907effd640 0 client.6201 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:33.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 1 -- 172.21.15.131:0/2552491223 >> 172.21.15.131:0/2552491223 conn(0x556b034445c0 msgr2=0x556b0393ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:33.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 1 -- 172.21.15.131:0/2552491223 shutdown_connections 2024-09-17T02:44:33.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 1 -- 172.21.15.131:0/2552491223 wait complete. 2024-09-17T02:44:33.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.191+0000 7f909dac0000 20 client.6201 trim_cache size 0 max 16384 2024-09-17T02:44:33.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.207+0000 7f909dac0000 1 Processor -- start 2024-09-17T02:44:33.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 -- start start 2024-09-17T02:44:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03470890 0x556b039f7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03534aa0 0x556b039f7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b038196d0 0x556b039f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556b03940b20 con 0x556b03534aa0 2024-09-17T02:44:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9090205950 con 0x556b03470890 2024-09-17T02:44:33.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f909015d830 con 0x556b038196d0 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f9087fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03534aa0 0x556b039f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f9087fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03534aa0 0x556b039f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60884/0 (socket says 172.21.15.131:60884) 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f9087fff640 1 -- 172.21.15.131:0/952835838 learned_addr learned my addr 172.21.15.131:0/952835838 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f9087fff640 1 -- 172.21.15.131:0/952835838 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b038196d0 msgr2=0x556b039f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f9087fff640 1 --2- 172.21.15.131:0/952835838 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b038196d0 0x556b039f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f9087fff640 1 -- 172.21.15.131:0/952835838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03470890 msgr2=0x556b039f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f9087fff640 1 --2- 172.21.15.131:0/952835838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03470890 0x556b039f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f9087fff640 1 -- 172.21.15.131:0/952835838 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b037e4680 con 0x556b03534aa0 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f9087fff640 1 --2- 172.21.15.131:0/952835838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03534aa0 0x556b039f7940 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x556b0383a760 tx=0x7f90780046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942e6c3f96d22c65 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f90857fa640 1 -- 172.21.15.131:0/952835838 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078028070 con 0x556b03534aa0 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f90857fa640 1 -- 172.21.15.131:0/952835838 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f907800ed60 con 0x556b03534aa0 2024-09-17T02:44:33.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f90857fa640 1 -- 172.21.15.131:0/952835838 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078016040 con 0x556b03534aa0 2024-09-17T02:44:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 -- 172.21.15.131:0/952835838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03534aa0 msgr2=0x556b039f7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 --2- 172.21.15.131:0/952835838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03534aa0 0x556b039f7940 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x556b0383a760 tx=0x7f90780046d0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 -- 172.21.15.131:0/952835838 shutdown_connections 2024-09-17T02:44:33.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 --2- 172.21.15.131:0/952835838 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b038196d0 0x556b039f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 --2- 172.21.15.131:0/952835838 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03534aa0 0x556b039f7940 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 --2- 172.21.15.131:0/952835838 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03470890 0x556b039f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 -- 172.21.15.131:0/952835838 >> 172.21.15.131:0/952835838 conn(0x556b034445c0 msgr2=0x556b03997fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 -- 172.21.15.131:0/952835838 shutdown_connections 2024-09-17T02:44:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 -- 172.21.15.131:0/952835838 wait complete. 2024-09-17T02:44:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 -- start start 2024-09-17T02:44:33.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.211+0000 7f909dac0000 1 Processor -- start 2024-09-17T02:44:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b039fb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03534aa0 0x556b039fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b038196d0 0x556b039fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f909015d1e0 con 0x556b03470890 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f90901f94e0 con 0x556b03534aa0 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x556b0344c680 con 0x556b038196d0 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 10 client.0 _ll_register_callbacks cb 0x7f9088100040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f908ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b039fb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f908ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b039fb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:60896/0 (socket says 172.21.15.131:60896) 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f908ffff640 1 -- 172.21.15.131:0/3993790967 learned_addr learned my addr 172.21.15.131:0/3993790967 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f908ffff640 1 -- 172.21.15.131:0/3993790967 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b038196d0 msgr2=0x556b039fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f908ffff640 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b038196d0 0x556b039fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f908ffff640 1 -- 172.21.15.131:0/3993790967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03534aa0 msgr2=0x556b039fbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f9087fff640 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03534aa0 0x556b039fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f908ffff640 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03534aa0 0x556b039fbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f908ffff640 1 -- 172.21.15.131:0/3993790967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b037510b0 con 0x556b03470890 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f908ffff640 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b039fb650 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f90800148a0 tx=0x7f9080000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=845bdabcc37e034c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f907effd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f907effd640 1 -- 172.21.15.131:0/3993790967 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908002c020 con 0x556b03470890 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f907effd640 1 -- 172.21.15.131:0/3993790967 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(65..65 src has 1..65) ==== 6101+0+0 (secure 0 0 0) 0x7f9080028450 con 0x556b03470890 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 -- 172.21.15.131:0/3993790967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f907800ed60 con 0x556b03470890 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f907effd640 1 -- 172.21.15.131:0/3993790967 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9080036050 con 0x556b03470890 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 10 client.6207 fetch_fsmap learned FSMap version 37 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 -- 172.21.15.131:0/3993790967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x556b0371d610 con 0x556b03470890 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f907effd640 1 -- 172.21.15.131:0/3993790967 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f907800ed60 con 0x556b03470890 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f907effd640 1 -- 172.21.15.131:0/3993790967 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7f9080039020 con 0x556b03470890 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 10 client.6207 fetch_fsmap finished waiting for FSMap version 37 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 10 client.6207 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:44:33.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 20 client.6207 populate_metadata read hostname 'smithi131' 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x556b03417990 0x556b0393ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 4 client.6207 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 -- 172.21.15.131:0/3993790967 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556b037955b0 con 0x556b03417990 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f9087fff640 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x556b03417990 0x556b0393ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f907effd640 10 client.6207 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f9087fff640 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x556b03417990 0x556b0393ccd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ff33c74f4401db7 server_cookie=efd09a0491209aed in_seq=0 out_seq=0 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f907effd640 1 -- 172.21.15.131:0/3993790967 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+861 (crc 0 0 0) 0x556b037955b0 con 0x556b03417990 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f907effd640 10 client.6207 handle_command_reply: tid=0 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 client.6207 shutdown 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 -- 172.21.15.131:0/3993790967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 msgr2=0x556b039fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b039fb650 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f90800148a0 tx=0x7f9080000ee0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 -- 172.21.15.131:0/3993790967 shutdown_connections 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x556b03417990 0x556b0393ccd0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556b038196d0 0x556b039fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556b03534aa0 0x556b039fbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f907effd640 0 client.6207 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:44:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 --2- 172.21.15.131:0/3993790967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556b03470890 0x556b039fb650 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.215+0000 7f909dac0000 1 -- 172.21.15.131:0/3993790967 >> 172.21.15.131:0/3993790967 conn(0x556b034445c0 msgr2=0x556b039475a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f909dac0000 1 -- 172.21.15.131:0/3993790967 shutdown_connections 2024-09-17T02:44:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f909dac0000 1 -- 172.21.15.131:0/3993790967 wait complete. 2024-09-17T02:44:33.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f909dac0000 20 client.6207 trim_cache size 0 max 16384 2024-09-17T02:44:33.219 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:44:33.219 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-17T02:44:33.219 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "3099326f-5d6e-47b0-bf0f-8774042d29fb": { 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "3099326f-5d6e-47b0-bf0f-8774042d29fb", 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,repair,force" 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "3099326f-5d6e-47b0-bf0f-8774042d29fb": { 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:44:33.001222+0000", 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:44:33.220 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:44:33.221 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:44:33.221 INFO:teuthology.orchestra.run.smithi131.stdout: "7c955166-b5f6-4490-8c58-d278c543ef77": { 2024-09-17T02:44:33.221 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:44:01.373757+0000", 2024-09-17T02:44:33.221 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:44:33.221 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:44:33.221 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:44:33.221 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:44:33.221 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:44:33.221 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:44:33.224 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:44:33.224 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:44:33.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 -- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9074073230 msgr2=0x7f9074075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9074073230 0x7f9074075740 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9080001570 tx=0x7f9080009040 comp rx=0 tx=0).stop 2024-09-17T02:44:33.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 -- 172.21.15.131:0/1278166941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90900ce470 msgr2=0x7f90900c8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:44:33.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90900ce470 0x7f90900c8a50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f90880198e0 tx=0x7f9088005b10 comp rx=0 tx=0).stop 2024-09-17T02:44:33.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 -- 172.21.15.131:0/1278166941 shutdown_connections 2024-09-17T02:44:33.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9074073230 0x7f9074075740 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f90900ce470 0x7f90900c8a50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f90900cac80 0x7f90900c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 --2- 172.21.15.131:0/1278166941 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f909006dea0 0x7f90900c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:44:33.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 -- 172.21.15.131:0/1278166941 >> 172.21.15.131:0/1278166941 conn(0x7f909006eea0 msgr2=0x7f9090074570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:44:33.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 -- 172.21.15.131:0/1278166941 shutdown_connections 2024-09-17T02:44:33.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:44:33.219+0000 7f90967f5640 1 -- 172.21.15.131:0/1278166941 wait complete. 2024-09-17T02:44:33.232 INFO:tasks.cephfs.filesystem:scrub status for tag:3099326f-5d6e-47b0-bf0f-8774042d29fb - {'path': '/', 'tag': '3099326f-5d6e-47b0-bf0f-8774042d29fb', 'options': 'recursive,repair,force'} 2024-09-17T02:44:44.143 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:44:44.146 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:45:03.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:03.235 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-17T02:45:03.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.402+0000 7f8d96238640 1 -- 172.21.15.131:0/2499604600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d78004810 msgr2=0x7f8d78004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.402+0000 7f8d96238640 1 --2- 172.21.15.131:0/2499604600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d78004810 0x7f8d78004c20 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8d8400bb10 tx=0x7f8d8401cde0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.402+0000 7f8d8ffff640 1 -- 172.21.15.131:0/2499604600 reap_dead start 2024-09-17T02:45:03.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.402+0000 7f8d96238640 1 -- 172.21.15.131:0/2499604600 shutdown_connections 2024-09-17T02:45:03.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.402+0000 7f8d96238640 1 -- 172.21.15.131:0/2499604600 >> 172.21.15.131:0/2499604600 conn(0x7f8d9006eea0 msgr2=0x7f8d9006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:03.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.402+0000 7f8d96238640 1 -- 172.21.15.131:0/2499604600 shutdown_connections 2024-09-17T02:45:03.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.402+0000 7f8d96238640 1 -- 172.21.15.131:0/2499604600 wait complete. 2024-09-17T02:45:03.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d96238640 1 Processor -- start 2024-09-17T02:45:03.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d96238640 1 -- start start 2024-09-17T02:45:03.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d96238640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d90158540 0x7f8d901f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d96238640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d901f5f60 0x7f8d9014a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d96238640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8d9015adf0 0x7f8d9015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d96238640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d900cbaf0 con 0x7f8d901f5f60 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d96238640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8d900cf320 con 0x7f8d90158540 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d96238640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8d900cbc70 con 0x7f8d9015adf0 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d901f5f60 0x7f8d9014a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d901f5f60 0x7f8d9014a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56552/0 (socket says 172.21.15.131:56552) 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8effd640 1 -- 172.21.15.131:0/3360194813 learned_addr learned my addr 172.21.15.131:0/3360194813 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d90158540 0x7f8d901f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8ffff640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8d9015adf0 0x7f8d9015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8effd640 1 -- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8d9015adf0 msgr2=0x7f8d9015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8effd640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8d9015adf0 0x7f8d9015b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8effd640 1 -- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d90158540 msgr2=0x7f8d901f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8effd640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d90158540 0x7f8d901f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8effd640 1 -- 172.21.15.131:0/3360194813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9015b740 con 0x7f8d901f5f60 2024-09-17T02:45:03.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d90158540 0x7f8d901f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:03.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d8effd640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d901f5f60 0x7f8d9014a3c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8d84000c10 tx=0x7f8d84004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:03.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d77fff640 1 -- 172.21.15.131:0/3360194813 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840be070 con 0x7f8d901f5f60 2024-09-17T02:45:03.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d77fff640 1 -- 172.21.15.131:0/3360194813 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d84005330 con 0x7f8d901f5f60 2024-09-17T02:45:03.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d77fff640 1 -- 172.21.15.131:0/3360194813 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d840ab050 con 0x7f8d901f5f60 2024-09-17T02:45:03.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d96238640 1 -- 172.21.15.131:0/3360194813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d90158f00 con 0x7f8d901f5f60 2024-09-17T02:45:03.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.406+0000 7f8d96238640 1 -- 172.21.15.131:0/3360194813 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d900c3fc0 con 0x7f8d901f5f60 2024-09-17T02:45:03.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.410+0000 7f8d77fff640 1 -- 172.21.15.131:0/3360194813 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8d840b6070 con 0x7f8d901f5f60 2024-09-17T02:45:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.410+0000 7f8d77fff640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d7c073200 0x7f8d7c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.410+0000 7f8d77fff640 1 -- 172.21.15.131:0/3360194813 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f8d8412e5a0 con 0x7f8d901f5f60 2024-09-17T02:45:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.410+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d7c073200 0x7f8d7c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.410+0000 7f8d9d503000 1 Processor -- start 2024-09-17T02:45:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.410+0000 7f8d9d503000 1 -- start start 2024-09-17T02:45:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.410+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d7c073200 0x7f8d7c075710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8d900b5fe0 tx=0x7f8d80001570 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad948049f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad942ef890 0x55ad94804f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad94805470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ad946a3cd0 con 0x55ad946986d0 2024-09-17T02:45:03.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ad945b9330 con 0x55ad943b3aa0 2024-09-17T02:45:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55ad94616bb0 con 0x55ad942ef890 2024-09-17T02:45:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad948049f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad948049f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56564/0 (socket says 172.21.15.131:56564) 2024-09-17T02:45:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8f7fe640 1 -- 172.21.15.131:0/3563807962 learned_addr learned my addr 172.21.15.131:0/3563807962 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8effd640 1 --2- 172.21.15.131:0/3563807962 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad942ef890 0x55ad94804f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8ffff640 1 --2- 172.21.15.131:0/3563807962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad94805470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8f7fe640 1 -- 172.21.15.131:0/3563807962 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad942ef890 msgr2=0x55ad94804f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/3563807962 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad942ef890 0x55ad94804f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8f7fe640 1 -- 172.21.15.131:0/3563807962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 msgr2=0x55ad94805470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/3563807962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad94805470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8f7fe640 1 -- 172.21.15.131:0/3563807962 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad945bac80 con 0x55ad946986d0 2024-09-17T02:45:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8effd640 1 --2- 172.21.15.131:0/3563807962 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad942ef890 0x55ad94804f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8ffff640 1 --2- 172.21.15.131:0/3563807962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad94805470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/3563807962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad948049f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8d80010c30 tx=0x7f8d80013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8359362a9ec145b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d757fa640 1 -- 172.21.15.131:0/3563807962 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80022070 con 0x55ad946986d0 2024-09-17T02:45:03.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d757fa640 1 -- 172.21.15.131:0/3563807962 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d80012e30 con 0x55ad946986d0 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d757fa640 1 -- 172.21.15.131:0/3563807962 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8001e670 con 0x55ad946986d0 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 -- 172.21.15.131:0/3563807962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 msgr2=0x55ad948049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 --2- 172.21.15.131:0/3563807962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad948049f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f8d80010c30 tx=0x7f8d80013000 comp rx=0 tx=0).stop 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 -- 172.21.15.131:0/3563807962 shutdown_connections 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 --2- 172.21.15.131:0/3563807962 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad94805470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 --2- 172.21.15.131:0/3563807962 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad942ef890 0x55ad94804f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 --2- 172.21.15.131:0/3563807962 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad948049f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 -- 172.21.15.131:0/3563807962 >> 172.21.15.131:0/3563807962 conn(0x55ad942c35c0 msgr2=0x55ad947c1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 -- 172.21.15.131:0/3563807962 shutdown_connections 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 -- 172.21.15.131:0/3563807962 wait complete. 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 -- start start 2024-09-17T02:45:03.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.414+0000 7f8d9d503000 1 Processor -- start 2024-09-17T02:45:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad942ef890 0x55ad947ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad947ba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad947bae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ad94609df0 con 0x55ad946986d0 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ad94605fe0 con 0x55ad943b3aa0 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55ad945ab4d0 con 0x55ad942ef890 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 10 client.0 _ll_register_callbacks cb 0x55ad945897f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad947bae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad947bae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56566/0 (socket says 172.21.15.131:56566) 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8ffff640 1 -- 172.21.15.131:0/2317129450 learned_addr learned my addr 172.21.15.131:0/2317129450 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad942ef890 0x55ad947ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8effd640 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad947ba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8ffff640 1 -- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad942ef890 msgr2=0x55ad947ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8ffff640 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad942ef890 0x55ad947ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8ffff640 1 -- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 msgr2=0x55ad947ba900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8ffff640 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad947ba900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8ffff640 1 -- 172.21.15.131:0/2317129450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad945a4210 con 0x55ad946986d0 2024-09-17T02:45:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8ffff640 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad947bae40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8d9015ffe0 tx=0x7f8d880127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b00d13b794ead85 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d6e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/2317129450 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8801b640 con 0x55ad946986d0 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/2317129450 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f8d88010450 con 0x55ad946986d0 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 1 -- 172.21.15.131:0/2317129450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ad9459e350 con 0x55ad946986d0 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/2317129450 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8d8801b7e0 con 0x55ad946986d0 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 10 client.6219 fetch_fsmap learned FSMap version 38 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 1 -- 172.21.15.131:0/2317129450 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x55ad9458afc0 con 0x55ad946986d0 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/2317129450 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55ad9459e350 con 0x55ad946986d0 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/2317129450 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 38) ==== 3502+0+0 (secure 0 0 0) 0x7f8d88027080 con 0x55ad946986d0 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 10 client.6219 fetch_fsmap finished waiting for FSMap version 38 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 10 client.6219 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 20 client.6219 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55ad942a8930 0x55ad947bb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 4 client.6219 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d9d503000 1 -- 172.21.15.131:0/2317129450 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad94618b00 con 0x55ad942a8930 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55ad942a8930 0x55ad947bb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d6e7fc640 10 client.6219 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.418+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55ad942a8930 0x55ad947bb850 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efb403e26809cefc server_cookie=1abde15ee537614 in_seq=0 out_seq=0 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/2317129450 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ad94618b00 con 0x55ad942a8930 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d6e7fc640 10 client.6219 handle_command_reply: tid=0 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 1 client.6219 shutdown 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 1 -- 172.21.15.131:0/2317129450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 msgr2=0x55ad947bae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad947bae40 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f8d9015ffe0 tx=0x7f8d880127c0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 1 -- 172.21.15.131:0/2317129450 shutdown_connections 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad947bae40 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad947ba900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55ad942a8930 0x55ad947bb850 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 1 --2- 172.21.15.131:0/2317129450 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad942ef890 0x55ad947ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d6e7fc640 0 client.6219 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:45:03.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 1 -- 172.21.15.131:0/2317129450 >> 172.21.15.131:0/2317129450 conn(0x55ad942c35c0 msgr2=0x55ad948075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:03.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 1 -- 172.21.15.131:0/2317129450 shutdown_connections 2024-09-17T02:45:03.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 1 -- 172.21.15.131:0/2317129450 wait complete. 2024-09-17T02:45:03.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.422+0000 7f8d9d503000 20 client.6219 trim_cache size 0 max 16384 2024-09-17T02:45:03.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.438+0000 7f8d9d503000 1 Processor -- start 2024-09-17T02:45:03.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.438+0000 7f8d9d503000 1 -- start start 2024-09-17T02:45:03.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad942ef890 0x55ad9481d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad9481d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad946986d0 0x55ad9480bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901fef60 con 0x55ad942ef890 2024-09-17T02:45:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8d901fdf70 con 0x55ad943b3aa0 2024-09-17T02:45:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8d900ce630 con 0x55ad946986d0 2024-09-17T02:45:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad942ef890 0x55ad9481d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d8f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad942ef890 0x55ad9481d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56582/0 (socket says 172.21.15.131:56582) 2024-09-17T02:45:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d8f7fe640 1 -- 172.21.15.131:0/2268549387 learned_addr learned my addr 172.21.15.131:0/2268549387 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d8f7fe640 1 -- 172.21.15.131:0/2268549387 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad946986d0 msgr2=0x55ad9480bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/2268549387 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad946986d0 0x55ad9480bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d8f7fe640 1 -- 172.21.15.131:0/2268549387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 msgr2=0x55ad9481d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:03.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/2268549387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad9481d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d8f7fe640 1 -- 172.21.15.131:0/2268549387 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad94663680 con 0x55ad942ef890 2024-09-17T02:45:03.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/2268549387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad942ef890 0x55ad9481d1c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55ad946b9760 tx=0x7f8d80020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be2d43ee22e57f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:03.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d74ff9640 1 -- 172.21.15.131:0/2268549387 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80022040 con 0x55ad942ef890 2024-09-17T02:45:03.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d74ff9640 1 -- 172.21.15.131:0/2268549387 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d80028a30 con 0x55ad942ef890 2024-09-17T02:45:03.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d74ff9640 1 -- 172.21.15.131:0/2268549387 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8001e430 con 0x55ad942ef890 2024-09-17T02:45:03.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 -- 172.21.15.131:0/2268549387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad942ef890 msgr2=0x55ad9481d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 --2- 172.21.15.131:0/2268549387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad942ef890 0x55ad9481d1c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55ad946b9760 tx=0x7f8d80020040 comp rx=0 tx=0).stop 2024-09-17T02:45:03.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 -- 172.21.15.131:0/2268549387 shutdown_connections 2024-09-17T02:45:03.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 --2- 172.21.15.131:0/2268549387 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad946986d0 0x55ad9480bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 --2- 172.21.15.131:0/2268549387 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad943b3aa0 0x55ad9481d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 --2- 172.21.15.131:0/2268549387 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad942ef890 0x55ad9481d1c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 -- 172.21.15.131:0/2268549387 >> 172.21.15.131:0/2268549387 conn(0x55ad942c35c0 msgr2=0x55ad947bc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:03.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 -- 172.21.15.131:0/2268549387 shutdown_connections 2024-09-17T02:45:03.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 -- 172.21.15.131:0/2268549387 wait complete. 2024-09-17T02:45:03.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 -- start start 2024-09-17T02:45:03.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.442+0000 7f8d9d503000 1 Processor -- start 2024-09-17T02:45:03.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad942ef890 0x55ad9487e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad943b3aa0 0x55ad9487ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad9487f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ad94609df0 con 0x55ad946986d0 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ad947c4d60 con 0x55ad942ef890 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55ad94600ee0 con 0x55ad943b3aa0 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 10 client.0 _ll_register_callbacks cb 0x7f8d880094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad9487f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad9487f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56592/0 (socket says 172.21.15.131:56592) 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8ffff640 1 -- 172.21.15.131:0/1983044182 learned_addr learned my addr 172.21.15.131:0/1983044182 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8effd640 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad943b3aa0 0x55ad9487ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad942ef890 0x55ad9487e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8ffff640 1 -- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad943b3aa0 msgr2=0x55ad9487ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8ffff640 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad943b3aa0 0x55ad9487ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8ffff640 1 -- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad942ef890 msgr2=0x55ad9487e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8ffff640 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad942ef890 0x55ad9487e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8ffff640 1 -- 172.21.15.131:0/1983044182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad945d00b0 con 0x55ad946986d0 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad942ef890 0x55ad9487e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8ffff640 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad9487f280 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8d8801bbb0 tx=0x7f8d8801bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ec3eb9c666ca0c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d6e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/1983044182 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d88003c00 con 0x55ad946986d0 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/1983044182 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f8d88003da0 con 0x55ad946986d0 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 1 -- 172.21.15.131:0/1983044182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d80028a30 con 0x55ad946986d0 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/1983044182 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f8d88035050 con 0x55ad946986d0 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 10 client.6225 fetch_fsmap learned FSMap version 38 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 1 -- 172.21.15.131:0/1983044182 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x55ad9459c610 con 0x55ad946986d0 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/1983044182 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d80028a30 con 0x55ad946986d0 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/1983044182 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 38) ==== 3502+0+0 (secure 0 0 0) 0x7f8d88038020 con 0x55ad946986d0 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 10 client.6225 fetch_fsmap finished waiting for FSMap version 38 2024-09-17T02:45:03.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 10 client.6225 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2024-09-17T02:45:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 20 client.6225 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55ad94296990 0x55ad948117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 4 client.6225 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:45:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 1 -- 172.21.15.131:0/1983044182 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ad946145b0 con 0x55ad94296990 2024-09-17T02:45:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55ad94296990 0x55ad948117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d8f7fe640 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55ad94296990 0x55ad948117e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f114bee34dd1b17 server_cookie=26e3552e924e1621 in_seq=0 out_seq=0 2024-09-17T02:45:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d6e7fc640 10 client.6225 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:45:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d6e7fc640 1 -- 172.21.15.131:0/1983044182 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55ad946145b0 con 0x55ad94296990 2024-09-17T02:45:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d6e7fc640 10 client.6225 handle_command_reply: tid=0 2024-09-17T02:45:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.446+0000 7f8d9d503000 1 client.6225 shutdown 2024-09-17T02:45:03.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d9d503000 1 -- 172.21.15.131:0/1983044182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 msgr2=0x55ad9487f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d9d503000 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad9487f280 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8d8801bbb0 tx=0x7f8d8801bbe0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d9d503000 1 -- 172.21.15.131:0/1983044182 shutdown_connections 2024-09-17T02:45:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d9d503000 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ad946986d0 0x55ad9487f280 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d9d503000 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x55ad94296990 0x55ad948117e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d9d503000 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ad943b3aa0 0x55ad9487ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d9d503000 1 --2- 172.21.15.131:0/1983044182 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ad942ef890 0x55ad9487e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d6e7fc640 0 client.6225 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:45:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d9d503000 1 -- 172.21.15.131:0/1983044182 >> 172.21.15.131:0/1983044182 conn(0x55ad942c35c0 msgr2=0x55ad9481c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d9d503000 1 -- 172.21.15.131:0/1983044182 shutdown_connections 2024-09-17T02:45:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d9d503000 1 -- 172.21.15.131:0/1983044182 wait complete. 2024-09-17T02:45:03.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d9d503000 20 client.6225 trim_cache size 0 max 16384 2024-09-17T02:45:03.453 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:45:03.453 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-09-17T02:45:03.453 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {}, 2024-09-17T02:45:03.453 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:45:03.453 INFO:teuthology.orchestra.run.smithi131.stdout: "3099326f-5d6e-47b0-bf0f-8774042d29fb": { 2024-09-17T02:45:03.453 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:44:33.001222+0000", 2024-09-17T02:45:03.453 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:45:03.453 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:45:03.453 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:45:03.453 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:45:03.453 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:45:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: "7c955166-b5f6-4490-8c58-d278c543ef77": { 2024-09-17T02:45:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:44:01.373757+0000", 2024-09-17T02:45:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:45:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:45:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:45:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:45:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:45:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:03.454 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:03.454 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:45:03.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 -- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d7c073200 msgr2=0x7f8d7c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d7c073200 0x7f8d7c075710 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8d900b5fe0 tx=0x7f8d80001570 comp rx=0 tx=0).stop 2024-09-17T02:45:03.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 -- 172.21.15.131:0/3360194813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d901f5f60 msgr2=0x7f8d9014a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d901f5f60 0x7f8d9014a3c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f8d84000c10 tx=0x7f8d84004620 comp rx=0 tx=0).stop 2024-09-17T02:45:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 -- 172.21.15.131:0/3360194813 shutdown_connections 2024-09-17T02:45:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d7c073200 0x7f8d7c075710 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8d9015adf0 0x7f8d9015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8d901f5f60 0x7f8d9014a3c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 --2- 172.21.15.131:0/3360194813 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8d90158540 0x7f8d901f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 -- 172.21.15.131:0/3360194813 >> 172.21.15.131:0/3360194813 conn(0x7f8d9006eea0 msgr2=0x7f8d900cb200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 -- 172.21.15.131:0/3360194813 shutdown_connections 2024-09-17T02:45:03.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.450+0000 7f8d96238640 1 -- 172.21.15.131:0/3360194813 wait complete. 2024-09-17T02:45:03.466 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-17T02:45:03.467 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:03.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.618+0000 7ff0426cb640 1 -- 172.21.15.131:0/619189511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff024005360 msgr2=0x7ff02400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.618+0000 7ff0426cb640 1 --2- 172.21.15.131:0/619189511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff024005360 0x7ff02400b9d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff03400c640 tx=0x7ff03401fa40 comp rx=0 tx=0).stop 2024-09-17T02:45:03.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.618+0000 7ff0426cb640 1 -- 172.21.15.131:0/619189511 shutdown_connections 2024-09-17T02:45:03.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.618+0000 7ff0426cb640 1 --2- 172.21.15.131:0/619189511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff024005360 0x7ff02400b9d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.618+0000 7ff0426cb640 1 -- 172.21.15.131:0/619189511 >> 172.21.15.131:0/619189511 conn(0x7ff03c06eea0 msgr2=0x7ff03c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:03.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.618+0000 7ff0426cb640 1 -- 172.21.15.131:0/619189511 shutdown_connections 2024-09-17T02:45:03.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.618+0000 7ff0426cb640 1 -- 172.21.15.131:0/619189511 wait complete. 2024-09-17T02:45:03.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.618+0000 7ff0426cb640 1 Processor -- start 2024-09-17T02:45:03.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.618+0000 7ff0426cb640 1 -- start start 2024-09-17T02:45:03.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff0426cb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024005360 0x7ff03c0a9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff0426cb640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff03c156c50 0x7ff03c14ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff0426cb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff03c0c8fc0 0x7ff03c0c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff0426cb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c06e640 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff0426cb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c06e4c0 con 0x7ff024005360 2024-09-17T02:45:03.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff0426cb640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff03c06e7c0 con 0x7ff03c156c50 2024-09-17T02:45:03.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff040c68640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff03c0c8fc0 0x7ff03c0c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff040c68640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff03c0c8fc0 0x7ff03c0c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56614/0 (socket says 172.21.15.131:56614) 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff040c68640 1 -- 172.21.15.131:0/1390026500 learned_addr learned my addr 172.21.15.131:0/1390026500 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff03b7fe640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff03c156c50 0x7ff03c14ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff03bfff640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024005360 0x7ff03c0a9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff040c68640 1 -- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff03c156c50 msgr2=0x7ff03c14ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff040c68640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff03c156c50 0x7ff03c14ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff040c68640 1 -- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024005360 msgr2=0x7ff03c0a9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff040c68640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024005360 0x7ff03c0a9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff040c68640 1 -- 172.21.15.131:0/1390026500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c0c9910 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff03b7fe640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff03c156c50 0x7ff03c14ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff040c68640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff03c0c8fc0 0x7ff03c0c93d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff0340202e0 tx=0x7ff034004f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff038ff9640 1 -- 172.21.15.131:0/1390026500 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034001a80 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff038ff9640 1 -- 172.21.15.131:0/1390026500 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0340a9720 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff0426cb640 1 -- 172.21.15.131:0/1390026500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c1f3890 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff038ff9640 1 -- 172.21.15.131:0/1390026500 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0340a9b30 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff0426cb640 1 -- 172.21.15.131:0/1390026500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c1f3e50 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff0426cb640 1 -- 172.21.15.131:0/1390026500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff03c0caa40 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff038ff9640 1 -- 172.21.15.131:0/1390026500 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff0340a9cd0 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff038ff9640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff028073200 0x7ff028075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:03.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff038ff9640 1 -- 172.21.15.131:0/1390026500 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7ff034131700 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.622+0000 7ff03bfff640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff028073200 0x7ff028075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:03.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.626+0000 7ff03bfff640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff028073200 0x7ff028075710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff03c06fe80 tx=0x7ff02c010040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:03.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.630+0000 7ff038ff9640 1 -- 172.21.15.131:0/1390026500 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff03c0c9910 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.838+0000 7ff0426cb640 1 -- 172.21.15.131:0/1390026500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff03c06fa20 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.838+0000 7ff038ff9640 1 -- 172.21.15.131:0/1390026500 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5234 (secure 0 0 0) 0x7ff0340197f0 con 0x7ff03c0c8fc0 2024-09-17T02:45:03.839 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:03.839 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":38,"btime":"2024-09-17T02:44:38:778548+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3554877374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3554877374},{"type":"v1","addr":"172.21.15.189:6837","nonce":3554877374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1670590113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1670590113},{"type":"v1","addr":"172.21.15.131:6834","nonce":1670590113}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1046159039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1046159039},{"type":"v1","addr":"172.21.15.131:6835","nonce":1046159039}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:43:46.061550+0000","modified":"2024-09-17T02:44:38.728223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5871":{"gid":5871,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":16,"addr":"172.21.15.189:6835/730392436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":730392436},{"type":"v1","addr":"172.21.15.189:6835","nonce":730392436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5871,"qdb_cluster":[5871]},"id":4}]} 2024-09-17T02:45:03.839 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 38 2024-09-17T02:45:03.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 -- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff028073200 msgr2=0x7ff028075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff028073200 0x7ff028075710 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff03c06fe80 tx=0x7ff02c010040 comp rx=0 tx=0).stop 2024-09-17T02:45:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 -- 172.21.15.131:0/1390026500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff03c0c8fc0 msgr2=0x7ff03c0c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff03c0c8fc0 0x7ff03c0c93d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff0340202e0 tx=0x7ff034004f70 comp rx=0 tx=0).stop 2024-09-17T02:45:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 -- 172.21.15.131:0/1390026500 shutdown_connections 2024-09-17T02:45:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff028073200 0x7ff028075710 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff03c0c8fc0 0x7ff03c0c93d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff03c156c50 0x7ff03c14ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 --2- 172.21.15.131:0/1390026500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff024005360 0x7ff03c0a9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 -- 172.21.15.131:0/1390026500 >> 172.21.15.131:0/1390026500 conn(0x7ff03c06eea0 msgr2=0x7ff03c0cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 -- 172.21.15.131:0/1390026500 shutdown_connections 2024-09-17T02:45:03.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:03.842+0000 7ff0426cb640 1 -- 172.21.15.131:0/1390026500 wait complete. 2024-09-17T02:45:03.858 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-09-17T02:45:04.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.002+0000 7f726c8bf640 1 -- 172.21.15.131:0/755829780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72680719c0 msgr2=0x7f72680cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.002+0000 7f726c8bf640 1 --2- 172.21.15.131:0/755829780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72680719c0 0x7f72680cf760 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7260009900 tx=0x7f726001c850 comp rx=0 tx=0).stop 2024-09-17T02:45:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.002+0000 7f726c8bf640 1 -- 172.21.15.131:0/755829780 shutdown_connections 2024-09-17T02:45:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.002+0000 7f726c8bf640 1 --2- 172.21.15.131:0/755829780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f72680719c0 0x7f72680cf760 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.002+0000 7f726c8bf640 1 --2- 172.21.15.131:0/755829780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f726815f930 0x7f7268071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.002+0000 7f726c8bf640 1 -- 172.21.15.131:0/755829780 >> 172.21.15.131:0/755829780 conn(0x7f72680d13d0 msgr2=0x7f72680734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.002+0000 7f726c8bf640 1 -- 172.21.15.131:0/755829780 shutdown_connections 2024-09-17T02:45:04.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.002+0000 7f726c8bf640 1 -- 172.21.15.131:0/755829780 wait complete. 2024-09-17T02:45:04.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f726c8bf640 1 Processor -- start 2024-09-17T02:45:04.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f726c8bf640 1 -- start start 2024-09-17T02:45:04.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f726c8bf640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f72680719c0 0x7f72681519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f726c8bf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7268074500 0x7f726815b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f726c8bf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f726815f930 0x7f7268145f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f726c8bf640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7268169a90 con 0x7f7268074500 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f726c8bf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7268169910 con 0x7f726815f930 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f726c8bf640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7268169c10 con 0x7f72680719c0 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f7265d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7268074500 0x7f726815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f7265d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7268074500 0x7f726815b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56634/0 (socket says 172.21.15.131:56634) 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f7265d9b640 1 -- 172.21.15.131:0/3820186929 learned_addr learned my addr 172.21.15.131:0/3820186929 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f7266d9d640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f726815f930 0x7f7268145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f726659c640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f72680719c0 0x7f72681519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f7265d9b640 1 -- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f72680719c0 msgr2=0x7f72681519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f7265d9b640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f72680719c0 0x7f72681519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f7265d9b640 1 -- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f726815f930 msgr2=0x7f7268145f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f7265d9b640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f726815f930 0x7f7268145f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f7265d9b640 1 -- 172.21.15.131:0/3820186929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72680b05e0 con 0x7f7268074500 2024-09-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f7265d9b640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7268074500 0x7f726815b660 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f725000d750 tx=0x7f725000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f725effd640 1 -- 172.21.15.131:0/3820186929 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250017070 con 0x7f7268074500 2024-09-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f725effd640 1 -- 172.21.15.131:0/3820186929 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7250009d20 con 0x7f7268074500 2024-09-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f726c8bf640 1 -- 172.21.15.131:0/3820186929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7268152360 con 0x7f7268074500 2024-09-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f725effd640 1 -- 172.21.15.131:0/3820186929 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250005280 con 0x7f7268074500 2024-09-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.006+0000 7f726c8bf640 1 -- 172.21.15.131:0/3820186929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f726814e640 con 0x7f7268074500 2024-09-17T02:45:04.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f725effd640 1 -- 172.21.15.131:0/3820186929 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7250005420 con 0x7f7268074500 2024-09-17T02:45:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f725effd640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7248073230 0x7f7248075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f725effd640 1 -- 172.21.15.131:0/3820186929 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f72500944d0 con 0x7f7268074500 2024-09-17T02:45:04.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f726659c640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7248073230 0x7f7248075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f7273b8a000 1 Processor -- start 2024-09-17T02:45:04.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f7273b8a000 1 -- start start 2024-09-17T02:45:04.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f726659c640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7248073230 0x7f7248075740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7258001cf0 tx=0x7f7258009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:04.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef3fb60 0x5651bf0852e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 0x5651bf085820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef1ed90 0x5651bf085d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f7273b8a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5651bef23d00 con 0x5651bef3fb60 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f7273b8a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5651bee97c90 con 0x5651bef1ed90 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f7273b8a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5651bee870c0 con 0x5651bef351c0 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f726659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef3fb60 0x5651bf0852e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f726659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef3fb60 0x5651bf0852e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56642/0 (socket says 172.21.15.131:56642) 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f726659c640 1 -- 172.21.15.131:0/657652357 learned_addr learned my addr 172.21.15.131:0/657652357 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f7266d9d640 1 --2- 172.21.15.131:0/657652357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef1ed90 0x5651bf085d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f7265d9b640 1 --2- 172.21.15.131:0/657652357 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 0x5651bf085820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f726659c640 1 -- 172.21.15.131:0/657652357 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 msgr2=0x5651bf085820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f726659c640 1 --2- 172.21.15.131:0/657652357 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 0x5651bf085820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f726659c640 1 -- 172.21.15.131:0/657652357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef1ed90 msgr2=0x5651bf085d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f726659c640 1 --2- 172.21.15.131:0/657652357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef1ed90 0x5651bf085d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f726659c640 1 -- 172.21.15.131:0/657652357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651bee62290 con 0x5651bef3fb60 2024-09-17T02:45:04.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.010+0000 7f7266d9d640 1 --2- 172.21.15.131:0/657652357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef1ed90 0x5651bf085d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f726659c640 1 --2- 172.21.15.131:0/657652357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef3fb60 0x5651bf0852e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7258003ae0 tx=0x7f7258011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317247d3c5cc7f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7247fff640 1 -- 172.21.15.131:0/657652357 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725800ee90 con 0x5651bef3fb60 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7247fff640 1 -- 172.21.15.131:0/657652357 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72580053d0 con 0x5651bef3fb60 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7247fff640 1 -- 172.21.15.131:0/657652357 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72580057e0 con 0x5651bef3fb60 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 -- 172.21.15.131:0/657652357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef3fb60 msgr2=0x5651bf0852e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 --2- 172.21.15.131:0/657652357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef3fb60 0x5651bf0852e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7258003ae0 tx=0x7f7258011000 comp rx=0 tx=0).stop 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 -- 172.21.15.131:0/657652357 shutdown_connections 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 --2- 172.21.15.131:0/657652357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef1ed90 0x5651bf085d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 --2- 172.21.15.131:0/657652357 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 0x5651bf085820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 --2- 172.21.15.131:0/657652357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef3fb60 0x5651bf0852e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 -- 172.21.15.131:0/657652357 >> 172.21.15.131:0/657652357 conn(0x5651beb44760 msgr2=0x5651bf068790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:04.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 -- 172.21.15.131:0/657652357 shutdown_connections 2024-09-17T02:45:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 -- 172.21.15.131:0/657652357 wait complete. 2024-09-17T02:45:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 -- start start 2024-09-17T02:45:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 Processor -- start 2024-09-17T02:45:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf041070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 0x5651bf0415b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef3fb60 0x5651bf0678d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5651bee21940 con 0x5651bef1ed90 2024-09-17T02:45:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5651bee8aed0 con 0x5651bef3fb60 2024-09-17T02:45:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5651beb4c820 con 0x5651bef351c0 2024-09-17T02:45:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f7273b8a000 10 client.0 _ll_register_callbacks cb 0x5651bee657f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:04.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f726659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf041070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f726659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf041070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56650/0 (socket says 172.21.15.131:56650) 2024-09-17T02:45:04.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f726659c640 1 -- 172.21.15.131:0/1223370076 learned_addr learned my addr 172.21.15.131:0/1223370076 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:04.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f726659c640 1 -- 172.21.15.131:0/1223370076 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 msgr2=0x5651bf0415b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:04.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f726659c640 1 --2- 172.21.15.131:0/1223370076 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 0x5651bf0415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f726659c640 1 -- 172.21.15.131:0/1223370076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef3fb60 msgr2=0x5651bf0678d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:04.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f726659c640 1 --2- 172.21.15.131:0/1223370076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef3fb60 0x5651bf0678d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f726659c640 1 -- 172.21.15.131:0/1223370076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651bee3a550 con 0x5651bef1ed90 2024-09-17T02:45:04.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f726659c640 1 --2- 172.21.15.131:0/1223370076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf041070 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7258001a10 tx=0x7f7258005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c59b5ddfa3d062 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:04.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.014+0000 7f72457fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f72457fa640 1 -- 172.21.15.131:0/1223370076 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258021450 con 0x5651bef1ed90 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f72457fa640 1 -- 172.21.15.131:0/1223370076 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f7258010050 con 0x5651bef1ed90 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 -- 172.21.15.131:0/1223370076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651bee1f400 con 0x5651bef1ed90 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f72457fa640 1 -- 172.21.15.131:0/1223370076 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7258033050 con 0x5651bef1ed90 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 10 client.6243 fetch_fsmap learned FSMap version 38 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 -- 172.21.15.131:0/1223370076 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x5651bee25480 con 0x5651bef1ed90 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f72457fa640 1 -- 172.21.15.131:0/1223370076 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5651bee1f400 con 0x5651bef1ed90 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f72457fa640 1 -- 172.21.15.131:0/1223370076 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 38) ==== 3502+0+0 (secure 0 0 0) 0x7f7258036020 con 0x5651bef1ed90 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 10 client.6243 fetch_fsmap finished waiting for FSMap version 38 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 10 client.6243 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 20 client.6243 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 --2- 172.21.15.131:0/1223370076 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5651beb29ad0 0x5651bf0968a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 4 client.6243 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:45:04.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 -- 172.21.15.131:0/1223370076 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651bee99be0 con 0x5651beb29ad0 2024-09-17T02:45:04.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7265d9b640 1 --2- 172.21.15.131:0/1223370076 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5651beb29ad0 0x5651bf0968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f72457fa640 10 client.6243 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:45:04.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7265d9b640 1 --2- 172.21.15.131:0/1223370076 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5651beb29ad0 0x5651bf0968a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82837b1501ffa47d server_cookie=4645ecac1853bc66 in_seq=0 out_seq=0 2024-09-17T02:45:04.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f72457fa640 1 -- 172.21.15.131:0/1223370076 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5651bee99be0 con 0x5651beb29ad0 2024-09-17T02:45:04.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f72457fa640 10 client.6243 handle_command_reply: tid=0 2024-09-17T02:45:04.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 client.6243 shutdown 2024-09-17T02:45:04.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 -- 172.21.15.131:0/1223370076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 msgr2=0x5651bf041070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 --2- 172.21.15.131:0/1223370076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf041070 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7258001a10 tx=0x7f7258005510 comp rx=0 tx=0).stop 2024-09-17T02:45:04.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 -- 172.21.15.131:0/1223370076 shutdown_connections 2024-09-17T02:45:04.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 --2- 172.21.15.131:0/1223370076 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef3fb60 0x5651bf0678d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 --2- 172.21.15.131:0/1223370076 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5651beb29ad0 0x5651bf0968a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 --2- 172.21.15.131:0/1223370076 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 0x5651bf0415b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 --2- 172.21.15.131:0/1223370076 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf041070 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f72457fa640 0 client.6243 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:45:04.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 -- 172.21.15.131:0/1223370076 >> 172.21.15.131:0/1223370076 conn(0x5651beb44760 msgr2=0x5651bf039680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:04.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 -- 172.21.15.131:0/1223370076 shutdown_connections 2024-09-17T02:45:04.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 1 -- 172.21.15.131:0/1223370076 wait complete. 2024-09-17T02:45:04.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.018+0000 7f7273b8a000 20 client.6243 trim_cache size 0 max 16384 2024-09-17T02:45:04.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 Processor -- start 2024-09-17T02:45:04.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 -- start start 2024-09-17T02:45:04.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef1ed90 0x5651bf03c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef351c0 0x5651bf03cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef3fb60 0x5651bf0998d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5651bee8a1e0 con 0x5651bef351c0 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5651bee96a60 con 0x5651bef1ed90 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5651bee53300 con 0x5651bef3fb60 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7265d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef351c0 0x5651bf03cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7265d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef351c0 0x5651bf03cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56652/0 (socket says 172.21.15.131:56652) 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7265d9b640 1 -- 172.21.15.131:0/2275042721 learned_addr learned my addr 172.21.15.131:0/2275042721 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7265d9b640 1 -- 172.21.15.131:0/2275042721 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef3fb60 msgr2=0x5651bf0998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7265d9b640 1 --2- 172.21.15.131:0/2275042721 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef3fb60 0x5651bf0998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7265d9b640 1 -- 172.21.15.131:0/2275042721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef1ed90 msgr2=0x5651bf03c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7265d9b640 1 --2- 172.21.15.131:0/2275042721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef1ed90 0x5651bf03c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7265d9b640 1 -- 172.21.15.131:0/2275042721 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651bef0cd10 con 0x5651bef351c0 2024-09-17T02:45:04.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7265d9b640 1 --2- 172.21.15.131:0/2275042721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef351c0 0x5651bf03cbe0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7250009610 tx=0x7f7250005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab1c84a30e46f42 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f72477fe640 1 -- 172.21.15.131:0/2275042721 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725004c070 con 0x5651bef351c0 2024-09-17T02:45:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f72477fe640 1 -- 172.21.15.131:0/2275042721 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f725005ecc0 con 0x5651bef351c0 2024-09-17T02:45:04.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f72477fe640 1 -- 172.21.15.131:0/2275042721 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250020040 con 0x5651bef351c0 2024-09-17T02:45:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 -- 172.21.15.131:0/2275042721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef351c0 msgr2=0x5651bf03cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 --2- 172.21.15.131:0/2275042721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef351c0 0x5651bf03cbe0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f7250009610 tx=0x7f7250005bc0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 -- 172.21.15.131:0/2275042721 shutdown_connections 2024-09-17T02:45:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 --2- 172.21.15.131:0/2275042721 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef3fb60 0x5651bf0998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 --2- 172.21.15.131:0/2275042721 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef351c0 0x5651bf03cbe0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 --2- 172.21.15.131:0/2275042721 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef1ed90 0x5651bf03c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 -- 172.21.15.131:0/2275042721 >> 172.21.15.131:0/2275042721 conn(0x5651beb44760 msgr2=0x5651bf03fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 -- 172.21.15.131:0/2275042721 shutdown_connections 2024-09-17T02:45:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.034+0000 7f7273b8a000 1 -- 172.21.15.131:0/2275042721 wait complete. 2024-09-17T02:45:04.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 -- start start 2024-09-17T02:45:04.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 Processor -- start 2024-09-17T02:45:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 0x5651bf099410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef3fb60 0x5651bf0b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5651bee8f500 con 0x5651bef1ed90 2024-09-17T02:45:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5651bee32300 con 0x5651bef3fb60 2024-09-17T02:45:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5651bee44670 con 0x5651bef351c0 2024-09-17T02:45:04.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 10 client.0 _ll_register_callbacks cb 0x7f72580039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f726659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f726659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf098ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56662/0 (socket says 172.21.15.131:56662) 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f726659c640 1 -- 172.21.15.131:0/4025651929 learned_addr learned my addr 172.21.15.131:0/4025651929 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f726659c640 1 -- 172.21.15.131:0/4025651929 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 msgr2=0x5651bf099410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f726659c640 1 --2- 172.21.15.131:0/4025651929 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 0x5651bf099410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f726659c640 1 -- 172.21.15.131:0/4025651929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef3fb60 msgr2=0x5651bf0b6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f726659c640 1 --2- 172.21.15.131:0/4025651929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef3fb60 0x5651bf0b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f726659c640 1 -- 172.21.15.131:0/4025651929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651bee3de50 con 0x5651bef1ed90 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f726659c640 1 --2- 172.21.15.131:0/4025651929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf098ed0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f725802bba0 tx=0x7f725802bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b2ed43cd22c5e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f72457fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f72457fa640 1 -- 172.21.15.131:0/4025651929 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7258021450 con 0x5651bef1ed90 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 -- 172.21.15.131:0/4025651929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f725005ecc0 con 0x5651bef1ed90 2024-09-17T02:45:04.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f72457fa640 1 -- 172.21.15.131:0/4025651929 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f7258010030 con 0x5651bef1ed90 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f72457fa640 1 -- 172.21.15.131:0/4025651929 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f725803f050 con 0x5651bef1ed90 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 10 client.6249 fetch_fsmap learned FSMap version 38 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 -- 172.21.15.131:0/4025651929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) -- 0x5651bee2c2d0 con 0x5651bef1ed90 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f72457fa640 1 -- 172.21.15.131:0/4025651929 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f725005ecc0 con 0x5651bef1ed90 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f72457fa640 1 -- 172.21.15.131:0/4025651929 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 38) ==== 3502+0+0 (secure 0 0 0) 0x7f7258042020 con 0x5651bef1ed90 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 10 client.6249 fetch_fsmap finished waiting for FSMap version 38 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 10 client.6249 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 20 client.6249 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 --2- 172.21.15.131:0/4025651929 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5651beb17b30 0x5651bf0b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 4 client.6249 mds_command: new command op to 5871 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7273b8a000 1 -- 172.21.15.131:0/4025651929 --> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5651bee95690 con 0x5651beb17b30 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.038+0000 7f7265d9b640 1 --2- 172.21.15.131:0/4025651929 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5651beb17b30 0x5651bf0b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f72457fa640 10 client.6249 ms_handle_connect on v2:172.21.15.189:6834/730392436 2024-09-17T02:45:04.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7265d9b640 1 --2- 172.21.15.131:0/4025651929 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5651beb17b30 0x5651bf0b7780 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77ec8367319db5ad server_cookie=31c5cfdd1c8ce33c in_seq=0 out_seq=0 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f72457fa640 1 -- 172.21.15.131:0/4025651929 <== mds.0 v2:172.21.15.189:6834/730392436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651bee95690 con 0x5651beb17b30 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f72457fa640 10 client.6249 handle_command_reply: tid=0 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 1 client.6249 shutdown 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 1 -- 172.21.15.131:0/4025651929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 msgr2=0x5651bf098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 1 --2- 172.21.15.131:0/4025651929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf098ed0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f725802bba0 tx=0x7f725802bbd0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 1 -- 172.21.15.131:0/4025651929 shutdown_connections 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 1 --2- 172.21.15.131:0/4025651929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5651bef3fb60 0x5651bf0b6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 1 --2- 172.21.15.131:0/4025651929 >> [v2:172.21.15.189:6834/730392436,v1:172.21.15.189:6835/730392436] conn(0x5651beb17b30 0x5651bf0b7780 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 1 --2- 172.21.15.131:0/4025651929 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5651bef351c0 0x5651bf099410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f72457fa640 0 client.6249 ms_handle_reset on v2:172.21.15.189:6834/730392436 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 1 --2- 172.21.15.131:0/4025651929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5651bef1ed90 0x5651bf098ed0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 1 -- 172.21.15.131:0/4025651929 >> 172.21.15.131:0/4025651929 conn(0x5651beb44760 msgr2=0x5651bf0405a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 1 -- 172.21.15.131:0/4025651929 shutdown_connections 2024-09-17T02:45:04.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 1 -- 172.21.15.131:0/4025651929 wait complete. 2024-09-17T02:45:04.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f7273b8a000 20 client.6249 trim_cache size 0 max 16384 2024-09-17T02:45:04.044 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:04.044 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-09-17T02:45:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 -- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7248073230 msgr2=0x7f7248075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7248073230 0x7f7248075740 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7258001cf0 tx=0x7f7258009040 comp rx=0 tx=0).stop 2024-09-17T02:45:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 -- 172.21.15.131:0/3820186929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7268074500 msgr2=0x7f726815b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7268074500 0x7f726815b660 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f725000d750 tx=0x7f725000dc10 comp rx=0 tx=0).stop 2024-09-17T02:45:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 -- 172.21.15.131:0/3820186929 shutdown_connections 2024-09-17T02:45:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7248073230 0x7f7248075740 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f726815f930 0x7f7268145f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7268074500 0x7f726815b660 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 --2- 172.21.15.131:0/3820186929 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f72680719c0 0x7f72681519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 -- 172.21.15.131:0/3820186929 >> 172.21.15.131:0/3820186929 conn(0x7f72680d13d0 msgr2=0x7f726814da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:04.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 -- 172.21.15.131:0/3820186929 shutdown_connections 2024-09-17T02:45:04.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.042+0000 7f726c8bf640 1 -- 172.21.15.131:0/3820186929 wait complete. 2024-09-17T02:45:04.058 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T02:45:04.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.202+0000 7f29b0a29640 1 -- 172.21.15.131:0/2382269073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0cab60 msgr2=0x7f29ac0caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 --2- 172.21.15.131:0/2382269073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0cab60 0x7f29ac0caf70 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009c80 tx=0x7f29a401cb90 comp rx=0 tx=0).stop 2024-09-17T02:45:04.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 -- 172.21.15.131:0/2382269073 shutdown_connections 2024-09-17T02:45:04.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 --2- 172.21.15.131:0/2382269073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0cab60 0x7f29ac0caf70 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 --2- 172.21.15.131:0/2382269073 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29ac06dea0 0x7f29ac06e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 -- 172.21.15.131:0/2382269073 >> 172.21.15.131:0/2382269073 conn(0x7f29ac06eea0 msgr2=0x7f29ac06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:04.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 -- 172.21.15.131:0/2382269073 shutdown_connections 2024-09-17T02:45:04.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 -- 172.21.15.131:0/2382269073 wait complete. 2024-09-17T02:45:04.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 Processor -- start 2024-09-17T02:45:04.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 -- start start 2024-09-17T02:45:04.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29ac06dea0 0x7f29ac14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0cab60 0x7f29ac0c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29ac155240 0x7f29ac0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac0cbdd0 con 0x7f29ac0cab60 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29ac0cbc50 con 0x7f29ac06dea0 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29b0a29640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f29ac165b60 con 0x7f29ac155240 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29a9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0cab60 0x7f29ac0c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29a9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0cab60 0x7f29ac0c5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56678/0 (socket says 172.21.15.131:56678) 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29a9d9b640 1 -- 172.21.15.131:0/1284315196 learned_addr learned my addr 172.21.15.131:0/1284315196 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29aa59c640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29ac06dea0 0x7f29ac14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29aad9d640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29ac155240 0x7f29ac0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29a9d9b640 1 -- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29ac155240 msgr2=0x7f29ac0c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29a9d9b640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29ac155240 0x7f29ac0c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29a9d9b640 1 -- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29ac06dea0 msgr2=0x7f29ac14cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29a9d9b640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29ac06dea0 0x7f29ac14cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.206+0000 7f29a9d9b640 1 -- 172.21.15.131:0/1284315196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac14d190 con 0x7f29ac0cab60 2024-09-17T02:45:04.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f29a9d9b640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0cab60 0x7f29ac0c5600 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f29a000c970 tx=0x7f29a000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f299affd640 1 -- 172.21.15.131:0/1284315196 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0004220 con 0x7f29ac0cab60 2024-09-17T02:45:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f299affd640 1 -- 172.21.15.131:0/1284315196 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29a00079b0 con 0x7f29ac0cab60 2024-09-17T02:45:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f29b0a29640 1 -- 172.21.15.131:0/1284315196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ac14d380 con 0x7f29ac0cab60 2024-09-17T02:45:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f299affd640 1 -- 172.21.15.131:0/1284315196 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a00044b0 con 0x7f29ac0cab60 2024-09-17T02:45:04.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f29b0a29640 1 -- 172.21.15.131:0/1284315196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ac0c7790 con 0x7f29ac0cab60 2024-09-17T02:45:04.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f29b0a29640 1 -- 172.21.15.131:0/1284315196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29ac14d5d0 con 0x7f29ac0cab60 2024-09-17T02:45:04.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f299affd640 1 -- 172.21.15.131:0/1284315196 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f29a0003600 con 0x7f29ac0cab60 2024-09-17T02:45:04.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f299affd640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f298c073230 0x7f298c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f299affd640 1 -- 172.21.15.131:0/1284315196 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f29a0094570 con 0x7f29ac0cab60 2024-09-17T02:45:04.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f29aa59c640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f298c073230 0x7f298c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.210+0000 7f29aa59c640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f298c073230 0x7f298c075740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f299c00b810 tx=0x7f299c008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:04.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.214+0000 7f299affd640 1 -- 172.21.15.131:0/1284315196 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f29a0014030 con 0x7f29ac0cab60 2024-09-17T02:45:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 -- 172.21.15.131:0/1284315196 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f29ac06e2b0 con 0x7f29ac0cab60 2024-09-17T02:45:04.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f299affd640 1 -- 172.21.15.131:0/1284315196 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f29a005f3d0 con 0x7f29ac0cab60 2024-09-17T02:45:04.416 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:04.416 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T02:45:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 -- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f298c073230 msgr2=0x7f298c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f298c073230 0x7f298c075740 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f299c00b810 tx=0x7f299c008040 comp rx=0 tx=0).stop 2024-09-17T02:45:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 -- 172.21.15.131:0/1284315196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0cab60 msgr2=0x7f29ac0c5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0cab60 0x7f29ac0c5600 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f29a000c970 tx=0x7f29a000ce30 comp rx=0 tx=0).stop 2024-09-17T02:45:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 -- 172.21.15.131:0/1284315196 shutdown_connections 2024-09-17T02:45:04.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f298c073230 0x7f298c075740 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29ac155240 0x7f29ac0c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29ac0cab60 0x7f29ac0c5600 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 --2- 172.21.15.131:0/1284315196 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29ac06dea0 0x7f29ac14cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 -- 172.21.15.131:0/1284315196 >> 172.21.15.131:0/1284315196 conn(0x7f29ac06eea0 msgr2=0x7f29ac154460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:04.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 -- 172.21.15.131:0/1284315196 shutdown_connections 2024-09-17T02:45:04.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.418+0000 7f29b0a29640 1 -- 172.21.15.131:0/1284315196 wait complete. 2024-09-17T02:45:04.436 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T02:45:04.437 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:45:04.437 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:45:04.517 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:45:04.605 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:45:04.605 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:45:04.605 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:45:04.606 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:04.612 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:45:04.612 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:04.660 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:45:04.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.866+0000 7f803f5d9640 1 -- 172.21.15.131:0/3530088012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80380c4c90 msgr2=0x7f80380c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.866+0000 7f803f5d9640 1 --2- 172.21.15.131:0/3530088012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80380c4c90 0x7f80380c50a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8030009bc0 tx=0x7f803001cb10 comp rx=0 tx=0).stop 2024-09-17T02:45:04.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.866+0000 7f803f5d9640 1 -- 172.21.15.131:0/3530088012 shutdown_connections 2024-09-17T02:45:04.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.866+0000 7f803f5d9640 1 --2- 172.21.15.131:0/3530088012 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80380c4c90 0x7f80380c50a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.866+0000 7f803f5d9640 1 --2- 172.21.15.131:0/3530088012 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80380cec00 0x7f80380cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.866+0000 7f803f5d9640 1 -- 172.21.15.131:0/3530088012 >> 172.21.15.131:0/3530088012 conn(0x7f803806eea0 msgr2=0x7f803806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:04.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.866+0000 7f803f5d9640 1 -- 172.21.15.131:0/3530088012 shutdown_connections 2024-09-17T02:45:04.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.866+0000 7f803f5d9640 1 -- 172.21.15.131:0/3530088012 wait complete. 2024-09-17T02:45:04.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.866+0000 7f803f5d9640 1 Processor -- start 2024-09-17T02:45:04.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.866+0000 7f803f5d9640 1 -- start start 2024-09-17T02:45:04.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803f5d9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803806e2c0 0x7f8038156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803f5d9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80380c4c90 0x7f8038152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803f5d9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80380cec00 0x7f803814b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803f5d9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8038169cf0 con 0x7f80380c4c90 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803f5d9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f803815cb90 con 0x7f803806e2c0 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803f5d9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8038169e70 con 0x7f80380cec00 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803cb74640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80380c4c90 0x7f8038152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803cb74640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80380c4c90 0x7f8038152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56694/0 (socket says 172.21.15.131:56694) 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803cb74640 1 -- 172.21.15.131:0/451846704 learned_addr learned my addr 172.21.15.131:0/451846704 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803db76640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80380cec00 0x7f803814b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803d375640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803806e2c0 0x7f8038156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803cb74640 1 -- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80380cec00 msgr2=0x7f803814b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803cb74640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80380cec00 0x7f803814b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803cb74640 1 -- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803806e2c0 msgr2=0x7f8038156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803cb74640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803806e2c0 0x7f8038156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:04.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803cb74640 1 -- 172.21.15.131:0/451846704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8038151760 con 0x7f80380c4c90 2024-09-17T02:45:04.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803db76640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80380cec00 0x7f803814b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:45:04.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803d375640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803806e2c0 0x7f8038156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:45:04.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803cb74640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80380c4c90 0x7f8038152de0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f802800e9c0 tx=0x7f802800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:04.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f802dffb640 1 -- 172.21.15.131:0/451846704 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802800cd80 con 0x7f80380c4c90 2024-09-17T02:45:04.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803f5d9640 1 -- 172.21.15.131:0/451846704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8038158a40 con 0x7f80380c4c90 2024-09-17T02:45:04.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f802dffb640 1 -- 172.21.15.131:0/451846704 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8028004020 con 0x7f80380c4c90 2024-09-17T02:45:04.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f802dffb640 1 -- 172.21.15.131:0/451846704 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8028004470 con 0x7f80380c4c90 2024-09-17T02:45:04.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803f5d9640 1 -- 172.21.15.131:0/451846704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f803814ba80 con 0x7f80380c4c90 2024-09-17T02:45:04.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f803f5d9640 1 -- 172.21.15.131:0/451846704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80380c50a0 con 0x7f80380c4c90 2024-09-17T02:45:04.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f802dffb640 1 -- 172.21.15.131:0/451846704 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8028003600 con 0x7f80380c4c90 2024-09-17T02:45:04.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.870+0000 7f802dffb640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f801c073200 0x7f801c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:04.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.874+0000 7f802dffb640 1 -- 172.21.15.131:0/451846704 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f8028094ed0 con 0x7f80380c4c90 2024-09-17T02:45:04.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.874+0000 7f803d375640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f801c073200 0x7f801c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:04.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.874+0000 7f803d375640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f801c073200 0x7f801c075710 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f80340016b0 tx=0x7f8034009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:04.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:04.878+0000 7f802dffb640 1 -- 172.21.15.131:0/451846704 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f802805fc20 con 0x7f80380c4c90 2024-09-17T02:45:05.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.050+0000 7f803f5d9640 1 -- 172.21.15.131:0/451846704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f80381510c0 con 0x7f80380c4c90 2024-09-17T02:45:05.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.050+0000 7f802dffb640 1 -- 172.21.15.131:0/451846704 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) ==== 80+0+0 (secure 0 0 0) 0x7f802805f520 con 0x7f80380c4c90 2024-09-17T02:45:05.050 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:45:05.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 -- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f801c073200 msgr2=0x7f801c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:05.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f801c073200 0x7f801c075710 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f80340016b0 tx=0x7f8034009040 comp rx=0 tx=0).stop 2024-09-17T02:45:05.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 -- 172.21.15.131:0/451846704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80380c4c90 msgr2=0x7f8038152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:05.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80380c4c90 0x7f8038152de0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f802800e9c0 tx=0x7f802800ee80 comp rx=0 tx=0).stop 2024-09-17T02:45:05.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 -- 172.21.15.131:0/451846704 shutdown_connections 2024-09-17T02:45:05.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f801c073200 0x7f801c075710 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80380cec00 0x7f803814b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80380c4c90 0x7f8038152de0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 --2- 172.21.15.131:0/451846704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f803806e2c0 0x7f8038156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 -- 172.21.15.131:0/451846704 >> 172.21.15.131:0/451846704 conn(0x7f803806eea0 msgr2=0x7f80381503d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:05.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 -- 172.21.15.131:0/451846704 shutdown_connections 2024-09-17T02:45:05.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.054+0000 7f803f5d9640 1 -- 172.21.15.131:0/451846704 wait complete. 2024-09-17T02:45:05.070 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:45:05.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:05.070 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:05.081 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[41958]: fuse finished with error 0 and tester_r 0 2024-09-17T02:45:05.082 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:45:05.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:05.082 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:45:05.255 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:45:05.255 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:45:05.255 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:05.262 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:45:05.262 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:45:05.262 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:45:05.263 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:05.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:45:05.303 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:45:05.304 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:45:05.304 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:05.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:45:05.310 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 -- 172.21.15.131:0/3732837582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd6415c5c0 msgr2=0x7fcd640cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:05.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 --2- 172.21.15.131:0/3732837582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd6415c5c0 0x7fcd640cba00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcd54009840 tx=0x7fcd5401c820 comp rx=0 tx=0).stop 2024-09-17T02:45:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 -- 172.21.15.131:0/3732837582 shutdown_connections 2024-09-17T02:45:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 --2- 172.21.15.131:0/3732837582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd6415c5c0 0x7fcd640cba00 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 -- 172.21.15.131:0/3732837582 >> 172.21.15.131:0/3732837582 conn(0x7fcd64073d10 msgr2=0x7fcd64074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 -- 172.21.15.131:0/3732837582 shutdown_connections 2024-09-17T02:45:05.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 -- 172.21.15.131:0/3732837582 wait complete. 2024-09-17T02:45:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 Processor -- start 2024-09-17T02:45:05.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 -- start start 2024-09-17T02:45:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd6406efb0 0x7fcd6415f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:05.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd640c4ff0 0x7fcd64152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd6415c5c0 0x7fcd64152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6416b0f0 con 0x7fcd6415c5c0 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6416af70 con 0x7fcd6406efb0 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd6ae55640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcd6416b270 con 0x7fcd640c4ff0 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd693f2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd6415c5c0 0x7fcd64152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.510+0000 7fcd693f2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd6415c5c0 0x7fcd64152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56708/0 (socket says 172.21.15.131:56708) 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd693f2640 1 -- 172.21.15.131:0/1774718029 learned_addr learned my addr 172.21.15.131:0/1774718029 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd68bf1640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd6406efb0 0x7fcd6415f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd63fff640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd640c4ff0 0x7fcd64152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd693f2640 1 -- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd640c4ff0 msgr2=0x7fcd64152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd693f2640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd640c4ff0 0x7fcd64152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd693f2640 1 -- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd6406efb0 msgr2=0x7fcd6415f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd693f2640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd6406efb0 0x7fcd6415f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd693f2640 1 -- 172.21.15.131:0/1774718029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd640c9670 con 0x7fcd6415c5c0 2024-09-17T02:45:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd68bf1640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd6406efb0 0x7fcd6415f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd693f2640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd6415c5c0 0x7fcd64152ad0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c760 tx=0x7fcd5c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd617fa640 1 -- 172.21.15.131:0/1774718029 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c003b80 con 0x7fcd6415c5c0 2024-09-17T02:45:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd617fa640 1 -- 172.21.15.131:0/1774718029 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd5c003d20 con 0x7fcd6415c5c0 2024-09-17T02:45:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd6ae55640 1 -- 172.21.15.131:0/1774718029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd641fa820 con 0x7fcd6415c5c0 2024-09-17T02:45:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd617fa640 1 -- 172.21.15.131:0/1774718029 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00aab0 con 0x7fcd6415c5c0 2024-09-17T02:45:05.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd6ae55640 1 -- 172.21.15.131:0/1774718029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd641fa540 con 0x7fcd6415c5c0 2024-09-17T02:45:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd6ae55640 1 -- 172.21.15.131:0/1774718029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd640c9840 con 0x7fcd6415c5c0 2024-09-17T02:45:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd617fa640 1 -- 172.21.15.131:0/1774718029 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcd5c00ac50 con 0x7fcd6415c5c0 2024-09-17T02:45:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd617fa640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd50073230 0x7fcd50075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd617fa640 1 -- 172.21.15.131:0/1774718029 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7fcd5c094280 con 0x7fcd6415c5c0 2024-09-17T02:45:05.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.514+0000 7fcd68bf1640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd50073230 0x7fcd50075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:05.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.518+0000 7fcd68bf1640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd50073230 0x7fcd50075740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd540af000 tx=0x7fcd5401b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:05.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.518+0000 7fcd617fa640 1 -- 172.21.15.131:0/1774718029 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcd5c05f0e0 con 0x7fcd6415c5c0 2024-09-17T02:45:05.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.730+0000 7fcd6ae55640 1 -- 172.21.15.131:0/1774718029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd640b67d0 con 0x7fcd6415c5c0 2024-09-17T02:45:05.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.730+0000 7fcd617fa640 1 -- 172.21.15.131:0/1774718029 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5234 (secure 0 0 0) 0x7fcd5c05e9e0 con 0x7fcd6415c5c0 2024-09-17T02:45:05.728 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:05.728 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":38,"btime":"2024-09-17T02:44:38:778548+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3554877374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3554877374},{"type":"v1","addr":"172.21.15.189:6837","nonce":3554877374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1670590113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1670590113},{"type":"v1","addr":"172.21.15.131:6834","nonce":1670590113}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1046159039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1046159039},{"type":"v1","addr":"172.21.15.131:6835","nonce":1046159039}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:43:46.061550+0000","modified":"2024-09-17T02:44:38.728223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5871":{"gid":5871,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":16,"addr":"172.21.15.189:6835/730392436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":730392436},{"type":"v1","addr":"172.21.15.189:6835","nonce":730392436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5871,"qdb_cluster":[5871]},"id":4}]} 2024-09-17T02:45:05.729 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 38 2024-09-17T02:45:05.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.730+0000 7fcd6ae55640 1 -- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd50073230 msgr2=0x7fcd50075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:05.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.734+0000 7fcd6ae55640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd50073230 0x7fcd50075740 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcd540af000 tx=0x7fcd5401b060 comp rx=0 tx=0).stop 2024-09-17T02:45:05.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.734+0000 7fcd6ae55640 1 -- 172.21.15.131:0/1774718029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd6415c5c0 msgr2=0x7fcd64152ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:05.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.734+0000 7fcd6ae55640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd6415c5c0 0x7fcd64152ad0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00c760 tx=0x7fcd5c00cc20 comp rx=0 tx=0).stop 2024-09-17T02:45:05.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.734+0000 7fcd6ae55640 1 -- 172.21.15.131:0/1774718029 shutdown_connections 2024-09-17T02:45:05.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.734+0000 7fcd6ae55640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd50073230 0x7fcd50075740 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.734+0000 7fcd6ae55640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd6415c5c0 0x7fcd64152ad0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.734+0000 7fcd6ae55640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd640c4ff0 0x7fcd64152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.734+0000 7fcd6ae55640 1 --2- 172.21.15.131:0/1774718029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd6406efb0 0x7fcd6415f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.734+0000 7fcd6ae55640 1 -- 172.21.15.131:0/1774718029 >> 172.21.15.131:0/1774718029 conn(0x7fcd64073d10 msgr2=0x7fcd64153f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:05.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.734+0000 7fcd6ae55640 1 -- 172.21.15.131:0/1774718029 shutdown_connections 2024-09-17T02:45:05.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.734+0000 7fcd6ae55640 1 -- 172.21.15.131:0/1774718029 wait complete. 2024-09-17T02:45:05.751 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:05.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.910+0000 7f3beb4c3640 1 -- 172.21.15.131:0/3355035716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bd0003fe0 msgr2=0x7f3bd0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:05.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.910+0000 7f3beb4c3640 1 --2- 172.21.15.131:0/3355035716 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3bd0003fe0 0x7f3bd0004810 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009b30 tx=0x7f3bd401d270 comp rx=0 tx=0).stop 2024-09-17T02:45:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.910+0000 7f3be9a60640 1 -- 172.21.15.131:0/3355035716 reap_dead start 2024-09-17T02:45:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.910+0000 7f3beb4c3640 1 -- 172.21.15.131:0/3355035716 shutdown_connections 2024-09-17T02:45:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.910+0000 7f3beb4c3640 1 -- 172.21.15.131:0/3355035716 >> 172.21.15.131:0/3355035716 conn(0x7f3be406eea0 msgr2=0x7f3be406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.910+0000 7f3beb4c3640 1 -- 172.21.15.131:0/3355035716 shutdown_connections 2024-09-17T02:45:05.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.910+0000 7f3beb4c3640 1 -- 172.21.15.131:0/3355035716 wait complete. 2024-09-17T02:45:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3beb4c3640 1 Processor -- start 2024-09-17T02:45:05.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3beb4c3640 1 -- start start 2024-09-17T02:45:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3beb4c3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd0003fe0 0x7f3be40c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3beb4c3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3be40c82c0 0x7f3be41f9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3beb4c3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41f98a0 0x7f3be4164620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:05.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3beb4c3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4167c50 con 0x7f3be41f98a0 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3beb4c3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3be4167ad0 con 0x7f3bd0003fe0 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3beb4c3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3be4167dd0 con 0x7f3be40c82c0 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be9a60640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41f98a0 0x7f3be4164620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be9a60640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41f98a0 0x7f3be4164620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56720/0 (socket says 172.21.15.131:56720) 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be9a60640 1 -- 172.21.15.131:0/4026073881 learned_addr learned my addr 172.21.15.131:0/4026073881 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be925f640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd0003fe0 0x7f3be40c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be8a5e640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3be40c82c0 0x7f3be41f9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be9a60640 1 -- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3be40c82c0 msgr2=0x7f3be41f9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be9a60640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3be40c82c0 0x7f3be41f9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be9a60640 1 -- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd0003fe0 msgr2=0x7f3be40c7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be9a60640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd0003fe0 0x7f3be40c7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be9a60640 1 -- 172.21.15.131:0/4026073881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be415ccd0 con 0x7f3be41f98a0 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be925f640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd0003fe0 0x7f3be40c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:05.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3be9a60640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41f98a0 0x7f3be4164620 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3be40b6130 tx=0x7f3bdc010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:05.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3bd9ffb640 1 -- 172.21.15.131:0/4026073881 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc00ee70 con 0x7f3be41f98a0 2024-09-17T02:45:05.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3bd9ffb640 1 -- 172.21.15.131:0/4026073881 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bdc015070 con 0x7f3be41f98a0 2024-09-17T02:45:05.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3bd9ffb640 1 -- 172.21.15.131:0/4026073881 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc003d30 con 0x7f3be41f98a0 2024-09-17T02:45:05.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3beb4c3640 1 -- 172.21.15.131:0/4026073881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be40c8800 con 0x7f3be41f98a0 2024-09-17T02:45:05.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.914+0000 7f3beb4c3640 1 -- 172.21.15.131:0/4026073881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be415d250 con 0x7f3be41f98a0 2024-09-17T02:45:05.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.918+0000 7f3beb4c3640 1 -- 172.21.15.131:0/4026073881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3be4154d90 con 0x7f3be41f98a0 2024-09-17T02:45:05.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.918+0000 7f3bd9ffb640 1 -- 172.21.15.131:0/4026073881 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3bdc004030 con 0x7f3be41f98a0 2024-09-17T02:45:05.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.918+0000 7f3bd9ffb640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3bc8073200 0x7f3bc8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:05.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.918+0000 7f3bd9ffb640 1 -- 172.21.15.131:0/4026073881 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f3bdc095190 con 0x7f3be41f98a0 2024-09-17T02:45:05.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.918+0000 7f3be925f640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3bc8073200 0x7f3bc8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:05.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.918+0000 7f3be925f640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3bc8073200 0x7f3bc8075710 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3be0001ec0 tx=0x7f3be0001670 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:05.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:05.922+0000 7f3bd9ffb640 1 -- 172.21.15.131:0/4026073881 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3be4154d90 con 0x7f3be41f98a0 2024-09-17T02:45:06.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.126+0000 7f3beb4c3640 1 -- 172.21.15.131:0/4026073881 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3be40cf010 con 0x7f3be41f98a0 2024-09-17T02:45:06.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.126+0000 7f3bd9ffb640 1 -- 172.21.15.131:0/4026073881 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5234 (secure 0 0 0) 0x7f3bdc05ff70 con 0x7f3be41f98a0 2024-09-17T02:45:06.126 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:06.126 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":38,"btime":"2024-09-17T02:44:38:778548+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3554877374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3554877374},{"type":"v1","addr":"172.21.15.189:6837","nonce":3554877374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1670590113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1670590113},{"type":"v1","addr":"172.21.15.131:6834","nonce":1670590113}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1046159039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1046159039},{"type":"v1","addr":"172.21.15.131:6835","nonce":1046159039}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:43:46.061550+0000","modified":"2024-09-17T02:44:38.728223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5871":{"gid":5871,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":16,"addr":"172.21.15.189:6835/730392436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":730392436},{"type":"v1","addr":"172.21.15.189:6835","nonce":730392436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5871,"qdb_cluster":[5871]},"id":4}]} 2024-09-17T02:45:06.126 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 38 2024-09-17T02:45:06.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 -- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3bc8073200 msgr2=0x7f3bc8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3bc8073200 0x7f3bc8075710 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3be0001ec0 tx=0x7f3be0001670 comp rx=0 tx=0).stop 2024-09-17T02:45:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 -- 172.21.15.131:0/4026073881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41f98a0 msgr2=0x7f3be4164620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41f98a0 0x7f3be4164620 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f3be40b6130 tx=0x7f3bdc010f40 comp rx=0 tx=0).stop 2024-09-17T02:45:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 -- 172.21.15.131:0/4026073881 shutdown_connections 2024-09-17T02:45:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3bc8073200 0x7f3bc8075710 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3be41f98a0 0x7f3be4164620 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3be40c82c0 0x7f3be41f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 --2- 172.21.15.131:0/4026073881 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3bd0003fe0 0x7f3be40c7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 -- 172.21.15.131:0/4026073881 >> 172.21.15.131:0/4026073881 conn(0x7f3be406eea0 msgr2=0x7f3be40c5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 -- 172.21.15.131:0/4026073881 shutdown_connections 2024-09-17T02:45:06.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.130+0000 7f3beb4c3640 1 -- 172.21.15.131:0/4026073881 wait complete. 2024-09-17T02:45:06.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:45:06.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.306+0000 7f29baf63640 1 -- 172.21.15.131:0/901288908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c004d50 msgr2=0x7f299c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.306+0000 7f29baf63640 1 --2- 172.21.15.131:0/901288908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f299c004d50 0x7f299c005220 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c610 tx=0x7f29ac01fa40 comp rx=0 tx=0).stop 2024-09-17T02:45:06.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.306+0000 7f29b9500640 1 -- 172.21.15.131:0/901288908 reap_dead start 2024-09-17T02:45:06.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.306+0000 7f29baf63640 1 -- 172.21.15.131:0/901288908 shutdown_connections 2024-09-17T02:45:06.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.306+0000 7f29baf63640 1 -- 172.21.15.131:0/901288908 >> 172.21.15.131:0/901288908 conn(0x7f29b406eea0 msgr2=0x7f29b406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:06.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.306+0000 7f29baf63640 1 -- 172.21.15.131:0/901288908 shutdown_connections 2024-09-17T02:45:06.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.306+0000 7f29baf63640 1 -- 172.21.15.131:0/901288908 wait complete. 2024-09-17T02:45:06.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29baf63640 1 Processor -- start 2024-09-17T02:45:06.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29baf63640 1 -- start start 2024-09-17T02:45:06.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29baf63640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29b41f6930 0x7f29b41f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:06.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29baf63640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29b41f94a0 0x7f29b41f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:06.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29baf63640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b4158770 0x7f29b4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:06.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29baf63640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29b40cf190 con 0x7f29b4158770 2024-09-17T02:45:06.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29baf63640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29b40cf010 con 0x7f29b41f6930 2024-09-17T02:45:06.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29baf63640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f29b40cf310 con 0x7f29b41f94a0 2024-09-17T02:45:06.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b9500640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b4158770 0x7f29b4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:06.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b9500640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b4158770 0x7f29b4158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56728/0 (socket says 172.21.15.131:56728) 2024-09-17T02:45:06.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b9500640 1 -- 172.21.15.131:0/1058635573 learned_addr learned my addr 172.21.15.131:0/1058635573 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:06.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b3fff640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29b41f94a0 0x7f29b41f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:06.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b8cff640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29b41f6930 0x7f29b41f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:06.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b9500640 1 -- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29b41f94a0 msgr2=0x7f29b41f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b9500640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29b41f94a0 0x7f29b41f98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b9500640 1 -- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29b41f6930 msgr2=0x7f29b41f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b9500640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29b41f6930 0x7f29b41f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b9500640 1 -- 172.21.15.131:0/1058635573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b41590c0 con 0x7f29b4158770 2024-09-17T02:45:06.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b9500640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b4158770 0x7f29b4158b80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c700 tx=0x7f29ac01fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:06.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b17fa640 1 -- 172.21.15.131:0/1058635573 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac002050 con 0x7f29b4158770 2024-09-17T02:45:06.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b17fa640 1 -- 172.21.15.131:0/1058635573 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29ac001880 con 0x7f29b4158770 2024-09-17T02:45:06.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29baf63640 1 -- 172.21.15.131:0/1058635573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b41f9f60 con 0x7f29b4158770 2024-09-17T02:45:06.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29b17fa640 1 -- 172.21.15.131:0/1058635573 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ac003cc0 con 0x7f29b4158770 2024-09-17T02:45:06.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.310+0000 7f29baf63640 1 -- 172.21.15.131:0/1058635573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b41f6130 con 0x7f29b4158770 2024-09-17T02:45:06.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.314+0000 7f29baf63640 1 -- 172.21.15.131:0/1058635573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b4159d10 con 0x7f29b4158770 2024-09-17T02:45:06.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.314+0000 7f29b17fa640 1 -- 172.21.15.131:0/1058635573 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f29ac0a9610 con 0x7f29b4158770 2024-09-17T02:45:06.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.314+0000 7f29b17fa640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29a0073200 0x7f29a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:06.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.314+0000 7f29b17fa640 1 -- 172.21.15.131:0/1058635573 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f29ac0b9020 con 0x7f29b4158770 2024-09-17T02:45:06.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.314+0000 7f29b8cff640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29a0073200 0x7f29a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:06.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.314+0000 7f29b8cff640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29a0073200 0x7f29a0075710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f29a4000c10 tx=0x7f29a4006f00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:06.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.318+0000 7f29b17fa640 1 -- 172.21.15.131:0/1058635573 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f29b41590c0 con 0x7f29b4158770 2024-09-17T02:45:06.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.490+0000 7f29baf63640 1 -- 172.21.15.131:0/1058635573 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29b4073310 con 0x7f29b4158770 2024-09-17T02:45:06.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.490+0000 7f29b17fa640 1 -- 172.21.15.131:0/1058635573 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+17378 (secure 0 0 0) 0x7f29ac0197f0 con 0x7f29b4158770 2024-09-17T02:45:06.489 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:06.489 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":68,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:44:35.331723+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:43:43.717619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-17T02:43:44.992949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.16","mappings":[{"from":2,"to":7}]},{"pgid":"10.1b","mappings":[{"from":2,"to":5}]},{"pgid":"10.24","mappings":[{"from":2,"to":7}]},{"pgid":"10.2b","mappings":[{"from":4,"to":1}]},{"pgid":"10.2d","mappings":[{"from":2,"to":5}]},{"pgid":"10.33","mappings":[{"from":2,"to":7}]},{"pgid":"10.34","mappings":[{"from":2,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:45:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 -- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29a0073200 msgr2=0x7f29a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29a0073200 0x7f29a0075710 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f29a4000c10 tx=0x7f29a4006f00 comp rx=0 tx=0).stop 2024-09-17T02:45:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 -- 172.21.15.131:0/1058635573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b4158770 msgr2=0x7f29b4158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b4158770 0x7f29b4158b80 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00c700 tx=0x7f29ac01fc90 comp rx=0 tx=0).stop 2024-09-17T02:45:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 -- 172.21.15.131:0/1058635573 shutdown_connections 2024-09-17T02:45:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29a0073200 0x7f29a0075710 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29b4158770 0x7f29b4158b80 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29b41f94a0 0x7f29b41f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 --2- 172.21.15.131:0/1058635573 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29b41f6930 0x7f29b41f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 -- 172.21.15.131:0/1058635573 >> 172.21.15.131:0/1058635573 conn(0x7f29b406eea0 msgr2=0x7f29b415b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 -- 172.21.15.131:0/1058635573 shutdown_connections 2024-09-17T02:45:06.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.494+0000 7f29baf63640 1 -- 172.21.15.131:0/1058635573 wait complete. 2024-09-17T02:45:06.513 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T02:45:06.515 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.674+0000 7fea9f9d9640 1 -- 172.21.15.131:0/3275374344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea9806e3e0 msgr2=0x7fea98151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.674+0000 7fea9f9d9640 1 --2- 172.21.15.131:0/3275374344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea9806e3e0 0x7fea98151940 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fea98076660 tx=0x7fea9001ccd0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.674+0000 7fea9f9d9640 1 -- 172.21.15.131:0/3275374344 shutdown_connections 2024-09-17T02:45:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.674+0000 7fea9f9d9640 1 --2- 172.21.15.131:0/3275374344 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea9806e3e0 0x7fea98151940 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.674+0000 7fea9f9d9640 1 --2- 172.21.15.131:0/3275374344 >> v2:172.21.15.189:3300/0 conn(0x7fea980cf9a0 0x7fea9806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.674+0000 7fea9f9d9640 1 --2- 172.21.15.131:0/3275374344 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea98071c00 0x7fea980c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.674+0000 7fea9f9d9640 1 -- 172.21.15.131:0/3275374344 >> 172.21.15.131:0/3275374344 conn(0x7fea980734f0 msgr2=0x7fea98073900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.674+0000 7fea9f9d9640 1 -- 172.21.15.131:0/3275374344 shutdown_connections 2024-09-17T02:45:06.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.674+0000 7fea9f9d9640 1 -- 172.21.15.131:0/3275374344 wait complete. 2024-09-17T02:45:06.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9f9d9640 1 Processor -- start 2024-09-17T02:45:06.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9f9d9640 1 -- start start 2024-09-17T02:45:06.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9f9d9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9806e3e0 0x7fea9815b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:06.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9f9d9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98071c00 0x7fea98070880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:06.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9f9d9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea980cf9a0 0x7fea981420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:06.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9f9d9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea9815f580 con 0x7fea98071c00 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9f9d9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea9815f400 con 0x7fea9806e3e0 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9f9d9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fea9815f700 con 0x7fea980cf9a0 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9cf74640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98071c00 0x7fea98070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9cf74640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98071c00 0x7fea98070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56740/0 (socket says 172.21.15.131:56740) 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9cf74640 1 -- 172.21.15.131:0/2567976092 learned_addr learned my addr 172.21.15.131:0/2567976092 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9d775640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9806e3e0 0x7fea9815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9df76640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea980cf9a0 0x7fea981420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9cf74640 1 -- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea980cf9a0 msgr2=0x7fea981420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9cf74640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea980cf9a0 0x7fea981420c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9cf74640 1 -- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9806e3e0 msgr2=0x7fea9815b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9cf74640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9806e3e0 0x7fea9815b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9cf74640 1 -- 172.21.15.131:0/2567976092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea98142f70 con 0x7fea98071c00 2024-09-17T02:45:06.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.678+0000 7fea9d775640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9806e3e0 0x7fea9815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:45:06.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.682+0000 7fea9cf74640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98071c00 0x7fea98070880 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fea880058d0 tx=0x7fea8800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:06.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.682+0000 7fea8dffb640 1 -- 172.21.15.131:0/2567976092 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88017070 con 0x7fea98071c00 2024-09-17T02:45:06.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.682+0000 7fea8dffb640 1 -- 172.21.15.131:0/2567976092 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea88009d20 con 0x7fea98071c00 2024-09-17T02:45:06.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.682+0000 7fea9f9d9640 1 -- 172.21.15.131:0/2567976092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea981fc4a0 con 0x7fea98071c00 2024-09-17T02:45:06.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.682+0000 7fea8dffb640 1 -- 172.21.15.131:0/2567976092 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea880106c0 con 0x7fea98071c00 2024-09-17T02:45:06.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.682+0000 7fea9f9d9640 1 -- 172.21.15.131:0/2567976092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea981fca60 con 0x7fea98071c00 2024-09-17T02:45:06.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.682+0000 7fea9f9d9640 1 -- 172.21.15.131:0/2567976092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea980c7180 con 0x7fea98071c00 2024-09-17T02:45:06.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.682+0000 7fea8dffb640 1 -- 172.21.15.131:0/2567976092 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fea8801a070 con 0x7fea98071c00 2024-09-17T02:45:06.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.682+0000 7fea8dffb640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea7c073200 0x7fea7c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:06.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.682+0000 7fea8dffb640 1 -- 172.21.15.131:0/2567976092 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7fea880945b0 con 0x7fea98071c00 2024-09-17T02:45:06.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.682+0000 7fea9d775640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea7c073200 0x7fea7c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:06.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.686+0000 7fea9d775640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea7c073200 0x7fea7c075710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fea94005fd0 tx=0x7fea94005e40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:06.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.686+0000 7fea8dffb640 1 -- 172.21.15.131:0/2567976092 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fea8805f410 con 0x7fea98071c00 2024-09-17T02:45:06.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.890+0000 7fea9f9d9640 1 -- 172.21.15.131:0/2567976092 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea980714a0 con 0x7fea98071c00 2024-09-17T02:45:06.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.890+0000 7fea8dffb640 1 -- 172.21.15.131:0/2567976092 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5234 (secure 0 0 0) 0x7fea8805ed10 con 0x7fea98071c00 2024-09-17T02:45:06.890 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:06.890 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":38,"btime":"2024-09-17T02:44:38:778548+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3554877374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3554877374},{"type":"v1","addr":"172.21.15.189:6837","nonce":3554877374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1670590113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1670590113},{"type":"v1","addr":"172.21.15.131:6834","nonce":1670590113}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1046159039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1046159039},{"type":"v1","addr":"172.21.15.131:6835","nonce":1046159039}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:43:46.061550+0000","modified":"2024-09-17T02:44:38.728223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5871":{"gid":5871,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":16,"addr":"172.21.15.189:6835/730392436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":730392436},{"type":"v1","addr":"172.21.15.189:6835","nonce":730392436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5871,"qdb_cluster":[5871]},"id":4}]} 2024-09-17T02:45:06.890 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 38 2024-09-17T02:45:06.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 -- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea7c073200 msgr2=0x7fea7c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea7c073200 0x7fea7c075710 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fea94005fd0 tx=0x7fea94005e40 comp rx=0 tx=0).stop 2024-09-17T02:45:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 -- 172.21.15.131:0/2567976092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98071c00 msgr2=0x7fea98070880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98071c00 0x7fea98070880 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fea880058d0 tx=0x7fea8800dde0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 -- 172.21.15.131:0/2567976092 shutdown_connections 2024-09-17T02:45:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fea7c073200 0x7fea7c075710 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea980cf9a0 0x7fea981420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea98071c00 0x7fea98070880 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 --2- 172.21.15.131:0/2567976092 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea9806e3e0 0x7fea9815b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 -- 172.21.15.131:0/2567976092 >> 172.21.15.131:0/2567976092 conn(0x7fea980734f0 msgr2=0x7fea980b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 -- 172.21.15.131:0/2567976092 shutdown_connections 2024-09-17T02:45:06.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:06.894+0000 7fea9f9d9640 1 -- 172.21.15.131:0/2567976092 wait complete. 2024-09-17T02:45:06.911 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.054+0000 7fce26785640 1 -- 172.21.15.131:0/2150087443 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce2006e3e0 msgr2=0x7fce2015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.054+0000 7fce26785640 1 --2- 172.21.15.131:0/2150087443 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce2006e3e0 0x7fce2015bea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fce10009840 tx=0x7fce1001c820 comp rx=0 tx=0).stop 2024-09-17T02:45:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.054+0000 7fce26785640 1 -- 172.21.15.131:0/2150087443 shutdown_connections 2024-09-17T02:45:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.054+0000 7fce26785640 1 --2- 172.21.15.131:0/2150087443 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce2006e3e0 0x7fce2015bea0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.054+0000 7fce26785640 1 --2- 172.21.15.131:0/2150087443 >> v2:172.21.15.189:3300/0 conn(0x7fce200ced00 0x7fce2006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.054+0000 7fce26785640 1 -- 172.21.15.131:0/2150087443 >> 172.21.15.131:0/2150087443 conn(0x7fce2006eea0 msgr2=0x7fce2006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:07.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.054+0000 7fce26785640 1 -- 172.21.15.131:0/2150087443 shutdown_connections 2024-09-17T02:45:07.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.054+0000 7fce26785640 1 -- 172.21.15.131:0/2150087443 wait complete. 2024-09-17T02:45:07.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.054+0000 7fce26785640 1 Processor -- start 2024-09-17T02:45:07.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce26785640 1 -- start start 2024-09-17T02:45:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce26785640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce2006e3e0 0x7fce20146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce26785640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce200ced00 0x7fce200afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce26785640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce2015c3e0 0x7fce20160e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce26785640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fce20168b60 con 0x7fce2015c3e0 2024-09-17T02:45:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce26785640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fce201689e0 con 0x7fce2006e3e0 2024-09-17T02:45:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce26785640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fce20168ce0 con 0x7fce200ced00 2024-09-17T02:45:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce24d22640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce2015c3e0 0x7fce20160e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:07.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce24d22640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce2015c3e0 0x7fce20160e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56756/0 (socket says 172.21.15.131:56756) 2024-09-17T02:45:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce24d22640 1 -- 172.21.15.131:0/2693392251 learned_addr learned my addr 172.21.15.131:0/2693392251 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce1f7fe640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce200ced00 0x7fce200afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce1ffff640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce2006e3e0 0x7fce20146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce24d22640 1 -- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce200ced00 msgr2=0x7fce200afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce24d22640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce200ced00 0x7fce200afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce24d22640 1 -- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce2006e3e0 msgr2=0x7fce20146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce24d22640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce2006e3e0 0x7fce20146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce24d22640 1 -- 172.21.15.131:0/2693392251 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce20160100 con 0x7fce2015c3e0 2024-09-17T02:45:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce24d22640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce2015c3e0 0x7fce20160e80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c710 tx=0x7fce1800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce1cff9640 1 -- 172.21.15.131:0/2693392251 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce18003b80 con 0x7fce2015c3e0 2024-09-17T02:45:07.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce26785640 1 -- 172.21.15.131:0/2693392251 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce201603a0 con 0x7fce2015c3e0 2024-09-17T02:45:07.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce1cff9640 1 -- 172.21.15.131:0/2693392251 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce18003d20 con 0x7fce2015c3e0 2024-09-17T02:45:07.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce1cff9640 1 -- 172.21.15.131:0/2693392251 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1800aab0 con 0x7fce2015c3e0 2024-09-17T02:45:07.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce26785640 1 -- 172.21.15.131:0/2693392251 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce20160960 con 0x7fce2015c3e0 2024-09-17T02:45:07.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.058+0000 7fce26785640 1 -- 172.21.15.131:0/2693392251 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce2015bea0 con 0x7fce2015c3e0 2024-09-17T02:45:07.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.062+0000 7fce1cff9640 1 -- 172.21.15.131:0/2693392251 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fce1800ac50 con 0x7fce2015c3e0 2024-09-17T02:45:07.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.062+0000 7fce1cff9640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce0c073200 0x7fce0c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.062+0000 7fce1cff9640 1 -- 172.21.15.131:0/2693392251 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7fce180942a0 con 0x7fce2015c3e0 2024-09-17T02:45:07.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.062+0000 7fce1ffff640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce0c073200 0x7fce0c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:07.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.062+0000 7fce1ffff640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce0c073200 0x7fce0c075710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fce14009ad0 tx=0x7fce14009260 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:07.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.066+0000 7fce1cff9640 1 -- 172.21.15.131:0/2693392251 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fce2006e7f0 con 0x7fce2015c3e0 2024-09-17T02:45:07.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.258+0000 7fce26785640 1 -- 172.21.15.131:0/2693392251 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce200b0640 con 0x7fce2015c3e0 2024-09-17T02:45:07.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.258+0000 7fce1cff9640 1 -- 172.21.15.131:0/2693392251 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+5234 (secure 0 0 0) 0x7fce1800ce20 con 0x7fce2015c3e0 2024-09-17T02:45:07.257 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:07.257 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":38,"btime":"2024-09-17T02:44:38:778548+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3554877374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3554877374},{"type":"v1","addr":"172.21.15.189:6837","nonce":3554877374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1670590113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1670590113},{"type":"v1","addr":"172.21.15.131:6834","nonce":1670590113}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1046159039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1046159039},{"type":"v1","addr":"172.21.15.131:6835","nonce":1046159039}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:43:46.061550+0000","modified":"2024-09-17T02:44:38.728223+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5871},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5871":{"gid":5871,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":16,"addr":"172.21.15.189:6835/730392436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":730392436},{"type":"v1","addr":"172.21.15.189:6835","nonce":730392436}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5871,"qdb_cluster":[5871]},"id":4}]} 2024-09-17T02:45:07.257 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 38 2024-09-17T02:45:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 -- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce0c073200 msgr2=0x7fce0c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce0c073200 0x7fce0c075710 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fce14009ad0 tx=0x7fce14009260 comp rx=0 tx=0).stop 2024-09-17T02:45:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 -- 172.21.15.131:0/2693392251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce2015c3e0 msgr2=0x7fce20160e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce2015c3e0 0x7fce20160e80 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c710 tx=0x7fce1800cbd0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 -- 172.21.15.131:0/2693392251 shutdown_connections 2024-09-17T02:45:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fce0c073200 0x7fce0c075710 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fce2015c3e0 0x7fce20160e80 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fce200ced00 0x7fce200afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 --2- 172.21.15.131:0/2693392251 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fce2006e3e0 0x7fce20146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 -- 172.21.15.131:0/2693392251 >> 172.21.15.131:0/2693392251 conn(0x7fce2006eea0 msgr2=0x7fce200ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 -- 172.21.15.131:0/2693392251 shutdown_connections 2024-09-17T02:45:07.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.262+0000 7fce26785640 1 -- 172.21.15.131:0/2693392251 wait complete. 2024-09-17T02:45:07.277 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:45:07.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.422+0000 7f10802f1640 1 -- 172.21.15.131:0/2908171496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f107815c5b0 msgr2=0x7f10780cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.422+0000 7f10802f1640 1 --2- 172.21.15.131:0/2908171496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f107815c5b0 0x7f10780cbaf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f1070009bc0 tx=0x7f107001cba0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.422+0000 7f10802f1640 1 -- 172.21.15.131:0/2908171496 shutdown_connections 2024-09-17T02:45:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.422+0000 7f10802f1640 1 --2- 172.21.15.131:0/2908171496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f107815c5b0 0x7f10780cbaf0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.422+0000 7f10802f1640 1 -- 172.21.15.131:0/2908171496 >> 172.21.15.131:0/2908171496 conn(0x7f107806eea0 msgr2=0x7f107806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.422+0000 7f10802f1640 1 -- 172.21.15.131:0/2908171496 shutdown_connections 2024-09-17T02:45:07.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.422+0000 7f10802f1640 1 -- 172.21.15.131:0/2908171496 wait complete. 2024-09-17T02:45:07.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f10802f1640 1 Processor -- start 2024-09-17T02:45:07.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f10802f1640 1 -- start start 2024-09-17T02:45:07.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f10802f1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10780ced00 0x7f1078154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f10802f1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f107815c5b0 0x7f1078158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f10802f1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1078158980 0x7f107814e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f10802f1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f10780ce440 con 0x7f10780ced00 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f10802f1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f10780ce2c0 con 0x7f107815c5b0 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f10802f1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f10780ce5c0 con 0x7f1078158980 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f107e08d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10780ced00 0x7f1078154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f107e08d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10780ced00 0x7f1078154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56780/0 (socket says 172.21.15.131:56780) 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f107e08d640 1 -- 172.21.15.131:0/3405727953 learned_addr learned my addr 172.21.15.131:0/3405727953 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f107d88c640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f107815c5b0 0x7f1078158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f107e08d640 1 -- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1078158980 msgr2=0x7f107814e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f107e08d640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1078158980 0x7f107814e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f107e08d640 1 -- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f107815c5b0 msgr2=0x7f1078158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f107e08d640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f107815c5b0 0x7f1078158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f107e08d640 1 -- 172.21.15.131:0/3405727953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10780b3220 con 0x7f10780ced00 2024-09-17T02:45:07.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f107e08d640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10780ced00 0x7f1078154d10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f106800c400 tx=0x7f106800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:07.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f106effd640 1 -- 172.21.15.131:0/3405727953 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068011790 con 0x7f10780ced00 2024-09-17T02:45:07.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f106effd640 1 -- 172.21.15.131:0/3405727953 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1068017070 con 0x7f10780ced00 2024-09-17T02:45:07.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f106effd640 1 -- 172.21.15.131:0/3405727953 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1068011cc0 con 0x7f10780ced00 2024-09-17T02:45:07.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f10802f1640 1 -- 172.21.15.131:0/3405727953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10780b3500 con 0x7f10780ced00 2024-09-17T02:45:07.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f10802f1640 1 -- 172.21.15.131:0/3405727953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10780b3ac0 con 0x7f10780ced00 2024-09-17T02:45:07.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f10802f1640 1 -- 172.21.15.131:0/3405727953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1078075b80 con 0x7f10780ced00 2024-09-17T02:45:07.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.426+0000 7f106effd640 1 -- 172.21.15.131:0/3405727953 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f106801c460 con 0x7f10780ced00 2024-09-17T02:45:07.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.430+0000 7f106effd640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f105c073200 0x7f105c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.430+0000 7f106effd640 1 -- 172.21.15.131:0/3405727953 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f1068094d40 con 0x7f10780ced00 2024-09-17T02:45:07.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.430+0000 7f107d88c640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f105c073200 0x7f105c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:07.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.430+0000 7f107d88c640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f105c073200 0x7f105c075710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f107806a6c0 tx=0x7f1074008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:07.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.434+0000 7f106effd640 1 -- 172.21.15.131:0/3405727953 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f107815c9c0 con 0x7f10780ced00 2024-09-17T02:45:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.590+0000 7f10802f1640 1 -- 172.21.15.131:0/3405727953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f107815cf20 con 0x7f10780ced00 2024-09-17T02:45:07.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f106effd640 1 -- 172.21.15.131:0/3405727953 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+17378 (secure 0 0 0) 0x7f106805fb10 con 0x7f10780ced00 2024-09-17T02:45:07.592 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:07.592 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":68,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:44:35.331723+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:43:43.717619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-17T02:43:44.992949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.16","mappings":[{"from":2,"to":7}]},{"pgid":"10.1b","mappings":[{"from":2,"to":5}]},{"pgid":"10.24","mappings":[{"from":2,"to":7}]},{"pgid":"10.2b","mappings":[{"from":4,"to":1}]},{"pgid":"10.2d","mappings":[{"from":2,"to":5}]},{"pgid":"10.33","mappings":[{"from":2,"to":7}]},{"pgid":"10.34","mappings":[{"from":2,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:45:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 -- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f105c073200 msgr2=0x7f105c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f105c073200 0x7f105c075710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f107806a6c0 tx=0x7f1074008040 comp rx=0 tx=0).stop 2024-09-17T02:45:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 -- 172.21.15.131:0/3405727953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10780ced00 msgr2=0x7f1078154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10780ced00 0x7f1078154d10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f106800c400 tx=0x7f106800c8c0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 -- 172.21.15.131:0/3405727953 shutdown_connections 2024-09-17T02:45:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f105c073200 0x7f105c075710 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1078158980 0x7f107814e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f107815c5b0 0x7f1078158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 --2- 172.21.15.131:0/3405727953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10780ced00 0x7f1078154d10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 -- 172.21.15.131:0/3405727953 >> 172.21.15.131:0/3405727953 conn(0x7f107806eea0 msgr2=0x7f107815d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 -- 172.21.15.131:0/3405727953 shutdown_connections 2024-09-17T02:45:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.594+0000 7f10802f1640 1 -- 172.21.15.131:0/3405727953 wait complete. 2024-09-17T02:45:07.613 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:45:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.754+0000 7f8da9b95640 1 -- 172.21.15.131:0/3779733472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da4152550 msgr2=0x7f8da406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.754+0000 7f8da9b95640 1 --2- 172.21.15.131:0/3779733472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da4152550 0x7f8da406e840 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009840 tx=0x7f8d9c01c760 comp rx=0 tx=0).stop 2024-09-17T02:45:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.754+0000 7f8da9b95640 1 -- 172.21.15.131:0/3779733472 shutdown_connections 2024-09-17T02:45:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.754+0000 7f8da9b95640 1 --2- 172.21.15.131:0/3779733472 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8da4071920 0x7f8da40b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.754+0000 7f8da9b95640 1 --2- 172.21.15.131:0/3779733472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da4152550 0x7f8da406e840 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.754+0000 7f8da9b95640 1 -- 172.21.15.131:0/3779733472 >> 172.21.15.131:0/3779733472 conn(0x7f8da40d1130 msgr2=0x7f8da40d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.754+0000 7f8da9b95640 1 -- 172.21.15.131:0/3779733472 shutdown_connections 2024-09-17T02:45:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.754+0000 7f8da9b95640 1 -- 172.21.15.131:0/3779733472 wait complete. 2024-09-17T02:45:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da9b95640 1 Processor -- start 2024-09-17T02:45:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da9b95640 1 -- start start 2024-09-17T02:45:07.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da9b95640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da4152550 0x7f8da414c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da9b95640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8da415ba70 0x7f8da414d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da9b95640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8da414ddf0 0x7f8da414e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da9b95640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8da4169160 con 0x7f8da4152550 2024-09-17T02:45:07.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da9b95640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8da4168fe0 con 0x7f8da414ddf0 2024-09-17T02:45:07.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da9b95640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8da41692e0 con 0x7f8da415ba70 2024-09-17T02:45:07.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da4152550 0x7f8da414c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da4152550 0x7f8da414c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56790/0 (socket says 172.21.15.131:56790) 2024-09-17T02:45:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da37fe640 1 -- 172.21.15.131:0/4079124398 learned_addr learned my addr 172.21.15.131:0/4079124398 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da37fe640 1 -- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8da415ba70 msgr2=0x7f8da414d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da37fe640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8da415ba70 0x7f8da414d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da37fe640 1 -- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8da414ddf0 msgr2=0x7f8da414e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da37fe640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8da414ddf0 0x7f8da414e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da37fe640 1 -- 172.21.15.131:0/4079124398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da414e740 con 0x7f8da4152550 2024-09-17T02:45:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da37fe640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da4152550 0x7f8da414c4f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c01ae10 tx=0x7f8d9c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:07.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8d8bfff640 1 -- 172.21.15.131:0/4079124398 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0bb070 con 0x7f8da4152550 2024-09-17T02:45:07.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8d8bfff640 1 -- 172.21.15.131:0/4079124398 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d9c005030 con 0x7f8da4152550 2024-09-17T02:45:07.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da9b95640 1 -- 172.21.15.131:0/4079124398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da41590e0 con 0x7f8da4152550 2024-09-17T02:45:07.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8d8bfff640 1 -- 172.21.15.131:0/4079124398 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0b2040 con 0x7f8da4152550 2024-09-17T02:45:07.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.758+0000 7f8da9b95640 1 -- 172.21.15.131:0/4079124398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da41564b0 con 0x7f8da4152550 2024-09-17T02:45:07.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.762+0000 7f8da9b95640 1 -- 172.21.15.131:0/4079124398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da40b40e0 con 0x7f8da4152550 2024-09-17T02:45:07.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.762+0000 7f8d8bfff640 1 -- 172.21.15.131:0/4079124398 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8d9c0be070 con 0x7f8da4152550 2024-09-17T02:45:07.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.762+0000 7f8d8bfff640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d90073200 0x7f8d90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:07.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.762+0000 7f8d8bfff640 1 -- 172.21.15.131:0/4079124398 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6304+0+0 (secure 0 0 0) 0x7f8d9c12df50 con 0x7f8da4152550 2024-09-17T02:45:07.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.762+0000 7f8da2ffd640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d90073200 0x7f8d90075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:07.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.762+0000 7f8da2ffd640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d90073200 0x7f8d90075710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d98007fd0 tx=0x7f8d98007d40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:07.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.766+0000 7f8d8bfff640 1 -- 172.21.15.131:0/4079124398 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8d9c016410 con 0x7f8da4152550 2024-09-17T02:45:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.962+0000 7f8da9b95640 1 -- 172.21.15.131:0/4079124398 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f8da4071ff0 con 0x7f8da4152550 2024-09-17T02:45:07.963 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:45:07.966+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:45:07.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.966+0000 7f8d8bfff640 1 -- 172.21.15.131:0/4079124398 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) ==== 161+0+0 (secure 0 0 0) 0x7f8d9c0f8db0 con 0x7f8da4152550 2024-09-17T02:45:07.966 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:45:07.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 -- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d90073200 msgr2=0x7f8d90075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d90073200 0x7f8d90075710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d98007fd0 tx=0x7f8d98007d40 comp rx=0 tx=0).stop 2024-09-17T02:45:07.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 -- 172.21.15.131:0/4079124398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da4152550 msgr2=0x7f8da414c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:07.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da4152550 0x7f8da414c4f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c01ae10 tx=0x7f8d9c004040 comp rx=0 tx=0).stop 2024-09-17T02:45:07.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 -- 172.21.15.131:0/4079124398 shutdown_connections 2024-09-17T02:45:07.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8d90073200 0x7f8d90075710 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8da414ddf0 0x7f8da414e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8da415ba70 0x7f8da414d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 --2- 172.21.15.131:0/4079124398 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8da4152550 0x7f8da414c4f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:07.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 -- 172.21.15.131:0/4079124398 >> 172.21.15.131:0/4079124398 conn(0x7f8da40d1130 msgr2=0x7f8da4165ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:07.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 -- 172.21.15.131:0/4079124398 shutdown_connections 2024-09-17T02:45:07.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:07.970+0000 7f8da9b95640 1 -- 172.21.15.131:0/4079124398 wait complete. 2024-09-17T02:45:07.985 INFO:tasks.ceph.mds.d.smithi189.stderr: -7829> 2024-09-17T02:43:48.601+0000 7fbd29b6b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:45:07.988 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T02:45:07.991 INFO:tasks.ceph.mds.d.smithi189.stderr: -5579> 2024-09-17T02:44:01.377+0000 7fbd25b63640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 pv4 ap=4 s=0 n(v0 1=1+0)->n(v0 rc2024-09-17T02:44:00.158448+0000 1=1+0) (iauth excl w=1) (ifile lock->xlock w=1) (iquiesce lock w=1) cr={6048=0-4194304@1} caps={6048=pAsLsXs/-@3},l=6048(-1) | ptrwaiter=1 request=1 lock=4 caps=1 dirtyparent=0 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x55b59e570580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-17T02:45:07.996 INFO:tasks.ceph.mds.d.smithi189.stderr: -3422> 2024-09-17T02:44:33.001+0000 7fbd25b63640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-09-17T02:44:00.158448+0000 1=1+0) caps={6048=pLsXs/-@4} | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x55b59e570580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-17T02:45:08.069 INFO:tasks.ceph.mds.d.smithi189.stdout:starting mds.d at 2024-09-17T02:45:08.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.134+0000 7f58ffd01640 1 -- 172.21.15.131:0/906553550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58e4005920 msgr2=0x7f58e4005d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:08.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.134+0000 7f58ffd01640 1 --2- 172.21.15.131:0/906553550 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58e4005920 0x7f58e4005d30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b9e0 tx=0x7f58e801cc00 comp rx=0 tx=0).stop 2024-09-17T02:45:08.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.134+0000 7f58fe29e640 1 -- 172.21.15.131:0/906553550 reap_dead start 2024-09-17T02:45:08.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.134+0000 7f58ffd01640 1 -- 172.21.15.131:0/906553550 shutdown_connections 2024-09-17T02:45:08.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.134+0000 7f58ffd01640 1 -- 172.21.15.131:0/906553550 >> 172.21.15.131:0/906553550 conn(0x7f58f806eea0 msgr2=0x7f58f806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:08.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.134+0000 7f58ffd01640 1 -- 172.21.15.131:0/906553550 shutdown_connections 2024-09-17T02:45:08.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.134+0000 7f58ffd01640 1 -- 172.21.15.131:0/906553550 wait complete. 2024-09-17T02:45:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ffd01640 1 Processor -- start 2024-09-17T02:45:08.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ffd01640 1 -- start start 2024-09-17T02:45:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ffd01640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e4005920 0x7f58f815f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ffd01640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f8146790 0x7f58f8144a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:08.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ffd01640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f58f8158db0 0x7f58f815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ffd01640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f58f806f9c0 con 0x7f58f8146790 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ffd01640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58f806e570 con 0x7f58e4005920 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ffd01640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f58f806fb40 con 0x7f58f8158db0 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58fd29c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f8146790 0x7f58f8144a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58fd29c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f8146790 0x7f58f8144a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56808/0 (socket says 172.21.15.131:56808) 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58fd29c640 1 -- 172.21.15.131:0/3164791179 learned_addr learned my addr 172.21.15.131:0/3164791179 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58fe29e640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f58f8158db0 0x7f58f815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58fda9d640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e4005920 0x7f58f815f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58fd29c640 1 -- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f58f8158db0 msgr2=0x7f58f815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58fd29c640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f58f8158db0 0x7f58f815dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58fd29c640 1 -- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e4005920 msgr2=0x7f58f815f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58fd29c640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e4005920 0x7f58f815f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58fd29c640 1 -- 172.21.15.131:0/3164791179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58f8152690 con 0x7f58f8146790 2024-09-17T02:45:08.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58fd29c640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f8146790 0x7f58f8144a70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f58e80b1040 tx=0x7f58e80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ee7fc640 1 -- 172.21.15.131:0/3164791179 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e80ab040 con 0x7f58f8146790 2024-09-17T02:45:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ee7fc640 1 -- 172.21.15.131:0/3164791179 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58e80048e0 con 0x7f58f8146790 2024-09-17T02:45:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ffd01640 1 -- 172.21.15.131:0/3164791179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58f815aeb0 con 0x7f58f8146790 2024-09-17T02:45:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ee7fc640 1 -- 172.21.15.131:0/3164791179 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8004d30 con 0x7f58f8146790 2024-09-17T02:45:08.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ffd01640 1 -- 172.21.15.131:0/3164791179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58f8143f90 con 0x7f58f8146790 2024-09-17T02:45:08.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.138+0000 7f58ffd01640 1 -- 172.21.15.131:0/3164791179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58f8073430 con 0x7f58f8146790 2024-09-17T02:45:08.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.142+0000 7f58ee7fc640 1 -- 172.21.15.131:0/3164791179 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f58e80c2020 con 0x7f58f8146790 2024-09-17T02:45:08.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.142+0000 7f58ee7fc640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58dc073200 0x7f58dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:08.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.142+0000 7f58ee7fc640 1 -- 172.21.15.131:0/3164791179 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6390+0+0 (secure 0 0 0) 0x7f58e8131310 con 0x7f58f8146790 2024-09-17T02:45:08.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.142+0000 7f58fda9d640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58dc073200 0x7f58dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:08.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.142+0000 7f58fda9d640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58dc073200 0x7f58dc075710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f58f4001020 tx=0x7f58f4006740 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:08.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.146+0000 7f58ee7fc640 1 -- 172.21.15.131:0/3164791179 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f58e80c9020 con 0x7f58f8146790 2024-09-17T02:45:08.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.338+0000 7f58ffd01640 1 -- 172.21.15.131:0/3164791179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f58f806e9d0 con 0x7f58f8146790 2024-09-17T02:45:08.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.970+0000 7f58ee7fc640 1 -- 172.21.15.131:0/3164791179 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) ==== 104+0+0 (secure 0 0 0) 0x7f58f8152690 con 0x7f58f8146790 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 -- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58dc073200 msgr2=0x7f58dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58dc073200 0x7f58dc075710 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f58f4001020 tx=0x7f58f4006740 comp rx=0 tx=0).stop 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 -- 172.21.15.131:0/3164791179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f8146790 msgr2=0x7f58f8144a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f8146790 0x7f58f8144a70 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f58e80b1040 tx=0x7f58e80056a0 comp rx=0 tx=0).stop 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 -- 172.21.15.131:0/3164791179 shutdown_connections 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58dc073200 0x7f58dc075710 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f58f8158db0 0x7f58f815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58f8146790 0x7f58f8144a70 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 --2- 172.21.15.131:0/3164791179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58e4005920 0x7f58f815f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 -- 172.21.15.131:0/3164791179 >> 172.21.15.131:0/3164791179 conn(0x7f58f806eea0 msgr2=0x7f58f80ca0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 -- 172.21.15.131:0/3164791179 shutdown_connections 2024-09-17T02:45:08.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:08.974+0000 7f58ffd01640 1 -- 172.21.15.131:0/3164791179 wait complete. 2024-09-17T02:45:08.999 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T02:45:09.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.162+0000 7f3712b02640 1 -- 172.21.15.131:0/3990347709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c15f7d0 msgr2=0x7f370c073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:09.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.162+0000 7f3712b02640 1 --2- 172.21.15.131:0/3990347709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c15f7d0 0x7f370c073120 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f3704009cb0 tx=0x7f370401cda0 comp rx=0 tx=0).stop 2024-09-17T02:45:09.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.162+0000 7f3712b02640 1 -- 172.21.15.131:0/3990347709 shutdown_connections 2024-09-17T02:45:09.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.162+0000 7f3712b02640 1 --2- 172.21.15.131:0/3990347709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c15f7d0 0x7f370c073120 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:09.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.162+0000 7f3712b02640 1 --2- 172.21.15.131:0/3990347709 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f370c0b2ef0 0x7f370c152200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:09.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.162+0000 7f3712b02640 1 -- 172.21.15.131:0/3990347709 >> 172.21.15.131:0/3990347709 conn(0x7f370c0d05a0 msgr2=0x7f370c0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:09.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.162+0000 7f3712b02640 1 -- 172.21.15.131:0/3990347709 shutdown_connections 2024-09-17T02:45:09.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.162+0000 7f3712b02640 1 -- 172.21.15.131:0/3990347709 wait complete. 2024-09-17T02:45:09.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.162+0000 7f3712b02640 1 Processor -- start 2024-09-17T02:45:09.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.162+0000 7f3712b02640 1 -- start start 2024-09-17T02:45:09.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f3712b02640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f370c0b2ef0 0x7f370c1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:09.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f3712b02640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f370c158bb0 0x7f370c145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:09.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f3712b02640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c15de50 0x7f370c15e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f3712b02640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f370c1687a0 con 0x7f370c15de50 2024-09-17T02:45:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f3712b02640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f370c168620 con 0x7f370c158bb0 2024-09-17T02:45:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f3712b02640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f370c168920 con 0x7f370c0b2ef0 2024-09-17T02:45:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f371109f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c15de50 0x7f370c15e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f371109f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c15de50 0x7f370c15e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56826/0 (socket says 172.21.15.131:56826) 2024-09-17T02:45:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f371109f640 1 -- 172.21.15.131:0/622338953 learned_addr learned my addr 172.21.15.131:0/622338953 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f371089e640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f370c0b2ef0 0x7f370c1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f370bfff640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f370c158bb0 0x7f370c145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:09.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f371109f640 1 -- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f370c0b2ef0 msgr2=0x7f370c1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f371109f640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f370c0b2ef0 0x7f370c1564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f371109f640 1 -- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f370c158bb0 msgr2=0x7f370c145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f371109f640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f370c158bb0 0x7f370c145300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f371109f640 1 -- 172.21.15.131:0/622338953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c145840 con 0x7f370c15de50 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f371089e640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f370c0b2ef0 0x7f370c1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f370bfff640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f370c158bb0 0x7f370c145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f371109f640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c15de50 0x7f370c15e260 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3704009bd0 tx=0x7f37040092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f37097fa640 1 -- 172.21.15.131:0/622338953 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37040bb070 con 0x7f370c15de50 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f37097fa640 1 -- 172.21.15.131:0/622338953 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3704004d40 con 0x7f370c15de50 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f37097fa640 1 -- 172.21.15.131:0/622338953 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37040b2040 con 0x7f370c15de50 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f3712b02640 1 -- 172.21.15.131:0/622338953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370c149250 con 0x7f370c15de50 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f3712b02640 1 -- 172.21.15.131:0/622338953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370c15a470 con 0x7f370c15de50 2024-09-17T02:45:09.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f3712b02640 1 -- 172.21.15.131:0/622338953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f370c15ee50 con 0x7f370c15de50 2024-09-17T02:45:09.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f37097fa640 1 -- 172.21.15.131:0/622338953 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f37040be070 con 0x7f370c15de50 2024-09-17T02:45:09.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.166+0000 7f37097fa640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f36f8073230 0x7f36f8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:09.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.170+0000 7f37097fa640 1 -- 172.21.15.131:0/622338953 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6390+0+0 (secure 0 0 0) 0x7f370412e470 con 0x7f370c15de50 2024-09-17T02:45:09.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.170+0000 7f371089e640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f36f8073230 0x7f36f8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:09.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.170+0000 7f371089e640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f36f8073230 0x7f36f8075740 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00a870 tx=0x7f36fc008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:09.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.174+0000 7f37097fa640 1 -- 172.21.15.131:0/622338953 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f370c145840 con 0x7f370c15de50 2024-09-17T02:45:09.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.342+0000 7f3712b02640 1 -- 172.21.15.131:0/622338953 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f370c073120 con 0x7f370c15de50 2024-09-17T02:45:09.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.978+0000 7f37097fa640 1 -- 172.21.15.131:0/622338953 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v70) ==== 181+0+0 (secure 0 0 0) 0x7f3704016990 con 0x7f370c15de50 2024-09-17T02:45:09.978 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-09-17T02:45:09.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.982+0000 7f3712b02640 1 -- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f36f8073230 msgr2=0x7f36f8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:09.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.982+0000 7f3712b02640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f36f8073230 0x7f36f8075740 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00a870 tx=0x7f36fc008040 comp rx=0 tx=0).stop 2024-09-17T02:45:09.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.982+0000 7f3712b02640 1 -- 172.21.15.131:0/622338953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c15de50 msgr2=0x7f370c15e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:09.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.982+0000 7f3712b02640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c15de50 0x7f370c15e260 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3704009bd0 tx=0x7f37040092c0 comp rx=0 tx=0).stop 2024-09-17T02:45:09.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.982+0000 7f3712b02640 1 -- 172.21.15.131:0/622338953 shutdown_connections 2024-09-17T02:45:09.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.982+0000 7f3712b02640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f36f8073230 0x7f36f8075740 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:09.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.982+0000 7f3712b02640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f370c15de50 0x7f370c15e260 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:09.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.982+0000 7f3712b02640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f370c158bb0 0x7f370c145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:09.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.982+0000 7f3712b02640 1 --2- 172.21.15.131:0/622338953 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f370c0b2ef0 0x7f370c1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:09.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.982+0000 7f3712b02640 1 -- 172.21.15.131:0/622338953 >> 172.21.15.131:0/622338953 conn(0x7f370c0d05a0 msgr2=0x7f370c155270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:09.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.986+0000 7f3712b02640 1 -- 172.21.15.131:0/622338953 shutdown_connections 2024-09-17T02:45:09.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:09.986+0000 7f3712b02640 1 -- 172.21.15.131:0/622338953 wait complete. 2024-09-17T02:45:10.008 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T02:45:10.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.162+0000 7f703384b640 1 -- 172.21.15.131:0/280932314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018003a10 msgr2=0x7f7018003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:10.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.162+0000 7f703384b640 1 --2- 172.21.15.131:0/280932314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018003a10 0x7f7018003e80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7028009b60 tx=0x7f702801d290 comp rx=0 tx=0).stop 2024-09-17T02:45:10.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.162+0000 7f7031de8640 1 -- 172.21.15.131:0/280932314 reap_dead start 2024-09-17T02:45:10.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.162+0000 7f703384b640 1 -- 172.21.15.131:0/280932314 shutdown_connections 2024-09-17T02:45:10.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.162+0000 7f703384b640 1 -- 172.21.15.131:0/280932314 >> 172.21.15.131:0/280932314 conn(0x7f702c0d42b0 msgr2=0x7f702c0d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:10.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.162+0000 7f703384b640 1 -- 172.21.15.131:0/280932314 shutdown_connections 2024-09-17T02:45:10.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.162+0000 7f703384b640 1 -- 172.21.15.131:0/280932314 wait complete. 2024-09-17T02:45:10.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.162+0000 7f703384b640 1 Processor -- start 2024-09-17T02:45:10.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.162+0000 7f703384b640 1 -- start start 2024-09-17T02:45:10.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f703384b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018003a10 0x7f702c156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:10.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f703384b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f702c150d60 0x7f702c1f95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:10.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f703384b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f702c1684f0 0x7f702c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:10.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f703384b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f702c0b9070 con 0x7f7018003a10 2024-09-17T02:45:10.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f703384b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f702c0b8ef0 con 0x7f702c1684f0 2024-09-17T02:45:10.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f703384b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f702c0b91f0 con 0x7f702c150d60 2024-09-17T02:45:10.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f70315e7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018003a10 0x7f702c156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:10.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f70315e7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018003a10 0x7f702c156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:56840/0 (socket says 172.21.15.131:56840) 2024-09-17T02:45:10.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f70315e7640 1 -- 172.21.15.131:0/217613784 learned_addr learned my addr 172.21.15.131:0/217613784 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f70315e7640 1 -- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f702c150d60 msgr2=0x7f702c1f95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f70315e7640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f702c150d60 0x7f702c1f95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f70315e7640 1 -- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f702c1684f0 msgr2=0x7f702c154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f70315e7640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f702c1684f0 0x7f702c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f70315e7640 1 -- 172.21.15.131:0/217613784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c169880 con 0x7f7018003a10 2024-09-17T02:45:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f70315e7640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018003a10 0x7f702c156b10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f702801d0c0 tx=0x7f7028005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f7021ffb640 1 -- 172.21.15.131:0/217613784 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70280b1040 con 0x7f7018003a10 2024-09-17T02:45:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f7021ffb640 1 -- 172.21.15.131:0/217613784 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70280bb070 con 0x7f7018003a10 2024-09-17T02:45:10.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f7021ffb640 1 -- 172.21.15.131:0/217613784 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028004290 con 0x7f7018003a10 2024-09-17T02:45:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f703384b640 1 -- 172.21.15.131:0/217613784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702c169b60 con 0x7f7018003a10 2024-09-17T02:45:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f703384b640 1 -- 172.21.15.131:0/217613784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702c156310 con 0x7f7018003a10 2024-09-17T02:45:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.166+0000 7f703384b640 1 -- 172.21.15.131:0/217613784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f702c0b3590 con 0x7f7018003a10 2024-09-17T02:45:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.170+0000 7f7021ffb640 1 -- 172.21.15.131:0/217613784 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f70280046d0 con 0x7f7018003a10 2024-09-17T02:45:10.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.170+0000 7f7021ffb640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7010073230 0x7f7010075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:10.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.170+0000 7f7021ffb640 1 -- 172.21.15.131:0/217613784 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5952+0+0 (secure 0 0 0) 0x7f702812fd00 con 0x7f7018003a10 2024-09-17T02:45:10.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.170+0000 7f7030de6640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7010073230 0x7f7010075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:10.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.170+0000 7f7030de6640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7010073230 0x7f7010075740 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f702c1f66b0 tx=0x7f701c009540 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:10.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.174+0000 7f7021ffb640 1 -- 172.21.15.131:0/217613784 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7028016fa0 con 0x7f7018003a10 2024-09-17T02:45:10.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.354+0000 7f703384b640 1 -- 172.21.15.131:0/217613784 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f702c06e000 con 0x7f7018003a10 2024-09-17T02:45:10.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.990+0000 7f7021ffb640 1 -- 172.21.15.131:0/217613784 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v71) ==== 169+0+0 (secure 0 0 0) 0x7f70280fadc0 con 0x7f7018003a10 2024-09-17T02:45:10.989 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-09-17T02:45:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 -- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7010073230 msgr2=0x7f7010075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7010073230 0x7f7010075740 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f702c1f66b0 tx=0x7f701c009540 comp rx=0 tx=0).stop 2024-09-17T02:45:10.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 -- 172.21.15.131:0/217613784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018003a10 msgr2=0x7f702c156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018003a10 0x7f702c156b10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f702801d0c0 tx=0x7f7028005520 comp rx=0 tx=0).stop 2024-09-17T02:45:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 -- 172.21.15.131:0/217613784 shutdown_connections 2024-09-17T02:45:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7010073230 0x7f7010075740 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f702c1684f0 0x7f702c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f702c150d60 0x7f702c1f95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 --2- 172.21.15.131:0/217613784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7018003a10 0x7f702c156b10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:10.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 -- 172.21.15.131:0/217613784 >> 172.21.15.131:0/217613784 conn(0x7f702c0d42b0 msgr2=0x7f702c15c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:10.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 -- 172.21.15.131:0/217613784 shutdown_connections 2024-09-17T02:45:10.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:10.994+0000 7f703384b640 1 -- 172.21.15.131:0/217613784 wait complete. 2024-09-17T02:45:11.022 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T02:45:11.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.182+0000 7f5a419dc640 1 -- 172.21.15.131:0/946323341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c155240 msgr2=0x7f5a3c073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:11.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.182+0000 7f5a419dc640 1 --2- 172.21.15.131:0/946323341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c155240 0x7f5a3c073430 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009bc0 tx=0x7f5a2c01cb60 comp rx=0 tx=0).stop 2024-09-17T02:45:11.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.182+0000 7f5a419dc640 1 -- 172.21.15.131:0/946323341 shutdown_connections 2024-09-17T02:45:11.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.182+0000 7f5a419dc640 1 --2- 172.21.15.131:0/946323341 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c155240 0x7f5a3c073430 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.182+0000 7f5a419dc640 1 -- 172.21.15.131:0/946323341 >> 172.21.15.131:0/946323341 conn(0x7f5a3c06eea0 msgr2=0x7f5a3c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:11.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.182+0000 7f5a419dc640 1 -- 172.21.15.131:0/946323341 shutdown_connections 2024-09-17T02:45:11.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.182+0000 7f5a419dc640 1 -- 172.21.15.131:0/946323341 wait complete. 2024-09-17T02:45:11.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a419dc640 1 Processor -- start 2024-09-17T02:45:11.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a419dc640 1 -- start start 2024-09-17T02:45:11.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a419dc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a3c073970 0x7f5a3c153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:11.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a419dc640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a3c155240 0x7f5a3c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:11.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a419dc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c156ca0 0x7f5a3c1f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:11.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a419dc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c168b80 con 0x7f5a3c156ca0 2024-09-17T02:45:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a419dc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a3c0cbee0 con 0x7f5a3c073970 2024-09-17T02:45:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a419dc640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3c168d00 con 0x7f5a3c155240 2024-09-17T02:45:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c156ca0 0x7f5a3c1f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c156ca0 0x7f5a3c1f4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37650/0 (socket says 172.21.15.131:37650) 2024-09-17T02:45:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3b7fe640 1 -- 172.21.15.131:0/37988342 learned_addr learned my addr 172.21.15.131:0/37988342 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3affd640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a3c073970 0x7f5a3c153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3a7fc640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a3c155240 0x7f5a3c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3b7fe640 1 -- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a3c155240 msgr2=0x7f5a3c152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3b7fe640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a3c155240 0x7f5a3c152a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3b7fe640 1 -- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a3c073970 msgr2=0x7f5a3c153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:11.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3b7fe640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a3c073970 0x7f5a3c153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3b7fe640 1 -- 172.21.15.131:0/37988342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3c152fa0 con 0x7f5a3c156ca0 2024-09-17T02:45:11.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3a7fc640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a3c155240 0x7f5a3c152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:11.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3affd640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a3c073970 0x7f5a3c153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:11.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a3b7fe640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c156ca0 0x7f5a3c1f4650 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400c400 tx=0x7f5a3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:11.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a23fff640 1 -- 172.21.15.131:0/37988342 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34011790 con 0x7f5a3c156ca0 2024-09-17T02:45:11.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a23fff640 1 -- 172.21.15.131:0/37988342 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a34017070 con 0x7f5a3c156ca0 2024-09-17T02:45:11.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a23fff640 1 -- 172.21.15.131:0/37988342 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a34011cc0 con 0x7f5a3c156ca0 2024-09-17T02:45:11.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a419dc640 1 -- 172.21.15.131:0/37988342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a3c155a20 con 0x7f5a3c156ca0 2024-09-17T02:45:11.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.186+0000 7f5a419dc640 1 -- 172.21.15.131:0/37988342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a3c155fa0 con 0x7f5a3c156ca0 2024-09-17T02:45:11.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.190+0000 7f5a419dc640 1 -- 172.21.15.131:0/37988342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3c073430 con 0x7f5a3c156ca0 2024-09-17T02:45:11.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.190+0000 7f5a23fff640 1 -- 172.21.15.131:0/37988342 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a3401c460 con 0x7f5a3c156ca0 2024-09-17T02:45:11.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.190+0000 7f5a23fff640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a28073200 0x7f5a28075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:11.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.190+0000 7f5a23fff640 1 -- 172.21.15.131:0/37988342 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7f5a34093ee0 con 0x7f5a3c156ca0 2024-09-17T02:45:11.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.190+0000 7f5a3affd640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a28073200 0x7f5a28075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:11.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.190+0000 7f5a3affd640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a28073200 0x7f5a28075710 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0af000 tx=0x7f5a2c01b3a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:11.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.194+0000 7f5a23fff640 1 -- 172.21.15.131:0/37988342 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a3c152fa0 con 0x7f5a3c156ca0 2024-09-17T02:45:11.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.358+0000 7f5a419dc640 1 -- 172.21.15.131:0/37988342 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f5a3c154500 con 0x7f5a3c156ca0 2024-09-17T02:45:11.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.366+0000 7f5a23fff640 1 -- 172.21.15.131:0/37988342 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7f5a3405f070 con 0x7f5a3c156ca0 2024-09-17T02:45:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 -- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a28073200 msgr2=0x7f5a28075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a28073200 0x7f5a28075710 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c0af000 tx=0x7f5a2c01b3a0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 -- 172.21.15.131:0/37988342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c156ca0 msgr2=0x7f5a3c1f4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c156ca0 0x7f5a3c1f4650 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5a3400c400 tx=0x7f5a3400c8c0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 -- 172.21.15.131:0/37988342 shutdown_connections 2024-09-17T02:45:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a28073200 0x7f5a28075710 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a3c156ca0 0x7f5a3c1f4650 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a3c155240 0x7f5a3c152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 --2- 172.21.15.131:0/37988342 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a3c073970 0x7f5a3c153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 -- 172.21.15.131:0/37988342 >> 172.21.15.131:0/37988342 conn(0x7f5a3c06eea0 msgr2=0x7f5a3c0b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 -- 172.21.15.131:0/37988342 shutdown_connections 2024-09-17T02:45:11.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.370+0000 7f5a419dc640 1 -- 172.21.15.131:0/37988342 wait complete. 2024-09-17T02:45:11.386 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-09-17T02:45:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.546+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1279406375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa90003a90 msgr2=0x7ffa90003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.546+0000 7ffaaccc3640 1 --2- 172.21.15.131:0/1279406375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa90003a90 0x7ffa90003f00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ffa980016f0 tx=0x7ffa9801c780 comp rx=0 tx=0).stop 2024-09-17T02:45:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.546+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1279406375 shutdown_connections 2024-09-17T02:45:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.546+0000 7ffaaccc3640 1 --2- 172.21.15.131:0/1279406375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa90003a90 0x7ffa90003f00 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.546+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1279406375 >> 172.21.15.131:0/1279406375 conn(0x7ffaa806eea0 msgr2=0x7ffaa806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.546+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1279406375 shutdown_connections 2024-09-17T02:45:11.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.546+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1279406375 wait complete. 2024-09-17T02:45:11.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.546+0000 7ffaaccc3640 1 Processor -- start 2024-09-17T02:45:11.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.546+0000 7ffaaccc3640 1 -- start start 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaaccc3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa90003a90 0x7ffaa80ab130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaaccc3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffaa81f6520 0x7ffaa81f6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaaccc3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa815a4e0 0x7ffaa815a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaaccc3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa815ec50 con 0x7ffaa815a4e0 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaaccc3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa815ead0 con 0x7ffa90003a90 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaaccc3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa815edd0 con 0x7ffaa81f6520 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa815a4e0 0x7ffaa815a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa815a4e0 0x7ffaa815a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37668/0 (socket says 172.21.15.131:37668) 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa6d9d640 1 -- 172.21.15.131:0/1383411734 learned_addr learned my addr 172.21.15.131:0/1383411734 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa659c640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa90003a90 0x7ffaa80ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa5d9b640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffaa81f6520 0x7ffaa81f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:11.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa6d9d640 1 -- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffaa81f6520 msgr2=0x7ffaa81f6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa6d9d640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffaa81f6520 0x7ffaa81f6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa6d9d640 1 -- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa90003a90 msgr2=0x7ffaa80ab130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa6d9d640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa90003a90 0x7ffaa80ab130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa6d9d640 1 -- 172.21.15.131:0/1383411734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa815ae30 con 0x7ffaa815a4e0 2024-09-17T02:45:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa659c640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa90003a90 0x7ffaa80ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:45:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaa6d9d640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa815a4e0 0x7ffaa815a8f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8071a50 tx=0x7ffaa000dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffa96ffd640 1 -- 172.21.15.131:0/1383411734 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0015070 con 0x7ffaa815a4e0 2024-09-17T02:45:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffa96ffd640 1 -- 172.21.15.131:0/1383411734 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaa0003db0 con 0x7ffaa815a4e0 2024-09-17T02:45:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1383411734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa80aa4f0 con 0x7ffaa815a4e0 2024-09-17T02:45:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffa96ffd640 1 -- 172.21.15.131:0/1383411734 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0005330 con 0x7ffaa815a4e0 2024-09-17T02:45:11.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1383411734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa80a9730 con 0x7ffaa815a4e0 2024-09-17T02:45:11.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1383411734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa815f330 con 0x7ffaa815a4e0 2024-09-17T02:45:11.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.550+0000 7ffa96ffd640 1 -- 172.21.15.131:0/1383411734 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffaa0018070 con 0x7ffaa815a4e0 2024-09-17T02:45:11.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.554+0000 7ffa96ffd640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffa88073200 0x7ffa88075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:11.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.554+0000 7ffa96ffd640 1 -- 172.21.15.131:0/1383411734 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7ffaa0095be0 con 0x7ffaa815a4e0 2024-09-17T02:45:11.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.554+0000 7ffaa659c640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffa88073200 0x7ffa88075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:11.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.554+0000 7ffaa659c640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffa88073200 0x7ffa88075710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffa980016c0 tx=0x7ffa980a8040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:11.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.558+0000 7ffa96ffd640 1 -- 172.21.15.131:0/1383411734 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ffaa815ae30 con 0x7ffaa815a4e0 2024-09-17T02:45:11.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:11.774+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1383411734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7ffaa806e9d0 con 0x7ffaa815a4e0 2024-09-17T02:45:11.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffa96ffd640 1 -- 172.21.15.131:0/1383411734 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v216) ==== 169+0+0 (secure 0 0 0) 0x7ffaa0060e90 con 0x7ffaa815a4e0 2024-09-17T02:45:12.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffa88073200 msgr2=0x7ffa88075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffa88073200 0x7ffa88075710 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffa980016c0 tx=0x7ffa980a8040 comp rx=0 tx=0).stop 2024-09-17T02:45:12.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1383411734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa815a4e0 msgr2=0x7ffaa815a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa815a4e0 0x7ffaa815a8f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8071a50 tx=0x7ffaa000dca0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1383411734 shutdown_connections 2024-09-17T02:45:12.002 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffa88073200 0x7ffa88075710 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffaa815a4e0 0x7ffaa815a8f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffaa81f6520 0x7ffaa81f6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 --2- 172.21.15.131:0/1383411734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa90003a90 0x7ffaa80ab130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1383411734 >> 172.21.15.131:0/1383411734 conn(0x7ffaa806eea0 msgr2=0x7ffaa815b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:12.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1383411734 shutdown_connections 2024-09-17T02:45:12.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.002+0000 7ffaaccc3640 1 -- 172.21.15.131:0/1383411734 wait complete. 2024-09-17T02:45:12.027 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-09-17T02:45:12.027 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-09-17T02:45:12.027 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-17T02:45:12.028 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:45:12.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 -- 172.21.15.131:0/3647838345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec155040 msgr2=0x7facec0ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 --2- 172.21.15.131:0/3647838345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec155040 0x7facec0ca880 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7facec0cc6f0 tx=0x7facdc01c830 comp rx=0 tx=0).stop 2024-09-17T02:45:12.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 -- 172.21.15.131:0/3647838345 shutdown_connections 2024-09-17T02:45:12.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 --2- 172.21.15.131:0/3647838345 >> v2:172.21.15.189:3300/0 conn(0x7facec073800 0x7facec073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 --2- 172.21.15.131:0/3647838345 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec155040 0x7facec0ca880 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 -- 172.21.15.131:0/3647838345 >> 172.21.15.131:0/3647838345 conn(0x7facec06eea0 msgr2=0x7facec06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:12.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 -- 172.21.15.131:0/3647838345 shutdown_connections 2024-09-17T02:45:12.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 -- 172.21.15.131:0/3647838345 wait complete. 2024-09-17T02:45:12.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 Processor -- start 2024-09-17T02:45:12.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 -- start start 2024-09-17T02:45:12.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec06dea0 0x7facec0b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec073800 0x7facec1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec155040 0x7facec152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7facec168530 con 0x7facec06dea0 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7facec1683b0 con 0x7facec073800 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facf08f0640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7facec1686b0 con 0x7facec155040 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facea59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec06dea0 0x7facec0b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facea59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec06dea0 0x7facec0b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37672/0 (socket says 172.21.15.131:37672) 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.190+0000 7facea59c640 1 -- 172.21.15.131:0/989475647 learned_addr learned my addr 172.21.15.131:0/989475647 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7face9d9b640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec073800 0x7facec1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facead9d640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec155040 0x7facec152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facea59c640 1 -- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec155040 msgr2=0x7facec152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facea59c640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec155040 0x7facec152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facea59c640 1 -- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec073800 msgr2=0x7facec1529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facea59c640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec073800 0x7facec1529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facea59c640 1 -- 172.21.15.131:0/989475647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facec0b0f00 con 0x7facec06dea0 2024-09-17T02:45:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facead9d640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec155040 0x7facec152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7face9d9b640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec073800 0x7facec1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facea59c640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec06dea0 0x7facec0b09c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7facdc0b0040 tx=0x7facdc0a5660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facdaffd640 1 -- 172.21.15.131:0/989475647 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc0b2040 con 0x7facec06dea0 2024-09-17T02:45:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facdaffd640 1 -- 172.21.15.131:0/989475647 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facdc0042c0 con 0x7facec06dea0 2024-09-17T02:45:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facdaffd640 1 -- 172.21.15.131:0/989475647 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc005030 con 0x7facec06dea0 2024-09-17T02:45:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facf08f0640 1 -- 172.21.15.131:0/989475647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facec0c9790 con 0x7facec06dea0 2024-09-17T02:45:12.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facf08f0640 1 -- 172.21.15.131:0/989475647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facec1521e0 con 0x7facec06dea0 2024-09-17T02:45:12.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facf08f0640 1 -- 172.21.15.131:0/989475647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facec073c10 con 0x7facec06dea0 2024-09-17T02:45:12.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facdaffd640 1 -- 172.21.15.131:0/989475647 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7facdc0051f0 con 0x7facec06dea0 2024-09-17T02:45:12.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facdaffd640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faccc073200 0x7faccc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7facdaffd640 1 -- 172.21.15.131:0/989475647 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7facdc12e2c0 con 0x7facec06dea0 2024-09-17T02:45:12.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.194+0000 7face9d9b640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faccc073200 0x7faccc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.198+0000 7face9d9b640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faccc073200 0x7faccc075710 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7facec0b1340 tx=0x7face0008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:12.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.198+0000 7facdaffd640 1 -- 172.21.15.131:0/989475647 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7facdc0164b0 con 0x7facec06dea0 2024-09-17T02:45:12.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.378+0000 7facf08f0640 1 -- 172.21.15.131:0/989475647 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facec06fa20 con 0x7facec06dea0 2024-09-17T02:45:12.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facdaffd640 1 -- 172.21.15.131:0/989475647 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+13354 (secure 0 0 0) 0x7facdc0f9450 con 0x7facec06dea0 2024-09-17T02:45:12.379 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:12.379 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":71,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:45:10.982357+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.189:6835/730392436":"2024-09-18T02:45:07.966724+0000","172.21.15.189:6834/730392436":"2024-09-18T02:45:07.966724+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:45:12.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 -- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faccc073200 msgr2=0x7faccc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faccc073200 0x7faccc075710 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7facec0b1340 tx=0x7face0008040 comp rx=0 tx=0).stop 2024-09-17T02:45:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 -- 172.21.15.131:0/989475647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec06dea0 msgr2=0x7facec0b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec06dea0 0x7facec0b09c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7facdc0b0040 tx=0x7facdc0a5660 comp rx=0 tx=0).stop 2024-09-17T02:45:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 -- 172.21.15.131:0/989475647 shutdown_connections 2024-09-17T02:45:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faccc073200 0x7faccc075710 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7facec155040 0x7facec152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7facec073800 0x7facec1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 --2- 172.21.15.131:0/989475647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7facec06dea0 0x7facec0b09c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 -- 172.21.15.131:0/989475647 >> 172.21.15.131:0/989475647 conn(0x7facec06eea0 msgr2=0x7facec0ca2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 -- 172.21.15.131:0/989475647 shutdown_connections 2024-09-17T02:45:12.382 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.382+0000 7facf08f0640 1 -- 172.21.15.131:0/989475647 wait complete. 2024-09-17T02:45:12.402 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T02:40:41.163614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T02:45:12.402 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T02:45:12.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.566+0000 7f95be1df640 1 -- 172.21.15.131:0/4205348031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b80cca70 msgr2=0x7f95b8167a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.566+0000 7f95be1df640 1 --2- 172.21.15.131:0/4205348031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b80cca70 0x7f95b8167a40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009840 tx=0x7f95a801c7f0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.566+0000 7f95be1df640 1 -- 172.21.15.131:0/4205348031 shutdown_connections 2024-09-17T02:45:12.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.566+0000 7f95be1df640 1 --2- 172.21.15.131:0/4205348031 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b80cca70 0x7f95b8167a40 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.566+0000 7f95be1df640 1 -- 172.21.15.131:0/4205348031 >> 172.21.15.131:0/4205348031 conn(0x7f95b80cf150 msgr2=0x7f95b80cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:12.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.566+0000 7f95be1df640 1 -- 172.21.15.131:0/4205348031 shutdown_connections 2024-09-17T02:45:12.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.566+0000 7f95be1df640 1 -- 172.21.15.131:0/4205348031 wait complete. 2024-09-17T02:45:12.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95be1df640 1 Processor -- start 2024-09-17T02:45:12.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95be1df640 1 -- start start 2024-09-17T02:45:12.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95be1df640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b80cca70 0x7f95b8155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95be1df640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95b8156bd0 0x7f95b815b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95be1df640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f95b8156190 0x7f95b806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95be1df640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f95b816ab30 con 0x7f95b80cca70 2024-09-17T02:45:12.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95be1df640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f95b816a9b0 con 0x7f95b8156bd0 2024-09-17T02:45:12.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95be1df640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f95b816acb0 con 0x7f95b8156190 2024-09-17T02:45:12.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b80cca70 0x7f95b8155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b80cca70 0x7f95b8155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37680/0 (socket says 172.21.15.131:37680) 2024-09-17T02:45:12.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b77fe640 1 -- 172.21.15.131:0/703363330 learned_addr learned my addr 172.21.15.131:0/703363330 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:12.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b6ffd640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95b8156bd0 0x7f95b815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b7fff640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f95b8156190 0x7f95b806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b77fe640 1 -- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f95b8156190 msgr2=0x7f95b806dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b77fe640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f95b8156190 0x7f95b806dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b77fe640 1 -- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95b8156bd0 msgr2=0x7f95b815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b77fe640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95b8156bd0 0x7f95b815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b77fe640 1 -- 172.21.15.131:0/703363330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b815c020 con 0x7f95b80cca70 2024-09-17T02:45:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b6ffd640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95b8156bd0 0x7f95b815b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95b77fe640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b80cca70 0x7f95b8155b00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f95b000c7c0 tx=0x7f95b000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95affff640 1 -- 172.21.15.131:0/703363330 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b0003b80 con 0x7f95b80cca70 2024-09-17T02:45:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95affff640 1 -- 172.21.15.131:0/703363330 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95b0003d20 con 0x7f95b80cca70 2024-09-17T02:45:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95be1df640 1 -- 172.21.15.131:0/703363330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b815fce0 con 0x7f95b80cca70 2024-09-17T02:45:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95affff640 1 -- 172.21.15.131:0/703363330 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b000aa90 con 0x7f95b80cca70 2024-09-17T02:45:12.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.570+0000 7f95be1df640 1 -- 172.21.15.131:0/703363330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b815b5c0 con 0x7f95b80cca70 2024-09-17T02:45:12.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.574+0000 7f95be1df640 1 -- 172.21.15.131:0/703363330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b80b41a0 con 0x7f95b80cca70 2024-09-17T02:45:12.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.574+0000 7f95affff640 1 -- 172.21.15.131:0/703363330 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f95b000ac30 con 0x7f95b80cca70 2024-09-17T02:45:12.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.574+0000 7f95affff640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9598073200 0x7f9598075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.574+0000 7f95affff640 1 -- 172.21.15.131:0/703363330 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7f95b0094930 con 0x7f95b80cca70 2024-09-17T02:45:12.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.574+0000 7f95b6ffd640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9598073200 0x7f9598075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.574+0000 7f95b6ffd640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9598073200 0x7f9598075710 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f95b8071600 tx=0x7f95a0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:12.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.578+0000 7f95affff640 1 -- 172.21.15.131:0/703363330 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f95b005fbc0 con 0x7f95b80cca70 2024-09-17T02:45:12.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 -- 172.21.15.131:0/703363330 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f95b815cd10 con 0x7f95b80cca70 2024-09-17T02:45:12.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95affff640 1 -- 172.21.15.131:0/703363330 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) ==== 93+0+10 (secure 0 0 0) 0x7f95b005f4c0 con 0x7f95b80cca70 2024-09-17T02:45:12.747 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-09-17T02:45:12.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 -- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9598073200 msgr2=0x7f9598075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9598073200 0x7f9598075710 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f95b8071600 tx=0x7f95a0009040 comp rx=0 tx=0).stop 2024-09-17T02:45:12.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 -- 172.21.15.131:0/703363330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b80cca70 msgr2=0x7f95b8155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b80cca70 0x7f95b8155b00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f95b000c7c0 tx=0x7f95b000cc80 comp rx=0 tx=0).stop 2024-09-17T02:45:12.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 -- 172.21.15.131:0/703363330 shutdown_connections 2024-09-17T02:45:12.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9598073200 0x7f9598075710 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f95b8156190 0x7f95b806dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f95b8156bd0 0x7f95b815b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 --2- 172.21.15.131:0/703363330 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f95b80cca70 0x7f95b8155b00 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 -- 172.21.15.131:0/703363330 >> 172.21.15.131:0/703363330 conn(0x7f95b80cf150 msgr2=0x7f95b815dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:12.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 -- 172.21.15.131:0/703363330 shutdown_connections 2024-09-17T02:45:12.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.750+0000 7f95be1df640 1 -- 172.21.15.131:0/703363330 wait complete. 2024-09-17T02:45:12.771 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-09-17T02:45:12.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.926+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/4223571730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d00cab60 msgr2=0x7fe3d00caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.926+0000 7fe3d6fa3640 1 --2- 172.21.15.131:0/4223571730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d00cab60 0x7fe3d00caf70 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fe3d00afaf0 tx=0x7fe3c801cb70 comp rx=0 tx=0).stop 2024-09-17T02:45:12.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.926+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/4223571730 shutdown_connections 2024-09-17T02:45:12.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.926+0000 7fe3d6fa3640 1 --2- 172.21.15.131:0/4223571730 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d00cab60 0x7fe3d00caf70 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.926+0000 7fe3d6fa3640 1 --2- 172.21.15.131:0/4223571730 >> v2:172.21.15.189:3300/0 conn(0x7fe3d006dea0 0x7fe3d006e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.926+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/4223571730 >> 172.21.15.131:0/4223571730 conn(0x7fe3d006eea0 msgr2=0x7fe3d006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:12.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.926+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/4223571730 shutdown_connections 2024-09-17T02:45:12.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.926+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/4223571730 wait complete. 2024-09-17T02:45:12.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d6fa3640 1 Processor -- start 2024-09-17T02:45:12.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d6fa3640 1 -- start start 2024-09-17T02:45:12.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d6fa3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d006dea0 0x7fe3d014e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d6fa3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe3d00cab60 0x7fe3d00b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d6fa3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d0155240 0x7fe3d00b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d6fa3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d00cbda0 con 0x7fe3d0155240 2024-09-17T02:45:12.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d6fa3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d00cbc20 con 0x7fe3d006dea0 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d6fa3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d00cbf20 con 0x7fe3d00cab60 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d5540640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d0155240 0x7fe3d00b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d5540640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d0155240 0x7fe3d00b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37700/0 (socket says 172.21.15.131:37700) 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d5540640 1 -- 172.21.15.131:0/3163602750 learned_addr learned my addr 172.21.15.131:0/3163602750 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d4d3f640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d006dea0 0x7fe3d014e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3cffff640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe3d00cab60 0x7fe3d00b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d5540640 1 -- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe3d00cab60 msgr2=0x7fe3d00b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d5540640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe3d00cab60 0x7fe3d00b3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d5540640 1 -- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d006dea0 msgr2=0x7fe3d014e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d5540640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d006dea0 0x7fe3d014e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d5540640 1 -- 172.21.15.131:0/3163602750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d01507e0 con 0x7fe3d0155240 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3cffff640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe3d00cab60 0x7fe3d00b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:12.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d4d3f640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d006dea0 0x7fe3d014e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:12.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d5540640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d0155240 0x7fe3d00b3f50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009bd0 tx=0x7fe3c8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:12.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3cd7fa640 1 -- 172.21.15.131:0/3163602750 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80bb070 con 0x7fe3d0155240 2024-09-17T02:45:12.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3cd7fa640 1 -- 172.21.15.131:0/3163602750 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3c8004a50 con 0x7fe3d0155240 2024-09-17T02:45:12.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/3163602750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d00c5530 con 0x7fe3d0155240 2024-09-17T02:45:12.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3cd7fa640 1 -- 172.21.15.131:0/3163602750 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80b2040 con 0x7fe3d0155240 2024-09-17T02:45:12.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.930+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/3163602750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d01f8ab0 con 0x7fe3d0155240 2024-09-17T02:45:12.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.934+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/3163602750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d006e2b0 con 0x7fe3d0155240 2024-09-17T02:45:12.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.934+0000 7fe3cd7fa640 1 -- 172.21.15.131:0/3163602750 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe3c80be070 con 0x7fe3d0155240 2024-09-17T02:45:12.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.934+0000 7fe3cd7fa640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe3b0073200 0x7fe3b0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:12.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.934+0000 7fe3cd7fa640 1 -- 172.21.15.131:0/3163602750 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7fe3c812dd90 con 0x7fe3d0155240 2024-09-17T02:45:12.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.934+0000 7fe3d4d3f640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe3b0073200 0x7fe3b0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:12.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.934+0000 7fe3d4d3f640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe3b0073200 0x7fe3b0075710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800a850 tx=0x7fe3b8008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:12.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:12.938+0000 7fe3cd7fa640 1 -- 172.21.15.131:0/3163602750 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe3d006e7f0 con 0x7fe3d0155240 2024-09-17T02:45:13.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:13.162+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/3163602750 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7fe3d0150570 con 0x7fe3d0155240 2024-09-17T02:45:14.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.014+0000 7fe3cd7fa640 1 -- 172.21.15.131:0/3163602750 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v218) ==== 169+0+0 (secure 0 0 0) 0x7fe3c80f8fb0 con 0x7fe3d0155240 2024-09-17T02:45:14.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe3b0073200 msgr2=0x7fe3b0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe3b0073200 0x7fe3b0075710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe3b800a850 tx=0x7fe3b8008040 comp rx=0 tx=0).stop 2024-09-17T02:45:14.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/3163602750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d0155240 msgr2=0x7fe3d00b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d0155240 0x7fe3d00b3f50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8009bd0 tx=0x7fe3c8005b00 comp rx=0 tx=0).stop 2024-09-17T02:45:14.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/3163602750 shutdown_connections 2024-09-17T02:45:14.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe3b0073200 0x7fe3b0075710 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe3d0155240 0x7fe3d00b3f50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe3d00cab60 0x7fe3d00b3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 --2- 172.21.15.131:0/3163602750 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe3d006dea0 0x7fe3d014e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/3163602750 >> 172.21.15.131:0/3163602750 conn(0x7fe3d006eea0 msgr2=0x7fe3d00733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:14.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/3163602750 shutdown_connections 2024-09-17T02:45:14.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.018+0000 7fe3d6fa3640 1 -- 172.21.15.131:0/3163602750 wait complete. 2024-09-17T02:45:14.039 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T02:45:14.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.186+0000 7fcc5a45d640 1 -- 172.21.15.131:0/2351679064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc3c004810 msgr2=0x7fcc3c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.186+0000 7fcc5a45d640 1 --2- 172.21.15.131:0/2351679064 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc3c004810 0x7fcc3c004c20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800c2e0 tx=0x7fcc4801f6d0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.186+0000 7fcc589fa640 1 -- 172.21.15.131:0/2351679064 reap_dead start 2024-09-17T02:45:14.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.186+0000 7fcc5a45d640 1 -- 172.21.15.131:0/2351679064 shutdown_connections 2024-09-17T02:45:14.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.186+0000 7fcc5a45d640 1 -- 172.21.15.131:0/2351679064 >> 172.21.15.131:0/2351679064 conn(0x7fcc5406eea0 msgr2=0x7fcc5406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:14.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.186+0000 7fcc5a45d640 1 -- 172.21.15.131:0/2351679064 shutdown_connections 2024-09-17T02:45:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.186+0000 7fcc5a45d640 1 -- 172.21.15.131:0/2351679064 wait complete. 2024-09-17T02:45:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc5a45d640 1 Processor -- start 2024-09-17T02:45:14.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc5a45d640 1 -- start start 2024-09-17T02:45:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc5a45d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc3c004810 0x7fcc541f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc5a45d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc541f5450 0x7fcc540c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc5a45d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc541f4750 0x7fcc541f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc5a45d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5406fe90 con 0x7fcc541f4750 2024-09-17T02:45:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc5a45d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5406fd10 con 0x7fcc541f5450 2024-09-17T02:45:14.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc5a45d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcc54070010 con 0x7fcc3c004810 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc589fa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc541f4750 0x7fcc541f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc589fa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc541f4750 0x7fcc541f4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37720/0 (socket says 172.21.15.131:37720) 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc589fa640 1 -- 172.21.15.131:0/1210744649 learned_addr learned my addr 172.21.15.131:0/1210744649 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc53fff640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc3c004810 0x7fcc541f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc537fe640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc541f5450 0x7fcc540c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc589fa640 1 -- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc3c004810 msgr2=0x7fcc541f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc589fa640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc3c004810 0x7fcc541f1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc589fa640 1 -- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc541f5450 msgr2=0x7fcc540c2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc589fa640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc541f5450 0x7fcc540c2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc589fa640 1 -- 172.21.15.131:0/1210744649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc541f50a0 con 0x7fcc541f4750 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc53fff640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc3c004810 0x7fcc541f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc537fe640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc541f5450 0x7fcc540c2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc589fa640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc541f4750 0x7fcc541f4b60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcc540ca620 tx=0x7fcc4c00d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:14.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc50ff9640 1 -- 172.21.15.131:0/1210744649 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c004d30 con 0x7fcc541f4750 2024-09-17T02:45:14.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc50ff9640 1 -- 172.21.15.131:0/1210744649 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc4c003980 con 0x7fcc541f4750 2024-09-17T02:45:14.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc5a45d640 1 -- 172.21.15.131:0/1210744649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc540c7110 con 0x7fcc541f4750 2024-09-17T02:45:14.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc50ff9640 1 -- 172.21.15.131:0/1210744649 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c003db0 con 0x7fcc541f4750 2024-09-17T02:45:14.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.190+0000 7fcc5a45d640 1 -- 172.21.15.131:0/1210744649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc540c8dd0 con 0x7fcc541f4750 2024-09-17T02:45:14.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.194+0000 7fcc5a45d640 1 -- 172.21.15.131:0/1210744649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc54073430 con 0x7fcc541f4750 2024-09-17T02:45:14.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.194+0000 7fcc50ff9640 1 -- 172.21.15.131:0/1210744649 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcc4c01e020 con 0x7fcc541f4750 2024-09-17T02:45:14.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.194+0000 7fcc50ff9640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc40073200 0x7fcc40075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:14.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.194+0000 7fcc50ff9640 1 -- 172.21.15.131:0/1210744649 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7fcc4c095ad0 con 0x7fcc541f4750 2024-09-17T02:45:14.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.194+0000 7fcc53fff640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc40073200 0x7fcc40075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:14.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.194+0000 7fcc53fff640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc40073200 0x7fcc40075710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009b40 tx=0x7fcc44005770 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:14.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.198+0000 7fcc50ff9640 1 -- 172.21.15.131:0/1210744649 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcc541f50a0 con 0x7fcc541f4750 2024-09-17T02:45:14.270 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:45:14.272 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:45:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.358+0000 7fcc5a45d640 1 -- 172.21.15.131:0/1210744649 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fcc5406e9d0 con 0x7fcc541f4750 2024-09-17T02:45:14.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.358+0000 7fcc50ff9640 1 -- 172.21.15.131:0/1210744649 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+2294 (secure 0 0 0) 0x7fcc4c060cd0 con 0x7fcc541f4750 2024-09-17T02:45:14.358 INFO:teuthology.orchestra.run.smithi131.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T02:45:14.358 INFO:teuthology.orchestra.run.smithi131.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T02:45:14.358 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced client_mount_timeout 30 2024-09-17T02:45:14.358 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_client 20/20 2024-09-17T02:45:14.358 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_mgr 20/20 2024-09-17T02:45:14.358 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_ms 1/1 2024-09-17T02:45:14.358 INFO:teuthology.orchestra.run.smithi131.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T02:45:14.358 INFO:teuthology.orchestra.run.smithi131.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T02:45:14.358 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3855.598857 2024-09-17T02:45:14.358 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3598.912856 2024-09-17T02:45:14.358 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3977.024117 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3823.557728 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4643.710284 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4452.598518 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6659.946233 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4539.778076 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds 20/20 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_ms 2/2 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_frag true 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_scatterstat true 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_verify_scatter true 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T02:45:14.359 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T02:45:14.360 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T02:45:14.360 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced client_mount_timeout 600 2024-09-17T02:45:14.360 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_client 20/20 2024-09-17T02:45:14.360 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_ms 1/1 2024-09-17T02:45:14.360 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T02:45:14.360 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T02:45:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 -- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc40073200 msgr2=0x7fcc40075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc40073200 0x7fcc40075710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcc44009b40 tx=0x7fcc44005770 comp rx=0 tx=0).stop 2024-09-17T02:45:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 -- 172.21.15.131:0/1210744649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc541f4750 msgr2=0x7fcc541f4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc541f4750 0x7fcc541f4b60 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcc540ca620 tx=0x7fcc4c00d7b0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 -- 172.21.15.131:0/1210744649 shutdown_connections 2024-09-17T02:45:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc40073200 0x7fcc40075710 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc541f4750 0x7fcc541f4b60 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc541f5450 0x7fcc540c2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 --2- 172.21.15.131:0/1210744649 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc3c004810 0x7fcc541f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 -- 172.21.15.131:0/1210744649 >> 172.21.15.131:0/1210744649 conn(0x7fcc5406eea0 msgr2=0x7fcc540cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 -- 172.21.15.131:0/1210744649 shutdown_connections 2024-09-17T02:45:14.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.362+0000 7fcc5a45d640 1 -- 172.21.15.131:0/1210744649 wait complete. 2024-09-17T02:45:14.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T02:45:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.526+0000 7fe8775a5640 1 -- 172.21.15.131:0/1321987813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe870155240 msgr2=0x7fe870073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.526+0000 7fe8775a5640 1 --2- 172.21.15.131:0/1321987813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe870155240 0x7fe870073430 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe86c009840 tx=0x7fe86c01c7f0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.526+0000 7fe8775a5640 1 -- 172.21.15.131:0/1321987813 shutdown_connections 2024-09-17T02:45:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.526+0000 7fe8775a5640 1 --2- 172.21.15.131:0/1321987813 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe870073970 0x7fe87006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.526+0000 7fe8775a5640 1 --2- 172.21.15.131:0/1321987813 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe870155240 0x7fe870073430 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.526+0000 7fe8775a5640 1 -- 172.21.15.131:0/1321987813 >> 172.21.15.131:0/1321987813 conn(0x7fe87006eea0 msgr2=0x7fe87006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:14.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.526+0000 7fe8775a5640 1 -- 172.21.15.131:0/1321987813 shutdown_connections 2024-09-17T02:45:14.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.526+0000 7fe8775a5640 1 -- 172.21.15.131:0/1321987813 wait complete. 2024-09-17T02:45:14.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.526+0000 7fe8775a5640 1 Processor -- start 2024-09-17T02:45:14.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.526+0000 7fe8775a5640 1 -- start start 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe8775a5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe87006e3e0 0x7fe8701523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe8775a5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe870073970 0x7fe8700b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe8775a5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe870155240 0x7fe8700b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe8775a5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe870167f00 con 0x7fe87006e3e0 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe8775a5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe870167d80 con 0x7fe870073970 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe8775a5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe870168080 con 0x7fe870155240 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe875341640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe87006e3e0 0x7fe8701523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe875341640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe87006e3e0 0x7fe8701523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37750/0 (socket says 172.21.15.131:37750) 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe875341640 1 -- 172.21.15.131:0/2191604043 learned_addr learned my addr 172.21.15.131:0/2191604043 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe874b40640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe870073970 0x7fe8700b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe875341640 1 -- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe870155240 msgr2=0x7fe8700b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe875341640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe870155240 0x7fe8700b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe875341640 1 -- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe870073970 msgr2=0x7fe8700b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe875341640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe870073970 0x7fe8700b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe875341640 1 -- 172.21.15.131:0/2191604043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8701fc3b0 con 0x7fe87006e3e0 2024-09-17T02:45:14.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe875341640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe87006e3e0 0x7fe8701523d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe86c01ccd0 tx=0x7fe86c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:14.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe861ffb640 1 -- 172.21.15.131:0/2191604043 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0b2040 con 0x7fe87006e3e0 2024-09-17T02:45:14.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe861ffb640 1 -- 172.21.15.131:0/2191604043 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe86c004130 con 0x7fe87006e3e0 2024-09-17T02:45:14.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe8775a5640 1 -- 172.21.15.131:0/2191604043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8701fc690 con 0x7fe87006e3e0 2024-09-17T02:45:14.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe861ffb640 1 -- 172.21.15.131:0/2191604043 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0044e0 con 0x7fe87006e3e0 2024-09-17T02:45:14.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe8775a5640 1 -- 172.21.15.131:0/2191604043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8701fcc50 con 0x7fe87006e3e0 2024-09-17T02:45:14.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.530+0000 7fe8775a5640 1 -- 172.21.15.131:0/2191604043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe87006dea0 con 0x7fe87006e3e0 2024-09-17T02:45:14.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.534+0000 7fe861ffb640 1 -- 172.21.15.131:0/2191604043 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe86c0bd030 con 0x7fe87006e3e0 2024-09-17T02:45:14.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.534+0000 7fe861ffb640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8540733d0 0x7fe8540758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:14.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.534+0000 7fe861ffb640 1 -- 172.21.15.131:0/2191604043 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7fe86c12dc00 con 0x7fe87006e3e0 2024-09-17T02:45:14.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.534+0000 7fe874b40640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8540733d0 0x7fe8540758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:14.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.534+0000 7fe874b40640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8540733d0 0x7fe8540758e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe864005c50 tx=0x7fe864005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:14.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.538+0000 7fe861ffb640 1 -- 172.21.15.131:0/2191604043 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe86c134080 con 0x7fe87006e3e0 2024-09-17T02:45:14.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.694+0000 7fe8775a5640 1 -- 172.21.15.131:0/2191604043 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe87006e9d0 con 0x7fe87006e3e0 2024-09-17T02:45:14.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.702+0000 7fe861ffb640 1 -- 172.21.15.131:0/2191604043 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7fe86c0c4060 con 0x7fe87006e3e0 2024-09-17T02:45:14.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 -- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8540733d0 msgr2=0x7fe8540758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8540733d0 0x7fe8540758e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe864005c50 tx=0x7fe864005be0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 -- 172.21.15.131:0/2191604043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe87006e3e0 msgr2=0x7fe8701523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe87006e3e0 0x7fe8701523d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe86c01ccd0 tx=0x7fe86c005db0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 -- 172.21.15.131:0/2191604043 shutdown_connections 2024-09-17T02:45:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8540733d0 0x7fe8540758e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe870155240 0x7fe8700b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe870073970 0x7fe8700b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 --2- 172.21.15.131:0/2191604043 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe87006e3e0 0x7fe8701523d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 -- 172.21.15.131:0/2191604043 >> 172.21.15.131:0/2191604043 conn(0x7fe87006eea0 msgr2=0x7fe87006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 -- 172.21.15.131:0/2191604043 shutdown_connections 2024-09-17T02:45:14.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:14.706+0000 7fe8775a5640 1 -- 172.21.15.131:0/2191604043 wait complete. 2024-09-17T02:45:14.723 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:45:14.723 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:45:14.796 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:45:14.846 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:45:14.846 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:45:14.846 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:14.851 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:45:14.851 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:45:14.852 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:14.857 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:45:14.857 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:45:14.857 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:14.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:45:14.899 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:45:14.899 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:14.904 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:45:14.904 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T02:45:14.905 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:15.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.094+0000 7f192724e640 1 -- 172.21.15.131:0/4190662197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f192015f9a0 msgr2=0x7f19200d2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:15.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.094+0000 7f192724e640 1 --2- 172.21.15.131:0/4190662197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f192015f9a0 0x7f19200d2460 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f19200d42c0 tx=0x7f190801cd80 comp rx=0 tx=0).stop 2024-09-17T02:45:15.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.094+0000 7f192724e640 1 -- 172.21.15.131:0/4190662197 shutdown_connections 2024-09-17T02:45:15.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.094+0000 7f192724e640 1 --2- 172.21.15.131:0/4190662197 >> v2:172.21.15.189:3300/0 conn(0x7f19200d29a0 0x7f19200d2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.094+0000 7f192724e640 1 --2- 172.21.15.131:0/4190662197 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f192015f9a0 0x7f19200d2460 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.094+0000 7f192724e640 1 -- 172.21.15.131:0/4190662197 >> 172.21.15.131:0/4190662197 conn(0x7f19200d1b10 msgr2=0x7f19200d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:15.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.094+0000 7f192724e640 1 -- 172.21.15.131:0/4190662197 shutdown_connections 2024-09-17T02:45:15.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.094+0000 7f192724e640 1 -- 172.21.15.131:0/4190662197 wait complete. 2024-09-17T02:45:15.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f192724e640 1 Processor -- start 2024-09-17T02:45:15.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f192724e640 1 -- start start 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f192724e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19200cf710 0x7f1920152e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f192724e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19200d29a0 0x7f19200cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f192724e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f192015f9a0 0x7f1920155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f192724e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f192016c140 con 0x7f19200cf710 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f192724e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f192016bfc0 con 0x7f192015f9a0 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f192724e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f192016c2c0 con 0x7f19200d29a0 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f1924fea640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19200cf710 0x7f1920152e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f1924fea640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19200cf710 0x7f1920152e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37776/0 (socket says 172.21.15.131:37776) 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f1924fea640 1 -- 172.21.15.131:0/2712427042 learned_addr learned my addr 172.21.15.131:0/2712427042 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f19257eb640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f192015f9a0 0x7f1920155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f191ffff640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19200d29a0 0x7f19200cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:15.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f1924fea640 1 -- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19200d29a0 msgr2=0x7f19200cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f1924fea640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19200d29a0 0x7f19200cc1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f1924fea640 1 -- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f192015f9a0 msgr2=0x7f1920155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f1924fea640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f192015f9a0 0x7f1920155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f1924fea640 1 -- 172.21.15.131:0/2712427042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1920154550 con 0x7f19200cf710 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f19257eb640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f192015f9a0 0x7f1920155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f1924fea640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19200cf710 0x7f1920152e30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f191800c3f0 tx=0x7f191800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f191d7fa640 1 -- 172.21.15.131:0/2712427042 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918017070 con 0x7f19200cf710 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f191d7fa640 1 -- 172.21.15.131:0/2712427042 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19180053c0 con 0x7f19200cf710 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f192724e640 1 -- 172.21.15.131:0/2712427042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1920154830 con 0x7f19200cf710 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f191d7fa640 1 -- 172.21.15.131:0/2712427042 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1918003860 con 0x7f19200cf710 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f192724e640 1 -- 172.21.15.131:0/2712427042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1920154df0 con 0x7f19200cf710 2024-09-17T02:45:15.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.098+0000 7f192724e640 1 -- 172.21.15.131:0/2712427042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19200d2db0 con 0x7f19200cf710 2024-09-17T02:45:15.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.102+0000 7f191d7fa640 1 -- 172.21.15.131:0/2712427042 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f191801a070 con 0x7f19200cf710 2024-09-17T02:45:15.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.102+0000 7f191d7fa640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1900073230 0x7f1900075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:15.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.102+0000 7f191d7fa640 1 -- 172.21.15.131:0/2712427042 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7f1918093410 con 0x7f19200cf710 2024-09-17T02:45:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.102+0000 7f191ffff640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1900073230 0x7f1900075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:15.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.102+0000 7f191ffff640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1900073230 0x7f1900075740 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f19080af000 tx=0x7f190801b5c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:15.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.106+0000 7f191d7fa640 1 -- 172.21.15.131:0/2712427042 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1918001090 con 0x7f19200cf710 2024-09-17T02:45:15.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.298+0000 7f192724e640 1 -- 172.21.15.131:0/2712427042 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19200b7fa0 con 0x7f19200cf710 2024-09-17T02:45:15.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.298+0000 7f191d7fa640 1 -- 172.21.15.131:0/2712427042 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+3920 (secure 0 0 0) 0x7f191805e5a0 con 0x7f19200cf710 2024-09-17T02:45:15.297 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:15.297 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":40,"btime":"2024-09-17T02:45:08:969975+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5329,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/3554877374","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":3554877374},{"type":"v1","addr":"172.21.15.189:6837","nonce":3554877374}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/546739435","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":546739435},{"type":"v1","addr":"172.21.15.189:6835","nonce":546739435}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":5856,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1670590113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1670590113},{"type":"v1","addr":"172.21.15.131:6834","nonce":1670590113}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5859,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/1046159039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1046159039},{"type":"v1","addr":"172.21.15.131:6835","nonce":1046159039}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[]} 2024-09-17T02:45:15.297 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 40 2024-09-17T02:45:15.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 -- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1900073230 msgr2=0x7f1900075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:15.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1900073230 0x7f1900075740 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f19080af000 tx=0x7f190801b5c0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 -- 172.21.15.131:0/2712427042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19200cf710 msgr2=0x7f1920152e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:15.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19200cf710 0x7f1920152e30 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f191800c3f0 tx=0x7f191800c8b0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 -- 172.21.15.131:0/2712427042 shutdown_connections 2024-09-17T02:45:15.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1900073230 0x7f1900075740 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f192015f9a0 0x7f1920155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f19200d29a0 0x7f19200cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 --2- 172.21.15.131:0/2712427042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f19200cf710 0x7f1920152e30 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 -- 172.21.15.131:0/2712427042 >> 172.21.15.131:0/2712427042 conn(0x7f19200d1b10 msgr2=0x7f192015f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:15.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 -- 172.21.15.131:0/2712427042 shutdown_connections 2024-09-17T02:45:15.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.302+0000 7f192724e640 1 -- 172.21.15.131:0/2712427042 wait complete. 2024-09-17T02:45:15.319 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T02:45:15.319 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T02:45:15.319 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T02:45:15.319 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:45:15.320 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T02:45:15.320 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T02:45:15.320 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T02:45:15.320 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:45:15.320 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T02:45:15.320 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T02:45:15.320 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T02:45:15.321 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:45:15.321 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T02:45:15.321 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T02:45:15.321 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T02:45:15.321 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:45:15.335 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T02:45:15.335 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T02:45:15.336 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T02:45:15.336 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T02:45:15.339 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T02:45:15.339 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T02:45:15.344 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T02:45:15.344 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T02:45:15.379 INFO:tasks.ceph.mds.c:Started 2024-09-17T02:45:15.380 DEBUG:teuthology.parallel:result is None 2024-09-17T02:45:15.380 INFO:tasks.ceph.mds.a:Started 2024-09-17T02:45:15.381 DEBUG:teuthology.parallel:result is None 2024-09-17T02:45:15.388 INFO:tasks.ceph.mds.b:Started 2024-09-17T02:45:15.389 DEBUG:teuthology.parallel:result is None 2024-09-17T02:45:15.390 INFO:tasks.ceph.mds.d:Started 2024-09-17T02:45:15.390 DEBUG:teuthology.parallel:result is None 2024-09-17T02:45:15.390 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T02:45:15.464 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:45:15.464 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:45:15.488 INFO:tasks.ceph.mds.d.smithi189.stdout:starting mds.d at 2024-09-17T02:45:15.488 INFO:tasks.ceph.mds.b.smithi189.stdout:starting mds.b at 2024-09-17T02:45:15.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.538+0000 7f2662ff0640 1 -- 172.21.15.131:0/1763634375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c152550 msgr2=0x7f265c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:15.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.538+0000 7f2662ff0640 1 --2- 172.21.15.131:0/1763634375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c152550 0x7f265c06e840 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f265c06e530 tx=0x7f265001c820 comp rx=0 tx=0).stop 2024-09-17T02:45:15.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.538+0000 7f2662ff0640 1 -- 172.21.15.131:0/1763634375 shutdown_connections 2024-09-17T02:45:15.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.538+0000 7f2662ff0640 1 --2- 172.21.15.131:0/1763634375 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f265c15ba70 0x7f265c071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.538+0000 7f2662ff0640 1 --2- 172.21.15.131:0/1763634375 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c152550 0x7f265c06e840 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.538+0000 7f2662ff0640 1 -- 172.21.15.131:0/1763634375 >> 172.21.15.131:0/1763634375 conn(0x7f265c0d1130 msgr2=0x7f265c0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:15.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.538+0000 7f2662ff0640 1 -- 172.21.15.131:0/1763634375 shutdown_connections 2024-09-17T02:45:15.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.538+0000 7f2662ff0640 1 -- 172.21.15.131:0/1763634375 wait complete. 2024-09-17T02:45:15.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.538+0000 7f2662ff0640 1 Processor -- start 2024-09-17T02:45:15.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.538+0000 7f2662ff0640 1 -- start start 2024-09-17T02:45:15.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2662ff0640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f265c152550 0x7f265c0cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2662ff0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f265c15ba70 0x7f265c14eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2662ff0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c15be80 0x7f265c06d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2662ff0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f265c169210 con 0x7f265c15be80 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2662ff0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f265c169090 con 0x7f265c15ba70 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2662ff0640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f265c169390 con 0x7f265c152550 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f266158d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c15be80 0x7f265c06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f266158d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c15be80 0x7f265c06d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37822/0 (socket says 172.21.15.131:37822) 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f266158d640 1 -- 172.21.15.131:0/1384913096 learned_addr learned my addr 172.21.15.131:0/1384913096 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2660d8c640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f265c152550 0x7f265c0cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f265bfff640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f265c15ba70 0x7f265c14eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f266158d640 1 -- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f265c152550 msgr2=0x7f265c0cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f266158d640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f265c152550 0x7f265c0cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f266158d640 1 -- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f265c15ba70 msgr2=0x7f265c14eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:15.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f266158d640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f265c15ba70 0x7f265c14eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:15.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f266158d640 1 -- 172.21.15.131:0/1384913096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265c1fdb80 con 0x7f265c15be80 2024-09-17T02:45:15.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f265bfff640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f265c15ba70 0x7f265c14eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:15.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f266158d640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c15be80 0x7f265c06d060 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f265400ba10 tx=0x7f265400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:15.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f26597fa640 1 -- 172.21.15.131:0/1384913096 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2654019070 con 0x7f265c15be80 2024-09-17T02:45:15.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f26597fa640 1 -- 172.21.15.131:0/1384913096 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f265400cc90 con 0x7f265c15be80 2024-09-17T02:45:15.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2662ff0640 1 -- 172.21.15.131:0/1384913096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265c1fd000 con 0x7f265c15be80 2024-09-17T02:45:15.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f26597fa640 1 -- 172.21.15.131:0/1384913096 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26540146a0 con 0x7f265c15be80 2024-09-17T02:45:15.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2662ff0640 1 -- 172.21.15.131:0/1384913096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265c1fd5c0 con 0x7f265c15be80 2024-09-17T02:45:15.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2662ff0640 1 -- 172.21.15.131:0/1384913096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265c06e840 con 0x7f265c15be80 2024-09-17T02:45:15.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f26597fa640 1 -- 172.21.15.131:0/1384913096 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f265401c070 con 0x7f265c15be80 2024-09-17T02:45:15.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f26597fa640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2640073200 0x7f2640075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:15.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f26597fa640 1 -- 172.21.15.131:0/1384913096 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5335+0+0 (secure 0 0 0) 0x7f2654093d50 con 0x7f265c15be80 2024-09-17T02:45:15.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2660d8c640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2640073200 0x7f2640075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:15.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.542+0000 7f2660d8c640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2640073200 0x7f2640075710 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26500af000 tx=0x7f265001b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:15.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.546+0000 7f26597fa640 1 -- 172.21.15.131:0/1384913096 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2654097030 con 0x7f265c15be80 2024-09-17T02:45:15.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:15.722+0000 7f2662ff0640 1 -- 172.21.15.131:0/1384913096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f265c071ff0 con 0x7f265c15be80 2024-09-17T02:45:16.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.042+0000 7f26597fa640 1 -- 172.21.15.131:0/1384913096 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) ==== 97+0+0 (secure 0 0 0) 0x7f265405ef70 con 0x7f265c15be80 2024-09-17T02:45:16.040 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-09-17T02:45:16.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 -- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2640073200 msgr2=0x7f2640075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2640073200 0x7f2640075710 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f26500af000 tx=0x7f265001b060 comp rx=0 tx=0).stop 2024-09-17T02:45:16.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 -- 172.21.15.131:0/1384913096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c15be80 msgr2=0x7f265c06d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c15be80 0x7f265c06d060 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f265400ba10 tx=0x7f265400bed0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 -- 172.21.15.131:0/1384913096 shutdown_connections 2024-09-17T02:45:16.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2640073200 0x7f2640075710 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f265c15be80 0x7f265c06d060 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f265c15ba70 0x7f265c14eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 --2- 172.21.15.131:0/1384913096 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f265c152550 0x7f265c0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 -- 172.21.15.131:0/1384913096 >> 172.21.15.131:0/1384913096 conn(0x7f265c0d1130 msgr2=0x7f265c151a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:16.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 -- 172.21.15.131:0/1384913096 shutdown_connections 2024-09-17T02:45:16.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.046+0000 7f2662ff0640 1 -- 172.21.15.131:0/1384913096 wait complete. 2024-09-17T02:45:16.072 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T02:45:16.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.218+0000 7faf933f2640 1 -- 172.21.15.131:0/3105105774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf78004d50 msgr2=0x7faf78005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.218+0000 7faf933f2640 1 --2- 172.21.15.131:0/3105105774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf78004d50 0x7faf78005160 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7faf84009eb0 tx=0x7faf8401d5b0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.218+0000 7faf9198f640 1 -- 172.21.15.131:0/3105105774 reap_dead start 2024-09-17T02:45:16.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.218+0000 7faf933f2640 1 -- 172.21.15.131:0/3105105774 shutdown_connections 2024-09-17T02:45:16.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.218+0000 7faf933f2640 1 -- 172.21.15.131:0/3105105774 >> 172.21.15.131:0/3105105774 conn(0x7faf8c0d1b10 msgr2=0x7faf8c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:16.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.218+0000 7faf933f2640 1 -- 172.21.15.131:0/3105105774 shutdown_connections 2024-09-17T02:45:16.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.218+0000 7faf933f2640 1 -- 172.21.15.131:0/3105105774 wait complete. 2024-09-17T02:45:16.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf933f2640 1 Processor -- start 2024-09-17T02:45:16.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf933f2640 1 -- start start 2024-09-17T02:45:16.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf933f2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf78004d50 0x7faf8c1e5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf933f2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf8c0cde40 0x7faf8c1e72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf933f2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf8c0ce250 0x7faf8c1e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf933f2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c06d810 con 0x7faf8c0cde40 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf933f2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf8c06d690 con 0x7faf8c0ce250 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf933f2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7faf8c0d2db0 con 0x7faf78004d50 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf9098d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf8c0cde40 0x7faf8c1e72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf9098d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf8c0cde40 0x7faf8c1e72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37844/0 (socket says 172.21.15.131:37844) 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf9098d640 1 -- 172.21.15.131:0/2699118094 learned_addr learned my addr 172.21.15.131:0/2699118094 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf9118e640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf78004d50 0x7faf8c1e5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf9198f640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf8c0ce250 0x7faf8c1e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf9098d640 1 -- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf78004d50 msgr2=0x7faf8c1e5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf9098d640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf78004d50 0x7faf8c1e5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf9098d640 1 -- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf8c0ce250 msgr2=0x7faf8c1e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf9098d640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf8c0ce250 0x7faf8c1e77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf9098d640 1 -- 172.21.15.131:0/2699118094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf8c1e6200 con 0x7faf8c0cde40 2024-09-17T02:45:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf9098d640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf8c0cde40 0x7faf8c1e72a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7faf88010960 tx=0x7faf88010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf81ffb640 1 -- 172.21.15.131:0/2699118094 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8800eda0 con 0x7faf8c0cde40 2024-09-17T02:45:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf81ffb640 1 -- 172.21.15.131:0/2699118094 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf88005020 con 0x7faf8c0cde40 2024-09-17T02:45:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf933f2640 1 -- 172.21.15.131:0/2699118094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf8c1e64e0 con 0x7faf8c0cde40 2024-09-17T02:45:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf81ffb640 1 -- 172.21.15.131:0/2699118094 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88005450 con 0x7faf8c0cde40 2024-09-17T02:45:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.222+0000 7faf933f2640 1 -- 172.21.15.131:0/2699118094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf8c1e6aa0 con 0x7faf8c0cde40 2024-09-17T02:45:16.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.226+0000 7faf933f2640 1 -- 172.21.15.131:0/2699118094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf8c0d2460 con 0x7faf8c0cde40 2024-09-17T02:45:16.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.226+0000 7faf81ffb640 1 -- 172.21.15.131:0/2699118094 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faf880047b0 con 0x7faf8c0cde40 2024-09-17T02:45:16.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.226+0000 7faf81ffb640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf70073200 0x7faf70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.226+0000 7faf81ffb640 1 -- 172.21.15.131:0/2699118094 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5249+0+0 (secure 0 0 0) 0x7faf88095310 con 0x7faf8c0cde40 2024-09-17T02:45:16.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.226+0000 7faf9118e640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf70073200 0x7faf70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.226+0000 7faf9118e640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf70073200 0x7faf70075710 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0018f0 tx=0x7faf7c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:16.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.230+0000 7faf81ffb640 1 -- 172.21.15.131:0/2699118094 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf880605d0 con 0x7faf8c0cde40 2024-09-17T02:45:16.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.442+0000 7faf933f2640 1 -- 172.21.15.131:0/2699118094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7faf8c1e5f90 con 0x7faf8c0cde40 2024-09-17T02:45:16.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf81ffb640 1 -- 172.21.15.131:0/2699118094 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) ==== 80+0+5304 (secure 0 0 0) 0x7faf8805fed0 con 0x7faf8c0cde40 2024-09-17T02:45:16.443 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:16.443 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm4U1cHhAATu86VrwT2vMjL/cW2SICMw==", 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.444 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmVpUSKxAA4l5zGHXg0VRasiyBI/j4ig==", 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmwHuCJRAAlOj+za8ZmqMI55PsShjKDw==", 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:45:16.445 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm9f+9MhAAS8z9jZFYon3nrmu9/4JMPg==", 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.446 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD16+hmV9xkBxAAT+IDf2J4awJfqYcx2xDoNA==", 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD46+hmOhJsGBAAdlA4BSdOPk8BySUTfo1ghg==", 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:45:16.447 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD76+hm3mSoKBAAQPoGFy816yWZQW6vPIK6bA==", 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.448 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD+6+hmpZaPNxAAp0EV6bn5SugDMwA4RnnJwg==", 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAF7OhmgDfVAxAALACK3bk+PK3UhG20b498HQ==", 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:45:16.449 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:45:16.450 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.450 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.450 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.450 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.450 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-09-17T02:45:16.450 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAI7Ohm6w44FBAA90GkelUAbiNjAwn+6IJWcg==", 2024-09-17T02:45:16.450 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.450 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:45:16.450 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:45:16.450 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.450 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAL7Ohm6S6TIxAArISJW//rmEONG2bC/sfkXA==", 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAO7OhmICwTMxAArAr7DhcO8A0FDSu7VEBehw==", 2024-09-17T02:45:16.451 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA==", 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T02:45:16.452 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmduVoBRAAwr50jSXw6sCyo+5t+3iT7A==", 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-09-17T02:45:16.453 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmBBCDABAAMw9581QyaOxJk5fs35o+ng==", 2024-09-17T02:45:16.454 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.454 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:45:16.454 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:45:16.454 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:45:16.454 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:45:16.454 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.454 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.454 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.454 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-09-17T02:45:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmgznRCRAA1Z8d32ADoSG+7RHtdPGmTQ==", 2024-09-17T02:45:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:45:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:45:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:45:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:45:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-09-17T02:45:16.455 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmpUfvABAAzclxwjR4E+UxvtkbwVqDXw==", 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmnWmJGBAA6fzewrm8ELqUAz/MczTOGA==", 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:45:16.456 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmrTq3ExAAuR8+2bItlpHqqFzlvPxASA==", 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-09-17T02:45:16.457 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:45:16.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 -- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf70073200 msgr2=0x7faf70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf70073200 0x7faf70075710 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0018f0 tx=0x7faf7c009040 comp rx=0 tx=0).stop 2024-09-17T02:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 -- 172.21.15.131:0/2699118094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf8c0cde40 msgr2=0x7faf8c1e72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf8c0cde40 0x7faf8c1e72a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7faf88010960 tx=0x7faf88010e20 comp rx=0 tx=0).stop 2024-09-17T02:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 -- 172.21.15.131:0/2699118094 shutdown_connections 2024-09-17T02:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf70073200 0x7faf70075710 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf8c0ce250 0x7faf8c1e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf8c0cde40 0x7faf8c1e72a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 --2- 172.21.15.131:0/2699118094 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf78004d50 0x7faf8c1e5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 -- 172.21.15.131:0/2699118094 >> 172.21.15.131:0/2699118094 conn(0x7faf8c0d1b10 msgr2=0x7faf8c159630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 -- 172.21.15.131:0/2699118094 shutdown_connections 2024-09-17T02:45:16.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.446+0000 7faf933f2640 1 -- 172.21.15.131:0/2699118094 wait complete. 2024-09-17T02:45:16.464 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T02:45:16.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.606+0000 7fedcca8b640 1 -- 172.21.15.131:0/403151198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc80c5e50 msgr2=0x7fedc80c6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.606+0000 7fedcca8b640 1 --2- 172.21.15.131:0/403151198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc80c5e50 0x7fedc80c6260 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fedc8072fc0 tx=0x7fedbc01c740 comp rx=0 tx=0).stop 2024-09-17T02:45:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.606+0000 7fedcca8b640 1 -- 172.21.15.131:0/403151198 shutdown_connections 2024-09-17T02:45:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.606+0000 7fedcca8b640 1 --2- 172.21.15.131:0/403151198 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc80c5e50 0x7fedc80c6260 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.606+0000 7fedcca8b640 1 --2- 172.21.15.131:0/403151198 >> v2:172.21.15.189:3300/0 conn(0x7fedc80ce230 0x7fedc80ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.606+0000 7fedcca8b640 1 -- 172.21.15.131:0/403151198 >> 172.21.15.131:0/403151198 conn(0x7fedc80734f0 msgr2=0x7fedc8073900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.606+0000 7fedcca8b640 1 -- 172.21.15.131:0/403151198 shutdown_connections 2024-09-17T02:45:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.606+0000 7fedcca8b640 1 -- 172.21.15.131:0/403151198 wait complete. 2024-09-17T02:45:16.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.606+0000 7fedcca8b640 1 Processor -- start 2024-09-17T02:45:16.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.606+0000 7fedcca8b640 1 -- start start 2024-09-17T02:45:16.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedcca8b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedc806dea0 0x7fedc80708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedcca8b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fedc80c5e50 0x7fedc8168ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedcca8b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc80ce230 0x7fedc806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedcca8b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fedc8169a50 con 0x7fedc80ce230 2024-09-17T02:45:16.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedcca8b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fedc80cc600 con 0x7fedc806dea0 2024-09-17T02:45:16.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedcca8b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fedc8169bd0 con 0x7fedc80c5e50 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedc6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc80ce230 0x7fedc806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedc6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc80ce230 0x7fedc806fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37848/0 (socket says 172.21.15.131:37848) 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedc6d9d640 1 -- 172.21.15.131:0/2844281336 learned_addr learned my addr 172.21.15.131:0/2844281336 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedc5d9b640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fedc80c5e50 0x7fedc8168ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedc659c640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedc806dea0 0x7fedc80708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedc6d9d640 1 -- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fedc80c5e50 msgr2=0x7fedc8168ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedc6d9d640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fedc80c5e50 0x7fedc8168ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedc6d9d640 1 -- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedc806dea0 msgr2=0x7fedc80708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedc6d9d640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedc806dea0 0x7fedc80708f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedc6d9d640 1 -- 172.21.15.131:0/2844281336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedc815e360 con 0x7fedc80ce230 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedc6d9d640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc80ce230 0x7fedc806fc90 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fedc000c3c0 tx=0x7fedc000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:16.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedb6ffd640 1 -- 172.21.15.131:0/2844281336 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0011790 con 0x7fedc80ce230 2024-09-17T02:45:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedcca8b640 1 -- 172.21.15.131:0/2844281336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedc81fc170 con 0x7fedc80ce230 2024-09-17T02:45:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedb6ffd640 1 -- 172.21.15.131:0/2844281336 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedc0017070 con 0x7fedc80ce230 2024-09-17T02:45:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedb6ffd640 1 -- 172.21.15.131:0/2844281336 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc0011ce0 con 0x7fedc80ce230 2024-09-17T02:45:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedcca8b640 1 -- 172.21.15.131:0/2844281336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedc81fc730 con 0x7fedc80ce230 2024-09-17T02:45:16.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedcca8b640 1 -- 172.21.15.131:0/2844281336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedc80c6260 con 0x7fedc80ce230 2024-09-17T02:45:16.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedb6ffd640 1 -- 172.21.15.131:0/2844281336 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fedc001c460 con 0x7fedc80ce230 2024-09-17T02:45:16.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedb6ffd640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feda8073200 0x7feda8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.610+0000 7fedb6ffd640 1 -- 172.21.15.131:0/2844281336 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5249+0+0 (secure 0 0 0) 0x7fedc0093ea0 con 0x7fedc80ce230 2024-09-17T02:45:16.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.614+0000 7fedc659c640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feda8073200 0x7feda8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.614+0000 7fedc659c640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feda8073200 0x7feda8075710 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fedb8005fd0 tx=0x7fedb8008230 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:16.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.618+0000 7fedb6ffd640 1 -- 172.21.15.131:0/2844281336 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fedc005f090 con 0x7fedc80ce230 2024-09-17T02:45:16.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.786+0000 7fedcca8b640 1 -- 172.21.15.131:0/2844281336 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fedc8071ca0 con 0x7fedc80ce230 2024-09-17T02:45:16.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.786+0000 7fedb6ffd640 1 -- 172.21.15.131:0/2844281336 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7fedc005e990 con 0x7fedc80ce230 2024-09-17T02:45:16.785 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:16.785 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:45:16.785 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:45:16.785 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-09-17T02:45:16.785 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-09-17T02:45:16.785 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:16.785 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:45:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 -- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feda8073200 msgr2=0x7feda8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feda8073200 0x7feda8075710 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fedb8005fd0 tx=0x7fedb8008230 comp rx=0 tx=0).stop 2024-09-17T02:45:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 -- 172.21.15.131:0/2844281336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc80ce230 msgr2=0x7fedc806fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc80ce230 0x7fedc806fc90 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fedc000c3c0 tx=0x7fedc000c7c0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 -- 172.21.15.131:0/2844281336 shutdown_connections 2024-09-17T02:45:16.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feda8073200 0x7feda8075710 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fedc80ce230 0x7fedc806fc90 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fedc80c5e50 0x7fedc8168ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 --2- 172.21.15.131:0/2844281336 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fedc806dea0 0x7fedc80708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 -- 172.21.15.131:0/2844281336 >> 172.21.15.131:0/2844281336 conn(0x7fedc80734f0 msgr2=0x7fedc8154160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 -- 172.21.15.131:0/2844281336 shutdown_connections 2024-09-17T02:45:16.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.790+0000 7fedcca8b640 1 -- 172.21.15.131:0/2844281336 wait complete. 2024-09-17T02:45:16.806 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T02:45:16.806 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.970+0000 7f18c6bda640 1 -- 172.21.15.131:0/936421641 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18c00b3760 msgr2=0x7f18c00cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.970+0000 7f18c6bda640 1 --2- 172.21.15.131:0/936421641 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18c00b3760 0x7f18c00cf5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009c80 tx=0x7f18b801cda0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.970+0000 7f18c6bda640 1 -- 172.21.15.131:0/936421641 shutdown_connections 2024-09-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.970+0000 7f18c6bda640 1 --2- 172.21.15.131:0/936421641 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18c00b3760 0x7f18c00cf5a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.970+0000 7f18c6bda640 1 --2- 172.21.15.131:0/936421641 >> v2:172.21.15.189:3300/0 conn(0x7f18c00b3350 0x7f18c00cf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.970+0000 7f18c6bda640 1 -- 172.21.15.131:0/936421641 >> 172.21.15.131:0/936421641 conn(0x7f18c00d42b0 msgr2=0x7f18c00d46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.970+0000 7f18c6bda640 1 -- 172.21.15.131:0/936421641 shutdown_connections 2024-09-17T02:45:16.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c6bda640 1 -- 172.21.15.131:0/936421641 wait complete. 2024-09-17T02:45:16.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c6bda640 1 Processor -- start 2024-09-17T02:45:16.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c6bda640 1 -- start start 2024-09-17T02:45:16.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c6bda640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c00b3350 0x7f18c014ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c6bda640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18c00b3760 0x7f18c00b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c6bda640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18c015e750 0x7f18c00b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c6bda640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f18c00b91a0 con 0x7f18c00b3350 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c6bda640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18c00b9020 con 0x7f18c015e750 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c6bda640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f18c00b9320 con 0x7f18c00b3760 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c4976640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c00b3350 0x7f18c014ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c4976640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c00b3350 0x7f18c014ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37860/0 (socket says 172.21.15.131:37860) 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c4976640 1 -- 172.21.15.131:0/2581605024 learned_addr learned my addr 172.21.15.131:0/2581605024 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18bffff640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18c00b3760 0x7f18c00b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c5177640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18c015e750 0x7f18c00b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c4976640 1 -- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18c00b3760 msgr2=0x7f18c00b6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c4976640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18c00b3760 0x7f18c00b6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c4976640 1 -- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18c015e750 msgr2=0x7f18c00b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c4976640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18c015e750 0x7f18c00b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c4976640 1 -- 172.21.15.131:0/2581605024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c01512a0 con 0x7f18c00b3350 2024-09-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18bffff640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18c00b3760 0x7f18c00b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.974+0000 7f18c4976640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c00b3350 0x7f18c014ce20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f18b000c3f0 tx=0x7f18b000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.978+0000 7f18bd7fa640 1 -- 172.21.15.131:0/2581605024 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b00052d0 con 0x7f18c00b3350 2024-09-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.978+0000 7f18bd7fa640 1 -- 172.21.15.131:0/2581605024 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18b0007020 con 0x7f18c00b3350 2024-09-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.978+0000 7f18c6bda640 1 -- 172.21.15.131:0/2581605024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c0151580 con 0x7f18c00b3350 2024-09-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.978+0000 7f18bd7fa640 1 -- 172.21.15.131:0/2581605024 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b0007470 con 0x7f18c00b3350 2024-09-17T02:45:16.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.978+0000 7f18c6bda640 1 -- 172.21.15.131:0/2581605024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c01522e0 con 0x7f18c00b3350 2024-09-17T02:45:16.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.978+0000 7f18c6bda640 1 -- 172.21.15.131:0/2581605024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c00b2c40 con 0x7f18c00b3350 2024-09-17T02:45:16.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.978+0000 7f18bd7fa640 1 -- 172.21.15.131:0/2581605024 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f18b00067e0 con 0x7f18c00b3350 2024-09-17T02:45:16.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.978+0000 7f18bd7fa640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18ac073230 0x7f18ac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:16.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.978+0000 7f18bd7fa640 1 -- 172.21.15.131:0/2581605024 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5249+0+0 (secure 0 0 0) 0x7f18b0093470 con 0x7f18c00b3350 2024-09-17T02:45:16.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.978+0000 7f18bffff640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18ac073230 0x7f18ac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:16.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.978+0000 7f18bffff640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18ac073230 0x7f18ac075740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f18c014e4b0 tx=0x7f18b40091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:16.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:16.982+0000 7f18bd7fa640 1 -- 172.21.15.131:0/2581605024 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f18b005e6e0 con 0x7f18c00b3350 2024-09-17T02:45:17.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:17.146+0000 7f18c6bda640 1 -- 172.21.15.131:0/2581605024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f18c015f1f0 con 0x7f18c00b3350 2024-09-17T02:45:18.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.046+0000 7f18bd7fa640 1 -- 172.21.15.131:0/2581605024 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7f18b0001090 con 0x7f18c00b3350 2024-09-17T02:45:18.045 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-09-17T02:45:18.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 -- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18ac073230 msgr2=0x7f18ac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:18.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18ac073230 0x7f18ac075740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f18c014e4b0 tx=0x7f18b40091e0 comp rx=0 tx=0).stop 2024-09-17T02:45:18.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 -- 172.21.15.131:0/2581605024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c00b3350 msgr2=0x7f18c014ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:18.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c00b3350 0x7f18c014ce20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f18b000c3f0 tx=0x7f18b000c8b0 comp rx=0 tx=0).stop 2024-09-17T02:45:18.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 -- 172.21.15.131:0/2581605024 shutdown_connections 2024-09-17T02:45:18.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18ac073230 0x7f18ac075740 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:18.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18c015e750 0x7f18c00b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:18.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18c00b3760 0x7f18c00b6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:18.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 --2- 172.21.15.131:0/2581605024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18c00b3350 0x7f18c014ce20 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:18.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 -- 172.21.15.131:0/2581605024 >> 172.21.15.131:0/2581605024 conn(0x7f18c00d42b0 msgr2=0x7f18c015f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:18.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 -- 172.21.15.131:0/2581605024 shutdown_connections 2024-09-17T02:45:18.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.050+0000 7f18c6bda640 1 -- 172.21.15.131:0/2581605024 wait complete. 2024-09-17T02:45:18.081 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T02:45:18.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.254+0000 7fdb07e7b640 1 -- 172.21.15.131:0/2172870278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaec004810 msgr2=0x7fdaec004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:18.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.254+0000 7fdb07e7b640 1 --2- 172.21.15.131:0/2172870278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaec004810 0x7fdaec004c20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009b30 tx=0x7fdaf001d230 comp rx=0 tx=0).stop 2024-09-17T02:45:18.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.254+0000 7fdb07e7b640 1 -- 172.21.15.131:0/2172870278 shutdown_connections 2024-09-17T02:45:18.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.254+0000 7fdb07e7b640 1 --2- 172.21.15.131:0/2172870278 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdaec004810 0x7fdaec004c20 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:18.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.254+0000 7fdb07e7b640 1 -- 172.21.15.131:0/2172870278 >> 172.21.15.131:0/2172870278 conn(0x7fdb000d0490 msgr2=0x7fdb0006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:18.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.254+0000 7fdb07e7b640 1 -- 172.21.15.131:0/2172870278 shutdown_connections 2024-09-17T02:45:18.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.254+0000 7fdb07e7b640 1 -- 172.21.15.131:0/2172870278 wait complete. 2024-09-17T02:45:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb07e7b640 1 Processor -- start 2024-09-17T02:45:18.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb07e7b640 1 -- start start 2024-09-17T02:45:18.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb07e7b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdaec004810 0x7fdb001e7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:18.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb07e7b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb001e8300 0x7fdb0015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:18.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb07e7b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb0015a5c0 0x7fdb00147350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:18.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb07e7b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdb000b3750 con 0x7fdb0015a5c0 2024-09-17T02:45:18.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb07e7b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00160da0 con 0x7fdaec004810 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb07e7b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdb000b38d0 con 0x7fdb001e8300 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb06418640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb0015a5c0 0x7fdb00147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb06418640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb0015a5c0 0x7fdb00147350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37876/0 (socket says 172.21.15.131:37876) 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb06418640 1 -- 172.21.15.131:0/4118287185 learned_addr learned my addr 172.21.15.131:0/4118287185 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb05c17640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdaec004810 0x7fdb001e7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb05416640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb001e8300 0x7fdb0015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb06418640 1 -- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb001e8300 msgr2=0x7fdb0015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb06418640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb001e8300 0x7fdb0015a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb06418640 1 -- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdaec004810 msgr2=0x7fdb001e7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb06418640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdaec004810 0x7fdb001e7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb06418640 1 -- 172.21.15.131:0/4118287185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0015a9d0 con 0x7fdb0015a5c0 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb05416640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb001e8300 0x7fdb0015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:18.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb06418640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb0015a5c0 0x7fdb00147350 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fdb000b4c80 tx=0x7fdafc0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:18.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdaf67fc640 1 -- 172.21.15.131:0/4118287185 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0107d0 con 0x7fdb0015a5c0 2024-09-17T02:45:18.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdaf67fc640 1 -- 172.21.15.131:0/4118287185 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdafc016070 con 0x7fdb0015a5c0 2024-09-17T02:45:18.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb07e7b640 1 -- 172.21.15.131:0/4118287185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb001e8ab0 con 0x7fdb0015a5c0 2024-09-17T02:45:18.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdaf67fc640 1 -- 172.21.15.131:0/4118287185 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc010ca0 con 0x7fdb0015a5c0 2024-09-17T02:45:18.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb07e7b640 1 -- 172.21.15.131:0/4118287185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb001e97d0 con 0x7fdb0015a5c0 2024-09-17T02:45:18.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.258+0000 7fdb07e7b640 1 -- 172.21.15.131:0/4118287185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb001e8820 con 0x7fdb0015a5c0 2024-09-17T02:45:18.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.262+0000 7fdaf67fc640 1 -- 172.21.15.131:0/4118287185 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdafc01b460 con 0x7fdb0015a5c0 2024-09-17T02:45:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.262+0000 7fdaf67fc640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 0x7fdae4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.262+0000 7fdaf67fc640 1 -- 172.21.15.131:0/4118287185 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5619+0+0 (secure 0 0 0) 0x7fdafc095490 con 0x7fdb0015a5c0 2024-09-17T02:45:18.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.262+0000 7fdb05c17640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 0x7fdae4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:18.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.262+0000 7fdb05c17640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 0x7fdae4075710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8001870 tx=0x7fdaf8001bc0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:18.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.266+0000 7fdaf67fc640 1 -- 172.21.15.131:0/4118287185 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdb0015a9d0 con 0x7fdb0015a5c0 2024-09-17T02:45:18.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:18.438+0000 7fdb07e7b640 1 -- 172.21.15.131:0/4118287185 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdb000b3e30 con 0x7fdb0015a5c0 2024-09-17T02:45:19.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.050+0000 7fdaf67fc640 1 -- 172.21.15.131:0/4118287185 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7fdafc0605e0 con 0x7fdb0015a5c0 2024-09-17T02:45:19.050 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-09-17T02:45:19.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.054+0000 7fdb07e7b640 1 -- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 msgr2=0x7fdae4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:19.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.054+0000 7fdb07e7b640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 0x7fdae4075710 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8001870 tx=0x7fdaf8001bc0 comp rx=0 tx=0).stop 2024-09-17T02:45:19.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.054+0000 7fdb07e7b640 1 -- 172.21.15.131:0/4118287185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb0015a5c0 msgr2=0x7fdb00147350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:19.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.054+0000 7fdb07e7b640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb0015a5c0 0x7fdb00147350 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fdb000b4c80 tx=0x7fdafc0019c0 comp rx=0 tx=0).stop 2024-09-17T02:45:19.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.054+0000 7fdb07e7b640 1 -- 172.21.15.131:0/4118287185 shutdown_connections 2024-09-17T02:45:19.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.054+0000 7fdb07e7b640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 0x7fdae4075710 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:19.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.054+0000 7fdb07e7b640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb0015a5c0 0x7fdb00147350 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:19.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.054+0000 7fdb07e7b640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb001e8300 0x7fdb0015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:19.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.054+0000 7fdb07e7b640 1 --2- 172.21.15.131:0/4118287185 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdaec004810 0x7fdb001e7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:19.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.054+0000 7fdb07e7b640 1 -- 172.21.15.131:0/4118287185 >> 172.21.15.131:0/4118287185 conn(0x7fdb000d0490 msgr2=0x7fdb000b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:19.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.058+0000 7fdb07e7b640 1 -- 172.21.15.131:0/4118287185 shutdown_connections 2024-09-17T02:45:19.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.058+0000 7fdb07e7b640 1 -- 172.21.15.131:0/4118287185 wait complete. 2024-09-17T02:45:19.092 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T02:45:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.238+0000 7f31f843b640 1 -- 172.21.15.131:0/1400095332 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f31f0155240 msgr2=0x7f31f0073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.238+0000 7f31f843b640 1 --2- 172.21.15.131:0/1400095332 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f31f0155240 0x7f31f0073430 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009900 tx=0x7f31e001c8b0 comp rx=0 tx=0).stop 2024-09-17T02:45:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.238+0000 7f31f843b640 1 -- 172.21.15.131:0/1400095332 shutdown_connections 2024-09-17T02:45:19.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.238+0000 7f31f843b640 1 --2- 172.21.15.131:0/1400095332 >> v2:172.21.15.189:3300/0 conn(0x7f31f0073970 0x7f31f006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.238+0000 7f31f843b640 1 --2- 172.21.15.131:0/1400095332 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f31f0155240 0x7f31f0073430 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.238+0000 7f31f843b640 1 -- 172.21.15.131:0/1400095332 >> 172.21.15.131:0/1400095332 conn(0x7f31f006eea0 msgr2=0x7f31f006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.238+0000 7f31f843b640 1 -- 172.21.15.131:0/1400095332 shutdown_connections 2024-09-17T02:45:19.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.238+0000 7f31f843b640 1 -- 172.21.15.131:0/1400095332 wait complete. 2024-09-17T02:45:19.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f843b640 1 Processor -- start 2024-09-17T02:45:19.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f843b640 1 -- start start 2024-09-17T02:45:19.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f843b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31f006e3e0 0x7f31f014fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:19.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f843b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f31f0073970 0x7f31f015c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:19.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f843b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f0155240 0x7f31f015c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f843b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f31f0169010 con 0x7f31f0155240 2024-09-17T02:45:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f843b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f31f0168e90 con 0x7f31f006e3e0 2024-09-17T02:45:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f843b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f31f0169190 con 0x7f31f0073970 2024-09-17T02:45:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f69d8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f0155240 0x7f31f015c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f69d8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f0155240 0x7f31f015c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37892/0 (socket says 172.21.15.131:37892) 2024-09-17T02:45:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f69d8640 1 -- 172.21.15.131:0/1849300186 learned_addr learned my addr 172.21.15.131:0/1849300186 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f59d6640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f31f0073970 0x7f31f015c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f61d7640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31f006e3e0 0x7f31f014fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:19.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f69d8640 1 -- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f31f0073970 msgr2=0x7f31f015c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f69d8640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f31f0073970 0x7f31f015c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f69d8640 1 -- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31f006e3e0 msgr2=0x7f31f014fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f69d8640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31f006e3e0 0x7f31f014fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f69d8640 1 -- 172.21.15.131:0/1849300186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f01501c0 con 0x7f31f0155240 2024-09-17T02:45:19.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31f69d8640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f0155240 0x7f31f015c8c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f31e800c6b0 tx=0x7f31e800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.242+0000 7f31e6ffd640 1 -- 172.21.15.131:0/1849300186 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8003b80 con 0x7f31f0155240 2024-09-17T02:45:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.246+0000 7f31e6ffd640 1 -- 172.21.15.131:0/1849300186 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31e8003d20 con 0x7f31f0155240 2024-09-17T02:45:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.246+0000 7f31e6ffd640 1 -- 172.21.15.131:0/1849300186 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8005330 con 0x7f31f0155240 2024-09-17T02:45:19.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.246+0000 7f31f843b640 1 -- 172.21.15.131:0/1849300186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f0155a40 con 0x7f31f0155240 2024-09-17T02:45:19.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.246+0000 7f31f843b640 1 -- 172.21.15.131:0/1849300186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f0156000 con 0x7f31f0155240 2024-09-17T02:45:19.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.246+0000 7f31f843b640 1 -- 172.21.15.131:0/1849300186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f006dea0 con 0x7f31f0155240 2024-09-17T02:45:19.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.246+0000 7f31e6ffd640 1 -- 172.21.15.131:0/1849300186 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f31e80054d0 con 0x7f31f0155240 2024-09-17T02:45:19.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.246+0000 7f31e6ffd640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f31d4073200 0x7f31d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:19.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.246+0000 7f31e6ffd640 1 -- 172.21.15.131:0/1849300186 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6001+0+0 (secure 0 0 0) 0x7f31e8094cf0 con 0x7f31f0155240 2024-09-17T02:45:19.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.246+0000 7f31f61d7640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f31d4073200 0x7f31d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:19.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.250+0000 7f31f61d7640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f31d4073200 0x7f31d4075710 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f31e00af000 tx=0x7f31e001b0f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:19.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.250+0000 7f31e6ffd640 1 -- 172.21.15.131:0/1849300186 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f31f006e7f0 con 0x7f31f0155240 2024-09-17T02:45:19.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:19.458+0000 7f31f843b640 1 -- 172.21.15.131:0/1849300186 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f31f006e9d0 con 0x7f31f0155240 2024-09-17T02:45:20.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.122+0000 7f31e6ffd640 1 -- 172.21.15.131:0/1849300186 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v43) ==== 174+0+0 (secure 0 0 0) 0x7f31e805fb40 con 0x7f31f0155240 2024-09-17T02:45:20.128 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 11 and data pool 12 2024-09-17T02:45:20.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 -- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f31d4073200 msgr2=0x7f31d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f31d4073200 0x7f31d4075710 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f31e00af000 tx=0x7f31e001b0f0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 -- 172.21.15.131:0/1849300186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f0155240 msgr2=0x7f31f015c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f0155240 0x7f31f015c8c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f31e800c6b0 tx=0x7f31e800cb70 comp rx=0 tx=0).stop 2024-09-17T02:45:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 -- 172.21.15.131:0/1849300186 shutdown_connections 2024-09-17T02:45:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f31d4073200 0x7f31d4075710 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f31f0155240 0x7f31f015c8c0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f31f0073970 0x7f31f015c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 --2- 172.21.15.131:0/1849300186 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f31f006e3e0 0x7f31f014fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 -- 172.21.15.131:0/1849300186 >> 172.21.15.131:0/1849300186 conn(0x7f31f006eea0 msgr2=0x7f31f006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:20.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 -- 172.21.15.131:0/1849300186 shutdown_connections 2024-09-17T02:45:20.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.146+0000 7f31f843b640 1 -- 172.21.15.131:0/1849300186 wait complete. 2024-09-17T02:45:20.172 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:45:20.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.322+0000 7fc5cad97640 1 -- 172.21.15.131:0/962936319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c406f4f0 msgr2=0x7fc5c415b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.322+0000 7fc5cad97640 1 --2- 172.21.15.131:0/962936319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c406f4f0 0x7fc5c415b870 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc5c406fb70 tx=0x7fc5ac01c8a0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.322+0000 7fc5cad97640 1 -- 172.21.15.131:0/962936319 shutdown_connections 2024-09-17T02:45:20.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.322+0000 7fc5cad97640 1 --2- 172.21.15.131:0/962936319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c406f4f0 0x7fc5c415b870 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.322+0000 7fc5cad97640 1 --2- 172.21.15.131:0/962936319 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5c415bdb0 0x7fc5c4151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.322+0000 7fc5cad97640 1 --2- 172.21.15.131:0/962936319 >> v2:172.21.15.189:3300/0 conn(0x7fc5c40cd470 0x7fc5c406efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.322+0000 7fc5cad97640 1 -- 172.21.15.131:0/962936319 >> 172.21.15.131:0/962936319 conn(0x7fc5c4073d10 msgr2=0x7fc5c4074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:20.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.322+0000 7fc5cad97640 1 -- 172.21.15.131:0/962936319 shutdown_connections 2024-09-17T02:45:20.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.322+0000 7fc5cad97640 1 -- 172.21.15.131:0/962936319 wait complete. 2024-09-17T02:45:20.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5cad97640 1 Processor -- start 2024-09-17T02:45:20.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5cad97640 1 -- start start 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5cad97640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5c406f4f0 0x7fc5c414b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5cad97640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5c40cd470 0x7fc5c406ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5cad97640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c415bdb0 0x7fc5c406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5cad97640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c40cc2e0 con 0x7fc5c415bdb0 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5cad97640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c40cc160 con 0x7fc5c406f4f0 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5cad97640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c415ee60 con 0x7fc5c40cd470 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c9334640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c415bdb0 0x7fc5c406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c9334640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c415bdb0 0x7fc5c406d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37908/0 (socket says 172.21.15.131:37908) 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c9334640 1 -- 172.21.15.131:0/3802863688 learned_addr learned my addr 172.21.15.131:0/3802863688 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c3fff640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5c40cd470 0x7fc5c406ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c8b33640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5c406f4f0 0x7fc5c414b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:20.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c9334640 1 -- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5c40cd470 msgr2=0x7fc5c406ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c9334640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5c40cd470 0x7fc5c406ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c9334640 1 -- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5c406f4f0 msgr2=0x7fc5c414b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c9334640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5c406f4f0 0x7fc5c414b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c9334640 1 -- 172.21.15.131:0/3802863688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c414bf70 con 0x7fc5c415bdb0 2024-09-17T02:45:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c3fff640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5c40cd470 0x7fc5c406ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:45:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c9334640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c415bdb0 0x7fc5c406d3d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00d610 tx=0x7fc5bc00dad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c17fa640 1 -- 172.21.15.131:0/3802863688 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc017070 con 0x7fc5c415bdb0 2024-09-17T02:45:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c17fa640 1 -- 172.21.15.131:0/3802863688 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5bc005310 con 0x7fc5c415bdb0 2024-09-17T02:45:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5cad97640 1 -- 172.21.15.131:0/3802863688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c414c3c0 con 0x7fc5c415bdb0 2024-09-17T02:45:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5c17fa640 1 -- 172.21.15.131:0/3802863688 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0047b0 con 0x7fc5c415bdb0 2024-09-17T02:45:20.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.326+0000 7fc5cad97640 1 -- 172.21.15.131:0/3802863688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c41f9cd0 con 0x7fc5c415bdb0 2024-09-17T02:45:20.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.330+0000 7fc5cad97640 1 -- 172.21.15.131:0/3802863688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c406efb0 con 0x7fc5c415bdb0 2024-09-17T02:45:20.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.330+0000 7fc5c17fa640 1 -- 172.21.15.131:0/3802863688 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc5bc004950 con 0x7fc5c415bdb0 2024-09-17T02:45:20.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.330+0000 7fc5c17fa640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5a4073200 0x7fc5a4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:20.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.330+0000 7fc5c17fa640 1 -- 172.21.15.131:0/3802863688 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7fc5bc094710 con 0x7fc5c415bdb0 2024-09-17T02:45:20.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.330+0000 7fc5c8b33640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5a4073200 0x7fc5a4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:20.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.330+0000 7fc5c8b33640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5a4073200 0x7fc5a4075710 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400a400 tx=0x7fc5b400d040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:20.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.334+0000 7fc5c17fa640 1 -- 172.21.15.131:0/3802863688 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc5bc05f030 con 0x7fc5c415bdb0 2024-09-17T02:45:20.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.502+0000 7fc5cad97640 1 -- 172.21.15.131:0/3802863688 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc5c40d0d80 con 0x7fc5c415bdb0 2024-09-17T02:45:20.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.502+0000 7fc5c17fa640 1 -- 172.21.15.131:0/3802863688 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+17006 (secure 0 0 0) 0x7fc5bc063a60 con 0x7fc5c415bdb0 2024-09-17T02:45:20.501 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:20.502 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":78,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:45:19.460836+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:45:17.161688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-17T02:45:18.453383+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:45:20.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 -- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5a4073200 msgr2=0x7fc5a4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5a4073200 0x7fc5a4075710 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400a400 tx=0x7fc5b400d040 comp rx=0 tx=0).stop 2024-09-17T02:45:20.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 -- 172.21.15.131:0/3802863688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c415bdb0 msgr2=0x7fc5c406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c415bdb0 0x7fc5c406d3d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00d610 tx=0x7fc5bc00dad0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 -- 172.21.15.131:0/3802863688 shutdown_connections 2024-09-17T02:45:20.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5a4073200 0x7fc5a4075710 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5c415bdb0 0x7fc5c406d3d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5c40cd470 0x7fc5c406ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 --2- 172.21.15.131:0/3802863688 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5c406f4f0 0x7fc5c414b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 -- 172.21.15.131:0/3802863688 >> 172.21.15.131:0/3802863688 conn(0x7fc5c4073d10 msgr2=0x7fc5c415dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 -- 172.21.15.131:0/3802863688 shutdown_connections 2024-09-17T02:45:20.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.506+0000 7fc5cad97640 1 -- 172.21.15.131:0/3802863688 wait complete. 2024-09-17T02:45:20.523 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:45:20.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.666+0000 7f4658018640 1 -- 172.21.15.131:0/4201177146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46500ced00 msgr2=0x7f465006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.666+0000 7f4658018640 1 --2- 172.21.15.131:0/4201177146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46500ced00 0x7f465006dea0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f46500cfe00 tx=0x7f464001c6e0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.666+0000 7f4658018640 1 -- 172.21.15.131:0/4201177146 shutdown_connections 2024-09-17T02:45:20.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.666+0000 7f4658018640 1 --2- 172.21.15.131:0/4201177146 >> v2:172.21.15.189:3300/0 conn(0x7f465015c5b0 0x7f46500cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.666+0000 7f4658018640 1 --2- 172.21.15.131:0/4201177146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f46500ced00 0x7f465006dea0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.666+0000 7f4658018640 1 -- 172.21.15.131:0/4201177146 >> 172.21.15.131:0/4201177146 conn(0x7f465006eea0 msgr2=0x7f465006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:20.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.666+0000 7f4658018640 1 -- 172.21.15.131:0/4201177146 shutdown_connections 2024-09-17T02:45:20.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.666+0000 7f4658018640 1 -- 172.21.15.131:0/4201177146 wait complete. 2024-09-17T02:45:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.666+0000 7f4658018640 1 Processor -- start 2024-09-17T02:45:20.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.666+0000 7f4658018640 1 -- start start 2024-09-17T02:45:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4658018640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465006e3e0 0x7f46501f9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4658018640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f46500ced00 0x7f46501f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4658018640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f465015c5b0 0x7f46501563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4658018640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f46501686c0 con 0x7f465006e3e0 2024-09-17T02:45:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4658018640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4650168540 con 0x7f465015c5b0 2024-09-17T02:45:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4658018640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4650168840 con 0x7f46500ced00 2024-09-17T02:45:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4655db4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465006e3e0 0x7f46501f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4655db4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465006e3e0 0x7f46501f9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37928/0 (socket says 172.21.15.131:37928) 2024-09-17T02:45:20.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4655db4640 1 -- 172.21.15.131:0/4106106834 learned_addr learned my addr 172.21.15.131:0/4106106834 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f46555b3640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f46500ced00 0x7f46501f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4655db4640 1 -- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f46500ced00 msgr2=0x7f46501f9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4655db4640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f46500ced00 0x7f46501f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4655db4640 1 -- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f465015c5b0 msgr2=0x7f46501563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4655db4640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f465015c5b0 0x7f46501563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4655db4640 1 -- 172.21.15.131:0/4106106834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4650158c80 con 0x7f465006e3e0 2024-09-17T02:45:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4655db4640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465006e3e0 0x7f46501f9700 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f464001cbc0 tx=0x7f46400a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f46467fc640 1 -- 172.21.15.131:0/4106106834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46400ac070 con 0x7f465006e3e0 2024-09-17T02:45:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f46467fc640 1 -- 172.21.15.131:0/4106106834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4640004320 con 0x7f465006e3e0 2024-09-17T02:45:20.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4658018640 1 -- 172.21.15.131:0/4106106834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4650158f60 con 0x7f465006e3e0 2024-09-17T02:45:20.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f46467fc640 1 -- 172.21.15.131:0/4106106834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640005320 con 0x7f465006e3e0 2024-09-17T02:45:20.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4658018640 1 -- 172.21.15.131:0/4106106834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4650159520 con 0x7f465006e3e0 2024-09-17T02:45:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f4658018640 1 -- 172.21.15.131:0/4106106834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4650159720 con 0x7f465006e3e0 2024-09-17T02:45:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f46467fc640 1 -- 172.21.15.131:0/4106106834 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f46400af070 con 0x7f465006e3e0 2024-09-17T02:45:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f46467fc640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4634073230 0x7f4634075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f46467fc640 1 -- 172.21.15.131:0/4106106834 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f464012e3e0 con 0x7f465006e3e0 2024-09-17T02:45:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.670+0000 7f46555b3640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4634073230 0x7f4634075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:20.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.674+0000 7f46555b3640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4634073230 0x7f4634075740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4650160bb0 tx=0x7f464c008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:20.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.678+0000 7f46467fc640 1 -- 172.21.15.131:0/4106106834 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f465006e7f0 con 0x7f465006e3e0 2024-09-17T02:45:20.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.846+0000 7f4658018640 1 -- 172.21.15.131:0/4106106834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f465006dea0 con 0x7f465006e3e0 2024-09-17T02:45:20.846 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.846+0000 7f46467fc640 1 -- 172.21.15.131:0/4106106834 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+17006 (secure 0 0 0) 0x7f46400f9280 con 0x7f465006e3e0 2024-09-17T02:45:20.846 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:20.846 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":78,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:45:19.460836+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:45:17.161688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-17T02:45:18.453383+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:45:20.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 -- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4634073230 msgr2=0x7f4634075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4634073230 0x7f4634075740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4650160bb0 tx=0x7f464c008040 comp rx=0 tx=0).stop 2024-09-17T02:45:20.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 -- 172.21.15.131:0/4106106834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465006e3e0 msgr2=0x7f46501f9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:20.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465006e3e0 0x7f46501f9700 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f464001cbc0 tx=0x7f46400a64e0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 -- 172.21.15.131:0/4106106834 shutdown_connections 2024-09-17T02:45:20.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4634073230 0x7f4634075740 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f465015c5b0 0x7f46501563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f46500ced00 0x7f46501f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 --2- 172.21.15.131:0/4106106834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f465006e3e0 0x7f46501f9700 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:20.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 -- 172.21.15.131:0/4106106834 >> 172.21.15.131:0/4106106834 conn(0x7f465006eea0 msgr2=0x7f46500b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:20.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 -- 172.21.15.131:0/4106106834 shutdown_connections 2024-09-17T02:45:20.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:20.850+0000 7f4658018640 1 -- 172.21.15.131:0/4106106834 wait complete. 2024-09-17T02:45:20.869 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T02:45:21.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.030+0000 7f45228f9640 1 -- 172.21.15.131:0/2690792647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f451c0c4ac0 msgr2=0x7f451c0c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:21.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.030+0000 7f45228f9640 1 --2- 172.21.15.131:0/2690792647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f451c0c4ac0 0x7f451c0c4ed0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f451c06a1c0 tx=0x7f451001c830 comp rx=0 tx=0).stop 2024-09-17T02:45:21.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.030+0000 7f45228f9640 1 -- 172.21.15.131:0/2690792647 shutdown_connections 2024-09-17T02:45:21.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.030+0000 7f45228f9640 1 --2- 172.21.15.131:0/2690792647 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f451c0c4ac0 0x7f451c0c4ed0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:21.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.030+0000 7f45228f9640 1 --2- 172.21.15.131:0/2690792647 >> v2:172.21.15.189:3300/0 conn(0x7f451c06e2c0 0x7f451c0c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:21.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.030+0000 7f45228f9640 1 -- 172.21.15.131:0/2690792647 >> 172.21.15.131:0/2690792647 conn(0x7f451c06eea0 msgr2=0x7f451c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:21.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.030+0000 7f45228f9640 1 -- 172.21.15.131:0/2690792647 shutdown_connections 2024-09-17T02:45:21.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.030+0000 7f45228f9640 1 -- 172.21.15.131:0/2690792647 wait complete. 2024-09-17T02:45:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.030+0000 7f45228f9640 1 Processor -- start 2024-09-17T02:45:21.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.030+0000 7f45228f9640 1 -- start start 2024-09-17T02:45:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f45228f9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f451c06e2c0 0x7f451c156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f45228f9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f451c0c4ac0 0x7f451c149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f45228f9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f451c0cdbf0 0x7f451c1fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f45228f9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f451c0cef20 con 0x7f451c06e2c0 2024-09-17T02:45:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f45228f9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f451c0cf0a0 con 0x7f451c0cdbf0 2024-09-17T02:45:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f45228f9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f451c0cf220 con 0x7f451c0c4ac0 2024-09-17T02:45:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f451bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f451c06e2c0 0x7f451c156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f451bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f451c06e2c0 0x7f451c156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37952/0 (socket says 172.21.15.131:37952) 2024-09-17T02:45:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f451bfff640 1 -- 172.21.15.131:0/3189551513 learned_addr learned my addr 172.21.15.131:0/3189551513 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f451bfff640 1 -- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f451c0c4ac0 msgr2=0x7f451c149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:21.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f451b7fe640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f451c0c4ac0 0x7f451c149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f4520e96640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f451c0cdbf0 0x7f451c1fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f451bfff640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f451c0c4ac0 0x7f451c149a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f451bfff640 1 -- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f451c0cdbf0 msgr2=0x7f451c1fa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f451bfff640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f451c0cdbf0 0x7f451c1fa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f451bfff640 1 -- 172.21.15.131:0/3189551513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c155270 con 0x7f451c06e2c0 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f4520e96640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f451c0cdbf0 0x7f451c1fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f451bfff640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f451c06e2c0 0x7f451c156310 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f450c00c9f0 tx=0x7f450c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f4518ff9640 1 -- 172.21.15.131:0/3189551513 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c007bb0 con 0x7f451c06e2c0 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f4518ff9640 1 -- 172.21.15.131:0/3189551513 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f450c007e40 con 0x7f451c06e2c0 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f45228f9640 1 -- 172.21.15.131:0/3189551513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451c155550 con 0x7f451c06e2c0 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f4518ff9640 1 -- 172.21.15.131:0/3189551513 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c00f690 con 0x7f451c06e2c0 2024-09-17T02:45:21.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f45228f9640 1 -- 172.21.15.131:0/3189551513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451c155b10 con 0x7f451c06e2c0 2024-09-17T02:45:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.034+0000 7f45228f9640 1 -- 172.21.15.131:0/3189551513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451c0c4580 con 0x7f451c06e2c0 2024-09-17T02:45:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.038+0000 7f4518ff9640 1 -- 172.21.15.131:0/3189551513 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f450c005020 con 0x7f451c06e2c0 2024-09-17T02:45:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.038+0000 7f4518ff9640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4508073200 0x7f4508075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.038+0000 7f4518ff9640 1 -- 172.21.15.131:0/3189551513 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6101+0+0 (secure 0 0 0) 0x7f450c094ec0 con 0x7f451c06e2c0 2024-09-17T02:45:21.035 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.038+0000 7f451b7fe640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4508073200 0x7f4508075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:21.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.038+0000 7f451b7fe640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4508073200 0x7f4508075710 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f45100af000 tx=0x7f451001b070 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:21.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.042+0000 7f4518ff9640 1 -- 172.21.15.131:0/3189551513 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f450c026080 con 0x7f451c06e2c0 2024-09-17T02:45:21.046 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-09-17T02:45:21.046+0000 7fbb9b47c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:45:21.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:21.234+0000 7f45228f9640 1 -- 172.21.15.131:0/3189551513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f451c073310 con 0x7f451c06e2c0 2024-09-17T02:45:22.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.138+0000 7f4518ff9640 1 -- 172.21.15.131:0/3189551513 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f450c014030 con 0x7f451c06e2c0 2024-09-17T02:45:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 -- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4508073200 msgr2=0x7f4508075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4508073200 0x7f4508075710 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f45100af000 tx=0x7f451001b070 comp rx=0 tx=0).stop 2024-09-17T02:45:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 -- 172.21.15.131:0/3189551513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f451c06e2c0 msgr2=0x7f451c156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f451c06e2c0 0x7f451c156310 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f450c00c9f0 tx=0x7f450c00ceb0 comp rx=0 tx=0).stop 2024-09-17T02:45:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 -- 172.21.15.131:0/3189551513 shutdown_connections 2024-09-17T02:45:22.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4508073200 0x7f4508075710 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f451c0cdbf0 0x7f451c1fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f451c0c4ac0 0x7f451c149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 --2- 172.21.15.131:0/3189551513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f451c06e2c0 0x7f451c156310 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 -- 172.21.15.131:0/3189551513 >> 172.21.15.131:0/3189551513 conn(0x7f451c06eea0 msgr2=0x7f451c1533f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 -- 172.21.15.131:0/3189551513 shutdown_connections 2024-09-17T02:45:22.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.142+0000 7f45228f9640 1 -- 172.21.15.131:0/3189551513 wait complete. 2024-09-17T02:45:22.164 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T02:45:22.164 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T02:45:22.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 -- 172.21.15.131:0/2402841827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f710015cfa0 msgr2=0x7f710015d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:22.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 --2- 172.21.15.131:0/2402841827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f710015cfa0 0x7f710015d3b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f70f8009bc0 tx=0x7f70f801cad0 comp rx=0 tx=0).stop 2024-09-17T02:45:22.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 -- 172.21.15.131:0/2402841827 shutdown_connections 2024-09-17T02:45:22.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 --2- 172.21.15.131:0/2402841827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f710015cfa0 0x7f710015d3b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:22.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 --2- 172.21.15.131:0/2402841827 >> v2:172.21.15.189:3300/0 conn(0x7f71000cce60 0x7f7100156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:22.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 -- 172.21.15.131:0/2402841827 >> 172.21.15.131:0/2402841827 conn(0x7f71000d2100 msgr2=0x7f71000d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:22.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 -- 172.21.15.131:0/2402841827 shutdown_connections 2024-09-17T02:45:22.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 -- 172.21.15.131:0/2402841827 wait complete. 2024-09-17T02:45:22.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 Processor -- start 2024-09-17T02:45:22.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 -- start start 2024-09-17T02:45:22.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71001566f0 0x7f71000c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:22.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f710015cfa0 0x7f71000b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:22.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71000b64c0 0x7f71001fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:22.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7100071630 con 0x7f71001566f0 2024-09-17T02:45:22.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f71000714b0 con 0x7f71000b64c0 2024-09-17T02:45:22.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f7106032640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f71000717b0 con 0x7f710015cfa0 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f70ff7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71001566f0 0x7f71000c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f70ff7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71001566f0 0x7f71000c47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44052/0 (socket says 172.21.15.131:44052) 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.330+0000 7f70ff7fe640 1 -- 172.21.15.131:0/1255848513 learned_addr learned my addr 172.21.15.131:0/1255848513 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70feffd640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f710015cfa0 0x7f71000b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70ff7fe640 1 -- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f710015cfa0 msgr2=0x7f71000b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70ff7fe640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f710015cfa0 0x7f71000b5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70ff7fe640 1 -- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71000b64c0 msgr2=0x7f71001fbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70ff7fe640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71000b64c0 0x7f71001fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70ff7fe640 1 -- 172.21.15.131:0/1255848513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71000b68d0 con 0x7f71001566f0 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70ff7fe640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71001566f0 0x7f71000c47e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f70f000c460 tx=0x7f70f000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70e7fff640 1 -- 172.21.15.131:0/1255848513 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0016020 con 0x7f71001566f0 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70e7fff640 1 -- 172.21.15.131:0/1255848513 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70f0012940 con 0x7f71001566f0 2024-09-17T02:45:22.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f7106032640 1 -- 172.21.15.131:0/1255848513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71000b6b50 con 0x7f71001566f0 2024-09-17T02:45:22.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70e7fff640 1 -- 172.21.15.131:0/1255848513 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0012d90 con 0x7f71001566f0 2024-09-17T02:45:22.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f7106032640 1 -- 172.21.15.131:0/1255848513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7100168120 con 0x7f71001566f0 2024-09-17T02:45:22.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f7106032640 1 -- 172.21.15.131:0/1255848513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f710015d3b0 con 0x7f71001566f0 2024-09-17T02:45:22.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70e7fff640 1 -- 172.21.15.131:0/1255848513 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f70f0019020 con 0x7f71001566f0 2024-09-17T02:45:22.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70e7fff640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f70ec073200 0x7f70ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:22.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70e7fff640 1 -- 172.21.15.131:0/1255848513 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f70f00942b0 con 0x7f71001566f0 2024-09-17T02:45:22.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.334+0000 7f70feffd640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f70ec073200 0x7f70ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:22.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.338+0000 7f70feffd640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f70ec073200 0x7f70ec075710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f70f40058d0 tx=0x7f70f4005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:22.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.338+0000 7f70e7fff640 1 -- 172.21.15.131:0/1255848513 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f70f0020020 con 0x7f71001566f0 2024-09-17T02:45:22.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:22.550+0000 7f7106032640 1 -- 172.21.15.131:0/1255848513 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f71000d2f10 con 0x7f71001566f0 2024-09-17T02:45:23.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.142+0000 7f70e7fff640 1 -- 172.21.15.131:0/1255848513 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f71000b68d0 con 0x7f71001566f0 2024-09-17T02:45:23.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 -- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f70ec073200 msgr2=0x7f70ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f70ec073200 0x7f70ec075710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f70f40058d0 tx=0x7f70f4005860 comp rx=0 tx=0).stop 2024-09-17T02:45:23.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 -- 172.21.15.131:0/1255848513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71001566f0 msgr2=0x7f71000c47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71001566f0 0x7f71000c47e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f70f000c460 tx=0x7f70f000c920 comp rx=0 tx=0).stop 2024-09-17T02:45:23.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 -- 172.21.15.131:0/1255848513 shutdown_connections 2024-09-17T02:45:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f70ec073200 0x7f70ec075710 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f71000b64c0 0x7f71001fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f710015cfa0 0x7f71000b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 --2- 172.21.15.131:0/1255848513 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f71001566f0 0x7f71000c47e0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 -- 172.21.15.131:0/1255848513 >> 172.21.15.131:0/1255848513 conn(0x7f71000d2100 msgr2=0x7f71000cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 -- 172.21.15.131:0/1255848513 shutdown_connections 2024-09-17T02:45:23.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.146+0000 7f7106032640 1 -- 172.21.15.131:0/1255848513 wait complete. 2024-09-17T02:45:23.168 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T02:45:23.168 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:23.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.326+0000 7f34f904e640 1 -- 172.21.15.131:0/110326799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34e0005260 msgr2=0x7f34e000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.326+0000 7f34f904e640 1 --2- 172.21.15.131:0/110326799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34e0005260 0x7f34e000b9d0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f34f40ba520 tx=0x7f34ec01cad0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.326+0000 7f34f904e640 1 -- 172.21.15.131:0/110326799 shutdown_connections 2024-09-17T02:45:23.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.326+0000 7f34f904e640 1 --2- 172.21.15.131:0/110326799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34e0005260 0x7f34e000b9d0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.326+0000 7f34f904e640 1 -- 172.21.15.131:0/110326799 >> 172.21.15.131:0/110326799 conn(0x7f34f40d1b10 msgr2=0x7f34f40d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:23.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.326+0000 7f34f904e640 1 -- 172.21.15.131:0/110326799 shutdown_connections 2024-09-17T02:45:23.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.326+0000 7f34f904e640 1 -- 172.21.15.131:0/110326799 wait complete. 2024-09-17T02:45:23.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f904e640 1 Processor -- start 2024-09-17T02:45:23.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f904e640 1 -- start start 2024-09-17T02:45:23.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f904e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f34e0005260 0x7f34f40ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:23.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f904e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f4152810 0x7f34f4150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f904e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34f4150d40 0x7f34f4151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f904e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f34f406d810 con 0x7f34f4150d40 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f904e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f34f406d690 con 0x7f34f4152810 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f904e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f34f40d2460 con 0x7f34e0005260 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f359e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34f4150d40 0x7f34f4151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f359e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34f4150d40 0x7f34f4151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44070/0 (socket says 172.21.15.131:44070) 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f359e640 1 -- 172.21.15.131:0/4137486228 learned_addr learned my addr 172.21.15.131:0/4137486228 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f2d9d640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f34e0005260 0x7f34f40ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34ea59c640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f4152810 0x7f34f4150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f359e640 1 -- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f34e0005260 msgr2=0x7f34f40ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f359e640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f34e0005260 0x7f34f40ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f359e640 1 -- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f4152810 msgr2=0x7f34f4150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f359e640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f4152810 0x7f34f4150800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f359e640 1 -- 172.21.15.131:0/4137486228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f4151690 con 0x7f34f4150d40 2024-09-17T02:45:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34f359e640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34f4150d40 0x7f34f4151150 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f34ec01b1d0 tx=0x7f34ec004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.330+0000 7f34ebfff640 1 -- 172.21.15.131:0/4137486228 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0062e0 con 0x7f34f4150d40 2024-09-17T02:45:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.334+0000 7f34ebfff640 1 -- 172.21.15.131:0/4137486228 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34ec0b5070 con 0x7f34f4150d40 2024-09-17T02:45:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.334+0000 7f34f904e640 1 -- 172.21.15.131:0/4137486228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f4152f20 con 0x7f34f4150d40 2024-09-17T02:45:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.334+0000 7f34ebfff640 1 -- 172.21.15.131:0/4137486228 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0067f0 con 0x7f34f4150d40 2024-09-17T02:45:23.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.334+0000 7f34f904e640 1 -- 172.21.15.131:0/4137486228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f415a980 con 0x7f34f4150d40 2024-09-17T02:45:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.334+0000 7f34f904e640 1 -- 172.21.15.131:0/4137486228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f415f290 con 0x7f34f4150d40 2024-09-17T02:45:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.334+0000 7f34ebfff640 1 -- 172.21.15.131:0/4137486228 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f34ec0b1430 con 0x7f34f4150d40 2024-09-17T02:45:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.334+0000 7f34ebfff640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f34d4073230 0x7f34d4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.334+0000 7f34ebfff640 1 -- 172.21.15.131:0/4137486228 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f34ec12eeb0 con 0x7f34f4150d40 2024-09-17T02:45:23.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.334+0000 7f34f2d9d640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f34d4073230 0x7f34d4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:23.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.334+0000 7f34f2d9d640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f34d4073230 0x7f34d4075740 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0018b0 tx=0x7f34dc001120 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:23.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.338+0000 7f34ebfff640 1 -- 172.21.15.131:0/4137486228 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f34ec0c1070 con 0x7f34f4150d40 2024-09-17T02:45:23.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.546+0000 7f34f904e640 1 -- 172.21.15.131:0/4137486228 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34f4152cb0 con 0x7f34f4150d40 2024-09-17T02:45:23.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34ebfff640 1 -- 172.21.15.131:0/4137486228 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5228 (secure 0 0 0) 0x7f34ec016830 con 0x7f34f4150d40 2024-09-17T02:45:23.547 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:23.547 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":46,"btime":"2024-09-17T02:45:23:141401+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:45:22.552270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6399":{"gid":6399,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/655402879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":655402879},{"type":"v1","addr":"172.21.15.131:6835","nonce":655402879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6399,"qdb_cluster":[6399]},"id":5}]} 2024-09-17T02:45:23.547 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 46 2024-09-17T02:45:23.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 -- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f34d4073230 msgr2=0x7f34d4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f34d4073230 0x7f34d4075740 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f34dc0018b0 tx=0x7f34dc001120 comp rx=0 tx=0).stop 2024-09-17T02:45:23.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 -- 172.21.15.131:0/4137486228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34f4150d40 msgr2=0x7f34f4151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34f4150d40 0x7f34f4151150 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f34ec01b1d0 tx=0x7f34ec004f50 comp rx=0 tx=0).stop 2024-09-17T02:45:23.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 -- 172.21.15.131:0/4137486228 shutdown_connections 2024-09-17T02:45:23.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f34d4073230 0x7f34d4075740 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f34f4150d40 0x7f34f4151150 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f34f4152810 0x7f34f4150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 --2- 172.21.15.131:0/4137486228 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f34e0005260 0x7f34f40ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 -- 172.21.15.131:0/4137486228 >> 172.21.15.131:0/4137486228 conn(0x7f34f40d1b10 msgr2=0x7f34f415ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:23.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 -- 172.21.15.131:0/4137486228 shutdown_connections 2024-09-17T02:45:23.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.550+0000 7f34f904e640 1 -- 172.21.15.131:0/4137486228 wait complete. 2024-09-17T02:45:23.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:45:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 -- 172.21.15.131:0/24418205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb3406e2c0 msgr2=0x7fbb340c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 --2- 172.21.15.131:0/24418205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb3406e2c0 0x7fbb340c4580 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fbb34074bb0 tx=0x7fbb2401c830 comp rx=0 tx=0).stop 2024-09-17T02:45:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 -- 172.21.15.131:0/24418205 shutdown_connections 2024-09-17T02:45:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 --2- 172.21.15.131:0/24418205 >> v2:172.21.15.189:3300/0 conn(0x7fbb340c4ac0 0x7fbb340c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 --2- 172.21.15.131:0/24418205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb3406e2c0 0x7fbb340c4580 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 -- 172.21.15.131:0/24418205 >> 172.21.15.131:0/24418205 conn(0x7fbb3406eea0 msgr2=0x7fbb3406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:23.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 -- 172.21.15.131:0/24418205 shutdown_connections 2024-09-17T02:45:23.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 -- 172.21.15.131:0/24418205 wait complete. 2024-09-17T02:45:23.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 Processor -- start 2024-09-17T02:45:23.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 -- start start 2024-09-17T02:45:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb3406e2c0 0x7fbb341fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb340c4ac0 0x7fbb341f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbb340cdbf0 0x7fbb340c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbb340cf0a0 con 0x7fbb340c4ac0 2024-09-17T02:45:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbb340cef20 con 0x7fbb3406e2c0 2024-09-17T02:45:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb397b2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fbb340cf220 con 0x7fbb340cdbf0 2024-09-17T02:45:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb327fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb340c4ac0 0x7fbb341f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb327fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb340c4ac0 0x7fbb341f7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44086/0 (socket says 172.21.15.131:44086) 2024-09-17T02:45:23.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb327fc640 1 -- 172.21.15.131:0/2411232094 learned_addr learned my addr 172.21.15.131:0/2411232094 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb337fe640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbb340cdbf0 0x7fbb340c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.730+0000 7fbb32ffd640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb3406e2c0 0x7fbb341fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb327fc640 1 -- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbb340cdbf0 msgr2=0x7fbb340c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb327fc640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbb340cdbf0 0x7fbb340c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb327fc640 1 -- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb3406e2c0 msgr2=0x7fbb341fa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb327fc640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb3406e2c0 0x7fbb341fa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb327fc640 1 -- 172.21.15.131:0/2411232094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb341fa930 con 0x7fbb340c4ac0 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb32ffd640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb3406e2c0 0x7fbb341fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb327fc640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb340c4ac0 0x7fbb341f7400 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbb280058d0 tx=0x7fbb2800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb1b7fe640 1 -- 172.21.15.131:0/2411232094 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28017070 con 0x7fbb340c4ac0 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb1b7fe640 1 -- 172.21.15.131:0/2411232094 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb28009cc0 con 0x7fbb340c4ac0 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb397b2640 1 -- 172.21.15.131:0/2411232094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb341fac10 con 0x7fbb340c4ac0 2024-09-17T02:45:23.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb1b7fe640 1 -- 172.21.15.131:0/2411232094 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb280106c0 con 0x7fbb340c4ac0 2024-09-17T02:45:23.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb397b2640 1 -- 172.21.15.131:0/2411232094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb341f6c00 con 0x7fbb340c4ac0 2024-09-17T02:45:23.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb397b2640 1 -- 172.21.15.131:0/2411232094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb340c4580 con 0x7fbb340c4ac0 2024-09-17T02:45:23.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb1b7fe640 1 -- 172.21.15.131:0/2411232094 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbb2801a070 con 0x7fbb340c4ac0 2024-09-17T02:45:23.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb1b7fe640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbb20073200 0x7fbb20075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:23.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb1b7fe640 1 -- 172.21.15.131:0/2411232094 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7fbb28094930 con 0x7fbb340c4ac0 2024-09-17T02:45:23.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.734+0000 7fbb32ffd640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbb20073200 0x7fbb20075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:23.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.738+0000 7fbb32ffd640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbb20073200 0x7fbb20075710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbb240af000 tx=0x7fbb2401b070 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:23.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.738+0000 7fbb1b7fe640 1 -- 172.21.15.131:0/2411232094 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbb2805f8a0 con 0x7fbb340c4ac0 2024-09-17T02:45:23.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.906+0000 7fbb397b2640 1 -- 172.21.15.131:0/2411232094 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbb34153e30 con 0x7fbb340c4ac0 2024-09-17T02:45:23.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.906+0000 7fbb1b7fe640 1 -- 172.21.15.131:0/2411232094 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+16993 (secure 0 0 0) 0x7fbb2805f1a0 con 0x7fbb340c4ac0 2024-09-17T02:45:23.905 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:23.905 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":79,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:45:21.122454+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:45:17.161688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-17T02:45:18.453383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:45:23.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 -- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbb20073200 msgr2=0x7fbb20075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbb20073200 0x7fbb20075710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbb240af000 tx=0x7fbb2401b070 comp rx=0 tx=0).stop 2024-09-17T02:45:23.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 -- 172.21.15.131:0/2411232094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb340c4ac0 msgr2=0x7fbb341f7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:23.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb340c4ac0 0x7fbb341f7400 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbb280058d0 tx=0x7fbb2800dde0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 -- 172.21.15.131:0/2411232094 shutdown_connections 2024-09-17T02:45:23.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbb20073200 0x7fbb20075710 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbb340cdbf0 0x7fbb340c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbb340c4ac0 0x7fbb341f7400 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 --2- 172.21.15.131:0/2411232094 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbb3406e2c0 0x7fbb341fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:23.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 -- 172.21.15.131:0/2411232094 >> 172.21.15.131:0/2411232094 conn(0x7fbb3406eea0 msgr2=0x7fbb340b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:23.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 -- 172.21.15.131:0/2411232094 shutdown_connections 2024-09-17T02:45:23.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:23.910+0000 7fbb397b2640 1 -- 172.21.15.131:0/2411232094 wait complete. 2024-09-17T02:45:23.926 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T02:45:23.927 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:45:24.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.066+0000 7f1da92c1640 1 -- 172.21.15.131:0/2000051126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d8c003a40 msgr2=0x7f1d8c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.066+0000 7f1da92c1640 1 --2- 172.21.15.131:0/2000051126 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d8c003a40 0x7f1d8c003eb0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1d940016f0 tx=0x7f1d9401c7a0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.066+0000 7f1da37fe640 1 -- 172.21.15.131:0/2000051126 reap_dead start 2024-09-17T02:45:24.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.066+0000 7f1da92c1640 1 -- 172.21.15.131:0/2000051126 shutdown_connections 2024-09-17T02:45:24.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.066+0000 7f1da92c1640 1 -- 172.21.15.131:0/2000051126 >> 172.21.15.131:0/2000051126 conn(0x7f1da406eea0 msgr2=0x7f1da406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:24.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.066+0000 7f1da92c1640 1 -- 172.21.15.131:0/2000051126 shutdown_connections 2024-09-17T02:45:24.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.066+0000 7f1da92c1640 1 -- 172.21.15.131:0/2000051126 wait complete. 2024-09-17T02:45:24.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da92c1640 1 Processor -- start 2024-09-17T02:45:24.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da92c1640 1 -- start start 2024-09-17T02:45:24.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da92c1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d8c003a40 0x7f1da41522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da92c1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1da41fae80 0x7f1da414b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da92c1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1da4167250 0x7f1da4167660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da92c1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4168e70 con 0x7f1da41fae80 2024-09-17T02:45:24.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da92c1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1da4168cf0 con 0x7f1d8c003a40 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da92c1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f1da4168ff0 con 0x7f1da4167250 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da27fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1da41fae80 0x7f1da414b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da27fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1da41fae80 0x7f1da414b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44110/0 (socket says 172.21.15.131:44110) 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da27fc640 1 -- 172.21.15.131:0/2579026997 learned_addr learned my addr 172.21.15.131:0/2579026997 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da37fe640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1da4167250 0x7f1da4167660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da27fc640 1 -- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1da4167250 msgr2=0x7f1da4167660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da27fc640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1da4167250 0x7f1da4167660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da27fc640 1 -- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d8c003a40 msgr2=0x7f1da41522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da27fc640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d8c003a40 0x7f1da41522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da27fc640 1 -- 172.21.15.131:0/2579026997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da4167ba0 con 0x7f1da41fae80 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da27fc640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1da41fae80 0x7f1da414b560 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1d98010730 tx=0x7f1d98010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:24.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1d8b7fe640 1 -- 172.21.15.131:0/2579026997 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800eae0 con 0x7f1da41fae80 2024-09-17T02:45:24.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1d8b7fe640 1 -- 172.21.15.131:0/2579026997 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d9800ed70 con 0x7f1da41fae80 2024-09-17T02:45:24.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da92c1640 1 -- 172.21.15.131:0/2579026997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da4153ef0 con 0x7f1da41fae80 2024-09-17T02:45:24.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1d8b7fe640 1 -- 172.21.15.131:0/2579026997 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98004280 con 0x7f1da41fae80 2024-09-17T02:45:24.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.070+0000 7f1da92c1640 1 -- 172.21.15.131:0/2579026997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da415b990 con 0x7f1da41fae80 2024-09-17T02:45:24.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.074+0000 7f1da92c1640 1 -- 172.21.15.131:0/2579026997 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da40701a0 con 0x7f1da41fae80 2024-09-17T02:45:24.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.074+0000 7f1d8b7fe640 1 -- 172.21.15.131:0/2579026997 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1d980047b0 con 0x7f1da41fae80 2024-09-17T02:45:24.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.074+0000 7f1d8b7fe640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d90073230 0x7f1d90075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.074+0000 7f1d8b7fe640 1 -- 172.21.15.131:0/2579026997 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f1d98094f90 con 0x7f1da41fae80 2024-09-17T02:45:24.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.074+0000 7f1da2ffd640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d90073230 0x7f1d90075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:24.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.074+0000 7f1da2ffd640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d90073230 0x7f1d90075740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1d940016c0 tx=0x7f1d8c005840 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:24.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.078+0000 7f1d8b7fe640 1 -- 172.21.15.131:0/2579026997 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1da4167ba0 con 0x7f1da41fae80 2024-09-17T02:45:24.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.234+0000 7f1da92c1640 1 -- 172.21.15.131:0/2579026997 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1da4073310 con 0x7f1d90073230 2024-09-17T02:45:24.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.246+0000 7f1d8b7fe640 1 -- 172.21.15.131:0/2579026997 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251077 (secure 0 0 0) 0x7f1da4073310 con 0x7f1d90073230 2024-09-17T02:45:24.249 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:24.251 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:45:24.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 -- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d90073230 msgr2=0x7f1d90075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d90073230 0x7f1d90075740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1d940016c0 tx=0x7f1d8c005840 comp rx=0 tx=0).stop 2024-09-17T02:45:24.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 -- 172.21.15.131:0/2579026997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1da41fae80 msgr2=0x7f1da414b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1da41fae80 0x7f1da414b560 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f1d98010730 tx=0x7f1d98010bf0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 -- 172.21.15.131:0/2579026997 shutdown_connections 2024-09-17T02:45:24.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d90073230 0x7f1d90075740 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1da4167250 0x7f1da4167660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1da41fae80 0x7f1da414b560 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 --2- 172.21.15.131:0/2579026997 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d8c003a40 0x7f1da41522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 -- 172.21.15.131:0/2579026997 >> 172.21.15.131:0/2579026997 conn(0x7f1da406eea0 msgr2=0x7f1da4154d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:24.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 -- 172.21.15.131:0/2579026997 shutdown_connections 2024-09-17T02:45:24.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.250+0000 7f1da92c1640 1 -- 172.21.15.131:0/2579026997 wait complete. 2024-09-17T02:45:24.267 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-17T02:45:23.225560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258104,"kb_used_data":8584,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716616,"statfs":{"total":773094113280,"available":772829814784,"internally_reserved":0,"allocated":8790016,"data_stored":4617026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.173689"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135946+0000","last_change":"2024-09-17T02:45:20.134496+0000","last_active":"2024-09-17T02:45:21.135946+0000","last_peered":"2024-09-17T02:45:21.135946+0000","last_clean":"2024-09-17T02:45:21.135946+0000","last_became_active":"2024-09-17T02:45:20.134052+0000","last_became_peered":"2024-09-17T02:45:20.134052+0000","last_unstale":"2024-09-17T02:45:21.135946+0000","last_undegraded":"2024-09-17T02:45:21.135946+0000","last_fullsized":"2024-09-17T02:45:21.135946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:33.880550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.066330+0000","last_change":"2024-09-17T02:45:19.066330+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.066330+0000","last_undegraded":"2024-09-17T02:45:19.066330+0000","last_fullsized":"2024-09-17T02:45:19.066330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.071003+0000","last_change":"2024-09-17T02:45:19.071003+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.071003+0000","last_undegraded":"2024-09-17T02:45:19.071003+0000","last_fullsized":"2024-09-17T02:45:19.071003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.127671+0000","last_change":"2024-09-17T02:45:20.124901+0000","last_active":"2024-09-17T02:45:20.127671+0000","last_peered":"2024-09-17T02:45:20.127671+0000","last_clean":"2024-09-17T02:45:20.127671+0000","last_became_active":"2024-09-17T02:45:20.124392+0000","last_became_peered":"2024-09-17T02:45:20.124392+0000","last_unstale":"2024-09-17T02:45:20.127671+0000","last_undegraded":"2024-09-17T02:45:20.127671+0000","last_fullsized":"2024-09-17T02:45:20.127671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:56.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.157150+0000","last_change":"2024-09-17T02:45:20.151934+0000","last_active":"2024-09-17T02:45:20.157150+0000","last_peered":"2024-09-17T02:45:20.157150+0000","last_clean":"2024-09-17T02:45:20.157150+0000","last_became_active":"2024-09-17T02:45:20.149652+0000","last_became_peered":"2024-09-17T02:45:20.149652+0000","last_unstale":"2024-09-17T02:45:20.157150+0000","last_undegraded":"2024-09-17T02:45:20.157150+0000","last_fullsized":"2024-09-17T02:45:20.157150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:06.754098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.162147+0000","last_change":"2024-09-17T02:45:20.156609+0000","last_active":"2024-09-17T02:45:20.162147+0000","last_peered":"2024-09-17T02:45:20.162147+0000","last_clean":"2024-09-17T02:45:20.162147+0000","last_became_active":"2024-09-17T02:45:20.155997+0000","last_became_peered":"2024-09-17T02:45:20.155997+0000","last_unstale":"2024-09-17T02:45:20.162147+0000","last_undegraded":"2024-09-17T02:45:20.162147+0000","last_fullsized":"2024-09-17T02:45:20.162147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:49.213100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.141570+0000","last_change":"2024-09-17T02:45:20.138831+0000","last_active":"2024-09-17T02:45:20.141570+0000","last_peered":"2024-09-17T02:45:20.141570+0000","last_clean":"2024-09-17T02:45:20.141570+0000","last_became_active":"2024-09-17T02:45:20.138468+0000","last_became_peered":"2024-09-17T02:45:20.138468+0000","last_unstale":"2024-09-17T02:45:20.141570+0000","last_undegraded":"2024-09-17T02:45:20.141570+0000","last_fullsized":"2024-09-17T02:45:20.141570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:17.239517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137912+0000","last_change":"2024-09-17T02:45:20.151755+0000","last_active":"2024-09-17T02:45:21.137912+0000","last_peered":"2024-09-17T02:45:21.137912+0000","last_clean":"2024-09-17T02:45:21.137912+0000","last_became_active":"2024-09-17T02:45:20.139390+0000","last_became_peered":"2024-09-17T02:45:20.139390+0000","last_unstale":"2024-09-17T02:45:21.137912+0000","last_undegraded":"2024-09-17T02:45:21.137912+0000","last_fullsized":"2024-09-17T02:45:21.137912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:04.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130320+0000","last_change":"2024-09-17T02:45:20.128764+0000","last_active":"2024-09-17T02:45:20.130320+0000","last_peered":"2024-09-17T02:45:20.130320+0000","last_clean":"2024-09-17T02:45:20.130320+0000","last_became_active":"2024-09-17T02:45:20.128381+0000","last_became_peered":"2024-09-17T02:45:20.128381+0000","last_unstale":"2024-09-17T02:45:20.130320+0000","last_undegraded":"2024-09-17T02:45:20.130320+0000","last_fullsized":"2024-09-17T02:45:20.130320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:35.763836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135793+0000","last_change":"2024-09-17T02:45:20.134184+0000","last_active":"2024-09-17T02:45:21.135793+0000","last_peered":"2024-09-17T02:45:21.135793+0000","last_clean":"2024-09-17T02:45:21.135793+0000","last_became_active":"2024-09-17T02:45:20.133826+0000","last_became_peered":"2024-09-17T02:45:20.133826+0000","last_unstale":"2024-09-17T02:45:21.135793+0000","last_undegraded":"2024-09-17T02:45:21.135793+0000","last_fullsized":"2024-09-17T02:45:21.135793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:41.858427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.068073+0000","last_change":"2024-09-17T02:45:19.068073+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.068073+0000","last_undegraded":"2024-09-17T02:45:19.068073+0000","last_fullsized":"2024-09-17T02:45:19.068073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.146278+0000","last_change":"2024-09-17T02:45:20.137823+0000","last_active":"2024-09-17T02:45:20.146278+0000","last_peered":"2024-09-17T02:45:20.146278+0000","last_clean":"2024-09-17T02:45:20.146278+0000","last_became_active":"2024-09-17T02:45:20.137420+0000","last_became_peered":"2024-09-17T02:45:20.137420+0000","last_unstale":"2024-09-17T02:45:20.146278+0000","last_undegraded":"2024-09-17T02:45:20.146278+0000","last_fullsized":"2024-09-17T02:45:20.146278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:14.885437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.072804+0000","last_change":"2024-09-17T02:45:19.072804+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.072804+0000","last_undegraded":"2024-09-17T02:45:19.072804+0000","last_fullsized":"2024-09-17T02:45:19.072804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139109+0000","last_change":"2024-09-17T02:45:20.154359+0000","last_active":"2024-09-17T02:45:21.139109+0000","last_peered":"2024-09-17T02:45:21.139109+0000","last_clean":"2024-09-17T02:45:21.139109+0000","last_became_active":"2024-09-17T02:45:20.138831+0000","last_became_peered":"2024-09-17T02:45:20.138831+0000","last_unstale":"2024-09-17T02:45:21.139109+0000","last_undegraded":"2024-09-17T02:45:21.139109+0000","last_fullsized":"2024-09-17T02:45:21.139109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:26.541024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151644+0000","last_change":"2024-09-17T02:45:20.136343+0000","last_active":"2024-09-17T02:45:20.151644+0000","last_peered":"2024-09-17T02:45:20.151644+0000","last_clean":"2024-09-17T02:45:20.151644+0000","last_became_active":"2024-09-17T02:45:20.135036+0000","last_became_peered":"2024-09-17T02:45:20.135036+0000","last_unstale":"2024-09-17T02:45:20.151644+0000","last_undegraded":"2024-09-17T02:45:20.151644+0000","last_fullsized":"2024-09-17T02:45:20.151644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:15.507022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134377+0000","last_change":"2024-09-17T02:45:20.133650+0000","last_active":"2024-09-17T02:45:20.134377+0000","last_peered":"2024-09-17T02:45:20.134377+0000","last_clean":"2024-09-17T02:45:20.134377+0000","last_became_active":"2024-09-17T02:45:20.133246+0000","last_became_peered":"2024-09-17T02:45:20.133246+0000","last_unstale":"2024-09-17T02:45:20.134377+0000","last_undegraded":"2024-09-17T02:45:20.134377+0000","last_fullsized":"2024-09-17T02:45:20.134377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:31.528537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138676+0000","last_change":"2024-09-17T02:45:20.138431+0000","last_active":"2024-09-17T02:45:21.138676+0000","last_peered":"2024-09-17T02:45:21.138676+0000","last_clean":"2024-09-17T02:45:21.138676+0000","last_became_active":"2024-09-17T02:45:20.137633+0000","last_became_peered":"2024-09-17T02:45:20.137633+0000","last_unstale":"2024-09-17T02:45:21.138676+0000","last_undegraded":"2024-09-17T02:45:21.138676+0000","last_fullsized":"2024-09-17T02:45:21.138676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:25.489158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.065085+0000","last_change":"2024-09-17T02:45:19.065085+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.065085+0000","last_undegraded":"2024-09-17T02:45:19.065085+0000","last_fullsized":"2024-09-17T02:45:19.065085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138525+0000","last_change":"2024-09-17T02:45:20.138096+0000","last_active":"2024-09-17T02:45:21.138525+0000","last_peered":"2024-09-17T02:45:21.138525+0000","last_clean":"2024-09-17T02:45:21.138525+0000","last_became_active":"2024-09-17T02:45:20.137246+0000","last_became_peered":"2024-09-17T02:45:20.137246+0000","last_unstale":"2024-09-17T02:45:21.138525+0000","last_undegraded":"2024-09-17T02:45:21.138525+0000","last_fullsized":"2024-09-17T02:45:21.138525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:08.405979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.148092+0000","last_change":"2024-09-17T02:45:20.144454+0000","last_active":"2024-09-17T02:45:20.148092+0000","last_peered":"2024-09-17T02:45:20.148092+0000","last_clean":"2024-09-17T02:45:20.148092+0000","last_became_active":"2024-09-17T02:45:20.143671+0000","last_became_peered":"2024-09-17T02:45:20.143671+0000","last_unstale":"2024-09-17T02:45:20.148092+0000","last_undegraded":"2024-09-17T02:45:20.148092+0000","last_fullsized":"2024-09-17T02:45:20.148092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:26.359990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.147886+0000","last_change":"2024-09-17T02:45:20.140711+0000","last_active":"2024-09-17T02:45:20.147886+0000","last_peered":"2024-09-17T02:45:20.147886+0000","last_clean":"2024-09-17T02:45:20.147886+0000","last_became_active":"2024-09-17T02:45:20.140390+0000","last_became_peered":"2024-09-17T02:45:20.140390+0000","last_unstale":"2024-09-17T02:45:20.147886+0000","last_undegraded":"2024-09-17T02:45:20.147886+0000","last_fullsized":"2024-09-17T02:45:20.147886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.473725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.069206+0000","last_change":"2024-09-17T02:45:19.069206+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.069206+0000","last_undegraded":"2024-09-17T02:45:19.069206+0000","last_fullsized":"2024-09-17T02:45:19.069206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135331+0000","last_change":"2024-09-17T02:45:20.120377+0000","last_active":"2024-09-17T02:45:21.135331+0000","last_peered":"2024-09-17T02:45:21.135331+0000","last_clean":"2024-09-17T02:45:21.135331+0000","last_became_active":"2024-09-17T02:45:20.119336+0000","last_became_peered":"2024-09-17T02:45:20.119336+0000","last_unstale":"2024-09-17T02:45:21.135331+0000","last_undegraded":"2024-09-17T02:45:21.135331+0000","last_fullsized":"2024-09-17T02:45:21.135331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:43.174152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.073370+0000","last_change":"2024-09-17T02:45:19.073370+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.073370+0000","last_undegraded":"2024-09-17T02:45:19.073370+0000","last_fullsized":"2024-09-17T02:45:19.073370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.143641+0000","last_change":"2024-09-17T02:45:20.136877+0000","last_active":"2024-09-17T02:45:20.143641+0000","last_peered":"2024-09-17T02:45:20.143641+0000","last_clean":"2024-09-17T02:45:20.143641+0000","last_became_active":"2024-09-17T02:45:20.136227+0000","last_became_peered":"2024-09-17T02:45:20.136227+0000","last_unstale":"2024-09-17T02:45:20.143641+0000","last_undegraded":"2024-09-17T02:45:20.143641+0000","last_fullsized":"2024-09-17T02:45:20.143641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:06.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.131033+0000","last_change":"2024-09-17T02:45:20.129949+0000","last_active":"2024-09-17T02:45:20.131033+0000","last_peered":"2024-09-17T02:45:20.131033+0000","last_clean":"2024-09-17T02:45:20.131033+0000","last_became_active":"2024-09-17T02:45:20.129519+0000","last_became_peered":"2024-09-17T02:45:20.129519+0000","last_unstale":"2024-09-17T02:45:20.131033+0000","last_undegraded":"2024-09-17T02:45:20.131033+0000","last_fullsized":"2024-09-17T02:45:20.131033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:53.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.127890+0000","last_change":"2024-09-17T02:45:20.125488+0000","last_active":"2024-09-17T02:45:20.127890+0000","last_peered":"2024-09-17T02:45:20.127890+0000","last_clean":"2024-09-17T02:45:20.127890+0000","last_became_active":"2024-09-17T02:45:20.124765+0000","last_became_peered":"2024-09-17T02:45:20.124765+0000","last_unstale":"2024-09-17T02:45:20.127890+0000","last_undegraded":"2024-09-17T02:45:20.127890+0000","last_fullsized":"2024-09-17T02:45:20.127890+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:41.858427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.125889+0000","last_change":"2024-09-17T02:45:20.121902+0000","last_active":"2024-09-17T02:45:20.125889+0000","last_peered":"2024-09-17T02:45:20.125889+0000","last_clean":"2024-09-17T02:45:20.125889+0000","last_became_active":"2024-09-17T02:45:20.121274+0000","last_became_peered":"2024-09-17T02:45:20.121274+0000","last_unstale":"2024-09-17T02:45:20.125889+0000","last_undegraded":"2024-09-17T02:45:20.125889+0000","last_fullsized":"2024-09-17T02:45:20.125889+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:19.026083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.150174+0000","last_change":"2024-09-17T02:45:20.147039+0000","last_active":"2024-09-17T02:45:20.150174+0000","last_peered":"2024-09-17T02:45:20.150174+0000","last_clean":"2024-09-17T02:45:20.150174+0000","last_became_active":"2024-09-17T02:45:20.145977+0000","last_became_peered":"2024-09-17T02:45:20.145977+0000","last_unstale":"2024-09-17T02:45:20.150174+0000","last_undegraded":"2024-09-17T02:45:20.150174+0000","last_fullsized":"2024-09-17T02:45:20.150174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:25.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135630+0000","last_change":"2024-09-17T02:45:20.134028+0000","last_active":"2024-09-17T02:45:21.135630+0000","last_peered":"2024-09-17T02:45:21.135630+0000","last_clean":"2024-09-17T02:45:21.135630+0000","last_became_active":"2024-09-17T02:45:20.133628+0000","last_became_peered":"2024-09-17T02:45:20.133628+0000","last_unstale":"2024-09-17T02:45:21.135630+0000","last_undegraded":"2024-09-17T02:45:21.135630+0000","last_fullsized":"2024-09-17T02:45:21.135630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:56.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.074374+0000","last_change":"2024-09-17T02:45:19.074374+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.074374+0000","last_undegraded":"2024-09-17T02:45:19.074374+0000","last_fullsized":"2024-09-17T02:45:19.074374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.129861+0000","last_change":"2024-09-17T02:45:20.127695+0000","last_active":"2024-09-17T02:45:20.129861+0000","last_peered":"2024-09-17T02:45:20.129861+0000","last_clean":"2024-09-17T02:45:20.129861+0000","last_became_active":"2024-09-17T02:45:20.127323+0000","last_became_peered":"2024-09-17T02:45:20.127323+0000","last_unstale":"2024-09-17T02:45:20.129861+0000","last_undegraded":"2024-09-17T02:45:20.129861+0000","last_fullsized":"2024-09-17T02:45:20.129861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:42.063380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.132767+0000","last_change":"2024-09-17T02:45:20.122630+0000","last_active":"2024-09-17T02:45:20.132767+0000","last_peered":"2024-09-17T02:45:20.132767+0000","last_clean":"2024-09-17T02:45:20.132767+0000","last_became_active":"2024-09-17T02:45:20.120596+0000","last_became_peered":"2024-09-17T02:45:20.120596+0000","last_unstale":"2024-09-17T02:45:20.132767+0000","last_undegraded":"2024-09-17T02:45:20.132767+0000","last_fullsized":"2024-09-17T02:45:20.132767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:16.342312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"78'4","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165404+0000","last_change":"2024-09-17T02:45:19.105415+0000","last_active":"2024-09-17T02:45:20.165404+0000","last_peered":"2024-09-17T02:45:20.165404+0000","last_clean":"2024-09-17T02:45:20.165404+0000","last_became_active":"2024-09-17T02:45:19.104799+0000","last_became_peered":"2024-09-17T02:45:19.104799+0000","last_unstale":"2024-09-17T02:45:20.165404+0000","last_undegraded":"2024-09-17T02:45:20.165404+0000","last_fullsized":"2024-09-17T02:45:20.165404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:15:52.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.153720+0000","last_change":"2024-09-17T02:45:20.151127+0000","last_active":"2024-09-17T02:45:20.153720+0000","last_peered":"2024-09-17T02:45:20.153720+0000","last_clean":"2024-09-17T02:45:20.153720+0000","last_became_active":"2024-09-17T02:45:20.150324+0000","last_became_peered":"2024-09-17T02:45:20.150324+0000","last_unstale":"2024-09-17T02:45:20.153720+0000","last_undegraded":"2024-09-17T02:45:20.153720+0000","last_fullsized":"2024-09-17T02:45:20.153720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:26.406192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151669+0000","last_change":"2024-09-17T02:45:19.079006+0000","last_active":"2024-09-17T02:45:20.151669+0000","last_peered":"2024-09-17T02:45:20.151669+0000","last_clean":"2024-09-17T02:45:20.151669+0000","last_became_active":"2024-09-17T02:45:19.078512+0000","last_became_peered":"2024-09-17T02:45:19.078512+0000","last_unstale":"2024-09-17T02:45:20.151669+0000","last_undegraded":"2024-09-17T02:45:20.151669+0000","last_fullsized":"2024-09-17T02:45:20.151669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:48.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134859+0000","last_change":"2024-09-17T02:45:20.131374+0000","last_active":"2024-09-17T02:45:20.134859+0000","last_peered":"2024-09-17T02:45:20.134859+0000","last_clean":"2024-09-17T02:45:20.134859+0000","last_became_active":"2024-09-17T02:45:20.129652+0000","last_became_peered":"2024-09-17T02:45:20.129652+0000","last_unstale":"2024-09-17T02:45:20.134859+0000","last_undegraded":"2024-09-17T02:45:20.134859+0000","last_fullsized":"2024-09-17T02:45:20.134859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:02.100162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136882+0000","last_change":"2024-09-17T02:45:19.078238+0000","last_active":"2024-09-17T02:45:21.136882+0000","last_peered":"2024-09-17T02:45:21.136882+0000","last_clean":"2024-09-17T02:45:21.136882+0000","last_became_active":"2024-09-17T02:45:19.077495+0000","last_became_peered":"2024-09-17T02:45:19.077495+0000","last_unstale":"2024-09-17T02:45:21.136882+0000","last_undegraded":"2024-09-17T02:45:21.136882+0000","last_fullsized":"2024-09-17T02:45:21.136882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:25.395603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130326+0000","last_change":"2024-09-17T02:45:20.127140+0000","last_active":"2024-09-17T02:45:20.130326+0000","last_peered":"2024-09-17T02:45:20.130326+0000","last_clean":"2024-09-17T02:45:20.130326+0000","last_became_active":"2024-09-17T02:45:20.126455+0000","last_became_peered":"2024-09-17T02:45:20.126455+0000","last_unstale":"2024-09-17T02:45:20.130326+0000","last_undegraded":"2024-09-17T02:45:20.130326+0000","last_fullsized":"2024-09-17T02:45:20.130326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:33.880550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"78'2","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149384+0000","last_change":"2024-09-17T02:45:19.092392+0000","last_active":"2024-09-17T02:45:20.149384+0000","last_peered":"2024-09-17T02:45:20.149384+0000","last_clean":"2024-09-17T02:45:20.149384+0000","last_became_active":"2024-09-17T02:45:19.091367+0000","last_became_peered":"2024-09-17T02:45:19.091367+0000","last_unstale":"2024-09-17T02:45:20.149384+0000","last_undegraded":"2024-09-17T02:45:20.149384+0000","last_fullsized":"2024-09-17T02:45:20.149384+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:57.648928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.143331+0000","last_change":"2024-09-17T02:45:20.134096+0000","last_active":"2024-09-17T02:45:20.143331+0000","last_peered":"2024-09-17T02:45:20.143331+0000","last_clean":"2024-09-17T02:45:20.143331+0000","last_became_active":"2024-09-17T02:45:20.133166+0000","last_became_peered":"2024-09-17T02:45:20.133166+0000","last_unstale":"2024-09-17T02:45:20.143331+0000","last_undegraded":"2024-09-17T02:45:20.143331+0000","last_fullsized":"2024-09-17T02:45:20.143331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:17.390929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.147932+0000","last_change":"2024-09-17T02:45:19.095580+0000","last_active":"2024-09-17T02:45:20.147932+0000","last_peered":"2024-09-17T02:45:20.147932+0000","last_clean":"2024-09-17T02:45:20.147932+0000","last_became_active":"2024-09-17T02:45:19.094437+0000","last_became_peered":"2024-09-17T02:45:19.094437+0000","last_unstale":"2024-09-17T02:45:20.147932+0000","last_undegraded":"2024-09-17T02:45:20.147932+0000","last_fullsized":"2024-09-17T02:45:20.147932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:59.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.066618+0000","last_change":"2024-09-17T02:45:19.066618+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.066618+0000","last_undegraded":"2024-09-17T02:45:19.066618+0000","last_fullsized":"2024-09-17T02:45:19.066618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165349+0000","last_change":"2024-09-17T02:45:19.086585+0000","last_active":"2024-09-17T02:45:20.165349+0000","last_peered":"2024-09-17T02:45:20.165349+0000","last_clean":"2024-09-17T02:45:20.165349+0000","last_became_active":"2024-09-17T02:45:19.086060+0000","last_became_peered":"2024-09-17T02:45:19.086060+0000","last_unstale":"2024-09-17T02:45:20.165349+0000","last_undegraded":"2024-09-17T02:45:20.165349+0000","last_fullsized":"2024-09-17T02:45:20.165349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:06.921612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.128800+0000","last_change":"2024-09-17T02:45:20.122719+0000","last_active":"2024-09-17T02:45:20.128800+0000","last_peered":"2024-09-17T02:45:20.128800+0000","last_clean":"2024-09-17T02:45:20.128800+0000","last_became_active":"2024-09-17T02:45:20.120568+0000","last_became_peered":"2024-09-17T02:45:20.120568+0000","last_unstale":"2024-09-17T02:45:20.128800+0000","last_undegraded":"2024-09-17T02:45:20.128800+0000","last_fullsized":"2024-09-17T02:45:20.128800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.038462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137866+0000","last_change":"2024-09-17T02:45:19.093274+0000","last_active":"2024-09-17T02:45:21.137866+0000","last_peered":"2024-09-17T02:45:21.137866+0000","last_clean":"2024-09-17T02:45:21.137866+0000","last_became_active":"2024-09-17T02:45:19.091525+0000","last_became_peered":"2024-09-17T02:45:19.091525+0000","last_unstale":"2024-09-17T02:45:21.137866+0000","last_undegraded":"2024-09-17T02:45:21.137866+0000","last_fullsized":"2024-09-17T02:45:21.137866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:05.164896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.136305+0000","last_change":"2024-09-17T02:45:20.132688+0000","last_active":"2024-09-17T02:45:20.136305+0000","last_peered":"2024-09-17T02:45:20.136305+0000","last_clean":"2024-09-17T02:45:20.136305+0000","last_became_active":"2024-09-17T02:45:20.132335+0000","last_became_peered":"2024-09-17T02:45:20.132335+0000","last_unstale":"2024-09-17T02:45:20.136305+0000","last_undegraded":"2024-09-17T02:45:20.136305+0000","last_fullsized":"2024-09-17T02:45:20.136305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:01.061964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.094857+0000","last_change":"2024-09-17T02:45:19.086961+0000","last_active":"2024-09-17T02:45:19.094857+0000","last_peered":"2024-09-17T02:45:19.094857+0000","last_clean":"2024-09-17T02:45:19.094857+0000","last_became_active":"2024-09-17T02:45:19.084997+0000","last_became_peered":"2024-09-17T02:45:19.084997+0000","last_unstale":"2024-09-17T02:45:19.094857+0000","last_undegraded":"2024-09-17T02:45:19.094857+0000","last_fullsized":"2024-09-17T02:45:19.094857+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:07.303479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.071853+0000","last_change":"2024-09-17T02:45:19.071853+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.071853+0000","last_undegraded":"2024-09-17T02:45:19.071853+0000","last_fullsized":"2024-09-17T02:45:19.071853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.103868+0000","last_change":"2024-09-17T02:45:19.081505+0000","last_active":"2024-09-17T02:45:20.103868+0000","last_peered":"2024-09-17T02:45:20.103868+0000","last_clean":"2024-09-17T02:45:20.103868+0000","last_became_active":"2024-09-17T02:45:19.081196+0000","last_became_peered":"2024-09-17T02:45:19.081196+0000","last_unstale":"2024-09-17T02:45:20.103868+0000","last_undegraded":"2024-09-17T02:45:20.103868+0000","last_fullsized":"2024-09-17T02:45:20.103868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:33.638898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.146953+0000","last_change":"2024-09-17T02:45:20.139575+0000","last_active":"2024-09-17T02:45:20.146953+0000","last_peered":"2024-09-17T02:45:20.146953+0000","last_clean":"2024-09-17T02:45:20.146953+0000","last_became_active":"2024-09-17T02:45:20.137448+0000","last_became_peered":"2024-09-17T02:45:20.137448+0000","last_unstale":"2024-09-17T02:45:20.146953+0000","last_undegraded":"2024-09-17T02:45:20.146953+0000","last_fullsized":"2024-09-17T02:45:20.146953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:28.895745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135457+0000","last_change":"2024-09-17T02:45:19.074384+0000","last_active":"2024-09-17T02:45:21.135457+0000","last_peered":"2024-09-17T02:45:21.135457+0000","last_clean":"2024-09-17T02:45:21.135457+0000","last_became_active":"2024-09-17T02:45:19.073958+0000","last_became_peered":"2024-09-17T02:45:19.073958+0000","last_unstale":"2024-09-17T02:45:21.135457+0000","last_undegraded":"2024-09-17T02:45:21.135457+0000","last_fullsized":"2024-09-17T02:45:21.135457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:36.384573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139981+0000","last_change":"2024-09-17T02:45:20.152408+0000","last_active":"2024-09-17T02:45:21.139981+0000","last_peered":"2024-09-17T02:45:21.139981+0000","last_clean":"2024-09-17T02:45:21.139981+0000","last_became_active":"2024-09-17T02:45:20.137766+0000","last_became_peered":"2024-09-17T02:45:20.137766+0000","last_unstale":"2024-09-17T02:45:21.139981+0000","last_undegraded":"2024-09-17T02:45:21.139981+0000","last_fullsized":"2024-09-17T02:45:21.139981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:54.493799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"78'3","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165499+0000","last_change":"2024-09-17T02:45:19.101534+0000","last_active":"2024-09-17T02:45:20.165499+0000","last_peered":"2024-09-17T02:45:20.165499+0000","last_clean":"2024-09-17T02:45:20.165499+0000","last_became_active":"2024-09-17T02:45:19.099790+0000","last_became_peered":"2024-09-17T02:45:19.099790+0000","last_unstale":"2024-09-17T02:45:20.165499+0000","last_undegraded":"2024-09-17T02:45:20.165499+0000","last_fullsized":"2024-09-17T02:45:20.165499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:59.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.155826+0000","last_change":"2024-09-17T02:45:20.154562+0000","last_active":"2024-09-17T02:45:20.155826+0000","last_peered":"2024-09-17T02:45:20.155826+0000","last_clean":"2024-09-17T02:45:20.155826+0000","last_became_active":"2024-09-17T02:45:20.153866+0000","last_became_peered":"2024-09-17T02:45:20.153866+0000","last_unstale":"2024-09-17T02:45:20.155826+0000","last_undegraded":"2024-09-17T02:45:20.155826+0000","last_fullsized":"2024-09-17T02:45:20.155826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:31.599827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"78'6","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.154097+0000","last_change":"2024-09-17T02:45:19.079828+0000","last_active":"2024-09-17T02:45:20.154097+0000","last_peered":"2024-09-17T02:45:20.154097+0000","last_clean":"2024-09-17T02:45:20.154097+0000","last_became_active":"2024-09-17T02:45:19.079230+0000","last_became_peered":"2024-09-17T02:45:19.079230+0000","last_unstale":"2024-09-17T02:45:20.154097+0000","last_undegraded":"2024-09-17T02:45:20.154097+0000","last_fullsized":"2024-09-17T02:45:20.154097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:46.676590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.133641+0000","last_change":"2024-09-17T02:45:20.130719+0000","last_active":"2024-09-17T02:45:20.133641+0000","last_peered":"2024-09-17T02:45:20.133641+0000","last_clean":"2024-09-17T02:45:20.133641+0000","last_became_active":"2024-09-17T02:45:20.130351+0000","last_became_peered":"2024-09-17T02:45:20.130351+0000","last_unstale":"2024-09-17T02:45:20.133641+0000","last_undegraded":"2024-09-17T02:45:20.133641+0000","last_fullsized":"2024-09-17T02:45:20.133641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:15.730199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136247+0000","last_change":"2024-09-17T02:45:19.085733+0000","last_active":"2024-09-17T02:45:21.136247+0000","last_peered":"2024-09-17T02:45:21.136247+0000","last_clean":"2024-09-17T02:45:21.136247+0000","last_became_active":"2024-09-17T02:45:19.085445+0000","last_became_peered":"2024-09-17T02:45:19.085445+0000","last_unstale":"2024-09-17T02:45:21.136247+0000","last_undegraded":"2024-09-17T02:45:21.136247+0000","last_fullsized":"2024-09-17T02:45:21.136247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:35.839507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136624+0000","last_change":"2024-09-17T02:45:20.127481+0000","last_active":"2024-09-17T02:45:21.136624+0000","last_peered":"2024-09-17T02:45:21.136624+0000","last_clean":"2024-09-17T02:45:21.136624+0000","last_became_active":"2024-09-17T02:45:20.127085+0000","last_became_peered":"2024-09-17T02:45:20.127085+0000","last_unstale":"2024-09-17T02:45:21.136624+0000","last_undegraded":"2024-09-17T02:45:21.136624+0000","last_fullsized":"2024-09-17T02:45:21.136624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:42.063380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.094987+0000","last_change":"2024-09-17T02:45:19.094438+0000","last_active":"2024-09-17T02:45:20.094987+0000","last_peered":"2024-09-17T02:45:20.094987+0000","last_clean":"2024-09-17T02:45:20.094987+0000","last_became_active":"2024-09-17T02:45:19.093709+0000","last_became_peered":"2024-09-17T02:45:19.093709+0000","last_unstale":"2024-09-17T02:45:20.094987+0000","last_undegraded":"2024-09-17T02:45:20.094987+0000","last_fullsized":"2024-09-17T02:45:20.094987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:45.773745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.135664+0000","last_change":"2024-09-17T02:45:20.131160+0000","last_active":"2024-09-17T02:45:20.135664+0000","last_peered":"2024-09-17T02:45:20.135664+0000","last_clean":"2024-09-17T02:45:20.135664+0000","last_became_active":"2024-09-17T02:45:20.129742+0000","last_became_peered":"2024-09-17T02:45:20.129742+0000","last_unstale":"2024-09-17T02:45:20.135664+0000","last_undegraded":"2024-09-17T02:45:20.135664+0000","last_fullsized":"2024-09-17T02:45:20.135664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:11.528887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.113545+0000","last_change":"2024-09-17T02:45:19.105626+0000","last_active":"2024-09-17T02:45:20.113545+0000","last_peered":"2024-09-17T02:45:20.113545+0000","last_clean":"2024-09-17T02:45:20.113545+0000","last_became_active":"2024-09-17T02:45:19.104904+0000","last_became_peered":"2024-09-17T02:45:19.104904+0000","last_unstale":"2024-09-17T02:45:20.113545+0000","last_undegraded":"2024-09-17T02:45:20.113545+0000","last_fullsized":"2024-09-17T02:45:20.113545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:38.203466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135464+0000","last_change":"2024-09-17T02:45:20.138086+0000","last_active":"2024-09-17T02:45:21.135464+0000","last_peered":"2024-09-17T02:45:21.135464+0000","last_clean":"2024-09-17T02:45:21.135464+0000","last_became_active":"2024-09-17T02:45:20.134972+0000","last_became_peered":"2024-09-17T02:45:20.134972+0000","last_unstale":"2024-09-17T02:45:21.135464+0000","last_undegraded":"2024-09-17T02:45:21.135464+0000","last_fullsized":"2024-09-17T02:45:21.135464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:18.254883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"78'8","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.164732+0000","last_change":"2024-09-17T02:45:19.102331+0000","last_active":"2024-09-17T02:45:20.164732+0000","last_peered":"2024-09-17T02:45:20.164732+0000","last_clean":"2024-09-17T02:45:20.164732+0000","last_became_active":"2024-09-17T02:45:19.101721+0000","last_became_peered":"2024-09-17T02:45:19.101721+0000","last_unstale":"2024-09-17T02:45:20.164732+0000","last_undegraded":"2024-09-17T02:45:20.164732+0000","last_fullsized":"2024-09-17T02:45:20.164732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:45.773745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149140+0000","last_change":"2024-09-17T02:45:20.131992+0000","last_active":"2024-09-17T02:45:20.149140+0000","last_peered":"2024-09-17T02:45:20.149140+0000","last_clean":"2024-09-17T02:45:20.149140+0000","last_became_active":"2024-09-17T02:45:20.126966+0000","last_became_peered":"2024-09-17T02:45:20.126966+0000","last_unstale":"2024-09-17T02:45:20.149140+0000","last_undegraded":"2024-09-17T02:45:20.149140+0000","last_fullsized":"2024-09-17T02:45:20.149140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:07.337735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.164565+0000","last_change":"2024-09-17T02:45:19.095009+0000","last_active":"2024-09-17T02:45:20.164565+0000","last_peered":"2024-09-17T02:45:20.164565+0000","last_clean":"2024-09-17T02:45:20.164565+0000","last_became_active":"2024-09-17T02:45:19.094610+0000","last_became_peered":"2024-09-17T02:45:19.094610+0000","last_unstale":"2024-09-17T02:45:20.164565+0000","last_undegraded":"2024-09-17T02:45:20.164565+0000","last_fullsized":"2024-09-17T02:45:20.164565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:42.163563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.073129+0000","last_change":"2024-09-17T02:45:19.073129+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.073129+0000","last_undegraded":"2024-09-17T02:45:19.073129+0000","last_fullsized":"2024-09-17T02:45:19.073129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"78'2","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151200+0000","last_change":"2024-09-17T02:45:19.089116+0000","last_active":"2024-09-17T02:45:20.151200+0000","last_peered":"2024-09-17T02:45:20.151200+0000","last_clean":"2024-09-17T02:45:20.151200+0000","last_became_active":"2024-09-17T02:45:19.088667+0000","last_became_peered":"2024-09-17T02:45:19.088667+0000","last_unstale":"2024-09-17T02:45:20.151200+0000","last_undegraded":"2024-09-17T02:45:20.151200+0000","last_fullsized":"2024-09-17T02:45:20.151200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:54:35.726780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139554+0000","last_change":"2024-09-17T02:45:20.162326+0000","last_active":"2024-09-17T02:45:21.139554+0000","last_peered":"2024-09-17T02:45:21.139554+0000","last_clean":"2024-09-17T02:45:21.139554+0000","last_became_active":"2024-09-17T02:45:20.161849+0000","last_became_peered":"2024-09-17T02:45:20.161849+0000","last_unstale":"2024-09-17T02:45:21.139554+0000","last_undegraded":"2024-09-17T02:45:21.139554+0000","last_fullsized":"2024-09-17T02:45:21.139554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:53.540992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.161729+0000","last_change":"2024-09-17T02:45:19.099466+0000","last_active":"2024-09-17T02:45:20.161729+0000","last_peered":"2024-09-17T02:45:20.161729+0000","last_clean":"2024-09-17T02:45:20.161729+0000","last_became_active":"2024-09-17T02:45:19.098488+0000","last_became_peered":"2024-09-17T02:45:19.098488+0000","last_unstale":"2024-09-17T02:45:20.161729+0000","last_undegraded":"2024-09-17T02:45:20.161729+0000","last_fullsized":"2024-09-17T02:45:20.161729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:32.496598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.157082+0000","last_change":"2024-09-17T02:45:20.152366+0000","last_active":"2024-09-17T02:45:20.157082+0000","last_peered":"2024-09-17T02:45:20.157082+0000","last_clean":"2024-09-17T02:45:20.157082+0000","last_became_active":"2024-09-17T02:45:20.150703+0000","last_became_peered":"2024-09-17T02:45:20.150703+0000","last_unstale":"2024-09-17T02:45:20.157082+0000","last_undegraded":"2024-09-17T02:45:20.157082+0000","last_fullsized":"2024-09-17T02:45:20.157082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:04.950418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.115551+0000","last_change":"2024-09-17T02:45:19.105629+0000","last_active":"2024-09-17T02:45:20.115551+0000","last_peered":"2024-09-17T02:45:20.115551+0000","last_clean":"2024-09-17T02:45:20.115551+0000","last_became_active":"2024-09-17T02:45:19.104942+0000","last_became_peered":"2024-09-17T02:45:19.104942+0000","last_unstale":"2024-09-17T02:45:20.115551+0000","last_undegraded":"2024-09-17T02:45:20.115551+0000","last_fullsized":"2024-09-17T02:45:20.115551+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:55.509905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.150727+0000","last_change":"2024-09-17T02:45:20.141760+0000","last_active":"2024-09-17T02:45:20.150727+0000","last_peered":"2024-09-17T02:45:20.150727+0000","last_clean":"2024-09-17T02:45:20.150727+0000","last_became_active":"2024-09-17T02:45:20.141069+0000","last_became_peered":"2024-09-17T02:45:20.141069+0000","last_unstale":"2024-09-17T02:45:20.150727+0000","last_undegraded":"2024-09-17T02:45:20.150727+0000","last_fullsized":"2024-09-17T02:45:20.150727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:36.850096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.106245+0000","last_change":"2024-09-17T02:45:19.101863+0000","last_active":"2024-09-17T02:45:19.106245+0000","last_peered":"2024-09-17T02:45:19.106245+0000","last_clean":"2024-09-17T02:45:19.106245+0000","last_became_active":"2024-09-17T02:45:19.101175+0000","last_became_peered":"2024-09-17T02:45:19.101175+0000","last_unstale":"2024-09-17T02:45:19.106245+0000","last_undegraded":"2024-09-17T02:45:19.106245+0000","last_fullsized":"2024-09-17T02:45:19.106245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:02.512375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149604+0000","last_change":"2024-09-17T02:45:20.127175+0000","last_active":"2024-09-17T02:45:20.149604+0000","last_peered":"2024-09-17T02:45:20.149604+0000","last_clean":"2024-09-17T02:45:20.149604+0000","last_became_active":"2024-09-17T02:45:20.126335+0000","last_became_peered":"2024-09-17T02:45:20.126335+0000","last_unstale":"2024-09-17T02:45:20.149604+0000","last_undegraded":"2024-09-17T02:45:20.149604+0000","last_fullsized":"2024-09-17T02:45:20.149604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:52.206612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138147+0000","last_change":"2024-09-17T02:45:19.093429+0000","last_active":"2024-09-17T02:45:21.138147+0000","last_peered":"2024-09-17T02:45:21.138147+0000","last_clean":"2024-09-17T02:45:21.138147+0000","last_became_active":"2024-09-17T02:45:19.092623+0000","last_became_peered":"2024-09-17T02:45:19.092623+0000","last_unstale":"2024-09-17T02:45:21.138147+0000","last_undegraded":"2024-09-17T02:45:21.138147+0000","last_fullsized":"2024-09-17T02:45:21.138147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:25.349401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.129570+0000","last_change":"2024-09-17T02:45:20.128671+0000","last_active":"2024-09-17T02:45:20.129570+0000","last_peered":"2024-09-17T02:45:20.129570+0000","last_clean":"2024-09-17T02:45:20.129570+0000","last_became_active":"2024-09-17T02:45:20.128268+0000","last_became_peered":"2024-09-17T02:45:20.128268+0000","last_unstale":"2024-09-17T02:45:20.129570+0000","last_undegraded":"2024-09-17T02:45:20.129570+0000","last_fullsized":"2024-09-17T02:45:20.129570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:05.898373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.096027+0000","last_change":"2024-09-17T02:45:19.089801+0000","last_active":"2024-09-17T02:45:19.096027+0000","last_peered":"2024-09-17T02:45:19.096027+0000","last_clean":"2024-09-17T02:45:19.096027+0000","last_became_active":"2024-09-17T02:45:19.083865+0000","last_became_peered":"2024-09-17T02:45:19.083865+0000","last_unstale":"2024-09-17T02:45:19.096027+0000","last_undegraded":"2024-09-17T02:45:19.096027+0000","last_fullsized":"2024-09-17T02:45:19.096027+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:05.560812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.122736+0000","last_change":"2024-09-17T02:45:20.121012+0000","last_active":"2024-09-17T02:45:20.122736+0000","last_peered":"2024-09-17T02:45:20.122736+0000","last_clean":"2024-09-17T02:45:20.122736+0000","last_became_active":"2024-09-17T02:45:20.120133+0000","last_became_peered":"2024-09-17T02:45:20.120133+0000","last_unstale":"2024-09-17T02:45:20.122736+0000","last_undegraded":"2024-09-17T02:45:20.122736+0000","last_fullsized":"2024-09-17T02:45:20.122736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:18.254883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135613+0000","last_change":"2024-09-17T02:45:19.082147+0000","last_active":"2024-09-17T02:45:21.135613+0000","last_peered":"2024-09-17T02:45:21.135613+0000","last_clean":"2024-09-17T02:45:21.135613+0000","last_became_active":"2024-09-17T02:45:19.081729+0000","last_became_peered":"2024-09-17T02:45:19.081729+0000","last_unstale":"2024-09-17T02:45:21.135613+0000","last_undegraded":"2024-09-17T02:45:21.135613+0000","last_fullsized":"2024-09-17T02:45:21.135613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:27.939745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135204+0000","last_change":"2024-09-17T02:45:20.136717+0000","last_active":"2024-09-17T02:45:21.135204+0000","last_peered":"2024-09-17T02:45:21.135204+0000","last_clean":"2024-09-17T02:45:21.135204+0000","last_became_active":"2024-09-17T02:45:20.136343+0000","last_became_peered":"2024-09-17T02:45:20.136343+0000","last_unstale":"2024-09-17T02:45:21.135204+0000","last_undegraded":"2024-09-17T02:45:21.135204+0000","last_fullsized":"2024-09-17T02:45:21.135204+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:44.913811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137044+0000","last_change":"2024-09-17T02:45:19.102757+0000","last_active":"2024-09-17T02:45:21.137044+0000","last_peered":"2024-09-17T02:45:21.137044+0000","last_clean":"2024-09-17T02:45:21.137044+0000","last_became_active":"2024-09-17T02:45:19.100744+0000","last_became_peered":"2024-09-17T02:45:19.100744+0000","last_unstale":"2024-09-17T02:45:21.137044+0000","last_undegraded":"2024-09-17T02:45:21.137044+0000","last_fullsized":"2024-09-17T02:45:21.137044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:57.648928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134832+0000","last_change":"2024-09-17T02:45:20.130812+0000","last_active":"2024-09-17T02:45:20.134832+0000","last_peered":"2024-09-17T02:45:20.134832+0000","last_clean":"2024-09-17T02:45:20.134832+0000","last_became_active":"2024-09-17T02:45:20.130480+0000","last_became_peered":"2024-09-17T02:45:20.130480+0000","last_unstale":"2024-09-17T02:45:20.134832+0000","last_undegraded":"2024-09-17T02:45:20.134832+0000","last_fullsized":"2024-09-17T02:45:20.134832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:43.366504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136826+0000","last_change":"2024-09-17T02:45:19.093011+0000","last_active":"2024-09-17T02:45:21.136826+0000","last_peered":"2024-09-17T02:45:21.136826+0000","last_clean":"2024-09-17T02:45:21.136826+0000","last_became_active":"2024-09-17T02:45:19.091917+0000","last_became_peered":"2024-09-17T02:45:19.091917+0000","last_unstale":"2024-09-17T02:45:21.136826+0000","last_undegraded":"2024-09-17T02:45:21.136826+0000","last_fullsized":"2024-09-17T02:45:21.136826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:03.939829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139221+0000","last_change":"2024-09-17T02:45:20.153283+0000","last_active":"2024-09-17T02:45:21.139221+0000","last_peered":"2024-09-17T02:45:21.139221+0000","last_clean":"2024-09-17T02:45:21.139221+0000","last_became_active":"2024-09-17T02:45:20.152428+0000","last_became_peered":"2024-09-17T02:45:20.152428+0000","last_unstale":"2024-09-17T02:45:21.139221+0000","last_undegraded":"2024-09-17T02:45:21.139221+0000","last_fullsized":"2024-09-17T02:45:21.139221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:36.231357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.104378+0000","last_change":"2024-09-17T02:45:19.096590+0000","last_active":"2024-09-17T02:45:20.104378+0000","last_peered":"2024-09-17T02:45:20.104378+0000","last_clean":"2024-09-17T02:45:20.104378+0000","last_became_active":"2024-09-17T02:45:19.096024+0000","last_became_peered":"2024-09-17T02:45:19.096024+0000","last_unstale":"2024-09-17T02:45:20.104378+0000","last_undegraded":"2024-09-17T02:45:20.104378+0000","last_fullsized":"2024-09-17T02:45:20.104378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:17.702397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.126109+0000","last_change":"2024-09-17T02:45:20.120878+0000","last_active":"2024-09-17T02:45:20.126109+0000","last_peered":"2024-09-17T02:45:20.126109+0000","last_clean":"2024-09-17T02:45:20.126109+0000","last_became_active":"2024-09-17T02:45:20.119874+0000","last_became_peered":"2024-09-17T02:45:20.119874+0000","last_unstale":"2024-09-17T02:45:20.126109+0000","last_undegraded":"2024-09-17T02:45:20.126109+0000","last_fullsized":"2024-09-17T02:45:20.126109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:55.317365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.108369+0000","last_change":"2024-09-17T02:45:19.084757+0000","last_active":"2024-09-17T02:45:20.108369+0000","last_peered":"2024-09-17T02:45:20.108369+0000","last_clean":"2024-09-17T02:45:20.108369+0000","last_became_active":"2024-09-17T02:45:19.084284+0000","last_became_peered":"2024-09-17T02:45:19.084284+0000","last_unstale":"2024-09-17T02:45:20.108369+0000","last_undegraded":"2024-09-17T02:45:20.108369+0000","last_fullsized":"2024-09-17T02:45:20.108369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:31.618471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130874+0000","last_change":"2024-09-17T02:45:20.128870+0000","last_active":"2024-09-17T02:45:20.130874+0000","last_peered":"2024-09-17T02:45:20.130874+0000","last_clean":"2024-09-17T02:45:20.130874+0000","last_became_active":"2024-09-17T02:45:20.128450+0000","last_became_peered":"2024-09-17T02:45:20.128450+0000","last_unstale":"2024-09-17T02:45:20.130874+0000","last_undegraded":"2024-09-17T02:45:20.130874+0000","last_fullsized":"2024-09-17T02:45:20.130874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:39.366822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.111875+0000","last_change":"2024-09-17T02:45:19.106516+0000","last_active":"2024-09-17T02:45:20.111875+0000","last_peered":"2024-09-17T02:45:20.111875+0000","last_clean":"2024-09-17T02:45:20.111875+0000","last_became_active":"2024-09-17T02:45:19.105883+0000","last_became_peered":"2024-09-17T02:45:19.105883+0000","last_unstale":"2024-09-17T02:45:20.111875+0000","last_undegraded":"2024-09-17T02:45:20.111875+0000","last_fullsized":"2024-09-17T02:45:20.111875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:39.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.063677+0000","last_change":"2024-09-17T02:45:19.063677+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.063677+0000","last_undegraded":"2024-09-17T02:45:19.063677+0000","last_fullsized":"2024-09-17T02:45:19.063677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.110077+0000","last_change":"2024-09-17T02:45:19.086253+0000","last_active":"2024-09-17T02:45:20.110077+0000","last_peered":"2024-09-17T02:45:20.110077+0000","last_clean":"2024-09-17T02:45:20.110077+0000","last_became_active":"2024-09-17T02:45:19.085807+0000","last_became_peered":"2024-09-17T02:45:19.085807+0000","last_unstale":"2024-09-17T02:45:20.110077+0000","last_undegraded":"2024-09-17T02:45:20.110077+0000","last_fullsized":"2024-09-17T02:45:20.110077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:43.903222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149388+0000","last_change":"2024-09-17T02:45:20.124232+0000","last_active":"2024-09-17T02:45:20.149388+0000","last_peered":"2024-09-17T02:45:20.149388+0000","last_clean":"2024-09-17T02:45:20.149388+0000","last_became_active":"2024-09-17T02:45:20.123450+0000","last_became_peered":"2024-09-17T02:45:20.123450+0000","last_unstale":"2024-09-17T02:45:20.149388+0000","last_undegraded":"2024-09-17T02:45:20.149388+0000","last_fullsized":"2024-09-17T02:45:20.149388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:01.142409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.094923+0000","last_change":"2024-09-17T02:45:19.087894+0000","last_active":"2024-09-17T02:45:19.094923+0000","last_peered":"2024-09-17T02:45:19.094923+0000","last_clean":"2024-09-17T02:45:19.094923+0000","last_became_active":"2024-09-17T02:45:19.086041+0000","last_became_peered":"2024-09-17T02:45:19.086041+0000","last_unstale":"2024-09-17T02:45:19.094923+0000","last_undegraded":"2024-09-17T02:45:19.094923+0000","last_fullsized":"2024-09-17T02:45:19.094923+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:30.454811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130153+0000","last_change":"2024-09-17T02:45:20.128777+0000","last_active":"2024-09-17T02:45:20.130153+0000","last_peered":"2024-09-17T02:45:20.130153+0000","last_clean":"2024-09-17T02:45:20.130153+0000","last_became_active":"2024-09-17T02:45:20.128362+0000","last_became_peered":"2024-09-17T02:45:20.128362+0000","last_unstale":"2024-09-17T02:45:20.130153+0000","last_undegraded":"2024-09-17T02:45:20.130153+0000","last_fullsized":"2024-09-17T02:45:20.130153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:46.647418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.111264+0000","last_change":"2024-09-17T02:45:19.099271+0000","last_active":"2024-09-17T02:45:20.111264+0000","last_peered":"2024-09-17T02:45:20.111264+0000","last_clean":"2024-09-17T02:45:20.111264+0000","last_became_active":"2024-09-17T02:45:19.088289+0000","last_became_peered":"2024-09-17T02:45:19.088289+0000","last_unstale":"2024-09-17T02:45:20.111264+0000","last_undegraded":"2024-09-17T02:45:20.111264+0000","last_fullsized":"2024-09-17T02:45:20.111264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:54.306776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":220,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.134645+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:45:21.134645+0000","last_peered":"2024-09-17T02:45:21.134645+0000","last_clean":"2024-09-17T02:45:21.134645+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:45:21.134645+0000","last_undegraded":"2024-09-17T02:45:21.134645+0000","last_fullsized":"2024-09-17T02:45:21.134645+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509500,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":954368,"data_stored":431399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509500,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1544192,"data_stored":1021205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509501,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":430769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509499,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":432463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509500,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":430747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509500,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":1022761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":950272,"data_stored":430769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":416913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:45:24.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:45:24.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.462+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/385997580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f24155240 msgr2=0x7f6f24073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.462+0000 7f6f2c1f0640 1 --2- 172.21.15.131:0/385997580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f24155240 0x7f6f24073430 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6f2406a1c0 tx=0x7f6f2001c690 comp rx=0 tx=0).stop 2024-09-17T02:45:24.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.462+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/385997580 shutdown_connections 2024-09-17T02:45:24.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.462+0000 7f6f2c1f0640 1 --2- 172.21.15.131:0/385997580 >> v2:172.21.15.189:3300/0 conn(0x7f6f24073970 0x7f6f2406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.462+0000 7f6f2c1f0640 1 --2- 172.21.15.131:0/385997580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f24155240 0x7f6f24073430 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.462+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/385997580 >> 172.21.15.131:0/385997580 conn(0x7f6f2406eea0 msgr2=0x7f6f2406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:24.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.462+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/385997580 shutdown_connections 2024-09-17T02:45:24.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.462+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/385997580 wait complete. 2024-09-17T02:45:24.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2c1f0640 1 Processor -- start 2024-09-17T02:45:24.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2c1f0640 1 -- start start 2024-09-17T02:45:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2c1f0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f2406e3e0 0x7f6f2414f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2c1f0640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6f24073970 0x7f6f24157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2c1f0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f24155240 0x7f6f24157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2c1f0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6f24165ce0 con 0x7f6f24155240 2024-09-17T02:45:24.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2c1f0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6f24165b60 con 0x7f6f2406e3e0 2024-09-17T02:45:24.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2c1f0640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6f24165e60 con 0x7f6f24073970 2024-09-17T02:45:24.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2a78d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f24155240 0x7f6f24157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:24.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2a78d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f24155240 0x7f6f24157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44128/0 (socket says 172.21.15.131:44128) 2024-09-17T02:45:24.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2a78d640 1 -- 172.21.15.131:0/2464837402 learned_addr learned my addr 172.21.15.131:0/2464837402 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:24.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f29f8c640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f2406e3e0 0x7f6f2414f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:24.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2978b640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6f24073970 0x7f6f24157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:24.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2a78d640 1 -- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6f24073970 msgr2=0x7f6f24157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2a78d640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6f24073970 0x7f6f24157830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2a78d640 1 -- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f2406e3e0 msgr2=0x7f6f2414f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2a78d640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f2406e3e0 0x7f6f2414f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2a78d640 1 -- 172.21.15.131:0/2464837402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2415a4e0 con 0x7f6f24155240 2024-09-17T02:45:24.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2a78d640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f24155240 0x7f6f24157d70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b990 tx=0x7f6f1c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:24.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f1a7fc640 1 -- 172.21.15.131:0/2464837402 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c019070 con 0x7f6f24155240 2024-09-17T02:45:24.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f1a7fc640 1 -- 172.21.15.131:0/2464837402 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f1c00cc90 con 0x7f6f24155240 2024-09-17T02:45:24.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/2464837402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f2415a750 con 0x7f6f24155240 2024-09-17T02:45:24.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f1a7fc640 1 -- 172.21.15.131:0/2464837402 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c0146a0 con 0x7f6f24155240 2024-09-17T02:45:24.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/2464837402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f24156930 con 0x7f6f24155240 2024-09-17T02:45:24.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.466+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/2464837402 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f24156b90 con 0x7f6f24155240 2024-09-17T02:45:24.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.470+0000 7f6f1a7fc640 1 -- 172.21.15.131:0/2464837402 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6f1c01c070 con 0x7f6f24155240 2024-09-17T02:45:24.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.470+0000 7f6f1a7fc640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f08073200 0x7f6f08075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.470+0000 7f6f1a7fc640 1 -- 172.21.15.131:0/2464837402 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f6f1c0940f0 con 0x7f6f24155240 2024-09-17T02:45:24.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.470+0000 7f6f29f8c640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f08073200 0x7f6f08075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:24.470 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.470+0000 7f6f29f8c640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f08073200 0x7f6f08075710 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f200af000 tx=0x7f6f2001aed0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:24.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.474+0000 7f6f1a7fc640 1 -- 172.21.15.131:0/2464837402 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6f1c05f010 con 0x7f6f24155240 2024-09-17T02:45:24.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.634+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/2464837402 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f24167640 con 0x7f6f08073200 2024-09-17T02:45:24.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f1a7fc640 1 -- 172.21.15.131:0/2464837402 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251077 (secure 0 0 0) 0x7f6f24167640 con 0x7f6f08073200 2024-09-17T02:45:24.649 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:24.651 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:45:24.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f08073200 msgr2=0x7f6f08075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f08073200 0x7f6f08075710 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6f200af000 tx=0x7f6f2001aed0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/2464837402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f24155240 msgr2=0x7f6f24157d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f24155240 0x7f6f24157d70 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00b990 tx=0x7f6f1c00be50 comp rx=0 tx=0).stop 2024-09-17T02:45:24.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/2464837402 shutdown_connections 2024-09-17T02:45:24.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6f08073200 0x7f6f08075710 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6f24155240 0x7f6f24157d70 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6f24073970 0x7f6f24157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 --2- 172.21.15.131:0/2464837402 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6f2406e3e0 0x7f6f2414f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/2464837402 >> 172.21.15.131:0/2464837402 conn(0x7f6f2406eea0 msgr2=0x7f6f2415e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:24.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/2464837402 shutdown_connections 2024-09-17T02:45:24.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.650+0000 7f6f2c1f0640 1 -- 172.21.15.131:0/2464837402 wait complete. 2024-09-17T02:45:24.669 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-17T02:45:23.225560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258104,"kb_used_data":8584,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716616,"statfs":{"total":773094113280,"available":772829814784,"internally_reserved":0,"allocated":8790016,"data_stored":4617026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.173689"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135946+0000","last_change":"2024-09-17T02:45:20.134496+0000","last_active":"2024-09-17T02:45:21.135946+0000","last_peered":"2024-09-17T02:45:21.135946+0000","last_clean":"2024-09-17T02:45:21.135946+0000","last_became_active":"2024-09-17T02:45:20.134052+0000","last_became_peered":"2024-09-17T02:45:20.134052+0000","last_unstale":"2024-09-17T02:45:21.135946+0000","last_undegraded":"2024-09-17T02:45:21.135946+0000","last_fullsized":"2024-09-17T02:45:21.135946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:33.880550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.066330+0000","last_change":"2024-09-17T02:45:19.066330+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.066330+0000","last_undegraded":"2024-09-17T02:45:19.066330+0000","last_fullsized":"2024-09-17T02:45:19.066330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.071003+0000","last_change":"2024-09-17T02:45:19.071003+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.071003+0000","last_undegraded":"2024-09-17T02:45:19.071003+0000","last_fullsized":"2024-09-17T02:45:19.071003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.127671+0000","last_change":"2024-09-17T02:45:20.124901+0000","last_active":"2024-09-17T02:45:20.127671+0000","last_peered":"2024-09-17T02:45:20.127671+0000","last_clean":"2024-09-17T02:45:20.127671+0000","last_became_active":"2024-09-17T02:45:20.124392+0000","last_became_peered":"2024-09-17T02:45:20.124392+0000","last_unstale":"2024-09-17T02:45:20.127671+0000","last_undegraded":"2024-09-17T02:45:20.127671+0000","last_fullsized":"2024-09-17T02:45:20.127671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:56.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.157150+0000","last_change":"2024-09-17T02:45:20.151934+0000","last_active":"2024-09-17T02:45:20.157150+0000","last_peered":"2024-09-17T02:45:20.157150+0000","last_clean":"2024-09-17T02:45:20.157150+0000","last_became_active":"2024-09-17T02:45:20.149652+0000","last_became_peered":"2024-09-17T02:45:20.149652+0000","last_unstale":"2024-09-17T02:45:20.157150+0000","last_undegraded":"2024-09-17T02:45:20.157150+0000","last_fullsized":"2024-09-17T02:45:20.157150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:06.754098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.162147+0000","last_change":"2024-09-17T02:45:20.156609+0000","last_active":"2024-09-17T02:45:20.162147+0000","last_peered":"2024-09-17T02:45:20.162147+0000","last_clean":"2024-09-17T02:45:20.162147+0000","last_became_active":"2024-09-17T02:45:20.155997+0000","last_became_peered":"2024-09-17T02:45:20.155997+0000","last_unstale":"2024-09-17T02:45:20.162147+0000","last_undegraded":"2024-09-17T02:45:20.162147+0000","last_fullsized":"2024-09-17T02:45:20.162147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:49.213100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.141570+0000","last_change":"2024-09-17T02:45:20.138831+0000","last_active":"2024-09-17T02:45:20.141570+0000","last_peered":"2024-09-17T02:45:20.141570+0000","last_clean":"2024-09-17T02:45:20.141570+0000","last_became_active":"2024-09-17T02:45:20.138468+0000","last_became_peered":"2024-09-17T02:45:20.138468+0000","last_unstale":"2024-09-17T02:45:20.141570+0000","last_undegraded":"2024-09-17T02:45:20.141570+0000","last_fullsized":"2024-09-17T02:45:20.141570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:17.239517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137912+0000","last_change":"2024-09-17T02:45:20.151755+0000","last_active":"2024-09-17T02:45:21.137912+0000","last_peered":"2024-09-17T02:45:21.137912+0000","last_clean":"2024-09-17T02:45:21.137912+0000","last_became_active":"2024-09-17T02:45:20.139390+0000","last_became_peered":"2024-09-17T02:45:20.139390+0000","last_unstale":"2024-09-17T02:45:21.137912+0000","last_undegraded":"2024-09-17T02:45:21.137912+0000","last_fullsized":"2024-09-17T02:45:21.137912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:04.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130320+0000","last_change":"2024-09-17T02:45:20.128764+0000","last_active":"2024-09-17T02:45:20.130320+0000","last_peered":"2024-09-17T02:45:20.130320+0000","last_clean":"2024-09-17T02:45:20.130320+0000","last_became_active":"2024-09-17T02:45:20.128381+0000","last_became_peered":"2024-09-17T02:45:20.128381+0000","last_unstale":"2024-09-17T02:45:20.130320+0000","last_undegraded":"2024-09-17T02:45:20.130320+0000","last_fullsized":"2024-09-17T02:45:20.130320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:35.763836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135793+0000","last_change":"2024-09-17T02:45:20.134184+0000","last_active":"2024-09-17T02:45:21.135793+0000","last_peered":"2024-09-17T02:45:21.135793+0000","last_clean":"2024-09-17T02:45:21.135793+0000","last_became_active":"2024-09-17T02:45:20.133826+0000","last_became_peered":"2024-09-17T02:45:20.133826+0000","last_unstale":"2024-09-17T02:45:21.135793+0000","last_undegraded":"2024-09-17T02:45:21.135793+0000","last_fullsized":"2024-09-17T02:45:21.135793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:41.858427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.068073+0000","last_change":"2024-09-17T02:45:19.068073+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.068073+0000","last_undegraded":"2024-09-17T02:45:19.068073+0000","last_fullsized":"2024-09-17T02:45:19.068073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.146278+0000","last_change":"2024-09-17T02:45:20.137823+0000","last_active":"2024-09-17T02:45:20.146278+0000","last_peered":"2024-09-17T02:45:20.146278+0000","last_clean":"2024-09-17T02:45:20.146278+0000","last_became_active":"2024-09-17T02:45:20.137420+0000","last_became_peered":"2024-09-17T02:45:20.137420+0000","last_unstale":"2024-09-17T02:45:20.146278+0000","last_undegraded":"2024-09-17T02:45:20.146278+0000","last_fullsized":"2024-09-17T02:45:20.146278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:14.885437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.072804+0000","last_change":"2024-09-17T02:45:19.072804+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.072804+0000","last_undegraded":"2024-09-17T02:45:19.072804+0000","last_fullsized":"2024-09-17T02:45:19.072804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139109+0000","last_change":"2024-09-17T02:45:20.154359+0000","last_active":"2024-09-17T02:45:21.139109+0000","last_peered":"2024-09-17T02:45:21.139109+0000","last_clean":"2024-09-17T02:45:21.139109+0000","last_became_active":"2024-09-17T02:45:20.138831+0000","last_became_peered":"2024-09-17T02:45:20.138831+0000","last_unstale":"2024-09-17T02:45:21.139109+0000","last_undegraded":"2024-09-17T02:45:21.139109+0000","last_fullsized":"2024-09-17T02:45:21.139109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:26.541024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151644+0000","last_change":"2024-09-17T02:45:20.136343+0000","last_active":"2024-09-17T02:45:20.151644+0000","last_peered":"2024-09-17T02:45:20.151644+0000","last_clean":"2024-09-17T02:45:20.151644+0000","last_became_active":"2024-09-17T02:45:20.135036+0000","last_became_peered":"2024-09-17T02:45:20.135036+0000","last_unstale":"2024-09-17T02:45:20.151644+0000","last_undegraded":"2024-09-17T02:45:20.151644+0000","last_fullsized":"2024-09-17T02:45:20.151644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:15.507022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134377+0000","last_change":"2024-09-17T02:45:20.133650+0000","last_active":"2024-09-17T02:45:20.134377+0000","last_peered":"2024-09-17T02:45:20.134377+0000","last_clean":"2024-09-17T02:45:20.134377+0000","last_became_active":"2024-09-17T02:45:20.133246+0000","last_became_peered":"2024-09-17T02:45:20.133246+0000","last_unstale":"2024-09-17T02:45:20.134377+0000","last_undegraded":"2024-09-17T02:45:20.134377+0000","last_fullsized":"2024-09-17T02:45:20.134377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:31.528537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138676+0000","last_change":"2024-09-17T02:45:20.138431+0000","last_active":"2024-09-17T02:45:21.138676+0000","last_peered":"2024-09-17T02:45:21.138676+0000","last_clean":"2024-09-17T02:45:21.138676+0000","last_became_active":"2024-09-17T02:45:20.137633+0000","last_became_peered":"2024-09-17T02:45:20.137633+0000","last_unstale":"2024-09-17T02:45:21.138676+0000","last_undegraded":"2024-09-17T02:45:21.138676+0000","last_fullsized":"2024-09-17T02:45:21.138676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:25.489158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.065085+0000","last_change":"2024-09-17T02:45:19.065085+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.065085+0000","last_undegraded":"2024-09-17T02:45:19.065085+0000","last_fullsized":"2024-09-17T02:45:19.065085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138525+0000","last_change":"2024-09-17T02:45:20.138096+0000","last_active":"2024-09-17T02:45:21.138525+0000","last_peered":"2024-09-17T02:45:21.138525+0000","last_clean":"2024-09-17T02:45:21.138525+0000","last_became_active":"2024-09-17T02:45:20.137246+0000","last_became_peered":"2024-09-17T02:45:20.137246+0000","last_unstale":"2024-09-17T02:45:21.138525+0000","last_undegraded":"2024-09-17T02:45:21.138525+0000","last_fullsized":"2024-09-17T02:45:21.138525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:08.405979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.148092+0000","last_change":"2024-09-17T02:45:20.144454+0000","last_active":"2024-09-17T02:45:20.148092+0000","last_peered":"2024-09-17T02:45:20.148092+0000","last_clean":"2024-09-17T02:45:20.148092+0000","last_became_active":"2024-09-17T02:45:20.143671+0000","last_became_peered":"2024-09-17T02:45:20.143671+0000","last_unstale":"2024-09-17T02:45:20.148092+0000","last_undegraded":"2024-09-17T02:45:20.148092+0000","last_fullsized":"2024-09-17T02:45:20.148092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:26.359990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.147886+0000","last_change":"2024-09-17T02:45:20.140711+0000","last_active":"2024-09-17T02:45:20.147886+0000","last_peered":"2024-09-17T02:45:20.147886+0000","last_clean":"2024-09-17T02:45:20.147886+0000","last_became_active":"2024-09-17T02:45:20.140390+0000","last_became_peered":"2024-09-17T02:45:20.140390+0000","last_unstale":"2024-09-17T02:45:20.147886+0000","last_undegraded":"2024-09-17T02:45:20.147886+0000","last_fullsized":"2024-09-17T02:45:20.147886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.473725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.069206+0000","last_change":"2024-09-17T02:45:19.069206+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.069206+0000","last_undegraded":"2024-09-17T02:45:19.069206+0000","last_fullsized":"2024-09-17T02:45:19.069206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135331+0000","last_change":"2024-09-17T02:45:20.120377+0000","last_active":"2024-09-17T02:45:21.135331+0000","last_peered":"2024-09-17T02:45:21.135331+0000","last_clean":"2024-09-17T02:45:21.135331+0000","last_became_active":"2024-09-17T02:45:20.119336+0000","last_became_peered":"2024-09-17T02:45:20.119336+0000","last_unstale":"2024-09-17T02:45:21.135331+0000","last_undegraded":"2024-09-17T02:45:21.135331+0000","last_fullsized":"2024-09-17T02:45:21.135331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:43.174152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.073370+0000","last_change":"2024-09-17T02:45:19.073370+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.073370+0000","last_undegraded":"2024-09-17T02:45:19.073370+0000","last_fullsized":"2024-09-17T02:45:19.073370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.143641+0000","last_change":"2024-09-17T02:45:20.136877+0000","last_active":"2024-09-17T02:45:20.143641+0000","last_peered":"2024-09-17T02:45:20.143641+0000","last_clean":"2024-09-17T02:45:20.143641+0000","last_became_active":"2024-09-17T02:45:20.136227+0000","last_became_peered":"2024-09-17T02:45:20.136227+0000","last_unstale":"2024-09-17T02:45:20.143641+0000","last_undegraded":"2024-09-17T02:45:20.143641+0000","last_fullsized":"2024-09-17T02:45:20.143641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:06.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.131033+0000","last_change":"2024-09-17T02:45:20.129949+0000","last_active":"2024-09-17T02:45:20.131033+0000","last_peered":"2024-09-17T02:45:20.131033+0000","last_clean":"2024-09-17T02:45:20.131033+0000","last_became_active":"2024-09-17T02:45:20.129519+0000","last_became_peered":"2024-09-17T02:45:20.129519+0000","last_unstale":"2024-09-17T02:45:20.131033+0000","last_undegraded":"2024-09-17T02:45:20.131033+0000","last_fullsized":"2024-09-17T02:45:20.131033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:53.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.127890+0000","last_change":"2024-09-17T02:45:20.125488+0000","last_active":"2024-09-17T02:45:20.127890+0000","last_peered":"2024-09-17T02:45:20.127890+0000","last_clean":"2024-09-17T02:45:20.127890+0000","last_became_active":"2024-09-17T02:45:20.124765+0000","last_became_peered":"2024-09-17T02:45:20.124765+0000","last_unstale":"2024-09-17T02:45:20.127890+0000","last_undegraded":"2024-09-17T02:45:20.127890+0000","last_fullsized":"2024-09-17T02:45:20.127890+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:41.858427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.125889+0000","last_change":"2024-09-17T02:45:20.121902+0000","last_active":"2024-09-17T02:45:20.125889+0000","last_peered":"2024-09-17T02:45:20.125889+0000","last_clean":"2024-09-17T02:45:20.125889+0000","last_became_active":"2024-09-17T02:45:20.121274+0000","last_became_peered":"2024-09-17T02:45:20.121274+0000","last_unstale":"2024-09-17T02:45:20.125889+0000","last_undegraded":"2024-09-17T02:45:20.125889+0000","last_fullsized":"2024-09-17T02:45:20.125889+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:19.026083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.150174+0000","last_change":"2024-09-17T02:45:20.147039+0000","last_active":"2024-09-17T02:45:20.150174+0000","last_peered":"2024-09-17T02:45:20.150174+0000","last_clean":"2024-09-17T02:45:20.150174+0000","last_became_active":"2024-09-17T02:45:20.145977+0000","last_became_peered":"2024-09-17T02:45:20.145977+0000","last_unstale":"2024-09-17T02:45:20.150174+0000","last_undegraded":"2024-09-17T02:45:20.150174+0000","last_fullsized":"2024-09-17T02:45:20.150174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:25.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135630+0000","last_change":"2024-09-17T02:45:20.134028+0000","last_active":"2024-09-17T02:45:21.135630+0000","last_peered":"2024-09-17T02:45:21.135630+0000","last_clean":"2024-09-17T02:45:21.135630+0000","last_became_active":"2024-09-17T02:45:20.133628+0000","last_became_peered":"2024-09-17T02:45:20.133628+0000","last_unstale":"2024-09-17T02:45:21.135630+0000","last_undegraded":"2024-09-17T02:45:21.135630+0000","last_fullsized":"2024-09-17T02:45:21.135630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:56.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.074374+0000","last_change":"2024-09-17T02:45:19.074374+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.074374+0000","last_undegraded":"2024-09-17T02:45:19.074374+0000","last_fullsized":"2024-09-17T02:45:19.074374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.129861+0000","last_change":"2024-09-17T02:45:20.127695+0000","last_active":"2024-09-17T02:45:20.129861+0000","last_peered":"2024-09-17T02:45:20.129861+0000","last_clean":"2024-09-17T02:45:20.129861+0000","last_became_active":"2024-09-17T02:45:20.127323+0000","last_became_peered":"2024-09-17T02:45:20.127323+0000","last_unstale":"2024-09-17T02:45:20.129861+0000","last_undegraded":"2024-09-17T02:45:20.129861+0000","last_fullsized":"2024-09-17T02:45:20.129861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:42.063380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.132767+0000","last_change":"2024-09-17T02:45:20.122630+0000","last_active":"2024-09-17T02:45:20.132767+0000","last_peered":"2024-09-17T02:45:20.132767+0000","last_clean":"2024-09-17T02:45:20.132767+0000","last_became_active":"2024-09-17T02:45:20.120596+0000","last_became_peered":"2024-09-17T02:45:20.120596+0000","last_unstale":"2024-09-17T02:45:20.132767+0000","last_undegraded":"2024-09-17T02:45:20.132767+0000","last_fullsized":"2024-09-17T02:45:20.132767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:16.342312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"78'4","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165404+0000","last_change":"2024-09-17T02:45:19.105415+0000","last_active":"2024-09-17T02:45:20.165404+0000","last_peered":"2024-09-17T02:45:20.165404+0000","last_clean":"2024-09-17T02:45:20.165404+0000","last_became_active":"2024-09-17T02:45:19.104799+0000","last_became_peered":"2024-09-17T02:45:19.104799+0000","last_unstale":"2024-09-17T02:45:20.165404+0000","last_undegraded":"2024-09-17T02:45:20.165404+0000","last_fullsized":"2024-09-17T02:45:20.165404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:15:52.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.153720+0000","last_change":"2024-09-17T02:45:20.151127+0000","last_active":"2024-09-17T02:45:20.153720+0000","last_peered":"2024-09-17T02:45:20.153720+0000","last_clean":"2024-09-17T02:45:20.153720+0000","last_became_active":"2024-09-17T02:45:20.150324+0000","last_became_peered":"2024-09-17T02:45:20.150324+0000","last_unstale":"2024-09-17T02:45:20.153720+0000","last_undegraded":"2024-09-17T02:45:20.153720+0000","last_fullsized":"2024-09-17T02:45:20.153720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:26.406192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151669+0000","last_change":"2024-09-17T02:45:19.079006+0000","last_active":"2024-09-17T02:45:20.151669+0000","last_peered":"2024-09-17T02:45:20.151669+0000","last_clean":"2024-09-17T02:45:20.151669+0000","last_became_active":"2024-09-17T02:45:19.078512+0000","last_became_peered":"2024-09-17T02:45:19.078512+0000","last_unstale":"2024-09-17T02:45:20.151669+0000","last_undegraded":"2024-09-17T02:45:20.151669+0000","last_fullsized":"2024-09-17T02:45:20.151669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:48.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134859+0000","last_change":"2024-09-17T02:45:20.131374+0000","last_active":"2024-09-17T02:45:20.134859+0000","last_peered":"2024-09-17T02:45:20.134859+0000","last_clean":"2024-09-17T02:45:20.134859+0000","last_became_active":"2024-09-17T02:45:20.129652+0000","last_became_peered":"2024-09-17T02:45:20.129652+0000","last_unstale":"2024-09-17T02:45:20.134859+0000","last_undegraded":"2024-09-17T02:45:20.134859+0000","last_fullsized":"2024-09-17T02:45:20.134859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:02.100162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136882+0000","last_change":"2024-09-17T02:45:19.078238+0000","last_active":"2024-09-17T02:45:21.136882+0000","last_peered":"2024-09-17T02:45:21.136882+0000","last_clean":"2024-09-17T02:45:21.136882+0000","last_became_active":"2024-09-17T02:45:19.077495+0000","last_became_peered":"2024-09-17T02:45:19.077495+0000","last_unstale":"2024-09-17T02:45:21.136882+0000","last_undegraded":"2024-09-17T02:45:21.136882+0000","last_fullsized":"2024-09-17T02:45:21.136882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:25.395603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130326+0000","last_change":"2024-09-17T02:45:20.127140+0000","last_active":"2024-09-17T02:45:20.130326+0000","last_peered":"2024-09-17T02:45:20.130326+0000","last_clean":"2024-09-17T02:45:20.130326+0000","last_became_active":"2024-09-17T02:45:20.126455+0000","last_became_peered":"2024-09-17T02:45:20.126455+0000","last_unstale":"2024-09-17T02:45:20.130326+0000","last_undegraded":"2024-09-17T02:45:20.130326+0000","last_fullsized":"2024-09-17T02:45:20.130326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:33.880550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"78'2","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149384+0000","last_change":"2024-09-17T02:45:19.092392+0000","last_active":"2024-09-17T02:45:20.149384+0000","last_peered":"2024-09-17T02:45:20.149384+0000","last_clean":"2024-09-17T02:45:20.149384+0000","last_became_active":"2024-09-17T02:45:19.091367+0000","last_became_peered":"2024-09-17T02:45:19.091367+0000","last_unstale":"2024-09-17T02:45:20.149384+0000","last_undegraded":"2024-09-17T02:45:20.149384+0000","last_fullsized":"2024-09-17T02:45:20.149384+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:57.648928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.143331+0000","last_change":"2024-09-17T02:45:20.134096+0000","last_active":"2024-09-17T02:45:20.143331+0000","last_peered":"2024-09-17T02:45:20.143331+0000","last_clean":"2024-09-17T02:45:20.143331+0000","last_became_active":"2024-09-17T02:45:20.133166+0000","last_became_peered":"2024-09-17T02:45:20.133166+0000","last_unstale":"2024-09-17T02:45:20.143331+0000","last_undegraded":"2024-09-17T02:45:20.143331+0000","last_fullsized":"2024-09-17T02:45:20.143331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:17.390929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.147932+0000","last_change":"2024-09-17T02:45:19.095580+0000","last_active":"2024-09-17T02:45:20.147932+0000","last_peered":"2024-09-17T02:45:20.147932+0000","last_clean":"2024-09-17T02:45:20.147932+0000","last_became_active":"2024-09-17T02:45:19.094437+0000","last_became_peered":"2024-09-17T02:45:19.094437+0000","last_unstale":"2024-09-17T02:45:20.147932+0000","last_undegraded":"2024-09-17T02:45:20.147932+0000","last_fullsized":"2024-09-17T02:45:20.147932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:59.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.066618+0000","last_change":"2024-09-17T02:45:19.066618+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.066618+0000","last_undegraded":"2024-09-17T02:45:19.066618+0000","last_fullsized":"2024-09-17T02:45:19.066618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165349+0000","last_change":"2024-09-17T02:45:19.086585+0000","last_active":"2024-09-17T02:45:20.165349+0000","last_peered":"2024-09-17T02:45:20.165349+0000","last_clean":"2024-09-17T02:45:20.165349+0000","last_became_active":"2024-09-17T02:45:19.086060+0000","last_became_peered":"2024-09-17T02:45:19.086060+0000","last_unstale":"2024-09-17T02:45:20.165349+0000","last_undegraded":"2024-09-17T02:45:20.165349+0000","last_fullsized":"2024-09-17T02:45:20.165349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:06.921612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.128800+0000","last_change":"2024-09-17T02:45:20.122719+0000","last_active":"2024-09-17T02:45:20.128800+0000","last_peered":"2024-09-17T02:45:20.128800+0000","last_clean":"2024-09-17T02:45:20.128800+0000","last_became_active":"2024-09-17T02:45:20.120568+0000","last_became_peered":"2024-09-17T02:45:20.120568+0000","last_unstale":"2024-09-17T02:45:20.128800+0000","last_undegraded":"2024-09-17T02:45:20.128800+0000","last_fullsized":"2024-09-17T02:45:20.128800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.038462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137866+0000","last_change":"2024-09-17T02:45:19.093274+0000","last_active":"2024-09-17T02:45:21.137866+0000","last_peered":"2024-09-17T02:45:21.137866+0000","last_clean":"2024-09-17T02:45:21.137866+0000","last_became_active":"2024-09-17T02:45:19.091525+0000","last_became_peered":"2024-09-17T02:45:19.091525+0000","last_unstale":"2024-09-17T02:45:21.137866+0000","last_undegraded":"2024-09-17T02:45:21.137866+0000","last_fullsized":"2024-09-17T02:45:21.137866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:05.164896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.136305+0000","last_change":"2024-09-17T02:45:20.132688+0000","last_active":"2024-09-17T02:45:20.136305+0000","last_peered":"2024-09-17T02:45:20.136305+0000","last_clean":"2024-09-17T02:45:20.136305+0000","last_became_active":"2024-09-17T02:45:20.132335+0000","last_became_peered":"2024-09-17T02:45:20.132335+0000","last_unstale":"2024-09-17T02:45:20.136305+0000","last_undegraded":"2024-09-17T02:45:20.136305+0000","last_fullsized":"2024-09-17T02:45:20.136305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:01.061964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.094857+0000","last_change":"2024-09-17T02:45:19.086961+0000","last_active":"2024-09-17T02:45:19.094857+0000","last_peered":"2024-09-17T02:45:19.094857+0000","last_clean":"2024-09-17T02:45:19.094857+0000","last_became_active":"2024-09-17T02:45:19.084997+0000","last_became_peered":"2024-09-17T02:45:19.084997+0000","last_unstale":"2024-09-17T02:45:19.094857+0000","last_undegraded":"2024-09-17T02:45:19.094857+0000","last_fullsized":"2024-09-17T02:45:19.094857+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:07.303479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.071853+0000","last_change":"2024-09-17T02:45:19.071853+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.071853+0000","last_undegraded":"2024-09-17T02:45:19.071853+0000","last_fullsized":"2024-09-17T02:45:19.071853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.103868+0000","last_change":"2024-09-17T02:45:19.081505+0000","last_active":"2024-09-17T02:45:20.103868+0000","last_peered":"2024-09-17T02:45:20.103868+0000","last_clean":"2024-09-17T02:45:20.103868+0000","last_became_active":"2024-09-17T02:45:19.081196+0000","last_became_peered":"2024-09-17T02:45:19.081196+0000","last_unstale":"2024-09-17T02:45:20.103868+0000","last_undegraded":"2024-09-17T02:45:20.103868+0000","last_fullsized":"2024-09-17T02:45:20.103868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:33.638898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.146953+0000","last_change":"2024-09-17T02:45:20.139575+0000","last_active":"2024-09-17T02:45:20.146953+0000","last_peered":"2024-09-17T02:45:20.146953+0000","last_clean":"2024-09-17T02:45:20.146953+0000","last_became_active":"2024-09-17T02:45:20.137448+0000","last_became_peered":"2024-09-17T02:45:20.137448+0000","last_unstale":"2024-09-17T02:45:20.146953+0000","last_undegraded":"2024-09-17T02:45:20.146953+0000","last_fullsized":"2024-09-17T02:45:20.146953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:28.895745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135457+0000","last_change":"2024-09-17T02:45:19.074384+0000","last_active":"2024-09-17T02:45:21.135457+0000","last_peered":"2024-09-17T02:45:21.135457+0000","last_clean":"2024-09-17T02:45:21.135457+0000","last_became_active":"2024-09-17T02:45:19.073958+0000","last_became_peered":"2024-09-17T02:45:19.073958+0000","last_unstale":"2024-09-17T02:45:21.135457+0000","last_undegraded":"2024-09-17T02:45:21.135457+0000","last_fullsized":"2024-09-17T02:45:21.135457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:36.384573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139981+0000","last_change":"2024-09-17T02:45:20.152408+0000","last_active":"2024-09-17T02:45:21.139981+0000","last_peered":"2024-09-17T02:45:21.139981+0000","last_clean":"2024-09-17T02:45:21.139981+0000","last_became_active":"2024-09-17T02:45:20.137766+0000","last_became_peered":"2024-09-17T02:45:20.137766+0000","last_unstale":"2024-09-17T02:45:21.139981+0000","last_undegraded":"2024-09-17T02:45:21.139981+0000","last_fullsized":"2024-09-17T02:45:21.139981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:54.493799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"78'3","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165499+0000","last_change":"2024-09-17T02:45:19.101534+0000","last_active":"2024-09-17T02:45:20.165499+0000","last_peered":"2024-09-17T02:45:20.165499+0000","last_clean":"2024-09-17T02:45:20.165499+0000","last_became_active":"2024-09-17T02:45:19.099790+0000","last_became_peered":"2024-09-17T02:45:19.099790+0000","last_unstale":"2024-09-17T02:45:20.165499+0000","last_undegraded":"2024-09-17T02:45:20.165499+0000","last_fullsized":"2024-09-17T02:45:20.165499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:59.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.155826+0000","last_change":"2024-09-17T02:45:20.154562+0000","last_active":"2024-09-17T02:45:20.155826+0000","last_peered":"2024-09-17T02:45:20.155826+0000","last_clean":"2024-09-17T02:45:20.155826+0000","last_became_active":"2024-09-17T02:45:20.153866+0000","last_became_peered":"2024-09-17T02:45:20.153866+0000","last_unstale":"2024-09-17T02:45:20.155826+0000","last_undegraded":"2024-09-17T02:45:20.155826+0000","last_fullsized":"2024-09-17T02:45:20.155826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:31.599827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"78'6","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.154097+0000","last_change":"2024-09-17T02:45:19.079828+0000","last_active":"2024-09-17T02:45:20.154097+0000","last_peered":"2024-09-17T02:45:20.154097+0000","last_clean":"2024-09-17T02:45:20.154097+0000","last_became_active":"2024-09-17T02:45:19.079230+0000","last_became_peered":"2024-09-17T02:45:19.079230+0000","last_unstale":"2024-09-17T02:45:20.154097+0000","last_undegraded":"2024-09-17T02:45:20.154097+0000","last_fullsized":"2024-09-17T02:45:20.154097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:46.676590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.133641+0000","last_change":"2024-09-17T02:45:20.130719+0000","last_active":"2024-09-17T02:45:20.133641+0000","last_peered":"2024-09-17T02:45:20.133641+0000","last_clean":"2024-09-17T02:45:20.133641+0000","last_became_active":"2024-09-17T02:45:20.130351+0000","last_became_peered":"2024-09-17T02:45:20.130351+0000","last_unstale":"2024-09-17T02:45:20.133641+0000","last_undegraded":"2024-09-17T02:45:20.133641+0000","last_fullsized":"2024-09-17T02:45:20.133641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:15.730199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136247+0000","last_change":"2024-09-17T02:45:19.085733+0000","last_active":"2024-09-17T02:45:21.136247+0000","last_peered":"2024-09-17T02:45:21.136247+0000","last_clean":"2024-09-17T02:45:21.136247+0000","last_became_active":"2024-09-17T02:45:19.085445+0000","last_became_peered":"2024-09-17T02:45:19.085445+0000","last_unstale":"2024-09-17T02:45:21.136247+0000","last_undegraded":"2024-09-17T02:45:21.136247+0000","last_fullsized":"2024-09-17T02:45:21.136247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:35.839507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136624+0000","last_change":"2024-09-17T02:45:20.127481+0000","last_active":"2024-09-17T02:45:21.136624+0000","last_peered":"2024-09-17T02:45:21.136624+0000","last_clean":"2024-09-17T02:45:21.136624+0000","last_became_active":"2024-09-17T02:45:20.127085+0000","last_became_peered":"2024-09-17T02:45:20.127085+0000","last_unstale":"2024-09-17T02:45:21.136624+0000","last_undegraded":"2024-09-17T02:45:21.136624+0000","last_fullsized":"2024-09-17T02:45:21.136624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:42.063380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.094987+0000","last_change":"2024-09-17T02:45:19.094438+0000","last_active":"2024-09-17T02:45:20.094987+0000","last_peered":"2024-09-17T02:45:20.094987+0000","last_clean":"2024-09-17T02:45:20.094987+0000","last_became_active":"2024-09-17T02:45:19.093709+0000","last_became_peered":"2024-09-17T02:45:19.093709+0000","last_unstale":"2024-09-17T02:45:20.094987+0000","last_undegraded":"2024-09-17T02:45:20.094987+0000","last_fullsized":"2024-09-17T02:45:20.094987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:45.773745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.135664+0000","last_change":"2024-09-17T02:45:20.131160+0000","last_active":"2024-09-17T02:45:20.135664+0000","last_peered":"2024-09-17T02:45:20.135664+0000","last_clean":"2024-09-17T02:45:20.135664+0000","last_became_active":"2024-09-17T02:45:20.129742+0000","last_became_peered":"2024-09-17T02:45:20.129742+0000","last_unstale":"2024-09-17T02:45:20.135664+0000","last_undegraded":"2024-09-17T02:45:20.135664+0000","last_fullsized":"2024-09-17T02:45:20.135664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:11.528887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.113545+0000","last_change":"2024-09-17T02:45:19.105626+0000","last_active":"2024-09-17T02:45:20.113545+0000","last_peered":"2024-09-17T02:45:20.113545+0000","last_clean":"2024-09-17T02:45:20.113545+0000","last_became_active":"2024-09-17T02:45:19.104904+0000","last_became_peered":"2024-09-17T02:45:19.104904+0000","last_unstale":"2024-09-17T02:45:20.113545+0000","last_undegraded":"2024-09-17T02:45:20.113545+0000","last_fullsized":"2024-09-17T02:45:20.113545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:38.203466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135464+0000","last_change":"2024-09-17T02:45:20.138086+0000","last_active":"2024-09-17T02:45:21.135464+0000","last_peered":"2024-09-17T02:45:21.135464+0000","last_clean":"2024-09-17T02:45:21.135464+0000","last_became_active":"2024-09-17T02:45:20.134972+0000","last_became_peered":"2024-09-17T02:45:20.134972+0000","last_unstale":"2024-09-17T02:45:21.135464+0000","last_undegraded":"2024-09-17T02:45:21.135464+0000","last_fullsized":"2024-09-17T02:45:21.135464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:18.254883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"78'8","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.164732+0000","last_change":"2024-09-17T02:45:19.102331+0000","last_active":"2024-09-17T02:45:20.164732+0000","last_peered":"2024-09-17T02:45:20.164732+0000","last_clean":"2024-09-17T02:45:20.164732+0000","last_became_active":"2024-09-17T02:45:19.101721+0000","last_became_peered":"2024-09-17T02:45:19.101721+0000","last_unstale":"2024-09-17T02:45:20.164732+0000","last_undegraded":"2024-09-17T02:45:20.164732+0000","last_fullsized":"2024-09-17T02:45:20.164732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:45.773745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149140+0000","last_change":"2024-09-17T02:45:20.131992+0000","last_active":"2024-09-17T02:45:20.149140+0000","last_peered":"2024-09-17T02:45:20.149140+0000","last_clean":"2024-09-17T02:45:20.149140+0000","last_became_active":"2024-09-17T02:45:20.126966+0000","last_became_peered":"2024-09-17T02:45:20.126966+0000","last_unstale":"2024-09-17T02:45:20.149140+0000","last_undegraded":"2024-09-17T02:45:20.149140+0000","last_fullsized":"2024-09-17T02:45:20.149140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:07.337735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.164565+0000","last_change":"2024-09-17T02:45:19.095009+0000","last_active":"2024-09-17T02:45:20.164565+0000","last_peered":"2024-09-17T02:45:20.164565+0000","last_clean":"2024-09-17T02:45:20.164565+0000","last_became_active":"2024-09-17T02:45:19.094610+0000","last_became_peered":"2024-09-17T02:45:19.094610+0000","last_unstale":"2024-09-17T02:45:20.164565+0000","last_undegraded":"2024-09-17T02:45:20.164565+0000","last_fullsized":"2024-09-17T02:45:20.164565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:42.163563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.073129+0000","last_change":"2024-09-17T02:45:19.073129+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.073129+0000","last_undegraded":"2024-09-17T02:45:19.073129+0000","last_fullsized":"2024-09-17T02:45:19.073129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"78'2","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151200+0000","last_change":"2024-09-17T02:45:19.089116+0000","last_active":"2024-09-17T02:45:20.151200+0000","last_peered":"2024-09-17T02:45:20.151200+0000","last_clean":"2024-09-17T02:45:20.151200+0000","last_became_active":"2024-09-17T02:45:19.088667+0000","last_became_peered":"2024-09-17T02:45:19.088667+0000","last_unstale":"2024-09-17T02:45:20.151200+0000","last_undegraded":"2024-09-17T02:45:20.151200+0000","last_fullsized":"2024-09-17T02:45:20.151200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:54:35.726780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139554+0000","last_change":"2024-09-17T02:45:20.162326+0000","last_active":"2024-09-17T02:45:21.139554+0000","last_peered":"2024-09-17T02:45:21.139554+0000","last_clean":"2024-09-17T02:45:21.139554+0000","last_became_active":"2024-09-17T02:45:20.161849+0000","last_became_peered":"2024-09-17T02:45:20.161849+0000","last_unstale":"2024-09-17T02:45:21.139554+0000","last_undegraded":"2024-09-17T02:45:21.139554+0000","last_fullsized":"2024-09-17T02:45:21.139554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:53.540992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.161729+0000","last_change":"2024-09-17T02:45:19.099466+0000","last_active":"2024-09-17T02:45:20.161729+0000","last_peered":"2024-09-17T02:45:20.161729+0000","last_clean":"2024-09-17T02:45:20.161729+0000","last_became_active":"2024-09-17T02:45:19.098488+0000","last_became_peered":"2024-09-17T02:45:19.098488+0000","last_unstale":"2024-09-17T02:45:20.161729+0000","last_undegraded":"2024-09-17T02:45:20.161729+0000","last_fullsized":"2024-09-17T02:45:20.161729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:32.496598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.157082+0000","last_change":"2024-09-17T02:45:20.152366+0000","last_active":"2024-09-17T02:45:20.157082+0000","last_peered":"2024-09-17T02:45:20.157082+0000","last_clean":"2024-09-17T02:45:20.157082+0000","last_became_active":"2024-09-17T02:45:20.150703+0000","last_became_peered":"2024-09-17T02:45:20.150703+0000","last_unstale":"2024-09-17T02:45:20.157082+0000","last_undegraded":"2024-09-17T02:45:20.157082+0000","last_fullsized":"2024-09-17T02:45:20.157082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:04.950418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.115551+0000","last_change":"2024-09-17T02:45:19.105629+0000","last_active":"2024-09-17T02:45:20.115551+0000","last_peered":"2024-09-17T02:45:20.115551+0000","last_clean":"2024-09-17T02:45:20.115551+0000","last_became_active":"2024-09-17T02:45:19.104942+0000","last_became_peered":"2024-09-17T02:45:19.104942+0000","last_unstale":"2024-09-17T02:45:20.115551+0000","last_undegraded":"2024-09-17T02:45:20.115551+0000","last_fullsized":"2024-09-17T02:45:20.115551+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:55.509905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.150727+0000","last_change":"2024-09-17T02:45:20.141760+0000","last_active":"2024-09-17T02:45:20.150727+0000","last_peered":"2024-09-17T02:45:20.150727+0000","last_clean":"2024-09-17T02:45:20.150727+0000","last_became_active":"2024-09-17T02:45:20.141069+0000","last_became_peered":"2024-09-17T02:45:20.141069+0000","last_unstale":"2024-09-17T02:45:20.150727+0000","last_undegraded":"2024-09-17T02:45:20.150727+0000","last_fullsized":"2024-09-17T02:45:20.150727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:36.850096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.106245+0000","last_change":"2024-09-17T02:45:19.101863+0000","last_active":"2024-09-17T02:45:19.106245+0000","last_peered":"2024-09-17T02:45:19.106245+0000","last_clean":"2024-09-17T02:45:19.106245+0000","last_became_active":"2024-09-17T02:45:19.101175+0000","last_became_peered":"2024-09-17T02:45:19.101175+0000","last_unstale":"2024-09-17T02:45:19.106245+0000","last_undegraded":"2024-09-17T02:45:19.106245+0000","last_fullsized":"2024-09-17T02:45:19.106245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:02.512375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149604+0000","last_change":"2024-09-17T02:45:20.127175+0000","last_active":"2024-09-17T02:45:20.149604+0000","last_peered":"2024-09-17T02:45:20.149604+0000","last_clean":"2024-09-17T02:45:20.149604+0000","last_became_active":"2024-09-17T02:45:20.126335+0000","last_became_peered":"2024-09-17T02:45:20.126335+0000","last_unstale":"2024-09-17T02:45:20.149604+0000","last_undegraded":"2024-09-17T02:45:20.149604+0000","last_fullsized":"2024-09-17T02:45:20.149604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:52.206612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138147+0000","last_change":"2024-09-17T02:45:19.093429+0000","last_active":"2024-09-17T02:45:21.138147+0000","last_peered":"2024-09-17T02:45:21.138147+0000","last_clean":"2024-09-17T02:45:21.138147+0000","last_became_active":"2024-09-17T02:45:19.092623+0000","last_became_peered":"2024-09-17T02:45:19.092623+0000","last_unstale":"2024-09-17T02:45:21.138147+0000","last_undegraded":"2024-09-17T02:45:21.138147+0000","last_fullsized":"2024-09-17T02:45:21.138147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:25.349401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.129570+0000","last_change":"2024-09-17T02:45:20.128671+0000","last_active":"2024-09-17T02:45:20.129570+0000","last_peered":"2024-09-17T02:45:20.129570+0000","last_clean":"2024-09-17T02:45:20.129570+0000","last_became_active":"2024-09-17T02:45:20.128268+0000","last_became_peered":"2024-09-17T02:45:20.128268+0000","last_unstale":"2024-09-17T02:45:20.129570+0000","last_undegraded":"2024-09-17T02:45:20.129570+0000","last_fullsized":"2024-09-17T02:45:20.129570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:05.898373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.096027+0000","last_change":"2024-09-17T02:45:19.089801+0000","last_active":"2024-09-17T02:45:19.096027+0000","last_peered":"2024-09-17T02:45:19.096027+0000","last_clean":"2024-09-17T02:45:19.096027+0000","last_became_active":"2024-09-17T02:45:19.083865+0000","last_became_peered":"2024-09-17T02:45:19.083865+0000","last_unstale":"2024-09-17T02:45:19.096027+0000","last_undegraded":"2024-09-17T02:45:19.096027+0000","last_fullsized":"2024-09-17T02:45:19.096027+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:05.560812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.122736+0000","last_change":"2024-09-17T02:45:20.121012+0000","last_active":"2024-09-17T02:45:20.122736+0000","last_peered":"2024-09-17T02:45:20.122736+0000","last_clean":"2024-09-17T02:45:20.122736+0000","last_became_active":"2024-09-17T02:45:20.120133+0000","last_became_peered":"2024-09-17T02:45:20.120133+0000","last_unstale":"2024-09-17T02:45:20.122736+0000","last_undegraded":"2024-09-17T02:45:20.122736+0000","last_fullsized":"2024-09-17T02:45:20.122736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:18.254883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135613+0000","last_change":"2024-09-17T02:45:19.082147+0000","last_active":"2024-09-17T02:45:21.135613+0000","last_peered":"2024-09-17T02:45:21.135613+0000","last_clean":"2024-09-17T02:45:21.135613+0000","last_became_active":"2024-09-17T02:45:19.081729+0000","last_became_peered":"2024-09-17T02:45:19.081729+0000","last_unstale":"2024-09-17T02:45:21.135613+0000","last_undegraded":"2024-09-17T02:45:21.135613+0000","last_fullsized":"2024-09-17T02:45:21.135613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:27.939745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135204+0000","last_change":"2024-09-17T02:45:20.136717+0000","last_active":"2024-09-17T02:45:21.135204+0000","last_peered":"2024-09-17T02:45:21.135204+0000","last_clean":"2024-09-17T02:45:21.135204+0000","last_became_active":"2024-09-17T02:45:20.136343+0000","last_became_peered":"2024-09-17T02:45:20.136343+0000","last_unstale":"2024-09-17T02:45:21.135204+0000","last_undegraded":"2024-09-17T02:45:21.135204+0000","last_fullsized":"2024-09-17T02:45:21.135204+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:44.913811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137044+0000","last_change":"2024-09-17T02:45:19.102757+0000","last_active":"2024-09-17T02:45:21.137044+0000","last_peered":"2024-09-17T02:45:21.137044+0000","last_clean":"2024-09-17T02:45:21.137044+0000","last_became_active":"2024-09-17T02:45:19.100744+0000","last_became_peered":"2024-09-17T02:45:19.100744+0000","last_unstale":"2024-09-17T02:45:21.137044+0000","last_undegraded":"2024-09-17T02:45:21.137044+0000","last_fullsized":"2024-09-17T02:45:21.137044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:57.648928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134832+0000","last_change":"2024-09-17T02:45:20.130812+0000","last_active":"2024-09-17T02:45:20.134832+0000","last_peered":"2024-09-17T02:45:20.134832+0000","last_clean":"2024-09-17T02:45:20.134832+0000","last_became_active":"2024-09-17T02:45:20.130480+0000","last_became_peered":"2024-09-17T02:45:20.130480+0000","last_unstale":"2024-09-17T02:45:20.134832+0000","last_undegraded":"2024-09-17T02:45:20.134832+0000","last_fullsized":"2024-09-17T02:45:20.134832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:43.366504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136826+0000","last_change":"2024-09-17T02:45:19.093011+0000","last_active":"2024-09-17T02:45:21.136826+0000","last_peered":"2024-09-17T02:45:21.136826+0000","last_clean":"2024-09-17T02:45:21.136826+0000","last_became_active":"2024-09-17T02:45:19.091917+0000","last_became_peered":"2024-09-17T02:45:19.091917+0000","last_unstale":"2024-09-17T02:45:21.136826+0000","last_undegraded":"2024-09-17T02:45:21.136826+0000","last_fullsized":"2024-09-17T02:45:21.136826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:03.939829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139221+0000","last_change":"2024-09-17T02:45:20.153283+0000","last_active":"2024-09-17T02:45:21.139221+0000","last_peered":"2024-09-17T02:45:21.139221+0000","last_clean":"2024-09-17T02:45:21.139221+0000","last_became_active":"2024-09-17T02:45:20.152428+0000","last_became_peered":"2024-09-17T02:45:20.152428+0000","last_unstale":"2024-09-17T02:45:21.139221+0000","last_undegraded":"2024-09-17T02:45:21.139221+0000","last_fullsized":"2024-09-17T02:45:21.139221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:36.231357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.104378+0000","last_change":"2024-09-17T02:45:19.096590+0000","last_active":"2024-09-17T02:45:20.104378+0000","last_peered":"2024-09-17T02:45:20.104378+0000","last_clean":"2024-09-17T02:45:20.104378+0000","last_became_active":"2024-09-17T02:45:19.096024+0000","last_became_peered":"2024-09-17T02:45:19.096024+0000","last_unstale":"2024-09-17T02:45:20.104378+0000","last_undegraded":"2024-09-17T02:45:20.104378+0000","last_fullsized":"2024-09-17T02:45:20.104378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:17.702397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.126109+0000","last_change":"2024-09-17T02:45:20.120878+0000","last_active":"2024-09-17T02:45:20.126109+0000","last_peered":"2024-09-17T02:45:20.126109+0000","last_clean":"2024-09-17T02:45:20.126109+0000","last_became_active":"2024-09-17T02:45:20.119874+0000","last_became_peered":"2024-09-17T02:45:20.119874+0000","last_unstale":"2024-09-17T02:45:20.126109+0000","last_undegraded":"2024-09-17T02:45:20.126109+0000","last_fullsized":"2024-09-17T02:45:20.126109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:55.317365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.108369+0000","last_change":"2024-09-17T02:45:19.084757+0000","last_active":"2024-09-17T02:45:20.108369+0000","last_peered":"2024-09-17T02:45:20.108369+0000","last_clean":"2024-09-17T02:45:20.108369+0000","last_became_active":"2024-09-17T02:45:19.084284+0000","last_became_peered":"2024-09-17T02:45:19.084284+0000","last_unstale":"2024-09-17T02:45:20.108369+0000","last_undegraded":"2024-09-17T02:45:20.108369+0000","last_fullsized":"2024-09-17T02:45:20.108369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:31.618471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130874+0000","last_change":"2024-09-17T02:45:20.128870+0000","last_active":"2024-09-17T02:45:20.130874+0000","last_peered":"2024-09-17T02:45:20.130874+0000","last_clean":"2024-09-17T02:45:20.130874+0000","last_became_active":"2024-09-17T02:45:20.128450+0000","last_became_peered":"2024-09-17T02:45:20.128450+0000","last_unstale":"2024-09-17T02:45:20.130874+0000","last_undegraded":"2024-09-17T02:45:20.130874+0000","last_fullsized":"2024-09-17T02:45:20.130874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:39.366822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.111875+0000","last_change":"2024-09-17T02:45:19.106516+0000","last_active":"2024-09-17T02:45:20.111875+0000","last_peered":"2024-09-17T02:45:20.111875+0000","last_clean":"2024-09-17T02:45:20.111875+0000","last_became_active":"2024-09-17T02:45:19.105883+0000","last_became_peered":"2024-09-17T02:45:19.105883+0000","last_unstale":"2024-09-17T02:45:20.111875+0000","last_undegraded":"2024-09-17T02:45:20.111875+0000","last_fullsized":"2024-09-17T02:45:20.111875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:39.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.063677+0000","last_change":"2024-09-17T02:45:19.063677+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.063677+0000","last_undegraded":"2024-09-17T02:45:19.063677+0000","last_fullsized":"2024-09-17T02:45:19.063677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.110077+0000","last_change":"2024-09-17T02:45:19.086253+0000","last_active":"2024-09-17T02:45:20.110077+0000","last_peered":"2024-09-17T02:45:20.110077+0000","last_clean":"2024-09-17T02:45:20.110077+0000","last_became_active":"2024-09-17T02:45:19.085807+0000","last_became_peered":"2024-09-17T02:45:19.085807+0000","last_unstale":"2024-09-17T02:45:20.110077+0000","last_undegraded":"2024-09-17T02:45:20.110077+0000","last_fullsized":"2024-09-17T02:45:20.110077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:43.903222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149388+0000","last_change":"2024-09-17T02:45:20.124232+0000","last_active":"2024-09-17T02:45:20.149388+0000","last_peered":"2024-09-17T02:45:20.149388+0000","last_clean":"2024-09-17T02:45:20.149388+0000","last_became_active":"2024-09-17T02:45:20.123450+0000","last_became_peered":"2024-09-17T02:45:20.123450+0000","last_unstale":"2024-09-17T02:45:20.149388+0000","last_undegraded":"2024-09-17T02:45:20.149388+0000","last_fullsized":"2024-09-17T02:45:20.149388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:01.142409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.094923+0000","last_change":"2024-09-17T02:45:19.087894+0000","last_active":"2024-09-17T02:45:19.094923+0000","last_peered":"2024-09-17T02:45:19.094923+0000","last_clean":"2024-09-17T02:45:19.094923+0000","last_became_active":"2024-09-17T02:45:19.086041+0000","last_became_peered":"2024-09-17T02:45:19.086041+0000","last_unstale":"2024-09-17T02:45:19.094923+0000","last_undegraded":"2024-09-17T02:45:19.094923+0000","last_fullsized":"2024-09-17T02:45:19.094923+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:30.454811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130153+0000","last_change":"2024-09-17T02:45:20.128777+0000","last_active":"2024-09-17T02:45:20.130153+0000","last_peered":"2024-09-17T02:45:20.130153+0000","last_clean":"2024-09-17T02:45:20.130153+0000","last_became_active":"2024-09-17T02:45:20.128362+0000","last_became_peered":"2024-09-17T02:45:20.128362+0000","last_unstale":"2024-09-17T02:45:20.130153+0000","last_undegraded":"2024-09-17T02:45:20.130153+0000","last_fullsized":"2024-09-17T02:45:20.130153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:46.647418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.111264+0000","last_change":"2024-09-17T02:45:19.099271+0000","last_active":"2024-09-17T02:45:20.111264+0000","last_peered":"2024-09-17T02:45:20.111264+0000","last_clean":"2024-09-17T02:45:20.111264+0000","last_became_active":"2024-09-17T02:45:19.088289+0000","last_became_peered":"2024-09-17T02:45:19.088289+0000","last_unstale":"2024-09-17T02:45:20.111264+0000","last_undegraded":"2024-09-17T02:45:20.111264+0000","last_fullsized":"2024-09-17T02:45:20.111264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:54.306776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":220,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.134645+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:45:21.134645+0000","last_peered":"2024-09-17T02:45:21.134645+0000","last_clean":"2024-09-17T02:45:21.134645+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:45:21.134645+0000","last_undegraded":"2024-09-17T02:45:21.134645+0000","last_fullsized":"2024-09-17T02:45:21.134645+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509500,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":954368,"data_stored":431399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509500,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1544192,"data_stored":1021205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509501,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":430769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509499,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":432463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509500,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":430747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509500,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":1022761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":950272,"data_stored":430769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":416913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:45:24.673 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:45:24.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.874+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4140620062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fdc005260 msgr2=0x7f4fdc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.874+0000 7f4ff89aa640 1 --2- 172.21.15.131:0/4140620062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fdc005260 0x7f4fdc00b9d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40ba520 tx=0x7f4fec01cad0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.874+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4140620062 shutdown_connections 2024-09-17T02:45:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.874+0000 7f4ff89aa640 1 --2- 172.21.15.131:0/4140620062 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fdc005260 0x7f4fdc00b9d0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.874+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4140620062 >> 172.21.15.131:0/4140620062 conn(0x7f4ff40d1b10 msgr2=0x7f4ff40d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.874+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4140620062 shutdown_connections 2024-09-17T02:45:24.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.874+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4140620062 wait complete. 2024-09-17T02:45:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.874+0000 7f4ff89aa640 1 Processor -- start 2024-09-17T02:45:24.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.874+0000 7f4ff89aa640 1 -- start start 2024-09-17T02:45:24.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff89aa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fdc005260 0x7f4ff40ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff89aa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ff4152810 0x7f4ff4150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff89aa640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4ff4150d40 0x7f4ff4151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff89aa640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff406d810 con 0x7f4fdc005260 2024-09-17T02:45:24.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff89aa640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff406d690 con 0x7f4ff4152810 2024-09-17T02:45:24.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff89aa640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff40d2460 con 0x7f4ff4150d40 2024-09-17T02:45:24.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fdc005260 0x7f4ff40ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:24.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fdc005260 0x7f4ff40ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44156/0 (socket says 172.21.15.131:44156) 2024-09-17T02:45:24.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff259c640 1 -- 172.21.15.131:0/4225098038 learned_addr learned my addr 172.21.15.131:0/4225098038 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:24.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff1d9b640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ff4152810 0x7f4ff4150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:24.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff2d9d640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4ff4150d40 0x7f4ff4151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:24.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff259c640 1 -- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4ff4150d40 msgr2=0x7f4ff4151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff259c640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4ff4150d40 0x7f4ff4151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff259c640 1 -- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ff4152810 msgr2=0x7f4ff4150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:24.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff259c640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ff4152810 0x7f4ff4150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:24.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff259c640 1 -- 172.21.15.131:0/4225098038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff4151690 con 0x7f4fdc005260 2024-09-17T02:45:24.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff259c640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fdc005260 0x7f4ff40ce0e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4000fe0 tx=0x7f4fe400e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:24.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4fe2ffd640 1 -- 172.21.15.131:0/4225098038 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4015020 con 0x7f4fdc005260 2024-09-17T02:45:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4fe2ffd640 1 -- 172.21.15.131:0/4225098038 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fe40109a0 con 0x7f4fdc005260 2024-09-17T02:45:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4225098038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff4152f20 con 0x7f4fdc005260 2024-09-17T02:45:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4fe2ffd640 1 -- 172.21.15.131:0/4225098038 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4010df0 con 0x7f4fdc005260 2024-09-17T02:45:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4225098038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff415a980 con 0x7f4fdc005260 2024-09-17T02:45:24.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4225098038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff415f290 con 0x7f4fdc005260 2024-09-17T02:45:24.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.878+0000 7f4fe2ffd640 1 -- 172.21.15.131:0/4225098038 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4fe4018020 con 0x7f4fdc005260 2024-09-17T02:45:24.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.882+0000 7f4fe2ffd640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4fd4073230 0x7f4fd4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:24.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.882+0000 7f4fe2ffd640 1 -- 172.21.15.131:0/4225098038 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f4fe4094a60 con 0x7f4fdc005260 2024-09-17T02:45:24.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.882+0000 7f4ff1d9b640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4fd4073230 0x7f4fd4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:24.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.882+0000 7f4ff1d9b640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4fd4073230 0x7f4fd4075740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4ff41f8840 tx=0x7f4fe80091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:24.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:24.886+0000 7f4fe2ffd640 1 -- 172.21.15.131:0/4225098038 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4fe401f020 con 0x7f4fdc005260 2024-09-17T02:45:25.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.046+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4225098038 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4ff40b9e50 con 0x7f4fd4073230 2024-09-17T02:45:25.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.058+0000 7f4fe2ffd640 1 -- 172.21.15.131:0/4225098038 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251077 (secure 0 0 0) 0x7f4ff40b9e50 con 0x7f4fd4073230 2024-09-17T02:45:25.058 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:25.059 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:45:25.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4fd4073230 msgr2=0x7f4fd4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4fd4073230 0x7f4fd4075740 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4ff41f8840 tx=0x7f4fe80091e0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4225098038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fdc005260 msgr2=0x7f4ff40ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fdc005260 0x7f4ff40ce0e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4000fe0 tx=0x7f4fe400e890 comp rx=0 tx=0).stop 2024-09-17T02:45:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4225098038 shutdown_connections 2024-09-17T02:45:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4fd4073230 0x7f4fd4075740 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4ff4150d40 0x7f4ff4151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ff4152810 0x7f4ff4150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 --2- 172.21.15.131:0/4225098038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4fdc005260 0x7f4ff40ce0e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4225098038 >> 172.21.15.131:0/4225098038 conn(0x7f4ff40d1b10 msgr2=0x7f4ff415ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4225098038 shutdown_connections 2024-09-17T02:45:25.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.062+0000 7f4ff89aa640 1 -- 172.21.15.131:0/4225098038 wait complete. 2024-09-17T02:45:25.079 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-17T02:45:23.225560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109,"ondisk_log_size":109,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258104,"kb_used_data":8584,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716616,"statfs":{"total":773094113280,"available":772829814784,"internally_reserved":0,"allocated":8790016,"data_stored":4617026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.173689"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135946+0000","last_change":"2024-09-17T02:45:20.134496+0000","last_active":"2024-09-17T02:45:21.135946+0000","last_peered":"2024-09-17T02:45:21.135946+0000","last_clean":"2024-09-17T02:45:21.135946+0000","last_became_active":"2024-09-17T02:45:20.134052+0000","last_became_peered":"2024-09-17T02:45:20.134052+0000","last_unstale":"2024-09-17T02:45:21.135946+0000","last_undegraded":"2024-09-17T02:45:21.135946+0000","last_fullsized":"2024-09-17T02:45:21.135946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:33.880550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.066330+0000","last_change":"2024-09-17T02:45:19.066330+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.066330+0000","last_undegraded":"2024-09-17T02:45:19.066330+0000","last_fullsized":"2024-09-17T02:45:19.066330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.071003+0000","last_change":"2024-09-17T02:45:19.071003+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.071003+0000","last_undegraded":"2024-09-17T02:45:19.071003+0000","last_fullsized":"2024-09-17T02:45:19.071003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.127671+0000","last_change":"2024-09-17T02:45:20.124901+0000","last_active":"2024-09-17T02:45:20.127671+0000","last_peered":"2024-09-17T02:45:20.127671+0000","last_clean":"2024-09-17T02:45:20.127671+0000","last_became_active":"2024-09-17T02:45:20.124392+0000","last_became_peered":"2024-09-17T02:45:20.124392+0000","last_unstale":"2024-09-17T02:45:20.127671+0000","last_undegraded":"2024-09-17T02:45:20.127671+0000","last_fullsized":"2024-09-17T02:45:20.127671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:56.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.157150+0000","last_change":"2024-09-17T02:45:20.151934+0000","last_active":"2024-09-17T02:45:20.157150+0000","last_peered":"2024-09-17T02:45:20.157150+0000","last_clean":"2024-09-17T02:45:20.157150+0000","last_became_active":"2024-09-17T02:45:20.149652+0000","last_became_peered":"2024-09-17T02:45:20.149652+0000","last_unstale":"2024-09-17T02:45:20.157150+0000","last_undegraded":"2024-09-17T02:45:20.157150+0000","last_fullsized":"2024-09-17T02:45:20.157150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:06.754098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.162147+0000","last_change":"2024-09-17T02:45:20.156609+0000","last_active":"2024-09-17T02:45:20.162147+0000","last_peered":"2024-09-17T02:45:20.162147+0000","last_clean":"2024-09-17T02:45:20.162147+0000","last_became_active":"2024-09-17T02:45:20.155997+0000","last_became_peered":"2024-09-17T02:45:20.155997+0000","last_unstale":"2024-09-17T02:45:20.162147+0000","last_undegraded":"2024-09-17T02:45:20.162147+0000","last_fullsized":"2024-09-17T02:45:20.162147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:49.213100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.141570+0000","last_change":"2024-09-17T02:45:20.138831+0000","last_active":"2024-09-17T02:45:20.141570+0000","last_peered":"2024-09-17T02:45:20.141570+0000","last_clean":"2024-09-17T02:45:20.141570+0000","last_became_active":"2024-09-17T02:45:20.138468+0000","last_became_peered":"2024-09-17T02:45:20.138468+0000","last_unstale":"2024-09-17T02:45:20.141570+0000","last_undegraded":"2024-09-17T02:45:20.141570+0000","last_fullsized":"2024-09-17T02:45:20.141570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:17.239517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137912+0000","last_change":"2024-09-17T02:45:20.151755+0000","last_active":"2024-09-17T02:45:21.137912+0000","last_peered":"2024-09-17T02:45:21.137912+0000","last_clean":"2024-09-17T02:45:21.137912+0000","last_became_active":"2024-09-17T02:45:20.139390+0000","last_became_peered":"2024-09-17T02:45:20.139390+0000","last_unstale":"2024-09-17T02:45:21.137912+0000","last_undegraded":"2024-09-17T02:45:21.137912+0000","last_fullsized":"2024-09-17T02:45:21.137912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:04.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130320+0000","last_change":"2024-09-17T02:45:20.128764+0000","last_active":"2024-09-17T02:45:20.130320+0000","last_peered":"2024-09-17T02:45:20.130320+0000","last_clean":"2024-09-17T02:45:20.130320+0000","last_became_active":"2024-09-17T02:45:20.128381+0000","last_became_peered":"2024-09-17T02:45:20.128381+0000","last_unstale":"2024-09-17T02:45:20.130320+0000","last_undegraded":"2024-09-17T02:45:20.130320+0000","last_fullsized":"2024-09-17T02:45:20.130320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:35.763836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135793+0000","last_change":"2024-09-17T02:45:20.134184+0000","last_active":"2024-09-17T02:45:21.135793+0000","last_peered":"2024-09-17T02:45:21.135793+0000","last_clean":"2024-09-17T02:45:21.135793+0000","last_became_active":"2024-09-17T02:45:20.133826+0000","last_became_peered":"2024-09-17T02:45:20.133826+0000","last_unstale":"2024-09-17T02:45:21.135793+0000","last_undegraded":"2024-09-17T02:45:21.135793+0000","last_fullsized":"2024-09-17T02:45:21.135793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:41.858427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.068073+0000","last_change":"2024-09-17T02:45:19.068073+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.068073+0000","last_undegraded":"2024-09-17T02:45:19.068073+0000","last_fullsized":"2024-09-17T02:45:19.068073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.146278+0000","last_change":"2024-09-17T02:45:20.137823+0000","last_active":"2024-09-17T02:45:20.146278+0000","last_peered":"2024-09-17T02:45:20.146278+0000","last_clean":"2024-09-17T02:45:20.146278+0000","last_became_active":"2024-09-17T02:45:20.137420+0000","last_became_peered":"2024-09-17T02:45:20.137420+0000","last_unstale":"2024-09-17T02:45:20.146278+0000","last_undegraded":"2024-09-17T02:45:20.146278+0000","last_fullsized":"2024-09-17T02:45:20.146278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:14.885437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.072804+0000","last_change":"2024-09-17T02:45:19.072804+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.072804+0000","last_undegraded":"2024-09-17T02:45:19.072804+0000","last_fullsized":"2024-09-17T02:45:19.072804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139109+0000","last_change":"2024-09-17T02:45:20.154359+0000","last_active":"2024-09-17T02:45:21.139109+0000","last_peered":"2024-09-17T02:45:21.139109+0000","last_clean":"2024-09-17T02:45:21.139109+0000","last_became_active":"2024-09-17T02:45:20.138831+0000","last_became_peered":"2024-09-17T02:45:20.138831+0000","last_unstale":"2024-09-17T02:45:21.139109+0000","last_undegraded":"2024-09-17T02:45:21.139109+0000","last_fullsized":"2024-09-17T02:45:21.139109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:26.541024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151644+0000","last_change":"2024-09-17T02:45:20.136343+0000","last_active":"2024-09-17T02:45:20.151644+0000","last_peered":"2024-09-17T02:45:20.151644+0000","last_clean":"2024-09-17T02:45:20.151644+0000","last_became_active":"2024-09-17T02:45:20.135036+0000","last_became_peered":"2024-09-17T02:45:20.135036+0000","last_unstale":"2024-09-17T02:45:20.151644+0000","last_undegraded":"2024-09-17T02:45:20.151644+0000","last_fullsized":"2024-09-17T02:45:20.151644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:15.507022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134377+0000","last_change":"2024-09-17T02:45:20.133650+0000","last_active":"2024-09-17T02:45:20.134377+0000","last_peered":"2024-09-17T02:45:20.134377+0000","last_clean":"2024-09-17T02:45:20.134377+0000","last_became_active":"2024-09-17T02:45:20.133246+0000","last_became_peered":"2024-09-17T02:45:20.133246+0000","last_unstale":"2024-09-17T02:45:20.134377+0000","last_undegraded":"2024-09-17T02:45:20.134377+0000","last_fullsized":"2024-09-17T02:45:20.134377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:31.528537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138676+0000","last_change":"2024-09-17T02:45:20.138431+0000","last_active":"2024-09-17T02:45:21.138676+0000","last_peered":"2024-09-17T02:45:21.138676+0000","last_clean":"2024-09-17T02:45:21.138676+0000","last_became_active":"2024-09-17T02:45:20.137633+0000","last_became_peered":"2024-09-17T02:45:20.137633+0000","last_unstale":"2024-09-17T02:45:21.138676+0000","last_undegraded":"2024-09-17T02:45:21.138676+0000","last_fullsized":"2024-09-17T02:45:21.138676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:25.489158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.065085+0000","last_change":"2024-09-17T02:45:19.065085+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.065085+0000","last_undegraded":"2024-09-17T02:45:19.065085+0000","last_fullsized":"2024-09-17T02:45:19.065085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138525+0000","last_change":"2024-09-17T02:45:20.138096+0000","last_active":"2024-09-17T02:45:21.138525+0000","last_peered":"2024-09-17T02:45:21.138525+0000","last_clean":"2024-09-17T02:45:21.138525+0000","last_became_active":"2024-09-17T02:45:20.137246+0000","last_became_peered":"2024-09-17T02:45:20.137246+0000","last_unstale":"2024-09-17T02:45:21.138525+0000","last_undegraded":"2024-09-17T02:45:21.138525+0000","last_fullsized":"2024-09-17T02:45:21.138525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:08.405979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.148092+0000","last_change":"2024-09-17T02:45:20.144454+0000","last_active":"2024-09-17T02:45:20.148092+0000","last_peered":"2024-09-17T02:45:20.148092+0000","last_clean":"2024-09-17T02:45:20.148092+0000","last_became_active":"2024-09-17T02:45:20.143671+0000","last_became_peered":"2024-09-17T02:45:20.143671+0000","last_unstale":"2024-09-17T02:45:20.148092+0000","last_undegraded":"2024-09-17T02:45:20.148092+0000","last_fullsized":"2024-09-17T02:45:20.148092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:26.359990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.147886+0000","last_change":"2024-09-17T02:45:20.140711+0000","last_active":"2024-09-17T02:45:20.147886+0000","last_peered":"2024-09-17T02:45:20.147886+0000","last_clean":"2024-09-17T02:45:20.147886+0000","last_became_active":"2024-09-17T02:45:20.140390+0000","last_became_peered":"2024-09-17T02:45:20.140390+0000","last_unstale":"2024-09-17T02:45:20.147886+0000","last_undegraded":"2024-09-17T02:45:20.147886+0000","last_fullsized":"2024-09-17T02:45:20.147886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.473725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.069206+0000","last_change":"2024-09-17T02:45:19.069206+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.069206+0000","last_undegraded":"2024-09-17T02:45:19.069206+0000","last_fullsized":"2024-09-17T02:45:19.069206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135331+0000","last_change":"2024-09-17T02:45:20.120377+0000","last_active":"2024-09-17T02:45:21.135331+0000","last_peered":"2024-09-17T02:45:21.135331+0000","last_clean":"2024-09-17T02:45:21.135331+0000","last_became_active":"2024-09-17T02:45:20.119336+0000","last_became_peered":"2024-09-17T02:45:20.119336+0000","last_unstale":"2024-09-17T02:45:21.135331+0000","last_undegraded":"2024-09-17T02:45:21.135331+0000","last_fullsized":"2024-09-17T02:45:21.135331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:43.174152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.073370+0000","last_change":"2024-09-17T02:45:19.073370+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.073370+0000","last_undegraded":"2024-09-17T02:45:19.073370+0000","last_fullsized":"2024-09-17T02:45:19.073370+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.143641+0000","last_change":"2024-09-17T02:45:20.136877+0000","last_active":"2024-09-17T02:45:20.143641+0000","last_peered":"2024-09-17T02:45:20.143641+0000","last_clean":"2024-09-17T02:45:20.143641+0000","last_became_active":"2024-09-17T02:45:20.136227+0000","last_became_peered":"2024-09-17T02:45:20.136227+0000","last_unstale":"2024-09-17T02:45:20.143641+0000","last_undegraded":"2024-09-17T02:45:20.143641+0000","last_fullsized":"2024-09-17T02:45:20.143641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:06.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.131033+0000","last_change":"2024-09-17T02:45:20.129949+0000","last_active":"2024-09-17T02:45:20.131033+0000","last_peered":"2024-09-17T02:45:20.131033+0000","last_clean":"2024-09-17T02:45:20.131033+0000","last_became_active":"2024-09-17T02:45:20.129519+0000","last_became_peered":"2024-09-17T02:45:20.129519+0000","last_unstale":"2024-09-17T02:45:20.131033+0000","last_undegraded":"2024-09-17T02:45:20.131033+0000","last_fullsized":"2024-09-17T02:45:20.131033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:53.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.127890+0000","last_change":"2024-09-17T02:45:20.125488+0000","last_active":"2024-09-17T02:45:20.127890+0000","last_peered":"2024-09-17T02:45:20.127890+0000","last_clean":"2024-09-17T02:45:20.127890+0000","last_became_active":"2024-09-17T02:45:20.124765+0000","last_became_peered":"2024-09-17T02:45:20.124765+0000","last_unstale":"2024-09-17T02:45:20.127890+0000","last_undegraded":"2024-09-17T02:45:20.127890+0000","last_fullsized":"2024-09-17T02:45:20.127890+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:41.858427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.125889+0000","last_change":"2024-09-17T02:45:20.121902+0000","last_active":"2024-09-17T02:45:20.125889+0000","last_peered":"2024-09-17T02:45:20.125889+0000","last_clean":"2024-09-17T02:45:20.125889+0000","last_became_active":"2024-09-17T02:45:20.121274+0000","last_became_peered":"2024-09-17T02:45:20.121274+0000","last_unstale":"2024-09-17T02:45:20.125889+0000","last_undegraded":"2024-09-17T02:45:20.125889+0000","last_fullsized":"2024-09-17T02:45:20.125889+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:19.026083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.150174+0000","last_change":"2024-09-17T02:45:20.147039+0000","last_active":"2024-09-17T02:45:20.150174+0000","last_peered":"2024-09-17T02:45:20.150174+0000","last_clean":"2024-09-17T02:45:20.150174+0000","last_became_active":"2024-09-17T02:45:20.145977+0000","last_became_peered":"2024-09-17T02:45:20.145977+0000","last_unstale":"2024-09-17T02:45:20.150174+0000","last_undegraded":"2024-09-17T02:45:20.150174+0000","last_fullsized":"2024-09-17T02:45:20.150174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:25.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135630+0000","last_change":"2024-09-17T02:45:20.134028+0000","last_active":"2024-09-17T02:45:21.135630+0000","last_peered":"2024-09-17T02:45:21.135630+0000","last_clean":"2024-09-17T02:45:21.135630+0000","last_became_active":"2024-09-17T02:45:20.133628+0000","last_became_peered":"2024-09-17T02:45:20.133628+0000","last_unstale":"2024-09-17T02:45:21.135630+0000","last_undegraded":"2024-09-17T02:45:21.135630+0000","last_fullsized":"2024-09-17T02:45:21.135630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:56.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.074374+0000","last_change":"2024-09-17T02:45:19.074374+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.074374+0000","last_undegraded":"2024-09-17T02:45:19.074374+0000","last_fullsized":"2024-09-17T02:45:19.074374+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.129861+0000","last_change":"2024-09-17T02:45:20.127695+0000","last_active":"2024-09-17T02:45:20.129861+0000","last_peered":"2024-09-17T02:45:20.129861+0000","last_clean":"2024-09-17T02:45:20.129861+0000","last_became_active":"2024-09-17T02:45:20.127323+0000","last_became_peered":"2024-09-17T02:45:20.127323+0000","last_unstale":"2024-09-17T02:45:20.129861+0000","last_undegraded":"2024-09-17T02:45:20.129861+0000","last_fullsized":"2024-09-17T02:45:20.129861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:42.063380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.132767+0000","last_change":"2024-09-17T02:45:20.122630+0000","last_active":"2024-09-17T02:45:20.132767+0000","last_peered":"2024-09-17T02:45:20.132767+0000","last_clean":"2024-09-17T02:45:20.132767+0000","last_became_active":"2024-09-17T02:45:20.120596+0000","last_became_peered":"2024-09-17T02:45:20.120596+0000","last_unstale":"2024-09-17T02:45:20.132767+0000","last_undegraded":"2024-09-17T02:45:20.132767+0000","last_fullsized":"2024-09-17T02:45:20.132767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:16.342312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"78'4","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165404+0000","last_change":"2024-09-17T02:45:19.105415+0000","last_active":"2024-09-17T02:45:20.165404+0000","last_peered":"2024-09-17T02:45:20.165404+0000","last_clean":"2024-09-17T02:45:20.165404+0000","last_became_active":"2024-09-17T02:45:19.104799+0000","last_became_peered":"2024-09-17T02:45:19.104799+0000","last_unstale":"2024-09-17T02:45:20.165404+0000","last_undegraded":"2024-09-17T02:45:20.165404+0000","last_fullsized":"2024-09-17T02:45:20.165404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:15:52.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.153720+0000","last_change":"2024-09-17T02:45:20.151127+0000","last_active":"2024-09-17T02:45:20.153720+0000","last_peered":"2024-09-17T02:45:20.153720+0000","last_clean":"2024-09-17T02:45:20.153720+0000","last_became_active":"2024-09-17T02:45:20.150324+0000","last_became_peered":"2024-09-17T02:45:20.150324+0000","last_unstale":"2024-09-17T02:45:20.153720+0000","last_undegraded":"2024-09-17T02:45:20.153720+0000","last_fullsized":"2024-09-17T02:45:20.153720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:26.406192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151669+0000","last_change":"2024-09-17T02:45:19.079006+0000","last_active":"2024-09-17T02:45:20.151669+0000","last_peered":"2024-09-17T02:45:20.151669+0000","last_clean":"2024-09-17T02:45:20.151669+0000","last_became_active":"2024-09-17T02:45:19.078512+0000","last_became_peered":"2024-09-17T02:45:19.078512+0000","last_unstale":"2024-09-17T02:45:20.151669+0000","last_undegraded":"2024-09-17T02:45:20.151669+0000","last_fullsized":"2024-09-17T02:45:20.151669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:48.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134859+0000","last_change":"2024-09-17T02:45:20.131374+0000","last_active":"2024-09-17T02:45:20.134859+0000","last_peered":"2024-09-17T02:45:20.134859+0000","last_clean":"2024-09-17T02:45:20.134859+0000","last_became_active":"2024-09-17T02:45:20.129652+0000","last_became_peered":"2024-09-17T02:45:20.129652+0000","last_unstale":"2024-09-17T02:45:20.134859+0000","last_undegraded":"2024-09-17T02:45:20.134859+0000","last_fullsized":"2024-09-17T02:45:20.134859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:02.100162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136882+0000","last_change":"2024-09-17T02:45:19.078238+0000","last_active":"2024-09-17T02:45:21.136882+0000","last_peered":"2024-09-17T02:45:21.136882+0000","last_clean":"2024-09-17T02:45:21.136882+0000","last_became_active":"2024-09-17T02:45:19.077495+0000","last_became_peered":"2024-09-17T02:45:19.077495+0000","last_unstale":"2024-09-17T02:45:21.136882+0000","last_undegraded":"2024-09-17T02:45:21.136882+0000","last_fullsized":"2024-09-17T02:45:21.136882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:25.395603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130326+0000","last_change":"2024-09-17T02:45:20.127140+0000","last_active":"2024-09-17T02:45:20.130326+0000","last_peered":"2024-09-17T02:45:20.130326+0000","last_clean":"2024-09-17T02:45:20.130326+0000","last_became_active":"2024-09-17T02:45:20.126455+0000","last_became_peered":"2024-09-17T02:45:20.126455+0000","last_unstale":"2024-09-17T02:45:20.130326+0000","last_undegraded":"2024-09-17T02:45:20.130326+0000","last_fullsized":"2024-09-17T02:45:20.130326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:33.880550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"78'2","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149384+0000","last_change":"2024-09-17T02:45:19.092392+0000","last_active":"2024-09-17T02:45:20.149384+0000","last_peered":"2024-09-17T02:45:20.149384+0000","last_clean":"2024-09-17T02:45:20.149384+0000","last_became_active":"2024-09-17T02:45:19.091367+0000","last_became_peered":"2024-09-17T02:45:19.091367+0000","last_unstale":"2024-09-17T02:45:20.149384+0000","last_undegraded":"2024-09-17T02:45:20.149384+0000","last_fullsized":"2024-09-17T02:45:20.149384+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:57.648928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.143331+0000","last_change":"2024-09-17T02:45:20.134096+0000","last_active":"2024-09-17T02:45:20.143331+0000","last_peered":"2024-09-17T02:45:20.143331+0000","last_clean":"2024-09-17T02:45:20.143331+0000","last_became_active":"2024-09-17T02:45:20.133166+0000","last_became_peered":"2024-09-17T02:45:20.133166+0000","last_unstale":"2024-09-17T02:45:20.143331+0000","last_undegraded":"2024-09-17T02:45:20.143331+0000","last_fullsized":"2024-09-17T02:45:20.143331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:17.390929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.147932+0000","last_change":"2024-09-17T02:45:19.095580+0000","last_active":"2024-09-17T02:45:20.147932+0000","last_peered":"2024-09-17T02:45:20.147932+0000","last_clean":"2024-09-17T02:45:20.147932+0000","last_became_active":"2024-09-17T02:45:19.094437+0000","last_became_peered":"2024-09-17T02:45:19.094437+0000","last_unstale":"2024-09-17T02:45:20.147932+0000","last_undegraded":"2024-09-17T02:45:20.147932+0000","last_fullsized":"2024-09-17T02:45:20.147932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:59.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.066618+0000","last_change":"2024-09-17T02:45:19.066618+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.066618+0000","last_undegraded":"2024-09-17T02:45:19.066618+0000","last_fullsized":"2024-09-17T02:45:19.066618+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165349+0000","last_change":"2024-09-17T02:45:19.086585+0000","last_active":"2024-09-17T02:45:20.165349+0000","last_peered":"2024-09-17T02:45:20.165349+0000","last_clean":"2024-09-17T02:45:20.165349+0000","last_became_active":"2024-09-17T02:45:19.086060+0000","last_became_peered":"2024-09-17T02:45:19.086060+0000","last_unstale":"2024-09-17T02:45:20.165349+0000","last_undegraded":"2024-09-17T02:45:20.165349+0000","last_fullsized":"2024-09-17T02:45:20.165349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:06.921612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.128800+0000","last_change":"2024-09-17T02:45:20.122719+0000","last_active":"2024-09-17T02:45:20.128800+0000","last_peered":"2024-09-17T02:45:20.128800+0000","last_clean":"2024-09-17T02:45:20.128800+0000","last_became_active":"2024-09-17T02:45:20.120568+0000","last_became_peered":"2024-09-17T02:45:20.120568+0000","last_unstale":"2024-09-17T02:45:20.128800+0000","last_undegraded":"2024-09-17T02:45:20.128800+0000","last_fullsized":"2024-09-17T02:45:20.128800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.038462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137866+0000","last_change":"2024-09-17T02:45:19.093274+0000","last_active":"2024-09-17T02:45:21.137866+0000","last_peered":"2024-09-17T02:45:21.137866+0000","last_clean":"2024-09-17T02:45:21.137866+0000","last_became_active":"2024-09-17T02:45:19.091525+0000","last_became_peered":"2024-09-17T02:45:19.091525+0000","last_unstale":"2024-09-17T02:45:21.137866+0000","last_undegraded":"2024-09-17T02:45:21.137866+0000","last_fullsized":"2024-09-17T02:45:21.137866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:05.164896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.136305+0000","last_change":"2024-09-17T02:45:20.132688+0000","last_active":"2024-09-17T02:45:20.136305+0000","last_peered":"2024-09-17T02:45:20.136305+0000","last_clean":"2024-09-17T02:45:20.136305+0000","last_became_active":"2024-09-17T02:45:20.132335+0000","last_became_peered":"2024-09-17T02:45:20.132335+0000","last_unstale":"2024-09-17T02:45:20.136305+0000","last_undegraded":"2024-09-17T02:45:20.136305+0000","last_fullsized":"2024-09-17T02:45:20.136305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:01.061964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.094857+0000","last_change":"2024-09-17T02:45:19.086961+0000","last_active":"2024-09-17T02:45:19.094857+0000","last_peered":"2024-09-17T02:45:19.094857+0000","last_clean":"2024-09-17T02:45:19.094857+0000","last_became_active":"2024-09-17T02:45:19.084997+0000","last_became_peered":"2024-09-17T02:45:19.084997+0000","last_unstale":"2024-09-17T02:45:19.094857+0000","last_undegraded":"2024-09-17T02:45:19.094857+0000","last_fullsized":"2024-09-17T02:45:19.094857+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:07.303479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.071853+0000","last_change":"2024-09-17T02:45:19.071853+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.071853+0000","last_undegraded":"2024-09-17T02:45:19.071853+0000","last_fullsized":"2024-09-17T02:45:19.071853+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.103868+0000","last_change":"2024-09-17T02:45:19.081505+0000","last_active":"2024-09-17T02:45:20.103868+0000","last_peered":"2024-09-17T02:45:20.103868+0000","last_clean":"2024-09-17T02:45:20.103868+0000","last_became_active":"2024-09-17T02:45:19.081196+0000","last_became_peered":"2024-09-17T02:45:19.081196+0000","last_unstale":"2024-09-17T02:45:20.103868+0000","last_undegraded":"2024-09-17T02:45:20.103868+0000","last_fullsized":"2024-09-17T02:45:20.103868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:33.638898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.146953+0000","last_change":"2024-09-17T02:45:20.139575+0000","last_active":"2024-09-17T02:45:20.146953+0000","last_peered":"2024-09-17T02:45:20.146953+0000","last_clean":"2024-09-17T02:45:20.146953+0000","last_became_active":"2024-09-17T02:45:20.137448+0000","last_became_peered":"2024-09-17T02:45:20.137448+0000","last_unstale":"2024-09-17T02:45:20.146953+0000","last_undegraded":"2024-09-17T02:45:20.146953+0000","last_fullsized":"2024-09-17T02:45:20.146953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:28.895745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135457+0000","last_change":"2024-09-17T02:45:19.074384+0000","last_active":"2024-09-17T02:45:21.135457+0000","last_peered":"2024-09-17T02:45:21.135457+0000","last_clean":"2024-09-17T02:45:21.135457+0000","last_became_active":"2024-09-17T02:45:19.073958+0000","last_became_peered":"2024-09-17T02:45:19.073958+0000","last_unstale":"2024-09-17T02:45:21.135457+0000","last_undegraded":"2024-09-17T02:45:21.135457+0000","last_fullsized":"2024-09-17T02:45:21.135457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:36.384573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139981+0000","last_change":"2024-09-17T02:45:20.152408+0000","last_active":"2024-09-17T02:45:21.139981+0000","last_peered":"2024-09-17T02:45:21.139981+0000","last_clean":"2024-09-17T02:45:21.139981+0000","last_became_active":"2024-09-17T02:45:20.137766+0000","last_became_peered":"2024-09-17T02:45:20.137766+0000","last_unstale":"2024-09-17T02:45:21.139981+0000","last_undegraded":"2024-09-17T02:45:21.139981+0000","last_fullsized":"2024-09-17T02:45:21.139981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:54.493799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"78'3","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165499+0000","last_change":"2024-09-17T02:45:19.101534+0000","last_active":"2024-09-17T02:45:20.165499+0000","last_peered":"2024-09-17T02:45:20.165499+0000","last_clean":"2024-09-17T02:45:20.165499+0000","last_became_active":"2024-09-17T02:45:19.099790+0000","last_became_peered":"2024-09-17T02:45:19.099790+0000","last_unstale":"2024-09-17T02:45:20.165499+0000","last_undegraded":"2024-09-17T02:45:20.165499+0000","last_fullsized":"2024-09-17T02:45:20.165499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:59.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.155826+0000","last_change":"2024-09-17T02:45:20.154562+0000","last_active":"2024-09-17T02:45:20.155826+0000","last_peered":"2024-09-17T02:45:20.155826+0000","last_clean":"2024-09-17T02:45:20.155826+0000","last_became_active":"2024-09-17T02:45:20.153866+0000","last_became_peered":"2024-09-17T02:45:20.153866+0000","last_unstale":"2024-09-17T02:45:20.155826+0000","last_undegraded":"2024-09-17T02:45:20.155826+0000","last_fullsized":"2024-09-17T02:45:20.155826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:31.599827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"78'6","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.154097+0000","last_change":"2024-09-17T02:45:19.079828+0000","last_active":"2024-09-17T02:45:20.154097+0000","last_peered":"2024-09-17T02:45:20.154097+0000","last_clean":"2024-09-17T02:45:20.154097+0000","last_became_active":"2024-09-17T02:45:19.079230+0000","last_became_peered":"2024-09-17T02:45:19.079230+0000","last_unstale":"2024-09-17T02:45:20.154097+0000","last_undegraded":"2024-09-17T02:45:20.154097+0000","last_fullsized":"2024-09-17T02:45:20.154097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:46.676590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.133641+0000","last_change":"2024-09-17T02:45:20.130719+0000","last_active":"2024-09-17T02:45:20.133641+0000","last_peered":"2024-09-17T02:45:20.133641+0000","last_clean":"2024-09-17T02:45:20.133641+0000","last_became_active":"2024-09-17T02:45:20.130351+0000","last_became_peered":"2024-09-17T02:45:20.130351+0000","last_unstale":"2024-09-17T02:45:20.133641+0000","last_undegraded":"2024-09-17T02:45:20.133641+0000","last_fullsized":"2024-09-17T02:45:20.133641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:15.730199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136247+0000","last_change":"2024-09-17T02:45:19.085733+0000","last_active":"2024-09-17T02:45:21.136247+0000","last_peered":"2024-09-17T02:45:21.136247+0000","last_clean":"2024-09-17T02:45:21.136247+0000","last_became_active":"2024-09-17T02:45:19.085445+0000","last_became_peered":"2024-09-17T02:45:19.085445+0000","last_unstale":"2024-09-17T02:45:21.136247+0000","last_undegraded":"2024-09-17T02:45:21.136247+0000","last_fullsized":"2024-09-17T02:45:21.136247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:35.839507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136624+0000","last_change":"2024-09-17T02:45:20.127481+0000","last_active":"2024-09-17T02:45:21.136624+0000","last_peered":"2024-09-17T02:45:21.136624+0000","last_clean":"2024-09-17T02:45:21.136624+0000","last_became_active":"2024-09-17T02:45:20.127085+0000","last_became_peered":"2024-09-17T02:45:20.127085+0000","last_unstale":"2024-09-17T02:45:21.136624+0000","last_undegraded":"2024-09-17T02:45:21.136624+0000","last_fullsized":"2024-09-17T02:45:21.136624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:42.063380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.094987+0000","last_change":"2024-09-17T02:45:19.094438+0000","last_active":"2024-09-17T02:45:20.094987+0000","last_peered":"2024-09-17T02:45:20.094987+0000","last_clean":"2024-09-17T02:45:20.094987+0000","last_became_active":"2024-09-17T02:45:19.093709+0000","last_became_peered":"2024-09-17T02:45:19.093709+0000","last_unstale":"2024-09-17T02:45:20.094987+0000","last_undegraded":"2024-09-17T02:45:20.094987+0000","last_fullsized":"2024-09-17T02:45:20.094987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:45.773745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.135664+0000","last_change":"2024-09-17T02:45:20.131160+0000","last_active":"2024-09-17T02:45:20.135664+0000","last_peered":"2024-09-17T02:45:20.135664+0000","last_clean":"2024-09-17T02:45:20.135664+0000","last_became_active":"2024-09-17T02:45:20.129742+0000","last_became_peered":"2024-09-17T02:45:20.129742+0000","last_unstale":"2024-09-17T02:45:20.135664+0000","last_undegraded":"2024-09-17T02:45:20.135664+0000","last_fullsized":"2024-09-17T02:45:20.135664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:11.528887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.113545+0000","last_change":"2024-09-17T02:45:19.105626+0000","last_active":"2024-09-17T02:45:20.113545+0000","last_peered":"2024-09-17T02:45:20.113545+0000","last_clean":"2024-09-17T02:45:20.113545+0000","last_became_active":"2024-09-17T02:45:19.104904+0000","last_became_peered":"2024-09-17T02:45:19.104904+0000","last_unstale":"2024-09-17T02:45:20.113545+0000","last_undegraded":"2024-09-17T02:45:20.113545+0000","last_fullsized":"2024-09-17T02:45:20.113545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:38.203466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135464+0000","last_change":"2024-09-17T02:45:20.138086+0000","last_active":"2024-09-17T02:45:21.135464+0000","last_peered":"2024-09-17T02:45:21.135464+0000","last_clean":"2024-09-17T02:45:21.135464+0000","last_became_active":"2024-09-17T02:45:20.134972+0000","last_became_peered":"2024-09-17T02:45:20.134972+0000","last_unstale":"2024-09-17T02:45:21.135464+0000","last_undegraded":"2024-09-17T02:45:21.135464+0000","last_fullsized":"2024-09-17T02:45:21.135464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:18.254883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"78'8","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.164732+0000","last_change":"2024-09-17T02:45:19.102331+0000","last_active":"2024-09-17T02:45:20.164732+0000","last_peered":"2024-09-17T02:45:20.164732+0000","last_clean":"2024-09-17T02:45:20.164732+0000","last_became_active":"2024-09-17T02:45:19.101721+0000","last_became_peered":"2024-09-17T02:45:19.101721+0000","last_unstale":"2024-09-17T02:45:20.164732+0000","last_undegraded":"2024-09-17T02:45:20.164732+0000","last_fullsized":"2024-09-17T02:45:20.164732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:45.773745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149140+0000","last_change":"2024-09-17T02:45:20.131992+0000","last_active":"2024-09-17T02:45:20.149140+0000","last_peered":"2024-09-17T02:45:20.149140+0000","last_clean":"2024-09-17T02:45:20.149140+0000","last_became_active":"2024-09-17T02:45:20.126966+0000","last_became_peered":"2024-09-17T02:45:20.126966+0000","last_unstale":"2024-09-17T02:45:20.149140+0000","last_undegraded":"2024-09-17T02:45:20.149140+0000","last_fullsized":"2024-09-17T02:45:20.149140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:07.337735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.164565+0000","last_change":"2024-09-17T02:45:19.095009+0000","last_active":"2024-09-17T02:45:20.164565+0000","last_peered":"2024-09-17T02:45:20.164565+0000","last_clean":"2024-09-17T02:45:20.164565+0000","last_became_active":"2024-09-17T02:45:19.094610+0000","last_became_peered":"2024-09-17T02:45:19.094610+0000","last_unstale":"2024-09-17T02:45:20.164565+0000","last_undegraded":"2024-09-17T02:45:20.164565+0000","last_fullsized":"2024-09-17T02:45:20.164565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:42.163563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.073129+0000","last_change":"2024-09-17T02:45:19.073129+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.073129+0000","last_undegraded":"2024-09-17T02:45:19.073129+0000","last_fullsized":"2024-09-17T02:45:19.073129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"78'2","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151200+0000","last_change":"2024-09-17T02:45:19.089116+0000","last_active":"2024-09-17T02:45:20.151200+0000","last_peered":"2024-09-17T02:45:20.151200+0000","last_clean":"2024-09-17T02:45:20.151200+0000","last_became_active":"2024-09-17T02:45:19.088667+0000","last_became_peered":"2024-09-17T02:45:19.088667+0000","last_unstale":"2024-09-17T02:45:20.151200+0000","last_undegraded":"2024-09-17T02:45:20.151200+0000","last_fullsized":"2024-09-17T02:45:20.151200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:54:35.726780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139554+0000","last_change":"2024-09-17T02:45:20.162326+0000","last_active":"2024-09-17T02:45:21.139554+0000","last_peered":"2024-09-17T02:45:21.139554+0000","last_clean":"2024-09-17T02:45:21.139554+0000","last_became_active":"2024-09-17T02:45:20.161849+0000","last_became_peered":"2024-09-17T02:45:20.161849+0000","last_unstale":"2024-09-17T02:45:21.139554+0000","last_undegraded":"2024-09-17T02:45:21.139554+0000","last_fullsized":"2024-09-17T02:45:21.139554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:53.540992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.161729+0000","last_change":"2024-09-17T02:45:19.099466+0000","last_active":"2024-09-17T02:45:20.161729+0000","last_peered":"2024-09-17T02:45:20.161729+0000","last_clean":"2024-09-17T02:45:20.161729+0000","last_became_active":"2024-09-17T02:45:19.098488+0000","last_became_peered":"2024-09-17T02:45:19.098488+0000","last_unstale":"2024-09-17T02:45:20.161729+0000","last_undegraded":"2024-09-17T02:45:20.161729+0000","last_fullsized":"2024-09-17T02:45:20.161729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:32.496598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.157082+0000","last_change":"2024-09-17T02:45:20.152366+0000","last_active":"2024-09-17T02:45:20.157082+0000","last_peered":"2024-09-17T02:45:20.157082+0000","last_clean":"2024-09-17T02:45:20.157082+0000","last_became_active":"2024-09-17T02:45:20.150703+0000","last_became_peered":"2024-09-17T02:45:20.150703+0000","last_unstale":"2024-09-17T02:45:20.157082+0000","last_undegraded":"2024-09-17T02:45:20.157082+0000","last_fullsized":"2024-09-17T02:45:20.157082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:04.950418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.115551+0000","last_change":"2024-09-17T02:45:19.105629+0000","last_active":"2024-09-17T02:45:20.115551+0000","last_peered":"2024-09-17T02:45:20.115551+0000","last_clean":"2024-09-17T02:45:20.115551+0000","last_became_active":"2024-09-17T02:45:19.104942+0000","last_became_peered":"2024-09-17T02:45:19.104942+0000","last_unstale":"2024-09-17T02:45:20.115551+0000","last_undegraded":"2024-09-17T02:45:20.115551+0000","last_fullsized":"2024-09-17T02:45:20.115551+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:55.509905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.150727+0000","last_change":"2024-09-17T02:45:20.141760+0000","last_active":"2024-09-17T02:45:20.150727+0000","last_peered":"2024-09-17T02:45:20.150727+0000","last_clean":"2024-09-17T02:45:20.150727+0000","last_became_active":"2024-09-17T02:45:20.141069+0000","last_became_peered":"2024-09-17T02:45:20.141069+0000","last_unstale":"2024-09-17T02:45:20.150727+0000","last_undegraded":"2024-09-17T02:45:20.150727+0000","last_fullsized":"2024-09-17T02:45:20.150727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:36.850096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.106245+0000","last_change":"2024-09-17T02:45:19.101863+0000","last_active":"2024-09-17T02:45:19.106245+0000","last_peered":"2024-09-17T02:45:19.106245+0000","last_clean":"2024-09-17T02:45:19.106245+0000","last_became_active":"2024-09-17T02:45:19.101175+0000","last_became_peered":"2024-09-17T02:45:19.101175+0000","last_unstale":"2024-09-17T02:45:19.106245+0000","last_undegraded":"2024-09-17T02:45:19.106245+0000","last_fullsized":"2024-09-17T02:45:19.106245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:35:02.512375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149604+0000","last_change":"2024-09-17T02:45:20.127175+0000","last_active":"2024-09-17T02:45:20.149604+0000","last_peered":"2024-09-17T02:45:20.149604+0000","last_clean":"2024-09-17T02:45:20.149604+0000","last_became_active":"2024-09-17T02:45:20.126335+0000","last_became_peered":"2024-09-17T02:45:20.126335+0000","last_unstale":"2024-09-17T02:45:20.149604+0000","last_undegraded":"2024-09-17T02:45:20.149604+0000","last_fullsized":"2024-09-17T02:45:20.149604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:52.206612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138147+0000","last_change":"2024-09-17T02:45:19.093429+0000","last_active":"2024-09-17T02:45:21.138147+0000","last_peered":"2024-09-17T02:45:21.138147+0000","last_clean":"2024-09-17T02:45:21.138147+0000","last_became_active":"2024-09-17T02:45:19.092623+0000","last_became_peered":"2024-09-17T02:45:19.092623+0000","last_unstale":"2024-09-17T02:45:21.138147+0000","last_undegraded":"2024-09-17T02:45:21.138147+0000","last_fullsized":"2024-09-17T02:45:21.138147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:25.349401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.129570+0000","last_change":"2024-09-17T02:45:20.128671+0000","last_active":"2024-09-17T02:45:20.129570+0000","last_peered":"2024-09-17T02:45:20.129570+0000","last_clean":"2024-09-17T02:45:20.129570+0000","last_became_active":"2024-09-17T02:45:20.128268+0000","last_became_peered":"2024-09-17T02:45:20.128268+0000","last_unstale":"2024-09-17T02:45:20.129570+0000","last_undegraded":"2024-09-17T02:45:20.129570+0000","last_fullsized":"2024-09-17T02:45:20.129570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:05.898373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.096027+0000","last_change":"2024-09-17T02:45:19.089801+0000","last_active":"2024-09-17T02:45:19.096027+0000","last_peered":"2024-09-17T02:45:19.096027+0000","last_clean":"2024-09-17T02:45:19.096027+0000","last_became_active":"2024-09-17T02:45:19.083865+0000","last_became_peered":"2024-09-17T02:45:19.083865+0000","last_unstale":"2024-09-17T02:45:19.096027+0000","last_undegraded":"2024-09-17T02:45:19.096027+0000","last_fullsized":"2024-09-17T02:45:19.096027+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:31:05.560812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.122736+0000","last_change":"2024-09-17T02:45:20.121012+0000","last_active":"2024-09-17T02:45:20.122736+0000","last_peered":"2024-09-17T02:45:20.122736+0000","last_clean":"2024-09-17T02:45:20.122736+0000","last_became_active":"2024-09-17T02:45:20.120133+0000","last_became_peered":"2024-09-17T02:45:20.120133+0000","last_unstale":"2024-09-17T02:45:20.122736+0000","last_undegraded":"2024-09-17T02:45:20.122736+0000","last_fullsized":"2024-09-17T02:45:20.122736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:18.254883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135613+0000","last_change":"2024-09-17T02:45:19.082147+0000","last_active":"2024-09-17T02:45:21.135613+0000","last_peered":"2024-09-17T02:45:21.135613+0000","last_clean":"2024-09-17T02:45:21.135613+0000","last_became_active":"2024-09-17T02:45:19.081729+0000","last_became_peered":"2024-09-17T02:45:19.081729+0000","last_unstale":"2024-09-17T02:45:21.135613+0000","last_undegraded":"2024-09-17T02:45:21.135613+0000","last_fullsized":"2024-09-17T02:45:21.135613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:27.939745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135204+0000","last_change":"2024-09-17T02:45:20.136717+0000","last_active":"2024-09-17T02:45:21.135204+0000","last_peered":"2024-09-17T02:45:21.135204+0000","last_clean":"2024-09-17T02:45:21.135204+0000","last_became_active":"2024-09-17T02:45:20.136343+0000","last_became_peered":"2024-09-17T02:45:20.136343+0000","last_unstale":"2024-09-17T02:45:21.135204+0000","last_undegraded":"2024-09-17T02:45:21.135204+0000","last_fullsized":"2024-09-17T02:45:21.135204+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:44.913811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137044+0000","last_change":"2024-09-17T02:45:19.102757+0000","last_active":"2024-09-17T02:45:21.137044+0000","last_peered":"2024-09-17T02:45:21.137044+0000","last_clean":"2024-09-17T02:45:21.137044+0000","last_became_active":"2024-09-17T02:45:19.100744+0000","last_became_peered":"2024-09-17T02:45:19.100744+0000","last_unstale":"2024-09-17T02:45:21.137044+0000","last_undegraded":"2024-09-17T02:45:21.137044+0000","last_fullsized":"2024-09-17T02:45:21.137044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:57.648928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134832+0000","last_change":"2024-09-17T02:45:20.130812+0000","last_active":"2024-09-17T02:45:20.134832+0000","last_peered":"2024-09-17T02:45:20.134832+0000","last_clean":"2024-09-17T02:45:20.134832+0000","last_became_active":"2024-09-17T02:45:20.130480+0000","last_became_peered":"2024-09-17T02:45:20.130480+0000","last_unstale":"2024-09-17T02:45:20.134832+0000","last_undegraded":"2024-09-17T02:45:20.134832+0000","last_fullsized":"2024-09-17T02:45:20.134832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:43.366504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136826+0000","last_change":"2024-09-17T02:45:19.093011+0000","last_active":"2024-09-17T02:45:21.136826+0000","last_peered":"2024-09-17T02:45:21.136826+0000","last_clean":"2024-09-17T02:45:21.136826+0000","last_became_active":"2024-09-17T02:45:19.091917+0000","last_became_peered":"2024-09-17T02:45:19.091917+0000","last_unstale":"2024-09-17T02:45:21.136826+0000","last_undegraded":"2024-09-17T02:45:21.136826+0000","last_fullsized":"2024-09-17T02:45:21.136826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:03.939829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139221+0000","last_change":"2024-09-17T02:45:20.153283+0000","last_active":"2024-09-17T02:45:21.139221+0000","last_peered":"2024-09-17T02:45:21.139221+0000","last_clean":"2024-09-17T02:45:21.139221+0000","last_became_active":"2024-09-17T02:45:20.152428+0000","last_became_peered":"2024-09-17T02:45:20.152428+0000","last_unstale":"2024-09-17T02:45:21.139221+0000","last_undegraded":"2024-09-17T02:45:21.139221+0000","last_fullsized":"2024-09-17T02:45:21.139221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:36.231357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.104378+0000","last_change":"2024-09-17T02:45:19.096590+0000","last_active":"2024-09-17T02:45:20.104378+0000","last_peered":"2024-09-17T02:45:20.104378+0000","last_clean":"2024-09-17T02:45:20.104378+0000","last_became_active":"2024-09-17T02:45:19.096024+0000","last_became_peered":"2024-09-17T02:45:19.096024+0000","last_unstale":"2024-09-17T02:45:20.104378+0000","last_undegraded":"2024-09-17T02:45:20.104378+0000","last_fullsized":"2024-09-17T02:45:20.104378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:17.702397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.126109+0000","last_change":"2024-09-17T02:45:20.120878+0000","last_active":"2024-09-17T02:45:20.126109+0000","last_peered":"2024-09-17T02:45:20.126109+0000","last_clean":"2024-09-17T02:45:20.126109+0000","last_became_active":"2024-09-17T02:45:20.119874+0000","last_became_peered":"2024-09-17T02:45:20.119874+0000","last_unstale":"2024-09-17T02:45:20.126109+0000","last_undegraded":"2024-09-17T02:45:20.126109+0000","last_fullsized":"2024-09-17T02:45:20.126109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:55.317365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.108369+0000","last_change":"2024-09-17T02:45:19.084757+0000","last_active":"2024-09-17T02:45:20.108369+0000","last_peered":"2024-09-17T02:45:20.108369+0000","last_clean":"2024-09-17T02:45:20.108369+0000","last_became_active":"2024-09-17T02:45:19.084284+0000","last_became_peered":"2024-09-17T02:45:19.084284+0000","last_unstale":"2024-09-17T02:45:20.108369+0000","last_undegraded":"2024-09-17T02:45:20.108369+0000","last_fullsized":"2024-09-17T02:45:20.108369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:31.618471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130874+0000","last_change":"2024-09-17T02:45:20.128870+0000","last_active":"2024-09-17T02:45:20.130874+0000","last_peered":"2024-09-17T02:45:20.130874+0000","last_clean":"2024-09-17T02:45:20.130874+0000","last_became_active":"2024-09-17T02:45:20.128450+0000","last_became_peered":"2024-09-17T02:45:20.128450+0000","last_unstale":"2024-09-17T02:45:20.130874+0000","last_undegraded":"2024-09-17T02:45:20.130874+0000","last_fullsized":"2024-09-17T02:45:20.130874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:39.366822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.111875+0000","last_change":"2024-09-17T02:45:19.106516+0000","last_active":"2024-09-17T02:45:20.111875+0000","last_peered":"2024-09-17T02:45:20.111875+0000","last_clean":"2024-09-17T02:45:20.111875+0000","last_became_active":"2024-09-17T02:45:19.105883+0000","last_became_peered":"2024-09-17T02:45:19.105883+0000","last_unstale":"2024-09-17T02:45:20.111875+0000","last_undegraded":"2024-09-17T02:45:20.111875+0000","last_fullsized":"2024-09-17T02:45:20.111875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:39.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":77,"state":"creating+peering","last_fresh":"2024-09-17T02:45:19.063677+0000","last_change":"2024-09-17T02:45:19.063677+0000","last_active":"2024-09-17T02:45:19.049447+0000","last_peered":"2024-09-17T02:45:19.049447+0000","last_clean":"2024-09-17T02:45:19.049447+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:45:19.063677+0000","last_undegraded":"2024-09-17T02:45:19.063677+0000","last_fullsized":"2024-09-17T02:45:19.063677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.110077+0000","last_change":"2024-09-17T02:45:19.086253+0000","last_active":"2024-09-17T02:45:20.110077+0000","last_peered":"2024-09-17T02:45:20.110077+0000","last_clean":"2024-09-17T02:45:20.110077+0000","last_became_active":"2024-09-17T02:45:19.085807+0000","last_became_peered":"2024-09-17T02:45:19.085807+0000","last_unstale":"2024-09-17T02:45:20.110077+0000","last_undegraded":"2024-09-17T02:45:20.110077+0000","last_fullsized":"2024-09-17T02:45:20.110077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:43.903222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149388+0000","last_change":"2024-09-17T02:45:20.124232+0000","last_active":"2024-09-17T02:45:20.149388+0000","last_peered":"2024-09-17T02:45:20.149388+0000","last_clean":"2024-09-17T02:45:20.149388+0000","last_became_active":"2024-09-17T02:45:20.123450+0000","last_became_peered":"2024-09-17T02:45:20.123450+0000","last_unstale":"2024-09-17T02:45:20.149388+0000","last_undegraded":"2024-09-17T02:45:20.149388+0000","last_fullsized":"2024-09-17T02:45:20.149388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:01.142409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-17T02:45:19.094923+0000","last_change":"2024-09-17T02:45:19.087894+0000","last_active":"2024-09-17T02:45:19.094923+0000","last_peered":"2024-09-17T02:45:19.094923+0000","last_clean":"2024-09-17T02:45:19.094923+0000","last_became_active":"2024-09-17T02:45:19.086041+0000","last_became_peered":"2024-09-17T02:45:19.086041+0000","last_unstale":"2024-09-17T02:45:19.094923+0000","last_undegraded":"2024-09-17T02:45:19.094923+0000","last_fullsized":"2024-09-17T02:45:19.094923+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:30.454811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130153+0000","last_change":"2024-09-17T02:45:20.128777+0000","last_active":"2024-09-17T02:45:20.130153+0000","last_peered":"2024-09-17T02:45:20.130153+0000","last_clean":"2024-09-17T02:45:20.130153+0000","last_became_active":"2024-09-17T02:45:20.128362+0000","last_became_peered":"2024-09-17T02:45:20.128362+0000","last_unstale":"2024-09-17T02:45:20.130153+0000","last_undegraded":"2024-09-17T02:45:20.130153+0000","last_fullsized":"2024-09-17T02:45:20.130153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:46.647418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.111264+0000","last_change":"2024-09-17T02:45:19.099271+0000","last_active":"2024-09-17T02:45:20.111264+0000","last_peered":"2024-09-17T02:45:20.111264+0000","last_clean":"2024-09-17T02:45:20.111264+0000","last_became_active":"2024-09-17T02:45:19.088289+0000","last_became_peered":"2024-09-17T02:45:19.088289+0000","last_unstale":"2024-09-17T02:45:20.111264+0000","last_undegraded":"2024-09-17T02:45:20.111264+0000","last_fullsized":"2024-09-17T02:45:20.111264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:54.306776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":220,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.134645+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:45:21.134645+0000","last_peered":"2024-09-17T02:45:21.134645+0000","last_clean":"2024-09-17T02:45:21.134645+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:45:21.134645+0000","last_undegraded":"2024-09-17T02:45:21.134645+0000","last_fullsized":"2024-09-17T02:45:21.134645+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509500,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":954368,"data_stored":431399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509500,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1544192,"data_stored":1021205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509501,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":430769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509499,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":432463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509500,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":430747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509500,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":1022761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":950272,"data_stored":430769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574908,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339752,"statfs":{"total":96636764160,"available":96603906048,"internally_reserved":0,"allocated":921600,"data_stored":416913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:45:25.083 INFO:tasks.ceph.ceph_manager.ceph:PG 12.3e is not active+clean 2024-09-17T02:45:25.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.066330+0000', 'last_change': '2024-09-17T02:45:19.066330+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.066330+0000', 'last_undegraded': '2024-09-17T02:45:19.066330+0000', 'last_fullsized': '2024-09-17T02:45:19.066330+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.083 INFO:tasks.ceph.ceph_manager.ceph:PG 12.3d is not active+clean 2024-09-17T02:45:25.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.071003+0000', 'last_change': '2024-09-17T02:45:19.071003+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.071003+0000', 'last_undegraded': '2024-09-17T02:45:19.071003+0000', 'last_fullsized': '2024-09-17T02:45:19.071003+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.083 INFO:tasks.ceph.ceph_manager.ceph:PG 12.35 is not active+clean 2024-09-17T02:45:25.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.068073+0000', 'last_change': '2024-09-17T02:45:19.068073+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.068073+0000', 'last_undegraded': '2024-09-17T02:45:19.068073+0000', 'last_fullsized': '2024-09-17T02:45:19.068073+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.083 INFO:tasks.ceph.ceph_manager.ceph:PG 12.33 is not active+clean 2024-09-17T02:45:25.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.072804+0000', 'last_change': '2024-09-17T02:45:19.072804+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.072804+0000', 'last_undegraded': '2024-09-17T02:45:19.072804+0000', 'last_fullsized': '2024-09-17T02:45:19.072804+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.084 INFO:tasks.ceph.ceph_manager.ceph:PG 12.2e is not active+clean 2024-09-17T02:45:25.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.065085+0000', 'last_change': '2024-09-17T02:45:19.065085+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.065085+0000', 'last_undegraded': '2024-09-17T02:45:19.065085+0000', 'last_fullsized': '2024-09-17T02:45:19.065085+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.084 INFO:tasks.ceph.ceph_manager.ceph:PG 12.2a is not active+clean 2024-09-17T02:45:25.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.069206+0000', 'last_change': '2024-09-17T02:45:19.069206+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.069206+0000', 'last_undegraded': '2024-09-17T02:45:19.069206+0000', 'last_fullsized': '2024-09-17T02:45:19.069206+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.084 INFO:tasks.ceph.ceph_manager.ceph:PG 12.28 is not active+clean 2024-09-17T02:45:25.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.073370+0000', 'last_change': '2024-09-17T02:45:19.073370+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.073370+0000', 'last_undegraded': '2024-09-17T02:45:19.073370+0000', 'last_fullsized': '2024-09-17T02:45:19.073370+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.084 INFO:tasks.ceph.ceph_manager.ceph:PG 12.21 is not active+clean 2024-09-17T02:45:25.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.074374+0000', 'last_change': '2024-09-17T02:45:19.074374+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.074374+0000', 'last_undegraded': '2024-09-17T02:45:19.074374+0000', 'last_fullsized': '2024-09-17T02:45:19.074374+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.085 INFO:tasks.ceph.ceph_manager.ceph:PG 12.1d is not active+clean 2024-09-17T02:45:25.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.066618+0000', 'last_change': '2024-09-17T02:45:19.066618+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.066618+0000', 'last_undegraded': '2024-09-17T02:45:19.066618+0000', 'last_fullsized': '2024-09-17T02:45:19.066618+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.085 INFO:tasks.ceph.ceph_manager.ceph:PG 12.10 is not active+clean 2024-09-17T02:45:25.085 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.071853+0000', 'last_change': '2024-09-17T02:45:19.071853+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.071853+0000', 'last_undegraded': '2024-09-17T02:45:19.071853+0000', 'last_fullsized': '2024-09-17T02:45:19.071853+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.086 INFO:tasks.ceph.ceph_manager.ceph:PG 12.9 is not active+clean 2024-09-17T02:45:25.086 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.073129+0000', 'last_change': '2024-09-17T02:45:19.073129+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.073129+0000', 'last_undegraded': '2024-09-17T02:45:19.073129+0000', 'last_fullsized': '2024-09-17T02:45:19.073129+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.086 INFO:tasks.ceph.ceph_manager.ceph:PG 12.6 is not active+clean 2024-09-17T02:45:25.086 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 77, 'state': 'creating+peering', 'last_fresh': '2024-09-17T02:45:19.063677+0000', 'last_change': '2024-09-17T02:45:19.063677+0000', 'last_active': '2024-09-17T02:45:19.049447+0000', 'last_peered': '2024-09-17T02:45:19.049447+0000', 'last_clean': '2024-09-17T02:45:19.049447+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T02:45:19.063677+0000', 'last_undegraded': '2024-09-17T02:45:19.063677+0000', 'last_fullsized': '2024-09-17T02:45:19.063677+0000', 'mapping_epoch': 77, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'last_clean_scrub_stamp': '2024-09-17T02:45:19.049447+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T02:45:25.087 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T02:45:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.270+0000 7fd770892640 1 -- 172.21.15.131:0/2064837975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd758004810 msgr2=0x7fd758004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.270+0000 7fd770892640 1 --2- 172.21.15.131:0/2064837975 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd758004810 0x7fd758004c20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0c70a0 tx=0x7fd75401d170 comp rx=0 tx=0).stop 2024-09-17T02:45:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.270+0000 7fd76ad9d640 1 -- 172.21.15.131:0/2064837975 reap_dead start 2024-09-17T02:45:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 -- 172.21.15.131:0/2064837975 shutdown_connections 2024-09-17T02:45:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 -- 172.21.15.131:0/2064837975 >> 172.21.15.131:0/2064837975 conn(0x7fd76c06eea0 msgr2=0x7fd76c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 -- 172.21.15.131:0/2064837975 shutdown_connections 2024-09-17T02:45:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 -- 172.21.15.131:0/2064837975 wait complete. 2024-09-17T02:45:25.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 Processor -- start 2024-09-17T02:45:25.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 -- start start 2024-09-17T02:45:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd758004810 0x7fd76c0b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76c158560 0x7fd76c1fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd76c152310 0x7fd76c160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c168cf0 con 0x7fd76c158560 2024-09-17T02:45:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c15cb90 con 0x7fd76c152310 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd770892640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd76c168e70 con 0x7fd758004810 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd769d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76c158560 0x7fd76c1fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd769d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76c158560 0x7fd76c1fa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44172/0 (socket says 172.21.15.131:44172) 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd769d9b640 1 -- 172.21.15.131:0/4014085713 learned_addr learned my addr 172.21.15.131:0/4014085713 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd76ad9d640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd76c152310 0x7fd76c160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd76a59c640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd758004810 0x7fd76c0b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd769d9b640 1 -- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd758004810 msgr2=0x7fd76c0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd769d9b640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd758004810 0x7fd76c0b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd769d9b640 1 -- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd76c152310 msgr2=0x7fd76c160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd769d9b640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd76c152310 0x7fd76c160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd769d9b640 1 -- 172.21.15.131:0/4014085713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c153810 con 0x7fd76c158560 2024-09-17T02:45:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd76ad9d640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd76c152310 0x7fd76c160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd769d9b640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76c158560 0x7fd76c1fa9c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd75401cfa0 tx=0x7fd754005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.274+0000 7fd762ffd640 1 -- 172.21.15.131:0/4014085713 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7540b1040 con 0x7fd76c158560 2024-09-17T02:45:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.278+0000 7fd762ffd640 1 -- 172.21.15.131:0/4014085713 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7540ac400 con 0x7fd76c158560 2024-09-17T02:45:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.278+0000 7fd770892640 1 -- 172.21.15.131:0/4014085713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c161400 con 0x7fd76c158560 2024-09-17T02:45:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.278+0000 7fd762ffd640 1 -- 172.21.15.131:0/4014085713 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd754004290 con 0x7fd76c158560 2024-09-17T02:45:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.278+0000 7fd770892640 1 -- 172.21.15.131:0/4014085713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c1fa1c0 con 0x7fd76c158560 2024-09-17T02:45:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.278+0000 7fd770892640 1 -- 172.21.15.131:0/4014085713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76c0c4580 con 0x7fd76c158560 2024-09-17T02:45:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.278+0000 7fd762ffd640 1 -- 172.21.15.131:0/4014085713 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd7540bd020 con 0x7fd76c158560 2024-09-17T02:45:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.278+0000 7fd762ffd640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd74c073200 0x7fd74c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.278+0000 7fd762ffd640 1 -- 172.21.15.131:0/4014085713 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7fd75412ec40 con 0x7fd76c158560 2024-09-17T02:45:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.278+0000 7fd76a59c640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd74c073200 0x7fd74c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.278+0000 7fd76a59c640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd74c073200 0x7fd74c075710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd75c000f90 tx=0x7fd75c006d50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:25.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.282+0000 7fd762ffd640 1 -- 172.21.15.131:0/4014085713 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd7540c4020 con 0x7fd76c158560 2024-09-17T02:45:25.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.498+0000 7fd770892640 1 -- 172.21.15.131:0/4014085713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd76c073310 con 0x7fd76c158560 2024-09-17T02:45:25.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd762ffd640 1 -- 172.21.15.131:0/4014085713 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1303 (secure 0 0 0) 0x7fd754016ee0 con 0x7fd76c158560 2024-09-17T02:45:25.498 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:25.499 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":299,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":79,"num_osds":8,"num_up_osds":8,"osd_up_since":1726540840,"num_in_osds":8,"osd_in_since":1726540830,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":85},{"state_name":"creating+peering","count":12}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":264298496,"bytes_avail":772829814784,"bytes_total":773094113280,"inactive_pgs_ratio":0.12371134012937546,"write_bytes_sec":2770,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":46,"btime":"2024-09-17T02:45:23:141401+0000","id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6399}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-17T02:44:45.210875+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T02:45:25.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 -- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd74c073200 msgr2=0x7fd74c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd74c073200 0x7fd74c075710 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd75c000f90 tx=0x7fd75c006d50 comp rx=0 tx=0).stop 2024-09-17T02:45:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 -- 172.21.15.131:0/4014085713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76c158560 msgr2=0x7fd76c1fa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76c158560 0x7fd76c1fa9c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fd75401cfa0 tx=0x7fd754005540 comp rx=0 tx=0).stop 2024-09-17T02:45:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 -- 172.21.15.131:0/4014085713 shutdown_connections 2024-09-17T02:45:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd74c073200 0x7fd74c075710 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd76c152310 0x7fd76c160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd76c158560 0x7fd76c1fa9c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 --2- 172.21.15.131:0/4014085713 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd758004810 0x7fd76c0b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 -- 172.21.15.131:0/4014085713 >> 172.21.15.131:0/4014085713 conn(0x7fd76c06eea0 msgr2=0x7fd76c0c6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 -- 172.21.15.131:0/4014085713 shutdown_connections 2024-09-17T02:45:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.502+0000 7fd770892640 1 -- 172.21.15.131:0/4014085713 wait complete. 2024-09-17T02:45:25.519 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T02:45:25.519 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:45:25.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.662+0000 7f94d0fda640 1 -- 172.21.15.131:0/3961760437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94cc15f7d0 msgr2=0x7f94cc162410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.662+0000 7f94d0fda640 1 --2- 172.21.15.131:0/3961760437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94cc15f7d0 0x7f94cc162410 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0ce990 tx=0x7f94b401c8a0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.662+0000 7f94d0fda640 1 -- 172.21.15.131:0/3961760437 shutdown_connections 2024-09-17T02:45:25.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.662+0000 7f94d0fda640 1 --2- 172.21.15.131:0/3961760437 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94cc15f7d0 0x7f94cc162410 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.662+0000 7f94d0fda640 1 --2- 172.21.15.131:0/3961760437 >> v2:172.21.15.189:3300/0 conn(0x7f94cc162950 0x7f94cc162d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.662+0000 7f94d0fda640 1 --2- 172.21.15.131:0/3961760437 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f94cc0cf710 0x7f94cc15f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.662+0000 7f94d0fda640 1 -- 172.21.15.131:0/3961760437 >> 172.21.15.131:0/3961760437 conn(0x7f94cc0d1b10 msgr2=0x7f94cc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:25.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.662+0000 7f94d0fda640 1 -- 172.21.15.131:0/3961760437 shutdown_connections 2024-09-17T02:45:25.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.662+0000 7f94d0fda640 1 -- 172.21.15.131:0/3961760437 wait complete. 2024-09-17T02:45:25.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94d0fda640 1 Processor -- start 2024-09-17T02:45:25.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94d0fda640 1 -- start start 2024-09-17T02:45:25.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94d0fda640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94cc0cf710 0x7f94cc159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:25.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94d0fda640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f94cc15f7d0 0x7f94cc160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:25.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94d0fda640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc162950 0x7f94cc15b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:25.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94d0fda640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc0d2df0 con 0x7f94cc0cf710 2024-09-17T02:45:25.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94d0fda640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f94cc0d2c70 con 0x7f94cc162950 2024-09-17T02:45:25.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94d0fda640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f94cc0d2f70 con 0x7f94cc15f7d0 2024-09-17T02:45:25.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94ca59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94cc0cf710 0x7f94cc159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:25.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94ca59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94cc0cf710 0x7f94cc159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44182/0 (socket says 172.21.15.131:44182) 2024-09-17T02:45:25.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94ca59c640 1 -- 172.21.15.131:0/937375216 learned_addr learned my addr 172.21.15.131:0/937375216 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:25.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94c3fff640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f94cc15f7d0 0x7f94cc160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:25.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94cad9d640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc162950 0x7f94cc15b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94ca59c640 1 -- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f94cc15f7d0 msgr2=0x7f94cc160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94ca59c640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f94cc15f7d0 0x7f94cc160de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94ca59c640 1 -- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc162950 msgr2=0x7f94cc15b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94ca59c640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc162950 0x7f94cc15b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94ca59c640 1 -- 172.21.15.131:0/937375216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94cc15a400 con 0x7f94cc0cf710 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94cad9d640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc162950 0x7f94cc15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94ca59c640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94cc0cf710 0x7f94cc159600 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d6b0 tx=0x7f94bc00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94c2ffd640 1 -- 172.21.15.131:0/937375216 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc017020 con 0x7f94cc0cf710 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94c2ffd640 1 -- 172.21.15.131:0/937375216 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94bc004320 con 0x7f94cc0cf710 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94d0fda640 1 -- 172.21.15.131:0/937375216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94cc159b40 con 0x7f94cc0cf710 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94c2ffd640 1 -- 172.21.15.131:0/937375216 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc009a80 con 0x7f94cc0cf710 2024-09-17T02:45:25.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94d0fda640 1 -- 172.21.15.131:0/937375216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94cc0b4dd0 con 0x7f94cc0cf710 2024-09-17T02:45:25.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.666+0000 7f94d0fda640 1 -- 172.21.15.131:0/937375216 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94cc0d3420 con 0x7f94cc0cf710 2024-09-17T02:45:25.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.670+0000 7f94c2ffd640 1 -- 172.21.15.131:0/937375216 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f94bc01a020 con 0x7f94cc0cf710 2024-09-17T02:45:25.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.670+0000 7f94c2ffd640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f94ac073200 0x7f94ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:25.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.670+0000 7f94c2ffd640 1 -- 172.21.15.131:0/937375216 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f94bc094ca0 con 0x7f94cc0cf710 2024-09-17T02:45:25.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.670+0000 7f94c3fff640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f94ac073200 0x7f94ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:25.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.670+0000 7f94c3fff640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f94ac073200 0x7f94ac075710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f94b40af000 tx=0x7f94b401b0e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:25.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.674+0000 7f94c2ffd640 1 -- 172.21.15.131:0/937375216 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f94bc021020 con 0x7f94cc0cf710 2024-09-17T02:45:25.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.826+0000 7f94d0fda640 1 -- 172.21.15.131:0/937375216 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f94cc0b5010 con 0x7f94ac073200 2024-09-17T02:45:25.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.838+0000 7f94c2ffd640 1 -- 172.21.15.131:0/937375216 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252060 (secure 0 0 0) 0x7f94cc0b5010 con 0x7f94ac073200 2024-09-17T02:45:25.839 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:25.841 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:45:25.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 -- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f94ac073200 msgr2=0x7f94ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f94ac073200 0x7f94ac075710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f94b40af000 tx=0x7f94b401b0e0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 -- 172.21.15.131:0/937375216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94cc0cf710 msgr2=0x7f94cc159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:25.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94cc0cf710 0x7f94cc159600 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f94bc00d6b0 tx=0x7f94bc00db70 comp rx=0 tx=0).stop 2024-09-17T02:45:25.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 -- 172.21.15.131:0/937375216 shutdown_connections 2024-09-17T02:45:25.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f94ac073200 0x7f94ac075710 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f94cc162950 0x7f94cc15b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f94cc15f7d0 0x7f94cc160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 --2- 172.21.15.131:0/937375216 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f94cc0cf710 0x7f94cc159600 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:25.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 -- 172.21.15.131:0/937375216 >> 172.21.15.131:0/937375216 conn(0x7f94cc0d1b10 msgr2=0x7f94cc15df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:25.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 -- 172.21.15.131:0/937375216 shutdown_connections 2024-09-17T02:45:25.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:25.842+0000 7f94d0fda640 1 -- 172.21.15.131:0/937375216 wait complete. 2024-09-17T02:45:25.860 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-17T02:45:25.226351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258148,"kb_used_data":8628,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716572,"statfs":{"total":773094113280,"available":772829769728,"internally_reserved":0,"allocated":8835072,"data_stored":4638224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.167965"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135946+0000","last_change":"2024-09-17T02:45:20.134496+0000","last_active":"2024-09-17T02:45:21.135946+0000","last_peered":"2024-09-17T02:45:21.135946+0000","last_clean":"2024-09-17T02:45:21.135946+0000","last_became_active":"2024-09-17T02:45:20.134052+0000","last_became_peered":"2024-09-17T02:45:20.134052+0000","last_unstale":"2024-09-17T02:45:21.135946+0000","last_undegraded":"2024-09-17T02:45:21.135946+0000","last_fullsized":"2024-09-17T02:45:21.135946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:33.880550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.140804+0000","last_change":"2024-09-17T02:45:20.144524+0000","last_active":"2024-09-17T02:45:21.140804+0000","last_peered":"2024-09-17T02:45:21.140804+0000","last_clean":"2024-09-17T02:45:21.140804+0000","last_became_active":"2024-09-17T02:45:20.143771+0000","last_became_peered":"2024-09-17T02:45:20.143771+0000","last_unstale":"2024-09-17T02:45:21.140804+0000","last_undegraded":"2024-09-17T02:45:21.140804+0000","last_fullsized":"2024-09-17T02:45:21.140804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:04.800055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.141004+0000","last_change":"2024-09-17T02:45:20.134523+0000","last_active":"2024-09-17T02:45:21.141004+0000","last_peered":"2024-09-17T02:45:21.141004+0000","last_clean":"2024-09-17T02:45:21.141004+0000","last_became_active":"2024-09-17T02:45:20.133784+0000","last_became_peered":"2024-09-17T02:45:20.133784+0000","last_unstale":"2024-09-17T02:45:21.141004+0000","last_undegraded":"2024-09-17T02:45:21.141004+0000","last_fullsized":"2024-09-17T02:45:21.141004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:50:03.665497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.127671+0000","last_change":"2024-09-17T02:45:20.124901+0000","last_active":"2024-09-17T02:45:20.127671+0000","last_peered":"2024-09-17T02:45:20.127671+0000","last_clean":"2024-09-17T02:45:20.127671+0000","last_became_active":"2024-09-17T02:45:20.124392+0000","last_became_peered":"2024-09-17T02:45:20.124392+0000","last_unstale":"2024-09-17T02:45:20.127671+0000","last_undegraded":"2024-09-17T02:45:20.127671+0000","last_fullsized":"2024-09-17T02:45:20.127671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:56.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.157150+0000","last_change":"2024-09-17T02:45:20.151934+0000","last_active":"2024-09-17T02:45:20.157150+0000","last_peered":"2024-09-17T02:45:20.157150+0000","last_clean":"2024-09-17T02:45:20.157150+0000","last_became_active":"2024-09-17T02:45:20.149652+0000","last_became_peered":"2024-09-17T02:45:20.149652+0000","last_unstale":"2024-09-17T02:45:20.157150+0000","last_undegraded":"2024-09-17T02:45:20.157150+0000","last_fullsized":"2024-09-17T02:45:20.157150+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:28:06.754098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.162147+0000","last_change":"2024-09-17T02:45:20.156609+0000","last_active":"2024-09-17T02:45:20.162147+0000","last_peered":"2024-09-17T02:45:20.162147+0000","last_clean":"2024-09-17T02:45:20.162147+0000","last_became_active":"2024-09-17T02:45:20.155997+0000","last_became_peered":"2024-09-17T02:45:20.155997+0000","last_unstale":"2024-09-17T02:45:20.162147+0000","last_undegraded":"2024-09-17T02:45:20.162147+0000","last_fullsized":"2024-09-17T02:45:20.162147+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:42:49.213100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.141570+0000","last_change":"2024-09-17T02:45:20.138831+0000","last_active":"2024-09-17T02:45:20.141570+0000","last_peered":"2024-09-17T02:45:20.141570+0000","last_clean":"2024-09-17T02:45:20.141570+0000","last_became_active":"2024-09-17T02:45:20.138468+0000","last_became_peered":"2024-09-17T02:45:20.138468+0000","last_unstale":"2024-09-17T02:45:20.141570+0000","last_undegraded":"2024-09-17T02:45:20.141570+0000","last_fullsized":"2024-09-17T02:45:20.141570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:17.239517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137912+0000","last_change":"2024-09-17T02:45:20.151755+0000","last_active":"2024-09-17T02:45:21.137912+0000","last_peered":"2024-09-17T02:45:21.137912+0000","last_clean":"2024-09-17T02:45:21.137912+0000","last_became_active":"2024-09-17T02:45:20.139390+0000","last_became_peered":"2024-09-17T02:45:20.139390+0000","last_unstale":"2024-09-17T02:45:21.137912+0000","last_undegraded":"2024-09-17T02:45:21.137912+0000","last_fullsized":"2024-09-17T02:45:21.137912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:04.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130320+0000","last_change":"2024-09-17T02:45:20.128764+0000","last_active":"2024-09-17T02:45:20.130320+0000","last_peered":"2024-09-17T02:45:20.130320+0000","last_clean":"2024-09-17T02:45:20.130320+0000","last_became_active":"2024-09-17T02:45:20.128381+0000","last_became_peered":"2024-09-17T02:45:20.128381+0000","last_unstale":"2024-09-17T02:45:20.130320+0000","last_undegraded":"2024-09-17T02:45:20.130320+0000","last_fullsized":"2024-09-17T02:45:20.130320+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:20:35.763836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135793+0000","last_change":"2024-09-17T02:45:20.134184+0000","last_active":"2024-09-17T02:45:21.135793+0000","last_peered":"2024-09-17T02:45:21.135793+0000","last_clean":"2024-09-17T02:45:21.135793+0000","last_became_active":"2024-09-17T02:45:20.133826+0000","last_became_peered":"2024-09-17T02:45:20.133826+0000","last_unstale":"2024-09-17T02:45:21.135793+0000","last_undegraded":"2024-09-17T02:45:21.135793+0000","last_fullsized":"2024-09-17T02:45:21.135793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:41.858427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.141636+0000","last_change":"2024-09-17T02:45:20.140353+0000","last_active":"2024-09-17T02:45:21.141636+0000","last_peered":"2024-09-17T02:45:21.141636+0000","last_clean":"2024-09-17T02:45:21.141636+0000","last_became_active":"2024-09-17T02:45:20.139415+0000","last_became_peered":"2024-09-17T02:45:20.139415+0000","last_unstale":"2024-09-17T02:45:21.141636+0000","last_undegraded":"2024-09-17T02:45:21.141636+0000","last_fullsized":"2024-09-17T02:45:21.141636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:43.957898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.146278+0000","last_change":"2024-09-17T02:45:20.137823+0000","last_active":"2024-09-17T02:45:20.146278+0000","last_peered":"2024-09-17T02:45:20.146278+0000","last_clean":"2024-09-17T02:45:20.146278+0000","last_became_active":"2024-09-17T02:45:20.137420+0000","last_became_peered":"2024-09-17T02:45:20.137420+0000","last_unstale":"2024-09-17T02:45:20.146278+0000","last_undegraded":"2024-09-17T02:45:20.146278+0000","last_fullsized":"2024-09-17T02:45:20.146278+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:14.885437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138648+0000","last_change":"2024-09-17T02:45:20.130548+0000","last_active":"2024-09-17T02:45:21.138648+0000","last_peered":"2024-09-17T02:45:21.138648+0000","last_clean":"2024-09-17T02:45:21.138648+0000","last_became_active":"2024-09-17T02:45:20.128892+0000","last_became_peered":"2024-09-17T02:45:20.128892+0000","last_unstale":"2024-09-17T02:45:21.138648+0000","last_undegraded":"2024-09-17T02:45:21.138648+0000","last_fullsized":"2024-09-17T02:45:21.138648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:04.543314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139109+0000","last_change":"2024-09-17T02:45:20.154359+0000","last_active":"2024-09-17T02:45:21.139109+0000","last_peered":"2024-09-17T02:45:21.139109+0000","last_clean":"2024-09-17T02:45:21.139109+0000","last_became_active":"2024-09-17T02:45:20.138831+0000","last_became_peered":"2024-09-17T02:45:20.138831+0000","last_unstale":"2024-09-17T02:45:21.139109+0000","last_undegraded":"2024-09-17T02:45:21.139109+0000","last_fullsized":"2024-09-17T02:45:21.139109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:26.541024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151644+0000","last_change":"2024-09-17T02:45:20.136343+0000","last_active":"2024-09-17T02:45:20.151644+0000","last_peered":"2024-09-17T02:45:20.151644+0000","last_clean":"2024-09-17T02:45:20.151644+0000","last_became_active":"2024-09-17T02:45:20.135036+0000","last_became_peered":"2024-09-17T02:45:20.135036+0000","last_unstale":"2024-09-17T02:45:20.151644+0000","last_undegraded":"2024-09-17T02:45:20.151644+0000","last_fullsized":"2024-09-17T02:45:20.151644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:15.507022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134377+0000","last_change":"2024-09-17T02:45:20.133650+0000","last_active":"2024-09-17T02:45:20.134377+0000","last_peered":"2024-09-17T02:45:20.134377+0000","last_clean":"2024-09-17T02:45:20.134377+0000","last_became_active":"2024-09-17T02:45:20.133246+0000","last_became_peered":"2024-09-17T02:45:20.133246+0000","last_unstale":"2024-09-17T02:45:20.134377+0000","last_undegraded":"2024-09-17T02:45:20.134377+0000","last_fullsized":"2024-09-17T02:45:20.134377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:23:31.528537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138676+0000","last_change":"2024-09-17T02:45:20.138431+0000","last_active":"2024-09-17T02:45:21.138676+0000","last_peered":"2024-09-17T02:45:21.138676+0000","last_clean":"2024-09-17T02:45:21.138676+0000","last_became_active":"2024-09-17T02:45:20.137633+0000","last_became_peered":"2024-09-17T02:45:20.137633+0000","last_unstale":"2024-09-17T02:45:21.138676+0000","last_undegraded":"2024-09-17T02:45:21.138676+0000","last_fullsized":"2024-09-17T02:45:21.138676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:25.489158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.141003+0000","last_change":"2024-09-17T02:45:20.132227+0000","last_active":"2024-09-17T02:45:21.141003+0000","last_peered":"2024-09-17T02:45:21.141003+0000","last_clean":"2024-09-17T02:45:21.141003+0000","last_became_active":"2024-09-17T02:45:20.130783+0000","last_became_peered":"2024-09-17T02:45:20.130783+0000","last_unstale":"2024-09-17T02:45:21.141003+0000","last_undegraded":"2024-09-17T02:45:21.141003+0000","last_fullsized":"2024-09-17T02:45:21.141003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:25.569753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138525+0000","last_change":"2024-09-17T02:45:20.138096+0000","last_active":"2024-09-17T02:45:21.138525+0000","last_peered":"2024-09-17T02:45:21.138525+0000","last_clean":"2024-09-17T02:45:21.138525+0000","last_became_active":"2024-09-17T02:45:20.137246+0000","last_became_peered":"2024-09-17T02:45:20.137246+0000","last_unstale":"2024-09-17T02:45:21.138525+0000","last_undegraded":"2024-09-17T02:45:21.138525+0000","last_fullsized":"2024-09-17T02:45:21.138525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:08.405979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.148092+0000","last_change":"2024-09-17T02:45:20.144454+0000","last_active":"2024-09-17T02:45:20.148092+0000","last_peered":"2024-09-17T02:45:20.148092+0000","last_clean":"2024-09-17T02:45:20.148092+0000","last_became_active":"2024-09-17T02:45:20.143671+0000","last_became_peered":"2024-09-17T02:45:20.143671+0000","last_unstale":"2024-09-17T02:45:20.148092+0000","last_undegraded":"2024-09-17T02:45:20.148092+0000","last_fullsized":"2024-09-17T02:45:20.148092+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:26.359990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.147886+0000","last_change":"2024-09-17T02:45:20.140711+0000","last_active":"2024-09-17T02:45:20.147886+0000","last_peered":"2024-09-17T02:45:20.147886+0000","last_clean":"2024-09-17T02:45:20.147886+0000","last_became_active":"2024-09-17T02:45:20.140390+0000","last_became_peered":"2024-09-17T02:45:20.140390+0000","last_unstale":"2024-09-17T02:45:20.147886+0000","last_undegraded":"2024-09-17T02:45:20.147886+0000","last_fullsized":"2024-09-17T02:45:20.147886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.473725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139684+0000","last_change":"2024-09-17T02:45:20.131403+0000","last_active":"2024-09-17T02:45:21.139684+0000","last_peered":"2024-09-17T02:45:21.139684+0000","last_clean":"2024-09-17T02:45:21.139684+0000","last_became_active":"2024-09-17T02:45:20.130541+0000","last_became_peered":"2024-09-17T02:45:20.130541+0000","last_unstale":"2024-09-17T02:45:21.139684+0000","last_undegraded":"2024-09-17T02:45:21.139684+0000","last_fullsized":"2024-09-17T02:45:21.139684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:02.517427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135331+0000","last_change":"2024-09-17T02:45:20.120377+0000","last_active":"2024-09-17T02:45:21.135331+0000","last_peered":"2024-09-17T02:45:21.135331+0000","last_clean":"2024-09-17T02:45:21.135331+0000","last_became_active":"2024-09-17T02:45:20.119336+0000","last_became_peered":"2024-09-17T02:45:20.119336+0000","last_unstale":"2024-09-17T02:45:21.135331+0000","last_undegraded":"2024-09-17T02:45:21.135331+0000","last_fullsized":"2024-09-17T02:45:21.135331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:43.174152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139403+0000","last_change":"2024-09-17T02:45:20.132058+0000","last_active":"2024-09-17T02:45:21.139403+0000","last_peered":"2024-09-17T02:45:21.139403+0000","last_clean":"2024-09-17T02:45:21.139403+0000","last_became_active":"2024-09-17T02:45:20.128816+0000","last_became_peered":"2024-09-17T02:45:20.128816+0000","last_unstale":"2024-09-17T02:45:21.139403+0000","last_undegraded":"2024-09-17T02:45:21.139403+0000","last_fullsized":"2024-09-17T02:45:21.139403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:21.117339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.143641+0000","last_change":"2024-09-17T02:45:20.136877+0000","last_active":"2024-09-17T02:45:20.143641+0000","last_peered":"2024-09-17T02:45:20.143641+0000","last_clean":"2024-09-17T02:45:20.143641+0000","last_became_active":"2024-09-17T02:45:20.136227+0000","last_became_peered":"2024-09-17T02:45:20.136227+0000","last_unstale":"2024-09-17T02:45:20.143641+0000","last_undegraded":"2024-09-17T02:45:20.143641+0000","last_fullsized":"2024-09-17T02:45:20.143641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:06.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.131033+0000","last_change":"2024-09-17T02:45:20.129949+0000","last_active":"2024-09-17T02:45:20.131033+0000","last_peered":"2024-09-17T02:45:20.131033+0000","last_clean":"2024-09-17T02:45:20.131033+0000","last_became_active":"2024-09-17T02:45:20.129519+0000","last_became_peered":"2024-09-17T02:45:20.129519+0000","last_unstale":"2024-09-17T02:45:20.131033+0000","last_undegraded":"2024-09-17T02:45:20.131033+0000","last_fullsized":"2024-09-17T02:45:20.131033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:53.379792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.127890+0000","last_change":"2024-09-17T02:45:20.125488+0000","last_active":"2024-09-17T02:45:20.127890+0000","last_peered":"2024-09-17T02:45:20.127890+0000","last_clean":"2024-09-17T02:45:20.127890+0000","last_became_active":"2024-09-17T02:45:20.124765+0000","last_became_peered":"2024-09-17T02:45:20.124765+0000","last_unstale":"2024-09-17T02:45:20.127890+0000","last_undegraded":"2024-09-17T02:45:20.127890+0000","last_fullsized":"2024-09-17T02:45:20.127890+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:41.858427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.125889+0000","last_change":"2024-09-17T02:45:20.121902+0000","last_active":"2024-09-17T02:45:20.125889+0000","last_peered":"2024-09-17T02:45:20.125889+0000","last_clean":"2024-09-17T02:45:20.125889+0000","last_became_active":"2024-09-17T02:45:20.121274+0000","last_became_peered":"2024-09-17T02:45:20.121274+0000","last_unstale":"2024-09-17T02:45:20.125889+0000","last_undegraded":"2024-09-17T02:45:20.125889+0000","last_fullsized":"2024-09-17T02:45:20.125889+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:54:19.026083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.150174+0000","last_change":"2024-09-17T02:45:20.147039+0000","last_active":"2024-09-17T02:45:20.150174+0000","last_peered":"2024-09-17T02:45:20.150174+0000","last_clean":"2024-09-17T02:45:20.150174+0000","last_became_active":"2024-09-17T02:45:20.145977+0000","last_became_peered":"2024-09-17T02:45:20.145977+0000","last_unstale":"2024-09-17T02:45:20.150174+0000","last_undegraded":"2024-09-17T02:45:20.150174+0000","last_fullsized":"2024-09-17T02:45:20.150174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:25.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135630+0000","last_change":"2024-09-17T02:45:20.134028+0000","last_active":"2024-09-17T02:45:21.135630+0000","last_peered":"2024-09-17T02:45:21.135630+0000","last_clean":"2024-09-17T02:45:21.135630+0000","last_became_active":"2024-09-17T02:45:20.133628+0000","last_became_peered":"2024-09-17T02:45:20.133628+0000","last_unstale":"2024-09-17T02:45:21.135630+0000","last_undegraded":"2024-09-17T02:45:21.135630+0000","last_fullsized":"2024-09-17T02:45:21.135630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:56.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139176+0000","last_change":"2024-09-17T02:45:20.147793+0000","last_active":"2024-09-17T02:45:21.139176+0000","last_peered":"2024-09-17T02:45:21.139176+0000","last_clean":"2024-09-17T02:45:21.139176+0000","last_became_active":"2024-09-17T02:45:20.147199+0000","last_became_peered":"2024-09-17T02:45:20.147199+0000","last_unstale":"2024-09-17T02:45:21.139176+0000","last_undegraded":"2024-09-17T02:45:21.139176+0000","last_fullsized":"2024-09-17T02:45:21.139176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:38.337307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.129861+0000","last_change":"2024-09-17T02:45:20.127695+0000","last_active":"2024-09-17T02:45:20.129861+0000","last_peered":"2024-09-17T02:45:20.129861+0000","last_clean":"2024-09-17T02:45:20.129861+0000","last_became_active":"2024-09-17T02:45:20.127323+0000","last_became_peered":"2024-09-17T02:45:20.127323+0000","last_unstale":"2024-09-17T02:45:20.129861+0000","last_undegraded":"2024-09-17T02:45:20.129861+0000","last_fullsized":"2024-09-17T02:45:20.129861+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:42.063380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.132767+0000","last_change":"2024-09-17T02:45:20.122630+0000","last_active":"2024-09-17T02:45:20.132767+0000","last_peered":"2024-09-17T02:45:20.132767+0000","last_clean":"2024-09-17T02:45:20.132767+0000","last_became_active":"2024-09-17T02:45:20.120596+0000","last_became_peered":"2024-09-17T02:45:20.120596+0000","last_unstale":"2024-09-17T02:45:20.132767+0000","last_undegraded":"2024-09-17T02:45:20.132767+0000","last_fullsized":"2024-09-17T02:45:20.132767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:16.342312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"78'4","reported_seq":17,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165404+0000","last_change":"2024-09-17T02:45:19.105415+0000","last_active":"2024-09-17T02:45:20.165404+0000","last_peered":"2024-09-17T02:45:20.165404+0000","last_clean":"2024-09-17T02:45:20.165404+0000","last_became_active":"2024-09-17T02:45:19.104799+0000","last_became_peered":"2024-09-17T02:45:19.104799+0000","last_unstale":"2024-09-17T02:45:20.165404+0000","last_undegraded":"2024-09-17T02:45:20.165404+0000","last_fullsized":"2024-09-17T02:45:20.165404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:15:52.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.153720+0000","last_change":"2024-09-17T02:45:20.151127+0000","last_active":"2024-09-17T02:45:20.153720+0000","last_peered":"2024-09-17T02:45:20.153720+0000","last_clean":"2024-09-17T02:45:20.153720+0000","last_became_active":"2024-09-17T02:45:20.150324+0000","last_became_peered":"2024-09-17T02:45:20.150324+0000","last_unstale":"2024-09-17T02:45:20.153720+0000","last_undegraded":"2024-09-17T02:45:20.153720+0000","last_fullsized":"2024-09-17T02:45:20.153720+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:26.406192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151669+0000","last_change":"2024-09-17T02:45:19.079006+0000","last_active":"2024-09-17T02:45:20.151669+0000","last_peered":"2024-09-17T02:45:20.151669+0000","last_clean":"2024-09-17T02:45:20.151669+0000","last_became_active":"2024-09-17T02:45:19.078512+0000","last_became_peered":"2024-09-17T02:45:19.078512+0000","last_unstale":"2024-09-17T02:45:20.151669+0000","last_undegraded":"2024-09-17T02:45:20.151669+0000","last_fullsized":"2024-09-17T02:45:20.151669+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:48.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134859+0000","last_change":"2024-09-17T02:45:20.131374+0000","last_active":"2024-09-17T02:45:20.134859+0000","last_peered":"2024-09-17T02:45:20.134859+0000","last_clean":"2024-09-17T02:45:20.134859+0000","last_became_active":"2024-09-17T02:45:20.129652+0000","last_became_peered":"2024-09-17T02:45:20.129652+0000","last_unstale":"2024-09-17T02:45:20.134859+0000","last_undegraded":"2024-09-17T02:45:20.134859+0000","last_fullsized":"2024-09-17T02:45:20.134859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:03:02.100162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136882+0000","last_change":"2024-09-17T02:45:19.078238+0000","last_active":"2024-09-17T02:45:21.136882+0000","last_peered":"2024-09-17T02:45:21.136882+0000","last_clean":"2024-09-17T02:45:21.136882+0000","last_became_active":"2024-09-17T02:45:19.077495+0000","last_became_peered":"2024-09-17T02:45:19.077495+0000","last_unstale":"2024-09-17T02:45:21.136882+0000","last_undegraded":"2024-09-17T02:45:21.136882+0000","last_fullsized":"2024-09-17T02:45:21.136882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:25.395603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130326+0000","last_change":"2024-09-17T02:45:20.127140+0000","last_active":"2024-09-17T02:45:20.130326+0000","last_peered":"2024-09-17T02:45:20.130326+0000","last_clean":"2024-09-17T02:45:20.130326+0000","last_became_active":"2024-09-17T02:45:20.126455+0000","last_became_peered":"2024-09-17T02:45:20.126455+0000","last_unstale":"2024-09-17T02:45:20.130326+0000","last_undegraded":"2024-09-17T02:45:20.130326+0000","last_fullsized":"2024-09-17T02:45:20.130326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:33.880550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"78'2","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149384+0000","last_change":"2024-09-17T02:45:19.092392+0000","last_active":"2024-09-17T02:45:20.149384+0000","last_peered":"2024-09-17T02:45:20.149384+0000","last_clean":"2024-09-17T02:45:20.149384+0000","last_became_active":"2024-09-17T02:45:19.091367+0000","last_became_peered":"2024-09-17T02:45:19.091367+0000","last_unstale":"2024-09-17T02:45:20.149384+0000","last_undegraded":"2024-09-17T02:45:20.149384+0000","last_fullsized":"2024-09-17T02:45:20.149384+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:57.648928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.143331+0000","last_change":"2024-09-17T02:45:20.134096+0000","last_active":"2024-09-17T02:45:20.143331+0000","last_peered":"2024-09-17T02:45:20.143331+0000","last_clean":"2024-09-17T02:45:20.143331+0000","last_became_active":"2024-09-17T02:45:20.133166+0000","last_became_peered":"2024-09-17T02:45:20.133166+0000","last_unstale":"2024-09-17T02:45:20.143331+0000","last_undegraded":"2024-09-17T02:45:20.143331+0000","last_fullsized":"2024-09-17T02:45:20.143331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:47:17.390929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.147932+0000","last_change":"2024-09-17T02:45:19.095580+0000","last_active":"2024-09-17T02:45:20.147932+0000","last_peered":"2024-09-17T02:45:20.147932+0000","last_clean":"2024-09-17T02:45:20.147932+0000","last_became_active":"2024-09-17T02:45:19.094437+0000","last_became_peered":"2024-09-17T02:45:19.094437+0000","last_unstale":"2024-09-17T02:45:20.147932+0000","last_undegraded":"2024-09-17T02:45:20.147932+0000","last_fullsized":"2024-09-17T02:45:20.147932+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:59.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.142142+0000","last_change":"2024-09-17T02:45:20.123316+0000","last_active":"2024-09-17T02:45:21.142142+0000","last_peered":"2024-09-17T02:45:21.142142+0000","last_clean":"2024-09-17T02:45:21.142142+0000","last_became_active":"2024-09-17T02:45:20.122550+0000","last_became_peered":"2024-09-17T02:45:20.122550+0000","last_unstale":"2024-09-17T02:45:21.142142+0000","last_undegraded":"2024-09-17T02:45:21.142142+0000","last_fullsized":"2024-09-17T02:45:21.142142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:55.953622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165349+0000","last_change":"2024-09-17T02:45:19.086585+0000","last_active":"2024-09-17T02:45:20.165349+0000","last_peered":"2024-09-17T02:45:20.165349+0000","last_clean":"2024-09-17T02:45:20.165349+0000","last_became_active":"2024-09-17T02:45:19.086060+0000","last_became_peered":"2024-09-17T02:45:19.086060+0000","last_unstale":"2024-09-17T02:45:20.165349+0000","last_undegraded":"2024-09-17T02:45:20.165349+0000","last_fullsized":"2024-09-17T02:45:20.165349+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:06.921612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.128800+0000","last_change":"2024-09-17T02:45:20.122719+0000","last_active":"2024-09-17T02:45:20.128800+0000","last_peered":"2024-09-17T02:45:20.128800+0000","last_clean":"2024-09-17T02:45:20.128800+0000","last_became_active":"2024-09-17T02:45:20.120568+0000","last_became_peered":"2024-09-17T02:45:20.120568+0000","last_unstale":"2024-09-17T02:45:20.128800+0000","last_undegraded":"2024-09-17T02:45:20.128800+0000","last_fullsized":"2024-09-17T02:45:20.128800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:40.038462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137866+0000","last_change":"2024-09-17T02:45:19.093274+0000","last_active":"2024-09-17T02:45:21.137866+0000","last_peered":"2024-09-17T02:45:21.137866+0000","last_clean":"2024-09-17T02:45:21.137866+0000","last_became_active":"2024-09-17T02:45:19.091525+0000","last_became_peered":"2024-09-17T02:45:19.091525+0000","last_unstale":"2024-09-17T02:45:21.137866+0000","last_undegraded":"2024-09-17T02:45:21.137866+0000","last_fullsized":"2024-09-17T02:45:21.137866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:05.164896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.136305+0000","last_change":"2024-09-17T02:45:20.132688+0000","last_active":"2024-09-17T02:45:20.136305+0000","last_peered":"2024-09-17T02:45:20.136305+0000","last_clean":"2024-09-17T02:45:20.136305+0000","last_became_active":"2024-09-17T02:45:20.132335+0000","last_became_peered":"2024-09-17T02:45:20.132335+0000","last_unstale":"2024-09-17T02:45:20.136305+0000","last_undegraded":"2024-09-17T02:45:20.136305+0000","last_fullsized":"2024-09-17T02:45:20.136305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:01.061964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.140589+0000","last_change":"2024-09-17T02:45:19.086961+0000","last_active":"2024-09-17T02:45:21.140589+0000","last_peered":"2024-09-17T02:45:21.140589+0000","last_clean":"2024-09-17T02:45:21.140589+0000","last_became_active":"2024-09-17T02:45:19.084997+0000","last_became_peered":"2024-09-17T02:45:19.084997+0000","last_unstale":"2024-09-17T02:45:21.140589+0000","last_undegraded":"2024-09-17T02:45:21.140589+0000","last_fullsized":"2024-09-17T02:45:21.140589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:11.507133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.140103+0000","last_change":"2024-09-17T02:45:20.127506+0000","last_active":"2024-09-17T02:45:21.140103+0000","last_peered":"2024-09-17T02:45:21.140103+0000","last_clean":"2024-09-17T02:45:21.140103+0000","last_became_active":"2024-09-17T02:45:20.126699+0000","last_became_peered":"2024-09-17T02:45:20.126699+0000","last_unstale":"2024-09-17T02:45:21.140103+0000","last_undegraded":"2024-09-17T02:45:21.140103+0000","last_fullsized":"2024-09-17T02:45:21.140103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:45.078719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.103868+0000","last_change":"2024-09-17T02:45:19.081505+0000","last_active":"2024-09-17T02:45:20.103868+0000","last_peered":"2024-09-17T02:45:20.103868+0000","last_clean":"2024-09-17T02:45:20.103868+0000","last_became_active":"2024-09-17T02:45:19.081196+0000","last_became_peered":"2024-09-17T02:45:19.081196+0000","last_unstale":"2024-09-17T02:45:20.103868+0000","last_undegraded":"2024-09-17T02:45:20.103868+0000","last_fullsized":"2024-09-17T02:45:20.103868+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:33.638898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.146953+0000","last_change":"2024-09-17T02:45:20.139575+0000","last_active":"2024-09-17T02:45:20.146953+0000","last_peered":"2024-09-17T02:45:20.146953+0000","last_clean":"2024-09-17T02:45:20.146953+0000","last_became_active":"2024-09-17T02:45:20.137448+0000","last_became_peered":"2024-09-17T02:45:20.137448+0000","last_unstale":"2024-09-17T02:45:20.146953+0000","last_undegraded":"2024-09-17T02:45:20.146953+0000","last_fullsized":"2024-09-17T02:45:20.146953+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:28.895745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135457+0000","last_change":"2024-09-17T02:45:19.074384+0000","last_active":"2024-09-17T02:45:21.135457+0000","last_peered":"2024-09-17T02:45:21.135457+0000","last_clean":"2024-09-17T02:45:21.135457+0000","last_became_active":"2024-09-17T02:45:19.073958+0000","last_became_peered":"2024-09-17T02:45:19.073958+0000","last_unstale":"2024-09-17T02:45:21.135457+0000","last_undegraded":"2024-09-17T02:45:21.135457+0000","last_fullsized":"2024-09-17T02:45:21.135457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:36.384573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139981+0000","last_change":"2024-09-17T02:45:20.152408+0000","last_active":"2024-09-17T02:45:21.139981+0000","last_peered":"2024-09-17T02:45:21.139981+0000","last_clean":"2024-09-17T02:45:21.139981+0000","last_became_active":"2024-09-17T02:45:20.137766+0000","last_became_peered":"2024-09-17T02:45:20.137766+0000","last_unstale":"2024-09-17T02:45:21.139981+0000","last_undegraded":"2024-09-17T02:45:21.139981+0000","last_fullsized":"2024-09-17T02:45:21.139981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:54.493799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"78'3","reported_seq":16,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.165499+0000","last_change":"2024-09-17T02:45:19.101534+0000","last_active":"2024-09-17T02:45:20.165499+0000","last_peered":"2024-09-17T02:45:20.165499+0000","last_clean":"2024-09-17T02:45:20.165499+0000","last_became_active":"2024-09-17T02:45:19.099790+0000","last_became_peered":"2024-09-17T02:45:19.099790+0000","last_unstale":"2024-09-17T02:45:20.165499+0000","last_undegraded":"2024-09-17T02:45:20.165499+0000","last_fullsized":"2024-09-17T02:45:20.165499+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:59.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.155826+0000","last_change":"2024-09-17T02:45:20.154562+0000","last_active":"2024-09-17T02:45:20.155826+0000","last_peered":"2024-09-17T02:45:20.155826+0000","last_clean":"2024-09-17T02:45:20.155826+0000","last_became_active":"2024-09-17T02:45:20.153866+0000","last_became_peered":"2024-09-17T02:45:20.153866+0000","last_unstale":"2024-09-17T02:45:20.155826+0000","last_undegraded":"2024-09-17T02:45:20.155826+0000","last_fullsized":"2024-09-17T02:45:20.155826+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:56:31.599827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"78'6","reported_seq":19,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.154097+0000","last_change":"2024-09-17T02:45:19.079828+0000","last_active":"2024-09-17T02:45:20.154097+0000","last_peered":"2024-09-17T02:45:20.154097+0000","last_clean":"2024-09-17T02:45:20.154097+0000","last_became_active":"2024-09-17T02:45:19.079230+0000","last_became_peered":"2024-09-17T02:45:19.079230+0000","last_unstale":"2024-09-17T02:45:20.154097+0000","last_undegraded":"2024-09-17T02:45:20.154097+0000","last_fullsized":"2024-09-17T02:45:20.154097+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:46.676590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.133641+0000","last_change":"2024-09-17T02:45:20.130719+0000","last_active":"2024-09-17T02:45:20.133641+0000","last_peered":"2024-09-17T02:45:20.133641+0000","last_clean":"2024-09-17T02:45:20.133641+0000","last_became_active":"2024-09-17T02:45:20.130351+0000","last_became_peered":"2024-09-17T02:45:20.130351+0000","last_unstale":"2024-09-17T02:45:20.133641+0000","last_undegraded":"2024-09-17T02:45:20.133641+0000","last_fullsized":"2024-09-17T02:45:20.133641+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:15.730199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136247+0000","last_change":"2024-09-17T02:45:19.085733+0000","last_active":"2024-09-17T02:45:21.136247+0000","last_peered":"2024-09-17T02:45:21.136247+0000","last_clean":"2024-09-17T02:45:21.136247+0000","last_became_active":"2024-09-17T02:45:19.085445+0000","last_became_peered":"2024-09-17T02:45:19.085445+0000","last_unstale":"2024-09-17T02:45:21.136247+0000","last_undegraded":"2024-09-17T02:45:21.136247+0000","last_fullsized":"2024-09-17T02:45:21.136247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:35.839507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136624+0000","last_change":"2024-09-17T02:45:20.127481+0000","last_active":"2024-09-17T02:45:21.136624+0000","last_peered":"2024-09-17T02:45:21.136624+0000","last_clean":"2024-09-17T02:45:21.136624+0000","last_became_active":"2024-09-17T02:45:20.127085+0000","last_became_peered":"2024-09-17T02:45:20.127085+0000","last_unstale":"2024-09-17T02:45:21.136624+0000","last_undegraded":"2024-09-17T02:45:21.136624+0000","last_fullsized":"2024-09-17T02:45:21.136624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:42.063380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.094987+0000","last_change":"2024-09-17T02:45:19.094438+0000","last_active":"2024-09-17T02:45:20.094987+0000","last_peered":"2024-09-17T02:45:20.094987+0000","last_clean":"2024-09-17T02:45:20.094987+0000","last_became_active":"2024-09-17T02:45:19.093709+0000","last_became_peered":"2024-09-17T02:45:19.093709+0000","last_unstale":"2024-09-17T02:45:20.094987+0000","last_undegraded":"2024-09-17T02:45:20.094987+0000","last_fullsized":"2024-09-17T02:45:20.094987+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:45.773745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.135664+0000","last_change":"2024-09-17T02:45:20.131160+0000","last_active":"2024-09-17T02:45:20.135664+0000","last_peered":"2024-09-17T02:45:20.135664+0000","last_clean":"2024-09-17T02:45:20.135664+0000","last_became_active":"2024-09-17T02:45:20.129742+0000","last_became_peered":"2024-09-17T02:45:20.129742+0000","last_unstale":"2024-09-17T02:45:20.135664+0000","last_undegraded":"2024-09-17T02:45:20.135664+0000","last_fullsized":"2024-09-17T02:45:20.135664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:08:11.528887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.113545+0000","last_change":"2024-09-17T02:45:19.105626+0000","last_active":"2024-09-17T02:45:20.113545+0000","last_peered":"2024-09-17T02:45:20.113545+0000","last_clean":"2024-09-17T02:45:20.113545+0000","last_became_active":"2024-09-17T02:45:19.104904+0000","last_became_peered":"2024-09-17T02:45:19.104904+0000","last_unstale":"2024-09-17T02:45:20.113545+0000","last_undegraded":"2024-09-17T02:45:20.113545+0000","last_fullsized":"2024-09-17T02:45:20.113545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:38.203466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135464+0000","last_change":"2024-09-17T02:45:20.138086+0000","last_active":"2024-09-17T02:45:21.135464+0000","last_peered":"2024-09-17T02:45:21.135464+0000","last_clean":"2024-09-17T02:45:21.135464+0000","last_became_active":"2024-09-17T02:45:20.134972+0000","last_became_peered":"2024-09-17T02:45:20.134972+0000","last_unstale":"2024-09-17T02:45:21.135464+0000","last_undegraded":"2024-09-17T02:45:21.135464+0000","last_fullsized":"2024-09-17T02:45:21.135464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:18.254883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"78'8","reported_seq":21,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.164732+0000","last_change":"2024-09-17T02:45:19.102331+0000","last_active":"2024-09-17T02:45:20.164732+0000","last_peered":"2024-09-17T02:45:20.164732+0000","last_clean":"2024-09-17T02:45:20.164732+0000","last_became_active":"2024-09-17T02:45:19.101721+0000","last_became_peered":"2024-09-17T02:45:19.101721+0000","last_unstale":"2024-09-17T02:45:20.164732+0000","last_undegraded":"2024-09-17T02:45:20.164732+0000","last_fullsized":"2024-09-17T02:45:20.164732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:45.773745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149140+0000","last_change":"2024-09-17T02:45:20.131992+0000","last_active":"2024-09-17T02:45:20.149140+0000","last_peered":"2024-09-17T02:45:20.149140+0000","last_clean":"2024-09-17T02:45:20.149140+0000","last_became_active":"2024-09-17T02:45:20.126966+0000","last_became_peered":"2024-09-17T02:45:20.126966+0000","last_unstale":"2024-09-17T02:45:20.149140+0000","last_undegraded":"2024-09-17T02:45:20.149140+0000","last_fullsized":"2024-09-17T02:45:20.149140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:07.337735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.164565+0000","last_change":"2024-09-17T02:45:19.095009+0000","last_active":"2024-09-17T02:45:20.164565+0000","last_peered":"2024-09-17T02:45:20.164565+0000","last_clean":"2024-09-17T02:45:20.164565+0000","last_became_active":"2024-09-17T02:45:19.094610+0000","last_became_peered":"2024-09-17T02:45:19.094610+0000","last_unstale":"2024-09-17T02:45:20.164565+0000","last_undegraded":"2024-09-17T02:45:20.164565+0000","last_fullsized":"2024-09-17T02:45:20.164565+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:42.163563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139855+0000","last_change":"2024-09-17T02:45:20.147442+0000","last_active":"2024-09-17T02:45:21.139855+0000","last_peered":"2024-09-17T02:45:21.139855+0000","last_clean":"2024-09-17T02:45:21.139855+0000","last_became_active":"2024-09-17T02:45:20.146649+0000","last_became_peered":"2024-09-17T02:45:20.146649+0000","last_unstale":"2024-09-17T02:45:21.139855+0000","last_undegraded":"2024-09-17T02:45:21.139855+0000","last_fullsized":"2024-09-17T02:45:21.139855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:49:44.498880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"78'2","reported_seq":15,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.151200+0000","last_change":"2024-09-17T02:45:19.089116+0000","last_active":"2024-09-17T02:45:20.151200+0000","last_peered":"2024-09-17T02:45:20.151200+0000","last_clean":"2024-09-17T02:45:20.151200+0000","last_became_active":"2024-09-17T02:45:19.088667+0000","last_became_peered":"2024-09-17T02:45:19.088667+0000","last_unstale":"2024-09-17T02:45:20.151200+0000","last_undegraded":"2024-09-17T02:45:20.151200+0000","last_fullsized":"2024-09-17T02:45:20.151200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:54:35.726780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139554+0000","last_change":"2024-09-17T02:45:20.162326+0000","last_active":"2024-09-17T02:45:21.139554+0000","last_peered":"2024-09-17T02:45:21.139554+0000","last_clean":"2024-09-17T02:45:21.139554+0000","last_became_active":"2024-09-17T02:45:20.161849+0000","last_became_peered":"2024-09-17T02:45:20.161849+0000","last_unstale":"2024-09-17T02:45:21.139554+0000","last_undegraded":"2024-09-17T02:45:21.139554+0000","last_fullsized":"2024-09-17T02:45:21.139554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:53.540992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.161729+0000","last_change":"2024-09-17T02:45:19.099466+0000","last_active":"2024-09-17T02:45:20.161729+0000","last_peered":"2024-09-17T02:45:20.161729+0000","last_clean":"2024-09-17T02:45:20.161729+0000","last_became_active":"2024-09-17T02:45:19.098488+0000","last_became_peered":"2024-09-17T02:45:19.098488+0000","last_unstale":"2024-09-17T02:45:20.161729+0000","last_undegraded":"2024-09-17T02:45:20.161729+0000","last_fullsized":"2024-09-17T02:45:20.161729+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:32.496598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.157082+0000","last_change":"2024-09-17T02:45:20.152366+0000","last_active":"2024-09-17T02:45:20.157082+0000","last_peered":"2024-09-17T02:45:20.157082+0000","last_clean":"2024-09-17T02:45:20.157082+0000","last_became_active":"2024-09-17T02:45:20.150703+0000","last_became_peered":"2024-09-17T02:45:20.150703+0000","last_unstale":"2024-09-17T02:45:20.157082+0000","last_undegraded":"2024-09-17T02:45:20.157082+0000","last_fullsized":"2024-09-17T02:45:20.157082+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:04.950418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.115551+0000","last_change":"2024-09-17T02:45:19.105629+0000","last_active":"2024-09-17T02:45:20.115551+0000","last_peered":"2024-09-17T02:45:20.115551+0000","last_clean":"2024-09-17T02:45:20.115551+0000","last_became_active":"2024-09-17T02:45:19.104942+0000","last_became_peered":"2024-09-17T02:45:19.104942+0000","last_unstale":"2024-09-17T02:45:20.115551+0000","last_undegraded":"2024-09-17T02:45:20.115551+0000","last_fullsized":"2024-09-17T02:45:20.115551+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:55.509905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.150727+0000","last_change":"2024-09-17T02:45:20.141760+0000","last_active":"2024-09-17T02:45:20.150727+0000","last_peered":"2024-09-17T02:45:20.150727+0000","last_clean":"2024-09-17T02:45:20.150727+0000","last_became_active":"2024-09-17T02:45:20.141069+0000","last_became_peered":"2024-09-17T02:45:20.141069+0000","last_unstale":"2024-09-17T02:45:20.150727+0000","last_undegraded":"2024-09-17T02:45:20.150727+0000","last_fullsized":"2024-09-17T02:45:20.150727+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:36.850096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139358+0000","last_change":"2024-09-17T02:45:19.101863+0000","last_active":"2024-09-17T02:45:21.139358+0000","last_peered":"2024-09-17T02:45:21.139358+0000","last_clean":"2024-09-17T02:45:21.139358+0000","last_became_active":"2024-09-17T02:45:19.101175+0000","last_became_peered":"2024-09-17T02:45:19.101175+0000","last_unstale":"2024-09-17T02:45:21.139358+0000","last_undegraded":"2024-09-17T02:45:21.139358+0000","last_fullsized":"2024-09-17T02:45:21.139358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:00.618208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149604+0000","last_change":"2024-09-17T02:45:20.127175+0000","last_active":"2024-09-17T02:45:20.149604+0000","last_peered":"2024-09-17T02:45:20.149604+0000","last_clean":"2024-09-17T02:45:20.149604+0000","last_became_active":"2024-09-17T02:45:20.126335+0000","last_became_peered":"2024-09-17T02:45:20.126335+0000","last_unstale":"2024-09-17T02:45:20.149604+0000","last_undegraded":"2024-09-17T02:45:20.149604+0000","last_fullsized":"2024-09-17T02:45:20.149604+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:52.206612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138147+0000","last_change":"2024-09-17T02:45:19.093429+0000","last_active":"2024-09-17T02:45:21.138147+0000","last_peered":"2024-09-17T02:45:21.138147+0000","last_clean":"2024-09-17T02:45:21.138147+0000","last_became_active":"2024-09-17T02:45:19.092623+0000","last_became_peered":"2024-09-17T02:45:19.092623+0000","last_unstale":"2024-09-17T02:45:21.138147+0000","last_undegraded":"2024-09-17T02:45:21.138147+0000","last_fullsized":"2024-09-17T02:45:21.138147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:25.349401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.129570+0000","last_change":"2024-09-17T02:45:20.128671+0000","last_active":"2024-09-17T02:45:20.129570+0000","last_peered":"2024-09-17T02:45:20.129570+0000","last_clean":"2024-09-17T02:45:20.129570+0000","last_became_active":"2024-09-17T02:45:20.128268+0000","last_became_peered":"2024-09-17T02:45:20.128268+0000","last_unstale":"2024-09-17T02:45:20.129570+0000","last_undegraded":"2024-09-17T02:45:20.129570+0000","last_fullsized":"2024-09-17T02:45:20.129570+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:05.898373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.140911+0000","last_change":"2024-09-17T02:45:19.089801+0000","last_active":"2024-09-17T02:45:21.140911+0000","last_peered":"2024-09-17T02:45:21.140911+0000","last_clean":"2024-09-17T02:45:21.140911+0000","last_became_active":"2024-09-17T02:45:19.083865+0000","last_became_peered":"2024-09-17T02:45:19.083865+0000","last_unstale":"2024-09-17T02:45:21.140911+0000","last_undegraded":"2024-09-17T02:45:21.140911+0000","last_fullsized":"2024-09-17T02:45:21.140911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:45.046413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.122736+0000","last_change":"2024-09-17T02:45:20.121012+0000","last_active":"2024-09-17T02:45:20.122736+0000","last_peered":"2024-09-17T02:45:20.122736+0000","last_clean":"2024-09-17T02:45:20.122736+0000","last_became_active":"2024-09-17T02:45:20.120133+0000","last_became_peered":"2024-09-17T02:45:20.120133+0000","last_unstale":"2024-09-17T02:45:20.122736+0000","last_undegraded":"2024-09-17T02:45:20.122736+0000","last_fullsized":"2024-09-17T02:45:20.122736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:18.254883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135613+0000","last_change":"2024-09-17T02:45:19.082147+0000","last_active":"2024-09-17T02:45:21.135613+0000","last_peered":"2024-09-17T02:45:21.135613+0000","last_clean":"2024-09-17T02:45:21.135613+0000","last_became_active":"2024-09-17T02:45:19.081729+0000","last_became_peered":"2024-09-17T02:45:19.081729+0000","last_unstale":"2024-09-17T02:45:21.135613+0000","last_undegraded":"2024-09-17T02:45:21.135613+0000","last_fullsized":"2024-09-17T02:45:21.135613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:27.939745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135204+0000","last_change":"2024-09-17T02:45:20.136717+0000","last_active":"2024-09-17T02:45:21.135204+0000","last_peered":"2024-09-17T02:45:21.135204+0000","last_clean":"2024-09-17T02:45:21.135204+0000","last_became_active":"2024-09-17T02:45:20.136343+0000","last_became_peered":"2024-09-17T02:45:20.136343+0000","last_unstale":"2024-09-17T02:45:21.135204+0000","last_undegraded":"2024-09-17T02:45:21.135204+0000","last_fullsized":"2024-09-17T02:45:21.135204+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:44.913811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"78'1","reported_seq":14,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137044+0000","last_change":"2024-09-17T02:45:19.102757+0000","last_active":"2024-09-17T02:45:21.137044+0000","last_peered":"2024-09-17T02:45:21.137044+0000","last_clean":"2024-09-17T02:45:21.137044+0000","last_became_active":"2024-09-17T02:45:19.100744+0000","last_became_peered":"2024-09-17T02:45:19.100744+0000","last_unstale":"2024-09-17T02:45:21.137044+0000","last_undegraded":"2024-09-17T02:45:21.137044+0000","last_fullsized":"2024-09-17T02:45:21.137044+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:57.648928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.134832+0000","last_change":"2024-09-17T02:45:20.130812+0000","last_active":"2024-09-17T02:45:20.134832+0000","last_peered":"2024-09-17T02:45:20.134832+0000","last_clean":"2024-09-17T02:45:20.134832+0000","last_became_active":"2024-09-17T02:45:20.130480+0000","last_became_peered":"2024-09-17T02:45:20.130480+0000","last_unstale":"2024-09-17T02:45:20.134832+0000","last_undegraded":"2024-09-17T02:45:20.134832+0000","last_fullsized":"2024-09-17T02:45:20.134832+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:43.366504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136826+0000","last_change":"2024-09-17T02:45:19.093011+0000","last_active":"2024-09-17T02:45:21.136826+0000","last_peered":"2024-09-17T02:45:21.136826+0000","last_clean":"2024-09-17T02:45:21.136826+0000","last_became_active":"2024-09-17T02:45:19.091917+0000","last_became_peered":"2024-09-17T02:45:19.091917+0000","last_unstale":"2024-09-17T02:45:21.136826+0000","last_undegraded":"2024-09-17T02:45:21.136826+0000","last_fullsized":"2024-09-17T02:45:21.136826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:03.939829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139221+0000","last_change":"2024-09-17T02:45:20.153283+0000","last_active":"2024-09-17T02:45:21.139221+0000","last_peered":"2024-09-17T02:45:21.139221+0000","last_clean":"2024-09-17T02:45:21.139221+0000","last_became_active":"2024-09-17T02:45:20.152428+0000","last_became_peered":"2024-09-17T02:45:20.152428+0000","last_unstale":"2024-09-17T02:45:21.139221+0000","last_undegraded":"2024-09-17T02:45:21.139221+0000","last_fullsized":"2024-09-17T02:45:21.139221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:36.231357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.104378+0000","last_change":"2024-09-17T02:45:19.096590+0000","last_active":"2024-09-17T02:45:20.104378+0000","last_peered":"2024-09-17T02:45:20.104378+0000","last_clean":"2024-09-17T02:45:20.104378+0000","last_became_active":"2024-09-17T02:45:19.096024+0000","last_became_peered":"2024-09-17T02:45:19.096024+0000","last_unstale":"2024-09-17T02:45:20.104378+0000","last_undegraded":"2024-09-17T02:45:20.104378+0000","last_fullsized":"2024-09-17T02:45:20.104378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:17.702397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.126109+0000","last_change":"2024-09-17T02:45:20.120878+0000","last_active":"2024-09-17T02:45:20.126109+0000","last_peered":"2024-09-17T02:45:20.126109+0000","last_clean":"2024-09-17T02:45:20.126109+0000","last_became_active":"2024-09-17T02:45:20.119874+0000","last_became_peered":"2024-09-17T02:45:20.119874+0000","last_unstale":"2024-09-17T02:45:20.126109+0000","last_undegraded":"2024-09-17T02:45:20.126109+0000","last_fullsized":"2024-09-17T02:45:20.126109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:55.317365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.108369+0000","last_change":"2024-09-17T02:45:19.084757+0000","last_active":"2024-09-17T02:45:20.108369+0000","last_peered":"2024-09-17T02:45:20.108369+0000","last_clean":"2024-09-17T02:45:20.108369+0000","last_became_active":"2024-09-17T02:45:19.084284+0000","last_became_peered":"2024-09-17T02:45:19.084284+0000","last_unstale":"2024-09-17T02:45:20.108369+0000","last_undegraded":"2024-09-17T02:45:20.108369+0000","last_fullsized":"2024-09-17T02:45:20.108369+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:31.618471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130874+0000","last_change":"2024-09-17T02:45:20.128870+0000","last_active":"2024-09-17T02:45:20.130874+0000","last_peered":"2024-09-17T02:45:20.130874+0000","last_clean":"2024-09-17T02:45:20.130874+0000","last_became_active":"2024-09-17T02:45:20.128450+0000","last_became_peered":"2024-09-17T02:45:20.128450+0000","last_unstale":"2024-09-17T02:45:20.130874+0000","last_undegraded":"2024-09-17T02:45:20.130874+0000","last_fullsized":"2024-09-17T02:45:20.130874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:30:39.366822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.111875+0000","last_change":"2024-09-17T02:45:19.106516+0000","last_active":"2024-09-17T02:45:20.111875+0000","last_peered":"2024-09-17T02:45:20.111875+0000","last_clean":"2024-09-17T02:45:20.111875+0000","last_became_active":"2024-09-17T02:45:19.105883+0000","last_became_peered":"2024-09-17T02:45:19.105883+0000","last_unstale":"2024-09-17T02:45:20.111875+0000","last_undegraded":"2024-09-17T02:45:20.111875+0000","last_fullsized":"2024-09-17T02:45:20.111875+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:39.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.141439+0000","last_change":"2024-09-17T02:45:20.146992+0000","last_active":"2024-09-17T02:45:21.141439+0000","last_peered":"2024-09-17T02:45:21.141439+0000","last_clean":"2024-09-17T02:45:21.141439+0000","last_became_active":"2024-09-17T02:45:20.146129+0000","last_became_peered":"2024-09-17T02:45:20.146129+0000","last_unstale":"2024-09-17T02:45:21.141439+0000","last_undegraded":"2024-09-17T02:45:21.141439+0000","last_fullsized":"2024-09-17T02:45:21.141439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:08.007508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.110077+0000","last_change":"2024-09-17T02:45:19.086253+0000","last_active":"2024-09-17T02:45:20.110077+0000","last_peered":"2024-09-17T02:45:20.110077+0000","last_clean":"2024-09-17T02:45:20.110077+0000","last_became_active":"2024-09-17T02:45:19.085807+0000","last_became_peered":"2024-09-17T02:45:19.085807+0000","last_unstale":"2024-09-17T02:45:20.110077+0000","last_undegraded":"2024-09-17T02:45:20.110077+0000","last_fullsized":"2024-09-17T02:45:20.110077+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:43.903222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.149388+0000","last_change":"2024-09-17T02:45:20.124232+0000","last_active":"2024-09-17T02:45:20.149388+0000","last_peered":"2024-09-17T02:45:20.149388+0000","last_clean":"2024-09-17T02:45:20.149388+0000","last_became_active":"2024-09-17T02:45:20.123450+0000","last_became_peered":"2024-09-17T02:45:20.123450+0000","last_unstale":"2024-09-17T02:45:20.149388+0000","last_undegraded":"2024-09-17T02:45:20.149388+0000","last_fullsized":"2024-09-17T02:45:20.149388+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:01.142409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.141220+0000","last_change":"2024-09-17T02:45:19.087894+0000","last_active":"2024-09-17T02:45:21.141220+0000","last_peered":"2024-09-17T02:45:21.141220+0000","last_clean":"2024-09-17T02:45:21.141220+0000","last_became_active":"2024-09-17T02:45:19.086041+0000","last_became_peered":"2024-09-17T02:45:19.086041+0000","last_unstale":"2024-09-17T02:45:21.141220+0000","last_undegraded":"2024-09-17T02:45:21.141220+0000","last_fullsized":"2024-09-17T02:45:21.141220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:53.624694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.130153+0000","last_change":"2024-09-17T02:45:20.128777+0000","last_active":"2024-09-17T02:45:20.130153+0000","last_peered":"2024-09-17T02:45:20.130153+0000","last_clean":"2024-09-17T02:45:20.130153+0000","last_became_active":"2024-09-17T02:45:20.128362+0000","last_became_peered":"2024-09-17T02:45:20.128362+0000","last_unstale":"2024-09-17T02:45:20.130153+0000","last_undegraded":"2024-09-17T02:45:20.130153+0000","last_fullsized":"2024-09-17T02:45:20.130153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:09:46.647418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-17T02:45:20.111264+0000","last_change":"2024-09-17T02:45:19.099271+0000","last_active":"2024-09-17T02:45:20.111264+0000","last_peered":"2024-09-17T02:45:20.111264+0000","last_clean":"2024-09-17T02:45:20.111264+0000","last_became_active":"2024-09-17T02:45:19.088289+0000","last_became_peered":"2024-09-17T02:45:19.088289+0000","last_unstale":"2024-09-17T02:45:20.111264+0000","last_undegraded":"2024-09-17T02:45:20.111264+0000","last_fullsized":"2024-09-17T02:45:20.111264+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:54.306776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":220,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.134645+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:45:21.134645+0000","last_peered":"2024-09-17T02:45:21.134645+0000","last_clean":"2024-09-17T02:45:21.134645+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:45:21.134645+0000","last_undegraded":"2024-09-17T02:45:21.134645+0000","last_fullsized":"2024-09-17T02:45:21.134645+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509500,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":954368,"data_stored":431399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509500,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1544192,"data_stored":1021205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509501,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":430769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509499,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":432463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509500,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":430747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509500,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1556480,"data_stored":1022761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509501,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":950272,"data_stored":430769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":438111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:45:28.864 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:45:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.034+0000 7f57d1737640 1 -- 172.21.15.131:0/1961552674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57b8005360 msgr2=0x7f57b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.034+0000 7f57d1737640 1 --2- 172.21.15.131:0/1961552674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57b8005360 0x7f57b800b9d0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009b60 tx=0x7f57bc01d2b0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.034+0000 7f57d1737640 1 -- 172.21.15.131:0/1961552674 shutdown_connections 2024-09-17T02:45:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.034+0000 7f57d1737640 1 --2- 172.21.15.131:0/1961552674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57b8005360 0x7f57b800b9d0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.034+0000 7f57d1737640 1 -- 172.21.15.131:0/1961552674 >> 172.21.15.131:0/1961552674 conn(0x7f57cc06eea0 msgr2=0x7f57cc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.034+0000 7f57d1737640 1 -- 172.21.15.131:0/1961552674 shutdown_connections 2024-09-17T02:45:29.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.034+0000 7f57d1737640 1 -- 172.21.15.131:0/1961552674 wait complete. 2024-09-17T02:45:29.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57d1737640 1 Processor -- start 2024-09-17T02:45:29.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57d1737640 1 -- start start 2024-09-17T02:45:29.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57d1737640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57b8005360 0x7f57cc14ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57d1737640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f57cc0c9300 0x7f57cc0c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57d1737640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57cc15e3f0 0x7f57cc15e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57d1737640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc15cbb0 con 0x7f57b8005360 2024-09-17T02:45:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57d1737640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc15ca30 con 0x7f57cc15e3f0 2024-09-17T02:45:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57d1737640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc0cf010 con 0x7f57cc0c9300 2024-09-17T02:45:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57caffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57b8005360 0x7f57cc14ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57caffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57b8005360 0x7f57cc14ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44194/0 (socket says 172.21.15.131:44194) 2024-09-17T02:45:29.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57caffd640 1 -- 172.21.15.131:0/4141723801 learned_addr learned my addr 172.21.15.131:0/4141723801 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57cb7fe640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57cc15e3f0 0x7f57cc15e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57ca7fc640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f57cc0c9300 0x7f57cc0c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57caffd640 1 -- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f57cc0c9300 msgr2=0x7f57cc0c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57caffd640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f57cc0c9300 0x7f57cc0c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57caffd640 1 -- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57cc15e3f0 msgr2=0x7f57cc15e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57caffd640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57cc15e3f0 0x7f57cc15e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57caffd640 1 -- 172.21.15.131:0/4141723801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc15ed40 con 0x7f57b8005360 2024-09-17T02:45:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57caffd640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57b8005360 0x7f57cc14ffd0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f57c4010c90 tx=0x7f57c400e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:29.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57c37fe640 1 -- 172.21.15.131:0/4141723801 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4017070 con 0x7f57b8005360 2024-09-17T02:45:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57c37fe640 1 -- 172.21.15.131:0/4141723801 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57c400ee50 con 0x7f57b8005360 2024-09-17T02:45:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57c37fe640 1 -- 172.21.15.131:0/4141723801 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c40136c0 con 0x7f57b8005360 2024-09-17T02:45:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57d1737640 1 -- 172.21.15.131:0/4141723801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc150510 con 0x7f57b8005360 2024-09-17T02:45:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57d1737640 1 -- 172.21.15.131:0/4141723801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc15d2b0 con 0x7f57b8005360 2024-09-17T02:45:29.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.038+0000 7f57d1737640 1 -- 172.21.15.131:0/4141723801 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57cc0c4580 con 0x7f57b8005360 2024-09-17T02:45:29.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.042+0000 7f57c37fe640 1 -- 172.21.15.131:0/4141723801 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f57c4013860 con 0x7f57b8005360 2024-09-17T02:45:29.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.042+0000 7f57c37fe640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f57ac073200 0x7f57ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.042+0000 7f57c37fe640 1 -- 172.21.15.131:0/4141723801 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f57c4098270 con 0x7f57b8005360 2024-09-17T02:45:29.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.042+0000 7f57ca7fc640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f57ac073200 0x7f57ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.042+0000 7f57ca7fc640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f57ac073200 0x7f57ac075710 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f57cc1581f0 tx=0x7f57b4006b30 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:29.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.046+0000 7f57c37fe640 1 -- 172.21.15.131:0/4141723801 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f57cc15ed40 con 0x7f57b8005360 2024-09-17T02:45:29.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.206+0000 7f57d1737640 1 -- 172.21.15.131:0/4141723801 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f57cc15cd80 con 0x7f57ac073200 2024-09-17T02:45:29.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.218+0000 7f57c37fe640 1 -- 172.21.15.131:0/4141723801 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252040 (secure 0 0 0) 0x7f57cc15cd80 con 0x7f57ac073200 2024-09-17T02:45:29.225 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:29.226 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:45:29.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 -- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f57ac073200 msgr2=0x7f57ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f57ac073200 0x7f57ac075710 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f57cc1581f0 tx=0x7f57b4006b30 comp rx=0 tx=0).stop 2024-09-17T02:45:29.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 -- 172.21.15.131:0/4141723801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57b8005360 msgr2=0x7f57cc14ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57b8005360 0x7f57cc14ffd0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f57c4010c90 tx=0x7f57c400e650 comp rx=0 tx=0).stop 2024-09-17T02:45:29.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 -- 172.21.15.131:0/4141723801 shutdown_connections 2024-09-17T02:45:29.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f57ac073200 0x7f57ac075710 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57cc15e3f0 0x7f57cc15e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f57cc0c9300 0x7f57cc0c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 --2- 172.21.15.131:0/4141723801 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57b8005360 0x7f57cc14ffd0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 -- 172.21.15.131:0/4141723801 >> 172.21.15.131:0/4141723801 conn(0x7f57cc06eea0 msgr2=0x7f57cc0af610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:29.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 -- 172.21.15.131:0/4141723801 shutdown_connections 2024-09-17T02:45:29.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.222+0000 7f57d1737640 1 -- 172.21.15.131:0/4141723801 wait complete. 2024-09-17T02:45:29.241 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-17T02:45:27.227202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258260,"kb_used_data":8712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716460,"statfs":{"total":773094113280,"available":772829655040,"internally_reserved":0,"allocated":8921088,"data_stored":4685068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003617"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135946+0000","last_change":"2024-09-17T02:45:20.134496+0000","last_active":"2024-09-17T02:45:21.135946+0000","last_peered":"2024-09-17T02:45:21.135946+0000","last_clean":"2024-09-17T02:45:21.135946+0000","last_became_active":"2024-09-17T02:45:20.134052+0000","last_became_peered":"2024-09-17T02:45:20.134052+0000","last_unstale":"2024-09-17T02:45:21.135946+0000","last_undegraded":"2024-09-17T02:45:21.135946+0000","last_fullsized":"2024-09-17T02:45:21.135946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:33.880550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.140804+0000","last_change":"2024-09-17T02:45:20.144524+0000","last_active":"2024-09-17T02:45:21.140804+0000","last_peered":"2024-09-17T02:45:21.140804+0000","last_clean":"2024-09-17T02:45:21.140804+0000","last_became_active":"2024-09-17T02:45:20.143771+0000","last_became_peered":"2024-09-17T02:45:20.143771+0000","last_unstale":"2024-09-17T02:45:21.140804+0000","last_undegraded":"2024-09-17T02:45:21.140804+0000","last_fullsized":"2024-09-17T02:45:21.140804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:21:04.800055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.141004+0000","last_change":"2024-09-17T02:45:20.134523+0000","last_active":"2024-09-17T02:45:21.141004+0000","last_peered":"2024-09-17T02:45:21.141004+0000","last_clean":"2024-09-17T02:45:21.141004+0000","last_became_active":"2024-09-17T02:45:20.133784+0000","last_became_peered":"2024-09-17T02:45:20.133784+0000","last_unstale":"2024-09-17T02:45:21.141004+0000","last_undegraded":"2024-09-17T02:45:21.141004+0000","last_fullsized":"2024-09-17T02:45:21.141004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:50:03.665497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.422292+0000","last_change":"2024-09-17T02:45:20.124901+0000","last_active":"2024-09-17T02:45:21.422292+0000","last_peered":"2024-09-17T02:45:21.422292+0000","last_clean":"2024-09-17T02:45:21.422292+0000","last_became_active":"2024-09-17T02:45:20.124392+0000","last_became_peered":"2024-09-17T02:45:20.124392+0000","last_unstale":"2024-09-17T02:45:21.422292+0000","last_undegraded":"2024-09-17T02:45:21.422292+0000","last_fullsized":"2024-09-17T02:45:21.422292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:54:22.784243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.233590+0000","last_change":"2024-09-17T02:45:20.151934+0000","last_active":"2024-09-17T02:45:21.233590+0000","last_peered":"2024-09-17T02:45:21.233590+0000","last_clean":"2024-09-17T02:45:21.233590+0000","last_became_active":"2024-09-17T02:45:20.149652+0000","last_became_peered":"2024-09-17T02:45:20.149652+0000","last_unstale":"2024-09-17T02:45:21.233590+0000","last_undegraded":"2024-09-17T02:45:21.233590+0000","last_fullsized":"2024-09-17T02:45:21.233590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:04.588206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.224222+0000","last_change":"2024-09-17T02:45:20.156609+0000","last_active":"2024-09-17T02:45:21.224222+0000","last_peered":"2024-09-17T02:45:21.224222+0000","last_clean":"2024-09-17T02:45:21.224222+0000","last_became_active":"2024-09-17T02:45:20.155997+0000","last_became_peered":"2024-09-17T02:45:20.155997+0000","last_unstale":"2024-09-17T02:45:21.224222+0000","last_undegraded":"2024-09-17T02:45:21.224222+0000","last_fullsized":"2024-09-17T02:45:21.224222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:49.835408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.422118+0000","last_change":"2024-09-17T02:45:20.138831+0000","last_active":"2024-09-17T02:45:21.422118+0000","last_peered":"2024-09-17T02:45:21.422118+0000","last_clean":"2024-09-17T02:45:21.422118+0000","last_became_active":"2024-09-17T02:45:20.138468+0000","last_became_peered":"2024-09-17T02:45:20.138468+0000","last_unstale":"2024-09-17T02:45:21.422118+0000","last_undegraded":"2024-09-17T02:45:21.422118+0000","last_fullsized":"2024-09-17T02:45:21.422118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:19:25.885789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137912+0000","last_change":"2024-09-17T02:45:20.151755+0000","last_active":"2024-09-17T02:45:21.137912+0000","last_peered":"2024-09-17T02:45:21.137912+0000","last_clean":"2024-09-17T02:45:21.137912+0000","last_became_active":"2024-09-17T02:45:20.139390+0000","last_became_peered":"2024-09-17T02:45:20.139390+0000","last_unstale":"2024-09-17T02:45:21.137912+0000","last_undegraded":"2024-09-17T02:45:21.137912+0000","last_fullsized":"2024-09-17T02:45:21.137912+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:04.850276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.423075+0000","last_change":"2024-09-17T02:45:20.128764+0000","last_active":"2024-09-17T02:45:21.423075+0000","last_peered":"2024-09-17T02:45:21.423075+0000","last_clean":"2024-09-17T02:45:21.423075+0000","last_became_active":"2024-09-17T02:45:20.128381+0000","last_became_peered":"2024-09-17T02:45:20.128381+0000","last_unstale":"2024-09-17T02:45:21.423075+0000","last_undegraded":"2024-09-17T02:45:21.423075+0000","last_fullsized":"2024-09-17T02:45:21.423075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:26.359990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135793+0000","last_change":"2024-09-17T02:45:20.134184+0000","last_active":"2024-09-17T02:45:21.135793+0000","last_peered":"2024-09-17T02:45:21.135793+0000","last_clean":"2024-09-17T02:45:21.135793+0000","last_became_active":"2024-09-17T02:45:20.133826+0000","last_became_peered":"2024-09-17T02:45:20.133826+0000","last_unstale":"2024-09-17T02:45:21.135793+0000","last_undegraded":"2024-09-17T02:45:21.135793+0000","last_fullsized":"2024-09-17T02:45:21.135793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:41.858427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.141636+0000","last_change":"2024-09-17T02:45:20.140353+0000","last_active":"2024-09-17T02:45:21.141636+0000","last_peered":"2024-09-17T02:45:21.141636+0000","last_clean":"2024-09-17T02:45:21.141636+0000","last_became_active":"2024-09-17T02:45:20.139415+0000","last_became_peered":"2024-09-17T02:45:20.139415+0000","last_unstale":"2024-09-17T02:45:21.141636+0000","last_undegraded":"2024-09-17T02:45:21.141636+0000","last_fullsized":"2024-09-17T02:45:21.141636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:21:43.957898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.740489+0000","last_change":"2024-09-17T02:45:20.137823+0000","last_active":"2024-09-17T02:45:21.740489+0000","last_peered":"2024-09-17T02:45:21.740489+0000","last_clean":"2024-09-17T02:45:21.740489+0000","last_became_active":"2024-09-17T02:45:20.137420+0000","last_became_peered":"2024-09-17T02:45:20.137420+0000","last_unstale":"2024-09-17T02:45:21.740489+0000","last_undegraded":"2024-09-17T02:45:21.740489+0000","last_fullsized":"2024-09-17T02:45:21.740489+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:46.875820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138648+0000","last_change":"2024-09-17T02:45:20.130548+0000","last_active":"2024-09-17T02:45:21.138648+0000","last_peered":"2024-09-17T02:45:21.138648+0000","last_clean":"2024-09-17T02:45:21.138648+0000","last_became_active":"2024-09-17T02:45:20.128892+0000","last_became_peered":"2024-09-17T02:45:20.128892+0000","last_unstale":"2024-09-17T02:45:21.138648+0000","last_undegraded":"2024-09-17T02:45:21.138648+0000","last_fullsized":"2024-09-17T02:45:21.138648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:47:04.543314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139109+0000","last_change":"2024-09-17T02:45:20.154359+0000","last_active":"2024-09-17T02:45:21.139109+0000","last_peered":"2024-09-17T02:45:21.139109+0000","last_clean":"2024-09-17T02:45:21.139109+0000","last_became_active":"2024-09-17T02:45:20.138831+0000","last_became_peered":"2024-09-17T02:45:20.138831+0000","last_unstale":"2024-09-17T02:45:21.139109+0000","last_undegraded":"2024-09-17T02:45:21.139109+0000","last_fullsized":"2024-09-17T02:45:21.139109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:26.541024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.233167+0000","last_change":"2024-09-17T02:45:20.136343+0000","last_active":"2024-09-17T02:45:21.233167+0000","last_peered":"2024-09-17T02:45:21.233167+0000","last_clean":"2024-09-17T02:45:21.233167+0000","last_became_active":"2024-09-17T02:45:20.135036+0000","last_became_peered":"2024-09-17T02:45:20.135036+0000","last_unstale":"2024-09-17T02:45:21.233167+0000","last_undegraded":"2024-09-17T02:45:21.233167+0000","last_fullsized":"2024-09-17T02:45:21.233167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:28.950334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.421808+0000","last_change":"2024-09-17T02:45:20.133650+0000","last_active":"2024-09-17T02:45:21.421808+0000","last_peered":"2024-09-17T02:45:21.421808+0000","last_clean":"2024-09-17T02:45:21.421808+0000","last_became_active":"2024-09-17T02:45:20.133246+0000","last_became_peered":"2024-09-17T02:45:20.133246+0000","last_unstale":"2024-09-17T02:45:21.421808+0000","last_undegraded":"2024-09-17T02:45:21.421808+0000","last_fullsized":"2024-09-17T02:45:21.421808+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:36:16.903056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138676+0000","last_change":"2024-09-17T02:45:20.138431+0000","last_active":"2024-09-17T02:45:21.138676+0000","last_peered":"2024-09-17T02:45:21.138676+0000","last_clean":"2024-09-17T02:45:21.138676+0000","last_became_active":"2024-09-17T02:45:20.137633+0000","last_became_peered":"2024-09-17T02:45:20.137633+0000","last_unstale":"2024-09-17T02:45:21.138676+0000","last_undegraded":"2024-09-17T02:45:21.138676+0000","last_fullsized":"2024-09-17T02:45:21.138676+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:25.489158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.141003+0000","last_change":"2024-09-17T02:45:20.132227+0000","last_active":"2024-09-17T02:45:21.141003+0000","last_peered":"2024-09-17T02:45:21.141003+0000","last_clean":"2024-09-17T02:45:21.141003+0000","last_became_active":"2024-09-17T02:45:20.130783+0000","last_became_peered":"2024-09-17T02:45:20.130783+0000","last_unstale":"2024-09-17T02:45:21.141003+0000","last_undegraded":"2024-09-17T02:45:21.141003+0000","last_fullsized":"2024-09-17T02:45:21.141003+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:08:25.569753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138525+0000","last_change":"2024-09-17T02:45:20.138096+0000","last_active":"2024-09-17T02:45:21.138525+0000","last_peered":"2024-09-17T02:45:21.138525+0000","last_clean":"2024-09-17T02:45:21.138525+0000","last_became_active":"2024-09-17T02:45:20.137246+0000","last_became_peered":"2024-09-17T02:45:20.137246+0000","last_unstale":"2024-09-17T02:45:21.138525+0000","last_undegraded":"2024-09-17T02:45:21.138525+0000","last_fullsized":"2024-09-17T02:45:21.138525+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:08.405979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.224600+0000","last_change":"2024-09-17T02:45:20.144454+0000","last_active":"2024-09-17T02:45:21.224600+0000","last_peered":"2024-09-17T02:45:21.224600+0000","last_clean":"2024-09-17T02:45:21.224600+0000","last_became_active":"2024-09-17T02:45:20.143671+0000","last_became_peered":"2024-09-17T02:45:20.143671+0000","last_unstale":"2024-09-17T02:45:21.224600+0000","last_undegraded":"2024-09-17T02:45:21.224600+0000","last_fullsized":"2024-09-17T02:45:21.224600+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:37.115388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.740691+0000","last_change":"2024-09-17T02:45:20.140711+0000","last_active":"2024-09-17T02:45:21.740691+0000","last_peered":"2024-09-17T02:45:21.740691+0000","last_clean":"2024-09-17T02:45:21.740691+0000","last_became_active":"2024-09-17T02:45:20.140390+0000","last_became_peered":"2024-09-17T02:45:20.140390+0000","last_unstale":"2024-09-17T02:45:21.740691+0000","last_undegraded":"2024-09-17T02:45:21.740691+0000","last_fullsized":"2024-09-17T02:45:21.740691+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:17:48.025458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139684+0000","last_change":"2024-09-17T02:45:20.131403+0000","last_active":"2024-09-17T02:45:21.139684+0000","last_peered":"2024-09-17T02:45:21.139684+0000","last_clean":"2024-09-17T02:45:21.139684+0000","last_became_active":"2024-09-17T02:45:20.130541+0000","last_became_peered":"2024-09-17T02:45:20.130541+0000","last_unstale":"2024-09-17T02:45:21.139684+0000","last_undegraded":"2024-09-17T02:45:21.139684+0000","last_fullsized":"2024-09-17T02:45:21.139684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:16:02.517427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135331+0000","last_change":"2024-09-17T02:45:20.120377+0000","last_active":"2024-09-17T02:45:21.135331+0000","last_peered":"2024-09-17T02:45:21.135331+0000","last_clean":"2024-09-17T02:45:21.135331+0000","last_became_active":"2024-09-17T02:45:20.119336+0000","last_became_peered":"2024-09-17T02:45:20.119336+0000","last_unstale":"2024-09-17T02:45:21.135331+0000","last_undegraded":"2024-09-17T02:45:21.135331+0000","last_fullsized":"2024-09-17T02:45:21.135331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:43.174152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139403+0000","last_change":"2024-09-17T02:45:20.132058+0000","last_active":"2024-09-17T02:45:21.139403+0000","last_peered":"2024-09-17T02:45:21.139403+0000","last_clean":"2024-09-17T02:45:21.139403+0000","last_became_active":"2024-09-17T02:45:20.128816+0000","last_became_peered":"2024-09-17T02:45:20.128816+0000","last_unstale":"2024-09-17T02:45:21.139403+0000","last_undegraded":"2024-09-17T02:45:21.139403+0000","last_fullsized":"2024-09-17T02:45:21.139403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:21.117339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.224627+0000","last_change":"2024-09-17T02:45:20.136877+0000","last_active":"2024-09-17T02:45:21.224627+0000","last_peered":"2024-09-17T02:45:21.224627+0000","last_clean":"2024-09-17T02:45:21.224627+0000","last_became_active":"2024-09-17T02:45:20.136227+0000","last_became_peered":"2024-09-17T02:45:20.136227+0000","last_unstale":"2024-09-17T02:45:21.224627+0000","last_undegraded":"2024-09-17T02:45:21.224627+0000","last_fullsized":"2024-09-17T02:45:21.224627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:27.434002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.422756+0000","last_change":"2024-09-17T02:45:20.129949+0000","last_active":"2024-09-17T02:45:21.422756+0000","last_peered":"2024-09-17T02:45:21.422756+0000","last_clean":"2024-09-17T02:45:21.422756+0000","last_became_active":"2024-09-17T02:45:20.129519+0000","last_became_peered":"2024-09-17T02:45:20.129519+0000","last_unstale":"2024-09-17T02:45:21.422756+0000","last_undegraded":"2024-09-17T02:45:21.422756+0000","last_fullsized":"2024-09-17T02:45:21.422756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:06.175485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.422448+0000","last_change":"2024-09-17T02:45:20.125488+0000","last_active":"2024-09-17T02:45:21.422448+0000","last_peered":"2024-09-17T02:45:21.422448+0000","last_clean":"2024-09-17T02:45:21.422448+0000","last_became_active":"2024-09-17T02:45:20.124765+0000","last_became_peered":"2024-09-17T02:45:20.124765+0000","last_unstale":"2024-09-17T02:45:21.422448+0000","last_undegraded":"2024-09-17T02:45:21.422448+0000","last_fullsized":"2024-09-17T02:45:21.422448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:43.366504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.234308+0000","last_change":"2024-09-17T02:45:20.121902+0000","last_active":"2024-09-17T02:45:21.234308+0000","last_peered":"2024-09-17T02:45:21.234308+0000","last_clean":"2024-09-17T02:45:21.234308+0000","last_became_active":"2024-09-17T02:45:20.121274+0000","last_became_peered":"2024-09-17T02:45:20.121274+0000","last_unstale":"2024-09-17T02:45:21.234308+0000","last_undegraded":"2024-09-17T02:45:21.234308+0000","last_fullsized":"2024-09-17T02:45:21.234308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:44.913811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.225153+0000","last_change":"2024-09-17T02:45:20.147039+0000","last_active":"2024-09-17T02:45:21.225153+0000","last_peered":"2024-09-17T02:45:21.225153+0000","last_clean":"2024-09-17T02:45:21.225153+0000","last_became_active":"2024-09-17T02:45:20.145977+0000","last_became_peered":"2024-09-17T02:45:20.145977+0000","last_unstale":"2024-09-17T02:45:21.225153+0000","last_undegraded":"2024-09-17T02:45:21.225153+0000","last_fullsized":"2024-09-17T02:45:21.225153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:50.273086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135630+0000","last_change":"2024-09-17T02:45:20.134028+0000","last_active":"2024-09-17T02:45:21.135630+0000","last_peered":"2024-09-17T02:45:21.135630+0000","last_clean":"2024-09-17T02:45:21.135630+0000","last_became_active":"2024-09-17T02:45:20.133628+0000","last_became_peered":"2024-09-17T02:45:20.133628+0000","last_unstale":"2024-09-17T02:45:21.135630+0000","last_undegraded":"2024-09-17T02:45:21.135630+0000","last_fullsized":"2024-09-17T02:45:21.135630+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:16:56.401934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139176+0000","last_change":"2024-09-17T02:45:20.147793+0000","last_active":"2024-09-17T02:45:21.139176+0000","last_peered":"2024-09-17T02:45:21.139176+0000","last_clean":"2024-09-17T02:45:21.139176+0000","last_became_active":"2024-09-17T02:45:20.147199+0000","last_became_peered":"2024-09-17T02:45:20.147199+0000","last_unstale":"2024-09-17T02:45:21.139176+0000","last_undegraded":"2024-09-17T02:45:21.139176+0000","last_fullsized":"2024-09-17T02:45:21.139176+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:38.337307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.421973+0000","last_change":"2024-09-17T02:45:20.127695+0000","last_active":"2024-09-17T02:45:21.421973+0000","last_peered":"2024-09-17T02:45:21.421973+0000","last_clean":"2024-09-17T02:45:21.421973+0000","last_became_active":"2024-09-17T02:45:20.127323+0000","last_became_peered":"2024-09-17T02:45:20.127323+0000","last_unstale":"2024-09-17T02:45:21.421973+0000","last_undegraded":"2024-09-17T02:45:21.421973+0000","last_fullsized":"2024-09-17T02:45:21.421973+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:06:04.472277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.233041+0000","last_change":"2024-09-17T02:45:20.122630+0000","last_active":"2024-09-17T02:45:21.233041+0000","last_peered":"2024-09-17T02:45:21.233041+0000","last_clean":"2024-09-17T02:45:21.233041+0000","last_became_active":"2024-09-17T02:45:20.120596+0000","last_became_peered":"2024-09-17T02:45:20.120596+0000","last_unstale":"2024-09-17T02:45:21.233041+0000","last_undegraded":"2024-09-17T02:45:21.233041+0000","last_fullsized":"2024-09-17T02:45:21.233041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:36.831068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"78'4","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.234347+0000","last_change":"2024-09-17T02:45:19.105415+0000","last_active":"2024-09-17T02:45:21.234347+0000","last_peered":"2024-09-17T02:45:21.234347+0000","last_clean":"2024-09-17T02:45:21.234347+0000","last_became_active":"2024-09-17T02:45:19.104799+0000","last_became_peered":"2024-09-17T02:45:19.104799+0000","last_unstale":"2024-09-17T02:45:21.234347+0000","last_undegraded":"2024-09-17T02:45:21.234347+0000","last_fullsized":"2024-09-17T02:45:21.234347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:15:52.663858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.233207+0000","last_change":"2024-09-17T02:45:20.151127+0000","last_active":"2024-09-17T02:45:21.233207+0000","last_peered":"2024-09-17T02:45:21.233207+0000","last_clean":"2024-09-17T02:45:21.233207+0000","last_became_active":"2024-09-17T02:45:20.150324+0000","last_became_peered":"2024-09-17T02:45:20.150324+0000","last_unstale":"2024-09-17T02:45:21.233207+0000","last_undegraded":"2024-09-17T02:45:21.233207+0000","last_fullsized":"2024-09-17T02:45:21.233207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:56.394790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1e","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.238840+0000","last_change":"2024-09-17T02:45:19.079006+0000","last_active":"2024-09-17T02:45:21.238840+0000","last_peered":"2024-09-17T02:45:21.238840+0000","last_clean":"2024-09-17T02:45:21.238840+0000","last_became_active":"2024-09-17T02:45:19.078512+0000","last_became_peered":"2024-09-17T02:45:19.078512+0000","last_unstale":"2024-09-17T02:45:21.238840+0000","last_undegraded":"2024-09-17T02:45:21.238840+0000","last_fullsized":"2024-09-17T02:45:21.238840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:48.935522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.740192+0000","last_change":"2024-09-17T02:45:20.131374+0000","last_active":"2024-09-17T02:45:21.740192+0000","last_peered":"2024-09-17T02:45:21.740192+0000","last_clean":"2024-09-17T02:45:21.740192+0000","last_became_active":"2024-09-17T02:45:20.129652+0000","last_became_peered":"2024-09-17T02:45:20.129652+0000","last_unstale":"2024-09-17T02:45:21.740192+0000","last_undegraded":"2024-09-17T02:45:21.740192+0000","last_fullsized":"2024-09-17T02:45:21.740192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:37:27.434002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136882+0000","last_change":"2024-09-17T02:45:19.078238+0000","last_active":"2024-09-17T02:45:21.136882+0000","last_peered":"2024-09-17T02:45:21.136882+0000","last_clean":"2024-09-17T02:45:21.136882+0000","last_became_active":"2024-09-17T02:45:19.077495+0000","last_became_peered":"2024-09-17T02:45:19.077495+0000","last_unstale":"2024-09-17T02:45:21.136882+0000","last_undegraded":"2024-09-17T02:45:21.136882+0000","last_fullsized":"2024-09-17T02:45:21.136882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:25.395603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.422437+0000","last_change":"2024-09-17T02:45:20.127140+0000","last_active":"2024-09-17T02:45:21.422437+0000","last_peered":"2024-09-17T02:45:21.422437+0000","last_clean":"2024-09-17T02:45:21.422437+0000","last_became_active":"2024-09-17T02:45:20.126455+0000","last_became_peered":"2024-09-17T02:45:20.126455+0000","last_unstale":"2024-09-17T02:45:21.422437+0000","last_undegraded":"2024-09-17T02:45:21.422437+0000","last_fullsized":"2024-09-17T02:45:21.422437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:54.522801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.740977+0000","last_change":"2024-09-17T02:45:19.092392+0000","last_active":"2024-09-17T02:45:21.740977+0000","last_peered":"2024-09-17T02:45:21.740977+0000","last_clean":"2024-09-17T02:45:21.740977+0000","last_became_active":"2024-09-17T02:45:19.091367+0000","last_became_peered":"2024-09-17T02:45:19.091367+0000","last_unstale":"2024-09-17T02:45:21.740977+0000","last_undegraded":"2024-09-17T02:45:21.740977+0000","last_fullsized":"2024-09-17T02:45:21.740977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:57.648928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.225014+0000","last_change":"2024-09-17T02:45:20.134096+0000","last_active":"2024-09-17T02:45:21.225014+0000","last_peered":"2024-09-17T02:45:21.225014+0000","last_clean":"2024-09-17T02:45:21.225014+0000","last_became_active":"2024-09-17T02:45:20.133166+0000","last_became_peered":"2024-09-17T02:45:20.133166+0000","last_unstale":"2024-09-17T02:45:21.225014+0000","last_undegraded":"2024-09-17T02:45:21.225014+0000","last_fullsized":"2024-09-17T02:45:21.225014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:08:45.879068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1b","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.741228+0000","last_change":"2024-09-17T02:45:19.095580+0000","last_active":"2024-09-17T02:45:21.741228+0000","last_peered":"2024-09-17T02:45:21.741228+0000","last_clean":"2024-09-17T02:45:21.741228+0000","last_became_active":"2024-09-17T02:45:19.094437+0000","last_became_peered":"2024-09-17T02:45:19.094437+0000","last_unstale":"2024-09-17T02:45:21.741228+0000","last_undegraded":"2024-09-17T02:45:21.741228+0000","last_fullsized":"2024-09-17T02:45:21.741228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:59.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.142142+0000","last_change":"2024-09-17T02:45:20.123316+0000","last_active":"2024-09-17T02:45:21.142142+0000","last_peered":"2024-09-17T02:45:21.142142+0000","last_clean":"2024-09-17T02:45:21.142142+0000","last_became_active":"2024-09-17T02:45:20.122550+0000","last_became_peered":"2024-09-17T02:45:20.122550+0000","last_unstale":"2024-09-17T02:45:21.142142+0000","last_undegraded":"2024-09-17T02:45:21.142142+0000","last_fullsized":"2024-09-17T02:45:21.142142+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:01:55.953622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.238546+0000","last_change":"2024-09-17T02:45:19.086585+0000","last_active":"2024-09-17T02:45:21.238546+0000","last_peered":"2024-09-17T02:45:21.238546+0000","last_clean":"2024-09-17T02:45:21.238546+0000","last_became_active":"2024-09-17T02:45:19.086060+0000","last_became_peered":"2024-09-17T02:45:19.086060+0000","last_unstale":"2024-09-17T02:45:21.238546+0000","last_undegraded":"2024-09-17T02:45:21.238546+0000","last_fullsized":"2024-09-17T02:45:21.238546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:05:06.921612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.740606+0000","last_change":"2024-09-17T02:45:20.122719+0000","last_active":"2024-09-17T02:45:21.740606+0000","last_peered":"2024-09-17T02:45:21.740606+0000","last_clean":"2024-09-17T02:45:21.740606+0000","last_became_active":"2024-09-17T02:45:20.120568+0000","last_became_peered":"2024-09-17T02:45:20.120568+0000","last_unstale":"2024-09-17T02:45:21.740606+0000","last_undegraded":"2024-09-17T02:45:21.740606+0000","last_fullsized":"2024-09-17T02:45:21.740606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:49.835408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.137866+0000","last_change":"2024-09-17T02:45:19.093274+0000","last_active":"2024-09-17T02:45:21.137866+0000","last_peered":"2024-09-17T02:45:21.137866+0000","last_clean":"2024-09-17T02:45:21.137866+0000","last_became_active":"2024-09-17T02:45:19.091525+0000","last_became_peered":"2024-09-17T02:45:19.091525+0000","last_unstale":"2024-09-17T02:45:21.137866+0000","last_undegraded":"2024-09-17T02:45:21.137866+0000","last_fullsized":"2024-09-17T02:45:21.137866+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:05.164896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.239124+0000","last_change":"2024-09-17T02:45:20.132688+0000","last_active":"2024-09-17T02:45:21.239124+0000","last_peered":"2024-09-17T02:45:21.239124+0000","last_clean":"2024-09-17T02:45:21.239124+0000","last_became_active":"2024-09-17T02:45:20.132335+0000","last_became_peered":"2024-09-17T02:45:20.132335+0000","last_unstale":"2024-09-17T02:45:21.239124+0000","last_undegraded":"2024-09-17T02:45:21.239124+0000","last_fullsized":"2024-09-17T02:45:21.239124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:35:12.181971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.140589+0000","last_change":"2024-09-17T02:45:19.086961+0000","last_active":"2024-09-17T02:45:21.140589+0000","last_peered":"2024-09-17T02:45:21.140589+0000","last_clean":"2024-09-17T02:45:21.140589+0000","last_became_active":"2024-09-17T02:45:19.084997+0000","last_became_peered":"2024-09-17T02:45:19.084997+0000","last_unstale":"2024-09-17T02:45:21.140589+0000","last_undegraded":"2024-09-17T02:45:21.140589+0000","last_fullsized":"2024-09-17T02:45:21.140589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:58:11.507133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.140103+0000","last_change":"2024-09-17T02:45:20.127506+0000","last_active":"2024-09-17T02:45:21.140103+0000","last_peered":"2024-09-17T02:45:21.140103+0000","last_clean":"2024-09-17T02:45:21.140103+0000","last_became_active":"2024-09-17T02:45:20.126699+0000","last_became_peered":"2024-09-17T02:45:20.126699+0000","last_unstale":"2024-09-17T02:45:21.140103+0000","last_undegraded":"2024-09-17T02:45:21.140103+0000","last_fullsized":"2024-09-17T02:45:21.140103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:13:45.078719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.423037+0000","last_change":"2024-09-17T02:45:19.081505+0000","last_active":"2024-09-17T02:45:21.423037+0000","last_peered":"2024-09-17T02:45:21.423037+0000","last_clean":"2024-09-17T02:45:21.423037+0000","last_became_active":"2024-09-17T02:45:19.081196+0000","last_became_peered":"2024-09-17T02:45:19.081196+0000","last_unstale":"2024-09-17T02:45:21.423037+0000","last_undegraded":"2024-09-17T02:45:21.423037+0000","last_fullsized":"2024-09-17T02:45:21.423037+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:33.638898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.740579+0000","last_change":"2024-09-17T02:45:20.139575+0000","last_active":"2024-09-17T02:45:21.740579+0000","last_peered":"2024-09-17T02:45:21.740579+0000","last_clean":"2024-09-17T02:45:21.740579+0000","last_became_active":"2024-09-17T02:45:20.137448+0000","last_became_peered":"2024-09-17T02:45:20.137448+0000","last_unstale":"2024-09-17T02:45:21.740579+0000","last_undegraded":"2024-09-17T02:45:21.740579+0000","last_fullsized":"2024-09-17T02:45:21.740579+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:47:25.489158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135457+0000","last_change":"2024-09-17T02:45:19.074384+0000","last_active":"2024-09-17T02:45:21.135457+0000","last_peered":"2024-09-17T02:45:21.135457+0000","last_clean":"2024-09-17T02:45:21.135457+0000","last_became_active":"2024-09-17T02:45:19.073958+0000","last_became_peered":"2024-09-17T02:45:19.073958+0000","last_unstale":"2024-09-17T02:45:21.135457+0000","last_undegraded":"2024-09-17T02:45:21.135457+0000","last_fullsized":"2024-09-17T02:45:21.135457+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:31:36.384573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139981+0000","last_change":"2024-09-17T02:45:20.152408+0000","last_active":"2024-09-17T02:45:21.139981+0000","last_peered":"2024-09-17T02:45:21.139981+0000","last_clean":"2024-09-17T02:45:21.139981+0000","last_became_active":"2024-09-17T02:45:20.137766+0000","last_became_peered":"2024-09-17T02:45:20.137766+0000","last_unstale":"2024-09-17T02:45:21.139981+0000","last_undegraded":"2024-09-17T02:45:21.139981+0000","last_fullsized":"2024-09-17T02:45:21.139981+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:54.493799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"78'3","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.224043+0000","last_change":"2024-09-17T02:45:19.101534+0000","last_active":"2024-09-17T02:45:21.224043+0000","last_peered":"2024-09-17T02:45:21.224043+0000","last_clean":"2024-09-17T02:45:21.224043+0000","last_became_active":"2024-09-17T02:45:19.099790+0000","last_became_peered":"2024-09-17T02:45:19.099790+0000","last_unstale":"2024-09-17T02:45:21.224043+0000","last_undegraded":"2024-09-17T02:45:21.224043+0000","last_fullsized":"2024-09-17T02:45:21.224043+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:06:59.010855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.224109+0000","last_change":"2024-09-17T02:45:20.154562+0000","last_active":"2024-09-17T02:45:21.224109+0000","last_peered":"2024-09-17T02:45:21.224109+0000","last_clean":"2024-09-17T02:45:21.224109+0000","last_became_active":"2024-09-17T02:45:20.153866+0000","last_became_peered":"2024-09-17T02:45:20.153866+0000","last_unstale":"2024-09-17T02:45:21.224109+0000","last_undegraded":"2024-09-17T02:45:21.224109+0000","last_fullsized":"2024-09-17T02:45:21.224109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:52:46.875820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.14","version":"78'6","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.239098+0000","last_change":"2024-09-17T02:45:19.079828+0000","last_active":"2024-09-17T02:45:21.239098+0000","last_peered":"2024-09-17T02:45:21.239098+0000","last_clean":"2024-09-17T02:45:21.239098+0000","last_became_active":"2024-09-17T02:45:19.079230+0000","last_became_peered":"2024-09-17T02:45:19.079230+0000","last_unstale":"2024-09-17T02:45:21.239098+0000","last_undegraded":"2024-09-17T02:45:21.239098+0000","last_fullsized":"2024-09-17T02:45:21.239098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:46.676590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.422977+0000","last_change":"2024-09-17T02:45:20.130719+0000","last_active":"2024-09-17T02:45:21.422977+0000","last_peered":"2024-09-17T02:45:21.422977+0000","last_clean":"2024-09-17T02:45:21.422977+0000","last_became_active":"2024-09-17T02:45:20.130351+0000","last_became_peered":"2024-09-17T02:45:20.130351+0000","last_unstale":"2024-09-17T02:45:21.422977+0000","last_undegraded":"2024-09-17T02:45:21.422977+0000","last_fullsized":"2024-09-17T02:45:21.422977+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:06.473725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.13","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136247+0000","last_change":"2024-09-17T02:45:19.085733+0000","last_active":"2024-09-17T02:45:21.136247+0000","last_peered":"2024-09-17T02:45:21.136247+0000","last_clean":"2024-09-17T02:45:21.136247+0000","last_became_active":"2024-09-17T02:45:19.085445+0000","last_became_peered":"2024-09-17T02:45:19.085445+0000","last_unstale":"2024-09-17T02:45:21.136247+0000","last_undegraded":"2024-09-17T02:45:21.136247+0000","last_fullsized":"2024-09-17T02:45:21.136247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:02:35.839507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136624+0000","last_change":"2024-09-17T02:45:20.127481+0000","last_active":"2024-09-17T02:45:21.136624+0000","last_peered":"2024-09-17T02:45:21.136624+0000","last_clean":"2024-09-17T02:45:21.136624+0000","last_became_active":"2024-09-17T02:45:20.127085+0000","last_became_peered":"2024-09-17T02:45:20.127085+0000","last_unstale":"2024-09-17T02:45:21.136624+0000","last_undegraded":"2024-09-17T02:45:21.136624+0000","last_fullsized":"2024-09-17T02:45:21.136624+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:42.063380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.740637+0000","last_change":"2024-09-17T02:45:19.094438+0000","last_active":"2024-09-17T02:45:21.740637+0000","last_peered":"2024-09-17T02:45:21.740637+0000","last_clean":"2024-09-17T02:45:21.740637+0000","last_became_active":"2024-09-17T02:45:19.093709+0000","last_became_peered":"2024-09-17T02:45:19.093709+0000","last_unstale":"2024-09-17T02:45:21.740637+0000","last_undegraded":"2024-09-17T02:45:21.740637+0000","last_fullsized":"2024-09-17T02:45:21.740637+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:45.773745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.740874+0000","last_change":"2024-09-17T02:45:20.131160+0000","last_active":"2024-09-17T02:45:21.740874+0000","last_peered":"2024-09-17T02:45:21.740874+0000","last_clean":"2024-09-17T02:45:21.740874+0000","last_became_active":"2024-09-17T02:45:20.129742+0000","last_became_peered":"2024-09-17T02:45:20.129742+0000","last_unstale":"2024-09-17T02:45:21.740874+0000","last_undegraded":"2024-09-17T02:45:21.740874+0000","last_fullsized":"2024-09-17T02:45:21.740874+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:50.273086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.233831+0000","last_change":"2024-09-17T02:45:19.105626+0000","last_active":"2024-09-17T02:45:21.233831+0000","last_peered":"2024-09-17T02:45:21.233831+0000","last_clean":"2024-09-17T02:45:21.233831+0000","last_became_active":"2024-09-17T02:45:19.104904+0000","last_became_peered":"2024-09-17T02:45:19.104904+0000","last_unstale":"2024-09-17T02:45:21.233831+0000","last_undegraded":"2024-09-17T02:45:21.233831+0000","last_fullsized":"2024-09-17T02:45:21.233831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:38.203466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135464+0000","last_change":"2024-09-17T02:45:20.138086+0000","last_active":"2024-09-17T02:45:21.135464+0000","last_peered":"2024-09-17T02:45:21.135464+0000","last_clean":"2024-09-17T02:45:21.135464+0000","last_became_active":"2024-09-17T02:45:20.134972+0000","last_became_peered":"2024-09-17T02:45:20.134972+0000","last_unstale":"2024-09-17T02:45:21.135464+0000","last_undegraded":"2024-09-17T02:45:21.135464+0000","last_fullsized":"2024-09-17T02:45:21.135464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:18.254883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"78'8","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.223611+0000","last_change":"2024-09-17T02:45:19.102331+0000","last_active":"2024-09-17T02:45:21.223611+0000","last_peered":"2024-09-17T02:45:21.223611+0000","last_clean":"2024-09-17T02:45:21.223611+0000","last_became_active":"2024-09-17T02:45:19.101721+0000","last_became_peered":"2024-09-17T02:45:19.101721+0000","last_unstale":"2024-09-17T02:45:21.223611+0000","last_undegraded":"2024-09-17T02:45:21.223611+0000","last_fullsized":"2024-09-17T02:45:21.223611+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:45.773745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.233224+0000","last_change":"2024-09-17T02:45:20.131992+0000","last_active":"2024-09-17T02:45:21.233224+0000","last_peered":"2024-09-17T02:45:21.233224+0000","last_clean":"2024-09-17T02:45:21.233224+0000","last_became_active":"2024-09-17T02:45:20.126966+0000","last_became_peered":"2024-09-17T02:45:20.126966+0000","last_unstale":"2024-09-17T02:45:21.233224+0000","last_undegraded":"2024-09-17T02:45:21.233224+0000","last_fullsized":"2024-09-17T02:45:21.233224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:58:49.114942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.f","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.423337+0000","last_change":"2024-09-17T02:45:19.095009+0000","last_active":"2024-09-17T02:45:21.423337+0000","last_peered":"2024-09-17T02:45:21.423337+0000","last_clean":"2024-09-17T02:45:21.423337+0000","last_became_active":"2024-09-17T02:45:19.094610+0000","last_became_peered":"2024-09-17T02:45:19.094610+0000","last_unstale":"2024-09-17T02:45:21.423337+0000","last_undegraded":"2024-09-17T02:45:21.423337+0000","last_fullsized":"2024-09-17T02:45:21.423337+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:42.163563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139855+0000","last_change":"2024-09-17T02:45:20.147442+0000","last_active":"2024-09-17T02:45:21.139855+0000","last_peered":"2024-09-17T02:45:21.139855+0000","last_clean":"2024-09-17T02:45:21.139855+0000","last_became_active":"2024-09-17T02:45:20.146649+0000","last_became_peered":"2024-09-17T02:45:20.146649+0000","last_unstale":"2024-09-17T02:45:21.139855+0000","last_undegraded":"2024-09-17T02:45:21.139855+0000","last_fullsized":"2024-09-17T02:45:21.139855+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:49:44.498880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.e","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.423502+0000","last_change":"2024-09-17T02:45:19.089116+0000","last_active":"2024-09-17T02:45:21.423502+0000","last_peered":"2024-09-17T02:45:21.423502+0000","last_clean":"2024-09-17T02:45:21.423502+0000","last_became_active":"2024-09-17T02:45:19.088667+0000","last_became_peered":"2024-09-17T02:45:19.088667+0000","last_unstale":"2024-09-17T02:45:21.423502+0000","last_undegraded":"2024-09-17T02:45:21.423502+0000","last_fullsized":"2024-09-17T02:45:21.423502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:54:35.726780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139554+0000","last_change":"2024-09-17T02:45:20.162326+0000","last_active":"2024-09-17T02:45:21.139554+0000","last_peered":"2024-09-17T02:45:21.139554+0000","last_clean":"2024-09-17T02:45:21.139554+0000","last_became_active":"2024-09-17T02:45:20.161849+0000","last_became_peered":"2024-09-17T02:45:20.161849+0000","last_unstale":"2024-09-17T02:45:21.139554+0000","last_undegraded":"2024-09-17T02:45:21.139554+0000","last_fullsized":"2024-09-17T02:45:21.139554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:50:53.540992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.d","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.233870+0000","last_change":"2024-09-17T02:45:19.099466+0000","last_active":"2024-09-17T02:45:21.233870+0000","last_peered":"2024-09-17T02:45:21.233870+0000","last_clean":"2024-09-17T02:45:21.233870+0000","last_became_active":"2024-09-17T02:45:19.098488+0000","last_became_peered":"2024-09-17T02:45:19.098488+0000","last_unstale":"2024-09-17T02:45:21.233870+0000","last_undegraded":"2024-09-17T02:45:21.233870+0000","last_fullsized":"2024-09-17T02:45:21.233870+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:32.496598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.233930+0000","last_change":"2024-09-17T02:45:20.152366+0000","last_active":"2024-09-17T02:45:21.233930+0000","last_peered":"2024-09-17T02:45:21.233930+0000","last_clean":"2024-09-17T02:45:21.233930+0000","last_became_active":"2024-09-17T02:45:20.150703+0000","last_became_peered":"2024-09-17T02:45:20.150703+0000","last_unstale":"2024-09-17T02:45:21.233930+0000","last_undegraded":"2024-09-17T02:45:21.233930+0000","last_fullsized":"2024-09-17T02:45:21.233930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:13:34.649487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.234570+0000","last_change":"2024-09-17T02:45:19.105629+0000","last_active":"2024-09-17T02:45:21.234570+0000","last_peered":"2024-09-17T02:45:21.234570+0000","last_clean":"2024-09-17T02:45:21.234570+0000","last_became_active":"2024-09-17T02:45:19.104942+0000","last_became_peered":"2024-09-17T02:45:19.104942+0000","last_unstale":"2024-09-17T02:45:21.234570+0000","last_undegraded":"2024-09-17T02:45:21.234570+0000","last_fullsized":"2024-09-17T02:45:21.234570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:55.509905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.234013+0000","last_change":"2024-09-17T02:45:20.141760+0000","last_active":"2024-09-17T02:45:21.234013+0000","last_peered":"2024-09-17T02:45:21.234013+0000","last_clean":"2024-09-17T02:45:21.234013+0000","last_became_active":"2024-09-17T02:45:20.141069+0000","last_became_peered":"2024-09-17T02:45:20.141069+0000","last_unstale":"2024-09-17T02:45:21.234013+0000","last_undegraded":"2024-09-17T02:45:21.234013+0000","last_fullsized":"2024-09-17T02:45:21.234013+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:00:31.942333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139358+0000","last_change":"2024-09-17T02:45:19.101863+0000","last_active":"2024-09-17T02:45:21.139358+0000","last_peered":"2024-09-17T02:45:21.139358+0000","last_clean":"2024-09-17T02:45:21.139358+0000","last_became_active":"2024-09-17T02:45:19.101175+0000","last_became_peered":"2024-09-17T02:45:19.101175+0000","last_unstale":"2024-09-17T02:45:21.139358+0000","last_undegraded":"2024-09-17T02:45:21.139358+0000","last_fullsized":"2024-09-17T02:45:21.139358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:15:00.618208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.234406+0000","last_change":"2024-09-17T02:45:20.127175+0000","last_active":"2024-09-17T02:45:21.234406+0000","last_peered":"2024-09-17T02:45:21.234406+0000","last_clean":"2024-09-17T02:45:21.234406+0000","last_became_active":"2024-09-17T02:45:20.126335+0000","last_became_peered":"2024-09-17T02:45:20.126335+0000","last_unstale":"2024-09-17T02:45:21.234406+0000","last_undegraded":"2024-09-17T02:45:21.234406+0000","last_fullsized":"2024-09-17T02:45:21.234406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:33.202120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.138147+0000","last_change":"2024-09-17T02:45:19.093429+0000","last_active":"2024-09-17T02:45:21.138147+0000","last_peered":"2024-09-17T02:45:21.138147+0000","last_clean":"2024-09-17T02:45:21.138147+0000","last_became_active":"2024-09-17T02:45:19.092623+0000","last_became_peered":"2024-09-17T02:45:19.092623+0000","last_unstale":"2024-09-17T02:45:21.138147+0000","last_undegraded":"2024-09-17T02:45:21.138147+0000","last_fullsized":"2024-09-17T02:45:21.138147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:25.349401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.239395+0000","last_change":"2024-09-17T02:45:20.128671+0000","last_active":"2024-09-17T02:45:21.239395+0000","last_peered":"2024-09-17T02:45:21.239395+0000","last_clean":"2024-09-17T02:45:21.239395+0000","last_became_active":"2024-09-17T02:45:20.128268+0000","last_became_peered":"2024-09-17T02:45:20.128268+0000","last_unstale":"2024-09-17T02:45:21.239395+0000","last_undegraded":"2024-09-17T02:45:21.239395+0000","last_fullsized":"2024-09-17T02:45:21.239395+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:47.811577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.140911+0000","last_change":"2024-09-17T02:45:19.089801+0000","last_active":"2024-09-17T02:45:21.140911+0000","last_peered":"2024-09-17T02:45:21.140911+0000","last_clean":"2024-09-17T02:45:21.140911+0000","last_became_active":"2024-09-17T02:45:19.083865+0000","last_became_peered":"2024-09-17T02:45:19.083865+0000","last_unstale":"2024-09-17T02:45:21.140911+0000","last_undegraded":"2024-09-17T02:45:21.140911+0000","last_fullsized":"2024-09-17T02:45:21.140911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:23:45.046413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.423078+0000","last_change":"2024-09-17T02:45:20.121012+0000","last_active":"2024-09-17T02:45:21.423078+0000","last_peered":"2024-09-17T02:45:21.423078+0000","last_clean":"2024-09-17T02:45:21.423078+0000","last_became_active":"2024-09-17T02:45:20.120133+0000","last_became_peered":"2024-09-17T02:45:20.120133+0000","last_unstale":"2024-09-17T02:45:21.423078+0000","last_undegraded":"2024-09-17T02:45:21.423078+0000","last_fullsized":"2024-09-17T02:45:21.423078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:18:25.114096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"78'2","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135613+0000","last_change":"2024-09-17T02:45:19.082147+0000","last_active":"2024-09-17T02:45:21.135613+0000","last_peered":"2024-09-17T02:45:21.135613+0000","last_clean":"2024-09-17T02:45:21.135613+0000","last_became_active":"2024-09-17T02:45:19.081729+0000","last_became_peered":"2024-09-17T02:45:19.081729+0000","last_unstale":"2024-09-17T02:45:21.135613+0000","last_undegraded":"2024-09-17T02:45:21.135613+0000","last_fullsized":"2024-09-17T02:45:21.135613+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:27.939745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.135204+0000","last_change":"2024-09-17T02:45:20.136717+0000","last_active":"2024-09-17T02:45:21.135204+0000","last_peered":"2024-09-17T02:45:21.135204+0000","last_clean":"2024-09-17T02:45:21.135204+0000","last_became_active":"2024-09-17T02:45:20.136343+0000","last_became_peered":"2024-09-17T02:45:20.136343+0000","last_unstale":"2024-09-17T02:45:21.135204+0000","last_undegraded":"2024-09-17T02:45:21.135204+0000","last_fullsized":"2024-09-17T02:45:21.135204+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:44.913811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"78'1","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.225503+0000","last_change":"2024-09-17T02:45:19.102757+0000","last_active":"2024-09-17T02:45:21.225503+0000","last_peered":"2024-09-17T02:45:21.225503+0000","last_clean":"2024-09-17T02:45:21.225503+0000","last_became_active":"2024-09-17T02:45:19.100744+0000","last_became_peered":"2024-09-17T02:45:19.100744+0000","last_unstale":"2024-09-17T02:45:21.225503+0000","last_undegraded":"2024-09-17T02:45:21.225503+0000","last_fullsized":"2024-09-17T02:45:21.225503+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:23:57.648928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.740118+0000","last_change":"2024-09-17T02:45:20.130812+0000","last_active":"2024-09-17T02:45:21.740118+0000","last_peered":"2024-09-17T02:45:21.740118+0000","last_clean":"2024-09-17T02:45:21.740118+0000","last_became_active":"2024-09-17T02:45:20.130480+0000","last_became_peered":"2024-09-17T02:45:20.130480+0000","last_unstale":"2024-09-17T02:45:21.740118+0000","last_undegraded":"2024-09-17T02:45:21.740118+0000","last_fullsized":"2024-09-17T02:45:21.740118+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:37:58.215125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.136826+0000","last_change":"2024-09-17T02:45:19.093011+0000","last_active":"2024-09-17T02:45:21.136826+0000","last_peered":"2024-09-17T02:45:21.136826+0000","last_clean":"2024-09-17T02:45:21.136826+0000","last_became_active":"2024-09-17T02:45:19.091917+0000","last_became_peered":"2024-09-17T02:45:19.091917+0000","last_unstale":"2024-09-17T02:45:21.136826+0000","last_undegraded":"2024-09-17T02:45:21.136826+0000","last_fullsized":"2024-09-17T02:45:21.136826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:03.939829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.139221+0000","last_change":"2024-09-17T02:45:20.153283+0000","last_active":"2024-09-17T02:45:21.139221+0000","last_peered":"2024-09-17T02:45:21.139221+0000","last_clean":"2024-09-17T02:45:21.139221+0000","last_became_active":"2024-09-17T02:45:20.152428+0000","last_became_peered":"2024-09-17T02:45:20.152428+0000","last_unstale":"2024-09-17T02:45:21.139221+0000","last_undegraded":"2024-09-17T02:45:21.139221+0000","last_fullsized":"2024-09-17T02:45:21.139221+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:36.231357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.422973+0000","last_change":"2024-09-17T02:45:19.096590+0000","last_active":"2024-09-17T02:45:21.422973+0000","last_peered":"2024-09-17T02:45:21.422973+0000","last_clean":"2024-09-17T02:45:21.422973+0000","last_became_active":"2024-09-17T02:45:19.096024+0000","last_became_peered":"2024-09-17T02:45:19.096024+0000","last_unstale":"2024-09-17T02:45:21.422973+0000","last_undegraded":"2024-09-17T02:45:21.422973+0000","last_fullsized":"2024-09-17T02:45:21.422973+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:17.702397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.740661+0000","last_change":"2024-09-17T02:45:20.120878+0000","last_active":"2024-09-17T02:45:21.740661+0000","last_peered":"2024-09-17T02:45:21.740661+0000","last_clean":"2024-09-17T02:45:21.740661+0000","last_became_active":"2024-09-17T02:45:20.119874+0000","last_became_peered":"2024-09-17T02:45:20.119874+0000","last_unstale":"2024-09-17T02:45:21.740661+0000","last_undegraded":"2024-09-17T02:45:21.740661+0000","last_fullsized":"2024-09-17T02:45:21.740661+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:37.115388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.238608+0000","last_change":"2024-09-17T02:45:19.084757+0000","last_active":"2024-09-17T02:45:21.238608+0000","last_peered":"2024-09-17T02:45:21.238608+0000","last_clean":"2024-09-17T02:45:21.238608+0000","last_became_active":"2024-09-17T02:45:19.084284+0000","last_became_peered":"2024-09-17T02:45:19.084284+0000","last_unstale":"2024-09-17T02:45:21.238608+0000","last_undegraded":"2024-09-17T02:45:21.238608+0000","last_fullsized":"2024-09-17T02:45:21.238608+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:44:31.618471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.423137+0000","last_change":"2024-09-17T02:45:20.128870+0000","last_active":"2024-09-17T02:45:21.423137+0000","last_peered":"2024-09-17T02:45:21.423137+0000","last_clean":"2024-09-17T02:45:21.423137+0000","last_became_active":"2024-09-17T02:45:20.128450+0000","last_became_peered":"2024-09-17T02:45:20.128450+0000","last_unstale":"2024-09-17T02:45:21.423137+0000","last_undegraded":"2024-09-17T02:45:21.423137+0000","last_fullsized":"2024-09-17T02:45:21.423137+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:28.895745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.225029+0000","last_change":"2024-09-17T02:45:19.106516+0000","last_active":"2024-09-17T02:45:21.225029+0000","last_peered":"2024-09-17T02:45:21.225029+0000","last_clean":"2024-09-17T02:45:21.225029+0000","last_became_active":"2024-09-17T02:45:19.105883+0000","last_became_peered":"2024-09-17T02:45:19.105883+0000","last_unstale":"2024-09-17T02:45:21.225029+0000","last_undegraded":"2024-09-17T02:45:21.225029+0000","last_fullsized":"2024-09-17T02:45:21.225029+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:15:39.027873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.141439+0000","last_change":"2024-09-17T02:45:20.146992+0000","last_active":"2024-09-17T02:45:21.141439+0000","last_peered":"2024-09-17T02:45:21.141439+0000","last_clean":"2024-09-17T02:45:21.141439+0000","last_became_active":"2024-09-17T02:45:20.146129+0000","last_became_peered":"2024-09-17T02:45:20.146129+0000","last_unstale":"2024-09-17T02:45:21.141439+0000","last_undegraded":"2024-09-17T02:45:21.141439+0000","last_fullsized":"2024-09-17T02:45:21.141439+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:08.007508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.422707+0000","last_change":"2024-09-17T02:45:19.086253+0000","last_active":"2024-09-17T02:45:21.422707+0000","last_peered":"2024-09-17T02:45:21.422707+0000","last_clean":"2024-09-17T02:45:21.422707+0000","last_became_active":"2024-09-17T02:45:19.085807+0000","last_became_peered":"2024-09-17T02:45:19.085807+0000","last_unstale":"2024-09-17T02:45:21.422707+0000","last_undegraded":"2024-09-17T02:45:21.422707+0000","last_fullsized":"2024-09-17T02:45:21.422707+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:57:43.903222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.234330+0000","last_change":"2024-09-17T02:45:20.124232+0000","last_active":"2024-09-17T02:45:21.234330+0000","last_peered":"2024-09-17T02:45:21.234330+0000","last_clean":"2024-09-17T02:45:21.234330+0000","last_became_active":"2024-09-17T02:45:20.123450+0000","last_became_peered":"2024-09-17T02:45:20.123450+0000","last_unstale":"2024-09-17T02:45:21.234330+0000","last_undegraded":"2024-09-17T02:45:21.234330+0000","last_fullsized":"2024-09-17T02:45:21.234330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:00.805928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.141220+0000","last_change":"2024-09-17T02:45:19.087894+0000","last_active":"2024-09-17T02:45:21.141220+0000","last_peered":"2024-09-17T02:45:21.141220+0000","last_clean":"2024-09-17T02:45:21.141220+0000","last_became_active":"2024-09-17T02:45:19.086041+0000","last_became_peered":"2024-09-17T02:45:19.086041+0000","last_unstale":"2024-09-17T02:45:21.141220+0000","last_undegraded":"2024-09-17T02:45:21.141220+0000","last_fullsized":"2024-09-17T02:45:21.141220+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:53.624694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.238976+0000","last_change":"2024-09-17T02:45:20.128777+0000","last_active":"2024-09-17T02:45:21.238976+0000","last_peered":"2024-09-17T02:45:21.238976+0000","last_clean":"2024-09-17T02:45:21.238976+0000","last_became_active":"2024-09-17T02:45:20.128362+0000","last_became_peered":"2024-09-17T02:45:20.128362+0000","last_unstale":"2024-09-17T02:45:21.238976+0000","last_undegraded":"2024-09-17T02:45:21.238976+0000","last_fullsized":"2024-09-17T02:45:21.238976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:19.049447+0000","last_clean_scrub_stamp":"2024-09-17T02:45:19.049447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:42:16.585337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.224734+0000","last_change":"2024-09-17T02:45:19.099271+0000","last_active":"2024-09-17T02:45:21.224734+0000","last_peered":"2024-09-17T02:45:21.224734+0000","last_clean":"2024-09-17T02:45:21.224734+0000","last_became_active":"2024-09-17T02:45:19.088289+0000","last_became_peered":"2024-09-17T02:45:19.088289+0000","last_unstale":"2024-09-17T02:45:21.224734+0000","last_undegraded":"2024-09-17T02:45:21.224734+0000","last_fullsized":"2024-09-17T02:45:21.224734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:45:18.038858+0000","last_clean_scrub_stamp":"2024-09-17T02:45:18.038858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:54.306776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":220,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-17T02:45:21.134645+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:45:21.134645+0000","last_peered":"2024-09-17T02:45:21.134645+0000","last_clean":"2024-09-17T02:45:21.134645+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:45:21.134645+0000","last_undegraded":"2024-09-17T02:45:21.134645+0000","last_fullsized":"2024-09-17T02:45:21.134645+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509501,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":966656,"data_stored":438091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509501,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1556480,"data_stored":1027897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":962560,"data_stored":437461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":978944,"data_stored":439155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509501,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":958464,"data_stored":437439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1568768,"data_stored":1029453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509502,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":962560,"data_stored":437461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":438111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:45:29.245 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T02:45:29.245 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T02:45:29.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.450+0000 7f911acf1640 1 -- 172.21.15.131:0/1112758073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f911415ece0 msgr2=0x7f91141512f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.450+0000 7f911acf1640 1 --2- 172.21.15.131:0/1112758073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f911415ece0 0x7f91141512f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9114155650 tx=0x7f910801c830 comp rx=0 tx=0).stop 2024-09-17T02:45:29.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.450+0000 7f911acf1640 1 -- 172.21.15.131:0/1112758073 shutdown_connections 2024-09-17T02:45:29.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.450+0000 7f911acf1640 1 --2- 172.21.15.131:0/1112758073 >> v2:172.21.15.189:3300/0 conn(0x7f9114151830 0x7f911406f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.450+0000 7f911acf1640 1 --2- 172.21.15.131:0/1112758073 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f911415ece0 0x7f91141512f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.450+0000 7f911acf1640 1 -- 172.21.15.131:0/1112758073 >> 172.21.15.131:0/1112758073 conn(0x7f91140d0490 msgr2=0x7f911406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:29.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.450+0000 7f911acf1640 1 -- 172.21.15.131:0/1112758073 shutdown_connections 2024-09-17T02:45:29.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.450+0000 7f911acf1640 1 -- 172.21.15.131:0/1112758073 wait complete. 2024-09-17T02:45:29.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.450+0000 7f911acf1640 1 Processor -- start 2024-09-17T02:45:29.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.450+0000 7f911acf1640 1 -- start start 2024-09-17T02:45:29.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911acf1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91140cf680 0x7f91141443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911acf1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9114151830 0x7f91141f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911acf1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9114145080 0x7f91141fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911acf1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9114070820 con 0x7f9114145080 2024-09-17T02:45:29.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911acf1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f91140706a0 con 0x7f9114151830 2024-09-17T02:45:29.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911acf1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f91140709a0 con 0x7f91140cf680 2024-09-17T02:45:29.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911928e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9114145080 0x7f91141fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911928e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9114145080 0x7f91141fa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44204/0 (socket says 172.21.15.131:44204) 2024-09-17T02:45:29.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911928e640 1 -- 172.21.15.131:0/4166568123 learned_addr learned my addr 172.21.15.131:0/4166568123 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:29.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f9113fff640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9114151830 0x7f91141f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f9118a8d640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91140cf680 0x7f91141443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911928e640 1 -- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91140cf680 msgr2=0x7f91141443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911928e640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91140cf680 0x7f91141443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911928e640 1 -- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9114151830 msgr2=0x7f91141f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911928e640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9114151830 0x7f91141f69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911928e640 1 -- 172.21.15.131:0/4166568123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9114144930 con 0x7f9114145080 2024-09-17T02:45:29.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f9118a8d640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91140cf680 0x7f91141443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:29.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911928e640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9114145080 0x7f91141fa6d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f910c00e970 tx=0x7f910c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:29.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f91117fa640 1 -- 172.21.15.131:0/4166568123 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c004d30 con 0x7f9114145080 2024-09-17T02:45:29.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f91117fa640 1 -- 172.21.15.131:0/4166568123 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f910c00cb20 con 0x7f9114145080 2024-09-17T02:45:29.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911acf1640 1 -- 172.21.15.131:0/4166568123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91141f6f30 con 0x7f9114145080 2024-09-17T02:45:29.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f91117fa640 1 -- 172.21.15.131:0/4166568123 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c005020 con 0x7f9114145080 2024-09-17T02:45:29.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911acf1640 1 -- 172.21.15.131:0/4166568123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9114165ab0 con 0x7f9114145080 2024-09-17T02:45:29.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f911acf1640 1 -- 172.21.15.131:0/4166568123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91141fa030 con 0x7f9114145080 2024-09-17T02:45:29.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f91117fa640 1 -- 172.21.15.131:0/4166568123 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f910c0051c0 con 0x7f9114145080 2024-09-17T02:45:29.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.454+0000 7f91117fa640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9100073200 0x7f9100075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.458+0000 7f91117fa640 1 -- 172.21.15.131:0/4166568123 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f910c094cd0 con 0x7f9114145080 2024-09-17T02:45:29.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.458+0000 7f9118a8d640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9100073200 0x7f9100075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.458+0000 7f9118a8d640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9100073200 0x7f9100075710 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f91140cc540 tx=0x7f9104008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:29.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.462+0000 7f91117fa640 1 -- 172.21.15.131:0/4166568123 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9114144930 con 0x7f9114145080 2024-09-17T02:45:29.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.686+0000 7f911acf1640 1 -- 172.21.15.131:0/4166568123 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f911406f740 con 0x7f9114145080 2024-09-17T02:45:29.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.690+0000 7f91117fa640 1 -- 172.21.15.131:0/4166568123 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v32) ==== 190+0+147 (secure 0 0 0) 0x7f910c016030 con 0x7f9114145080 2024-09-17T02:45:29.690 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-09-17T02:45:29.691 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA== 2024-09-17T02:45:29.691 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-09-17T02:45:29.691 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-09-17T02:45:29.691 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T02:45:29.693 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-09-17T02:45:29.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 -- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9100073200 msgr2=0x7f9100075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9100073200 0x7f9100075710 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f91140cc540 tx=0x7f9104008040 comp rx=0 tx=0).stop 2024-09-17T02:45:29.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 -- 172.21.15.131:0/4166568123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9114145080 msgr2=0x7f91141fa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9114145080 0x7f91141fa6d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f910c00e970 tx=0x7f910c00ee30 comp rx=0 tx=0).stop 2024-09-17T02:45:29.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 -- 172.21.15.131:0/4166568123 shutdown_connections 2024-09-17T02:45:29.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9100073200 0x7f9100075710 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9114145080 0x7f91141fa6d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9114151830 0x7f91141f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 --2- 172.21.15.131:0/4166568123 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f91140cf680 0x7f91141443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 -- 172.21.15.131:0/4166568123 >> 172.21.15.131:0/4166568123 conn(0x7f91140d0490 msgr2=0x7f91140cd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:29.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 -- 172.21.15.131:0/4166568123 shutdown_connections 2024-09-17T02:45:29.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.694+0000 7f911acf1640 1 -- 172.21.15.131:0/4166568123 wait complete. 2024-09-17T02:45:29.710 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:29.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.858+0000 7f03475b8640 1 -- 172.21.15.131:0/332839488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032c003a60 msgr2=0x7f032c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.858+0000 7f03475b8640 1 --2- 172.21.15.131:0/332839488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032c003a60 0x7f032c003ed0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f033c009b40 tx=0x7f033c01d250 comp rx=0 tx=0).stop 2024-09-17T02:45:29.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.858+0000 7f03475b8640 1 -- 172.21.15.131:0/332839488 shutdown_connections 2024-09-17T02:45:29.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.858+0000 7f03475b8640 1 --2- 172.21.15.131:0/332839488 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f032c003a60 0x7f032c003ed0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.858+0000 7f03475b8640 1 -- 172.21.15.131:0/332839488 >> 172.21.15.131:0/332839488 conn(0x7f034006eea0 msgr2=0x7f034006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:29.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.858+0000 7f03475b8640 1 -- 172.21.15.131:0/332839488 shutdown_connections 2024-09-17T02:45:29.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.858+0000 7f03475b8640 1 -- 172.21.15.131:0/332839488 wait complete. 2024-09-17T02:45:29.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f03475b8640 1 Processor -- start 2024-09-17T02:45:29.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f03475b8640 1 -- start start 2024-09-17T02:45:29.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f03475b8640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f032c003a60 0x7f03401f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f03475b8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03400a9680 0x7f0340157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f03475b8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03401541c0 0x7f0340156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f03475b8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f03400cc480 con 0x7f03400a9680 2024-09-17T02:45:29.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f03475b8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f03400cc300 con 0x7f03401541c0 2024-09-17T02:45:29.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f03475b8640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f03400cc600 con 0x7f032c003a60 2024-09-17T02:45:29.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0344b53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03400a9680 0x7f0340157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0344b53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03400a9680 0x7f0340157430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44232/0 (socket says 172.21.15.131:44232) 2024-09-17T02:45:29.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0344b53640 1 -- 172.21.15.131:0/3798945822 learned_addr learned my addr 172.21.15.131:0/3798945822 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:29.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0345354640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f032c003a60 0x7f03401f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0345b55640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03401541c0 0x7f0340156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0344b53640 1 -- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f032c003a60 msgr2=0x7f03401f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0344b53640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f032c003a60 0x7f03401f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0344b53640 1 -- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03401541c0 msgr2=0x7f0340156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0344b53640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03401541c0 0x7f0340156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0344b53640 1 -- 172.21.15.131:0/3798945822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03400aa670 con 0x7f03400a9680 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0344b53640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03400a9680 0x7f0340157430 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0330004ee0 tx=0x7f0330001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0335ffb640 1 -- 172.21.15.131:0/3798945822 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330015020 con 0x7f03400a9680 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0335ffb640 1 -- 172.21.15.131:0/3798945822 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0330011960 con 0x7f03400a9680 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f0335ffb640 1 -- 172.21.15.131:0/3798945822 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330011d90 con 0x7f03400a9680 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f03475b8640 1 -- 172.21.15.131:0/3798945822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03400a9dd0 con 0x7f03400a9680 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f03475b8640 1 -- 172.21.15.131:0/3798945822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03400aa390 con 0x7f03400a9680 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.862+0000 7f03475b8640 1 -- 172.21.15.131:0/3798945822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0340075b80 con 0x7f03400a9680 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.866+0000 7f0335ffb640 1 -- 172.21.15.131:0/3798945822 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0330018020 con 0x7f03400a9680 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.866+0000 7f0335ffb640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0324073200 0x7f0324075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:29.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.866+0000 7f0335ffb640 1 -- 172.21.15.131:0/3798945822 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f0330094e10 con 0x7f03400a9680 2024-09-17T02:45:29.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.866+0000 7f0345354640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0324073200 0x7f0324075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:29.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.866+0000 7f0345354640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0324073200 0x7f0324075710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f033c0ad000 tx=0x7f033c019ff0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:29.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:29.870+0000 7f0335ffb640 1 -- 172.21.15.131:0/3798945822 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f033001f020 con 0x7f03400a9680 2024-09-17T02:45:30.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.070+0000 7f03475b8640 1 -- 172.21.15.131:0/3798945822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03400a9b60 con 0x7f03400a9680 2024-09-17T02:45:30.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.070+0000 7f0335ffb640 1 -- 172.21.15.131:0/3798945822 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5228 (secure 0 0 0) 0x7f03400aa670 con 0x7f03400a9680 2024-09-17T02:45:30.070 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:30.071 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":46,"btime":"2024-09-17T02:45:23:141401+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:45:22.552270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6399":{"gid":6399,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/655402879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":655402879},{"type":"v1","addr":"172.21.15.131:6835","nonce":655402879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6399,"qdb_cluster":[6399]},"id":5}]} 2024-09-17T02:45:30.071 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 46 2024-09-17T02:45:30.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 -- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0324073200 msgr2=0x7f0324075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0324073200 0x7f0324075710 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f033c0ad000 tx=0x7f033c019ff0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 -- 172.21.15.131:0/3798945822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03400a9680 msgr2=0x7f0340157430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03400a9680 0x7f0340157430 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0330004ee0 tx=0x7f0330001640 comp rx=0 tx=0).stop 2024-09-17T02:45:30.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 -- 172.21.15.131:0/3798945822 shutdown_connections 2024-09-17T02:45:30.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0324073200 0x7f0324075710 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f03401541c0 0x7f0340156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f03400a9680 0x7f0340157430 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 --2- 172.21.15.131:0/3798945822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f032c003a60 0x7f03401f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 -- 172.21.15.131:0/3798945822 >> 172.21.15.131:0/3798945822 conn(0x7f034006eea0 msgr2=0x7f034015ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:30.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 -- 172.21.15.131:0/3798945822 shutdown_connections 2024-09-17T02:45:30.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.074+0000 7f03475b8640 1 -- 172.21.15.131:0/3798945822 wait complete. 2024-09-17T02:45:30.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:45:19.460749+0000', 'modified': '2024-09-17T02:45:22.552270+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6399}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6399': {'gid': 6399, 'name': 'c', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6835/655402879', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6833', 'nonce': 655402879}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 655402879}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6399, 'qdb_cluster': [6399]} 2024-09-17T02:45:30.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:45:30.496 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T02:45:30.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.662+0000 7f1644f0f640 1 -- 172.21.15.131:0/1821973798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f164015c640 msgr2=0x7f16400ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.662+0000 7f1644f0f640 1 --2- 172.21.15.131:0/1821973798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f164015c640 0x7f16400ccb20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1640074bd0 tx=0x7f163001c800 comp rx=0 tx=0).stop 2024-09-17T02:45:30.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.662+0000 7f1644f0f640 1 -- 172.21.15.131:0/1821973798 shutdown_connections 2024-09-17T02:45:30.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.662+0000 7f1644f0f640 1 --2- 172.21.15.131:0/1821973798 >> v2:172.21.15.189:3300/0 conn(0x7f16400b2220 0x7f16400d12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.662+0000 7f1644f0f640 1 --2- 172.21.15.131:0/1821973798 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f16400cd060 0x7f16400b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.662+0000 7f1644f0f640 1 --2- 172.21.15.131:0/1821973798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f164015c640 0x7f16400ccb20 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.662+0000 7f1644f0f640 1 -- 172.21.15.131:0/1821973798 >> 172.21.15.131:0/1821973798 conn(0x7f16400b3f30 msgr2=0x7f16400b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:30.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.662+0000 7f1644f0f640 1 -- 172.21.15.131:0/1821973798 shutdown_connections 2024-09-17T02:45:30.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.662+0000 7f1644f0f640 1 -- 172.21.15.131:0/1821973798 wait complete. 2024-09-17T02:45:30.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1644f0f640 1 Processor -- start 2024-09-17T02:45:30.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1644f0f640 1 -- start start 2024-09-17T02:45:30.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1644f0f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16400b2220 0x7f1640155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1644f0f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16400cd060 0x7f1640151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1644f0f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f164015c640 0x7f16401544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1644f0f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f16401605f0 con 0x7f16400cd060 2024-09-17T02:45:30.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1644f0f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1640160470 con 0x7f16400b2220 2024-09-17T02:45:30.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1644f0f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f1640160770 con 0x7f164015c640 2024-09-17T02:45:30.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1635d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16400cd060 0x7f1640151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1635d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16400cd060 0x7f1640151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44248/0 (socket says 172.21.15.131:44248) 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1635d9b640 1 -- 172.21.15.131:0/2485686006 learned_addr learned my addr 172.21.15.131:0/2485686006 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f163ed9d640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f164015c640 0x7f16401544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1635d9b640 1 -- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f164015c640 msgr2=0x7f16401544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1635d9b640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f164015c640 0x7f16401544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1635d9b640 1 -- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16400b2220 msgr2=0x7f1640155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f163e59c640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16400b2220 0x7f1640155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1635d9b640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16400b2220 0x7f1640155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.666+0000 7f1635d9b640 1 -- 172.21.15.131:0/2485686006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16401f6370 con 0x7f16400cd060 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f1635d9b640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16400cd060 0x7f1640151ae0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f162800d720 tx=0x7f162800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f16377fe640 1 -- 172.21.15.131:0/2485686006 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628017070 con 0x7f16400cd060 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f16377fe640 1 -- 172.21.15.131:0/2485686006 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1628005310 con 0x7f16400cd060 2024-09-17T02:45:30.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f1644f0f640 1 -- 172.21.15.131:0/2485686006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1640152a90 con 0x7f16400cd060 2024-09-17T02:45:30.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f16377fe640 1 -- 172.21.15.131:0/2485686006 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16280097c0 con 0x7f16400cd060 2024-09-17T02:45:30.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f1644f0f640 1 -- 172.21.15.131:0/2485686006 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f164015d950 con 0x7f16400cd060 2024-09-17T02:45:30.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f16377fe640 1 -- 172.21.15.131:0/2485686006 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1628009960 con 0x7f16400cd060 2024-09-17T02:45:30.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f16377fe640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1620073200 0x7f1620075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f16377fe640 1 -- 172.21.15.131:0/2485686006 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f16280944b0 con 0x7f16400cd060 2024-09-17T02:45:30.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f163e59c640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1620073200 0x7f1620075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f164c1da000 1 Processor -- start 2024-09-17T02:45:30.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.670+0000 7f164c1da000 1 -- start start 2024-09-17T02:45:30.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f163e59c640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1620073200 0x7f1620075710 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f16300af000 tx=0x7f163001b040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:30.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a0406460 0x5591a09476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07f6950 0x5591a0947be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07dfca0 0x5591a0948120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5591a081c410 con 0x5591a07dfca0 2024-09-17T02:45:30.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5591a0759b40 con 0x5591a07f6950 2024-09-17T02:45:30.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5591a0748f70 con 0x5591a0406460 2024-09-17T02:45:30.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f163ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07dfca0 0x5591a0948120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f163ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07dfca0 0x5591a0948120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44250/0 (socket says 172.21.15.131:44250) 2024-09-17T02:45:30.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f163ed9d640 1 -- 172.21.15.131:0/872029748 learned_addr learned my addr 172.21.15.131:0/872029748 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:30.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f163e59c640 1 --2- 172.21.15.131:0/872029748 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a0406460 0x5591a09476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f1635d9b640 1 --2- 172.21.15.131:0/872029748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07f6950 0x5591a0947be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f163ed9d640 1 -- 172.21.15.131:0/872029748 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a0406460 msgr2=0x5591a09476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f163ed9d640 1 --2- 172.21.15.131:0/872029748 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a0406460 0x5591a09476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f163ed9d640 1 -- 172.21.15.131:0/872029748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07f6950 msgr2=0x5591a0947be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f163ed9d640 1 --2- 172.21.15.131:0/872029748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07f6950 0x5591a0947be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f163ed9d640 1 -- 172.21.15.131:0/872029748 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a07105b0 con 0x5591a07dfca0 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f1635d9b640 1 --2- 172.21.15.131:0/872029748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07f6950 0x5591a0947be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f163ed9d640 1 --2- 172.21.15.131:0/872029748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07dfca0 0x5591a0948120 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1638008c90 tx=0x7f163800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c998ff4790e6b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f161ffff640 1 -- 172.21.15.131:0/872029748 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638018680 con 0x5591a07dfca0 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f161ffff640 1 -- 172.21.15.131:0/872029748 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1638018910 con 0x5591a07dfca0 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f161ffff640 1 -- 172.21.15.131:0/872029748 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638018d40 con 0x5591a07dfca0 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 -- 172.21.15.131:0/872029748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07dfca0 msgr2=0x5591a0948120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 --2- 172.21.15.131:0/872029748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07dfca0 0x5591a0948120 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1638008c90 tx=0x7f163800d710 comp rx=0 tx=0).stop 2024-09-17T02:45:30.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 -- 172.21.15.131:0/872029748 shutdown_connections 2024-09-17T02:45:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 --2- 172.21.15.131:0/872029748 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07dfca0 0x5591a0948120 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 --2- 172.21.15.131:0/872029748 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07f6950 0x5591a0947be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 --2- 172.21.15.131:0/872029748 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a0406460 0x5591a09476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 -- 172.21.15.131:0/872029748 >> 172.21.15.131:0/872029748 conn(0x5591a08ccd90 msgr2=0x5591a08fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 -- 172.21.15.131:0/872029748 shutdown_connections 2024-09-17T02:45:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 -- 172.21.15.131:0/872029748 wait complete. 2024-09-17T02:45:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 -- start start 2024-09-17T02:45:30.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.674+0000 7f164c1da000 1 Processor -- start 2024-09-17T02:45:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a094e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07dfca0 0x5591a094ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07f6950 0x5591a094f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5591a06e35f0 con 0x5591a0406460 2024-09-17T02:45:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5591a074cd80 con 0x5591a07dfca0 2024-09-17T02:45:30.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5591a06ef520 con 0x5591a07f6950 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 10 client.0 _ll_register_callbacks cb 0x5591a06f3830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f163e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a094e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f163e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a094e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44266/0 (socket says 172.21.15.131:44266) 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f163e59c640 1 -- 172.21.15.131:0/4033671038 learned_addr learned my addr 172.21.15.131:0/4033671038 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f163ed9d640 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07f6950 0x5591a094f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f1635d9b640 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07dfca0 0x5591a094ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f163e59c640 1 -- 172.21.15.131:0/4033671038 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07f6950 msgr2=0x5591a094f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f163e59c640 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07f6950 0x5591a094f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f163e59c640 1 -- 172.21.15.131:0/4033671038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07dfca0 msgr2=0x5591a094ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f163e59c640 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07dfca0 0x5591a094ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f163e59c640 1 -- 172.21.15.131:0/4033671038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a06e7190 con 0x5591a0406460 2024-09-17T02:45:30.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f163e59c640 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a094e920 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f16300b0040 tx=0x7f1630009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ed41f12957636 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f161d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f161d7fa640 1 -- 172.21.15.131:0/4033671038 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300c8060 con 0x5591a0406460 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f161d7fa640 1 -- 172.21.15.131:0/4033671038 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f16300a5cf0 con 0x5591a0406460 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 -- 172.21.15.131:0/4033671038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5591a06e1110 con 0x5591a0406460 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f161d7fa640 1 -- 172.21.15.131:0/4033671038 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f16300ce050 con 0x5591a0406460 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 10 client.6546 fetch_fsmap learned FSMap version 46 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 -- 172.21.15.131:0/4033671038 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x5591a07cdf10 con 0x5591a0406460 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f161d7fa640 1 -- 172.21.15.131:0/4033671038 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5591a06e1110 con 0x5591a0406460 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f161d7fa640 1 -- 172.21.15.131:0/4033671038 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f16300d1020 con 0x5591a0406460 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 10 client.6546 fetch_fsmap finished waiting for FSMap version 46 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 10 client.6546 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 20 client.6546 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.131:6833/655402879,v1:172.21.15.131:6835/655402879] conn(0x5591a03eb7d0 0x5591a0958d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 4 client.6546 mds_command: new command op to 6399 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 -- 172.21.15.131:0/4033671038 --> [v2:172.21.15.131:6833/655402879,v1:172.21.15.131:6835/655402879] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591a075ba90 con 0x5591a03eb7d0 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f1635d9b640 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.131:6833/655402879,v1:172.21.15.131:6835/655402879] conn(0x5591a03eb7d0 0x5591a0958d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f161d7fa640 10 client.6546 ms_handle_connect on v2:172.21.15.131:6833/655402879 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f1635d9b640 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.131:6833/655402879,v1:172.21.15.131:6835/655402879] conn(0x5591a03eb7d0 0x5591a0958d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5fefebd78145955 server_cookie=c3b4fdbdb02caa83 in_seq=0 out_seq=0 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f161d7fa640 1 -- 172.21.15.131:0/4033671038 <== mds.0 v2:172.21.15.131:6833/655402879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591a075ba90 con 0x5591a03eb7d0 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f161d7fa640 10 client.6546 handle_command_reply: tid=0 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 client.6546 shutdown 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 -- 172.21.15.131:0/4033671038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 msgr2=0x5591a094e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a094e920 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f16300b0040 tx=0x7f1630009260 comp rx=0 tx=0).stop 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 -- 172.21.15.131:0/4033671038 shutdown_connections 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.131:6833/655402879,v1:172.21.15.131:6835/655402879] conn(0x5591a03eb7d0 0x5591a0958d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07f6950 0x5591a094f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07dfca0 0x5591a094ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f164c1da000 1 --2- 172.21.15.131:0/4033671038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a094e920 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.678+0000 7f161d7fa640 0 client.6546 ms_handle_reset on v2:172.21.15.131:6833/655402879 2024-09-17T02:45:30.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.682+0000 7f164c1da000 1 -- 172.21.15.131:0/4033671038 >> 172.21.15.131:0/4033671038 conn(0x5591a08ccd90 msgr2=0x5591a0903440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:30.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.682+0000 7f164c1da000 1 -- 172.21.15.131:0/4033671038 shutdown_connections 2024-09-17T02:45:30.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.682+0000 7f164c1da000 1 -- 172.21.15.131:0/4033671038 wait complete. 2024-09-17T02:45:30.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.682+0000 7f164c1da000 20 client.6546 trim_cache size 0 max 16384 2024-09-17T02:45:30.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.698+0000 7f164c1da000 1 Processor -- start 2024-09-17T02:45:30.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.698+0000 7f164c1da000 1 -- start start 2024-09-17T02:45:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a0406460 0x5591a09b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07dfca0 0x5591a09b9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07f6950 0x5591a09ba0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1640203450 con 0x5591a07f6950 2024-09-17T02:45:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1640202470 con 0x5591a0406460 2024-09-17T02:45:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f164015f5b0 con 0x5591a07dfca0 2024-09-17T02:45:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f163ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07f6950 0x5591a09ba0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f163ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07f6950 0x5591a09ba0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44272/0 (socket says 172.21.15.131:44272) 2024-09-17T02:45:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f163ed9d640 1 -- 172.21.15.131:0/2640331687 learned_addr learned my addr 172.21.15.131:0/2640331687 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f1635d9b640 1 --2- 172.21.15.131:0/2640331687 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07dfca0 0x5591a09b9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f163e59c640 1 --2- 172.21.15.131:0/2640331687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a0406460 0x5591a09b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f163ed9d640 1 -- 172.21.15.131:0/2640331687 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07dfca0 msgr2=0x5591a09b9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:30.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f163ed9d640 1 --2- 172.21.15.131:0/2640331687 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07dfca0 0x5591a09b9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f163ed9d640 1 -- 172.21.15.131:0/2640331687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a0406460 msgr2=0x5591a09b9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:30.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f163ed9d640 1 --2- 172.21.15.131:0/2640331687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a0406460 0x5591a09b9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f163ed9d640 1 -- 172.21.15.131:0/2640331687 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a07a4db0 con 0x5591a07f6950 2024-09-17T02:45:30.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f1635d9b640 1 --2- 172.21.15.131:0/2640331687 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07dfca0 0x5591a09b9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:30.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f163ed9d640 1 --2- 172.21.15.131:0/2640331687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07f6950 0x5591a09ba0a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1638008be0 tx=0x7f1638004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e60ce9651d9932 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:30.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f161f7fe640 1 -- 172.21.15.131:0/2640331687 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638004d30 con 0x5591a07f6950 2024-09-17T02:45:30.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f161f7fe640 1 -- 172.21.15.131:0/2640331687 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16380187c0 con 0x5591a07f6950 2024-09-17T02:45:30.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f161f7fe640 1 -- 172.21.15.131:0/2640331687 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638018c10 con 0x5591a07f6950 2024-09-17T02:45:30.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 -- 172.21.15.131:0/2640331687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07f6950 msgr2=0x5591a09ba0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 --2- 172.21.15.131:0/2640331687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07f6950 0x5591a09ba0a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f1638008be0 tx=0x7f1638004ac0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 -- 172.21.15.131:0/2640331687 shutdown_connections 2024-09-17T02:45:30.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 --2- 172.21.15.131:0/2640331687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a07f6950 0x5591a09ba0a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 --2- 172.21.15.131:0/2640331687 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07dfca0 0x5591a09b9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 --2- 172.21.15.131:0/2640331687 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a0406460 0x5591a09b9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 -- 172.21.15.131:0/2640331687 >> 172.21.15.131:0/2640331687 conn(0x5591a08ccd90 msgr2=0x5591a0945620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:30.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 -- 172.21.15.131:0/2640331687 shutdown_connections 2024-09-17T02:45:30.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 -- 172.21.15.131:0/2640331687 wait complete. 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 -- start start 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.702+0000 7f164c1da000 1 Processor -- start 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a09bd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07dfca0 0x5591a09bd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07f6950 0x5591a09bde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5591a07cda80 con 0x5591a0406460 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5591a0758910 con 0x5591a07f6950 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5591a07513b0 con 0x5591a07dfca0 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 10 client.0 _ll_register_callbacks cb 0x7f16300b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f163e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a09bd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f163e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a09bd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44280/0 (socket says 172.21.15.131:44280) 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f163e59c640 1 -- 172.21.15.131:0/1649596804 learned_addr learned my addr 172.21.15.131:0/1649596804 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:30.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f163ed9d640 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07f6950 0x5591a09bde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1635d9b640 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07dfca0 0x5591a09bd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f163e59c640 1 -- 172.21.15.131:0/1649596804 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07dfca0 msgr2=0x5591a09bd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f163e59c640 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07dfca0 0x5591a09bd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f163e59c640 1 -- 172.21.15.131:0/1649596804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07f6950 msgr2=0x5591a09bde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f163e59c640 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07f6950 0x5591a09bde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f163e59c640 1 -- 172.21.15.131:0/1649596804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a070fab0 con 0x5591a0406460 2024-09-17T02:45:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f163e59c640 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a09bd390 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f163001a430 tx=0x7f16300095a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f70c0d7294c7de server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f161d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f161d7fa640 1 -- 172.21.15.131:0/1649596804 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16300d7060 con 0x5591a0406460 2024-09-17T02:45:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f161d7fa640 1 -- 172.21.15.131:0/1649596804 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f16300054c0 con 0x5591a0406460 2024-09-17T02:45:30.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 -- 172.21.15.131:0/1649596804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16380187c0 con 0x5591a0406460 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f161d7fa640 1 -- 172.21.15.131:0/1649596804 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f16300b2040 con 0x5591a0406460 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 10 client.6552 fetch_fsmap learned FSMap version 46 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 -- 172.21.15.131:0/1649596804 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x5591a06ee070 con 0x5591a0406460 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f161d7fa640 1 -- 172.21.15.131:0/1649596804 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16380187c0 con 0x5591a0406460 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f161d7fa640 1 -- 172.21.15.131:0/1649596804 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f16300d1040 con 0x5591a0406460 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 10 client.6552 fetch_fsmap finished waiting for FSMap version 46 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 10 client.6552 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 20 client.6552 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.131:6833/655402879,v1:172.21.15.131:6835/655402879] conn(0x5591a03d9830 0x5591a0979870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 4 client.6552 mds_command: new command op to 6399 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 -- 172.21.15.131:0/1649596804 --> [v2:172.21.15.131:6833/655402879,v1:172.21.15.131:6835/655402879] -- command(tid 0: {"prefix": "status"}) -- 0x5591a0757540 con 0x5591a03d9830 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1635d9b640 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.131:6833/655402879,v1:172.21.15.131:6835/655402879] conn(0x5591a03d9830 0x5591a0979870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f161d7fa640 10 client.6552 ms_handle_connect on v2:172.21.15.131:6833/655402879 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1635d9b640 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.131:6833/655402879,v1:172.21.15.131:6835/655402879] conn(0x5591a03d9830 0x5591a0979870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=737f3f80004899b7 server_cookie=a8d3db36366cb9f9 in_seq=0 out_seq=0 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f161d7fa640 1 -- 172.21.15.131:0/1649596804 <== mds.0 v2:172.21.15.131:6833/655402879 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5591a0757540 con 0x5591a03d9830 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f161d7fa640 10 client.6552 handle_command_reply: tid=0 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 client.6552 shutdown 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 -- 172.21.15.131:0/1649596804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 msgr2=0x5591a09bd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a09bd390 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f163001a430 tx=0x7f16300095a0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 -- 172.21.15.131:0/1649596804 shutdown_connections 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.131:6833/655402879,v1:172.21.15.131:6835/655402879] conn(0x5591a03d9830 0x5591a0979870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5591a07f6950 0x5591a09bde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5591a07dfca0 0x5591a09bd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 --2- 172.21.15.131:0/1649596804 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5591a0406460 0x5591a09bd390 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f161d7fa640 0 client.6552 ms_handle_reset on v2:172.21.15.131:6833/655402879 2024-09-17T02:45:30.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 -- 172.21.15.131:0/1649596804 >> 172.21.15.131:0/1649596804 conn(0x5591a08ccd90 msgr2=0x5591a0945fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:30.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 -- 172.21.15.131:0/1649596804 shutdown_connections 2024-09-17T02:45:30.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 1 -- 172.21.15.131:0/1649596804 wait complete. 2024-09-17T02:45:30.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f164c1da000 20 client.6552 trim_cache size 0 max 16384 2024-09-17T02:45:30.712 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:45:30.712 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:45:30.713 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:45:30.713 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6399, 2024-09-17T02:45:30.713 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:45:30.713 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:45:30.713 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:45:30.713 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 10.584771012999999, 2024-09-17T02:45:30.713 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 46, 2024-09-17T02:45:30.713 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 79, 2024-09-17T02:45:30.713 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 79, 2024-09-17T02:45:30.713 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 15.239719416 2024-09-17T02:45:30.714 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:45:30.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1644f0f640 1 -- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1620073200 msgr2=0x7f1620075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1644f0f640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1620073200 0x7f1620075710 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f16300af000 tx=0x7f163001b040 comp rx=0 tx=0).stop 2024-09-17T02:45:30.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1644f0f640 1 -- 172.21.15.131:0/2485686006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16400cd060 msgr2=0x7f1640151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1644f0f640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16400cd060 0x7f1640151ae0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f162800d720 tx=0x7f162800dbe0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1644f0f640 1 -- 172.21.15.131:0/2485686006 shutdown_connections 2024-09-17T02:45:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1644f0f640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1620073200 0x7f1620075710 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1644f0f640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f164015c640 0x7f16401544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1644f0f640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f16400cd060 0x7f1640151ae0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1644f0f640 1 --2- 172.21.15.131:0/2485686006 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f16400b2220 0x7f1640155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.706+0000 7f1644f0f640 1 -- 172.21.15.131:0/2485686006 >> 172.21.15.131:0/2485686006 conn(0x7f16400b3f30 msgr2=0x7f16400b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:30.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.710+0000 7f1644f0f640 1 -- 172.21.15.131:0/2485686006 shutdown_connections 2024-09-17T02:45:30.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.710+0000 7f1644f0f640 1 -- 172.21.15.131:0/2485686006 wait complete. 2024-09-17T02:45:30.725 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:45:30.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.874+0000 7f045463e640 1 -- 172.21.15.131:0/1833022422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f044c155240 msgr2=0x7f044c073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.874+0000 7f045463e640 1 --2- 172.21.15.131:0/1833022422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f044c155240 0x7f044c073430 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f044c06a1c0 tx=0x7f044801cb30 comp rx=0 tx=0).stop 2024-09-17T02:45:30.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.874+0000 7f045463e640 1 -- 172.21.15.131:0/1833022422 shutdown_connections 2024-09-17T02:45:30.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.874+0000 7f045463e640 1 --2- 172.21.15.131:0/1833022422 >> v2:172.21.15.189:3300/0 conn(0x7f044c073970 0x7f044c06dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.874+0000 7f045463e640 1 --2- 172.21.15.131:0/1833022422 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f044c155240 0x7f044c073430 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.874+0000 7f045463e640 1 -- 172.21.15.131:0/1833022422 >> 172.21.15.131:0/1833022422 conn(0x7f044c06eea0 msgr2=0x7f044c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:30.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.874+0000 7f045463e640 1 -- 172.21.15.131:0/1833022422 shutdown_connections 2024-09-17T02:45:30.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.874+0000 7f045463e640 1 -- 172.21.15.131:0/1833022422 wait complete. 2024-09-17T02:45:30.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f045463e640 1 Processor -- start 2024-09-17T02:45:30.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f045463e640 1 -- start start 2024-09-17T02:45:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f045463e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f044c06e3e0 0x7f044c150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f045463e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f044c073970 0x7f044c15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f045463e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f044c155240 0x7f044c158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f045463e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f044c168560 con 0x7f044c06e3e0 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f045463e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f044c1683e0 con 0x7f044c073970 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f045463e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f044c1686e0 con 0x7f044c155240 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f04523da640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f044c06e3e0 0x7f044c150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f04523da640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f044c06e3e0 0x7f044c150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44298/0 (socket says 172.21.15.131:44298) 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f04523da640 1 -- 172.21.15.131:0/2598405695 learned_addr learned my addr 172.21.15.131:0/2598405695 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f0452bdb640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f044c155240 0x7f044c158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f0451bd9640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f044c073970 0x7f044c15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f04523da640 1 -- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f044c155240 msgr2=0x7f044c158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f04523da640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f044c155240 0x7f044c158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f04523da640 1 -- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f044c073970 msgr2=0x7f044c15aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:30.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f04523da640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f044c073970 0x7f044c15aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:30.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f04523da640 1 -- 172.21.15.131:0/2598405695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c157980 con 0x7f044c06e3e0 2024-09-17T02:45:30.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f04523da640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f044c06e3e0 0x7f044c150820 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f044801b1e0 tx=0x7f0448009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:30.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f0442ffd640 1 -- 172.21.15.131:0/2598405695 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04480b2040 con 0x7f044c06e3e0 2024-09-17T02:45:30.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f0442ffd640 1 -- 172.21.15.131:0/2598405695 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0448004d40 con 0x7f044c06e3e0 2024-09-17T02:45:30.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f045463e640 1 -- 172.21.15.131:0/2598405695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044c1576a0 con 0x7f044c06e3e0 2024-09-17T02:45:30.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f0442ffd640 1 -- 172.21.15.131:0/2598405695 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448004260 con 0x7f044c06e3e0 2024-09-17T02:45:30.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.878+0000 7f045463e640 1 -- 172.21.15.131:0/2598405695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044c156310 con 0x7f044c06e3e0 2024-09-17T02:45:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.882+0000 7f045463e640 1 -- 172.21.15.131:0/2598405695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f044c06dea0 con 0x7f044c06e3e0 2024-09-17T02:45:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.882+0000 7f0442ffd640 1 -- 172.21.15.131:0/2598405695 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f04480be060 con 0x7f044c06e3e0 2024-09-17T02:45:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.882+0000 7f0442ffd640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0430073200 0x7f0430075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.882+0000 7f0442ffd640 1 -- 172.21.15.131:0/2598405695 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f044812e570 con 0x7f044c06e3e0 2024-09-17T02:45:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.882+0000 7f0451bd9640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0430073200 0x7f0430075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:30.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.882+0000 7f0451bd9640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0430073200 0x7f0430075710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f043c0058d0 tx=0x7f043c005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:30.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:30.886+0000 7f0442ffd640 1 -- 172.21.15.131:0/2598405695 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f044c06e7f0 con 0x7f044c06e3e0 2024-09-17T02:45:31.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.086+0000 7f045463e640 1 -- 172.21.15.131:0/2598405695 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f044c06e9d0 con 0x7f044c06e3e0 2024-09-17T02:45:31.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.086+0000 7f0442ffd640 1 -- 172.21.15.131:0/2598405695 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f04480f94e0 con 0x7f044c06e3e0 2024-09-17T02:45:31.085 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:45:31.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 -- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0430073200 msgr2=0x7f0430075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:31.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0430073200 0x7f0430075710 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f043c0058d0 tx=0x7f043c005860 comp rx=0 tx=0).stop 2024-09-17T02:45:31.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 -- 172.21.15.131:0/2598405695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f044c06e3e0 msgr2=0x7f044c150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:31.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f044c06e3e0 0x7f044c150820 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f044801b1e0 tx=0x7f0448009340 comp rx=0 tx=0).stop 2024-09-17T02:45:31.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 -- 172.21.15.131:0/2598405695 shutdown_connections 2024-09-17T02:45:31.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0430073200 0x7f0430075710 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:31.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f044c155240 0x7f044c158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:31.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f044c073970 0x7f044c15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:31.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 --2- 172.21.15.131:0/2598405695 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f044c06e3e0 0x7f044c150820 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:31.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 -- 172.21.15.131:0/2598405695 >> 172.21.15.131:0/2598405695 conn(0x7f044c06eea0 msgr2=0x7f044c06d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:31.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 -- 172.21.15.131:0/2598405695 shutdown_connections 2024-09-17T02:45:31.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31.090+0000 7f045463e640 1 -- 172.21.15.131:0/2598405695 wait complete. 2024-09-17T02:45:31.105 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:45:31.106 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:45:31.106 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:45:31.106 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:45:31.106 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:31.106 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:45:31.106 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:45:31.106 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:45:31.106 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:45:31.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:31.106 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:31.113 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42175sec preferred_lft 42175sec 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:31.114 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:45:31.115 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:31.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:31.116 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:45:31.166 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:45:31.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:31.166 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:45:31.220 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:45:31.220 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:45:31.221 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:45:31.221 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:45:31.221 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:45:31.221 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:45:31.221 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:45:31.221 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:45:31.221 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:45:31.221 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:45:31.221 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:45:31.222 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:45:31.222 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:45:31.222 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:45:31.222 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:45:31.222 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:45:31.222 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:45:31.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:45:31.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:45:31.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:31.339 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:45:31.392 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:45:31.392 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:45:31.393 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:45:31.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:31.393 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:45:31.447 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:45:31.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:45:31.447 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:31.495 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:45:31.495 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:45:31.495 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:31.543 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:45:31.601 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:45:31.648 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:45:31.648 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:45:31.648 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:45:31.648 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:45:31.648 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:45:31.649 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:31.650 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:31.651 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:31.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:31.652 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:45:31.699 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:45:31.700 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:45:31.744 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:45:31.759 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:45:31.807 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:45:31.807 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:45:31.807 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:45:31.807 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:45:31.807 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:45:31.808 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:45:31.808 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:45:31.808 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:45:31.808 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:45:31.808 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:45:31.808 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:45:31.808 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:45:31.808 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:45:31.808 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:45:31.808 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:45:31.808 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:45:31.809 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:31.810 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:31.810 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:31.810 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:31.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:31.811 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:45:31.850 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:45:31.850+0000 7feebd787580 -1 init, newargv = 0x7feea0004730 newargc=15 2024-09-17T02:45:31.851 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[43806]: starting ceph client 2024-09-17T02:45:31.855 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[43806]: starting fuse 2024-09-17T02:45:32.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:45:32.973 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:45:33.021 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:45:33.021 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:45:33.022 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:45:33.023 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:45:33.024 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:45:33.024 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:45:33.024 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:45:33.024 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:33.024 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:33.024 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:33.024 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:33.024 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:33.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:33.025 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:45:33.071 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:45:33.071 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:45:33.072 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:45:33.073 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:45:33.196 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.43806.asok 2024-09-17T02:45:33.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:45:33.208 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43806.asok 2024-09-17T02:45:33.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:33.208 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43806.asok status 2024-09-17T02:45:33.391 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:45:33.391 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:45:33.391 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:45:33.391 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:45:33.391 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:45:33.391 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:45:33.391 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:45:33.391 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "43806", 2024-09-17T02:45:33.391 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6567, 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 6567 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:45:33.392 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3357475128 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3357475128 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.6567 192.168.0.1:0/3357475128", 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/3357475128", 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 46, 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 79, 2024-09-17T02:45:33.393 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:45:33.394 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:45:33.394 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:45:33.394 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:45:33.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:33.407 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:45:33.407 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:33.413 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:45:33.413 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:33.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:33.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:33.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:33.468 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-09-17T02:45:33.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:33.524 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-09-17T02:45:33.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:33.576 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-09-17T02:45:33.626 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:45:33.626 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:45:33.626 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:45:33.626 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:45:33.626 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-09-17T02:45:33.627 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:45:33.736 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627776 2024-09-17T02:45:33.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:45:33.744 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:45:33.744 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:33.791 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:45:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.002+0000 7fe1a210a640 1 -- 172.21.15.131:0/1565232834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe19c06e2c0 msgr2=0x7fe18400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.002+0000 7fe1a210a640 1 --2- 172.21.15.131:0/1565232834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe19c06e2c0 0x7fe18400b9d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0016f0 tx=0x7fe18c01c7b0 comp rx=0 tx=0).stop 2024-09-17T02:45:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.002+0000 7fe19bfff640 1 -- 172.21.15.131:0/1565232834 reap_dead start 2024-09-17T02:45:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.002+0000 7fe1a210a640 1 -- 172.21.15.131:0/1565232834 shutdown_connections 2024-09-17T02:45:33.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.002+0000 7fe1a210a640 1 -- 172.21.15.131:0/1565232834 >> 172.21.15.131:0/1565232834 conn(0x7fe19c06eea0 msgr2=0x7fe19c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.002+0000 7fe1a210a640 1 -- 172.21.15.131:0/1565232834 shutdown_connections 2024-09-17T02:45:34.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.002+0000 7fe1a210a640 1 -- 172.21.15.131:0/1565232834 wait complete. 2024-09-17T02:45:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.002+0000 7fe1a210a640 1 Processor -- start 2024-09-17T02:45:34.001 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.002+0000 7fe1a210a640 1 -- start start 2024-09-17T02:45:34.003 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe1a210a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe19c15b760 0x7fe19c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe1a210a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c151560 0x7fe19c150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe1a210a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe19c15a430 0x7fe19c15a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe1a210a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c169370 con 0x7fe19c15b760 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe1a210a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c1691f0 con 0x7fe19c151560 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe1a210a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe19c1694f0 con 0x7fe19c15a430 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe19c15b760 0x7fe19c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe19c15b760 0x7fe19c0b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46766/0 (socket says 172.21.15.131:46766) 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19b7fe640 1 -- 172.21.15.131:0/668260472 learned_addr learned my addr 172.21.15.131:0/668260472 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19affd640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c151560 0x7fe19c150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19bfff640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe19c15a430 0x7fe19c15a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19b7fe640 1 -- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe19c15a430 msgr2=0x7fe19c15a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:34.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19b7fe640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe19c15a430 0x7fe19c15a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19b7fe640 1 -- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c151560 msgr2=0x7fe19c150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19b7fe640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c151560 0x7fe19c150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19b7fe640 1 -- 172.21.15.131:0/668260472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c15ad80 con 0x7fe19c15b760 2024-09-17T02:45:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19bfff640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe19c15a430 0x7fe19c15a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:45:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19b7fe640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe19c15b760 0x7fe19c0b3690 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe194005f70 tx=0x7fe194009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe183fff640 1 -- 172.21.15.131:0/668260472 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19401e070 con 0x7fe19c15b760 2024-09-17T02:45:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe1a210a640 1 -- 172.21.15.131:0/668260472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c152fc0 con 0x7fe19c15b760 2024-09-17T02:45:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe183fff640 1 -- 172.21.15.131:0/668260472 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe19400fd90 con 0x7fe19c15b760 2024-09-17T02:45:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe183fff640 1 -- 172.21.15.131:0/668260472 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194019670 con 0x7fe19c15b760 2024-09-17T02:45:34.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe1a210a640 1 -- 172.21.15.131:0/668260472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c153860 con 0x7fe19c15b760 2024-09-17T02:45:34.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe1a210a640 1 -- 172.21.15.131:0/668260472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c06e2c0 con 0x7fe19c15b760 2024-09-17T02:45:34.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe183fff640 1 -- 172.21.15.131:0/668260472 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe194021070 con 0x7fe19c15b760 2024-09-17T02:45:34.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe183fff640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe188073200 0x7fe188075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:34.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe183fff640 1 -- 172.21.15.131:0/668260472 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7fe1940954b0 con 0x7fe19c15b760 2024-09-17T02:45:34.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.006+0000 7fe19affd640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe188073200 0x7fe188075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:34.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.010+0000 7fe19affd640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe188073200 0x7fe188075710 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe190003a00 tx=0x7fe190001260 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:34.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.010+0000 7fe183fff640 1 -- 172.21.15.131:0/668260472 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe19c15ad80 con 0x7fe19c15b760 2024-09-17T02:45:34.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 -- 172.21.15.131:0/668260472 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe19c0c65a0 con 0x7fe19c15b760 2024-09-17T02:45:34.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe183fff640 1 -- 172.21.15.131:0/668260472 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) ==== 80+0+0 (secure 0 0 0) 0x7fe19405fe10 con 0x7fe19c15b760 2024-09-17T02:45:34.192 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:45:34.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 -- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe188073200 msgr2=0x7fe188075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:34.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe188073200 0x7fe188075710 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe190003a00 tx=0x7fe190001260 comp rx=0 tx=0).stop 2024-09-17T02:45:34.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 -- 172.21.15.131:0/668260472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe19c15b760 msgr2=0x7fe19c0b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:34.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe19c15b760 0x7fe19c0b3690 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe194005f70 tx=0x7fe194009f70 comp rx=0 tx=0).stop 2024-09-17T02:45:34.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 -- 172.21.15.131:0/668260472 shutdown_connections 2024-09-17T02:45:34.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe188073200 0x7fe188075710 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe19c15a430 0x7fe19c15a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe19c151560 0x7fe19c150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 --2- 172.21.15.131:0/668260472 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe19c15b760 0x7fe19c0b3690 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 -- 172.21.15.131:0/668260472 >> 172.21.15.131:0/668260472 conn(0x7fe19c06eea0 msgr2=0x7fe19c154360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:34.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 -- 172.21.15.131:0/668260472 shutdown_connections 2024-09-17T02:45:34.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.194+0000 7fe1a210a640 1 -- 172.21.15.131:0/668260472 wait complete. 2024-09-17T02:45:34.213 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:45:34.213 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:34.220 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:45:34.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.426+0000 7f4b5093d640 1 -- 172.21.15.131:0/3847096274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b38004010 msgr2=0x7f4b38004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:34.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.426+0000 7f4b5093d640 1 --2- 172.21.15.131:0/3847096274 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b38004010 0x7f4b38004810 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4b3400c260 tx=0x7f4b3401f6c0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.426+0000 7f4b4ad9d640 1 -- 172.21.15.131:0/3847096274 reap_dead start 2024-09-17T02:45:34.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.426+0000 7f4b5093d640 1 -- 172.21.15.131:0/3847096274 shutdown_connections 2024-09-17T02:45:34.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.426+0000 7f4b5093d640 1 -- 172.21.15.131:0/3847096274 >> 172.21.15.131:0/3847096274 conn(0x7f4b4c06eea0 msgr2=0x7f4b4c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:34.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.426+0000 7f4b5093d640 1 -- 172.21.15.131:0/3847096274 shutdown_connections 2024-09-17T02:45:34.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.426+0000 7f4b5093d640 1 -- 172.21.15.131:0/3847096274 wait complete. 2024-09-17T02:45:34.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b5093d640 1 Processor -- start 2024-09-17T02:45:34.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b5093d640 1 -- start start 2024-09-17T02:45:34.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b5093d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b38004010 0x7f4b4c168b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:34.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b5093d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b4c0c7770 0x7f4b4c154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:34.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b5093d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b4c154780 0x7f4b4c1f8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:34.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b5093d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c06fd50 con 0x7f4b4c154780 2024-09-17T02:45:34.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b5093d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c06fbd0 con 0x7f4b4c0c7770 2024-09-17T02:45:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b5093d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4b4c06fed0 con 0x7f4b38004010 2024-09-17T02:45:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b4ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b4c154780 0x7f4b4c1f8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b4ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b4c154780 0x7f4b4c1f8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46790/0 (socket says 172.21.15.131:46790) 2024-09-17T02:45:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b4ad9d640 1 -- 172.21.15.131:0/3310124156 learned_addr learned my addr 172.21.15.131:0/3310124156 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b4a59c640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b38004010 0x7f4b4c168b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b43fff640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b4c0c7770 0x7f4b4c154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b4ad9d640 1 -- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b38004010 msgr2=0x7f4b4c168b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b4ad9d640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b38004010 0x7f4b4c168b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b4ad9d640 1 -- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b4c0c7770 msgr2=0x7f4b4c154240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:34.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b4ad9d640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b4c0c7770 0x7f4b4c154240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b4ad9d640 1 -- 172.21.15.131:0/3310124156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c154b90 con 0x7f4b4c154780 2024-09-17T02:45:34.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b43fff640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b4c0c7770 0x7f4b4c154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:34.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b4a59c640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b38004010 0x7f4b4c168b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:34.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b4ad9d640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b4c154780 0x7f4b4c1f8b70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c072ad0 tx=0x7f4b4400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:34.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b42ffd640 1 -- 172.21.15.131:0/3310124156 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4400f040 con 0x7f4b4c154780 2024-09-17T02:45:34.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b42ffd640 1 -- 172.21.15.131:0/3310124156 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b44018070 con 0x7f4b4c154780 2024-09-17T02:45:34.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b5093d640 1 -- 172.21.15.131:0/3310124156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c15af50 con 0x7f4b4c154780 2024-09-17T02:45:34.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b42ffd640 1 -- 172.21.15.131:0/3310124156 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4400c930 con 0x7f4b4c154780 2024-09-17T02:45:34.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b5093d640 1 -- 172.21.15.131:0/3310124156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c15a470 con 0x7f4b4c154780 2024-09-17T02:45:34.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.430+0000 7f4b5093d640 1 -- 172.21.15.131:0/3310124156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b4c0cab60 con 0x7f4b4c154780 2024-09-17T02:45:34.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.434+0000 7f4b42ffd640 1 -- 172.21.15.131:0/3310124156 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4b4400cad0 con 0x7f4b4c154780 2024-09-17T02:45:34.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.434+0000 7f4b42ffd640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b2c073200 0x7f4b2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:34.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.434+0000 7f4b42ffd640 1 -- 172.21.15.131:0/3310124156 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f4b44093f50 con 0x7f4b4c154780 2024-09-17T02:45:34.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.434+0000 7f4b4a59c640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b2c073200 0x7f4b2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:34.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.434+0000 7f4b4a59c640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b2c073200 0x7f4b2c075710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009ac0 tx=0x7f4b3c005750 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:34.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.438+0000 7f4b42ffd640 1 -- 172.21.15.131:0/3310124156 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4b4c154b90 con 0x7f4b4c154780 2024-09-17T02:45:34.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.622+0000 7f4b5093d640 1 -- 172.21.15.131:0/3310124156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4b4c169290 con 0x7f4b4c154780 2024-09-17T02:45:34.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.622+0000 7f4b42ffd640 1 -- 172.21.15.131:0/3310124156 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) ==== 80+0+0 (secure 0 0 0) 0x7f4b4405e820 con 0x7f4b4c154780 2024-09-17T02:45:34.620 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:45:34.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 -- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b2c073200 msgr2=0x7f4b2c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:34.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b2c073200 0x7f4b2c075710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009ac0 tx=0x7f4b3c005750 comp rx=0 tx=0).stop 2024-09-17T02:45:34.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 -- 172.21.15.131:0/3310124156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b4c154780 msgr2=0x7f4b4c1f8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:34.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b4c154780 0x7f4b4c1f8b70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c072ad0 tx=0x7f4b4400ec20 comp rx=0 tx=0).stop 2024-09-17T02:45:34.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 -- 172.21.15.131:0/3310124156 shutdown_connections 2024-09-17T02:45:34.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b2c073200 0x7f4b2c075710 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b4c154780 0x7f4b4c1f8b70 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b4c0c7770 0x7f4b4c154240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 --2- 172.21.15.131:0/3310124156 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b38004010 0x7f4b4c168b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:34.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 -- 172.21.15.131:0/3310124156 >> 172.21.15.131:0/3310124156 conn(0x7f4b4c06eea0 msgr2=0x7f4b4c15b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:34.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 -- 172.21.15.131:0/3310124156 shutdown_connections 2024-09-17T02:45:34.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:34.626+0000 7f4b5093d640 1 -- 172.21.15.131:0/3310124156 wait complete. 2024-09-17T02:45:34.642 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:45:34.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:34.643 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:34.653 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[43806]: fuse finished with error 0 and tester_r 0 2024-09-17T02:45:34.655 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:45:40.807 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:45:40.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:40.807 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:45:40.815 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:41.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.026+0000 7f4d36ba6640 1 -- 172.21.15.131:0/2335942404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d300c53a0 msgr2=0x7f4d300cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.026+0000 7f4d36ba6640 1 --2- 172.21.15.131:0/2335942404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d300c53a0 0x7f4d300cf010 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009bc0 tx=0x7f4d2801cb10 comp rx=0 tx=0).stop 2024-09-17T02:45:41.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.026+0000 7f4d36ba6640 1 -- 172.21.15.131:0/2335942404 shutdown_connections 2024-09-17T02:45:41.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.026+0000 7f4d36ba6640 1 --2- 172.21.15.131:0/2335942404 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d300c53a0 0x7f4d300cf010 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.026+0000 7f4d36ba6640 1 -- 172.21.15.131:0/2335942404 >> 172.21.15.131:0/2335942404 conn(0x7f4d3006eea0 msgr2=0x7f4d3006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:41.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.026+0000 7f4d36ba6640 1 -- 172.21.15.131:0/2335942404 shutdown_connections 2024-09-17T02:45:41.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.026+0000 7f4d36ba6640 1 -- 172.21.15.131:0/2335942404 wait complete. 2024-09-17T02:45:41.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d36ba6640 1 Processor -- start 2024-09-17T02:45:41.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d36ba6640 1 -- start start 2024-09-17T02:45:41.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d36ba6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d3006e2c0 0x7f4d3013e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d36ba6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d300c53a0 0x7f4d300c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d36ba6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4d300cec00 0x7f4d300c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d36ba6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3016ae80 con 0x7f4d3006e2c0 2024-09-17T02:45:41.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d36ba6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3016ad00 con 0x7f4d300c53a0 2024-09-17T02:45:41.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d36ba6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4d3016b000 con 0x7f4d300cec00 2024-09-17T02:45:41.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d34942640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d3006e2c0 0x7f4d3013e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d34942640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d3006e2c0 0x7f4d3013e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46812/0 (socket says 172.21.15.131:46812) 2024-09-17T02:45:41.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d34942640 1 -- 172.21.15.131:0/3884251414 learned_addr learned my addr 172.21.15.131:0/3884251414 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:41.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d34942640 1 -- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4d300cec00 msgr2=0x7f4d300c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d34942640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4d300cec00 0x7f4d300c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d34942640 1 -- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d300c53a0 msgr2=0x7f4d300c7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d2ffff640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d300c53a0 0x7f4d300c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d34942640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d300c53a0 0x7f4d300c7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d34942640 1 -- 172.21.15.131:0/3884251414 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3015b2b0 con 0x7f4d3006e2c0 2024-09-17T02:45:41.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d2ffff640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d300c53a0 0x7f4d300c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:41.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d34942640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d3006e2c0 0x7f4d3013e160 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800b440 tx=0x7f4d1800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:41.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d2d7fa640 1 -- 172.21.15.131:0/3884251414 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18015020 con 0x7f4d3006e2c0 2024-09-17T02:45:41.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d2d7fa640 1 -- 172.21.15.131:0/3884251414 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d18011980 con 0x7f4d3006e2c0 2024-09-17T02:45:41.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d2d7fa640 1 -- 172.21.15.131:0/3884251414 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d18011dd0 con 0x7f4d3006e2c0 2024-09-17T02:45:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d36ba6640 1 -- 172.21.15.131:0/3884251414 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3015b7b0 con 0x7f4d3006e2c0 2024-09-17T02:45:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.030+0000 7f4d36ba6640 1 -- 172.21.15.131:0/3884251414 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d3015bd70 con 0x7f4d3006e2c0 2024-09-17T02:45:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.034+0000 7f4d36ba6640 1 -- 172.21.15.131:0/3884251414 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d300cf010 con 0x7f4d3006e2c0 2024-09-17T02:45:41.031 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.034+0000 7f4d2d7fa640 1 -- 172.21.15.131:0/3884251414 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4d18018020 con 0x7f4d3006e2c0 2024-09-17T02:45:41.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.034+0000 7f4d2d7fa640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4d10073200 0x7f4d10075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.034+0000 7f4d2d7fa640 1 -- 172.21.15.131:0/3884251414 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f4d18093ed0 con 0x7f4d3006e2c0 2024-09-17T02:45:41.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.034+0000 7f4d2ffff640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4d10073200 0x7f4d10075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.034+0000 7f4d2ffff640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4d10073200 0x7f4d10075710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4d30154c40 tx=0x7f4d20009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:41.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.038+0000 7f4d2d7fa640 1 -- 172.21.15.131:0/3884251414 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4d1801f070 con 0x7f4d3006e2c0 2024-09-17T02:45:41.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.238+0000 7f4d36ba6640 1 -- 172.21.15.131:0/3884251414 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d30073310 con 0x7f4d3006e2c0 2024-09-17T02:45:41.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.238+0000 7f4d2d7fa640 1 -- 172.21.15.131:0/3884251414 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5228 (secure 0 0 0) 0x7f4d1800d040 con 0x7f4d3006e2c0 2024-09-17T02:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:41.237 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":46,"btime":"2024-09-17T02:45:23:141401+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:45:22.552270+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6399},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6399":{"gid":6399,"name":"c","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/655402879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":655402879},{"type":"v1","addr":"172.21.15.131:6835","nonce":655402879}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6399,"qdb_cluster":[6399]},"id":5}]} 2024-09-17T02:45:41.237 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 46 2024-09-17T02:45:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 -- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4d10073200 msgr2=0x7f4d10075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4d10073200 0x7f4d10075710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4d30154c40 tx=0x7f4d20009040 comp rx=0 tx=0).stop 2024-09-17T02:45:41.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 -- 172.21.15.131:0/3884251414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d3006e2c0 msgr2=0x7f4d3013e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d3006e2c0 0x7f4d3013e160 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f4d1800b440 tx=0x7f4d1800b900 comp rx=0 tx=0).stop 2024-09-17T02:45:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 -- 172.21.15.131:0/3884251414 shutdown_connections 2024-09-17T02:45:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4d10073200 0x7f4d10075710 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4d300cec00 0x7f4d300c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4d300c53a0 0x7f4d300c7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 --2- 172.21.15.131:0/3884251414 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4d3006e2c0 0x7f4d3013e160 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 -- 172.21.15.131:0/3884251414 >> 172.21.15.131:0/3884251414 conn(0x7f4d3006eea0 msgr2=0x7f4d300b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 -- 172.21.15.131:0/3884251414 shutdown_connections 2024-09-17T02:45:41.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.242+0000 7f4d36ba6640 1 -- 172.21.15.131:0/3884251414 wait complete. 2024-09-17T02:45:41.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-17T02:45:41.423 INFO:teuthology.orchestra.run.smithi131.stdout:{"message":"","return_code":0} 2024-09-17T02:45:41.424 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T02:45:41.424 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:45:41.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.582+0000 7ff0bb861640 1 -- 172.21.15.131:0/1964116629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0b40cec00 msgr2=0x7ff0b40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.582+0000 7ff0bb861640 1 --2- 172.21.15.131:0/1964116629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0b40cec00 0x7ff0b40cf010 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4071d30 tx=0x7ff0a401c770 comp rx=0 tx=0).stop 2024-09-17T02:45:41.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.582+0000 7ff0bb861640 1 -- 172.21.15.131:0/1964116629 shutdown_connections 2024-09-17T02:45:41.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.582+0000 7ff0bb861640 1 --2- 172.21.15.131:0/1964116629 >> v2:172.21.15.189:3300/0 conn(0x7ff0b40c4c90 0x7ff0b40c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.582+0000 7ff0bb861640 1 --2- 172.21.15.131:0/1964116629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0b40cec00 0x7ff0b40cf010 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.582+0000 7ff0bb861640 1 -- 172.21.15.131:0/1964116629 >> 172.21.15.131:0/1964116629 conn(0x7ff0b406eea0 msgr2=0x7ff0b406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:41.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.582+0000 7ff0bb861640 1 -- 172.21.15.131:0/1964116629 shutdown_connections 2024-09-17T02:45:41.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.582+0000 7ff0bb861640 1 -- 172.21.15.131:0/1964116629 wait complete. 2024-09-17T02:45:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0bb861640 1 Processor -- start 2024-09-17T02:45:41.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0bb861640 1 -- start start 2024-09-17T02:45:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0bb861640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0b406e2c0 0x7ff0b4149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0bb861640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0b40c4c90 0x7ff0b40c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0bb861640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0b40cec00 0x7ff0b41509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0bb861640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4169c60 con 0x7ff0b40cec00 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0bb861640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b4169ae0 con 0x7ff0b406e2c0 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0bb861640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b4169de0 con 0x7ff0b40c4c90 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0b9dfe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0b40cec00 0x7ff0b41509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0b9dfe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0b40cec00 0x7ff0b41509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38372/0 (socket says 172.21.15.131:38372) 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0b9dfe640 1 -- 172.21.15.131:0/3662947908 learned_addr learned my addr 172.21.15.131:0/3662947908 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0b8dfc640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0b40c4c90 0x7ff0b40c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0b95fd640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0b406e2c0 0x7ff0b4149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0b9dfe640 1 -- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0b40c4c90 msgr2=0x7ff0b40c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0b9dfe640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0b40c4c90 0x7ff0b40c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0b9dfe640 1 -- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0b406e2c0 msgr2=0x7ff0b4149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0b9dfe640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0b406e2c0 0x7ff0b4149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0b9dfe640 1 -- 172.21.15.131:0/3662947908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b41fab20 con 0x7ff0b40cec00 2024-09-17T02:45:41.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0b9dfe640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0b40cec00 0x7ff0b41509a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00ca90 tx=0x7ff0ac00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:41.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0a9ffb640 1 -- 172.21.15.131:0/3662947908 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac007df0 con 0x7ff0b40cec00 2024-09-17T02:45:41.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0a9ffb640 1 -- 172.21.15.131:0/3662947908 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0ac004b30 con 0x7ff0b40cec00 2024-09-17T02:45:41.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0bb861640 1 -- 172.21.15.131:0/3662947908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b414a730 con 0x7ff0b40cec00 2024-09-17T02:45:41.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0a9ffb640 1 -- 172.21.15.131:0/3662947908 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac005020 con 0x7ff0b40cec00 2024-09-17T02:45:41.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0bb861640 1 -- 172.21.15.131:0/3662947908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b41fa280 con 0x7ff0b40cec00 2024-09-17T02:45:41.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.586+0000 7ff0bb861640 1 -- 172.21.15.131:0/3662947908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b414a900 con 0x7ff0b40cec00 2024-09-17T02:45:41.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.590+0000 7ff0a9ffb640 1 -- 172.21.15.131:0/3662947908 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff0ac0051c0 con 0x7ff0b40cec00 2024-09-17T02:45:41.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.590+0000 7ff0a9ffb640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff098073230 0x7ff098075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.590+0000 7ff0a9ffb640 1 -- 172.21.15.131:0/3662947908 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7ff0ac094f10 con 0x7ff0b40cec00 2024-09-17T02:45:41.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.590+0000 7ff0b95fd640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff098073230 0x7ff098075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.590+0000 7ff0b95fd640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff098073230 0x7ff098075740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00015f0 tx=0x7ff0b0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:41.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.594+0000 7ff0a9ffb640 1 -- 172.21.15.131:0/3662947908 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff0ac014030 con 0x7ff0b40cec00 2024-09-17T02:45:41.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.794+0000 7ff0bb861640 1 -- 172.21.15.131:0/3662947908 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff0b40c50a0 con 0x7ff0b40cec00 2024-09-17T02:45:41.791 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:45:41.794+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:45:41.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.794+0000 7ff0a9ffb640 1 -- 172.21.15.131:0/3662947908 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7ff0ac05fe80 con 0x7ff0b40cec00 2024-09-17T02:45:41.794 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:45:41.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 -- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff098073230 msgr2=0x7ff098075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff098073230 0x7ff098075740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff0b00015f0 tx=0x7ff0b0009040 comp rx=0 tx=0).stop 2024-09-17T02:45:41.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 -- 172.21.15.131:0/3662947908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0b40cec00 msgr2=0x7ff0b41509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0b40cec00 0x7ff0b41509a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00ca90 tx=0x7ff0ac00cf50 comp rx=0 tx=0).stop 2024-09-17T02:45:41.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 -- 172.21.15.131:0/3662947908 shutdown_connections 2024-09-17T02:45:41.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff098073230 0x7ff098075740 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0b40cec00 0x7ff0b41509a0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0b40c4c90 0x7ff0b40c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 --2- 172.21.15.131:0/3662947908 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0b406e2c0 0x7ff0b4149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 -- 172.21.15.131:0/3662947908 >> 172.21.15.131:0/3662947908 conn(0x7ff0b406eea0 msgr2=0x7ff0b4152940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:41.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 -- 172.21.15.131:0/3662947908 shutdown_connections 2024-09-17T02:45:41.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.798+0000 7ff0bb861640 1 -- 172.21.15.131:0/3662947908 wait complete. 2024-09-17T02:45:41.805 INFO:tasks.ceph.mds.c.smithi131.stderr: -4357> 2024-09-17T02:45:21.046+0000 7fbb9b47c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:45:41.817 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T02:45:41.877 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:45:41.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe79da32080 1 Processor -- start 2024-09-17T02:45:41.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe79da32080 1 -- start start 2024-09-17T02:45:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe79da32080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e1a7fb9f0 0x560e1a815820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe79da32080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fbe00 0x560e1a7ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe79da32080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e1a8a20d0 0x560e1a8a2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe79da32080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560e1a71da00 con 0x560e1a7fbe00 2024-09-17T02:45:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe79da32080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e1a77bf40 con 0x560e1a7fb9f0 2024-09-17T02:45:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe79da32080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x560e1a7ff4b0 con 0x560e1a8a20d0 2024-09-17T02:45:41.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe796d08640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fbe00 0x560e1a7ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe796d08640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fbe00 0x560e1a7ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38410/0 (socket says 172.21.15.131:38410) 2024-09-17T02:45:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe796d08640 1 -- 172.21.15.131:0/1899430145 learned_addr learned my addr 172.21.15.131:0/1899430145 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe796d08640 1 -- 172.21.15.131:0/1899430145 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e1a8a20d0 msgr2=0x560e1a8a2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe796d08640 1 --2- 172.21.15.131:0/1899430145 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e1a8a20d0 0x560e1a8a2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe796d08640 1 -- 172.21.15.131:0/1899430145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e1a7fb9f0 msgr2=0x560e1a815820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe796d08640 1 --2- 172.21.15.131:0/1899430145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e1a7fb9f0 0x560e1a815820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe796d08640 1 -- 172.21.15.131:0/1899430145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1a6f6c10 con 0x560e1a7fbe00 2024-09-17T02:45:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe796d08640 1 --2- 172.21.15.131:0/1899430145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fbe00 0x560e1a7ccd40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00a3d0 tx=0x7fe78c00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775fcf7608dd1ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe795d06640 1 -- 172.21.15.131:0/1899430145 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c010070 con 0x560e1a7fbe00 2024-09-17T02:45:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe795d06640 1 -- 172.21.15.131:0/1899430145 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe78c00c9f0 con 0x560e1a7fbe00 2024-09-17T02:45:41.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.882+0000 7fe795d06640 1 -- 172.21.15.131:0/1899430145 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00ce40 con 0x560e1a7fbe00 2024-09-17T02:45:41.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 -- 172.21.15.131:0/1899430145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fbe00 msgr2=0x560e1a7ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 --2- 172.21.15.131:0/1899430145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fbe00 0x560e1a7ccd40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00a3d0 tx=0x7fe78c00a890 comp rx=0 tx=0).stop 2024-09-17T02:45:41.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 -- 172.21.15.131:0/1899430145 shutdown_connections 2024-09-17T02:45:41.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 --2- 172.21.15.131:0/1899430145 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e1a8a20d0 0x560e1a8a2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 --2- 172.21.15.131:0/1899430145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fbe00 0x560e1a7ccd40 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 --2- 172.21.15.131:0/1899430145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e1a7fb9f0 0x560e1a815820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 -- 172.21.15.131:0/1899430145 >> 172.21.15.131:0/1899430145 conn(0x560e1a7bbd00 msgr2=0x560e1a8a0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:41.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 -- 172.21.15.131:0/1899430145 shutdown_connections 2024-09-17T02:45:41.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 -- 172.21.15.131:0/1899430145 wait complete. 2024-09-17T02:45:41.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 Processor -- start 2024-09-17T02:45:41.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 -- start start 2024-09-17T02:45:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fb9f0 0x560e1a8aa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e1a7fbe00 0x560e1a814ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e1a8a20d0 0x560e1a80f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560e1a8a7ef0 con 0x560e1a7fb9f0 2024-09-17T02:45:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e1a8a4a30 con 0x560e1a8a20d0 2024-09-17T02:45:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe79da32080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x560e1a8a4cd0 con 0x560e1a7fbe00 2024-09-17T02:45:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe797509640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fb9f0 0x560e1a8aa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe797509640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fb9f0 0x560e1a8aa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38416/0 (socket says 172.21.15.131:38416) 2024-09-17T02:45:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe797509640 1 -- 172.21.15.131:0/881426988 learned_addr learned my addr 172.21.15.131:0/881426988 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe796d08640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e1a7fbe00 0x560e1a814ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe797d0a640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e1a8a20d0 0x560e1a80f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe797509640 1 -- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e1a7fbe00 msgr2=0x560e1a814ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe797509640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e1a7fbe00 0x560e1a814ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe797509640 1 -- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e1a8a20d0 msgr2=0x560e1a80f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe797509640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e1a8a20d0 0x560e1a80f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe797509640 1 -- 172.21.15.131:0/881426988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1a80ff20 con 0x560e1a7fb9f0 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe796d08640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e1a7fbe00 0x560e1a814ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe797d0a640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e1a8a20d0 0x560e1a80f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe797509640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fb9f0 0x560e1a8aa880 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe788019950 tx=0x7fe7880ab810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe794d04640 1 -- 172.21.15.131:0/881426988 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7880aba00 con 0x560e1a7fb9f0 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.886+0000 7fe794d04640 1 -- 172.21.15.131:0/881426988 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7880b9070 con 0x560e1a7fb9f0 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe794d04640 1 -- 172.21.15.131:0/881426988 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7880b4400 con 0x560e1a7fb9f0 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe79da32080 1 -- 172.21.15.131:0/881426988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560e1a8ab3b0 con 0x560e1a7fb9f0 2024-09-17T02:45:41.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe79da32080 1 -- 172.21.15.131:0/881426988 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1a8ab8f0 con 0x560e1a7fb9f0 2024-09-17T02:45:41.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe794d04640 1 -- 172.21.15.131:0/881426988 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe7880b3070 con 0x560e1a7fb9f0 2024-09-17T02:45:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe794d04640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe784071be0 0x7fe7840740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe794d04640 1 -- 172.21.15.131:0/881426988 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fe78812d7b0 con 0x560e1a7fb9f0 2024-09-17T02:45:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe79da32080 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x560e1a8abd90 0x560e1a8ac1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe79da32080 1 -- 172.21.15.131:0/881426988 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80) -- 0x560e1a7ccd40 con 0x560e1a8abd90 2024-09-17T02:45:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe796d08640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe784071be0 0x7fe7840740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe797d0a640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x560e1a8abd90 0x560e1a8ac1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe796d08640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe784071be0 0x7fe7840740f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe78c005fa0 tx=0x7fe78c002fb0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:41.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.890+0000 7fe797d0a640 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x560e1a8abd90 0x560e1a8ac1a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:41.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe797d0a640 1 -- 172.21.15.131:0/881426988 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v80'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fe790002040 con 0x560e1a8abd90 2024-09-17T02:45:41.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe79da32080 1 -- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x560e1a8abd90 msgr2=0x560e1a8ac1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe79da32080 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x560e1a8abd90 0x560e1a8ac1a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe79da32080 1 -- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe784071be0 msgr2=0x7fe7840740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe79da32080 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe784071be0 0x7fe7840740f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe78c005fa0 tx=0x7fe78c002fb0 comp rx=0 tx=0).stop 2024-09-17T02:45:41.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe79da32080 1 -- 172.21.15.131:0/881426988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fb9f0 msgr2=0x560e1a8aa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:41.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe79da32080 1 --2- 172.21.15.131:0/881426988 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e1a7fb9f0 0x560e1a8aa880 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe788019950 tx=0x7fe7880ab810 comp rx=0 tx=0).stop 2024-09-17T02:45:41.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe797d0a640 1 -- 172.21.15.131:0/881426988 reap_dead start 2024-09-17T02:45:41.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe79da32080 1 -- 172.21.15.131:0/881426988 shutdown_connections 2024-09-17T02:45:41.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe79da32080 1 -- 172.21.15.131:0/881426988 >> 172.21.15.131:0/881426988 conn(0x560e1a7bbd00 msgr2=0x560e1a8a0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:41.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe79da32080 1 -- 172.21.15.131:0/881426988 shutdown_connections 2024-09-17T02:45:41.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:41.898+0000 7fe79da32080 1 -- 172.21.15.131:0/881426988 wait complete. 2024-09-17T02:45:41.904 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T02:45:42.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.046+0000 7f87b3966640 1 -- 172.21.15.131:0/1279151784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ac0d3470 msgr2=0x7f87ac162b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:42.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.046+0000 7f87b3966640 1 --2- 172.21.15.131:0/1279151784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ac0d3470 0x7f87ac162b20 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009bc0 tx=0x7f87a401cba0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.046+0000 7f87b3966640 1 -- 172.21.15.131:0/1279151784 shutdown_connections 2024-09-17T02:45:42.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.046+0000 7f87b3966640 1 --2- 172.21.15.131:0/1279151784 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ac0d3470 0x7f87ac162b20 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.046+0000 7f87b3966640 1 --2- 172.21.15.131:0/1279151784 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87ac0cf710 0x7f87ac15f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.046+0000 7f87b3966640 1 -- 172.21.15.131:0/1279151784 >> 172.21.15.131:0/1279151784 conn(0x7f87ac0d1b10 msgr2=0x7f87ac0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:42.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.046+0000 7f87b3966640 1 -- 172.21.15.131:0/1279151784 shutdown_connections 2024-09-17T02:45:42.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.046+0000 7f87b3966640 1 -- 172.21.15.131:0/1279151784 wait complete. 2024-09-17T02:45:42.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b3966640 1 Processor -- start 2024-09-17T02:45:42.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b3966640 1 -- start start 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b3966640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87ac0cf710 0x7f87ac14fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b3966640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ac0d3470 0x7f87ac1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b3966640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87ac15f7d0 0x7f87ac1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b3966640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac15fdf0 con 0x7f87ac0d3470 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b3966640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac1632a0 con 0x7f87ac15f7d0 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b3966640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f87ac16bdb0 con 0x7f87ac0cf710 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b0f01640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ac0d3470 0x7f87ac1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b0f01640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ac0d3470 0x7f87ac1603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38426/0 (socket says 172.21.15.131:38426) 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b0f01640 1 -- 172.21.15.131:0/2855333620 learned_addr learned my addr 172.21.15.131:0/2855333620 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b1f03640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87ac15f7d0 0x7f87ac1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b1702640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87ac0cf710 0x7f87ac14fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b0f01640 1 -- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87ac0cf710 msgr2=0x7f87ac14fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:42.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b0f01640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87ac0cf710 0x7f87ac14fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b0f01640 1 -- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87ac15f7d0 msgr2=0x7f87ac1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b0f01640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87ac15f7d0 0x7f87ac1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b0f01640 1 -- 172.21.15.131:0/2855333620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac160930 con 0x7f87ac0d3470 2024-09-17T02:45:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b1f03640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87ac15f7d0 0x7f87ac1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b0f01640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ac0d3470 0x7f87ac1603f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f879c001ec0 tx=0x7f879c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87a1ffb640 1 -- 172.21.15.131:0/2855333620 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c017070 con 0x7f87ac0d3470 2024-09-17T02:45:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87a1ffb640 1 -- 172.21.15.131:0/2855333620 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f879c003600 con 0x7f87ac0d3470 2024-09-17T02:45:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b3966640 1 -- 172.21.15.131:0/2855333620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ac160e40 con 0x7f87ac0d3470 2024-09-17T02:45:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87b3966640 1 -- 172.21.15.131:0/2855333620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ac1f8930 con 0x7f87ac0d3470 2024-09-17T02:45:42.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.050+0000 7f87a1ffb640 1 -- 172.21.15.131:0/2855333620 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c003a50 con 0x7f87ac0d3470 2024-09-17T02:45:42.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.054+0000 7f87b3966640 1 -- 172.21.15.131:0/2855333620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878c0052a0 con 0x7f87ac0d3470 2024-09-17T02:45:42.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.054+0000 7f87a1ffb640 1 -- 172.21.15.131:0/2855333620 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f879c01a070 con 0x7f87ac0d3470 2024-09-17T02:45:42.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.054+0000 7f87a1ffb640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8790073230 0x7f8790075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:42.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.054+0000 7f87a1ffb640 1 -- 172.21.15.131:0/2855333620 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f879c093580 con 0x7f87ac0d3470 2024-09-17T02:45:42.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.054+0000 7f87b1702640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8790073230 0x7f8790075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:42.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.054+0000 7f87b1702640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8790073230 0x7f8790075740 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0b7610 tx=0x7f87a8005c50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:42.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.058+0000 7f87a1ffb640 1 -- 172.21.15.131:0/2855333620 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f879c05e450 con 0x7f87ac0d3470 2024-09-17T02:45:42.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.258+0000 7f87b3966640 1 -- 172.21.15.131:0/2855333620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f878c005860 con 0x7f87ac0d3470 2024-09-17T02:45:42.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.798+0000 7f87a1ffb640 1 -- 172.21.15.131:0/2855333620 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v48) ==== 162+0+0 (secure 0 0 0) 0x7f879c05dd50 con 0x7f87ac0d3470 2024-09-17T02:45:42.797 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T02:45:42.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 -- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8790073230 msgr2=0x7f8790075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:42.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8790073230 0x7f8790075740 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0b7610 tx=0x7f87a8005c50 comp rx=0 tx=0).stop 2024-09-17T02:45:42.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 -- 172.21.15.131:0/2855333620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ac0d3470 msgr2=0x7f87ac1603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:42.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ac0d3470 0x7f87ac1603f0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f879c001ec0 tx=0x7f879c00c890 comp rx=0 tx=0).stop 2024-09-17T02:45:42.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 -- 172.21.15.131:0/2855333620 shutdown_connections 2024-09-17T02:45:42.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8790073230 0x7f8790075740 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f87ac15f7d0 0x7f87ac1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f87ac0d3470 0x7f87ac1603f0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 --2- 172.21.15.131:0/2855333620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f87ac0cf710 0x7f87ac14fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 -- 172.21.15.131:0/2855333620 >> 172.21.15.131:0/2855333620 conn(0x7f87ac0d1b10 msgr2=0x7f87ac1522e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:42.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 -- 172.21.15.131:0/2855333620 shutdown_connections 2024-09-17T02:45:42.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.802+0000 7f87b3966640 1 -- 172.21.15.131:0/2855333620 wait complete. 2024-09-17T02:45:42.830 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:42.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.974+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2604754364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6a00d3470 msgr2=0x7fe6a0162b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:42.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.974+0000 7fe6a6d0e640 1 --2- 172.21.15.131:0/2604754364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6a00d3470 0x7fe6a0162b20 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe698009bc0 tx=0x7fe69801cba0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.974+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2604754364 shutdown_connections 2024-09-17T02:45:42.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.974+0000 7fe6a6d0e640 1 --2- 172.21.15.131:0/2604754364 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6a00d3470 0x7fe6a0162b20 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.974+0000 7fe6a6d0e640 1 --2- 172.21.15.131:0/2604754364 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe6a00cf710 0x7fe6a015f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.974+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2604754364 >> 172.21.15.131:0/2604754364 conn(0x7fe6a00d1b10 msgr2=0x7fe6a00d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:42.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.974+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2604754364 shutdown_connections 2024-09-17T02:45:42.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.974+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2604754364 wait complete. 2024-09-17T02:45:42.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a6d0e640 1 Processor -- start 2024-09-17T02:45:42.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a6d0e640 1 -- start start 2024-09-17T02:45:42.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a6d0e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe6a00cf710 0x7fe6a00ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a6d0e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6a00d3470 0x7fe6a015dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a6d0e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a015f7d0 0x7fe6a015e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a6d0e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a0163260 con 0x7fe6a00d3470 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a6d0e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a015fdf0 con 0x7fe6a015f7d0 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a6d0e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe6a016bdb0 con 0x7fe6a00cf710 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6a00d3470 0x7fe6a015dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6a00d3470 0x7fe6a015dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38434/0 (socket says 172.21.15.131:38434) 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69ffff640 1 -- 172.21.15.131:0/2919404598 learned_addr learned my addr 172.21.15.131:0/2919404598 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a4aaa640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe6a00cf710 0x7fe6a00ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a52ab640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a015f7d0 0x7fe6a015e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69ffff640 1 -- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe6a00cf710 msgr2=0x7fe6a00ce1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69ffff640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe6a00cf710 0x7fe6a00ce1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69ffff640 1 -- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a015f7d0 msgr2=0x7fe6a015e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:42.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69ffff640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a015f7d0 0x7fe6a015e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:42.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69ffff640 1 -- 172.21.15.131:0/2919404598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a00ce700 con 0x7fe6a00d3470 2024-09-17T02:45:42.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69ffff640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6a00d3470 0x7fe6a015dc50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00c420 tx=0x7fe68c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:42.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69d7fa640 1 -- 172.21.15.131:0/2919404598 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c011790 con 0x7fe6a00d3470 2024-09-17T02:45:42.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69d7fa640 1 -- 172.21.15.131:0/2919404598 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe68c017070 con 0x7fe6a00d3470 2024-09-17T02:45:42.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2919404598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a015c270 con 0x7fe6a00d3470 2024-09-17T02:45:42.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe69d7fa640 1 -- 172.21.15.131:0/2919404598 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c011ca0 con 0x7fe6a00d3470 2024-09-17T02:45:42.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2919404598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a0159c50 con 0x7fe6a00d3470 2024-09-17T02:45:42.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.978+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2919404598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a015bfe0 con 0x7fe6a00d3470 2024-09-17T02:45:42.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.982+0000 7fe69d7fa640 1 -- 172.21.15.131:0/2919404598 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe68c01c460 con 0x7fe6a00d3470 2024-09-17T02:45:42.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.982+0000 7fe69d7fa640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe684073200 0x7fe684075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:42.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.982+0000 7fe69d7fa640 1 -- 172.21.15.131:0/2919404598 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fe68c094160 con 0x7fe6a00d3470 2024-09-17T02:45:42.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.982+0000 7fe6a4aaa640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe684073200 0x7fe684075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:42.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.982+0000 7fe6a4aaa640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe684073200 0x7fe684075710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe6940058d0 tx=0x7fe694005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:42.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:42.986+0000 7fe69d7fa640 1 -- 172.21.15.131:0/2919404598 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe68c05f030 con 0x7fe6a00d3470 2024-09-17T02:45:43.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2919404598 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6a015f290 con 0x7fe6a00d3470 2024-09-17T02:45:43.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe69d7fa640 1 -- 172.21.15.131:0/2919404598 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5222 (secure 0 0 0) 0x7fe68c05e930 con 0x7fe6a00d3470 2024-09-17T02:45:43.176 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:43.176 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":49,"btime":"2024-09-17T02:45:42:807814+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:45:42.807766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.131:6835/962976667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":962976667},{"type":"v1","addr":"172.21.15.131:6835","nonce":962976667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-17T02:45:43.176 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 49 2024-09-17T02:45:43.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe684073200 msgr2=0x7fe684075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:43.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe684073200 0x7fe684075710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe6940058d0 tx=0x7fe694005860 comp rx=0 tx=0).stop 2024-09-17T02:45:43.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2919404598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6a00d3470 msgr2=0x7fe6a015dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:43.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6a00d3470 0x7fe6a015dc50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00c420 tx=0x7fe68c00c8e0 comp rx=0 tx=0).stop 2024-09-17T02:45:43.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2919404598 shutdown_connections 2024-09-17T02:45:43.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe684073200 0x7fe684075710 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:43.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe6a015f7d0 0x7fe6a015e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:43.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe6a00d3470 0x7fe6a015dc50 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:43.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 --2- 172.21.15.131:0/2919404598 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe6a00cf710 0x7fe6a00ce1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:43.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2919404598 >> 172.21.15.131:0/2919404598 conn(0x7fe6a00d1b10 msgr2=0x7fe6a0155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:43.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2919404598 shutdown_connections 2024-09-17T02:45:43.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:43.178+0000 7fe6a6d0e640 1 -- 172.21.15.131:0/2919404598 wait complete. 2024-09-17T02:45:43.600 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:45:19.460749+0000', 'modified': '2024-09-17T02:45:42.807766+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6609}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6609': {'gid': 6609, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/962976667', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6833', 'nonce': 962976667}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 962976667}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:45:43.600 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6609:up:replay 2024-09-17T02:45:44.313 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:45:44.316 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:45:44.602 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:44.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.770+0000 7f75a2b24640 1 -- 172.21.15.131:0/3906173613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7584004810 msgr2=0x7f7584004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:44.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.770+0000 7f75a2b24640 1 --2- 172.21.15.131:0/3906173613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7584004810 0x7f7584004c20 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7590009b40 tx=0x7f759001d270 comp rx=0 tx=0).stop 2024-09-17T02:45:44.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.770+0000 7f75a10c1640 1 -- 172.21.15.131:0/3906173613 reap_dead start 2024-09-17T02:45:44.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.770+0000 7f75a2b24640 1 -- 172.21.15.131:0/3906173613 shutdown_connections 2024-09-17T02:45:44.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.770+0000 7f75a2b24640 1 -- 172.21.15.131:0/3906173613 >> 172.21.15.131:0/3906173613 conn(0x7f759c06eea0 msgr2=0x7f759c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:44.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.770+0000 7f75a2b24640 1 -- 172.21.15.131:0/3906173613 shutdown_connections 2024-09-17T02:45:44.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.770+0000 7f75a2b24640 1 -- 172.21.15.131:0/3906173613 wait complete. 2024-09-17T02:45:44.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a2b24640 1 Processor -- start 2024-09-17T02:45:44.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a2b24640 1 -- start start 2024-09-17T02:45:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a2b24640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7584004810 0x7f759c149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a2b24640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c1f6590 0x7f759c0b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a2b24640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f759c0b5600 0x7f759c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a2b24640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f759c06e5b0 con 0x7f759c1f6590 2024-09-17T02:45:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a2b24640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f759c0cf320 con 0x7f7584004810 2024-09-17T02:45:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a2b24640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f759c06e730 con 0x7f759c0b5600 2024-09-17T02:45:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f759bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c1f6590 0x7f759c0b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f759bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c1f6590 0x7f759c0b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38454/0 (socket says 172.21.15.131:38454) 2024-09-17T02:45:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f759bfff640 1 -- 172.21.15.131:0/142855335 learned_addr learned my addr 172.21.15.131:0/142855335 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:44.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a10c1640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f759c0b5600 0x7f759c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a08c0640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7584004810 0x7f759c149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f759bfff640 1 -- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f759c0b5600 msgr2=0x7f759c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f759bfff640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f759c0b5600 0x7f759c0cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f759bfff640 1 -- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7584004810 msgr2=0x7f759c149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f759bfff640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7584004810 0x7f759c149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f759bfff640 1 -- 172.21.15.131:0/142855335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c0b5a10 con 0x7f759c1f6590 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f759bfff640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c1f6590 0x7f759c0b50c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7590098c90 tx=0x7f7590005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75997fa640 1 -- 172.21.15.131:0/142855335 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75900bb030 con 0x7f759c1f6590 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75997fa640 1 -- 172.21.15.131:0/142855335 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75900046d0 con 0x7f759c1f6590 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a2b24640 1 -- 172.21.15.131:0/142855335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c149980 con 0x7f759c1f6590 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75997fa640 1 -- 172.21.15.131:0/142855335 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590004aa0 con 0x7f759c1f6590 2024-09-17T02:45:44.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a2b24640 1 -- 172.21.15.131:0/142855335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c0a9470 con 0x7f759c1f6590 2024-09-17T02:45:44.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.774+0000 7f75a2b24640 1 -- 172.21.15.131:0/142855335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759c075b80 con 0x7f759c1f6590 2024-09-17T02:45:44.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.778+0000 7f75997fa640 1 -- 172.21.15.131:0/142855335 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f75900be030 con 0x7f759c1f6590 2024-09-17T02:45:44.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.778+0000 7f75997fa640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073230 0x7f7588075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:44.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.778+0000 7f75997fa640 1 -- 172.21.15.131:0/142855335 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f759012f890 con 0x7f759c1f6590 2024-09-17T02:45:44.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.778+0000 7f75a08c0640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073230 0x7f7588075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:44.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.778+0000 7f75a08c0640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073230 0x7f7588075740 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f758c0065b0 tx=0x7f758c006f00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:44.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.782+0000 7f75997fa640 1 -- 172.21.15.131:0/142855335 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f759c0b5a10 con 0x7f759c1f6590 2024-09-17T02:45:44.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.986+0000 7f75a2b24640 1 -- 172.21.15.131:0/142855335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f759c1f6d70 con 0x7f759c1f6590 2024-09-17T02:45:44.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.986+0000 7f75997fa640 1 -- 172.21.15.131:0/142855335 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5222 (secure 0 0 0) 0x7f7590016f80 con 0x7f759c1f6590 2024-09-17T02:45:44.986 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:44.986 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":51,"btime":"2024-09-17T02:45:44:819955+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:45:43.824810+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":49,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.131:6835/962976667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":962976667},{"type":"v1","addr":"172.21.15.131:6835","nonce":962976667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-17T02:45:44.987 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 51 2024-09-17T02:45:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.990+0000 7f75a2b24640 1 -- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073230 msgr2=0x7f7588075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.990+0000 7f75a2b24640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073230 0x7f7588075740 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f758c0065b0 tx=0x7f758c006f00 comp rx=0 tx=0).stop 2024-09-17T02:45:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.990+0000 7f75a2b24640 1 -- 172.21.15.131:0/142855335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c1f6590 msgr2=0x7f759c0b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.990+0000 7f75a2b24640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c1f6590 0x7f759c0b50c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7590098c90 tx=0x7f7590005500 comp rx=0 tx=0).stop 2024-09-17T02:45:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.990+0000 7f75a2b24640 1 -- 172.21.15.131:0/142855335 shutdown_connections 2024-09-17T02:45:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.990+0000 7f75a2b24640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073230 0x7f7588075740 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.990+0000 7f75a2b24640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f759c0b5600 0x7f759c0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.990+0000 7f75a2b24640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c1f6590 0x7f759c0b50c0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:44.990 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.990+0000 7f75a2b24640 1 --2- 172.21.15.131:0/142855335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7584004810 0x7f759c149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.990+0000 7f75a2b24640 1 -- 172.21.15.131:0/142855335 >> 172.21.15.131:0/142855335 conn(0x7f759c06eea0 msgr2=0x7f759c159d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.990+0000 7f75a2b24640 1 -- 172.21.15.131:0/142855335 shutdown_connections 2024-09-17T02:45:44.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:44.994+0000 7f75a2b24640 1 -- 172.21.15.131:0/142855335 wait complete. 2024-09-17T02:45:45.414 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:45:19.460749+0000', 'modified': '2024-09-17T02:45:43.824810+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6609}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6609': {'gid': 6609, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.131:6835/962976667', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6833', 'nonce': 962976667}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 962976667}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:45:45.414 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6609:up:rejoin 2024-09-17T02:45:46.415 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:45:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.586+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/489892001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd40ced00 msgr2=0x7f0dd406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.586+0000 7f0dd8b2a640 1 --2- 172.21.15.131:0/489892001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd40ced00 0x7f0dd406dea0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f0dd40b61b0 tx=0x7f0dc401c6e0 comp rx=0 tx=0).stop 2024-09-17T02:45:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.586+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/489892001 shutdown_connections 2024-09-17T02:45:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.586+0000 7f0dd8b2a640 1 --2- 172.21.15.131:0/489892001 >> v2:172.21.15.189:3300/0 conn(0x7f0dd406e3e0 0x7f0dd415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.586+0000 7f0dd8b2a640 1 --2- 172.21.15.131:0/489892001 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd40ced00 0x7f0dd406dea0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.586+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/489892001 >> 172.21.15.131:0/489892001 conn(0x7f0dd406eea0 msgr2=0x7f0dd406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.586+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/489892001 shutdown_connections 2024-09-17T02:45:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.586+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/489892001 wait complete. 2024-09-17T02:45:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd8b2a640 1 Processor -- start 2024-09-17T02:45:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd8b2a640 1 -- start start 2024-09-17T02:45:46.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd8b2a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd406e3e0 0x7f0dd415e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:46.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd8b2a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0dd40ced00 0x7f0dd4151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:46.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd8b2a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd415c3e0 0x7f0dd41523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:46.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd8b2a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd40cdf80 con 0x7f0dd415c3e0 2024-09-17T02:45:46.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd8b2a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd40cde00 con 0x7f0dd406e3e0 2024-09-17T02:45:46.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd8b2a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0dd40ce100 con 0x7f0dd40ced00 2024-09-17T02:45:46.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd415c3e0 0x7f0dd41523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:46.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd415c3e0 0x7f0dd41523c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38470/0 (socket says 172.21.15.131:38470) 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd2d9d640 1 -- 172.21.15.131:0/3699561587 learned_addr learned my addr 172.21.15.131:0/3699561587 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd259c640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd406e3e0 0x7f0dd415e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd1d9b640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0dd40ced00 0x7f0dd4151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd2d9d640 1 -- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0dd40ced00 msgr2=0x7f0dd4151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd2d9d640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0dd40ced00 0x7f0dd4151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd2d9d640 1 -- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd406e3e0 msgr2=0x7f0dd415e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd2d9d640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd406e3e0 0x7f0dd415e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd2d9d640 1 -- 172.21.15.131:0/3699561587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd4150240 con 0x7f0dd415c3e0 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd2d9d640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd415c3e0 0x7f0dd41523c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00ba10 tx=0x7f0dcc00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dc2ffd640 1 -- 172.21.15.131:0/3699561587 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc019070 con 0x7f0dd415c3e0 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dc2ffd640 1 -- 172.21.15.131:0/3699561587 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dcc00cc30 con 0x7f0dd415c3e0 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/3699561587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd414f1a0 con 0x7f0dd415c3e0 2024-09-17T02:45:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dc2ffd640 1 -- 172.21.15.131:0/3699561587 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc00ce60 con 0x7f0dd415c3e0 2024-09-17T02:45:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/3699561587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd414f760 con 0x7f0dd415c3e0 2024-09-17T02:45:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.590+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/3699561587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dd415bea0 con 0x7f0dd415c3e0 2024-09-17T02:45:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.594+0000 7f0dc2ffd640 1 -- 172.21.15.131:0/3699561587 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0dcc01c070 con 0x7f0dd415c3e0 2024-09-17T02:45:46.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.594+0000 7f0dc2ffd640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0db4073200 0x7f0db4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:46.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.594+0000 7f0dc2ffd640 1 -- 172.21.15.131:0/3699561587 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f0dcc094400 con 0x7f0dd415c3e0 2024-09-17T02:45:46.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.594+0000 7f0dd259c640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0db4073200 0x7f0db4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:46.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.594+0000 7f0dd259c640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0db4073200 0x7f0db4075710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40ae000 tx=0x7f0dc401af20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:46.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.598+0000 7f0dc2ffd640 1 -- 172.21.15.131:0/3699561587 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0dcc05f240 con 0x7f0dd415c3e0 2024-09-17T02:45:46.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/3699561587 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0dd415eb40 con 0x7f0dd415c3e0 2024-09-17T02:45:46.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dc2ffd640 1 -- 172.21.15.131:0/3699561587 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5229 (secure 0 0 0) 0x7f0dcc05eb40 con 0x7f0dd415c3e0 2024-09-17T02:45:46.803 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:45:46.804 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":52,"btime":"2024-09-17T02:45:45:829114+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:45:45.829099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/962976667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":962976667},{"type":"v1","addr":"172.21.15.131:6835","nonce":962976667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6609,"qdb_cluster":[6609]},"id":5}]} 2024-09-17T02:45:46.804 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 52 2024-09-17T02:45:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0db4073200 msgr2=0x7f0db4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0db4073200 0x7f0db4075710 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40ae000 tx=0x7f0dc401af20 comp rx=0 tx=0).stop 2024-09-17T02:45:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/3699561587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd415c3e0 msgr2=0x7f0dd41523c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd415c3e0 0x7f0dd41523c0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc00ba10 tx=0x7f0dcc00bed0 comp rx=0 tx=0).stop 2024-09-17T02:45:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/3699561587 shutdown_connections 2024-09-17T02:45:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0db4073200 0x7f0db4075710 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:46.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0dd415c3e0 0x7f0dd41523c0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0dd40ced00 0x7f0dd4151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 --2- 172.21.15.131:0/3699561587 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0dd406e3e0 0x7f0dd415e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/3699561587 >> 172.21.15.131:0/3699561587 conn(0x7f0dd406eea0 msgr2=0x7f0dd406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/3699561587 shutdown_connections 2024-09-17T02:45:46.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:46.806+0000 7f0dd8b2a640 1 -- 172.21.15.131:0/3699561587 wait complete. 2024-09-17T02:45:47.230 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:45:19.460749+0000', 'modified': '2024-09-17T02:45:45.829099+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6609}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6609': {'gid': 6609, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.131:6835/962976667', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6833', 'nonce': 962976667}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 962976667}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6609, 'qdb_cluster': [6609]} 2024-09-17T02:45:47.230 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:45:47.230 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T02:45:47.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.402+0000 7f15ef863640 1 -- 172.21.15.131:0/709354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e806e3e0 msgr2=0x7f15e815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.402+0000 7f15ef863640 1 --2- 172.21.15.131:0/709354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e806e3e0 0x7f15e815bea0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f15e80cca40 tx=0x7f15d801c8c0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.402+0000 7f15ef863640 1 -- 172.21.15.131:0/709354 shutdown_connections 2024-09-17T02:45:47.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.402+0000 7f15ef863640 1 --2- 172.21.15.131:0/709354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e806e3e0 0x7f15e815bea0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.402+0000 7f15ef863640 1 --2- 172.21.15.131:0/709354 >> v2:172.21.15.189:3300/0 conn(0x7f15e815c3e0 0x7f15e80cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.401 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.402+0000 7f15ef863640 1 --2- 172.21.15.131:0/709354 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f15e80ced00 0x7f15e806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.402+0000 7f15ef863640 1 -- 172.21.15.131:0/709354 >> 172.21.15.131:0/709354 conn(0x7f15e806eea0 msgr2=0x7f15e806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:47.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.402+0000 7f15ef863640 1 -- 172.21.15.131:0/709354 shutdown_connections 2024-09-17T02:45:47.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.402+0000 7f15ef863640 1 -- 172.21.15.131:0/709354 wait complete. 2024-09-17T02:45:47.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ef863640 1 Processor -- start 2024-09-17T02:45:47.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ef863640 1 -- start start 2024-09-17T02:45:47.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ef863640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15e806e3e0 0x7f15e815a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ef863640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e80ced00 0x7f15e8154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ef863640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f15e815c3e0 0x7f15e81553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ef863640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f15e80caa00 con 0x7f15e80ced00 2024-09-17T02:45:47.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ef863640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f15e80ca880 con 0x7f15e806e3e0 2024-09-17T02:45:47.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ef863640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f15e80cab80 con 0x7f15e815c3e0 2024-09-17T02:45:47.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ecdfe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e80ced00 0x7f15e8154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ecdfe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e80ced00 0x7f15e8154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38490/0 (socket says 172.21.15.131:38490) 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ecdfe640 1 -- 172.21.15.131:0/1840978548 learned_addr learned my addr 172.21.15.131:0/1840978548 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ede00640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f15e815c3e0 0x7f15e81553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15e806e3e0 0x7f15e815a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ecdfe640 1 -- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f15e815c3e0 msgr2=0x7f15e81553d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ecdfe640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f15e815c3e0 0x7f15e81553d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ecdfe640 1 -- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15e806e3e0 msgr2=0x7f15e815a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ecdfe640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15e806e3e0 0x7f15e815a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ecdfe640 1 -- 172.21.15.131:0/1840978548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e81546a0 con 0x7f15e80ced00 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ecdfe640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e80ced00 0x7f15e8154e90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f15d801af70 tx=0x7f15d8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ddffb640 1 -- 172.21.15.131:0/1840978548 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d80b2040 con 0x7f15e80ced00 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ddffb640 1 -- 172.21.15.131:0/1840978548 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15d8004330 con 0x7f15e80ced00 2024-09-17T02:45:47.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ef863640 1 -- 172.21.15.131:0/1840978548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e815b440 con 0x7f15e80ced00 2024-09-17T02:45:47.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ddffb640 1 -- 172.21.15.131:0/1840978548 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8005290 con 0x7f15e80ced00 2024-09-17T02:45:47.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.406+0000 7f15ef863640 1 -- 172.21.15.131:0/1840978548 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e8159560 con 0x7f15e80ced00 2024-09-17T02:45:47.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ddffb640 1 -- 172.21.15.131:0/1840978548 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f15d80bd060 con 0x7f15e80ced00 2024-09-17T02:45:47.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ddffb640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f15cc073230 0x7f15cc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ddffb640 1 -- 172.21.15.131:0/1840978548 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f15d812dc40 con 0x7f15e80ced00 2024-09-17T02:45:47.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f15cc073230 0x7f15cc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15f6b2e000 1 Processor -- start 2024-09-17T02:45:47.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15f6b2e000 1 -- start start 2024-09-17T02:45:47.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f15cc073230 0x7f15cc075740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009990 tx=0x7f15e40091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:47.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x5565965976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556596446950 0x556596597be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55659642fca0 0x556596598120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55659646c410 con 0x556596056460 2024-09-17T02:45:47.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5565963a9b40 con 0x556596446950 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x556596398f70 con 0x55659642fca0 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ed5ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x5565965976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ed5ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x5565965976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38496/0 (socket says 172.21.15.131:38496) 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ed5ff640 1 -- 172.21.15.131:0/618748178 learned_addr learned my addr 172.21.15.131:0/618748178 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ecdfe640 1 --2- 172.21.15.131:0/618748178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556596446950 0x556596597be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ede00640 1 --2- 172.21.15.131:0/618748178 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55659642fca0 0x556596598120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ed5ff640 1 -- 172.21.15.131:0/618748178 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55659642fca0 msgr2=0x556596598120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/618748178 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55659642fca0 0x556596598120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ed5ff640 1 -- 172.21.15.131:0/618748178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556596446950 msgr2=0x556596597be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/618748178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556596446950 0x556596597be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.410+0000 7f15ed5ff640 1 -- 172.21.15.131:0/618748178 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565963605b0 con 0x556596056460 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/618748178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x5565965976a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009fd0 tx=0x7f15e4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac348c0cab1cf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:47.411 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15d2ffd640 1 -- 172.21.15.131:0/618748178 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4023070 con 0x556596056460 2024-09-17T02:45:47.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15d2ffd640 1 -- 172.21.15.131:0/618748178 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15e4005020 con 0x556596056460 2024-09-17T02:45:47.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15d2ffd640 1 -- 172.21.15.131:0/618748178 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4009600 con 0x556596056460 2024-09-17T02:45:47.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 -- 172.21.15.131:0/618748178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 msgr2=0x5565965976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/618748178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x5565965976a0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009fd0 tx=0x7f15e4013000 comp rx=0 tx=0).stop 2024-09-17T02:45:47.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 -- 172.21.15.131:0/618748178 shutdown_connections 2024-09-17T02:45:47.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/618748178 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55659642fca0 0x556596598120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/618748178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556596446950 0x556596597be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/618748178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x5565965976a0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 -- 172.21.15.131:0/618748178 >> 172.21.15.131:0/618748178 conn(0x55659651cd90 msgr2=0x55659654c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:47.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 -- 172.21.15.131:0/618748178 shutdown_connections 2024-09-17T02:45:47.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 -- 172.21.15.131:0/618748178 wait complete. 2024-09-17T02:45:47.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 -- start start 2024-09-17T02:45:47.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 Processor -- start 2024-09-17T02:45:47.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659659e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55659642fca0 0x55659659ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556596446950 0x55659659f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5565963335f0 con 0x556596056460 2024-09-17T02:45:47.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55659639cd80 con 0x55659642fca0 2024-09-17T02:45:47.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55659633f520 con 0x556596446950 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15f6b2e000 10 client.0 _ll_register_callbacks cb 0x556596343830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15ed5ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659659e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15ed5ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659659e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38508/0 (socket says 172.21.15.131:38508) 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15ed5ff640 1 -- 172.21.15.131:0/3997858298 learned_addr learned my addr 172.21.15.131:0/3997858298 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15ede00640 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556596446950 0x55659659f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.414+0000 7f15ecdfe640 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55659642fca0 0x55659659ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15ed5ff640 1 -- 172.21.15.131:0/3997858298 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556596446950 msgr2=0x55659659f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556596446950 0x55659659f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15ed5ff640 1 -- 172.21.15.131:0/3997858298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55659642fca0 msgr2=0x55659659ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55659642fca0 0x55659659ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15ed5ff640 1 -- 172.21.15.131:0/3997858298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556596337190 con 0x556596056460 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659659e920 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x5565963a37e0 tx=0x7f15e4021470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b609b9b798db5d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:47.417 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15c7fff640 1 -- 172.21.15.131:0/3997858298 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4023040 con 0x556596056460 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15c7fff640 1 -- 172.21.15.131:0/3997858298 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f15e4021660 con 0x556596056460 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 -- 172.21.15.131:0/3997858298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556596331110 con 0x556596056460 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15c7fff640 1 -- 172.21.15.131:0/3997858298 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f15e4039050 con 0x556596056460 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 10 client.6657 fetch_fsmap learned FSMap version 52 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 -- 172.21.15.131:0/3997858298 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55659641df10 con 0x556596056460 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15c7fff640 1 -- 172.21.15.131:0/3997858298 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x556596331110 con 0x556596056460 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15c7fff640 1 -- 172.21.15.131:0/3997858298 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f15e403c020 con 0x556596056460 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 10 client.6657 fetch_fsmap finished waiting for FSMap version 52 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 10 client.6657 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 20 client.6657 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55659603b7d0 0x5565965a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 4 client.6657 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:45:47.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 -- 172.21.15.131:0/3997858298 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565963aba90 con 0x55659603b7d0 2024-09-17T02:45:47.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15ecdfe640 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55659603b7d0 0x5565965a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15c7fff640 10 client.6657 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:47.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15ecdfe640 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55659603b7d0 0x5565965a8d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976ed0d9faea53e4 server_cookie=b841058f88a6d42b in_seq=0 out_seq=0 2024-09-17T02:45:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15c7fff640 1 -- 172.21.15.131:0/3997858298 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5565963aba90 con 0x55659603b7d0 2024-09-17T02:45:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15c7fff640 10 client.6657 handle_command_reply: tid=0 2024-09-17T02:45:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 client.6657 shutdown 2024-09-17T02:45:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 -- 172.21.15.131:0/3997858298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 msgr2=0x55659659e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659659e920 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x5565963a37e0 tx=0x7f15e4021470 comp rx=0 tx=0).stop 2024-09-17T02:45:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 -- 172.21.15.131:0/3997858298 shutdown_connections 2024-09-17T02:45:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55659603b7d0 0x5565965a8d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556596446950 0x55659659f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55659642fca0 0x55659659ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/3997858298 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659659e920 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15c7fff640 0 client.6657 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:47.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 -- 172.21.15.131:0/3997858298 >> 172.21.15.131:0/3997858298 conn(0x55659651cd90 msgr2=0x556596553440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:47.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 -- 172.21.15.131:0/3997858298 shutdown_connections 2024-09-17T02:45:47.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 1 -- 172.21.15.131:0/3997858298 wait complete. 2024-09-17T02:45:47.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.418+0000 7f15f6b2e000 20 client.6657 trim_cache size 0 max 16384 2024-09-17T02:45:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15f6b2e000 1 Processor -- start 2024-09-17T02:45:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15f6b2e000 1 -- start start 2024-09-17T02:45:47.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x556596609620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55659642fca0 0x556596609b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556596446950 0x55659660a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8203400 con 0x556596056460 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8202400 con 0x55659642fca0 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f15e8201760 con 0x556596446950 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15ed5ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x556596609620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15ed5ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x556596609620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38522/0 (socket says 172.21.15.131:38522) 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15ed5ff640 1 -- 172.21.15.131:0/2755464830 learned_addr learned my addr 172.21.15.131:0/2755464830 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15ed5ff640 1 -- 172.21.15.131:0/2755464830 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556596446950 msgr2=0x55659660a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/2755464830 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556596446950 0x55659660a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15ed5ff640 1 -- 172.21.15.131:0/2755464830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55659642fca0 msgr2=0x556596609b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/2755464830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55659642fca0 0x556596609b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.438+0000 7f15ed5ff640 1 -- 172.21.15.131:0/2755464830 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565963f4db0 con 0x556596056460 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/2755464830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x556596609620 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f15e40216f0 tx=0x7f15e4021720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8403a5b286da900e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15d27fc640 1 -- 172.21.15.131:0/2755464830 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4016040 con 0x556596056460 2024-09-17T02:45:47.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15d27fc640 1 -- 172.21.15.131:0/2755464830 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15e4021b10 con 0x556596056460 2024-09-17T02:45:47.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15d27fc640 1 -- 172.21.15.131:0/2755464830 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4023040 con 0x556596056460 2024-09-17T02:45:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 -- 172.21.15.131:0/2755464830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 msgr2=0x556596609620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/2755464830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x556596609620 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f15e40216f0 tx=0x7f15e4021720 comp rx=0 tx=0).stop 2024-09-17T02:45:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 -- 172.21.15.131:0/2755464830 shutdown_connections 2024-09-17T02:45:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/2755464830 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x556596446950 0x55659660a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/2755464830 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55659642fca0 0x556596609b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/2755464830 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x556596609620 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 -- 172.21.15.131:0/2755464830 >> 172.21.15.131:0/2755464830 conn(0x55659651cd90 msgr2=0x556596595620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 -- 172.21.15.131:0/2755464830 shutdown_connections 2024-09-17T02:45:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 -- 172.21.15.131:0/2755464830 wait complete. 2024-09-17T02:45:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 -- start start 2024-09-17T02:45:47.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 Processor -- start 2024-09-17T02:45:47.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659660d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55659642fca0 0x55659660dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556596446950 0x55659660e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5565963a13b0 con 0x556596056460 2024-09-17T02:45:47.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556596458750 con 0x556596446950 2024-09-17T02:45:47.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55659641da80 con 0x55659642fca0 2024-09-17T02:45:47.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15f6b2e000 10 client.0 _ll_register_callbacks cb 0x7f15e4010c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15ed5ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659660d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15ed5ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659660d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38532/0 (socket says 172.21.15.131:38532) 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15ed5ff640 1 -- 172.21.15.131:0/767884148 learned_addr learned my addr 172.21.15.131:0/767884148 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15ecdfe640 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55659642fca0 0x55659660dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15ed5ff640 1 -- 172.21.15.131:0/767884148 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55659642fca0 msgr2=0x55659660dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55659642fca0 0x55659660dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15ed5ff640 1 -- 172.21.15.131:0/767884148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556596446950 msgr2=0x55659660e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15ede00640 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556596446950 0x55659660e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556596446950 0x55659660e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.442+0000 7f15ed5ff640 1 -- 172.21.15.131:0/767884148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55659635fab0 con 0x556596056460 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ed5ff640 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659660d5a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f15e4021320 tx=0x7f15e40107c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5de7cb8ac262458 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15c7fff640 1 -- 172.21.15.131:0/767884148 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4016070 con 0x556596056460 2024-09-17T02:45:47.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15c7fff640 1 -- 172.21.15.131:0/767884148 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f15e401f660 con 0x556596056460 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 -- 172.21.15.131:0/767884148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e4021b10 con 0x556596056460 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15c7fff640 1 -- 172.21.15.131:0/767884148 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f15e404c050 con 0x556596056460 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 10 client.6663 fetch_fsmap learned FSMap version 52 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 -- 172.21.15.131:0/767884148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55659633e070 con 0x556596056460 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15c7fff640 1 -- 172.21.15.131:0/767884148 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15e4021b10 con 0x556596056460 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15c7fff640 1 -- 172.21.15.131:0/767884148 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f15e404f020 con 0x556596056460 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 10 client.6663 fetch_fsmap finished waiting for FSMap version 52 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 10 client.6663 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 20 client.6663 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x556596029830 0x5565965c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 4 client.6663 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 -- 172.21.15.131:0/767884148 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "status"}) -- 0x5565963a7540 con 0x556596029830 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ecdfe640 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x556596029830 0x5565965c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15c7fff640 10 client.6663 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ecdfe640 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x556596029830 0x5565965c96a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e206c74dcdf5fee2 server_cookie=f6867df59558d879 in_seq=0 out_seq=0 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15c7fff640 1 -- 172.21.15.131:0/767884148 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5565963a7540 con 0x556596029830 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15c7fff640 10 client.6663 handle_command_reply: tid=0 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 client.6663 shutdown 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 -- 172.21.15.131:0/767884148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 msgr2=0x55659660d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659660d5a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f15e4021320 tx=0x7f15e40107c0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 -- 172.21.15.131:0/767884148 shutdown_connections 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x556596029830 0x5565965c96a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556596446950 0x55659660e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15c7fff640 0 client.6663 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55659642fca0 0x55659660dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 --2- 172.21.15.131:0/767884148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556596056460 0x55659660d5a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 -- 172.21.15.131:0/767884148 >> 172.21.15.131:0/767884148 conn(0x55659651cd90 msgr2=0x5565965a86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:47.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 -- 172.21.15.131:0/767884148 shutdown_connections 2024-09-17T02:45:47.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 1 -- 172.21.15.131:0/767884148 wait complete. 2024-09-17T02:45:47.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15f6b2e000 20 client.6663 trim_cache size 0 max 16384 2024-09-17T02:45:47.449 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:45:47.449 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:45:47.449 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:45:47.449 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6609, 2024-09-17T02:45:47.449 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:45:47.449 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:45:47.449 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:45:47.450 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 4.6318581400000003, 2024-09-17T02:45:47.450 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 52, 2024-09-17T02:45:47.450 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 80, 2024-09-17T02:45:47.450 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 80, 2024-09-17T02:45:47.450 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 5.5652723259999997 2024-09-17T02:45:47.450 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 -- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f15cc073230 msgr2=0x7f15cc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f15cc073230 0x7f15cc075740 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009990 tx=0x7f15e40091e0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 -- 172.21.15.131:0/1840978548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e80ced00 msgr2=0x7f15e8154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e80ced00 0x7f15e8154e90 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f15d801af70 tx=0x7f15d8005de0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 -- 172.21.15.131:0/1840978548 shutdown_connections 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f15cc073230 0x7f15cc075740 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f15e815c3e0 0x7f15e81553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f15e80ced00 0x7f15e8154e90 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 --2- 172.21.15.131:0/1840978548 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f15e806e3e0 0x7f15e815a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 -- 172.21.15.131:0/1840978548 >> 172.21.15.131:0/1840978548 conn(0x7f15e806eea0 msgr2=0x7f15e80cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 -- 172.21.15.131:0/1840978548 shutdown_connections 2024-09-17T02:45:47.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.446+0000 7f15ef863640 1 -- 172.21.15.131:0/1840978548 wait complete. 2024-09-17T02:45:47.465 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:45:47.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.614+0000 7f0179f65640 1 -- 172.21.15.131:0/489868221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f017415c3e0 msgr2=0x7f01740cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.614+0000 7f0179f65640 1 --2- 172.21.15.131:0/489868221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f017415c3e0 0x7f01740cbaf0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f016c009c80 tx=0x7f016c01cda0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.614+0000 7f0179f65640 1 -- 172.21.15.131:0/489868221 shutdown_connections 2024-09-17T02:45:47.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.614+0000 7f0179f65640 1 --2- 172.21.15.131:0/489868221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f017415c3e0 0x7f01740cbaf0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.614+0000 7f0179f65640 1 --2- 172.21.15.131:0/489868221 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f017406e5b0 0x7f017415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.614+0000 7f0179f65640 1 -- 172.21.15.131:0/489868221 >> 172.21.15.131:0/489868221 conn(0x7f017406eea0 msgr2=0x7f017406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:47.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.614+0000 7f0179f65640 1 -- 172.21.15.131:0/489868221 shutdown_connections 2024-09-17T02:45:47.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.614+0000 7f0179f65640 1 -- 172.21.15.131:0/489868221 wait complete. 2024-09-17T02:45:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0179f65640 1 Processor -- start 2024-09-17T02:45:47.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0179f65640 1 -- start start 2024-09-17T02:45:47.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0179f65640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740ced00 0x7f01740b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0179f65640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f017415c3e0 0x7f017414f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0179f65640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f017414d940 0x7f017414bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0179f65640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0174168830 con 0x7f01740ced00 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0179f65640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f01741686b0 con 0x7f017414d940 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0179f65640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f01741689b0 con 0x7f017415c3e0 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f01737fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740ced00 0x7f01740b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f01737fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740ced00 0x7f01740b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38556/0 (socket says 172.21.15.131:38556) 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f01737fe640 1 -- 172.21.15.131:0/346528374 learned_addr learned my addr 172.21.15.131:0/346528374 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0172ffd640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f017415c3e0 0x7f017414f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0173fff640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f017414d940 0x7f017414bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f01737fe640 1 -- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f017415c3e0 msgr2=0x7f017414f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f01737fe640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f017415c3e0 0x7f017414f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f01737fe640 1 -- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f017414d940 msgr2=0x7f017414bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f01737fe640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f017414d940 0x7f017414bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f01737fe640 1 -- 172.21.15.131:0/346528374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01740b2ef0 con 0x7f01740ced00 2024-09-17T02:45:47.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f01737fe640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740ced00 0x7f01740b39b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f016400c3c0 tx=0x7f016400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:47.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f015bfff640 1 -- 172.21.15.131:0/346528374 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164017070 con 0x7f01740ced00 2024-09-17T02:45:47.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f015bfff640 1 -- 172.21.15.131:0/346528374 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01640053c0 con 0x7f01740ced00 2024-09-17T02:45:47.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0179f65640 1 -- 172.21.15.131:0/346528374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017414fc10 con 0x7f01740ced00 2024-09-17T02:45:47.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f015bfff640 1 -- 172.21.15.131:0/346528374 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0164003860 con 0x7f01740ced00 2024-09-17T02:45:47.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.618+0000 7f0179f65640 1 -- 172.21.15.131:0/346528374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01740b31b0 con 0x7f01740ced00 2024-09-17T02:45:47.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.622+0000 7f0179f65640 1 -- 172.21.15.131:0/346528374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017415bea0 con 0x7f01740ced00 2024-09-17T02:45:47.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.622+0000 7f015bfff640 1 -- 172.21.15.131:0/346528374 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f016401a070 con 0x7f01740ced00 2024-09-17T02:45:47.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.622+0000 7f015bfff640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0160073200 0x7f0160075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:47.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.622+0000 7f015bfff640 1 -- 172.21.15.131:0/346528374 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f01640936d0 con 0x7f01740ced00 2024-09-17T02:45:47.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.622+0000 7f0172ffd640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0160073200 0x7f0160075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:47.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.622+0000 7f0172ffd640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0160073200 0x7f0160075710 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f01680058d0 tx=0x7f0168005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:47.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.626+0000 7f015bfff640 1 -- 172.21.15.131:0/346528374 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f016405e5a0 con 0x7f01740ced00 2024-09-17T02:45:47.814 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-09-17T02:45:47.814+0000 7f5131508640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:45:47.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.830+0000 7f0179f65640 1 -- 172.21.15.131:0/346528374 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f017415b450 con 0x7f01740ced00 2024-09-17T02:45:47.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.830+0000 7f015bfff640 1 -- 172.21.15.131:0/346528374 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7f016405dea0 con 0x7f01740ced00 2024-09-17T02:45:47.828 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:45:47.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 -- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0160073200 msgr2=0x7f0160075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0160073200 0x7f0160075710 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f01680058d0 tx=0x7f0168005860 comp rx=0 tx=0).stop 2024-09-17T02:45:47.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 -- 172.21.15.131:0/346528374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740ced00 msgr2=0x7f01740b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:47.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740ced00 0x7f01740b39b0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f016400c3c0 tx=0x7f016400c880 comp rx=0 tx=0).stop 2024-09-17T02:45:47.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 -- 172.21.15.131:0/346528374 shutdown_connections 2024-09-17T02:45:47.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0160073200 0x7f0160075710 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f017414d940 0x7f017414bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f017415c3e0 0x7f017414f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 --2- 172.21.15.131:0/346528374 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f01740ced00 0x7f01740b39b0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:47.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 -- 172.21.15.131:0/346528374 >> 172.21.15.131:0/346528374 conn(0x7f017406eea0 msgr2=0x7f01740b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:47.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 -- 172.21.15.131:0/346528374 shutdown_connections 2024-09-17T02:45:47.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:47.834+0000 7f0179f65640 1 -- 172.21.15.131:0/346528374 wait complete. 2024-09-17T02:45:47.850 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:45:47.851 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:45:47.851 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:45:47.851 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:45:47.851 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:47.851 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:45:47.851 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:45:47.851 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:45:47.851 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:45:47.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:47.852 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:47.857 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42158sec preferred_lft 42158sec 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:45:47.858 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:45:47.859 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:45:47.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:47.860 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:45:47.906 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:45:47.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:47.907 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:45:47.959 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:45:47.959 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:45:47.960 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:45:47.960 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:45:47.960 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:45:47.960 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:45:47.960 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:45:47.961 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:45:47.961 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:45:47.961 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:45:47.961 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:45:47.962 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:45:47.962 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:45:47.962 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:45:47.962 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:45:47.962 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:45:47.963 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:45:48.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:45:48.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:45:48.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:48.079 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:45:48.126 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:45:48.127 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:45:48.127 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:45:48.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:48.127 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:45:48.187 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:45:48.187 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:45:48.187 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:48.235 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:45:48.235 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:45:48.235 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:48.282 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:45:48.339 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:45:48.391 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:45:48.391 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:45:48.391 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:45:48.391 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:45:48.391 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:45:48.391 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:45:48.391 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:45:48.391 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:45:48.391 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:45:48.392 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:45:48.393 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:45:48.393 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:45:48.393 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:45:48.393 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:45:48.393 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:45:48.393 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:45:48.393 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:45:48.393 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:45:48.393 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:48.393 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:48.394 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:48.394 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:48.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:48.394 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:45:48.439 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:45:48.439 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:45:48.484 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:45:48.498 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:45:48.548 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:45:48.548 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:45:48.548 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:45:48.548 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:45:48.548 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:45:48.548 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:45:48.548 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:45:48.548 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:45:48.548 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:45:48.549 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:48.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:48.551 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:45:48.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:45:48.594+0000 7f090bc0b580 -1 init, newargv = 0x7f08fc0ad020 newargc=15 2024-09-17T02:45:48.594 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[44253]: starting ceph client 2024-09-17T02:45:48.598 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[44253]: starting fuse 2024-09-17T02:45:49.699 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:45:49.717 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:45:49.765 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:45:49.765 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:45:49.765 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:45:49.766 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:45:49.767 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:45:49.768 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:49.768 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:45:49.768 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:49.768 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:45:49.768 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:49.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:49.769 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:45:49.815 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:45:49.815 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:45:49.815 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:45:49.816 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:45:49.817 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:45:49.940 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.44253.asok 2024-09-17T02:45:49.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:45:49.953 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44253.asok 2024-09-17T02:45:49.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:49.953 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44253.asok status 2024-09-17T02:45:50.133 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:45:50.133 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "44253", 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6675, 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:45:50.134 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 6675 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2025616945 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2025616945 2024-09-17T02:45:50.135 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:50.136 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.6675 192.168.0.1:0/2025616945", 2024-09-17T02:45:50.136 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/2025616945", 2024-09-17T02:45:50.136 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:45:50.136 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 52, 2024-09-17T02:45:50.136 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 80, 2024-09-17T02:45:50.136 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:45:50.136 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:45:50.136 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:45:50.136 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:45:50.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:50.150 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:45:50.150 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:50.157 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:45:50.157 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:50.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:50.157 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:50.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:50.210 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-09-17T02:45:50.267 INFO:teuthology.orchestra.run.smithi131.stdout:file_undamaged 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> import json 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> import sys 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> try: 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> except OSError as e: 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> sys.exit(e.errno) 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> print(json.dumps( 2024-09-17T02:45:50.269 DEBUG:teuthology.orchestra.run.smithi131:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-17T02:45:50.270 DEBUG:teuthology.orchestra.run.smithi131:> indent=2)) 2024-09-17T02:45:50.270 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:45:50.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-09-17T02:45:50.397 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-17T02:45:50.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:50.398 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-17T02:45:50.448 INFO:teuthology.orchestra.run.smithi131.stdout:2 2024-09-17T02:45:50.448 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:45:50.448 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:50.496 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:45:50.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.706+0000 7f6df8aae640 1 -- 172.21.15.131:0/3631883186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ddc003a60 msgr2=0x7f6ddc003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.706+0000 7f6df8aae640 1 --2- 172.21.15.131:0/3631883186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ddc003a60 0x7f6ddc003ed0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6de400c2e0 tx=0x7f6de401cf80 comp rx=0 tx=0).stop 2024-09-17T02:45:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.706+0000 7f6df8aae640 1 -- 172.21.15.131:0/3631883186 shutdown_connections 2024-09-17T02:45:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.706+0000 7f6df8aae640 1 --2- 172.21.15.131:0/3631883186 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ddc003a60 0x7f6ddc003ed0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.706+0000 7f6df8aae640 1 -- 172.21.15.131:0/3631883186 >> 172.21.15.131:0/3631883186 conn(0x7f6df406eea0 msgr2=0x7f6df406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.706+0000 7f6df8aae640 1 -- 172.21.15.131:0/3631883186 shutdown_connections 2024-09-17T02:45:50.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.706+0000 7f6df8aae640 1 -- 172.21.15.131:0/3631883186 wait complete. 2024-09-17T02:45:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df8aae640 1 Processor -- start 2024-09-17T02:45:50.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df8aae640 1 -- start start 2024-09-17T02:45:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df8aae640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ddc003a60 0x7f6df40cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df8aae640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41ec3c0 0x7f6df41ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df8aae640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6df41fc700 0x7f6df41fcb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df8aae640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6df40c4f10 con 0x7f6ddc003a60 2024-09-17T02:45:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df8aae640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6df40c4d90 con 0x7f6df41ec3c0 2024-09-17T02:45:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df8aae640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6df40c5090 con 0x7f6df41fc700 2024-09-17T02:45:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ddc003a60 0x7f6df40cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ddc003a60 0x7f6df40cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:38568/0 (socket says 172.21.15.131:38568) 2024-09-17T02:45:50.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df259c640 1 -- 172.21.15.131:0/1953850433 learned_addr learned my addr 172.21.15.131:0/1953850433 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df2d9d640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6df41fc700 0x7f6df41fcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df1d9b640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41ec3c0 0x7f6df41ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df259c640 1 -- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6df41fc700 msgr2=0x7f6df41fcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df259c640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6df41fc700 0x7f6df41fcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df259c640 1 -- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41ec3c0 msgr2=0x7f6df41ec7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df259c640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41ec3c0 0x7f6df41ec7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df259c640 1 -- 172.21.15.131:0/1953850433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df41ecd10 con 0x7f6ddc003a60 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df2d9d640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6df41fc700 0x7f6df41fcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df259c640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ddc003a60 0x7f6df40cc280 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6de401b680 tx=0x7f6de4001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6de2ffd640 1 -- 172.21.15.131:0/1953850433 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4001500 con 0x7f6ddc003a60 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6de2ffd640 1 -- 172.21.15.131:0/1953850433 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6de40016a0 con 0x7f6ddc003a60 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df8aae640 1 -- 172.21.15.131:0/1953850433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df40ca8c0 con 0x7f6ddc003a60 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6de2ffd640 1 -- 172.21.15.131:0/1953850433 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de4006330 con 0x7f6ddc003a60 2024-09-17T02:45:50.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df8aae640 1 -- 172.21.15.131:0/1953850433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df40cae80 con 0x7f6ddc003a60 2024-09-17T02:45:50.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.710+0000 7f6df8aae640 1 -- 172.21.15.131:0/1953850433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df40cdd80 con 0x7f6ddc003a60 2024-09-17T02:45:50.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.714+0000 7f6de2ffd640 1 -- 172.21.15.131:0/1953850433 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6de40a5c60 con 0x7f6ddc003a60 2024-09-17T02:45:50.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.714+0000 7f6de2ffd640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6dd4073230 0x7f6dd4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:50.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.714+0000 7f6de2ffd640 1 -- 172.21.15.131:0/1953850433 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f6de412dcf0 con 0x7f6ddc003a60 2024-09-17T02:45:50.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.714+0000 7f6df1d9b640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6dd4073230 0x7f6dd4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:50.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.714+0000 7f6df1d9b640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6dd4073230 0x7f6dd4075740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6df40af7d0 tx=0x7f6de8006f00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:50.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.718+0000 7f6de2ffd640 1 -- 172.21.15.131:0/1953850433 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6de4132070 con 0x7f6ddc003a60 2024-09-17T02:45:50.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.898+0000 7f6df8aae640 1 -- 172.21.15.131:0/1953850433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6df4073310 con 0x7f6ddc003a60 2024-09-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.898+0000 7f6de2ffd640 1 -- 172.21.15.131:0/1953850433 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v80) ==== 80+0+122 (secure 0 0 0) 0x7f6df41ecd10 con 0x7f6ddc003a60 2024-09-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6835/655402879 2024-09-18T02:45:41.796095+0000 2024-09-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6833/655402879 2024-09-18T02:45:41.796095+0000 2024-09-17T02:45:50.898 INFO:teuthology.orchestra.run.smithi131.stderr:listed 2 entries 2024-09-17T02:45:50.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 -- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6dd4073230 msgr2=0x7f6dd4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:50.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6dd4073230 0x7f6dd4075740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6df40af7d0 tx=0x7f6de8006f00 comp rx=0 tx=0).stop 2024-09-17T02:45:50.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 -- 172.21.15.131:0/1953850433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ddc003a60 msgr2=0x7f6df40cc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:50.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ddc003a60 0x7f6df40cc280 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6de401b680 tx=0x7f6de4001d90 comp rx=0 tx=0).stop 2024-09-17T02:45:50.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 -- 172.21.15.131:0/1953850433 shutdown_connections 2024-09-17T02:45:50.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6dd4073230 0x7f6dd4075740 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:50.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6df41fc700 0x7f6df41fcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:50.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6df41ec3c0 0x7f6df41ec7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:50.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 --2- 172.21.15.131:0/1953850433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6ddc003a60 0x7f6df40cc280 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:50.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 -- 172.21.15.131:0/1953850433 >> 172.21.15.131:0/1953850433 conn(0x7f6df406eea0 msgr2=0x7f6df415a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:50.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 -- 172.21.15.131:0/1953850433 shutdown_connections 2024-09-17T02:45:50.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:50.902+0000 7f6df8aae640 1 -- 172.21.15.131:0/1953850433 wait complete. 2024-09-17T02:45:50.920 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:45:50.920 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:45:50.926 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:45:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 -- 172.21.15.131:0/3503023705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88040cec00 msgr2=0x7f88040cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 --2- 172.21.15.131:0/3503023705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88040cec00 0x7f88040cf010 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009900 tx=0x7f87f401c850 comp rx=0 tx=0).stop 2024-09-17T02:45:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 -- 172.21.15.131:0/3503023705 shutdown_connections 2024-09-17T02:45:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 --2- 172.21.15.131:0/3503023705 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f88040cec00 0x7f88040cf010 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 --2- 172.21.15.131:0/3503023705 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f880406e2c0 0x7f88040c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 -- 172.21.15.131:0/3503023705 >> 172.21.15.131:0/3503023705 conn(0x7f880406eea0 msgr2=0x7f880406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 -- 172.21.15.131:0/3503023705 shutdown_connections 2024-09-17T02:45:51.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 -- 172.21.15.131:0/3503023705 wait complete. 2024-09-17T02:45:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 Processor -- start 2024-09-17T02:45:51.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 -- start start 2024-09-17T02:45:51.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f880406e2c0 0x7f8804152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:51.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f88040c4ac0 0x7f880414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:51.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88040cec00 0x7f880414a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:51.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8804169cf0 con 0x7f880406e2c0 2024-09-17T02:45:51.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f880415cb90 con 0x7f88040cec00 2024-09-17T02:45:51.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8809361640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8804169e70 con 0x7f88040c4ac0 2024-09-17T02:45:51.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.138+0000 7f8802ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f880406e2c0 0x7f8804152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f8802ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f880406e2c0 0x7f8804152bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43104/0 (socket says 172.21.15.131:43104) 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f8802ffd640 1 -- 172.21.15.131:0/2590080381 learned_addr learned my addr 172.21.15.131:0/2590080381 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f88027fc640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f88040c4ac0 0x7f880414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f88037fe640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88040cec00 0x7f880414a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f8802ffd640 1 -- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f88040c4ac0 msgr2=0x7f880414c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f8802ffd640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f88040c4ac0 0x7f880414c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f8802ffd640 1 -- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88040cec00 msgr2=0x7f880414a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f8802ffd640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88040cec00 0x7f880414a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f8802ffd640 1 -- 172.21.15.131:0/2590080381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88041fa4c0 con 0x7f880406e2c0 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f8802ffd640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f880406e2c0 0x7f8804152bb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00cd10 tx=0x7f87fc007610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f87fb7fe640 1 -- 172.21.15.131:0/2590080381 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc004b30 con 0x7f880406e2c0 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f87fb7fe640 1 -- 172.21.15.131:0/2590080381 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87fc004de0 con 0x7f880406e2c0 2024-09-17T02:45:51.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f8809361640 1 -- 172.21.15.131:0/2590080381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88041fa7a0 con 0x7f880406e2c0 2024-09-17T02:45:51.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f87fb7fe640 1 -- 172.21.15.131:0/2590080381 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc005280 con 0x7f880406e2c0 2024-09-17T02:45:51.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f8809361640 1 -- 172.21.15.131:0/2590080381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88041f6ab0 con 0x7f880406e2c0 2024-09-17T02:45:51.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f8809361640 1 -- 172.21.15.131:0/2590080381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88040c4580 con 0x7f880406e2c0 2024-09-17T02:45:51.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f87fb7fe640 1 -- 172.21.15.131:0/2590080381 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f87fc005420 con 0x7f880406e2c0 2024-09-17T02:45:51.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f87fb7fe640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87e4073230 0x7f87e4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:51.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f87fb7fe640 1 -- 172.21.15.131:0/2590080381 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f87fc094fa0 con 0x7f880406e2c0 2024-09-17T02:45:51.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.142+0000 7f88027fc640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87e4073230 0x7f87e4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:51.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.146+0000 7f88027fc640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87e4073230 0x7f87e4075740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f88041f4500 tx=0x7f87ec008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:51.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.146+0000 7f87fb7fe640 1 -- 172.21.15.131:0/2590080381 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f87fc014030 con 0x7f880406e2c0 2024-09-17T02:45:51.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.334+0000 7f8809361640 1 -- 172.21.15.131:0/2590080381 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8804159d10 con 0x7f880406e2c0 2024-09-17T02:45:51.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.334+0000 7f87fb7fe640 1 -- 172.21.15.131:0/2590080381 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v80) ==== 80+0+122 (secure 0 0 0) 0x7f87fc05fde0 con 0x7f880406e2c0 2024-09-17T02:45:51.332 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6835/655402879 2024-09-18T02:45:41.796095+0000 2024-09-17T02:45:51.333 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6833/655402879 2024-09-18T02:45:41.796095+0000 2024-09-17T02:45:51.333 INFO:teuthology.orchestra.run.smithi131.stderr:listed 2 entries 2024-09-17T02:45:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 -- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87e4073230 msgr2=0x7f87e4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87e4073230 0x7f87e4075740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f88041f4500 tx=0x7f87ec008040 comp rx=0 tx=0).stop 2024-09-17T02:45:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 -- 172.21.15.131:0/2590080381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f880406e2c0 msgr2=0x7f8804152bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f880406e2c0 0x7f8804152bb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00cd10 tx=0x7f87fc007610 comp rx=0 tx=0).stop 2024-09-17T02:45:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 -- 172.21.15.131:0/2590080381 shutdown_connections 2024-09-17T02:45:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f87e4073230 0x7f87e4075740 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f88040cec00 0x7f880414a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f88040c4ac0 0x7f880414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 --2- 172.21.15.131:0/2590080381 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f880406e2c0 0x7f8804152bb0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 -- 172.21.15.131:0/2590080381 >> 172.21.15.131:0/2590080381 conn(0x7f880406eea0 msgr2=0x7f880406fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:51.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 -- 172.21.15.131:0/2590080381 shutdown_connections 2024-09-17T02:45:51.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:51.338+0000 7f8809361640 1 -- 172.21.15.131:0/2590080381 wait complete. 2024-09-17T02:45:51.357 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:45:51.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:51.358 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:45:51.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[44253]: fuse finished with error 0 and tester_r 0 2024-09-17T02:45:51.371 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:45:53.300 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:45:53.302+0000 7f3a39041640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-17T02:45:57.472 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:45:57.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:57.472 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:45:57.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:57.480 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-09-17T02:45:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.694+0000 7f577d18f640 1 -- 172.21.15.131:0/1509420700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57780cbed0 msgr2=0x7f57780cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.694+0000 7f577d18f640 1 --2- 172.21.15.131:0/1509420700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57780cbed0 0x7f57780cc2e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f5770009c80 tx=0x7f577001ca60 comp rx=0 tx=0).stop 2024-09-17T02:45:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.694+0000 7f577d18f640 1 -- 172.21.15.131:0/1509420700 shutdown_connections 2024-09-17T02:45:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.694+0000 7f577d18f640 1 --2- 172.21.15.131:0/1509420700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f57780cbed0 0x7f57780cc2e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.694+0000 7f577d18f640 1 --2- 172.21.15.131:0/1509420700 >> v2:172.21.15.189:3300/0 conn(0x7f577806f700 0x7f577806fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.694+0000 7f577d18f640 1 -- 172.21.15.131:0/1509420700 >> 172.21.15.131:0/1509420700 conn(0x7f57780d05a0 msgr2=0x7f57780d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.694+0000 7f577d18f640 1 -- 172.21.15.131:0/1509420700 shutdown_connections 2024-09-17T02:45:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.694+0000 7f577d18f640 1 -- 172.21.15.131:0/1509420700 wait complete. 2024-09-17T02:45:57.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577d18f640 1 Processor -- start 2024-09-17T02:45:57.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577d18f640 1 -- start start 2024-09-17T02:45:57.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577d18f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f577806f700 0x7f5778151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577d18f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f57780b2ef0 0x7f57780ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577d18f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57780cbed0 0x7f57780ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577d18f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f57780743a0 con 0x7f577806f700 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577d18f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5778074220 con 0x7f57780cbed0 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577d18f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5778168870 con 0x7f57780b2ef0 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f5776d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f577806f700 0x7f5778151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f5776d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f577806f700 0x7f5778151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43116/0 (socket says 172.21.15.131:43116) 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f5776d9d640 1 -- 172.21.15.131:0/1804191096 learned_addr learned my addr 172.21.15.131:0/1804191096 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577659c640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f57780b2ef0 0x7f57780ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577759e640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57780cbed0 0x7f57780ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f5776d9d640 1 -- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f57780b2ef0 msgr2=0x7f57780ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f5776d9d640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f57780b2ef0 0x7f57780ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f5776d9d640 1 -- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57780cbed0 msgr2=0x7f57780ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f5776d9d640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57780cbed0 0x7f57780ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f5776d9d640 1 -- 172.21.15.131:0/1804191096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5778151970 con 0x7f577806f700 2024-09-17T02:45:57.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f5776d9d640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f577806f700 0x7f5778151430 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f576800b3e0 tx=0x7f576800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f575f7fe640 1 -- 172.21.15.131:0/1804191096 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768015020 con 0x7f577806f700 2024-09-17T02:45:57.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f575f7fe640 1 -- 172.21.15.131:0/1804191096 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5768011940 con 0x7f577806f700 2024-09-17T02:45:57.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f575f7fe640 1 -- 172.21.15.131:0/1804191096 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768011d90 con 0x7f577806f700 2024-09-17T02:45:57.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577d18f640 1 -- 172.21.15.131:0/1804191096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57780cabc0 con 0x7f577806f700 2024-09-17T02:45:57.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.698+0000 7f577d18f640 1 -- 172.21.15.131:0/1804191096 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57780cb180 con 0x7f577806f700 2024-09-17T02:45:57.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.702+0000 7f575f7fe640 1 -- 172.21.15.131:0/1804191096 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5768018020 con 0x7f577806f700 2024-09-17T02:45:57.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.702+0000 7f575f7fe640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5764073200 0x7f5764075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.702+0000 7f575f7fe640 1 -- 172.21.15.131:0/1804191096 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f5768094410 con 0x7f577806f700 2024-09-17T02:45:57.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.702+0000 7f577659c640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5764073200 0x7f5764075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.702+0000 7f578445a000 1 Processor -- start 2024-09-17T02:45:57.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.702+0000 7f578445a000 1 -- start start 2024-09-17T02:45:57.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.702+0000 7f577659c640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5764073200 0x7f5764075710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57780b1b60 tx=0x7f576c005db0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d34940 0x562231e93620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231e93b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562231ca5a60 con 0x562231d1a9a0 2024-09-17T02:45:57.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562231c94e90 con 0x562231d34940 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562231c98ca0 con 0x562231d20ce0 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f577759e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f577759e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e940a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43118/0 (socket says 172.21.15.131:43118) 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f577759e640 1 -- 172.21.15.131:0/3967875205 learned_addr learned my addr 172.21.15.131:0/3967875205 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f5776d9d640 1 --2- 172.21.15.131:0/3967875205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d34940 0x562231e93620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f577659c640 1 --2- 172.21.15.131:0/3967875205 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231e93b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f577759e640 1 -- 172.21.15.131:0/3967875205 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 msgr2=0x562231e93b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f577759e640 1 --2- 172.21.15.131:0/3967875205 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231e93b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f577759e640 1 -- 172.21.15.131:0/3967875205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d34940 msgr2=0x562231e93620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f577759e640 1 --2- 172.21.15.131:0/3967875205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d34940 0x562231e93620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f577759e640 1 -- 172.21.15.131:0/3967875205 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562231cde060 con 0x562231d1a9a0 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f577759e640 1 --2- 172.21.15.131:0/3967875205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e940a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f577001b110 tx=0x7f5770019800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe0f9a22ec9c281 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f575cff9640 1 -- 172.21.15.131:0/3967875205 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700b2040 con 0x562231d1a9a0 2024-09-17T02:45:57.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f575cff9640 1 -- 172.21.15.131:0/3967875205 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5770004c50 con 0x562231d1a9a0 2024-09-17T02:45:57.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f575cff9640 1 -- 172.21.15.131:0/3967875205 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5770009340 con 0x562231d1a9a0 2024-09-17T02:45:57.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 -- 172.21.15.131:0/3967875205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 msgr2=0x562231e940a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 --2- 172.21.15.131:0/3967875205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e940a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f577001b110 tx=0x7f5770019800 comp rx=0 tx=0).stop 2024-09-17T02:45:57.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 -- 172.21.15.131:0/3967875205 shutdown_connections 2024-09-17T02:45:57.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 --2- 172.21.15.131:0/3967875205 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e940a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 --2- 172.21.15.131:0/3967875205 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231e93b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 --2- 172.21.15.131:0/3967875205 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d34940 0x562231e93620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 -- 172.21.15.131:0/3967875205 >> 172.21.15.131:0/3967875205 conn(0x562231953330 msgr2=0x562231e485d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 -- 172.21.15.131:0/3967875205 shutdown_connections 2024-09-17T02:45:57.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 -- 172.21.15.131:0/3967875205 wait complete. 2024-09-17T02:45:57.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 -- start start 2024-09-17T02:45:57.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.706+0000 7f578445a000 1 Processor -- start 2024-09-17T02:45:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e4f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231e4f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d34940 0x562231e82130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562231cd8b80 con 0x562231d1a9a0 2024-09-17T02:45:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562231c3e970 con 0x562231d34940 2024-09-17T02:45:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562231c4abd0 con 0x562231d20ce0 2024-09-17T02:45:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 10 client.0 _ll_register_callbacks cb 0x562231c3b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5776d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e4f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5776d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e4f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43132/0 (socket says 172.21.15.131:43132) 2024-09-17T02:45:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5776d9d640 1 -- 172.21.15.131:0/1803497187 learned_addr learned my addr 172.21.15.131:0/1803497187 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f577659c640 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231e4f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f577759e640 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d34940 0x562231e82130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5776d9d640 1 -- 172.21.15.131:0/1803497187 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 msgr2=0x562231e4f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5776d9d640 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231e4f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5776d9d640 1 -- 172.21.15.131:0/1803497187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d34940 msgr2=0x562231e82130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5776d9d640 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d34940 0x562231e82130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5776d9d640 1 -- 172.21.15.131:0/1803497187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562231c3db50 con 0x562231d1a9a0 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5776d9d640 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e4f460 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5768009fd0 tx=0x7f576805e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58c1aac6c28a20 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5755ffb640 1 -- 172.21.15.131:0/1803497187 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5768063590 con 0x562231d1a9a0 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 -- 172.21.15.131:0/1803497187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562231c2d200 con 0x562231d1a9a0 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5755ffb640 1 -- 172.21.15.131:0/1803497187 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f5768084030 con 0x562231d1a9a0 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5755ffb640 1 -- 172.21.15.131:0/1803497187 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f576807f050 con 0x562231d1a9a0 2024-09-17T02:45:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 10 client.6699 fetch_fsmap learned FSMap version 53 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 -- 172.21.15.131:0/1803497187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x562231c330c0 con 0x562231d1a9a0 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5755ffb640 1 -- 172.21.15.131:0/1803497187 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562231c2d200 con 0x562231d1a9a0 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5755ffb640 1 -- 172.21.15.131:0/1803497187 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f576805dd10 con 0x562231d1a9a0 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 10 client.6699 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 10 client.6699 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 20 client.6699 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562231937b10 0x562231e9a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 4 client.6699 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 -- 172.21.15.131:0/1803497187 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562231ca79b0 con 0x562231937b10 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f577659c640 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562231937b10 0x562231e9a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5755ffb640 10 client.6699 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f577659c640 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562231937b10 0x562231e9a8a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc71ac71e434b5f server_cookie=be36c94e04dbead9 in_seq=0 out_seq=0 2024-09-17T02:45:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5755ffb640 1 -- 172.21.15.131:0/1803497187 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562231ca79b0 con 0x562231937b10 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5755ffb640 10 client.6699 handle_command_reply: tid=0 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 client.6699 shutdown 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 -- 172.21.15.131:0/1803497187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 msgr2=0x562231e4f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e4f460 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5768009fd0 tx=0x7f576805e6f0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 -- 172.21.15.131:0/1803497187 shutdown_connections 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562231937b10 0x562231e9a8a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d34940 0x562231e82130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231e4f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f5755ffb640 0 client.6699 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 --2- 172.21.15.131:0/1803497187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d1a9a0 0x562231e4f460 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 -- 172.21.15.131:0/1803497187 >> 172.21.15.131:0/1803497187 conn(0x562231953330 msgr2=0x562231e48d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 -- 172.21.15.131:0/1803497187 shutdown_connections 2024-09-17T02:45:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 1 -- 172.21.15.131:0/1803497187 wait complete. 2024-09-17T02:45:57.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.710+0000 7f578445a000 20 client.6699 trim_cache size 0 max 16384 2024-09-17T02:45:57.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f578445a000 1 Processor -- start 2024-09-17T02:45:57.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f578445a000 1 -- start start 2024-09-17T02:45:57.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f578445a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d1a9a0 0x562231e47320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f578445a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d20ce0 0x562231e47860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f578445a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d34940 0x562231e47da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f578445a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5778202a30 con 0x562231d20ce0 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f578445a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f57781596b0 con 0x562231d1a9a0 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f578445a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5778202490 con 0x562231d34940 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f577659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d20ce0 0x562231e47860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f577659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d20ce0 0x562231e47860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43138/0 (socket says 172.21.15.131:43138) 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f577659c640 1 -- 172.21.15.131:0/3756620093 learned_addr learned my addr 172.21.15.131:0/3756620093 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f577759e640 1 --2- 172.21.15.131:0/3756620093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d34940 0x562231e47da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f5776d9d640 1 --2- 172.21.15.131:0/3756620093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d1a9a0 0x562231e47320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f577659c640 1 -- 172.21.15.131:0/3756620093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d34940 msgr2=0x562231e47da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.730+0000 7f577659c640 1 --2- 172.21.15.131:0/3756620093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d34940 0x562231e47da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f577659c640 1 -- 172.21.15.131:0/3756620093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d1a9a0 msgr2=0x562231e47320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f577659c640 1 --2- 172.21.15.131:0/3756620093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d1a9a0 0x562231e47320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f577659c640 1 -- 172.21.15.131:0/3756620093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562231d19de0 con 0x562231d20ce0 2024-09-17T02:45:57.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f577659c640 1 --2- 172.21.15.131:0/3756620093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d20ce0 0x562231e47860 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5778143290 tx=0x7f576c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d435e31eeae36b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f5757fff640 1 -- 172.21.15.131:0/3756620093 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c0286b0 con 0x562231d20ce0 2024-09-17T02:45:57.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f5757fff640 1 -- 172.21.15.131:0/3756620093 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f576c028940 con 0x562231d20ce0 2024-09-17T02:45:57.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f5757fff640 1 -- 172.21.15.131:0/3756620093 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c028d30 con 0x562231d20ce0 2024-09-17T02:45:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 -- 172.21.15.131:0/3756620093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d20ce0 msgr2=0x562231e47860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 --2- 172.21.15.131:0/3756620093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d20ce0 0x562231e47860 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5778143290 tx=0x7f576c014000 comp rx=0 tx=0).stop 2024-09-17T02:45:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 -- 172.21.15.131:0/3756620093 shutdown_connections 2024-09-17T02:45:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 --2- 172.21.15.131:0/3756620093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d34940 0x562231e47da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 --2- 172.21.15.131:0/3756620093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d20ce0 0x562231e47860 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 --2- 172.21.15.131:0/3756620093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d1a9a0 0x562231e47320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 -- 172.21.15.131:0/3756620093 >> 172.21.15.131:0/3756620093 conn(0x562231953330 msgr2=0x562231e45c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 -- 172.21.15.131:0/3756620093 shutdown_connections 2024-09-17T02:45:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 -- 172.21.15.131:0/3756620093 wait complete. 2024-09-17T02:45:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 -- start start 2024-09-17T02:45:57.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 Processor -- start 2024-09-17T02:45:57.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d1a9a0 0x562231f09f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231f0a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d34940 0x562231f0a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562231c60f60 con 0x562231d34940 2024-09-17T02:45:57.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562231c60780 con 0x562231d1a9a0 2024-09-17T02:45:57.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562231c97fb0 con 0x562231d20ce0 2024-09-17T02:45:57.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f578445a000 10 client.0 _ll_register_callbacks cb 0x7f576805b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:57.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f577759e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d34940 0x562231f0a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f577759e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d34940 0x562231f0a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43142/0 (socket says 172.21.15.131:43142) 2024-09-17T02:45:57.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.734+0000 7f577759e640 1 -- 172.21.15.131:0/433724600 learned_addr learned my addr 172.21.15.131:0/433724600 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5776d9d640 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d1a9a0 0x562231f09f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577659c640 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231f0a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577759e640 1 -- 172.21.15.131:0/433724600 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 msgr2=0x562231f0a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577759e640 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231f0a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577759e640 1 -- 172.21.15.131:0/433724600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d1a9a0 msgr2=0x562231f09f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577759e640 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d1a9a0 0x562231f09f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577759e640 1 -- 172.21.15.131:0/433724600 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562231c2b640 con 0x562231d34940 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577759e640 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d34940 0x562231f0a9d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5770009bd0 tx=0x7f57700099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72201e0f6ce0aca server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5755ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5755ffb640 1 -- 172.21.15.131:0/433724600 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700b2050 con 0x562231d34940 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5755ffb640 1 -- 172.21.15.131:0/433724600 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f57700bfc80 con 0x562231d34940 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 -- 172.21.15.131:0/433724600 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f576c028940 con 0x562231d34940 2024-09-17T02:45:57.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5755ffb640 1 -- 172.21.15.131:0/433724600 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f57700bfe80 con 0x562231d34940 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 10 client.6705 fetch_fsmap learned FSMap version 53 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 -- 172.21.15.131:0/433724600 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x562231c5db10 con 0x562231d34940 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5755ffb640 1 -- 172.21.15.131:0/433724600 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f576c028940 con 0x562231d34940 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5755ffb640 1 -- 172.21.15.131:0/433724600 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f57700b5070 con 0x562231d34940 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 10 client.6705 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 10 client.6705 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 20 client.6705 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562231925b70 0x562231ec6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 4 client.6705 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 -- 172.21.15.131:0/433724600 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x562231ca3460 con 0x562231925b70 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5776d9d640 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562231925b70 0x562231ec6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5755ffb640 10 client.6705 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5776d9d640 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562231925b70 0x562231ec6260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dadc61d609cacb6 server_cookie=37c4d643588b3684 in_seq=0 out_seq=0 2024-09-17T02:45:57.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5755ffb640 1 -- 172.21.15.131:0/433724600 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562231ca3460 con 0x562231925b70 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5755ffb640 10 client.6705 handle_command_reply: tid=0 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 client.6705 shutdown 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 -- 172.21.15.131:0/433724600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d34940 msgr2=0x562231f0a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d34940 0x562231f0a9d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5770009bd0 tx=0x7f57700099a0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 -- 172.21.15.131:0/433724600 shutdown_connections 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562231925b70 0x562231ec6260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562231d34940 0x562231f0a9d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562231d20ce0 0x562231f0a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 --2- 172.21.15.131:0/433724600 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562231d1a9a0 0x562231f09f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f5755ffb640 0 client.6705 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 -- 172.21.15.131:0/433724600 >> 172.21.15.131:0/433724600 conn(0x562231953330 msgr2=0x562231ea4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 -- 172.21.15.131:0/433724600 shutdown_connections 2024-09-17T02:45:57.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 1 -- 172.21.15.131:0/433724600 wait complete. 2024-09-17T02:45:57.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f578445a000 20 client.6705 trim_cache size 0 max 16384 2024-09-17T02:45:57.741 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:45:57.741 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-09-17T02:45:57.741 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "7665c7ca-bac8-4303-b3bb-a75393928ae5", 2024-09-17T02:45:57.741 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-09-17T02:45:57.741 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:45:57.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 -- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5764073200 msgr2=0x7f5764075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5764073200 0x7f5764075710 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f57780b1b60 tx=0x7f576c005db0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 -- 172.21.15.131:0/1804191096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f577806f700 msgr2=0x7f5778151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f577806f700 0x7f5778151430 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f576800b3e0 tx=0x7f576800b8a0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 -- 172.21.15.131:0/1804191096 shutdown_connections 2024-09-17T02:45:57.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5764073200 0x7f5764075710 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f57780cbed0 0x7f57780ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f57780b2ef0 0x7f57780ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 --2- 172.21.15.131:0/1804191096 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f577806f700 0x7f5778151430 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 -- 172.21.15.131:0/1804191096 >> 172.21.15.131:0/1804191096 conn(0x7f57780d05a0 msgr2=0x7f5778153e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 -- 172.21.15.131:0/1804191096 shutdown_connections 2024-09-17T02:45:57.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.738+0000 7f577d18f640 1 -- 172.21.15.131:0/1804191096 wait complete. 2024-09-17T02:45:57.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:45:57.758 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-17T02:45:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.910+0000 7fdedfac1640 1 -- 172.21.15.131:0/166975939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec4003a40 msgr2=0x7fdec4003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.910+0000 7fdedfac1640 1 --2- 172.21.15.131:0/166975939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec4003a40 0x7fdec4003eb0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009b60 tx=0x7fdecc01d280 comp rx=0 tx=0).stop 2024-09-17T02:45:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.910+0000 7fdede05e640 1 -- 172.21.15.131:0/166975939 reap_dead start 2024-09-17T02:45:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.910+0000 7fdedfac1640 1 -- 172.21.15.131:0/166975939 shutdown_connections 2024-09-17T02:45:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.910+0000 7fdedfac1640 1 -- 172.21.15.131:0/166975939 >> 172.21.15.131:0/166975939 conn(0x7fded80cf0b0 msgr2=0x7fded80cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.910+0000 7fdedfac1640 1 -- 172.21.15.131:0/166975939 shutdown_connections 2024-09-17T02:45:57.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.910+0000 7fdedfac1640 1 -- 172.21.15.131:0/166975939 wait complete. 2024-09-17T02:45:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedfac1640 1 Processor -- start 2024-09-17T02:45:57.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedfac1640 1 -- start start 2024-09-17T02:45:57.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedfac1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec4003a40 0x7fded81f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedfac1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fded81556f0 0x7fded80b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedfac1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fded80b7330 0x7fded8161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedfac1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fded8167cc0 con 0x7fdec4003a40 2024-09-17T02:45:57.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedfac1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fded80cd260 con 0x7fded80b7330 2024-09-17T02:45:57.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedfac1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fded8167e40 con 0x7fded81556f0 2024-09-17T02:45:57.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedd85d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec4003a40 0x7fded81f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedd85d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec4003a40 0x7fded81f9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43172/0 (socket says 172.21.15.131:43172) 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedd85d640 1 -- 172.21.15.131:0/3502245170 learned_addr learned my addr 172.21.15.131:0/3502245170 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedd85d640 1 -- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fded81556f0 msgr2=0x7fded80b6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedd85d640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fded81556f0 0x7fded80b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedd85d640 1 -- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fded80b7330 msgr2=0x7fded8161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdede05e640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fded80b7330 0x7fded8161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedd85d640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fded80b7330 0x7fded8161450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedd85d640 1 -- 172.21.15.131:0/3502245170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded81f9e60 con 0x7fdec4003a40 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedd85d640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec4003a40 0x7fded81f9920 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fdecc01d0b0 tx=0x7fdecc0055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdeca7fc640 1 -- 172.21.15.131:0/3502245170 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0b1040 con 0x7fdec4003a40 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdeca7fc640 1 -- 172.21.15.131:0/3502245170 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdecc0bb070 con 0x7fdec4003a40 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedfac1640 1 -- 172.21.15.131:0/3502245170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded80b7740 con 0x7fdec4003a40 2024-09-17T02:45:57.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdeca7fc640 1 -- 172.21.15.131:0/3502245170 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc004380 con 0x7fdec4003a40 2024-09-17T02:45:57.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.914+0000 7fdedfac1640 1 -- 172.21.15.131:0/3502245170 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded80ca580 con 0x7fdec4003a40 2024-09-17T02:45:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.918+0000 7fdeca7fc640 1 -- 172.21.15.131:0/3502245170 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdecc0046d0 con 0x7fdec4003a40 2024-09-17T02:45:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.918+0000 7fdeca7fc640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdebc073230 0x7fdebc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.918+0000 7fdeca7fc640 1 -- 172.21.15.131:0/3502245170 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fdecc12f440 con 0x7fdec4003a40 2024-09-17T02:45:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.918+0000 7fdedd05c640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdebc073230 0x7fdebc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.918+0000 7fdee6d8c000 1 Processor -- start 2024-09-17T02:45:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.918+0000 7fdee6d8c000 1 -- start start 2024-09-17T02:45:57.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.918+0000 7fdedd05c640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdebc073230 0x7fdebc075740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fded81f6a90 tx=0x7fded4009580 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9d3b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9826890 0x55d9a9d3bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a98eaaa0 0x55d9a9d3c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d9a9bdacd0 con 0x55d9a9826890 2024-09-17T02:45:57.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9a9af0330 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d9a9b4dbb0 con 0x55d9a98eaaa0 2024-09-17T02:45:57.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdedd05c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9826890 0x55d9a9d3bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdedd05c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9826890 0x55d9a9d3bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43188/0 (socket says 172.21.15.131:43188) 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdedd05c640 1 -- 172.21.15.131:0/3771799556 learned_addr learned my addr 172.21.15.131:0/3771799556 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdede05e640 1 --2- 172.21.15.131:0/3771799556 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a98eaaa0 0x55d9a9d3c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdedd85d640 1 --2- 172.21.15.131:0/3771799556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9d3b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdedd05c640 1 -- 172.21.15.131:0/3771799556 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a98eaaa0 msgr2=0x55d9a9d3c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdedd05c640 1 --2- 172.21.15.131:0/3771799556 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a98eaaa0 0x55d9a9d3c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdedd05c640 1 -- 172.21.15.131:0/3771799556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9bcf6d0 msgr2=0x55d9a9d3b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdedd05c640 1 --2- 172.21.15.131:0/3771799556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9d3b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdedd05c640 1 -- 172.21.15.131:0/3771799556 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a9af1c80 con 0x55d9a9826890 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdedd05c640 1 --2- 172.21.15.131:0/3771799556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9826890 0x55d9a9d3bf30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fded40058d0 tx=0x7fded4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e281ae21d5bc8b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdebb7fe640 1 -- 172.21.15.131:0/3771799556 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4023070 con 0x55d9a9826890 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdebb7fe640 1 -- 172.21.15.131:0/3771799556 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fded4009710 con 0x55d9a9826890 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdebb7fe640 1 -- 172.21.15.131:0/3771799556 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded401f690 con 0x55d9a9826890 2024-09-17T02:45:57.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 -- 172.21.15.131:0/3771799556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9826890 msgr2=0x55d9a9d3bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/3771799556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9826890 0x55d9a9d3bf30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fded40058d0 tx=0x7fded4013000 comp rx=0 tx=0).stop 2024-09-17T02:45:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 -- 172.21.15.131:0/3771799556 shutdown_connections 2024-09-17T02:45:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/3771799556 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a98eaaa0 0x55d9a9d3c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/3771799556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9826890 0x55d9a9d3bf30 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/3771799556 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9d3b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 -- 172.21.15.131:0/3771799556 >> 172.21.15.131:0/3771799556 conn(0x55d9a97fa5c0 msgr2=0x55d9a9cf8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 -- 172.21.15.131:0/3771799556 shutdown_connections 2024-09-17T02:45:57.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 -- 172.21.15.131:0/3771799556 wait complete. 2024-09-17T02:45:57.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 -- start start 2024-09-17T02:45:57.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.922+0000 7fdee6d8c000 1 Processor -- start 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a9826890 0x55d9a9cf79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a98eaaa0 0x55d9a9cf7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cfe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d9a9b40df0 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9a9b3cfe0 con 0x55d9a98eaaa0 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d9a9ae24d0 con 0x55d9a9826890 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 10 client.0 _ll_register_callbacks cb 0x55d9a9ac07f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdede05e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cfe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdede05e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cfe5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43190/0 (socket says 172.21.15.131:43190) 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdede05e640 1 -- 172.21.15.131:0/2599335791 learned_addr learned my addr 172.21.15.131:0/2599335791 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdedd85d640 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a9826890 0x55d9a9cf79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdedd05c640 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a98eaaa0 0x55d9a9cf7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdede05e640 1 -- 172.21.15.131:0/2599335791 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a9826890 msgr2=0x55d9a9cf79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdede05e640 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a9826890 0x55d9a9cf79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdede05e640 1 -- 172.21.15.131:0/2599335791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a98eaaa0 msgr2=0x55d9a9cf7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdede05e640 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a98eaaa0 0x55d9a9cf7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdede05e640 1 -- 172.21.15.131:0/2599335791 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a9adb210 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdede05e640 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cfe5f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fded80cc830 tx=0x7fded000fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69e2f10fbd3c13 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdeb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2599335791 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0003d90 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2599335791 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fded0014a10 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2599335791 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d9a9ad5350 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2599335791 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fded0014c10 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 10 client.6717 fetch_fsmap learned FSMap version 53 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2599335791 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55d9a9ac1fc0 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2599335791 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d9a9ad5350 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2599335791 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fded0006190 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 10 client.6717 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 10 client.6717 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 20 client.6717 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d9a97df930 0x55d9a9d42e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 4 client.6717 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2599335791 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9a9b4fb00 con 0x55d9a97df930 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdedd85d640 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d9a97df930 0x55d9a9d42e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdeb8ff9640 10 client.6717 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdedd85d640 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d9a97df930 0x55d9a9d42e80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ee539aadac6e66d server_cookie=aeacd7d4f06a9fb in_seq=0 out_seq=0 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2599335791 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9a9b4fb00 con 0x55d9a97df930 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdeb8ff9640 10 client.6717 handle_command_reply: tid=0 2024-09-17T02:45:57.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 client.6717 shutdown 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2599335791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 msgr2=0x55d9a9cfe5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cfe5f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fded80cc830 tx=0x7fded000fa70 comp rx=0 tx=0).stop 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2599335791 shutdown_connections 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d9a97df930 0x55d9a9d42e80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cfe5f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a98eaaa0 0x55d9a9cf7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdeb8ff9640 0 client.6717 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2599335791 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a9826890 0x55d9a9cf79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2599335791 >> 172.21.15.131:0/2599335791 conn(0x55d9a97fa5c0 msgr2=0x55d9a9cf0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2599335791 shutdown_connections 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2599335791 wait complete. 2024-09-17T02:45:57.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.926+0000 7fdee6d8c000 20 client.6717 trim_cache size 0 max 16384 2024-09-17T02:45:57.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.946+0000 7fdee6d8c000 1 Processor -- start 2024-09-17T02:45:57.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.946+0000 7fdee6d8c000 1 -- start start 2024-09-17T02:45:57.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.946+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9826890 0x55d9a9dad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.946+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a98eaaa0 0x55d9a9dad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.946+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a9bcf6d0 0x55d9a9dadc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.946+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d9a9cf6b20 con 0x55d9a98eaaa0 2024-09-17T02:45:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.946+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fded8206760 con 0x55d9a9826890 2024-09-17T02:45:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.946+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fded82057c0 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.946+0000 7fdedd05c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a98eaaa0 0x55d9a9dad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.946+0000 7fdedd05c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a98eaaa0 0x55d9a9dad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43198/0 (socket says 172.21.15.131:43198) 2024-09-17T02:45:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.946+0000 7fdedd05c640 1 -- 172.21.15.131:0/3403376294 learned_addr learned my addr 172.21.15.131:0/3403376294 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdedd85d640 1 --2- 172.21.15.131:0/3403376294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9826890 0x55d9a9dad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdede05e640 1 --2- 172.21.15.131:0/3403376294 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a9bcf6d0 0x55d9a9dadc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdedd05c640 1 -- 172.21.15.131:0/3403376294 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a9bcf6d0 msgr2=0x55d9a9dadc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdedd05c640 1 --2- 172.21.15.131:0/3403376294 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a9bcf6d0 0x55d9a9dadc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdedd05c640 1 -- 172.21.15.131:0/3403376294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9826890 msgr2=0x55d9a9dad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdedd05c640 1 --2- 172.21.15.131:0/3403376294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9826890 0x55d9a9dad1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdedd05c640 1 -- 172.21.15.131:0/3403376294 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a9b9a680 con 0x55d9a98eaaa0 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdedd85d640 1 --2- 172.21.15.131:0/3403376294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9826890 0x55d9a9dad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdedd05c640 1 --2- 172.21.15.131:0/3403376294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a98eaaa0 0x55d9a9dad730 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fded4022000 tx=0x7fded4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bdd5097ae3c412 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdebaffd640 1 -- 172.21.15.131:0/3403376294 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4037070 con 0x55d9a98eaaa0 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdebaffd640 1 -- 172.21.15.131:0/3403376294 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fded40294e0 con 0x55d9a98eaaa0 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdebaffd640 1 -- 172.21.15.131:0/3403376294 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4023040 con 0x55d9a98eaaa0 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 -- 172.21.15.131:0/3403376294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a98eaaa0 msgr2=0x55d9a9dad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/3403376294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a98eaaa0 0x55d9a9dad730 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fded4022000 tx=0x7fded4004740 comp rx=0 tx=0).stop 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 -- 172.21.15.131:0/3403376294 shutdown_connections 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/3403376294 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a9bcf6d0 0x55d9a9dadc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/3403376294 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a98eaaa0 0x55d9a9dad730 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/3403376294 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9826890 0x55d9a9dad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 -- 172.21.15.131:0/3403376294 >> 172.21.15.131:0/3403376294 conn(0x55d9a97fa5c0 msgr2=0x55d9a9cfd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 -- 172.21.15.131:0/3403376294 shutdown_connections 2024-09-17T02:45:57.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 -- 172.21.15.131:0/3403376294 wait complete. 2024-09-17T02:45:57.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 -- start start 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 Processor -- start 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9826890 0x55d9a9dad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a98eaaa0 0x55d9a9dadb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cf2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fded8205170 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fded8204b20 con 0x55d9a9826890 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d9a9802680 con 0x55d9a98eaaa0 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdee6d8c000 10 client.0 _ll_register_callbacks cb 0x7fded00014e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdede05e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cf2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdede05e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cf2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43200/0 (socket says 172.21.15.131:43200) 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.950+0000 7fdede05e640 1 -- 172.21.15.131:0/2435172494 learned_addr learned my addr 172.21.15.131:0/2435172494 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedd05c640 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a98eaaa0 0x55d9a9dadb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedd85d640 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9826890 0x55d9a9dad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdede05e640 1 -- 172.21.15.131:0/2435172494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a98eaaa0 msgr2=0x55d9a9dadb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdede05e640 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a98eaaa0 0x55d9a9dadb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdede05e640 1 -- 172.21.15.131:0/2435172494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9826890 msgr2=0x55d9a9dad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdede05e640 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9826890 0x55d9a9dad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdede05e640 1 -- 172.21.15.131:0/2435172494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a9b070b0 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdede05e640 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cf2990 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fded0002240 tx=0x7fded0014e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a493d52fb8d5a43c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdeb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2435172494 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0003b90 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2435172494 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fded0003d30 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2435172494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fded40294e0 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2435172494 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fded0014200 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 10 client.6723 fetch_fsmap learned FSMap version 53 2024-09-17T02:45:57.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2435172494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55d9a9ad3610 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2435172494 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fded40294e0 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2435172494 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fded0038070 con 0x55d9a9bcf6d0 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 10 client.6723 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 10 client.6723 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 20 client.6723 populate_metadata read hostname 'smithi131' 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d9a97cd990 0x55d9a9cf3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 4 client.6723 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2435172494 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9a9b4b5b0 con 0x55d9a97cd990 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedd85d640 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d9a97cd990 0x55d9a9cf3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdeb8ff9640 10 client.6723 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedd85d640 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d9a97cd990 0x55d9a9cf3320 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e002c5ec2f976a server_cookie=800ec5eb138ec5d4 in_seq=0 out_seq=0 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdeb8ff9640 1 -- 172.21.15.131:0/2435172494 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+576 (crc 0 0 0) 0x55d9a9b4b5b0 con 0x55d9a97cd990 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdeb8ff9640 10 client.6723 handle_command_reply: tid=0 2024-09-17T02:45:57.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 client.6723 shutdown 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2435172494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 msgr2=0x55d9a9cf2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cf2990 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fded0002240 tx=0x7fded0014e30 comp rx=0 tx=0).stop 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2435172494 shutdown_connections 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d9a97cd990 0x55d9a9cf3320 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9a9bcf6d0 0x55d9a9cf2990 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdeb8ff9640 0 client.6723 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9a98eaaa0 0x55d9a9dadb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 --2- 172.21.15.131:0/2435172494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9a9826890 0x55d9a9dad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2435172494 >> 172.21.15.131:0/2435172494 conn(0x55d9a97fa5c0 msgr2=0x55d9a9d4dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2435172494 shutdown_connections 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 1 -- 172.21.15.131:0/2435172494 wait complete. 2024-09-17T02:45:57.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdee6d8c000 20 client.6723 trim_cache size 0 max 16384 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: "7665c7ca-bac8-4303-b3bb-a75393928ae5": { 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/subdir", 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "7665c7ca-bac8-4303-b3bb-a75393928ae5", 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive" 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: "7665c7ca-bac8-4303-b3bb-a75393928ae5": { 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:45:57.738298+0000", 2024-09-17T02:45:57.959 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/subdir", 2024-09-17T02:45:57.960 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:45:57.960 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:45:57.960 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:45:57.960 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:45:57.960 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:57.960 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:45:57.960 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:45:57.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 -- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdebc073230 msgr2=0x7fdebc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdebc073230 0x7fdebc075740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fded81f6a90 tx=0x7fded4009580 comp rx=0 tx=0).stop 2024-09-17T02:45:57.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 -- 172.21.15.131:0/3502245170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec4003a40 msgr2=0x7fded81f9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:45:57.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec4003a40 0x7fded81f9920 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fdecc01d0b0 tx=0x7fdecc0055d0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 -- 172.21.15.131:0/3502245170 shutdown_connections 2024-09-17T02:45:57.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdebc073230 0x7fdebc075740 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fded80b7330 0x7fded8161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fded81556f0 0x7fded80b6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 --2- 172.21.15.131:0/3502245170 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdec4003a40 0x7fded81f9920 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:45:57.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 -- 172.21.15.131:0/3502245170 >> 172.21.15.131:0/3502245170 conn(0x7fded80cf0b0 msgr2=0x7fded8166410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:45:57.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 -- 172.21.15.131:0/3502245170 shutdown_connections 2024-09-17T02:45:57.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:45:57.954+0000 7fdedfac1640 1 -- 172.21.15.131:0/3502245170 wait complete. 2024-09-17T02:45:57.973 INFO:tasks.cephfs.filesystem:scrub status for tag:7665c7ca-bac8-4303-b3bb-a75393928ae5 - {'path': '/subdir', 'tag': '7665c7ca-bac8-4303-b3bb-a75393928ae5', 'options': 'recursive'} 2024-09-17T02:46:14.460 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:46:14.463 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:46:27.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:46:27.975 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-17T02:46:28.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.146+0000 7fd321ddd640 1 -- 172.21.15.131:0/2591979450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd304003a60 msgr2=0x7fd304003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.146+0000 7fd321ddd640 1 --2- 172.21.15.131:0/2591979450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd304003a60 0x7fd304003ed0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fd30c00c290 tx=0x7fd30c01cf60 comp rx=0 tx=0).stop 2024-09-17T02:46:28.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.146+0000 7fd321ddd640 1 -- 172.21.15.131:0/2591979450 shutdown_connections 2024-09-17T02:46:28.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.146+0000 7fd321ddd640 1 --2- 172.21.15.131:0/2591979450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd304003a60 0x7fd304003ed0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.146+0000 7fd321ddd640 1 -- 172.21.15.131:0/2591979450 >> 172.21.15.131:0/2591979450 conn(0x7fd31c06eea0 msgr2=0x7fd31c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd321ddd640 1 -- 172.21.15.131:0/2591979450 shutdown_connections 2024-09-17T02:46:28.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd321ddd640 1 -- 172.21.15.131:0/2591979450 wait complete. 2024-09-17T02:46:28.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd321ddd640 1 Processor -- start 2024-09-17T02:46:28.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd321ddd640 1 -- start start 2024-09-17T02:46:28.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd321ddd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd304003a60 0x7fd31c1fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd321ddd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd31c151a80 0x7fd31c0afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd321ddd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd31c0b0230 0x7fd31c1e78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd321ddd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c15e7d0 con 0x7fd31c151a80 2024-09-17T02:46:28.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd321ddd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c073b50 con 0x7fd31c0b0230 2024-09-17T02:46:28.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd321ddd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c15e950 con 0x7fd304003a60 2024-09-17T02:46:28.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd31affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd31c151a80 0x7fd31c0afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd31affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd31c151a80 0x7fd31c0afcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52342/0 (socket says 172.21.15.131:52342) 2024-09-17T02:46:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd31affd640 1 -- 172.21.15.131:0/3022179704 learned_addr learned my addr 172.21.15.131:0/3022179704 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd31b7fe640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd304003a60 0x7fd31c1fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd31bfff640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd31c0b0230 0x7fd31c1e78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd31affd640 1 -- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd304003a60 msgr2=0x7fd31c1fa0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd31affd640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd304003a60 0x7fd31c1fa0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd31affd640 1 -- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd31c0b0230 msgr2=0x7fd31c1e78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd31affd640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd31c0b0230 0x7fd31c1e78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd31affd640 1 -- 172.21.15.131:0/3022179704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31c1fa630 con 0x7fd31c151a80 2024-09-17T02:46:28.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.150+0000 7fd31bfff640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd31c0b0230 0x7fd31c1e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:28.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd31affd640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd31c151a80 0x7fd31c0afcf0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd310004d50 tx=0x7fd310001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd303fff640 1 -- 172.21.15.131:0/3022179704 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3100107d0 con 0x7fd31c151a80 2024-09-17T02:46:28.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd303fff640 1 -- 172.21.15.131:0/3022179704 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd310016070 con 0x7fd31c151a80 2024-09-17T02:46:28.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd321ddd640 1 -- 172.21.15.131:0/3022179704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31c0b0640 con 0x7fd31c151a80 2024-09-17T02:46:28.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd303fff640 1 -- 172.21.15.131:0/3022179704 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310010d00 con 0x7fd31c151a80 2024-09-17T02:46:28.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd321ddd640 1 -- 172.21.15.131:0/3022179704 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31c1f72b0 con 0x7fd31c151a80 2024-09-17T02:46:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd303fff640 1 -- 172.21.15.131:0/3022179704 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd31001b460 con 0x7fd31c151a80 2024-09-17T02:46:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd303fff640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd308073200 0x7fd308075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd303fff640 1 -- 172.21.15.131:0/3022179704 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fd310094b60 con 0x7fd31c151a80 2024-09-17T02:46:28.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd31b7fe640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd308073200 0x7fd308075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd3290a8000 1 Processor -- start 2024-09-17T02:46:28.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.154+0000 7fd3290a8000 1 -- start start 2024-09-17T02:46:28.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31b7fe640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd308073200 0x7fd308075710 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0ab000 tx=0x7fd30c019da0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d8399959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839480890 0x55d839995f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839996470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d839834cd0 con 0x55d839544aa0 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d83974a330 con 0x55d839480890 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d8397a7bb0 con 0x55d8398296d0 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839996470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839996470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52348/0 (socket says 172.21.15.131:52348) 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31bfff640 1 -- 172.21.15.131:0/641552743 learned_addr learned my addr 172.21.15.131:0/641552743 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31affd640 1 --2- 172.21.15.131:0/641552743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839480890 0x55d839995f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31b7fe640 1 --2- 172.21.15.131:0/641552743 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d8399959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31bfff640 1 -- 172.21.15.131:0/641552743 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 msgr2=0x55d8399959f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31bfff640 1 --2- 172.21.15.131:0/641552743 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d8399959f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31bfff640 1 -- 172.21.15.131:0/641552743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839480890 msgr2=0x55d839995f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31bfff640 1 --2- 172.21.15.131:0/641552743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839480890 0x55d839995f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31bfff640 1 -- 172.21.15.131:0/641552743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d83974bc80 con 0x55d839544aa0 2024-09-17T02:46:28.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31affd640 1 --2- 172.21.15.131:0/641552743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839480890 0x55d839995f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:28.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd31bfff640 1 --2- 172.21.15.131:0/641552743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839996470 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd31c15b1e0 tx=0x7fd3140047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d818f22b3b256fa server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3017fa640 1 -- 172.21.15.131:0/641552743 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314004d30 con 0x55d839544aa0 2024-09-17T02:46:28.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3017fa640 1 -- 172.21.15.131:0/641552743 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3140099f0 con 0x55d839544aa0 2024-09-17T02:46:28.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3017fa640 1 -- 172.21.15.131:0/641552743 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd314009e00 con 0x55d839544aa0 2024-09-17T02:46:28.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 -- 172.21.15.131:0/641552743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 msgr2=0x55d839996470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 --2- 172.21.15.131:0/641552743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839996470 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd31c15b1e0 tx=0x7fd3140047a0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 -- 172.21.15.131:0/641552743 shutdown_connections 2024-09-17T02:46:28.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 --2- 172.21.15.131:0/641552743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839996470 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 --2- 172.21.15.131:0/641552743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839480890 0x55d839995f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 --2- 172.21.15.131:0/641552743 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d8399959f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 -- 172.21.15.131:0/641552743 >> 172.21.15.131:0/641552743 conn(0x55d8394545c0 msgr2=0x55d839952ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 -- 172.21.15.131:0/641552743 shutdown_connections 2024-09-17T02:46:28.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 -- 172.21.15.131:0/641552743 wait complete. 2024-09-17T02:46:28.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 -- start start 2024-09-17T02:46:28.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.158+0000 7fd3290a8000 1 Processor -- start 2024-09-17T02:46:28.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d8399519b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839544aa0 0x55d839951ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d8399585f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d83979adf0 con 0x55d839480890 2024-09-17T02:46:28.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d839796fe0 con 0x55d839544aa0 2024-09-17T02:46:28.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d83973c4d0 con 0x55d8398296d0 2024-09-17T02:46:28.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 10 client.0 _ll_register_callbacks cb 0x55d83971a7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:46:28.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d8399519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d8399519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52352/0 (socket says 172.21.15.131:52352) 2024-09-17T02:46:28.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31b7fe640 1 -- 172.21.15.131:0/1649189720 learned_addr learned my addr 172.21.15.131:0/1649189720 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31affd640 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839544aa0 0x55d839951ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31bfff640 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d8399585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31b7fe640 1 -- 172.21.15.131:0/1649189720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 msgr2=0x55d8399585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31b7fe640 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d8399585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31b7fe640 1 -- 172.21.15.131:0/1649189720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839544aa0 msgr2=0x55d839951ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31b7fe640 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839544aa0 0x55d839951ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31b7fe640 1 -- 172.21.15.131:0/1649189720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d839735210 con 0x55d839480890 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31bfff640 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d8399585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31affd640 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839544aa0 0x55d839951ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31b7fe640 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d8399519b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd30c01b830 tx=0x7fd30c0067b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c091d8004d6f6a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd2fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/1649189720 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0ca060 con 0x55d839480890 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/1649189720 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fd30c003cd0 con 0x55d839480890 2024-09-17T02:46:28.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 1 -- 172.21.15.131:0/1649189720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d83972f350 con 0x55d839480890 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/1649189720 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fd30c0d0050 con 0x55d839480890 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 10 client.6735 fetch_fsmap learned FSMap version 53 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 1 -- 172.21.15.131:0/1649189720 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55d83971bfc0 con 0x55d839480890 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/1649189720 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d83972f350 con 0x55d839480890 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/1649189720 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fd30c0d3020 con 0x55d839480890 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 10 client.6735 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 10 client.6735 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 20 client.6735 populate_metadata read hostname 'smithi131' 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d839439930 0x55d83999ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 4 client.6735 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 1 -- 172.21.15.131:0/1649189720 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8397a9b00 con 0x55d839439930 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31affd640 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d839439930 0x55d83999ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd2fa7fc640 10 client.6735 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:28.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd31affd640 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d839439930 0x55d83999ce80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae5474ef1c42da18 server_cookie=47c25ba7b316851f in_seq=0 out_seq=0 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/1649189720 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d8397a9b00 con 0x55d839439930 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd2fa7fc640 10 client.6735 handle_command_reply: tid=0 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.162+0000 7fd3290a8000 1 client.6735 shutdown 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd3290a8000 1 -- 172.21.15.131:0/1649189720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 msgr2=0x55d8399519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd3290a8000 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d8399519b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd30c01b830 tx=0x7fd30c0067b0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd3290a8000 1 -- 172.21.15.131:0/1649189720 shutdown_connections 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd3290a8000 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d839439930 0x55d83999ce80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd3290a8000 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d8399585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd3290a8000 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839544aa0 0x55d839951ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd2fa7fc640 0 client.6735 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd3290a8000 1 --2- 172.21.15.131:0/1649189720 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d8399519b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd3290a8000 1 -- 172.21.15.131:0/1649189720 >> 172.21.15.131:0/1649189720 conn(0x55d8394545c0 msgr2=0x55d83994ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd3290a8000 1 -- 172.21.15.131:0/1649189720 shutdown_connections 2024-09-17T02:46:28.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd3290a8000 1 -- 172.21.15.131:0/1649189720 wait complete. 2024-09-17T02:46:28.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.166+0000 7fd3290a8000 20 client.6735 trim_cache size 0 max 16384 2024-09-17T02:46:28.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.182+0000 7fd3290a8000 1 Processor -- start 2024-09-17T02:46:28.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.182+0000 7fd3290a8000 1 -- start start 2024-09-17T02:46:28.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839480890 0x55d839a0b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839a0bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d839a0c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d839950b20 con 0x55d839544aa0 2024-09-17T02:46:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c203ef0 con 0x55d839480890 2024-09-17T02:46:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd31c202ed0 con 0x55d8398296d0 2024-09-17T02:46:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd31affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839a0bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd31affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839a0bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52360/0 (socket says 172.21.15.131:52360) 2024-09-17T02:46:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd31affd640 1 -- 172.21.15.131:0/2545376757 learned_addr learned my addr 172.21.15.131:0/2545376757 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd31bfff640 1 --2- 172.21.15.131:0/2545376757 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d839a0c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd31b7fe640 1 --2- 172.21.15.131:0/2545376757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839480890 0x55d839a0b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd31affd640 1 -- 172.21.15.131:0/2545376757 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 msgr2=0x55d839a0c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd31affd640 1 --2- 172.21.15.131:0/2545376757 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d839a0c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd31affd640 1 -- 172.21.15.131:0/2545376757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839480890 msgr2=0x55d839a0b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd31affd640 1 --2- 172.21.15.131:0/2545376757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839480890 0x55d839a0b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd31affd640 1 -- 172.21.15.131:0/2545376757 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8397f4680 con 0x55d839544aa0 2024-09-17T02:46:28.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd31affd640 1 --2- 172.21.15.131:0/2545376757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839a0bb40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd310001240 tx=0x7fd310092d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5701b5490678f59 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd300ff9640 1 -- 172.21.15.131:0/2545376757 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31005fab0 con 0x55d839544aa0 2024-09-17T02:46:28.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd300ff9640 1 -- 172.21.15.131:0/2545376757 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd310068040 con 0x55d839544aa0 2024-09-17T02:46:28.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd300ff9640 1 -- 172.21.15.131:0/2545376757 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310063e90 con 0x55d839544aa0 2024-09-17T02:46:28.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 -- 172.21.15.131:0/2545376757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 msgr2=0x55d839a0bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 --2- 172.21.15.131:0/2545376757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839a0bb40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fd310001240 tx=0x7fd310092d80 comp rx=0 tx=0).stop 2024-09-17T02:46:28.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 -- 172.21.15.131:0/2545376757 shutdown_connections 2024-09-17T02:46:28.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 --2- 172.21.15.131:0/2545376757 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d839a0c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 --2- 172.21.15.131:0/2545376757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839544aa0 0x55d839a0bb40 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 --2- 172.21.15.131:0/2545376757 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839480890 0x55d839a0b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 -- 172.21.15.131:0/2545376757 >> 172.21.15.131:0/2545376757 conn(0x55d8394545c0 msgr2=0x55d8399575a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 -- 172.21.15.131:0/2545376757 shutdown_connections 2024-09-17T02:46:28.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 -- 172.21.15.131:0/2545376757 wait complete. 2024-09-17T02:46:28.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 -- start start 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.186+0000 7fd3290a8000 1 Processor -- start 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d839a0f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839544aa0 0x55d839a0fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d839a100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c202880 con 0x55d839480890 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd31c202380 con 0x55d839544aa0 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d83945c680 con 0x55d8398296d0 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 10 client.0 _ll_register_callbacks cb 0x7fd31c074c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d839a0f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d839a0f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52376/0 (socket says 172.21.15.131:52376) 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31b7fe640 1 -- 172.21.15.131:0/2152335434 learned_addr learned my addr 172.21.15.131:0/2152335434 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31affd640 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839544aa0 0x55d839a0fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31b7fe640 1 -- 172.21.15.131:0/2152335434 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 msgr2=0x55d839a100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31b7fe640 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d839a100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31b7fe640 1 -- 172.21.15.131:0/2152335434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839544aa0 msgr2=0x55d839a0fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31b7fe640 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839544aa0 0x55d839a0fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31b7fe640 1 -- 172.21.15.131:0/2152335434 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8397610b0 con 0x55d839480890 2024-09-17T02:46:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31b7fe640 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d839a0f640 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd30c001060 tx=0x7fd30c003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40dea2eeadbc6d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd2fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:46:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/2152335434 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0bd020 con 0x55d839480890 2024-09-17T02:46:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/2152335434 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fd30c0ae8d0 con 0x55d839480890 2024-09-17T02:46:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 -- 172.21.15.131:0/2152335434 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd310068040 con 0x55d839480890 2024-09-17T02:46:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/2152335434 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fd30c0dc050 con 0x55d839480890 2024-09-17T02:46:28.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 10 client.6741 fetch_fsmap learned FSMap version 53 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 -- 172.21.15.131:0/2152335434 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55d83972d610 con 0x55d839480890 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/2152335434 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd310068040 con 0x55d839480890 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/2152335434 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fd30c0df020 con 0x55d839480890 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 10 client.6741 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 10 client.6741 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 20 client.6741 populate_metadata read hostname 'smithi131' 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d839427990 0x55d83994d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 4 client.6741 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 -- 172.21.15.131:0/2152335434 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d8397a55b0 con 0x55d839427990 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31affd640 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d839427990 0x55d83994d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd2fa7fc640 10 client.6741 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd31affd640 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d839427990 0x55d83994d2c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca82bdb9711104d server_cookie=75b8d22f9afcd13a in_seq=0 out_seq=0 2024-09-17T02:46:28.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd2fa7fc640 1 -- 172.21.15.131:0/2152335434 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+380 (crc 0 0 0) 0x55d8397a55b0 con 0x55d839427990 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd2fa7fc640 10 client.6741 handle_command_reply: tid=0 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 client.6741 shutdown 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 -- 172.21.15.131:0/2152335434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 msgr2=0x55d839a0f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d839a0f640 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fd30c001060 tx=0x7fd30c003dd0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 -- 172.21.15.131:0/2152335434 shutdown_connections 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55d839427990 0x55d83994d2c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d8398296d0 0x55d839a100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd2fa7fc640 0 client.6741 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d839544aa0 0x55d839a0fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 --2- 172.21.15.131:0/2152335434 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d839480890 0x55d839a0f640 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 -- 172.21.15.131:0/2152335434 >> 172.21.15.131:0/2152335434 conn(0x55d8394545c0 msgr2=0x55d8399a7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 -- 172.21.15.131:0/2152335434 shutdown_connections 2024-09-17T02:46:28.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 1 -- 172.21.15.131:0/2152335434 wait complete. 2024-09-17T02:46:28.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd3290a8000 20 client.6741 trim_cache size 0 max 16384 2024-09-17T02:46:28.194 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:46:28.194 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-09-17T02:46:28.194 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {}, 2024-09-17T02:46:28.194 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:46:28.194 INFO:teuthology.orchestra.run.smithi131.stdout: "7665c7ca-bac8-4303-b3bb-a75393928ae5": { 2024-09-17T02:46:28.194 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:45:57.738298+0000", 2024-09-17T02:46:28.194 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/subdir", 2024-09-17T02:46:28.195 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:46:28.195 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:46:28.195 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:46:28.195 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:46:28.195 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:46:28.195 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:46:28.195 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:46:28.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 -- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd308073200 msgr2=0x7fd308075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd308073200 0x7fd308075710 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0ab000 tx=0x7fd30c019da0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 -- 172.21.15.131:0/3022179704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd31c151a80 msgr2=0x7fd31c0afcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd31c151a80 0x7fd31c0afcf0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fd310004d50 tx=0x7fd310001640 comp rx=0 tx=0).stop 2024-09-17T02:46:28.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 -- 172.21.15.131:0/3022179704 shutdown_connections 2024-09-17T02:46:28.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd308073200 0x7fd308075710 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd31c0b0230 0x7fd31c1e78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd31c151a80 0x7fd31c0afcf0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 --2- 172.21.15.131:0/3022179704 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd304003a60 0x7fd31c1fa0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 -- 172.21.15.131:0/3022179704 >> 172.21.15.131:0/3022179704 conn(0x7fd31c06eea0 msgr2=0x7fd31c0b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 -- 172.21.15.131:0/3022179704 shutdown_connections 2024-09-17T02:46:28.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.190+0000 7fd321ddd640 1 -- 172.21.15.131:0/3022179704 wait complete. 2024-09-17T02:46:28.209 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-17T02:46:28.209 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:46:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.358+0000 7ff0d79df640 1 -- 172.21.15.131:0/3356980230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0d0160780 msgr2=0x7ff0d00cdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.358+0000 7ff0d79df640 1 --2- 172.21.15.131:0/3356980230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0d0160780 0x7ff0d00cdb50 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009840 tx=0x7ff0c001c790 comp rx=0 tx=0).stop 2024-09-17T02:46:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.358+0000 7ff0d79df640 1 -- 172.21.15.131:0/3356980230 shutdown_connections 2024-09-17T02:46:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.358+0000 7ff0d79df640 1 --2- 172.21.15.131:0/3356980230 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0d006fd50 0x7ff0d0070160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.358+0000 7ff0d79df640 1 --2- 172.21.15.131:0/3356980230 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0d0160780 0x7ff0d00cdb50 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.358+0000 7ff0d79df640 1 -- 172.21.15.131:0/3356980230 >> 172.21.15.131:0/3356980230 conn(0x7ff0d00d07a0 msgr2=0x7ff0d00d0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.358+0000 7ff0d79df640 1 -- 172.21.15.131:0/3356980230 shutdown_connections 2024-09-17T02:46:28.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.358+0000 7ff0d79df640 1 -- 172.21.15.131:0/3356980230 wait complete. 2024-09-17T02:46:28.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d79df640 1 Processor -- start 2024-09-17T02:46:28.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d79df640 1 -- start start 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d79df640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0d006fd50 0x7ff0d00b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d79df640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0d0156c10 0x7ff0d0145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d79df640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0d01552a0 0x7ff0d015b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d79df640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d016a4b0 con 0x7ff0d006fd50 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d79df640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d016a330 con 0x7ff0d0156c10 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d79df640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d016a630 con 0x7ff0d01552a0 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d577b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0d006fd50 0x7ff0d00b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d577b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0d006fd50 0x7ff0d00b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52386/0 (socket says 172.21.15.131:52386) 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d577b640 1 -- 172.21.15.131:0/2056058653 learned_addr learned my addr 172.21.15.131:0/2056058653 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d4f7a640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0d0156c10 0x7ff0d0145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d5f7c640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0d01552a0 0x7ff0d015b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d577b640 1 -- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0d01552a0 msgr2=0x7ff0d015b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d577b640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0d01552a0 0x7ff0d015b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d577b640 1 -- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0d0156c10 msgr2=0x7ff0d0145f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d577b640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0d0156c10 0x7ff0d0145f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d577b640 1 -- 172.21.15.131:0/2056058653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d0155ee0 con 0x7ff0d006fd50 2024-09-17T02:46:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d5f7c640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0d01552a0 0x7ff0d015b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d577b640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0d006fd50 0x7ff0d00b4ff0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c3f0 tx=0x7ff0cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0c5ffb640 1 -- 172.21.15.131:0/2056058653 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc011790 con 0x7ff0d006fd50 2024-09-17T02:46:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0c5ffb640 1 -- 172.21.15.131:0/2056058653 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0cc017070 con 0x7ff0d006fd50 2024-09-17T02:46:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d79df640 1 -- 172.21.15.131:0/2056058653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d01561c0 con 0x7ff0d006fd50 2024-09-17T02:46:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0c5ffb640 1 -- 172.21.15.131:0/2056058653 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc011ce0 con 0x7ff0d006fd50 2024-09-17T02:46:28.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.362+0000 7ff0d79df640 1 -- 172.21.15.131:0/2056058653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d0146490 con 0x7ff0d006fd50 2024-09-17T02:46:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.366+0000 7ff0d79df640 1 -- 172.21.15.131:0/2056058653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d0156400 con 0x7ff0d006fd50 2024-09-17T02:46:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.366+0000 7ff0c5ffb640 1 -- 172.21.15.131:0/2056058653 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff0cc01c460 con 0x7ff0d006fd50 2024-09-17T02:46:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.366+0000 7ff0c5ffb640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff0b4073230 0x7ff0b4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.366+0000 7ff0c5ffb640 1 -- 172.21.15.131:0/2056058653 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7ff0cc0941a0 con 0x7ff0d006fd50 2024-09-17T02:46:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.366+0000 7ff0d4f7a640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff0b4073230 0x7ff0b4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.366+0000 7ff0d4f7a640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff0b4073230 0x7ff0b4075740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00af000 tx=0x7ff0c001afd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.370+0000 7ff0c5ffb640 1 -- 172.21.15.131:0/2056058653 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff0d0156400 con 0x7ff0d006fd50 2024-09-17T02:46:28.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.566+0000 7ff0d79df640 1 -- 172.21.15.131:0/2056058653 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0d0070160 con 0x7ff0d006fd50 2024-09-17T02:46:28.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.566+0000 7ff0c5ffb640 1 -- 172.21.15.131:0/2056058653 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5229 (secure 0 0 0) 0x7ff0cc05efe0 con 0x7ff0d006fd50 2024-09-17T02:46:28.565 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:46:28.565 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":53,"btime":"2024-09-17T02:45:53:304690+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:45:52.828057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6835/962976667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":962976667},{"type":"v1","addr":"172.21.15.131:6835","nonce":962976667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6609,"qdb_cluster":[6609]},"id":5}]} 2024-09-17T02:46:28.565 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 53 2024-09-17T02:46:28.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 -- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff0b4073230 msgr2=0x7ff0b4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff0b4073230 0x7ff0b4075740 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00af000 tx=0x7ff0c001afd0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 -- 172.21.15.131:0/2056058653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0d006fd50 msgr2=0x7ff0d00b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0d006fd50 0x7ff0d00b4ff0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c3f0 tx=0x7ff0cc00c8b0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 -- 172.21.15.131:0/2056058653 shutdown_connections 2024-09-17T02:46:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff0b4073230 0x7ff0b4075740 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0d01552a0 0x7ff0d015b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0d0156c10 0x7ff0d0145f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 --2- 172.21.15.131:0/2056058653 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0d006fd50 0x7ff0d00b4ff0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 -- 172.21.15.131:0/2056058653 >> 172.21.15.131:0/2056058653 conn(0x7ff0d00d07a0 msgr2=0x7ff0d0153540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 -- 172.21.15.131:0/2056058653 shutdown_connections 2024-09-17T02:46:28.568 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.570+0000 7ff0d79df640 1 -- 172.21.15.131:0/2056058653 wait complete. 2024-09-17T02:46:28.586 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-17T02:46:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 -- 172.21.15.131:0/2595256696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cfc004000 msgr2=0x7f1cfc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 --2- 172.21.15.131:0/2595256696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1cfc004000 0x7f1cfc004810 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0c70a0 tx=0x7f1d1401d530 comp rx=0 tx=0).stop 2024-09-17T02:46:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d2114a640 1 -- 172.21.15.131:0/2595256696 reap_dead start 2024-09-17T02:46:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 -- 172.21.15.131:0/2595256696 shutdown_connections 2024-09-17T02:46:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 -- 172.21.15.131:0/2595256696 >> 172.21.15.131:0/2595256696 conn(0x7f1d1c06eea0 msgr2=0x7f1d1c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 -- 172.21.15.131:0/2595256696 shutdown_connections 2024-09-17T02:46:28.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 -- 172.21.15.131:0/2595256696 wait complete. 2024-09-17T02:46:28.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 Processor -- start 2024-09-17T02:46:28.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 -- start start 2024-09-17T02:46:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1cfc004000 0x7f1d1c0c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d1c15f880 0x7f1d1c0b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1c0b52b0 0x7f1d1c0c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c167c50 con 0x7f1d1c0b52b0 2024-09-17T02:46:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1c167ad0 con 0x7f1d1c15f880 2024-09-17T02:46:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d22bad640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f1d1c167dd0 con 0x7f1cfc004000 2024-09-17T02:46:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d2114a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1c0b52b0 0x7f1d1c0c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d2114a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1c0b52b0 0x7f1d1c0c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52412/0 (socket says 172.21.15.131:52412) 2024-09-17T02:46:28.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d2114a640 1 -- 172.21.15.131:0/1813791662 learned_addr learned my addr 172.21.15.131:0/1813791662 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d20949640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1cfc004000 0x7f1d1c0c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d1bfff640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d1c15f880 0x7f1d1c0b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d2114a640 1 -- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1cfc004000 msgr2=0x7f1d1c0c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d2114a640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1cfc004000 0x7f1d1c0c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d2114a640 1 -- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d1c15f880 msgr2=0x7f1d1c0b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d2114a640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d1c15f880 0x7f1d1c0b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.750+0000 7f1d2114a640 1 -- 172.21.15.131:0/1813791662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c0b56c0 con 0x7f1d1c0b52b0 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.754+0000 7f1d2114a640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1c0b52b0 0x7f1d1c0c7080 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c073b10 tx=0x7f1d0400fa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.754+0000 7f1d197fa640 1 -- 172.21.15.131:0/1813791662 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04011910 con 0x7f1d1c0b52b0 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.754+0000 7f1d197fa640 1 -- 172.21.15.131:0/1813791662 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d04011ad0 con 0x7f1d1c0b52b0 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.754+0000 7f1d22bad640 1 -- 172.21.15.131:0/1813791662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1c0c9160 con 0x7f1d1c0b52b0 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.754+0000 7f1d197fa640 1 -- 172.21.15.131:0/1813791662 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0400b420 con 0x7f1d1c0b52b0 2024-09-17T02:46:28.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.754+0000 7f1d22bad640 1 -- 172.21.15.131:0/1813791662 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d1c1503e0 con 0x7f1d1c0b52b0 2024-09-17T02:46:28.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.754+0000 7f1d197fa640 1 -- 172.21.15.131:0/1813791662 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1d04019020 con 0x7f1d1c0b52b0 2024-09-17T02:46:28.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.754+0000 7f1d197fa640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d00073230 0x7f1d00075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.754+0000 7f1d197fa640 1 -- 172.21.15.131:0/1813791662 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f1d04094880 con 0x7f1d1c0b52b0 2024-09-17T02:46:28.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.754+0000 7f1d20949640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d00073230 0x7f1d00075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.754+0000 7f1d29e78000 1 Processor -- start 2024-09-17T02:46:28.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 -- start start 2024-09-17T02:46:28.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d20949640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d00073230 0x7f1d00075740 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c001870 tx=0x7f1d0c001ba0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663969b60 0x55c663aaf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c66395f1c0 0x55c663aaf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663948d90 0x55c663aafd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c66394dd00 con 0x55c663948d90 2024-09-17T02:46:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c6638c1c90 con 0x55c663969b60 2024-09-17T02:46:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55c6638b10c0 con 0x55c66395f1c0 2024-09-17T02:46:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d2114a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663948d90 0x55c663aafd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d2114a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663948d90 0x55c663aafd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52422/0 (socket says 172.21.15.131:52422) 2024-09-17T02:46:28.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d2114a640 1 -- 172.21.15.131:0/1116020553 learned_addr learned my addr 172.21.15.131:0/1116020553 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d1bfff640 1 --2- 172.21.15.131:0/1116020553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c66395f1c0 0x55c663aaf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d20949640 1 --2- 172.21.15.131:0/1116020553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663969b60 0x55c663aaf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d2114a640 1 -- 172.21.15.131:0/1116020553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c66395f1c0 msgr2=0x55c663aaf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d2114a640 1 --2- 172.21.15.131:0/1116020553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c66395f1c0 0x55c663aaf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d2114a640 1 -- 172.21.15.131:0/1116020553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663969b60 msgr2=0x55c663aaf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d2114a640 1 --2- 172.21.15.131:0/1116020553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663969b60 0x55c663aaf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d2114a640 1 -- 172.21.15.131:0/1116020553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c66388c290 con 0x55c663948d90 2024-09-17T02:46:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d2114a640 1 --2- 172.21.15.131:0/1116020553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663948d90 0x55c663aafd60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1d04001ec0 tx=0x7f1d0405ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f95d330cc9d415 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d127fc640 1 -- 172.21.15.131:0/1116020553 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04076070 con 0x55c663948d90 2024-09-17T02:46:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d127fc640 1 -- 172.21.15.131:0/1116020553 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d04000e50 con 0x55c663948d90 2024-09-17T02:46:28.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d127fc640 1 -- 172.21.15.131:0/1116020553 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04068750 con 0x55c663948d90 2024-09-17T02:46:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 -- 172.21.15.131:0/1116020553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663948d90 msgr2=0x55c663aafd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 --2- 172.21.15.131:0/1116020553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663948d90 0x55c663aafd60 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1d04001ec0 tx=0x7f1d0405ead0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 -- 172.21.15.131:0/1116020553 shutdown_connections 2024-09-17T02:46:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 --2- 172.21.15.131:0/1116020553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663948d90 0x55c663aafd60 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 --2- 172.21.15.131:0/1116020553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c66395f1c0 0x55c663aaf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 --2- 172.21.15.131:0/1116020553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663969b60 0x55c663aaf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 -- 172.21.15.131:0/1116020553 >> 172.21.15.131:0/1116020553 conn(0x55c66356e760 msgr2=0x55c663a92790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 -- 172.21.15.131:0/1116020553 shutdown_connections 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.758+0000 7f1d29e78000 1 -- 172.21.15.131:0/1116020553 wait complete. 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 -- start start 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 Processor -- start 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663948d90 0x55c663a6b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663a6b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663969b60 0x55c663a918d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c66384b940 con 0x55c66395f1c0 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c6638b4ed0 con 0x55c663948d90 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55c663576820 con 0x55c663969b60 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 10 client.0 _ll_register_callbacks cb 0x55c66388f7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663a6b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663a6b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52424/0 (socket says 172.21.15.131:52424) 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d1bfff640 1 -- 172.21.15.131:0/411810759 learned_addr learned my addr 172.21.15.131:0/411810759 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d20949640 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663948d90 0x55c663a6b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d2114a640 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663969b60 0x55c663a918d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d1bfff640 1 -- 172.21.15.131:0/411810759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663969b60 msgr2=0x55c663a918d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d1bfff640 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663969b60 0x55c663a918d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d1bfff640 1 -- 172.21.15.131:0/411810759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663948d90 msgr2=0x55c663a6b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d1bfff640 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663948d90 0x55c663a6b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d1bfff640 1 -- 172.21.15.131:0/411810759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c663864550 con 0x55c66395f1c0 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d1bfff640 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663a6b610 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1d140af040 tx=0x7f1d14009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59a46b210c8e0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/411810759 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d140bc070 con 0x55c66395f1c0 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 -- 172.21.15.131:0/411810759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c663849400 con 0x55c66395f1c0 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/411810759 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f1d14004d90 con 0x55c66395f1c0 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/411810759 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f1d140c6050 con 0x55c66395f1c0 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 10 client.6762 fetch_fsmap learned FSMap version 53 2024-09-17T02:46:28.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 -- 172.21.15.131:0/411810759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55c66384f480 con 0x55c66395f1c0 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/411810759 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c663849400 con 0x55c66395f1c0 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/411810759 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f1d140c9020 con 0x55c66395f1c0 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 10 client.6762 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 10 client.6762 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 20 client.6762 populate_metadata read hostname 'smithi131' 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55c663553ad0 0x55c663ac08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 4 client.6762 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d29e78000 1 -- 172.21.15.131:0/411810759 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6638c3be0 con 0x55c663553ad0 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d20949640 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55c663553ad0 0x55c663ac08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d20949640 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55c663553ad0 0x55c663ac08a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4774ea1efb9ad9db server_cookie=51954f39fcb4ef0c in_seq=0 out_seq=0 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.762+0000 7f1d0b7fe640 10 client.6762 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/411810759 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c6638c3be0 con 0x55c663553ad0 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d0b7fe640 10 client.6762 handle_command_reply: tid=0 2024-09-17T02:46:28.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 1 client.6762 shutdown 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 1 -- 172.21.15.131:0/411810759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 msgr2=0x55c663a6b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663a6b610 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1d140af040 tx=0x7f1d14009200 comp rx=0 tx=0).stop 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 1 -- 172.21.15.131:0/411810759 shutdown_connections 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55c663553ad0 0x55c663ac08a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663969b60 0x55c663a918d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663a6b610 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 1 --2- 172.21.15.131:0/411810759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663948d90 0x55c663a6b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 1 -- 172.21.15.131:0/411810759 >> 172.21.15.131:0/411810759 conn(0x55c66356e760 msgr2=0x55c663a63680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d0b7fe640 0 client.6762 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 1 -- 172.21.15.131:0/411810759 shutdown_connections 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 1 -- 172.21.15.131:0/411810759 wait complete. 2024-09-17T02:46:28.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.766+0000 7f1d29e78000 20 client.6762 trim_cache size 0 max 16384 2024-09-17T02:46:28.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.778+0000 7f1d29e78000 1 Processor -- start 2024-09-17T02:46:28.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.778+0000 7f1d29e78000 1 -- start start 2024-09-17T02:46:28.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.778+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663948d90 0x55c663a666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66395f1c0 0x55c663a66be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663969b60 0x55c663a69c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c663970fc0 con 0x55c663969b60 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c66385c300 con 0x55c66395f1c0 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55c6638c0a60 con 0x55c663948d90 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d2114a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663969b60 0x55c663a69c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d2114a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663969b60 0x55c663a69c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52432/0 (socket says 172.21.15.131:52432) 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d2114a640 1 -- 172.21.15.131:0/4135876630 learned_addr learned my addr 172.21.15.131:0/4135876630 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d20949640 1 --2- 172.21.15.131:0/4135876630 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663948d90 0x55c663a666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d2114a640 1 -- 172.21.15.131:0/4135876630 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663948d90 msgr2=0x55c663a666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d2114a640 1 --2- 172.21.15.131:0/4135876630 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663948d90 0x55c663a666a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d1bfff640 1 --2- 172.21.15.131:0/4135876630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66395f1c0 0x55c663a66be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d2114a640 1 -- 172.21.15.131:0/4135876630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66395f1c0 msgr2=0x55c663a66be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d2114a640 1 --2- 172.21.15.131:0/4135876630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66395f1c0 0x55c663a66be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d2114a640 1 -- 172.21.15.131:0/4135876630 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c663936d10 con 0x55c663969b60 2024-09-17T02:46:28.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d2114a640 1 --2- 172.21.15.131:0/4135876630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663969b60 0x55c663a69c40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1d04092c20 tx=0x7f1d04092aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc80802ffec1108b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d11ffb640 1 -- 172.21.15.131:0/4135876630 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0407f070 con 0x55c663969b60 2024-09-17T02:46:28.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d11ffb640 1 -- 172.21.15.131:0/4135876630 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d04000e50 con 0x55c663969b60 2024-09-17T02:46:28.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d11ffb640 1 -- 172.21.15.131:0/4135876630 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04076030 con 0x55c663969b60 2024-09-17T02:46:28.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 -- 172.21.15.131:0/4135876630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663969b60 msgr2=0x55c663a69c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 --2- 172.21.15.131:0/4135876630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663969b60 0x55c663a69c40 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f1d04092c20 tx=0x7f1d04092aa0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 -- 172.21.15.131:0/4135876630 shutdown_connections 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 --2- 172.21.15.131:0/4135876630 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c663969b60 0x55c663a69c40 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 --2- 172.21.15.131:0/4135876630 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c66395f1c0 0x55c663a66be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 --2- 172.21.15.131:0/4135876630 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663948d90 0x55c663a666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 -- 172.21.15.131:0/4135876630 >> 172.21.15.131:0/4135876630 conn(0x55c66356e760 msgr2=0x55c663a914c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 -- 172.21.15.131:0/4135876630 shutdown_connections 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 -- 172.21.15.131:0/4135876630 wait complete. 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 -- start start 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 Processor -- start 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663948d90 0x55c663b23c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663b24160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.782+0000 7f1d29e78000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663969b60 0x55c663b246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c66387d300 con 0x55c66395f1c0 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c66386e670 con 0x55c663948d90 2024-09-17T02:46:28.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55c6638b41e0 con 0x55c663969b60 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 10 client.0 _ll_register_callbacks cb 0x7f1d140af070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663b24160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d1bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663b24160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52440/0 (socket says 172.21.15.131:52440) 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d1bfff640 1 -- 172.21.15.131:0/1813229217 learned_addr learned my addr 172.21.15.131:0/1813229217 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d2114a640 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663969b60 0x55c663b246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d1bfff640 1 -- 172.21.15.131:0/1813229217 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663969b60 msgr2=0x55c663b246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d1bfff640 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663969b60 0x55c663b246a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d1bfff640 1 -- 172.21.15.131:0/1813229217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663948d90 msgr2=0x55c663b23c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d20949640 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663948d90 0x55c663b23c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d1bfff640 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663948d90 0x55c663b23c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d1bfff640 1 -- 172.21.15.131:0/1813229217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c663867e50 con 0x55c66395f1c0 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d1bfff640 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663b24160 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009de0 tx=0x7f1d14009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84d6964235252ea server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d0b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:46:28.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/1813229217 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d140d0060 con 0x55c66395f1c0 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/1813229217 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f1d14005700 con 0x55c66395f1c0 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 -- 172.21.15.131:0/1813229217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d04000e50 con 0x55c66395f1c0 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/1813229217 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f1d140bc070 con 0x55c66395f1c0 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 10 client.6768 fetch_fsmap learned FSMap version 53 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 -- 172.21.15.131:0/1813229217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55c6638562d0 con 0x55c66395f1c0 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/1813229217 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d04000e50 con 0x55c66395f1c0 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/1813229217 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f1d140c9070 con 0x55c66395f1c0 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 10 client.6768 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 10 client.6768 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 20 client.6768 populate_metadata read hostname 'smithi131' 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55c663541b30 0x55c663b2ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 4 client.6768 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 -- 172.21.15.131:0/1813229217 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55c6638bf690 con 0x55c663541b30 2024-09-17T02:46:28.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d20949640 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55c663541b30 0x55c663b2ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d0b7fe640 10 client.6768 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d20949640 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55c663541b30 0x55c663b2ce80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52fddb34648e9667 server_cookie=da5586e4e0c39da4 in_seq=0 out_seq=0 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d0b7fe640 1 -- 172.21.15.131:0/1813229217 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x55c6638bf690 con 0x55c663541b30 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d0b7fe640 10 client.6768 handle_command_reply: tid=0 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 client.6768 shutdown 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 -- 172.21.15.131:0/1813229217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 msgr2=0x55c663b24160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663b24160 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009de0 tx=0x7f1d14009e10 comp rx=0 tx=0).stop 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 -- 172.21.15.131:0/1813229217 shutdown_connections 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55c663541b30 0x55c663b2ce80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c663969b60 0x55c663b246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c66395f1c0 0x55c663b24160 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 --2- 172.21.15.131:0/1813229217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c663948d90 0x55c663b23c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d0b7fe640 0 client.6768 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:28.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 -- 172.21.15.131:0/1813229217 >> 172.21.15.131:0/1813229217 conn(0x55c66356e760 msgr2=0x55c663a65690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 -- 172.21.15.131:0/1813229217 shutdown_connections 2024-09-17T02:46:28.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 1 -- 172.21.15.131:0/1813229217 wait complete. 2024-09-17T02:46:28.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d29e78000 20 client.6768 trim_cache size 0 max 16384 2024-09-17T02:46:28.789 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:46:28.789 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:46:28.789 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:46:28.789 INFO:teuthology.orchestra.run.smithi131.stdout: "damage_type": "dentry", 2024-09-17T02:46:28.789 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 2137140783, 2024-09-17T02:46:28.789 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 1099511627776, 2024-09-17T02:46:28.789 INFO:teuthology.orchestra.run.smithi131.stdout: "frag": "*", 2024-09-17T02:46:28.790 INFO:teuthology.orchestra.run.smithi131.stdout: "dname": "file_to_be_damaged", 2024-09-17T02:46:28.790 INFO:teuthology.orchestra.run.smithi131.stdout: "snap_id": "head", 2024-09-17T02:46:28.790 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/subdir/file_to_be_damaged" 2024-09-17T02:46:28.790 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:46:28.790 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:46:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 -- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d00073230 msgr2=0x7f1d00075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d00073230 0x7f1d00075740 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c001870 tx=0x7f1d0c001ba0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 -- 172.21.15.131:0/1813791662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1c0b52b0 msgr2=0x7f1d1c0c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1c0b52b0 0x7f1d1c0c7080 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c073b10 tx=0x7f1d0400fa90 comp rx=0 tx=0).stop 2024-09-17T02:46:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 -- 172.21.15.131:0/1813791662 shutdown_connections 2024-09-17T02:46:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1d00073230 0x7f1d00075740 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1d1c0b52b0 0x7f1d1c0c7080 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1d1c15f880 0x7f1d1c0b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 --2- 172.21.15.131:0/1813791662 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1cfc004000 0x7f1d1c0c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 -- 172.21.15.131:0/1813791662 >> 172.21.15.131:0/1813791662 conn(0x7f1d1c06eea0 msgr2=0x7f1d1c15a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 -- 172.21.15.131:0/1813791662 shutdown_connections 2024-09-17T02:46:28.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.786+0000 7f1d22bad640 1 -- 172.21.15.131:0/1813791662 wait complete. 2024-09-17T02:46:28.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T02:46:28.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.950+0000 7fda627ef640 1 -- 172.21.15.131:0/568382037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda44003a40 msgr2=0x7fda44003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.950+0000 7fda627ef640 1 --2- 172.21.15.131:0/568382037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda44003a40 0x7fda44003eb0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0016f0 tx=0x7fda4c01c7b0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.950+0000 7fda60d8c640 1 -- 172.21.15.131:0/568382037 reap_dead start 2024-09-17T02:46:28.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.950+0000 7fda627ef640 1 -- 172.21.15.131:0/568382037 shutdown_connections 2024-09-17T02:46:28.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.950+0000 7fda627ef640 1 -- 172.21.15.131:0/568382037 >> 172.21.15.131:0/568382037 conn(0x7fda5c06eea0 msgr2=0x7fda5c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:28.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.950+0000 7fda627ef640 1 -- 172.21.15.131:0/568382037 shutdown_connections 2024-09-17T02:46:28.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.950+0000 7fda627ef640 1 -- 172.21.15.131:0/568382037 wait complete. 2024-09-17T02:46:28.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.950+0000 7fda627ef640 1 Processor -- start 2024-09-17T02:46:28.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.950+0000 7fda627ef640 1 -- start start 2024-09-17T02:46:28.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda627ef640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda44003a40 0x7fda5c15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda627ef640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda5c15e610 0x7fda5c147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda627ef640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda5c1644b0 0x7fda5c1648c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda627ef640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c167560 con 0x7fda5c15e610 2024-09-17T02:46:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda627ef640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda5c1673e0 con 0x7fda5c1644b0 2024-09-17T02:46:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda627ef640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fda5c1676e0 con 0x7fda44003a40 2024-09-17T02:46:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda5b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda5c15e610 0x7fda5c147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda5b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda5c15e610 0x7fda5c147420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52460/0 (socket says 172.21.15.131:52460) 2024-09-17T02:46:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda5b7fe640 1 -- 172.21.15.131:0/2999618900 learned_addr learned my addr 172.21.15.131:0/2999618900 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda5bfff640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda44003a40 0x7fda5c15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda60d8c640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda5c1644b0 0x7fda5c1648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda5b7fe640 1 -- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda44003a40 msgr2=0x7fda5c15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda5b7fe640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda44003a40 0x7fda5c15a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda5b7fe640 1 -- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda5c1644b0 msgr2=0x7fda5c1648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:28.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda5b7fe640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda5c1644b0 0x7fda5c1648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:28.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda5b7fe640 1 -- 172.21.15.131:0/2999618900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda5c164e00 con 0x7fda5c15e610 2024-09-17T02:46:28.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda5bfff640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda44003a40 0x7fda5c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:28.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda60d8c640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda5c1644b0 0x7fda5c1648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:28.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda5b7fe640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda5c15e610 0x7fda5c147420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fda50000c10 tx=0x7fda500049e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda58ff9640 1 -- 172.21.15.131:0/2999618900 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda50015020 con 0x7fda5c15e610 2024-09-17T02:46:28.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda58ff9640 1 -- 172.21.15.131:0/2999618900 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda50011a20 con 0x7fda5c15e610 2024-09-17T02:46:28.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda627ef640 1 -- 172.21.15.131:0/2999618900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda5c147960 con 0x7fda5c15e610 2024-09-17T02:46:28.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda58ff9640 1 -- 172.21.15.131:0/2999618900 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda500103c0 con 0x7fda5c15e610 2024-09-17T02:46:28.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda627ef640 1 -- 172.21.15.131:0/2999618900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5c1f8240 con 0x7fda5c15e610 2024-09-17T02:46:28.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda627ef640 1 -- 172.21.15.131:0/2999618900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda5c075b80 con 0x7fda5c15e610 2024-09-17T02:46:28.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.954+0000 7fda58ff9640 1 -- 172.21.15.131:0/2999618900 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fda50018020 con 0x7fda5c15e610 2024-09-17T02:46:28.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.958+0000 7fda58ff9640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda48073260 0x7fda48075770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:28.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.958+0000 7fda58ff9640 1 -- 172.21.15.131:0/2999618900 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fda50095010 con 0x7fda5c15e610 2024-09-17T02:46:28.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.958+0000 7fda5bfff640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda48073260 0x7fda48075770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:28.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.958+0000 7fda5bfff640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda48073260 0x7fda48075770 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0016c0 tx=0x7fda4c0195a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:28.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:28.962+0000 7fda58ff9640 1 -- 172.21.15.131:0/2999618900 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda5001f020 con 0x7fda5c15e610 2024-09-17T02:46:29.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 -- 172.21.15.131:0/2999618900 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fda5c15bea0 con 0x7fda5c15e610 2024-09-17T02:46:29.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda58ff9640 1 -- 172.21.15.131:0/2999618900 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fda5c164e00 con 0x7fda5c15e610 2024-09-17T02:46:29.151 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:46:29.151 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-09-17T02:46:29.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 -- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda48073260 msgr2=0x7fda48075770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda48073260 0x7fda48075770 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0016c0 tx=0x7fda4c0195a0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 -- 172.21.15.131:0/2999618900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda5c15e610 msgr2=0x7fda5c147420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda5c15e610 0x7fda5c147420 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fda50000c10 tx=0x7fda500049e0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 -- 172.21.15.131:0/2999618900 shutdown_connections 2024-09-17T02:46:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda48073260 0x7fda48075770 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda5c1644b0 0x7fda5c1648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda5c15e610 0x7fda5c147420 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 --2- 172.21.15.131:0/2999618900 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda44003a40 0x7fda5c15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 -- 172.21.15.131:0/2999618900 >> 172.21.15.131:0/2999618900 conn(0x7fda5c06eea0 msgr2=0x7fda5c0b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 -- 172.21.15.131:0/2999618900 shutdown_connections 2024-09-17T02:46:29.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.154+0000 7fda627ef640 1 -- 172.21.15.131:0/2999618900 wait complete. 2024-09-17T02:46:29.172 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T02:46:29.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:46:29.172 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-09-17T02:46:29.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.330+0000 7fdb29e79640 1 -- 172.21.15.131:0/246351557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb0c004000 msgr2=0x7fdb0c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.330+0000 7fdb29e79640 1 --2- 172.21.15.131:0/246351557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb0c004000 0x7fdb0c004810 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fdb240b4830 tx=0x7fdb1801d250 comp rx=0 tx=0).stop 2024-09-17T02:46:29.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.330+0000 7fdb23fff640 1 -- 172.21.15.131:0/246351557 reap_dead start 2024-09-17T02:46:29.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.330+0000 7fdb29e79640 1 -- 172.21.15.131:0/246351557 shutdown_connections 2024-09-17T02:46:29.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.330+0000 7fdb29e79640 1 -- 172.21.15.131:0/246351557 >> 172.21.15.131:0/246351557 conn(0x7fdb240d05a0 msgr2=0x7fdb240d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.330+0000 7fdb29e79640 1 -- 172.21.15.131:0/246351557 shutdown_connections 2024-09-17T02:46:29.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.330+0000 7fdb29e79640 1 -- 172.21.15.131:0/246351557 wait complete. 2024-09-17T02:46:29.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb29e79640 1 Processor -- start 2024-09-17T02:46:29.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb29e79640 1 -- start start 2024-09-17T02:46:29.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb29e79640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb241512d0 0x7fdb241616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb29e79640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb24147150 0x7fdb2406fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb29e79640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2414c660 0x7fdb2414ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb29e79640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24166360 con 0x7fdb2414c660 2024-09-17T02:46:29.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb29e79640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb241661e0 con 0x7fdb24147150 2024-09-17T02:46:29.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb29e79640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdb241664e0 con 0x7fdb241512d0 2024-09-17T02:46:29.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2414c660 0x7fdb2414ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2414c660 0x7fdb2414ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52482/0 (socket says 172.21.15.131:52482) 2024-09-17T02:46:29.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb23fff640 1 -- 172.21.15.131:0/1669376280 learned_addr learned my addr 172.21.15.131:0/1669376280 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:29.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb237fe640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb241512d0 0x7fdb241616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb22ffd640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb24147150 0x7fdb2406fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb23fff640 1 -- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb241512d0 msgr2=0x7fdb241616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb23fff640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb241512d0 0x7fdb241616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb23fff640 1 -- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb24147150 msgr2=0x7fdb2406fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb23fff640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb24147150 0x7fdb2406fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb23fff640 1 -- 172.21.15.131:0/1669376280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2414cfb0 con 0x7fdb2414c660 2024-09-17T02:46:29.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.334+0000 7fdb23fff640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2414c660 0x7fdb2414ca70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdb24070e70 tx=0x7fdb1c00fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb0bfff640 1 -- 172.21.15.131:0/1669376280 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00dce0 con 0x7fdb2414c660 2024-09-17T02:46:29.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb0bfff640 1 -- 172.21.15.131:0/1669376280 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb1c003980 con 0x7fdb2414c660 2024-09-17T02:46:29.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb29e79640 1 -- 172.21.15.131:0/1669376280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb24161d30 con 0x7fdb2414c660 2024-09-17T02:46:29.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb0bfff640 1 -- 172.21.15.131:0/1669376280 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c003dd0 con 0x7fdb2414c660 2024-09-17T02:46:29.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb29e79640 1 -- 172.21.15.131:0/1669376280 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb241622f0 con 0x7fdb2414c660 2024-09-17T02:46:29.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb0bfff640 1 -- 172.21.15.131:0/1669376280 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdb1c004030 con 0x7fdb2414c660 2024-09-17T02:46:29.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb0bfff640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb10073230 0x7fdb10075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb0bfff640 1 -- 172.21.15.131:0/1669376280 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fdb1c094a30 con 0x7fdb2414c660 2024-09-17T02:46:29.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb237fe640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb10073230 0x7fdb10075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb31144000 1 Processor -- start 2024-09-17T02:46:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb31144000 1 -- start start 2024-09-17T02:46:29.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.338+0000 7fdb237fe640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb10073230 0x7fdb10075740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdb14001340 tx=0x7fdb14000ee0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f25e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dd5a40 0x562d95f263a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562d95d38bd0 con 0x562d95dbf640 2024-09-17T02:46:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562d95d28000 con 0x562d95de4ae0 2024-09-17T02:46:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562d95d2be10 con 0x562d95dd5a40 2024-09-17T02:46:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb23fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f268e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52494/0 (socket says 172.21.15.131:52494) 2024-09-17T02:46:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb23fff640 1 -- 172.21.15.131:0/3710978362 learned_addr learned my addr 172.21.15.131:0/3710978362 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb237fe640 1 --2- 172.21.15.131:0/3710978362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f25e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb22ffd640 1 --2- 172.21.15.131:0/3710978362 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dd5a40 0x562d95f263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb23fff640 1 -- 172.21.15.131:0/3710978362 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dd5a40 msgr2=0x562d95f263a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb23fff640 1 --2- 172.21.15.131:0/3710978362 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dd5a40 0x562d95f263a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb23fff640 1 -- 172.21.15.131:0/3710978362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 msgr2=0x562d95f25e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb23fff640 1 --2- 172.21.15.131:0/3710978362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f25e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb23fff640 1 -- 172.21.15.131:0/3710978362 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d95d034c0 con 0x562d95dbf640 2024-09-17T02:46:29.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb23fff640 1 --2- 172.21.15.131:0/3710978362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f268e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c010010 tx=0x7fdb1c05f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370a00cd874e1fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb097fa640 1 -- 172.21.15.131:0/3710978362 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c020070 con 0x562d95dbf640 2024-09-17T02:46:29.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb097fa640 1 -- 172.21.15.131:0/3710978362 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb1c063d70 con 0x562d95dbf640 2024-09-17T02:46:29.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb097fa640 1 -- 172.21.15.131:0/3710978362 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c0662d0 con 0x562d95dbf640 2024-09-17T02:46:29.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 -- 172.21.15.131:0/3710978362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 msgr2=0x562d95f268e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 --2- 172.21.15.131:0/3710978362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f268e0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c010010 tx=0x7fdb1c05f210 comp rx=0 tx=0).stop 2024-09-17T02:46:29.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 -- 172.21.15.131:0/3710978362 shutdown_connections 2024-09-17T02:46:29.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 --2- 172.21.15.131:0/3710978362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f268e0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 --2- 172.21.15.131:0/3710978362 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dd5a40 0x562d95f263a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 --2- 172.21.15.131:0/3710978362 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f25e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 -- 172.21.15.131:0/3710978362 >> 172.21.15.131:0/3710978362 conn(0x562d959e4b30 msgr2=0x562d95edae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 -- 172.21.15.131:0/3710978362 shutdown_connections 2024-09-17T02:46:29.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.342+0000 7fdb31144000 1 -- 172.21.15.131:0/3710978362 wait complete. 2024-09-17T02:46:29.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 -- start start 2024-09-17T02:46:29.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 Processor -- start 2024-09-17T02:46:29.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dbf640 0x562d95f11420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f11960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f11ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562d95de3a30 con 0x562d95dd5a40 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562d95cd1b50 con 0x562d95de4ae0 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562d95cce9b0 con 0x562d95dbf640 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 10 client.0 _ll_register_callbacks cb 0x562d95ddecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb22ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f11960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb22ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f11960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52508/0 (socket says 172.21.15.131:52508) 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb22ffd640 1 -- 172.21.15.131:0/1215793800 learned_addr learned my addr 172.21.15.131:0/1215793800 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb23fff640 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f11ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb237fe640 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dbf640 0x562d95f11420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb22ffd640 1 -- 172.21.15.131:0/1215793800 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dbf640 msgr2=0x562d95f11420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb22ffd640 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dbf640 0x562d95f11420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb22ffd640 1 -- 172.21.15.131:0/1215793800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 msgr2=0x562d95f11ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb22ffd640 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f11ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb22ffd640 1 -- 172.21.15.131:0/1215793800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d95d71620 con 0x562d95dd5a40 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb237fe640 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dbf640 0x562d95f11420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb23fff640 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f11ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb22ffd640 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f11960 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdb180af040 tx=0x7fdb180054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8b6a5b6bbc6197 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb027fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb027fc640 1 -- 172.21.15.131:0/1215793800 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb180bb030 con 0x562d95dd5a40 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb027fc640 1 -- 172.21.15.131:0/1215793800 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fdb180055a0 con 0x562d95dd5a40 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 -- 172.21.15.131:0/1215793800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562d95cc0260 con 0x562d95dd5a40 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb027fc640 1 -- 172.21.15.131:0/1215793800 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fdb180c7050 con 0x562d95dd5a40 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 10 client.6786 fetch_fsmap learned FSMap version 53 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 -- 172.21.15.131:0/1215793800 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x562d95cdb370 con 0x562d95dd5a40 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb027fc640 1 -- 172.21.15.131:0/1215793800 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562d95cc0260 con 0x562d95dd5a40 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb027fc640 1 -- 172.21.15.131:0/1215793800 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fdb180ca020 con 0x562d95dd5a40 2024-09-17T02:46:29.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 10 client.6786 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 10 client.6786 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 20 client.6786 populate_metadata read hostname 'smithi131' 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562d959cabd0 0x562d95f128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 4 client.6786 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb31144000 1 -- 172.21.15.131:0/1215793800 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d95d3ab20 con 0x562d959cabd0 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb237fe640 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562d959cabd0 0x562d95f128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.346+0000 7fdb027fc640 10 client.6786 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb237fe640 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562d959cabd0 0x562d95f128b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b8d5e0b4dfd6fbd server_cookie=df64069fc2513fe2 in_seq=0 out_seq=0 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb027fc640 1 -- 172.21.15.131:0/1215793800 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562d95d3ab20 con 0x562d959cabd0 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb027fc640 10 client.6786 handle_command_reply: tid=0 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 1 client.6786 shutdown 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 1 -- 172.21.15.131:0/1215793800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 msgr2=0x562d95f11960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f11960 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdb180af040 tx=0x7fdb180054b0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 1 -- 172.21.15.131:0/1215793800 shutdown_connections 2024-09-17T02:46:29.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562d959cabd0 0x562d95f128b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f11ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb027fc640 0 client.6786 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:29.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f11960 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 1 --2- 172.21.15.131:0/1215793800 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dbf640 0x562d95f11420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 1 -- 172.21.15.131:0/1215793800 >> 172.21.15.131:0/1215793800 conn(0x562d959e4b30 msgr2=0x562d95edb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 1 -- 172.21.15.131:0/1215793800 shutdown_connections 2024-09-17T02:46:29.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 1 -- 172.21.15.131:0/1215793800 wait complete. 2024-09-17T02:46:29.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.350+0000 7fdb31144000 20 client.6786 trim_cache size 0 max 16384 2024-09-17T02:46:29.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb31144000 1 Processor -- start 2024-09-17T02:46:29.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb31144000 1 -- start start 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb31144000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dbf640 0x562d95ee2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb31144000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f105a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb31144000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f10ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb31144000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24204220 con 0x562d95dd5a40 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb31144000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb24203280 con 0x562d95de4ae0 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb31144000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdb24202c30 con 0x562d95dbf640 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb22ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb22ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f105a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52516/0 (socket says 172.21.15.131:52516) 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb22ffd640 1 -- 172.21.15.131:0/464110878 learned_addr learned my addr 172.21.15.131:0/464110878 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb22ffd640 1 -- 172.21.15.131:0/464110878 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dbf640 msgr2=0x562d95ee2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb22ffd640 1 --2- 172.21.15.131:0/464110878 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dbf640 0x562d95ee2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb22ffd640 1 -- 172.21.15.131:0/464110878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 msgr2=0x562d95f10ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb22ffd640 1 --2- 172.21.15.131:0/464110878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f10ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.366+0000 7fdb22ffd640 1 -- 172.21.15.131:0/464110878 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d95cc6120 con 0x562d95dd5a40 2024-09-17T02:46:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb22ffd640 1 --2- 172.21.15.131:0/464110878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f105a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdb18000c10 tx=0x7fdb180056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396042a557eeb27e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb08ff9640 1 -- 172.21.15.131:0/464110878 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb180bb040 con 0x562d95dd5a40 2024-09-17T02:46:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb08ff9640 1 -- 172.21.15.131:0/464110878 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb18000e50 con 0x562d95dd5a40 2024-09-17T02:46:29.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb08ff9640 1 -- 172.21.15.131:0/464110878 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb180b1040 con 0x562d95dd5a40 2024-09-17T02:46:29.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 -- 172.21.15.131:0/464110878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 msgr2=0x562d95f105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 --2- 172.21.15.131:0/464110878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f105a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdb18000c10 tx=0x7fdb180056d0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 -- 172.21.15.131:0/464110878 shutdown_connections 2024-09-17T02:46:29.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 --2- 172.21.15.131:0/464110878 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95de4ae0 0x562d95f10ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 --2- 172.21.15.131:0/464110878 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dd5a40 0x562d95f105a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 --2- 172.21.15.131:0/464110878 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95dbf640 0x562d95ee2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 -- 172.21.15.131:0/464110878 >> 172.21.15.131:0/464110878 conn(0x562d959e4b30 msgr2=0x562d95f35410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 -- 172.21.15.131:0/464110878 shutdown_connections 2024-09-17T02:46:29.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 -- 172.21.15.131:0/464110878 wait complete. 2024-09-17T02:46:29.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 -- start start 2024-09-17T02:46:29.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 Processor -- start 2024-09-17T02:46:29.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f9f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.372 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95dd5a40 0x562d95f9f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95de4ae0 0x562d95f9fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562d95d2b120 con 0x562d95dbf640 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562d95cf5ab0 con 0x562d95dd5a40 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562d95d30440 con 0x562d95de4ae0 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb31144000 10 client.0 _ll_register_callbacks cb 0x7fdb180af070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb237fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb237fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f9f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52530/0 (socket says 172.21.15.131:52530) 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb237fe640 1 -- 172.21.15.131:0/2086876994 learned_addr learned my addr 172.21.15.131:0/2086876994 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb23fff640 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95de4ae0 0x562d95f9fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb237fe640 1 -- 172.21.15.131:0/2086876994 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95de4ae0 msgr2=0x562d95f9fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb237fe640 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95de4ae0 0x562d95f9fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb237fe640 1 -- 172.21.15.131:0/2086876994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95dd5a40 msgr2=0x562d95f9f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb237fe640 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95dd5a40 0x562d95f9f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb237fe640 1 -- 172.21.15.131:0/2086876994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d95dad2b0 con 0x562d95dbf640 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb237fe640 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f9f230 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdb14001000 tx=0x7fdb240b3f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d817a946934af3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.370+0000 7fdb027fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb027fc640 1 -- 172.21.15.131:0/2086876994 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb140238c0 con 0x562d95dbf640 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb027fc640 1 -- 172.21.15.131:0/2086876994 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fdb14023a60 con 0x562d95dbf640 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 -- 172.21.15.131:0/2086876994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb18000e50 con 0x562d95dbf640 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb027fc640 1 -- 172.21.15.131:0/2086876994 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fdb14023c60 con 0x562d95dbf640 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 10 client.6792 fetch_fsmap learned FSMap version 53 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 -- 172.21.15.131:0/2086876994 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x562d95cbe620 con 0x562d95dbf640 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb027fc640 1 -- 172.21.15.131:0/2086876994 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb1402b030 con 0x562d95dbf640 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb027fc640 1 -- 172.21.15.131:0/2086876994 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fdb1401e040 con 0x562d95dbf640 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 10 client.6792 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 10 client.6792 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:46:29.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 20 client.6792 populate_metadata read hostname 'smithi131' 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562d959b8c30 0x562d95f5ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 4 client.6792 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 -- 172.21.15.131:0/2086876994 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x562d95d365d0 con 0x562d959b8c30 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb22ffd640 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562d959b8c30 0x562d95f5ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb027fc640 10 client.6792 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb22ffd640 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562d959b8c30 0x562d95f5ae10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cf6ca8c88fece40 server_cookie=d8ff117dbfb1f77f in_seq=0 out_seq=0 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb027fc640 1 -- 172.21.15.131:0/2086876994 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562d95d365d0 con 0x562d959b8c30 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb027fc640 10 client.6792 handle_command_reply: tid=0 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 client.6792 shutdown 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 -- 172.21.15.131:0/2086876994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 msgr2=0x562d95f9f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f9f230 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fdb14001000 tx=0x7fdb240b3f60 comp rx=0 tx=0).stop 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 -- 172.21.15.131:0/2086876994 shutdown_connections 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x562d959b8c30 0x562d95f5ae10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562d95de4ae0 0x562d95f9fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562d95dd5a40 0x562d95f9f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb027fc640 0 client.6792 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:29.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 --2- 172.21.15.131:0/2086876994 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562d95dbf640 0x562d95f9f230 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 -- 172.21.15.131:0/2086876994 >> 172.21.15.131:0/2086876994 conn(0x562d959e4b30 msgr2=0x562d95f11980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 -- 172.21.15.131:0/2086876994 shutdown_connections 2024-09-17T02:46:29.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 1 -- 172.21.15.131:0/2086876994 wait complete. 2024-09-17T02:46:29.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb31144000 20 client.6792 trim_cache size 0 max 16384 2024-09-17T02:46:29.376 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:46:29.376 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-09-17T02:46:29.376 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "4a421807-ccc6-4e69-ad08-65dd56749a48", 2024-09-17T02:46:29.377 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-09-17T02:46:29.377 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:46:29.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 -- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb10073230 msgr2=0x7fdb10075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb10073230 0x7fdb10075740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fdb14001340 tx=0x7fdb14000ee0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 -- 172.21.15.131:0/1669376280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2414c660 msgr2=0x7fdb2414ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2414c660 0x7fdb2414ca70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdb24070e70 tx=0x7fdb1c00fd60 comp rx=0 tx=0).stop 2024-09-17T02:46:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 -- 172.21.15.131:0/1669376280 shutdown_connections 2024-09-17T02:46:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb10073230 0x7fdb10075740 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2414c660 0x7fdb2414ca70 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb24147150 0x7fdb2406fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 --2- 172.21.15.131:0/1669376280 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb241512d0 0x7fdb241616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 -- 172.21.15.131:0/1669376280 >> 172.21.15.131:0/1669376280 conn(0x7fdb240d05a0 msgr2=0x7fdb24068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 -- 172.21.15.131:0/1669376280 shutdown_connections 2024-09-17T02:46:29.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.374+0000 7fdb29e79640 1 -- 172.21.15.131:0/1669376280 wait complete. 2024-09-17T02:46:29.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:46:29.391 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-17T02:46:29.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 -- 172.21.15.131:0/2135470319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2c005280 msgr2=0x7fdb2c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 --2- 172.21.15.131:0/2135470319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb2c005280 0x7fdb2c00b9d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009eb0 tx=0x7fdb3801d210 comp rx=0 tx=0).stop 2024-09-17T02:46:29.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb45a90640 1 -- 172.21.15.131:0/2135470319 reap_dead start 2024-09-17T02:46:29.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 -- 172.21.15.131:0/2135470319 shutdown_connections 2024-09-17T02:46:29.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 -- 172.21.15.131:0/2135470319 >> 172.21.15.131:0/2135470319 conn(0x7fdb4006eea0 msgr2=0x7fdb4006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 -- 172.21.15.131:0/2135470319 shutdown_connections 2024-09-17T02:46:29.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 -- 172.21.15.131:0/2135470319 wait complete. 2024-09-17T02:46:29.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 Processor -- start 2024-09-17T02:46:29.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 -- start start 2024-09-17T02:46:29.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb40155280 0x7fdb40159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb401557b0 0x7fdb4014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb400aaf30 0x7fdb400ab340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4006e020 con 0x7fdb400aaf30 2024-09-17T02:46:29.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4006dea0 con 0x7fdb401557b0 2024-09-17T02:46:29.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb474f3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4006e1a0 con 0x7fdb40155280 2024-09-17T02:46:29.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb45a90640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb400aaf30 0x7fdb400ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.534+0000 7fdb45a90640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb400aaf30 0x7fdb400ab340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52542/0 (socket says 172.21.15.131:52542) 2024-09-17T02:46:29.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb45a90640 1 -- 172.21.15.131:0/4063669256 learned_addr learned my addr 172.21.15.131:0/4063669256 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:29.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb4528f640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb40155280 0x7fdb40159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb44a8e640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb401557b0 0x7fdb4014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb45a90640 1 -- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb40155280 msgr2=0x7fdb40159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb45a90640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb40155280 0x7fdb40159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb45a90640 1 -- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb401557b0 msgr2=0x7fdb4014c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb45a90640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb401557b0 0x7fdb4014c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb45a90640 1 -- 172.21.15.131:0/4063669256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4015b0a0 con 0x7fdb400aaf30 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb4528f640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb40155280 0x7fdb40159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb45a90640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb400aaf30 0x7fdb400ab340 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fdb380af040 tx=0x7fdb380adcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb35ffb640 1 -- 172.21.15.131:0/4063669256 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb380b1040 con 0x7fdb400aaf30 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb35ffb640 1 -- 172.21.15.131:0/4063669256 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb380bc070 con 0x7fdb400aaf30 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb35ffb640 1 -- 172.21.15.131:0/4063669256 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38009ab0 con 0x7fdb400aaf30 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb474f3640 1 -- 172.21.15.131:0/4063669256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4015aae0 con 0x7fdb400aaf30 2024-09-17T02:46:29.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb474f3640 1 -- 172.21.15.131:0/4063669256 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb40158920 con 0x7fdb400aaf30 2024-09-17T02:46:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb35ffb640 1 -- 172.21.15.131:0/4063669256 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdb38004a50 con 0x7fdb400aaf30 2024-09-17T02:46:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb35ffb640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb24073200 0x7fdb24075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb35ffb640 1 -- 172.21.15.131:0/4063669256 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fdb3812f000 con 0x7fdb400aaf30 2024-09-17T02:46:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb4528f640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb24073200 0x7fdb24075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb4e7be000 1 Processor -- start 2024-09-17T02:46:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.538+0000 7fdb4e7be000 1 -- start start 2024-09-17T02:46:29.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4528f640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb24073200 0x7fdb24075710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdb30005d10 tx=0x7fdb30005dc0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a190e6d0 0x55f0a1a7eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a7f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a7fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f0a1919cd0 con 0x55f0a1565890 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f0a182f330 con 0x55f0a1629aa0 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f0a188cbb0 con 0x55f0a190e6d0 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb44a8e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a7f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb44a8e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a7f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52554/0 (socket says 172.21.15.131:52554) 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb44a8e640 1 -- 172.21.15.131:0/1711420740 learned_addr learned my addr 172.21.15.131:0/1711420740 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb45a90640 1 --2- 172.21.15.131:0/1711420740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a7fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4528f640 1 --2- 172.21.15.131:0/1711420740 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a190e6d0 0x55f0a1a7eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb44a8e640 1 -- 172.21.15.131:0/1711420740 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a190e6d0 msgr2=0x55f0a1a7eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb44a8e640 1 --2- 172.21.15.131:0/1711420740 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a190e6d0 0x55f0a1a7eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb44a8e640 1 -- 172.21.15.131:0/1711420740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 msgr2=0x55f0a1a7fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb44a8e640 1 --2- 172.21.15.131:0/1711420740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a7fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb44a8e640 1 -- 172.21.15.131:0/1711420740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a1830c80 con 0x55f0a1565890 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb45a90640 1 --2- 172.21.15.131:0/1711420740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a7fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb44a8e640 1 --2- 172.21.15.131:0/1711420740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a7f530 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0063d0 tx=0x7fdb3c006be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcd1cb79efce3c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb22ffd640 1 -- 172.21.15.131:0/1711420740 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c016020 con 0x55f0a1565890 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb22ffd640 1 -- 172.21.15.131:0/1711420740 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb3c006580 con 0x55f0a1565890 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb22ffd640 1 -- 172.21.15.131:0/1711420740 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c004cd0 con 0x55f0a1565890 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 -- 172.21.15.131:0/1711420740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 msgr2=0x55f0a1a7f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/1711420740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a7f530 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0063d0 tx=0x7fdb3c006be0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 -- 172.21.15.131:0/1711420740 shutdown_connections 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/1711420740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a7fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/1711420740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a7f530 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/1711420740 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a190e6d0 0x55f0a1a7eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 -- 172.21.15.131:0/1711420740 >> 172.21.15.131:0/1711420740 conn(0x55f0a15395c0 msgr2=0x55f0a1a37ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 -- 172.21.15.131:0/1711420740 shutdown_connections 2024-09-17T02:46:29.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 -- 172.21.15.131:0/1711420740 wait complete. 2024-09-17T02:46:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 -- start start 2024-09-17T02:46:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.542+0000 7fdb4e7be000 1 Processor -- start 2024-09-17T02:46:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a40cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a190e6d0 0x55f0a1a41730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f0a187fdf0 con 0x55f0a1565890 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f0a187bfe0 con 0x55f0a1629aa0 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f0a18214d0 con 0x55f0a190e6d0 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 10 client.0 _ll_register_callbacks cb 0x55f0a17ff7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4528f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a40cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4528f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a40cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52564/0 (socket says 172.21.15.131:52564) 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4528f640 1 -- 172.21.15.131:0/2368478976 learned_addr learned my addr 172.21.15.131:0/2368478976 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb45a90640 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a190e6d0 0x55f0a1a41730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb44a8e640 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4528f640 1 -- 172.21.15.131:0/2368478976 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a190e6d0 msgr2=0x55f0a1a41730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4528f640 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a190e6d0 0x55f0a1a41730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4528f640 1 -- 172.21.15.131:0/2368478976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 msgr2=0x55f0a1a411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4528f640 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4528f640 1 -- 172.21.15.131:0/2368478976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0a181a210 con 0x55f0a1565890 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4528f640 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a40cb0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdb40075290 tx=0x7fdb30004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57987a8c58cab290 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2368478976 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3001f620 con 0x55f0a1565890 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2368478976 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fdb3001f7c0 con 0x55f0a1565890 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2368478976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f0a1814350 con 0x55f0a1565890 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2368478976 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fdb3002f050 con 0x55f0a1565890 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 10 client.6804 fetch_fsmap learned FSMap version 53 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2368478976 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55f0a1800fc0 con 0x55f0a1565890 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2368478976 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f0a1814350 con 0x55f0a1565890 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2368478976 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fdb3000f040 con 0x55f0a1565890 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 10 client.6804 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 10 client.6804 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:46:29.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 20 client.6804 populate_metadata read hostname 'smithi131' 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55f0a151e930 0x55f0a1a905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 4 client.6804 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2368478976 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0a188eb00 con 0x55f0a151e930 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb44a8e640 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55f0a151e930 0x55f0a1a905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb1bfff640 10 client.6804 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb44a8e640 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55f0a151e930 0x55f0a1a905b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9257e53d47b89ab server_cookie=18958adf214addfa in_seq=0 out_seq=0 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2368478976 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f0a188eb00 con 0x55f0a151e930 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb1bfff640 10 client.6804 handle_command_reply: tid=0 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 client.6804 shutdown 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2368478976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 msgr2=0x55f0a1a40cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a40cb0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fdb40075290 tx=0x7fdb30004740 comp rx=0 tx=0).stop 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2368478976 shutdown_connections 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55f0a151e930 0x55f0a1a905b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a190e6d0 0x55f0a1a41730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb1bfff640 0 client.6804 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2368478976 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a40cb0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2368478976 >> 172.21.15.131:0/2368478976 conn(0x55f0a15395c0 msgr2=0x55f0a1a2f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2368478976 shutdown_connections 2024-09-17T02:46:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2368478976 wait complete. 2024-09-17T02:46:29.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.546+0000 7fdb4e7be000 20 client.6804 trim_cache size 0 max 16384 2024-09-17T02:46:29.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 Processor -- start 2024-09-17T02:46:29.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 -- start start 2024-09-17T02:46:29.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a1565890 0x55f0a1a36780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a37500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a190e6d0 0x55f0a1a37a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f0a1a3ed80 con 0x55f0a190e6d0 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb401ff140 con 0x55f0a1629aa0 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdb401fe120 con 0x55f0a1565890 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb45a90640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a190e6d0 0x55f0a1a37a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb45a90640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a190e6d0 0x55f0a1a37a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52576/0 (socket says 172.21.15.131:52576) 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb45a90640 1 -- 172.21.15.131:0/2821840187 learned_addr learned my addr 172.21.15.131:0/2821840187 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb45a90640 1 -- 172.21.15.131:0/2821840187 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a1565890 msgr2=0x55f0a1a36780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb45a90640 1 --2- 172.21.15.131:0/2821840187 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a1565890 0x55f0a1a36780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb45a90640 1 -- 172.21.15.131:0/2821840187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 msgr2=0x55f0a1a37500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb45a90640 1 --2- 172.21.15.131:0/2821840187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a37500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb45a90640 1 -- 172.21.15.131:0/2821840187 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0a18d9680 con 0x55f0a190e6d0 2024-09-17T02:46:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb45a90640 1 --2- 172.21.15.131:0/2821840187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a190e6d0 0x55f0a1a37a40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdb38002240 tx=0x7fdb380f9270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f4506d09c7bf19 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb227fc640 1 -- 172.21.15.131:0/2821840187 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb380f9860 con 0x55f0a190e6d0 2024-09-17T02:46:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb227fc640 1 -- 172.21.15.131:0/2821840187 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb380fe340 con 0x55f0a190e6d0 2024-09-17T02:46:29.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb227fc640 1 -- 172.21.15.131:0/2821840187 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb381031f0 con 0x55f0a190e6d0 2024-09-17T02:46:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2821840187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a190e6d0 msgr2=0x55f0a1a37a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2821840187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a190e6d0 0x55f0a1a37a40 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdb38002240 tx=0x7fdb380f9270 comp rx=0 tx=0).stop 2024-09-17T02:46:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2821840187 shutdown_connections 2024-09-17T02:46:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2821840187 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a190e6d0 0x55f0a1a37a40 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2821840187 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a1629aa0 0x55f0a1a37500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2821840187 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a1565890 0x55f0a1a36780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2821840187 >> 172.21.15.131:0/2821840187 conn(0x55f0a15395c0 msgr2=0x55f0a1a36df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.566+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2821840187 shutdown_connections 2024-09-17T02:46:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2821840187 wait complete. 2024-09-17T02:46:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 -- start start 2024-09-17T02:46:29.567 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 Processor -- start 2024-09-17T02:46:29.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a91ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a1629aa0 0x55f0a1a92000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a190e6d0 0x55f0a1a92540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdb401fdad0 con 0x55f0a1565890 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb400ce740 con 0x55f0a190e6d0 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f0a1541680 con 0x55f0a1629aa0 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 10 client.0 _ll_register_callbacks cb 0x7fdb30016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4528f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a91ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4528f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a91ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:52588/0 (socket says 172.21.15.131:52588) 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4528f640 1 -- 172.21.15.131:0/2680762425 learned_addr learned my addr 172.21.15.131:0/2680762425 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb44a8e640 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a1629aa0 0x55f0a1a92000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4528f640 1 -- 172.21.15.131:0/2680762425 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a1629aa0 msgr2=0x55f0a1a92000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4528f640 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a1629aa0 0x55f0a1a92000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4528f640 1 -- 172.21.15.131:0/2680762425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a190e6d0 msgr2=0x55f0a1a92540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb45a90640 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a190e6d0 0x55f0a1a92540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4528f640 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a190e6d0 0x55f0a1a92540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4528f640 1 -- 172.21.15.131:0/2680762425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0a18460b0 con 0x55f0a1565890 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb44a8e640 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a1629aa0 0x55f0a1a92000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4528f640 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a91ac0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fdb30022040 tx=0x7fdb3001fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d711717c1840feb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb1bfff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2680762425 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30037070 con 0x55f0a1565890 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2680762425 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fdb30005040 con 0x55f0a1565890 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2680762425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb380fe340 con 0x55f0a1565890 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2680762425 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fdb30042050 con 0x55f0a1565890 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 10 client.6810 fetch_fsmap learned FSMap version 53 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2680762425 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55f0a1812610 con 0x55f0a1565890 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2680762425 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb380fe340 con 0x55f0a1565890 2024-09-17T02:46:29.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2680762425 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fdb30045020 con 0x55f0a1565890 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 10 client.6810 fetch_fsmap finished waiting for FSMap version 53 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 10 client.6810 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 20 client.6810 populate_metadata read hostname 'smithi131' 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55f0a150c990 0x55f0a1a92a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 4 client.6810 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2680762425 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f0a188a5b0 con 0x55f0a150c990 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb44a8e640 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55f0a150c990 0x55f0a1a92a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb1bfff640 10 client.6810 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.570+0000 7fdb44a8e640 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55f0a150c990 0x55f0a1a92a80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54bab36a07f2868 server_cookie=a8c4a701ea5106e2 in_seq=0 out_seq=0 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb1bfff640 1 -- 172.21.15.131:0/2680762425 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+873 (crc 0 0 0) 0x55f0a188a5b0 con 0x55f0a150c990 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb1bfff640 10 client.6810 handle_command_reply: tid=0 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 1 client.6810 shutdown 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2680762425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 msgr2=0x55f0a1a91ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a91ac0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fdb30022040 tx=0x7fdb3001fea0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2680762425 shutdown_connections 2024-09-17T02:46:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55f0a150c990 0x55f0a1a92a80 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f0a190e6d0 0x55f0a1a92540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb1bfff640 0 client.6810 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f0a1629aa0 0x55f0a1a92000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 1 --2- 172.21.15.131:0/2680762425 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f0a1565890 0x55f0a1a91ac0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2680762425 >> 172.21.15.131:0/2680762425 conn(0x55f0a15395c0 msgr2=0x55f0a1a377b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2680762425 shutdown_connections 2024-09-17T02:46:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 1 -- 172.21.15.131:0/2680762425 wait complete. 2024-09-17T02:46:29.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb4e7be000 20 client.6810 trim_cache size 0 max 16384 2024-09-17T02:46:29.575 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: "4a421807-ccc6-4e69-ad08-65dd56749a48": { 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/subdir", 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "4a421807-ccc6-4e69-ad08-65dd56749a48", 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,repair" 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: "4a421807-ccc6-4e69-ad08-65dd56749a48": { 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:46:29.374028+0000", 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/subdir", 2024-09-17T02:46:29.576 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: "7665c7ca-bac8-4303-b3bb-a75393928ae5": { 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:45:57.738298+0000", 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/subdir", 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:46:29.577 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:46:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 -- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb24073200 msgr2=0x7fdb24075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb24073200 0x7fdb24075710 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdb30005d10 tx=0x7fdb30005dc0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 -- 172.21.15.131:0/4063669256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb400aaf30 msgr2=0x7fdb400ab340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb400aaf30 0x7fdb400ab340 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fdb380af040 tx=0x7fdb380adcc0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 -- 172.21.15.131:0/4063669256 shutdown_connections 2024-09-17T02:46:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdb24073200 0x7fdb24075710 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb400aaf30 0x7fdb400ab340 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb401557b0 0x7fdb4014c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 --2- 172.21.15.131:0/4063669256 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb40155280 0x7fdb40159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 -- 172.21.15.131:0/4063669256 >> 172.21.15.131:0/4063669256 conn(0x7fdb4006eea0 msgr2=0x7fdb40073c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 -- 172.21.15.131:0/4063669256 shutdown_connections 2024-09-17T02:46:29.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:29.574+0000 7fdb474f3640 1 -- 172.21.15.131:0/4063669256 wait complete. 2024-09-17T02:46:29.589 INFO:tasks.cephfs.filesystem:scrub status for tag:4a421807-ccc6-4e69-ad08-65dd56749a48 - {'path': '/subdir', 'tag': '4a421807-ccc6-4e69-ad08-65dd56749a48', 'options': 'recursive,repair'} 2024-09-17T02:46:44.567 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:46:44.571 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:46:59.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:46:59.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-17T02:46:59.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 -- 172.21.15.131:0/1228766226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f593406e3e0 msgr2=0x7f593415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 --2- 172.21.15.131:0/1228766226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f593406e3e0 0x7f593415bea0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f593406a1c0 tx=0x7f591c01c8c0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 -- 172.21.15.131:0/1228766226 shutdown_connections 2024-09-17T02:46:59.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 --2- 172.21.15.131:0/1228766226 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f593406e3e0 0x7f593415bea0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 --2- 172.21.15.131:0/1228766226 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f593415c3e0 0x7f59340cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 --2- 172.21.15.131:0/1228766226 >> v2:172.21.15.189:3300/0 conn(0x7f59340ced00 0x7f593406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 -- 172.21.15.131:0/1228766226 >> 172.21.15.131:0/1228766226 conn(0x7f593406eea0 msgr2=0x7f593406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 -- 172.21.15.131:0/1228766226 shutdown_connections 2024-09-17T02:46:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 -- 172.21.15.131:0/1228766226 wait complete. 2024-09-17T02:46:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 Processor -- start 2024-09-17T02:46:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 -- start start 2024-09-17T02:46:59.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f593406e3e0 0x7f5934153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59340ced00 0x7f593414e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f593415c3e0 0x7f593415ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59340caae0 con 0x7f593415c3e0 2024-09-17T02:46:59.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59340ca960 con 0x7f59340ced00 2024-09-17T02:46:59.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.765+0000 7f5938bb1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f59340cac60 con 0x7f593406e3e0 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5932d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f593415c3e0 0x7f593415ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5932d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f593415c3e0 0x7f593415ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46084/0 (socket says 172.21.15.131:46084) 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5932d9d640 1 -- 172.21.15.131:0/3409977199 learned_addr learned my addr 172.21.15.131:0/3409977199 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f593259c640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f593406e3e0 0x7f5934153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5931d9b640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59340ced00 0x7f593414e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5932d9d640 1 -- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f593406e3e0 msgr2=0x7f5934153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5932d9d640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f593406e3e0 0x7f5934153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5932d9d640 1 -- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59340ced00 msgr2=0x7f593414e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5932d9d640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59340ced00 0x7f593414e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5932d9d640 1 -- 172.21.15.131:0/3409977199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59340b0e40 con 0x7f593415c3e0 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5932d9d640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f593415c3e0 0x7f593415ada0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f592400e910 tx=0x7f592400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:59.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f592affd640 1 -- 172.21.15.131:0/3409977199 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924004d30 con 0x7f593415c3e0 2024-09-17T02:46:59.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f592affd640 1 -- 172.21.15.131:0/3409977199 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f592400cb20 con 0x7f593415c3e0 2024-09-17T02:46:59.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f592affd640 1 -- 172.21.15.131:0/3409977199 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924005020 con 0x7f593415c3e0 2024-09-17T02:46:59.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5938bb1640 1 -- 172.21.15.131:0/3409977199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59340b1120 con 0x7f593415c3e0 2024-09-17T02:46:59.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f5938bb1640 1 -- 172.21.15.131:0/3409977199 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59341f65b0 con 0x7f593415c3e0 2024-09-17T02:46:59.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f592affd640 1 -- 172.21.15.131:0/3409977199 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f59240051c0 con 0x7f593415c3e0 2024-09-17T02:46:59.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f592affd640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5914073230 0x7f5914075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f592affd640 1 -- 172.21.15.131:0/3409977199 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f5924094b90 con 0x7f593415c3e0 2024-09-17T02:46:59.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.769+0000 7f593259c640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5914073230 0x7f5914075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 Processor -- start 2024-09-17T02:46:59.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 -- start start 2024-09-17T02:46:59.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593259c640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5914073230 0x7f5914075740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f592c005950 tx=0x7f592c00c460 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:59.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 0x5603cbb9b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbb9bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb74aaa0 0x5603cbb9c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5603cba3acd0 con 0x5603cb686890 2024-09-17T02:46:59.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603cb950330 con 0x5603cba2f6d0 2024-09-17T02:46:59.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5603cb9adbb0 con 0x5603cb74aaa0 2024-09-17T02:46:59.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5931d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbb9bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5931d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbb9bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46096/0 (socket says 172.21.15.131:46096) 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5931d9b640 1 -- 172.21.15.131:0/1697944088 learned_addr learned my addr 172.21.15.131:0/1697944088 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593259c640 1 --2- 172.21.15.131:0/1697944088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 0x5603cbb9b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5932d9d640 1 --2- 172.21.15.131:0/1697944088 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb74aaa0 0x5603cbb9c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5931d9b640 1 -- 172.21.15.131:0/1697944088 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb74aaa0 msgr2=0x5603cbb9c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5931d9b640 1 --2- 172.21.15.131:0/1697944088 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb74aaa0 0x5603cbb9c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5931d9b640 1 -- 172.21.15.131:0/1697944088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 msgr2=0x5603cbb9b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5931d9b640 1 --2- 172.21.15.131:0/1697944088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 0x5603cbb9b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5931d9b640 1 -- 172.21.15.131:0/1697944088 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603cb951c80 con 0x5603cb686890 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593259c640 1 --2- 172.21.15.131:0/1697944088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 0x5603cbb9b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5931d9b640 1 --2- 172.21.15.131:0/1697944088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbb9bf30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f591c01d160 tx=0x7f591c019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985400950496d1dd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5913fff640 1 -- 172.21.15.131:0/1697944088 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c0ba020 con 0x5603cb686890 2024-09-17T02:46:59.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5913fff640 1 -- 172.21.15.131:0/1697944088 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f591c004330 con 0x5603cb686890 2024-09-17T02:46:59.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f5913fff640 1 -- 172.21.15.131:0/1697944088 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c0b2040 con 0x5603cb686890 2024-09-17T02:46:59.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 -- 172.21.15.131:0/1697944088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 msgr2=0x5603cbb9bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 --2- 172.21.15.131:0/1697944088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbb9bf30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f591c01d160 tx=0x7f591c019660 comp rx=0 tx=0).stop 2024-09-17T02:46:59.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 -- 172.21.15.131:0/1697944088 shutdown_connections 2024-09-17T02:46:59.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 --2- 172.21.15.131:0/1697944088 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb74aaa0 0x5603cbb9c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 --2- 172.21.15.131:0/1697944088 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbb9bf30 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 --2- 172.21.15.131:0/1697944088 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 0x5603cbb9b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 -- 172.21.15.131:0/1697944088 >> 172.21.15.131:0/1697944088 conn(0x5603cb65a5c0 msgr2=0x5603cbb58ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:59.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 -- 172.21.15.131:0/1697944088 shutdown_connections 2024-09-17T02:46:59.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.773+0000 7f593fe7c000 1 -- 172.21.15.131:0/1697944088 wait complete. 2024-09-17T02:46:59.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 -- start start 2024-09-17T02:46:59.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 Processor -- start 2024-09-17T02:46:59.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb686890 0x5603cbb57a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cb74aaa0 0x5603cbb57f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cba2f6d0 0x5603cbb5e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5603cb9a0df0 con 0x5603cba2f6d0 2024-09-17T02:46:59.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603cb99cfe0 con 0x5603cb74aaa0 2024-09-17T02:46:59.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5603cb9424d0 con 0x5603cb686890 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 10 client.0 _ll_register_callbacks cb 0x5603cb9207f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f5932d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cba2f6d0 0x5603cbb5e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f5932d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cba2f6d0 0x5603cbb5e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46100/0 (socket says 172.21.15.131:46100) 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f5932d9d640 1 -- 172.21.15.131:0/2515092495 learned_addr learned my addr 172.21.15.131:0/2515092495 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f5931d9b640 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cb74aaa0 0x5603cbb57f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593259c640 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb686890 0x5603cbb57a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f5932d9d640 1 -- 172.21.15.131:0/2515092495 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb686890 msgr2=0x5603cbb57a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f5932d9d640 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb686890 0x5603cbb57a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f5932d9d640 1 -- 172.21.15.131:0/2515092495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cb74aaa0 msgr2=0x5603cbb57f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f5932d9d640 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cb74aaa0 0x5603cbb57f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f5932d9d640 1 -- 172.21.15.131:0/2515092495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603cb93b210 con 0x5603cba2f6d0 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f5931d9b640 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cb74aaa0 0x5603cbb57f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:59.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593259c640 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb686890 0x5603cbb57a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f5932d9d640 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cba2f6d0 0x5603cbb5e5f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f592405f2f0 tx=0x7f592405f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3030653fb1cd7795 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f59117fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f59117fa640 1 -- 172.21.15.131:0/2515092495 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924084070 con 0x5603cba2f6d0 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f59117fa640 1 -- 172.21.15.131:0/2515092495 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f5924012d00 con 0x5603cba2f6d0 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 -- 172.21.15.131:0/2515092495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603cb935350 con 0x5603cba2f6d0 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f59117fa640 1 -- 172.21.15.131:0/2515092495 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f592401f040 con 0x5603cba2f6d0 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 10 client.6822 fetch_fsmap learned FSMap version 54 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 -- 172.21.15.131:0/2515092495 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5603cb921fc0 con 0x5603cba2f6d0 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f59117fa640 1 -- 172.21.15.131:0/2515092495 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5603cb935350 con 0x5603cba2f6d0 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f59117fa640 1 -- 172.21.15.131:0/2515092495 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f5924022060 con 0x5603cba2f6d0 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 10 client.6822 fetch_fsmap finished waiting for FSMap version 54 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 10 client.6822 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 20 client.6822 populate_metadata read hostname 'smithi131' 2024-09-17T02:46:59.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x5603cb63f930 0x5603cbba2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 4 client.6822 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593fe7c000 1 -- 172.21.15.131:0/2515092495 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603cb9afb00 con 0x5603cb63f930 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593259c640 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x5603cb63f930 0x5603cbba2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f59117fa640 10 client.6822 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.777+0000 7f593259c640 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x5603cb63f930 0x5603cbba2e80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0acdda44766a597 server_cookie=13b946680295e7f2 in_seq=0 out_seq=0 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f59117fa640 1 -- 172.21.15.131:0/2515092495 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5603cb9afb00 con 0x5603cb63f930 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f59117fa640 10 client.6822 handle_command_reply: tid=0 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 1 client.6822 shutdown 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 1 -- 172.21.15.131:0/2515092495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cba2f6d0 msgr2=0x5603cbb5e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cba2f6d0 0x5603cbb5e5f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f592405f2f0 tx=0x7f592405f980 comp rx=0 tx=0).stop 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 1 -- 172.21.15.131:0/2515092495 shutdown_connections 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x5603cb63f930 0x5603cbba2e80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cba2f6d0 0x5603cbb5e5f0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cb74aaa0 0x5603cbb57f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 1 --2- 172.21.15.131:0/2515092495 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb686890 0x5603cbb57a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f59117fa640 0 client.6822 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 1 -- 172.21.15.131:0/2515092495 >> 172.21.15.131:0/2515092495 conn(0x5603cb65a5c0 msgr2=0x5603cbb50d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 1 -- 172.21.15.131:0/2515092495 shutdown_connections 2024-09-17T02:46:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 1 -- 172.21.15.131:0/2515092495 wait complete. 2024-09-17T02:46:59.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.781+0000 7f593fe7c000 20 client.6822 trim_cache size 0 max 16384 2024-09-17T02:46:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.797+0000 7f593fe7c000 1 Processor -- start 2024-09-17T02:46:59.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.797+0000 7f593fe7c000 1 -- start start 2024-09-17T02:46:59.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb686890 0x5603cbc0d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb74aaa0 0x5603cbc0d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 0x5603cbc0de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5603cbb56b20 con 0x5603cb74aaa0 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5934203570 con 0x5603cba2f6d0 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f59342025b0 con 0x5603cb686890 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f5931d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb74aaa0 0x5603cbc0d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f5931d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb74aaa0 0x5603cbc0d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46108/0 (socket says 172.21.15.131:46108) 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f5931d9b640 1 -- 172.21.15.131:0/3223701666 learned_addr learned my addr 172.21.15.131:0/3223701666 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f5931d9b640 1 -- 172.21.15.131:0/3223701666 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb686890 msgr2=0x5603cbc0d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593259c640 1 --2- 172.21.15.131:0/3223701666 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb686890 0x5603cbc0d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f5931d9b640 1 --2- 172.21.15.131:0/3223701666 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb686890 0x5603cbc0d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f5931d9b640 1 -- 172.21.15.131:0/3223701666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 msgr2=0x5603cbc0de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f5932d9d640 1 --2- 172.21.15.131:0/3223701666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 0x5603cbc0de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f5931d9b640 1 --2- 172.21.15.131:0/3223701666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 0x5603cbc0de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f5931d9b640 1 -- 172.21.15.131:0/3223701666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603cb9fa680 con 0x5603cb74aaa0 2024-09-17T02:46:59.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f5932d9d640 1 --2- 172.21.15.131:0/3223701666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 0x5603cbc0de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:59.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f5931d9b640 1 --2- 172.21.15.131:0/3223701666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb74aaa0 0x5603cbc0d940 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f591c0a5710 tx=0x7f591c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0fdc900474d7ae server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:59.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f59137fe640 1 -- 172.21.15.131:0/3223701666 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c0ba030 con 0x5603cb74aaa0 2024-09-17T02:46:59.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f59137fe640 1 -- 172.21.15.131:0/3223701666 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f591c005040 con 0x5603cb74aaa0 2024-09-17T02:46:59.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f59137fe640 1 -- 172.21.15.131:0/3223701666 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c0b2040 con 0x5603cb74aaa0 2024-09-17T02:46:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 -- 172.21.15.131:0/3223701666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb74aaa0 msgr2=0x5603cbc0d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 --2- 172.21.15.131:0/3223701666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb74aaa0 0x5603cbc0d940 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f591c0a5710 tx=0x7f591c009260 comp rx=0 tx=0).stop 2024-09-17T02:46:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 -- 172.21.15.131:0/3223701666 shutdown_connections 2024-09-17T02:46:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 --2- 172.21.15.131:0/3223701666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cba2f6d0 0x5603cbc0de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 --2- 172.21.15.131:0/3223701666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb74aaa0 0x5603cbc0d940 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 --2- 172.21.15.131:0/3223701666 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cb686890 0x5603cbc0d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 -- 172.21.15.131:0/3223701666 >> 172.21.15.131:0/3223701666 conn(0x5603cb65a5c0 msgr2=0x5603cbbadfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 -- 172.21.15.131:0/3223701666 shutdown_connections 2024-09-17T02:46:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 -- 172.21.15.131:0/3223701666 wait complete. 2024-09-17T02:46:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 -- start start 2024-09-17T02:46:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.801+0000 7f593fe7c000 1 Processor -- start 2024-09-17T02:46:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbc11440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cb74aaa0 0x5603cbc11980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cba2f6d0 0x5603cbc11ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5934201f60 con 0x5603cb686890 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59340ce650 con 0x5603cb74aaa0 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5603cb662680 con 0x5603cba2f6d0 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 10 client.0 _ll_register_callbacks cb 0x7f5924004a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbc11440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbc11440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46112/0 (socket says 172.21.15.131:46112) 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593259c640 1 -- 172.21.15.131:0/4147502870 learned_addr learned my addr 172.21.15.131:0/4147502870 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593259c640 1 -- 172.21.15.131:0/4147502870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cba2f6d0 msgr2=0x5603cbc11ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f5931d9b640 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cb74aaa0 0x5603cbc11980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f5932d9d640 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cba2f6d0 0x5603cbc11ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593259c640 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cba2f6d0 0x5603cbc11ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593259c640 1 -- 172.21.15.131:0/4147502870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cb74aaa0 msgr2=0x5603cbc11980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593259c640 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cb74aaa0 0x5603cbc11980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593259c640 1 -- 172.21.15.131:0/4147502870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603cb9670b0 con 0x5603cb686890 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593259c640 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbc11440 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x5603cb9adf00 tx=0x7f592c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57980a901f9695f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f59117fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f59117fa640 1 -- 172.21.15.131:0/4147502870 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c02e470 con 0x5603cb686890 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f59117fa640 1 -- 172.21.15.131:0/4147502870 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f592c01d420 con 0x5603cb686890 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 -- 172.21.15.131:0/4147502870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f591c005040 con 0x5603cb686890 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f59117fa640 1 -- 172.21.15.131:0/4147502870 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f592c01d620 con 0x5603cb686890 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 10 client.6828 fetch_fsmap learned FSMap version 54 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 -- 172.21.15.131:0/4147502870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5603cb933610 con 0x5603cb686890 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f59117fa640 1 -- 172.21.15.131:0/4147502870 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f591c005040 con 0x5603cb686890 2024-09-17T02:46:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f59117fa640 1 -- 172.21.15.131:0/4147502870 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f592c01d7c0 con 0x5603cb686890 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 10 client.6828 fetch_fsmap finished waiting for FSMap version 54 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 10 client.6828 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 20 client.6828 populate_metadata read hostname 'smithi131' 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x5603cb62d990 0x5603cbb52d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 4 client.6828 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 -- 172.21.15.131:0/4147502870 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5603cb9ab5b0 con 0x5603cb62d990 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f5931d9b640 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x5603cb62d990 0x5603cbb52d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f59117fa640 10 client.6828 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f5931d9b640 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x5603cb62d990 0x5603cbb52d40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d537f86f465b43a server_cookie=c6f5a6e1b76afe41 in_seq=0 out_seq=0 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f59117fa640 1 -- 172.21.15.131:0/4147502870 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+670 (crc 0 0 0) 0x5603cb9ab5b0 con 0x5603cb62d990 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f59117fa640 10 client.6828 handle_command_reply: tid=0 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 client.6828 shutdown 2024-09-17T02:46:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 -- 172.21.15.131:0/4147502870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 msgr2=0x5603cbc11440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbc11440 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x5603cb9adf00 tx=0x7f592c004740 comp rx=0 tx=0).stop 2024-09-17T02:46:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 -- 172.21.15.131:0/4147502870 shutdown_connections 2024-09-17T02:46:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x5603cb62d990 0x5603cbb52d40 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603cba2f6d0 0x5603cbc11ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603cb74aaa0 0x5603cbc11980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f59117fa640 0 client.6828 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:46:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 --2- 172.21.15.131:0/4147502870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603cb686890 0x5603cbc11440 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 -- 172.21.15.131:0/4147502870 >> 172.21.15.131:0/4147502870 conn(0x5603cb65a5c0 msgr2=0x5603cbb5d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 -- 172.21.15.131:0/4147502870 shutdown_connections 2024-09-17T02:46:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 1 -- 172.21.15.131:0/4147502870 wait complete. 2024-09-17T02:46:59.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f593fe7c000 20 client.6828 trim_cache size 0 max 16384 2024-09-17T02:46:59.810 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:46:59.810 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-09-17T02:46:59.810 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {}, 2024-09-17T02:46:59.810 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:46:59.810 INFO:teuthology.orchestra.run.smithi131.stdout: "4a421807-ccc6-4e69-ad08-65dd56749a48": { 2024-09-17T02:46:59.810 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:46:29.374028+0000", 2024-09-17T02:46:59.810 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/subdir", 2024-09-17T02:46:59.810 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:46:59.810 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:46:59.810 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:46:59.810 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:46:59.811 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:46:59.811 INFO:teuthology.orchestra.run.smithi131.stdout: "7665c7ca-bac8-4303-b3bb-a75393928ae5": { 2024-09-17T02:46:59.811 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:45:57.738298+0000", 2024-09-17T02:46:59.811 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/subdir", 2024-09-17T02:46:59.811 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:46:59.811 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:46:59.811 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:46:59.811 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:46:59.811 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:46:59.811 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:46:59.811 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:46:59.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.805+0000 7f5938bb1640 1 -- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5914073230 msgr2=0x7f5914075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.809+0000 7f5938bb1640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5914073230 0x7f5914075740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f592c005950 tx=0x7f592c00c460 comp rx=0 tx=0).stop 2024-09-17T02:46:59.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.809+0000 7f5938bb1640 1 -- 172.21.15.131:0/3409977199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f593415c3e0 msgr2=0x7f593415ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.809+0000 7f5938bb1640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f593415c3e0 0x7f593415ada0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f592400e910 tx=0x7f592400edd0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.809+0000 7f5938bb1640 1 -- 172.21.15.131:0/3409977199 shutdown_connections 2024-09-17T02:46:59.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.809+0000 7f5938bb1640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5914073230 0x7f5914075740 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.809+0000 7f5938bb1640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f593415c3e0 0x7f593415ada0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.809+0000 7f5938bb1640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59340ced00 0x7f593414e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.809+0000 7f5938bb1640 1 --2- 172.21.15.131:0/3409977199 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f593406e3e0 0x7f5934153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.809+0000 7f5938bb1640 1 -- 172.21.15.131:0/3409977199 >> 172.21.15.131:0/3409977199 conn(0x7f593406eea0 msgr2=0x7f593406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.809+0000 7f5938bb1640 1 -- 172.21.15.131:0/3409977199 shutdown_connections 2024-09-17T02:46:59.813 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.809+0000 7f5938bb1640 1 -- 172.21.15.131:0/3409977199 wait complete. 2024-09-17T02:46:59.825 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-17T02:46:59.825 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:46:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.969+0000 7fcce7805640 1 -- 172.21.15.131:0/3084087944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce015c5b0 msgr2=0x7fcce00cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.969+0000 7fcce7805640 1 --2- 172.21.15.131:0/3084087944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce015c5b0 0x7fcce00cbaf0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009c80 tx=0x7fccd801cda0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.969+0000 7fcce7805640 1 -- 172.21.15.131:0/3084087944 shutdown_connections 2024-09-17T02:46:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.969+0000 7fcce7805640 1 --2- 172.21.15.131:0/3084087944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce015c5b0 0x7fcce00cbaf0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.969+0000 7fcce7805640 1 -- 172.21.15.131:0/3084087944 >> 172.21.15.131:0/3084087944 conn(0x7fcce006eea0 msgr2=0x7fcce006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:46:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.969+0000 7fcce7805640 1 -- 172.21.15.131:0/3084087944 shutdown_connections 2024-09-17T02:46:59.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.969+0000 7fcce7805640 1 -- 172.21.15.131:0/3084087944 wait complete. 2024-09-17T02:46:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce7805640 1 Processor -- start 2024-09-17T02:46:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce7805640 1 -- start start 2024-09-17T02:46:59.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce7805640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcce00ced00 0x7fcce01f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce7805640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcce015c5b0 0x7fcce00b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce7805640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158b10 0x7fcce00b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce7805640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcce01699d0 con 0x7fcce0158b10 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce7805640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcce0169850 con 0x7fcce015c5b0 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce7805640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcce0169b50 con 0x7fcce00ced00 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce5da2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158b10 0x7fcce00b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce5da2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158b10 0x7fcce00b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46126/0 (socket says 172.21.15.131:46126) 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce5da2640 1 -- 172.21.15.131:0/102830917 learned_addr learned my addr 172.21.15.131:0/102830917 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce55a1640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcce00ced00 0x7fcce01f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce4da0640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcce015c5b0 0x7fcce00b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce5da2640 1 -- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcce00ced00 msgr2=0x7fcce01f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce5da2640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcce00ced00 0x7fcce01f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce5da2640 1 -- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcce015c5b0 msgr2=0x7fcce00b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce5da2640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcce015c5b0 0x7fcce00b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:46:59.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce5da2640 1 -- 172.21.15.131:0/102830917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce01f62e0 con 0x7fcce0158b10 2024-09-17T02:46:59.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce55a1640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcce00ced00 0x7fcce01f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:46:59.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce5da2640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158b10 0x7fcce00b10d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009bd0 tx=0x7fccd80093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:59.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fccd5ffb640 1 -- 172.21.15.131:0/102830917 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80bc070 con 0x7fcce0158b10 2024-09-17T02:46:59.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fccd5ffb640 1 -- 172.21.15.131:0/102830917 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccd8004d10 con 0x7fcce0158b10 2024-09-17T02:46:59.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce7805640 1 -- 172.21.15.131:0/102830917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce00b4e50 con 0x7fcce0158b10 2024-09-17T02:46:59.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fccd5ffb640 1 -- 172.21.15.131:0/102830917 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd80b2040 con 0x7fcce0158b10 2024-09-17T02:46:59.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce7805640 1 -- 172.21.15.131:0/102830917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce00b5410 con 0x7fcce0158b10 2024-09-17T02:46:59.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.973+0000 7fcce7805640 1 -- 172.21.15.131:0/102830917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce0075b80 con 0x7fcce0158b10 2024-09-17T02:46:59.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.977+0000 7fccd5ffb640 1 -- 172.21.15.131:0/102830917 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fccd80bf070 con 0x7fcce0158b10 2024-09-17T02:46:59.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.977+0000 7fccd5ffb640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccc4073200 0x7fccc4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:46:59.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.977+0000 7fccd5ffb640 1 -- 172.21.15.131:0/102830917 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fccd812e200 con 0x7fcce0158b10 2024-09-17T02:46:59.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.977+0000 7fcce55a1640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccc4073200 0x7fccc4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:46:59.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.977+0000 7fcce55a1640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccc4073200 0x7fccc4075710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fccd000a850 tx=0x7fccd0008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:46:59.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:46:59.981+0000 7fccd5ffb640 1 -- 172.21.15.131:0/102830917 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcce01f62e0 con 0x7fcce0158b10 2024-09-17T02:47:00.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.177+0000 7fcce7805640 1 -- 172.21.15.131:0/102830917 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcce00cbaf0 con 0x7fcce0158b10 2024-09-17T02:47:00.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.177+0000 7fccd5ffb640 1 -- 172.21.15.131:0/102830917 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5230 (secure 0 0 0) 0x7fccd8016990 con 0x7fcce0158b10 2024-09-17T02:47:00.177 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:00.177 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":54,"btime":"2024-09-17T02:46:32:884315+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:46:32.834031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":16,"addr":"172.21.15.131:6835/962976667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":962976667},{"type":"v1","addr":"172.21.15.131:6835","nonce":962976667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6609,"qdb_cluster":[6609]},"id":5}]} 2024-09-17T02:47:00.178 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 54 2024-09-17T02:47:00.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 -- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccc4073200 msgr2=0x7fccc4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccc4073200 0x7fccc4075710 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fccd000a850 tx=0x7fccd0008040 comp rx=0 tx=0).stop 2024-09-17T02:47:00.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 -- 172.21.15.131:0/102830917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158b10 msgr2=0x7fcce00b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158b10 0x7fcce00b10d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009bd0 tx=0x7fccd80093d0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 -- 172.21.15.131:0/102830917 shutdown_connections 2024-09-17T02:47:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fccc4073200 0x7fccc4075710 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcce0158b10 0x7fcce00b10d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcce015c5b0 0x7fcce00b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 --2- 172.21.15.131:0/102830917 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcce00ced00 0x7fcce01f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 -- 172.21.15.131:0/102830917 >> 172.21.15.131:0/102830917 conn(0x7fcce006eea0 msgr2=0x7fcce015a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 -- 172.21.15.131:0/102830917 shutdown_connections 2024-09-17T02:47:00.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.181+0000 7fcce7805640 1 -- 172.21.15.131:0/102830917 wait complete. 2024-09-17T02:47:00.198 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-17T02:47:00.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.357+0000 7f6503cc2640 1 -- 172.21.15.131:0/4269290743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64fc06fdd0 msgr2=0x7f64fc06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.357+0000 7f6503cc2640 1 --2- 172.21.15.131:0/4269290743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64fc06fdd0 0x7f64fc06dea0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009c80 tx=0x7f64f401cda0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.357+0000 7f6503cc2640 1 -- 172.21.15.131:0/4269290743 shutdown_connections 2024-09-17T02:47:00.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.357+0000 7f6503cc2640 1 --2- 172.21.15.131:0/4269290743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64fc06fdd0 0x7f64fc06dea0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.357+0000 7f6503cc2640 1 --2- 172.21.15.131:0/4269290743 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f64fc15b2e0 0x7f64fc155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.357+0000 7f6503cc2640 1 -- 172.21.15.131:0/4269290743 >> 172.21.15.131:0/4269290743 conn(0x7f64fc06eea0 msgr2=0x7f64fc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:00.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.357+0000 7f6503cc2640 1 -- 172.21.15.131:0/4269290743 shutdown_connections 2024-09-17T02:47:00.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.357+0000 7f6503cc2640 1 -- 172.21.15.131:0/4269290743 wait complete. 2024-09-17T02:47:00.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6503cc2640 1 Processor -- start 2024-09-17T02:47:00.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6503cc2640 1 -- start start 2024-09-17T02:47:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6503cc2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64fc06f9c0 0x7f64fc14f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6503cc2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f64fc06fdd0 0x7f64fc1f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6503cc2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64fc15b2e0 0x7f64fc146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6503cc2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc168f30 con 0x7f64fc15b2e0 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6503cc2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f64fc168db0 con 0x7f64fc06f9c0 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6503cc2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f64fc1690b0 con 0x7f64fc06fdd0 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f650225f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64fc15b2e0 0x7f64fc146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f650225f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64fc15b2e0 0x7f64fc146f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46140/0 (socket says 172.21.15.131:46140) 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f650225f640 1 -- 172.21.15.131:0/208583493 learned_addr learned my addr 172.21.15.131:0/208583493 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6501a5e640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64fc06f9c0 0x7f64fc14f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f650125d640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f64fc06fdd0 0x7f64fc1f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f650225f640 1 -- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f64fc06fdd0 msgr2=0x7f64fc1f6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f650225f640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f64fc06fdd0 0x7f64fc1f6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f650225f640 1 -- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64fc06f9c0 msgr2=0x7f64fc14f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f650225f640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64fc06f9c0 0x7f64fc14f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f650225f640 1 -- 172.21.15.131:0/208583493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64fc150060 con 0x7f64fc15b2e0 2024-09-17T02:47:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6501a5e640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64fc06f9c0 0x7f64fc14f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:47:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f650225f640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64fc15b2e0 0x7f64fc146f40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009bd0 tx=0x7f64f4005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f64f27fc640 1 -- 172.21.15.131:0/208583493 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40bb070 con 0x7f64fc15b2e0 2024-09-17T02:47:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f64f27fc640 1 -- 172.21.15.131:0/208583493 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64f4004b40 con 0x7f64fc15b2e0 2024-09-17T02:47:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6503cc2640 1 -- 172.21.15.131:0/208583493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64fc149d30 con 0x7f64fc15b2e0 2024-09-17T02:47:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f64f27fc640 1 -- 172.21.15.131:0/208583493 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40b2040 con 0x7f64fc15b2e0 2024-09-17T02:47:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.361+0000 7f6503cc2640 1 -- 172.21.15.131:0/208583493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64fc14a2f0 con 0x7f64fc15b2e0 2024-09-17T02:47:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f64f27fc640 1 -- 172.21.15.131:0/208583493 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f64f40be070 con 0x7f64fc15b2e0 2024-09-17T02:47:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f64f27fc640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f64e0073200 0x7f64e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f64f27fc640 1 -- 172.21.15.131:0/208583493 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f64f412dff0 con 0x7f64fc15b2e0 2024-09-17T02:47:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f6501a5e640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f64e0073200 0x7f64e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650af8d000 1 Processor -- start 2024-09-17T02:47:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650af8d000 1 -- start start 2024-09-17T02:47:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f6501a5e640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f64e0073200 0x7f64e0075710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009b90 tx=0x7f64f80091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650af8d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e75ab60 0x55904e8a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650af8d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e7501c0 0x55904e8a0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650af8d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e8a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650af8d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55904e73ed00 con 0x55904e739d90 2024-09-17T02:47:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650af8d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55904e6b2c90 con 0x55904e7501c0 2024-09-17T02:47:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650af8d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55904e6a20c0 con 0x55904e75ab60 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650225f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e8a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650225f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e8a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46156/0 (socket says 172.21.15.131:46156) 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650225f640 1 -- 172.21.15.131:0/1766542027 learned_addr learned my addr 172.21.15.131:0/1766542027 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f6501a5e640 1 --2- 172.21.15.131:0/1766542027 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e75ab60 0x55904e8a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650125d640 1 --2- 172.21.15.131:0/1766542027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e7501c0 0x55904e8a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650225f640 1 -- 172.21.15.131:0/1766542027 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e75ab60 msgr2=0x55904e8a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650225f640 1 --2- 172.21.15.131:0/1766542027 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e75ab60 0x55904e8a02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650225f640 1 -- 172.21.15.131:0/1766542027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e7501c0 msgr2=0x55904e8a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650225f640 1 --2- 172.21.15.131:0/1766542027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e7501c0 0x55904e8a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650225f640 1 -- 172.21.15.131:0/1766542027 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904e67d290 con 0x55904e739d90 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.365+0000 7f650125d640 1 --2- 172.21.15.131:0/1766542027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e7501c0 0x55904e8a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650225f640 1 --2- 172.21.15.131:0/1766542027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e8a0d60 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f64f40f8750 tx=0x7f64f40f4790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d9761aee4fb1af server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f64df7fe640 1 -- 172.21.15.131:0/1766542027 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f40f82d0 con 0x55904e739d90 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f64df7fe640 1 -- 172.21.15.131:0/1766542027 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64f40fd070 con 0x55904e739d90 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f64df7fe640 1 -- 172.21.15.131:0/1766542027 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4101f00 con 0x55904e739d90 2024-09-17T02:47:00.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 -- 172.21.15.131:0/1766542027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 msgr2=0x55904e8a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 --2- 172.21.15.131:0/1766542027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e8a0d60 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f64f40f8750 tx=0x7f64f40f4790 comp rx=0 tx=0).stop 2024-09-17T02:47:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 -- 172.21.15.131:0/1766542027 shutdown_connections 2024-09-17T02:47:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 --2- 172.21.15.131:0/1766542027 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e8a0d60 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 --2- 172.21.15.131:0/1766542027 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e7501c0 0x55904e8a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 --2- 172.21.15.131:0/1766542027 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e75ab60 0x55904e8a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 -- 172.21.15.131:0/1766542027 >> 172.21.15.131:0/1766542027 conn(0x55904e35f760 msgr2=0x55904e883790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 -- 172.21.15.131:0/1766542027 shutdown_connections 2024-09-17T02:47:00.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 -- 172.21.15.131:0/1766542027 wait complete. 2024-09-17T02:47:00.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 -- start start 2024-09-17T02:47:00.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 Processor -- start 2024-09-17T02:47:00.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e739d90 0x55904e85c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e7501c0 0x55904e85c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e75ab60 0x55904e8828d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55904e63c940 con 0x55904e7501c0 2024-09-17T02:47:00.373 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55904e6a5ed0 con 0x55904e75ab60 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55904e367820 con 0x55904e739d90 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650af8d000 10 client.0 _ll_register_callbacks cb 0x55904e6807f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650125d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e7501c0 0x55904e85c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650125d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e7501c0 0x55904e85c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46172/0 (socket says 172.21.15.131:46172) 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650125d640 1 -- 172.21.15.131:0/2610451329 learned_addr learned my addr 172.21.15.131:0/2610451329 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650225f640 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e75ab60 0x55904e8828d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f6501a5e640 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e739d90 0x55904e85c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650125d640 1 -- 172.21.15.131:0/2610451329 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e739d90 msgr2=0x55904e85c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650125d640 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e739d90 0x55904e85c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650125d640 1 -- 172.21.15.131:0/2610451329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e75ab60 msgr2=0x55904e8828d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650125d640 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e75ab60 0x55904e8828d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.369+0000 7f650125d640 1 -- 172.21.15.131:0/2610451329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904e655550 con 0x55904e7501c0 2024-09-17T02:47:00.374 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650125d640 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e7501c0 0x55904e85c5b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f64ec001e40 tx=0x7f64ec00f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9d121499ab1bd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f64dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f64dcff9640 1 -- 172.21.15.131:0/2610451329 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ec0050d0 con 0x55904e7501c0 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f64dcff9640 1 -- 172.21.15.131:0/2610451329 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f64ec005270 con 0x55904e7501c0 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 -- 172.21.15.131:0/2610451329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55904e63a400 con 0x55904e7501c0 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f64dcff9640 1 -- 172.21.15.131:0/2610451329 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f64ec020050 con 0x55904e7501c0 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 10 client.6849 fetch_fsmap learned FSMap version 54 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 -- 172.21.15.131:0/2610451329 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55904e640480 con 0x55904e7501c0 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f64dcff9640 1 -- 172.21.15.131:0/2610451329 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55904e63a400 con 0x55904e7501c0 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f64dcff9640 1 -- 172.21.15.131:0/2610451329 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f64ec023020 con 0x55904e7501c0 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 10 client.6849 fetch_fsmap finished waiting for FSMap version 54 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 10 client.6849 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 20 client.6849 populate_metadata read hostname 'smithi131' 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55904e344ad0 0x55904e8b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 4 client.6849 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:47:00.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 -- 172.21.15.131:0/2610451329 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55904e6b4be0 con 0x55904e344ad0 2024-09-17T02:47:00.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f6501a5e640 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55904e344ad0 0x55904e8b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f64dcff9640 10 client.6849 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:47:00.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f6501a5e640 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55904e344ad0 0x55904e8b18a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c3e6b9e9b22755 server_cookie=d4b554e8c6e13107 in_seq=0 out_seq=0 2024-09-17T02:47:00.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f64dcff9640 1 -- 172.21.15.131:0/2610451329 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55904e6b4be0 con 0x55904e344ad0 2024-09-17T02:47:00.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f64dcff9640 10 client.6849 handle_command_reply: tid=0 2024-09-17T02:47:00.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 client.6849 shutdown 2024-09-17T02:47:00.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 -- 172.21.15.131:0/2610451329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e7501c0 msgr2=0x55904e85c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e7501c0 0x55904e85c5b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f64ec001e40 tx=0x7f64ec00f660 comp rx=0 tx=0).stop 2024-09-17T02:47:00.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 -- 172.21.15.131:0/2610451329 shutdown_connections 2024-09-17T02:47:00.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55904e344ad0 0x55904e8b18a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e75ab60 0x55904e8828d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e7501c0 0x55904e85c5b0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 --2- 172.21.15.131:0/2610451329 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e739d90 0x55904e85c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f64dcff9640 0 client.6849 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:47:00.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 -- 172.21.15.131:0/2610451329 >> 172.21.15.131:0/2610451329 conn(0x55904e35f760 msgr2=0x55904e854680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:00.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 -- 172.21.15.131:0/2610451329 shutdown_connections 2024-09-17T02:47:00.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 1 -- 172.21.15.131:0/2610451329 wait complete. 2024-09-17T02:47:00.377 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.373+0000 7f650af8d000 20 client.6849 trim_cache size 0 max 16384 2024-09-17T02:47:00.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.385+0000 7f650af8d000 1 Processor -- start 2024-09-17T02:47:00.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.385+0000 7f650af8d000 1 -- start start 2024-09-17T02:47:00.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e9113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e7501c0 0x55904e9118f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e75ab60 0x55904e911e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55904e860510 con 0x55904e739d90 2024-09-17T02:47:00.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55904e64d300 con 0x55904e75ab60 2024-09-17T02:47:00.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55904e6b1a60 con 0x55904e7501c0 2024-09-17T02:47:00.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f6501a5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e9113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f6501a5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e9113b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46182/0 (socket says 172.21.15.131:46182) 2024-09-17T02:47:00.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f6501a5e640 1 -- 172.21.15.131:0/458983869 learned_addr learned my addr 172.21.15.131:0/458983869 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:00.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f6501a5e640 1 -- 172.21.15.131:0/458983869 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e7501c0 msgr2=0x55904e9118f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:00.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f6501a5e640 1 --2- 172.21.15.131:0/458983869 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e7501c0 0x55904e9118f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f6501a5e640 1 -- 172.21.15.131:0/458983869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e75ab60 msgr2=0x55904e911e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:00.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f6501a5e640 1 --2- 172.21.15.131:0/458983869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e75ab60 0x55904e911e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f6501a5e640 1 -- 172.21.15.131:0/458983869 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904e727d10 con 0x55904e739d90 2024-09-17T02:47:00.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f6501a5e640 1 --2- 172.21.15.131:0/458983869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e9113b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f64f80042f0 tx=0x7f64f8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3293c2f43bb2dbb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:00.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f64deffd640 1 -- 172.21.15.131:0/458983869 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8030070 con 0x55904e739d90 2024-09-17T02:47:00.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f64deffd640 1 -- 172.21.15.131:0/458983869 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64f8016070 con 0x55904e739d90 2024-09-17T02:47:00.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f64deffd640 1 -- 172.21.15.131:0/458983869 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8026030 con 0x55904e739d90 2024-09-17T02:47:00.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 -- 172.21.15.131:0/458983869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 msgr2=0x55904e9113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 --2- 172.21.15.131:0/458983869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e9113b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f64f80042f0 tx=0x7f64f8014000 comp rx=0 tx=0).stop 2024-09-17T02:47:00.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 -- 172.21.15.131:0/458983869 shutdown_connections 2024-09-17T02:47:00.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 --2- 172.21.15.131:0/458983869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e75ab60 0x55904e911e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 --2- 172.21.15.131:0/458983869 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e7501c0 0x55904e9118f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 --2- 172.21.15.131:0/458983869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e9113b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 -- 172.21.15.131:0/458983869 >> 172.21.15.131:0/458983869 conn(0x55904e35f760 msgr2=0x55904e856690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:00.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 -- 172.21.15.131:0/458983869 shutdown_connections 2024-09-17T02:47:00.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 -- 172.21.15.131:0/458983869 wait complete. 2024-09-17T02:47:00.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 -- start start 2024-09-17T02:47:00.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.389+0000 7f650af8d000 1 Processor -- start 2024-09-17T02:47:00.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e915810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e7501c0 0x55904e915d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e75ab60 0x55904e916290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55904e66e300 con 0x55904e739d90 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55904e65f670 con 0x55904e7501c0 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55904e6a51e0 con 0x55904e75ab60 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 10 client.0 _ll_register_callbacks cb 0x7f64ec007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f6501a5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e915810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f6501a5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e915810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46188/0 (socket says 172.21.15.131:46188) 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f6501a5e640 1 -- 172.21.15.131:0/848682834 learned_addr learned my addr 172.21.15.131:0/848682834 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650125d640 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e7501c0 0x55904e915d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650225f640 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e75ab60 0x55904e916290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f6501a5e640 1 -- 172.21.15.131:0/848682834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e75ab60 msgr2=0x55904e916290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f6501a5e640 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e75ab60 0x55904e916290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f6501a5e640 1 -- 172.21.15.131:0/848682834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e7501c0 msgr2=0x55904e915d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f6501a5e640 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e7501c0 0x55904e915d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f6501a5e640 1 -- 172.21.15.131:0/848682834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904e658e50 con 0x55904e739d90 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f6501a5e640 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e915810 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f64f8003fd0 tx=0x7f64f80045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e73ed62bffcb8eb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f64dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f64dcff9640 1 -- 172.21.15.131:0/848682834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f803b060 con 0x55904e739d90 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f64dcff9640 1 -- 172.21.15.131:0/848682834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f64f8012550 con 0x55904e739d90 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 -- 172.21.15.131:0/848682834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f8016070 con 0x55904e739d90 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f64dcff9640 1 -- 172.21.15.131:0/848682834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f64f8041050 con 0x55904e739d90 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 10 client.6855 fetch_fsmap learned FSMap version 54 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 -- 172.21.15.131:0/848682834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55904e6472d0 con 0x55904e739d90 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f64dcff9640 1 -- 172.21.15.131:0/848682834 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64f8016070 con 0x55904e739d90 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f64dcff9640 1 -- 172.21.15.131:0/848682834 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f64f8044020 con 0x55904e739d90 2024-09-17T02:47:00.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 10 client.6855 fetch_fsmap finished waiting for FSMap version 54 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 10 client.6855 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 20 client.6855 populate_metadata read hostname 'smithi131' 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55904e332b30 0x55904e8b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 4 client.6855 mds_command: new command op to 6609 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 -- 172.21.15.131:0/848682834 --> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55904e6b0690 con 0x55904e332b30 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650125d640 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55904e332b30 0x55904e8b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f64dcff9640 10 client.6855 ms_handle_connect on v2:172.21.15.131:6833/962976667 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650125d640 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55904e332b30 0x55904e8b3ad0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7a642bcca391d44 server_cookie=79a01fea189fbca4 in_seq=0 out_seq=0 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f64dcff9640 1 -- 172.21.15.131:0/848682834 <== mds.0 v2:172.21.15.131:6833/962976667 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55904e6b0690 con 0x55904e332b30 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f64dcff9640 10 client.6855 handle_command_reply: tid=0 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 client.6855 shutdown 2024-09-17T02:47:00.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 -- 172.21.15.131:0/848682834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 msgr2=0x55904e915810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e915810 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f64f8003fd0 tx=0x7f64f80045c0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 -- 172.21.15.131:0/848682834 shutdown_connections 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.131:6833/962976667,v1:172.21.15.131:6835/962976667] conn(0x55904e332b30 0x55904e8b3ad0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55904e75ab60 0x55904e916290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55904e7501c0 0x55904e915d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 --2- 172.21.15.131:0/848682834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55904e739d90 0x55904e915810 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 -- 172.21.15.131:0/848682834 >> 172.21.15.131:0/848682834 conn(0x55904e35f760 msgr2=0x55904e8acce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f64dcff9640 0 client.6855 ms_handle_reset on v2:172.21.15.131:6833/962976667 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 -- 172.21.15.131:0/848682834 shutdown_connections 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 1 -- 172.21.15.131:0/848682834 wait complete. 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.393+0000 7f650af8d000 20 client.6855 trim_cache size 0 max 16384 2024-09-17T02:47:00.397 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:00.398 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-09-17T02:47:00.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 -- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f64e0073200 msgr2=0x7f64e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f64e0073200 0x7f64e0075710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009b90 tx=0x7f64f80091e0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 -- 172.21.15.131:0/208583493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64fc15b2e0 msgr2=0x7f64fc146f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64fc15b2e0 0x7f64fc146f40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009bd0 tx=0x7f64f4005b00 comp rx=0 tx=0).stop 2024-09-17T02:47:00.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 -- 172.21.15.131:0/208583493 shutdown_connections 2024-09-17T02:47:00.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f64e0073200 0x7f64e0075710 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f64fc15b2e0 0x7f64fc146f40 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f64fc06fdd0 0x7f64fc1f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 --2- 172.21.15.131:0/208583493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f64fc06f9c0 0x7f64fc14f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 -- 172.21.15.131:0/208583493 >> 172.21.15.131:0/208583493 conn(0x7f64fc06eea0 msgr2=0x7f64fc0cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:00.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 -- 172.21.15.131:0/208583493 shutdown_connections 2024-09-17T02:47:00.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.397+0000 7f6503cc2640 1 -- 172.21.15.131:0/208583493 wait complete. 2024-09-17T02:47:00.413 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T02:47:00.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.549+0000 7f8c3577f640 1 -- 172.21.15.131:0/2360738512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c300cab60 msgr2=0x7f8c300caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.549+0000 7f8c3577f640 1 --2- 172.21.15.131:0/2360738512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c300cab60 0x7f8c300caf70 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009900 tx=0x7f8c2001c6d0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.549+0000 7f8c3577f640 1 -- 172.21.15.131:0/2360738512 shutdown_connections 2024-09-17T02:47:00.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.549+0000 7f8c3577f640 1 --2- 172.21.15.131:0/2360738512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c300cab60 0x7f8c300caf70 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.549+0000 7f8c3577f640 1 --2- 172.21.15.131:0/2360738512 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c3006dea0 0x7f8c3006e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.549+0000 7f8c3577f640 1 -- 172.21.15.131:0/2360738512 >> 172.21.15.131:0/2360738512 conn(0x7f8c3006eea0 msgr2=0x7f8c3006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:00.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.549+0000 7f8c3577f640 1 -- 172.21.15.131:0/2360738512 shutdown_connections 2024-09-17T02:47:00.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.549+0000 7f8c3577f640 1 -- 172.21.15.131:0/2360738512 wait complete. 2024-09-17T02:47:00.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.553+0000 7f8c3577f640 1 Processor -- start 2024-09-17T02:47:00.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.553+0000 7f8c3577f640 1 -- start start 2024-09-17T02:47:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.553+0000 7f8c3577f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c3006dea0 0x7f8c301f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.553+0000 7f8c3577f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c300cab60 0x7f8c3014ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.553+0000 7f8c3577f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c30155240 0x7f8c301f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.553+0000 7f8c3577f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8c30168430 con 0x7f8c300cab60 2024-09-17T02:47:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.553+0000 7f8c3577f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301682b0 con 0x7f8c30155240 2024-09-17T02:47:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.553+0000 7f8c3577f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8c301685b0 con 0x7f8c3006dea0 2024-09-17T02:47:00.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c300cab60 0x7f8c3014ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c300cab60 0x7f8c3014ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46200/0 (socket says 172.21.15.131:46200) 2024-09-17T02:47:00.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2e7fc640 1 -- 172.21.15.131:0/924095481 learned_addr learned my addr 172.21.15.131:0/924095481 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:00.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2effd640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c3006dea0 0x7f8c301f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2f7fe640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c30155240 0x7f8c301f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2e7fc640 1 -- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c3006dea0 msgr2=0x7f8c301f9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2e7fc640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c3006dea0 0x7f8c301f9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2e7fc640 1 -- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c30155240 msgr2=0x7f8c301f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2e7fc640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c30155240 0x7f8c301f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2e7fc640 1 -- 172.21.15.131:0/924095481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c301fa020 con 0x7f8c300cab60 2024-09-17T02:47:00.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2effd640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c3006dea0 0x7f8c301f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:00.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2e7fc640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c300cab60 0x7f8c3014ed80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400c940 tx=0x7f8c2400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:00.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c177fe640 1 -- 172.21.15.131:0/924095481 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24004220 con 0x7f8c300cab60 2024-09-17T02:47:00.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c177fe640 1 -- 172.21.15.131:0/924095481 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c240079b0 con 0x7f8c300cab60 2024-09-17T02:47:00.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c177fe640 1 -- 172.21.15.131:0/924095481 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c240044b0 con 0x7f8c300cab60 2024-09-17T02:47:00.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c3577f640 1 -- 172.21.15.131:0/924095481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c30151df0 con 0x7f8c300cab60 2024-09-17T02:47:00.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c3577f640 1 -- 172.21.15.131:0/924095481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c301523b0 con 0x7f8c300cab60 2024-09-17T02:47:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c3577f640 1 -- 172.21.15.131:0/924095481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c3006e2b0 con 0x7f8c300cab60 2024-09-17T02:47:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c177fe640 1 -- 172.21.15.131:0/924095481 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8c24003600 con 0x7f8c300cab60 2024-09-17T02:47:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c177fe640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c1c073230 0x7f8c1c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c177fe640 1 -- 172.21.15.131:0/924095481 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f8c24094bd0 con 0x7f8c300cab60 2024-09-17T02:47:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.557+0000 7f8c2effd640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c1c073230 0x7f8c1c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.561+0000 7f8c2effd640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c1c073230 0x7f8c1c075740 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c28007e90 tx=0x7f8c280073a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:00.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.561+0000 7f8c177fe640 1 -- 172.21.15.131:0/924095481 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8c301fa020 con 0x7f8c300cab60 2024-09-17T02:47:00.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.753+0000 7f8c3577f640 1 -- 172.21.15.131:0/924095481 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f8c300c6c80 con 0x7f8c300cab60 2024-09-17T02:47:00.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.753+0000 7f8c177fe640 1 -- 172.21.15.131:0/924095481 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8c24014030 con 0x7f8c300cab60 2024-09-17T02:47:00.753 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:00.753 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T02:47:00.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 -- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c1c073230 msgr2=0x7f8c1c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c1c073230 0x7f8c1c075740 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c28007e90 tx=0x7f8c280073a0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 -- 172.21.15.131:0/924095481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c300cab60 msgr2=0x7f8c3014ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c300cab60 0x7f8c3014ed80 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400c940 tx=0x7f8c2400ce00 comp rx=0 tx=0).stop 2024-09-17T02:47:00.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 -- 172.21.15.131:0/924095481 shutdown_connections 2024-09-17T02:47:00.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c1c073230 0x7f8c1c075740 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c30155240 0x7f8c301f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c300cab60 0x7f8c3014ed80 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 --2- 172.21.15.131:0/924095481 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c3006dea0 0x7f8c301f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 -- 172.21.15.131:0/924095481 >> 172.21.15.131:0/924095481 conn(0x7f8c3006eea0 msgr2=0x7f8c301544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:00.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 -- 172.21.15.131:0/924095481 shutdown_connections 2024-09-17T02:47:00.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.757+0000 7f8c3577f640 1 -- 172.21.15.131:0/924095481 wait complete. 2024-09-17T02:47:00.773 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T02:47:00.774 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:47:00.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.925+0000 7f625126d640 1 -- 172.21.15.131:0/856467429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6238003a10 msgr2=0x7f6238003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.925+0000 7f625126d640 1 --2- 172.21.15.131:0/856467429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6238003a10 0x7f6238003e80 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f624400c640 tx=0x7f624401d320 comp rx=0 tx=0).stop 2024-09-17T02:47:00.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.925+0000 7f624b7fe640 1 -- 172.21.15.131:0/856467429 reap_dead start 2024-09-17T02:47:00.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.925+0000 7f625126d640 1 -- 172.21.15.131:0/856467429 shutdown_connections 2024-09-17T02:47:00.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.925+0000 7f625126d640 1 -- 172.21.15.131:0/856467429 >> 172.21.15.131:0/856467429 conn(0x7f624c06eea0 msgr2=0x7f624c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:00.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.925+0000 7f625126d640 1 -- 172.21.15.131:0/856467429 shutdown_connections 2024-09-17T02:47:00.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.925+0000 7f625126d640 1 -- 172.21.15.131:0/856467429 wait complete. 2024-09-17T02:47:00.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.925+0000 7f625126d640 1 Processor -- start 2024-09-17T02:47:00.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.925+0000 7f625126d640 1 -- start start 2024-09-17T02:47:00.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f625126d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c151ff0 0x7f624c152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f625126d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f624c152b60 0x7f624c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f625126d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f624c15d810 0x7f624c0b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f625126d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f624c167c50 con 0x7f624c15d810 2024-09-17T02:47:00.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f625126d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f624c167ad0 con 0x7f624c151ff0 2024-09-17T02:47:00.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f625126d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f624c167dd0 con 0x7f624c152b60 2024-09-17T02:47:00.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f624c15d810 0x7f624c0b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f624c15d810 0x7f624c0b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46220/0 (socket says 172.21.15.131:46220) 2024-09-17T02:47:00.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624b7fe640 1 -- 172.21.15.131:0/1179110156 learned_addr learned my addr 172.21.15.131:0/1179110156 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624affd640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c151ff0 0x7f624c152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624a7fc640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f624c152b60 0x7f624c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624b7fe640 1 -- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f624c152b60 msgr2=0x7f624c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624b7fe640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f624c152b60 0x7f624c15d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624b7fe640 1 -- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c151ff0 msgr2=0x7f624c152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624b7fe640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c151ff0 0x7f624c152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624b7fe640 1 -- 172.21.15.131:0/1179110156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f624c152400 con 0x7f624c15d810 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624affd640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c151ff0 0x7f624c152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f624b7fe640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f624c15d810 0x7f624c0b4130 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f624c0727c0 tx=0x7f623c00f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f62437fe640 1 -- 172.21.15.131:0/1179110156 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c005220 con 0x7f624c15d810 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f62437fe640 1 -- 172.21.15.131:0/1179110156 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f623c0054d0 con 0x7f624c15d810 2024-09-17T02:47:00.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f62437fe640 1 -- 172.21.15.131:0/1179110156 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c004a10 con 0x7f624c15d810 2024-09-17T02:47:00.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f625126d640 1 -- 172.21.15.131:0/1179110156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624c0b3090 con 0x7f624c15d810 2024-09-17T02:47:00.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f625126d640 1 -- 172.21.15.131:0/1179110156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f624c0b3650 con 0x7f624c15d810 2024-09-17T02:47:00.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f625126d640 1 -- 172.21.15.131:0/1179110156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f624c0c50a0 con 0x7f624c15d810 2024-09-17T02:47:00.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.929+0000 7f62437fe640 1 -- 172.21.15.131:0/1179110156 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f623c019030 con 0x7f624c15d810 2024-09-17T02:47:00.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.933+0000 7f62437fe640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f622c073230 0x7f622c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:00.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.933+0000 7f62437fe640 1 -- 172.21.15.131:0/1179110156 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f623c094830 con 0x7f624c15d810 2024-09-17T02:47:00.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.933+0000 7f624affd640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f622c073230 0x7f622c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:00.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.933+0000 7f624affd640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f622c073230 0x7f622c075740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f62440ac000 tx=0x7f6244006040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:00.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:00.933+0000 7f62437fe640 1 -- 172.21.15.131:0/1179110156 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f623c020030 con 0x7f624c15d810 2024-09-17T02:47:01.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.137+0000 7f625126d640 1 -- 172.21.15.131:0/1179110156 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f624c0c4a70 con 0x7f624c15d810 2024-09-17T02:47:01.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f62437fe640 1 -- 172.21.15.131:0/1179110156 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7f623c05f870 con 0x7f624c15d810 2024-09-17T02:47:01.137 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:47:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 -- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f622c073230 msgr2=0x7f622c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f622c073230 0x7f622c075740 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f62440ac000 tx=0x7f6244006040 comp rx=0 tx=0).stop 2024-09-17T02:47:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 -- 172.21.15.131:0/1179110156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f624c15d810 msgr2=0x7f624c0b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f624c15d810 0x7f624c0b4130 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f624c0727c0 tx=0x7f623c00f960 comp rx=0 tx=0).stop 2024-09-17T02:47:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 -- 172.21.15.131:0/1179110156 shutdown_connections 2024-09-17T02:47:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f622c073230 0x7f622c075740 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f624c15d810 0x7f624c0b4130 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f624c152b60 0x7f624c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:01.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 --2- 172.21.15.131:0/1179110156 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f624c151ff0 0x7f624c152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:01.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 -- 172.21.15.131:0/1179110156 >> 172.21.15.131:0/1179110156 conn(0x7f624c06eea0 msgr2=0x7f624c155880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:01.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 -- 172.21.15.131:0/1179110156 shutdown_connections 2024-09-17T02:47:01.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01.141+0000 7f625126d640 1 -- 172.21.15.131:0/1179110156 wait complete. 2024-09-17T02:47:01.158 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:47:01.158 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:47:01.158 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:47:01.158 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:47:01.158 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:01.158 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:47:01.158 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:47:01.158 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:47:01.158 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:47:01.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:01.159 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:47:01.163 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:47:01.164 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42085sec preferred_lft 42085sec 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:01.165 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:47:01.166 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:01.166 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:47:01.166 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:01.166 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:47:01.166 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:01.166 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:47:01.166 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:01.166 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:47:01.166 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:01.166 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:47:01.166 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:01.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:01.167 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:47:01.215 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:47:01.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:01.217 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:47:01.269 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:47:01.270 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:47:01.270 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:47:01.270 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:47:01.270 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:47:01.270 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:47:01.270 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:47:01.270 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:47:01.270 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:47:01.270 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:47:01.270 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:47:01.271 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:47:01.271 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:47:01.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:47:01.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:47:01.271 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:47:01.271 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:47:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:47:01.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:47:01.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:01.392 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:47:01.438 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:47:01.438 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:47:01.439 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:47:01.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:01.439 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:47:01.498 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:47:01.498 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:47:01.499 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:01.550 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:47:01.550 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:47:01.550 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:01.602 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:47:01.663 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:47:01.711 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:47:01.711 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:47:01.711 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:47:01.711 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:47:01.711 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:47:01.711 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:47:01.711 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:47:01.711 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:47:01.711 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:47:01.711 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:47:01.712 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:47:01.713 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:47:01.713 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:47:01.713 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:47:01.713 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:47:01.713 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:47:01.713 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:47:01.713 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:01.713 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:01.713 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:01.713 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:01.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:01.714 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:47:01.762 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:47:01.762 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:47:01.807 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:47:01.821 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:47:01.870 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:47:01.870 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:47:01.870 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:47:01.870 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:47:01.870 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:47:01.871 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:01.872 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:01.873 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:01.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:01.874 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:47:01.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:47:01.921+0000 7ff550c08580 -1 init, newargv = 0x56393bd0f670 newargc=15 2024-09-17T02:47:01.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[44937]: starting ceph client 2024-09-17T02:47:01.925 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[44937]: starting fuse 2024-09-17T02:47:03.023 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:47:03.039 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:47:03.087 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:47:03.087 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:47:03.087 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:47:03.088 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:03.089 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:03.090 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:03.090 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:03.090 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:03.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:03.091 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:47:03.138 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:47:03.139 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:47:03.139 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:47:03.139 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:47:03.139 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:47:03.139 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:47:03.139 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:47:03.140 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:03.140 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:47:03.140 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:47:03.140 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:47:03.140 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:47:03.140 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:03.140 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:47:03.140 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:47:03.140 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:47:03.141 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:47:03.141 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:03.141 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:47:03.141 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:47:03.141 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:47:03.141 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:47:03.142 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:47:03.142 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:47:03.142 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:47:03.142 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:47:03.142 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:03.143 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:47:03.143 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:47:03.267 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.44937.asok 2024-09-17T02:47:03.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:47:03.279 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44937.asok 2024-09-17T02:47:03.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:03.280 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44937.asok status 2024-09-17T02:47:03.455 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:47:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:47:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:47:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:47:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:47:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:47:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:47:03.455 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "44937", 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 6873, 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 6873 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:47:03.456 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 731165427 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 731165427 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.6873 192.168.0.1:0/731165427", 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/731165427", 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 54, 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 80, 2024-09-17T02:47:03.457 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:47:03.458 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:47:03.458 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:47:03.458 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:47:03.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:03.472 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:47:03.472 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:03.478 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:47:03.478 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:03.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:03.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:03.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:03.534 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-17T02:47:03.586 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:47:03.587 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-17T02:47:03.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.701+0000 7fcede070080 1 Processor -- start 2024-09-17T02:47:03.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.701+0000 7fcede070080 1 -- start start 2024-09-17T02:47:03.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5560109e4360 0x5560109e5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:03.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556010a2b890 0x556010a23720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:03.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010ab5ed0 0x556010ab1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:03.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556010930a00 con 0x556010ab5ed0 2024-09-17T02:47:03.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55601098ef40 con 0x556010a2b890 2024-09-17T02:47:03.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x556010ab79c0 con 0x5560109e4360 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced8348640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010ab5ed0 0x556010ab1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced8348640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010ab5ed0 0x556010ab1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39292/0 (socket says 172.21.15.131:39292) 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced8348640 1 -- 172.21.15.131:0/2392283131 learned_addr learned my addr 172.21.15.131:0/2392283131 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced7346640 1 --2- 172.21.15.131:0/2392283131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556010a2b890 0x556010a23720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced7b47640 1 --2- 172.21.15.131:0/2392283131 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5560109e4360 0x5560109e5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced8348640 1 -- 172.21.15.131:0/2392283131 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5560109e4360 msgr2=0x5560109e5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced8348640 1 --2- 172.21.15.131:0/2392283131 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5560109e4360 0x5560109e5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced8348640 1 -- 172.21.15.131:0/2392283131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556010a2b890 msgr2=0x556010a23720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced8348640 1 --2- 172.21.15.131:0/2392283131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556010a2b890 0x556010a23720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced8348640 1 -- 172.21.15.131:0/2392283131 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556010909c10 con 0x556010ab5ed0 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced7b47640 1 --2- 172.21.15.131:0/2392283131 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5560109e4360 0x5560109e5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced8348640 1 --2- 172.21.15.131:0/2392283131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010ab5ed0 0x556010ab1380 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fced0018ed0 tx=0x7fced0018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3137f3e7dbfceed9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced6344640 1 -- 172.21.15.131:0/2392283131 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced00b0040 con 0x556010ab5ed0 2024-09-17T02:47:03.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced6344640 1 -- 172.21.15.131:0/2392283131 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fced00ba070 con 0x556010ab5ed0 2024-09-17T02:47:03.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fced6344640 1 -- 172.21.15.131:0/2392283131 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced00b5410 con 0x556010ab5ed0 2024-09-17T02:47:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 -- 172.21.15.131:0/2392283131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010ab5ed0 msgr2=0x556010ab1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 --2- 172.21.15.131:0/2392283131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010ab5ed0 0x556010ab1380 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fced0018ed0 tx=0x7fced0018040 comp rx=0 tx=0).stop 2024-09-17T02:47:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 -- 172.21.15.131:0/2392283131 shutdown_connections 2024-09-17T02:47:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 --2- 172.21.15.131:0/2392283131 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010ab5ed0 0x556010ab1380 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 --2- 172.21.15.131:0/2392283131 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556010a2b890 0x556010a23720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 --2- 172.21.15.131:0/2392283131 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5560109e4360 0x5560109e5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 -- 172.21.15.131:0/2392283131 >> 172.21.15.131:0/2392283131 conn(0x5560109ced00 msgr2=0x556010a296d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 -- 172.21.15.131:0/2392283131 shutdown_connections 2024-09-17T02:47:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.705+0000 7fcede070080 1 -- 172.21.15.131:0/2392283131 wait complete. 2024-09-17T02:47:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fcede070080 1 Processor -- start 2024-09-17T02:47:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fcede070080 1 -- start start 2024-09-17T02:47:03.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fcede070080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5560109e4360 0x556010ab3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fcede070080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010a2b890 0x556010ab2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fcede070080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556010ab5ed0 0x556010b50f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fcede070080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x556010a13320 con 0x556010a2b890 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fcede070080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x556010ab0d80 con 0x556010ab5ed0 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fcede070080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x556010a135c0 con 0x5560109e4360 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced7346640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010a2b890 0x556010ab2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced7346640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010a2b890 0x556010ab2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39298/0 (socket says 172.21.15.131:39298) 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced7346640 1 -- 172.21.15.131:0/3156344067 learned_addr learned my addr 172.21.15.131:0/3156344067 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced7b47640 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5560109e4360 0x556010ab3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced8348640 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556010ab5ed0 0x556010b50f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced7346640 1 -- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5560109e4360 msgr2=0x556010ab3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced7346640 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5560109e4360 0x556010ab3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced7346640 1 -- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556010ab5ed0 msgr2=0x556010b50f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced7346640 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556010ab5ed0 0x556010b50f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced7346640 1 -- 172.21.15.131:0/3156344067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556010ab2c70 con 0x556010a2b890 2024-09-17T02:47:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced8348640 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x556010ab5ed0 0x556010b50f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:47:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced7346640 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010a2b890 0x556010ab2360 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fcec000df90 tx=0x7fcec000b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced5342640 1 -- 172.21.15.131:0/3156344067 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec0014020 con 0x556010a2b890 2024-09-17T02:47:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced5342640 1 -- 172.21.15.131:0/3156344067 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcec00015a0 con 0x556010a2b890 2024-09-17T02:47:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fcede070080 1 -- 172.21.15.131:0/3156344067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556010ab2f10 con 0x556010a2b890 2024-09-17T02:47:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fced5342640 1 -- 172.21.15.131:0/3156344067 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec000a040 con 0x556010a2b890 2024-09-17T02:47:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.709+0000 7fcede070080 1 -- 172.21.15.131:0/3156344067 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556010ab3450 con 0x556010a2b890 2024-09-17T02:47:03.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.713+0000 7fced5342640 1 -- 172.21.15.131:0/3156344067 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcec0017020 con 0x556010a2b890 2024-09-17T02:47:03.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.713+0000 7fced5342640 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fceb8071be0 0x7fceb80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:03.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.713+0000 7fced5342640 1 -- 172.21.15.131:0/3156344067 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fcec0092fe0 con 0x556010a2b890 2024-09-17T02:47:03.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.713+0000 7fcede070080 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5560109e0c90 0x5560109e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:03.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.713+0000 7fced7b47640 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fceb8071be0 0x7fceb80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:03.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.713+0000 7fcede070080 1 -- 172.21.15.131:0/3156344067 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80) -- 0x556010b51880 con 0x5560109e0c90 2024-09-17T02:47:03.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.713+0000 7fced8348640 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5560109e0c90 0x5560109e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:03.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.713+0000 7fced7b47640 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fceb8071be0 0x7fceb80740f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcec800e990 tx=0x7fcec8004ba0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:03.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.713+0000 7fced8348640 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5560109e0c90 0x5560109e10a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:03.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fced8348640 1 -- 172.21.15.131:0/3156344067 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v80'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fced00b3040 con 0x5560109e0c90 2024-09-17T02:47:03.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fcede070080 1 -- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5560109e0c90 msgr2=0x5560109e10a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:03.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fcede070080 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5560109e0c90 0x5560109e10a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:03.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fcede070080 1 -- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fceb8071be0 msgr2=0x7fceb80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:03.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fcede070080 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fceb8071be0 0x7fceb80740f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcec800e990 tx=0x7fcec8004ba0 comp rx=0 tx=0).stop 2024-09-17T02:47:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fcede070080 1 -- 172.21.15.131:0/3156344067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010a2b890 msgr2=0x556010ab2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fcede070080 1 --2- 172.21.15.131:0/3156344067 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x556010a2b890 0x556010ab2360 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fcec000df90 tx=0x7fcec000b680 comp rx=0 tx=0).stop 2024-09-17T02:47:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fced8348640 1 -- 172.21.15.131:0/3156344067 reap_dead start 2024-09-17T02:47:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fcede070080 1 -- 172.21.15.131:0/3156344067 shutdown_connections 2024-09-17T02:47:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fcede070080 1 -- 172.21.15.131:0/3156344067 >> 172.21.15.131:0/3156344067 conn(0x5560109ced00 msgr2=0x556010a296d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fcede070080 1 -- 172.21.15.131:0/3156344067 shutdown_connections 2024-09-17T02:47:03.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:03.717+0000 7fcede070080 1 -- 172.21.15.131:0/3156344067 wait complete. 2024-09-17T02:47:03.719 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:47:03.720 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:47:03.791 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:47:03.853 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:47:03.853 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:47:03.853 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:47:03.853 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:47:03.859 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:47:03.859 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:47:03.906 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:47:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 -- 172.21.15.131:0/323455660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c0c53a0 msgr2=0x7f5a1c0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 --2- 172.21.15.131:0/323455660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c0c53a0 0x7f5a1c0cf010 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5a14009840 tx=0x7f5a1401c790 comp rx=0 tx=0).stop 2024-09-17T02:47:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 -- 172.21.15.131:0/323455660 shutdown_connections 2024-09-17T02:47:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 --2- 172.21.15.131:0/323455660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c0c53a0 0x7f5a1c0cf010 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 -- 172.21.15.131:0/323455660 >> 172.21.15.131:0/323455660 conn(0x7f5a1c06eea0 msgr2=0x7f5a1c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 -- 172.21.15.131:0/323455660 shutdown_connections 2024-09-17T02:47:04.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 -- 172.21.15.131:0/323455660 wait complete. 2024-09-17T02:47:04.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 Processor -- start 2024-09-17T02:47:04.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 -- start start 2024-09-17T02:47:04.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a1c06e2c0 0x7f5a1c0c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:04.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c0c53a0 0x7f5a1c1f9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:04.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a1c0cec00 0x7f5a1c1fa230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:04.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c16ae80 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c16ad00 con 0x7f5a1c0cec00 2024-09-17T02:47:04.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a22c87640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5a1c16b000 con 0x7f5a1c06e2c0 2024-09-17T02:47:04.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a1bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c0c53a0 0x7f5a1c1f9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:04.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a1bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c0c53a0 0x7f5a1c1f9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39318/0 (socket says 172.21.15.131:39318) 2024-09-17T02:47:04.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.117+0000 7f5a1bfff640 1 -- 172.21.15.131:0/3509104034 learned_addr learned my addr 172.21.15.131:0/3509104034 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a1bfff640 1 -- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a1c06e2c0 msgr2=0x7f5a1c0c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a1bfff640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a1c06e2c0 0x7f5a1c0c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a21224640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a1c0cec00 0x7f5a1c1fa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a1bfff640 1 -- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a1c0cec00 msgr2=0x7f5a1c1fa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a1bfff640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a1c0cec00 0x7f5a1c1fa230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a1bfff640 1 -- 172.21.15.131:0/3509104034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c155010 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a1bfff640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c0c53a0 0x7f5a1c1f9cf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400cbc0 tx=0x7f5a0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a197fa640 1 -- 172.21.15.131:0/3509104034 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04016020 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a197fa640 1 -- 172.21.15.131:0/3509104034 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a0400aae0 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a197fa640 1 -- 172.21.15.131:0/3509104034 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a04012420 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a22c87640 1 -- 172.21.15.131:0/3509104034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c1552f0 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a22c87640 1 -- 172.21.15.131:0/3509104034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c1558b0 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a22c87640 1 -- 172.21.15.131:0/3509104034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1c0cf010 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a197fa640 1 -- 172.21.15.131:0/3509104034 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a04019020 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.121+0000 7f5a197fa640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59fc073230 0x7f59fc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:04.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.125+0000 7f5a197fa640 1 -- 172.21.15.131:0/3509104034 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f5a04094990 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.125+0000 7f5a20a23640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59fc073230 0x7f59fc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:04.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.125+0000 7f5a20a23640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59fc073230 0x7f59fc075740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009670 tx=0x7f5a0c006bd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:04.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.129+0000 7f5a197fa640 1 -- 172.21.15.131:0/3509104034 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a04020020 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.305+0000 7f5a22c87640 1 -- 172.21.15.131:0/3509104034 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5a1c073310 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.305+0000 7f5a197fa640 1 -- 172.21.15.131:0/3509104034 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v80) ==== 80+0+122 (secure 0 0 0) 0x7f5a0405f940 con 0x7f5a1c0c53a0 2024-09-17T02:47:04.303 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6835/655402879 2024-09-18T02:45:41.796095+0000 2024-09-17T02:47:04.304 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6833/655402879 2024-09-18T02:45:41.796095+0000 2024-09-17T02:47:04.304 INFO:teuthology.orchestra.run.smithi131.stderr:listed 2 entries 2024-09-17T02:47:04.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 -- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59fc073230 msgr2=0x7f59fc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:04.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59fc073230 0x7f59fc075740 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009670 tx=0x7f5a0c006bd0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 -- 172.21.15.131:0/3509104034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c0c53a0 msgr2=0x7f5a1c1f9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:04.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c0c53a0 0x7f5a1c1f9cf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400cbc0 tx=0x7f5a0400ccb0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 -- 172.21.15.131:0/3509104034 shutdown_connections 2024-09-17T02:47:04.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59fc073230 0x7f59fc075740 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a1c0cec00 0x7f5a1c1fa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a1c0c53a0 0x7f5a1c1f9cf0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 --2- 172.21.15.131:0/3509104034 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a1c06e2c0 0x7f5a1c0c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 -- 172.21.15.131:0/3509104034 >> 172.21.15.131:0/3509104034 conn(0x7f5a1c06eea0 msgr2=0x7f5a1c152140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:04.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 -- 172.21.15.131:0/3509104034 shutdown_connections 2024-09-17T02:47:04.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.309+0000 7f5a22c87640 1 -- 172.21.15.131:0/3509104034 wait complete. 2024-09-17T02:47:04.324 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:47:04.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:04.324 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:04.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[44937]: fuse finished with error 0 and tester_r 0 2024-09-17T02:47:04.335 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:47:04.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:04.335 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:47:04.485 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:47:04.485 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:47:04.485 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:47:04.491 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:47:04.492 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:47:04.492 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:47:04.492 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:47:04.534 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:47:04.535 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:47:04.535 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:47:04.535 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:47:04.540 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:47:04.541 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:47:04.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.737+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2726309231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf20005340 msgr2=0x7fbf2000b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:04.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.737+0000 7fbf3ed40640 1 --2- 172.21.15.131:0/2726309231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf20005340 0x7fbf2000b9f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009e00 tx=0x7fbf3001d170 comp rx=0 tx=0).stop 2024-09-17T02:47:04.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.737+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2726309231 shutdown_connections 2024-09-17T02:47:04.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.737+0000 7fbf3ed40640 1 --2- 172.21.15.131:0/2726309231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf20005340 0x7fbf2000b9f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.737+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2726309231 >> 172.21.15.131:0/2726309231 conn(0x7fbf380cda20 msgr2=0x7fbf380cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:04.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.737+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2726309231 shutdown_connections 2024-09-17T02:47:04.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.737+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2726309231 wait complete. 2024-09-17T02:47:04.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3ed40640 1 Processor -- start 2024-09-17T02:47:04.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3ed40640 1 -- start start 2024-09-17T02:47:04.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3ed40640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf20005340 0x7fbf380b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3ed40640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf380b2200 0x7fbf381f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3ed40640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf381f5d70 0x7fbf381f6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3ed40640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbf38074080 con 0x7fbf380b2200 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3ed40640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf380b6f90 con 0x7fbf381f5d70 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3ed40640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fbf38074200 con 0x7fbf20005340 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf37fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf380b2200 0x7fbf381f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf37fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf380b2200 0x7fbf381f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39328/0 (socket says 172.21.15.131:39328) 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf37fff640 1 -- 172.21.15.131:0/2876891367 learned_addr learned my addr 172.21.15.131:0/2876891367 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3cadc640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf20005340 0x7fbf380b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3d2dd640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf381f5d70 0x7fbf381f6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf37fff640 1 -- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf20005340 msgr2=0x7fbf380b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:04.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf37fff640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf20005340 0x7fbf380b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf37fff640 1 -- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf381f5d70 msgr2=0x7fbf381f6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:04.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf37fff640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf381f5d70 0x7fbf381f6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf37fff640 1 -- 172.21.15.131:0/2876891367 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf380b2610 con 0x7fbf380b2200 2024-09-17T02:47:04.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf37fff640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf380b2200 0x7fbf381f5830 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0015a0 tx=0x7fbf2c00e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:04.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf357fa640 1 -- 172.21.15.131:0/2876891367 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c012640 con 0x7fbf380b2200 2024-09-17T02:47:04.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf357fa640 1 -- 172.21.15.131:0/2876891367 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf2c0127e0 con 0x7fbf380b2200 2024-09-17T02:47:04.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2876891367 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf3806d8e0 con 0x7fbf380b2200 2024-09-17T02:47:04.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf357fa640 1 -- 172.21.15.131:0/2876891367 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c012bf0 con 0x7fbf380b2200 2024-09-17T02:47:04.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.741+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2876891367 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf381f6990 con 0x7fbf380b2200 2024-09-17T02:47:04.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.745+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2876891367 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf380cd200 con 0x7fbf380b2200 2024-09-17T02:47:04.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.745+0000 7fbf357fa640 1 -- 172.21.15.131:0/2876891367 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbf2c00d030 con 0x7fbf380b2200 2024-09-17T02:47:04.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.745+0000 7fbf357fa640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf24073230 0x7fbf24075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:04.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.745+0000 7fbf357fa640 1 -- 172.21.15.131:0/2876891367 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7fbf2c0977e0 con 0x7fbf380b2200 2024-09-17T02:47:04.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.745+0000 7fbf3cadc640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf24073230 0x7fbf24075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:04.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.745+0000 7fbf3cadc640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf24073230 0x7fbf24075740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbf28003ea0 tx=0x7fbf28006bb0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:04.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.749+0000 7fbf357fa640 1 -- 172.21.15.131:0/2876891367 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbf2c029080 con 0x7fbf380b2200 2024-09-17T02:47:04.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.937+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2876891367 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf3806fec0 con 0x7fbf380b2200 2024-09-17T02:47:04.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.937+0000 7fbf357fa640 1 -- 172.21.15.131:0/2876891367 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5230 (secure 0 0 0) 0x7fbf380b2610 con 0x7fbf380b2200 2024-09-17T02:47:04.935 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:04.935 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":54,"btime":"2024-09-17T02:46:32:884315+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:46:32.834031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":16,"addr":"172.21.15.131:6835/962976667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":962976667},{"type":"v1","addr":"172.21.15.131:6835","nonce":962976667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6609,"qdb_cluster":[6609]},"id":5}]} 2024-09-17T02:47:04.936 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 54 2024-09-17T02:47:04.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf24073230 msgr2=0x7fbf24075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:04.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf24073230 0x7fbf24075740 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbf28003ea0 tx=0x7fbf28006bb0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2876891367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf380b2200 msgr2=0x7fbf381f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:04.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf380b2200 0x7fbf381f5830 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c0015a0 tx=0x7fbf2c00e6b0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2876891367 shutdown_connections 2024-09-17T02:47:04.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf24073230 0x7fbf24075740 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf381f5d70 0x7fbf381f6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf380b2200 0x7fbf381f5830 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 --2- 172.21.15.131:0/2876891367 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf20005340 0x7fbf380b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:04.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2876891367 >> 172.21.15.131:0/2876891367 conn(0x7fbf380cda20 msgr2=0x7fbf380b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:04.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2876891367 shutdown_connections 2024-09-17T02:47:04.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:04.941+0000 7fbf3ed40640 1 -- 172.21.15.131:0/2876891367 wait complete. 2024-09-17T02:47:04.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:47:05.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.109+0000 7f18fc3b0640 1 -- 172.21.15.131:0/1784798729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40ce470 msgr2=0x7f18f40ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.109+0000 7f18fc3b0640 1 --2- 172.21.15.131:0/1784798729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40ce470 0x7f18f40ce880 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009bc0 tx=0x7f18ec01cb10 comp rx=0 tx=0).stop 2024-09-17T02:47:05.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.109+0000 7f18fc3b0640 1 -- 172.21.15.131:0/1784798729 shutdown_connections 2024-09-17T02:47:05.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.109+0000 7f18fc3b0640 1 --2- 172.21.15.131:0/1784798729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40ce470 0x7f18f40ce880 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.109+0000 7f18fc3b0640 1 --2- 172.21.15.131:0/1784798729 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18f40cac80 0x7f18f40cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.109+0000 7f18fc3b0640 1 -- 172.21.15.131:0/1784798729 >> 172.21.15.131:0/1784798729 conn(0x7f18f406eea0 msgr2=0x7f18f406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.109+0000 7f18fc3b0640 1 -- 172.21.15.131:0/1784798729 shutdown_connections 2024-09-17T02:47:05.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.109+0000 7f18fc3b0640 1 -- 172.21.15.131:0/1784798729 wait complete. 2024-09-17T02:47:05.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.109+0000 7f18fc3b0640 1 Processor -- start 2024-09-17T02:47:05.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fc3b0640 1 -- start start 2024-09-17T02:47:05.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fc3b0640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18f406dea0 0x7f18f40c9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fc3b0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18f40cac80 0x7f18f40c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fc3b0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40ce470 0x7f18f4148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fc3b0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f18f416c060 con 0x7f18f40ce470 2024-09-17T02:47:05.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fc3b0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18f416bee0 con 0x7f18f40cac80 2024-09-17T02:47:05.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fc3b0640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f18f416c1e0 con 0x7f18f406dea0 2024-09-17T02:47:05.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fa94d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40ce470 0x7f18f4148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fa94d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40ce470 0x7f18f4148d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39352/0 (socket says 172.21.15.131:39352) 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fa94d640 1 -- 172.21.15.131:0/2846792936 learned_addr learned my addr 172.21.15.131:0/2846792936 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fa14c640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18f406dea0 0x7f18f40c9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18f994b640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18f40cac80 0x7f18f40c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fa94d640 1 -- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18f406dea0 msgr2=0x7f18f40c9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fa94d640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18f406dea0 0x7f18f40c9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fa94d640 1 -- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18f40cac80 msgr2=0x7f18f40c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fa94d640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18f40cac80 0x7f18f40c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fa94d640 1 -- 172.21.15.131:0/2846792936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f414a810 con 0x7f18f40ce470 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fa94d640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40ce470 0x7f18f4148d90 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f18ec01c580 tx=0x7f18ec009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18eaffd640 1 -- 172.21.15.131:0/2846792936 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec0b2040 con 0x7f18f40ce470 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18eaffd640 1 -- 172.21.15.131:0/2846792936 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18ec0bb070 con 0x7f18f40ce470 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fc3b0640 1 -- 172.21.15.131:0/2846792936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f40c7070 con 0x7f18f40ce470 2024-09-17T02:47:05.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18eaffd640 1 -- 172.21.15.131:0/2846792936 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ec004590 con 0x7f18f40ce470 2024-09-17T02:47:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fc3b0640 1 -- 172.21.15.131:0/2846792936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f40c7630 con 0x7f18f40ce470 2024-09-17T02:47:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.113+0000 7f18fc3b0640 1 -- 172.21.15.131:0/2846792936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f40cb090 con 0x7f18f40ce470 2024-09-17T02:47:05.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.117+0000 7f18eaffd640 1 -- 172.21.15.131:0/2846792936 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f18ec0ad440 con 0x7f18f40ce470 2024-09-17T02:47:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.117+0000 7f18eaffd640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18d8073200 0x7f18d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.117+0000 7f18eaffd640 1 -- 172.21.15.131:0/2846792936 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f18ec12f9e0 con 0x7f18f40ce470 2024-09-17T02:47:05.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.117+0000 7f18fa14c640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18d8073200 0x7f18d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.117+0000 7f18fa14c640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18d8073200 0x7f18d8075710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f18e4000f70 tx=0x7f18e4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:05.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.121+0000 7f18eaffd640 1 -- 172.21.15.131:0/2846792936 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f18ec016790 con 0x7f18f40ce470 2024-09-17T02:47:05.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.309+0000 7f18fc3b0640 1 -- 172.21.15.131:0/2846792936 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18f4070c70 con 0x7f18f40ce470 2024-09-17T02:47:05.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.309+0000 7f18eaffd640 1 -- 172.21.15.131:0/2846792936 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5230 (secure 0 0 0) 0x7f18ec0fa900 con 0x7f18f40ce470 2024-09-17T02:47:05.308 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:05.308 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":54,"btime":"2024-09-17T02:46:32:884315+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:46:32.834031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":16,"addr":"172.21.15.131:6835/962976667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":962976667},{"type":"v1","addr":"172.21.15.131:6835","nonce":962976667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6609,"qdb_cluster":[6609]},"id":5}]} 2024-09-17T02:47:05.309 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 54 2024-09-17T02:47:05.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 -- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18d8073200 msgr2=0x7f18d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18d8073200 0x7f18d8075710 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f18e4000f70 tx=0x7f18e4009040 comp rx=0 tx=0).stop 2024-09-17T02:47:05.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 -- 172.21.15.131:0/2846792936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40ce470 msgr2=0x7f18f4148d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40ce470 0x7f18f4148d90 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f18ec01c580 tx=0x7f18ec009200 comp rx=0 tx=0).stop 2024-09-17T02:47:05.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 -- 172.21.15.131:0/2846792936 shutdown_connections 2024-09-17T02:47:05.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f18d8073200 0x7f18d8075710 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f18f40ce470 0x7f18f4148d90 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f18f40cac80 0x7f18f40c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 --2- 172.21.15.131:0/2846792936 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f18f406dea0 0x7f18f40c9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 -- 172.21.15.131:0/2846792936 >> 172.21.15.131:0/2846792936 conn(0x7f18f406eea0 msgr2=0x7f18f40c6560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:05.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 -- 172.21.15.131:0/2846792936 shutdown_connections 2024-09-17T02:47:05.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.313+0000 7f18fc3b0640 1 -- 172.21.15.131:0/2846792936 wait complete. 2024-09-17T02:47:05.330 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:47:05.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.489+0000 7f17ed77e640 1 -- 172.21.15.131:0/1816797585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d4003a40 msgr2=0x7f17d4003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.489+0000 7f17ed77e640 1 --2- 172.21.15.131:0/1816797585 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17d4003a40 0x7f17d4003eb0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f17e00016f0 tx=0x7f17e001c7b0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.489+0000 7f17e77fe640 1 -- 172.21.15.131:0/1816797585 reap_dead start 2024-09-17T02:47:05.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.489+0000 7f17ed77e640 1 -- 172.21.15.131:0/1816797585 shutdown_connections 2024-09-17T02:47:05.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.489+0000 7f17ed77e640 1 -- 172.21.15.131:0/1816797585 >> 172.21.15.131:0/1816797585 conn(0x7f17e806eea0 msgr2=0x7f17e806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:05.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.489+0000 7f17ed77e640 1 -- 172.21.15.131:0/1816797585 shutdown_connections 2024-09-17T02:47:05.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.489+0000 7f17ed77e640 1 -- 172.21.15.131:0/1816797585 wait complete. 2024-09-17T02:47:05.487 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.489+0000 7f17ed77e640 1 Processor -- start 2024-09-17T02:47:05.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.489+0000 7f17ed77e640 1 -- start start 2024-09-17T02:47:05.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17ed77e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17d4003a40 0x7f17e80ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17ed77e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e814f0d0 0x7f17e80ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17ed77e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17e80c62b0 0x7f17e80c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17ed77e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17e81666e0 con 0x7f17e814f0d0 2024-09-17T02:47:05.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17ed77e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17e8166560 con 0x7f17e80c62b0 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17ed77e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f17e8166860 con 0x7f17d4003a40 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e67fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e814f0d0 0x7f17e80ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e67fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e814f0d0 0x7f17e80ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39374/0 (socket says 172.21.15.131:39374) 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e67fc640 1 -- 172.21.15.131:0/2144495983 learned_addr learned my addr 172.21.15.131:0/2144495983 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e6ffd640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17d4003a40 0x7f17e80ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e77fe640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17e80c62b0 0x7f17e80c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e67fc640 1 -- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17d4003a40 msgr2=0x7f17e80ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e67fc640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17d4003a40 0x7f17e80ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e67fc640 1 -- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17e80c62b0 msgr2=0x7f17e80c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e67fc640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17e80c62b0 0x7f17e80c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e67fc640 1 -- 172.21.15.131:0/2144495983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e80c68a0 con 0x7f17e814f0d0 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e67fc640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e814f0d0 0x7f17e80ceaf0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f17d0002090 tx=0x7f17d000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17df7fe640 1 -- 172.21.15.131:0/2144495983 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d0015070 con 0x7f17e814f0d0 2024-09-17T02:47:05.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17df7fe640 1 -- 172.21.15.131:0/2144495983 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17d000cd40 con 0x7f17e814f0d0 2024-09-17T02:47:05.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17ed77e640 1 -- 172.21.15.131:0/2144495983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e81f8350 con 0x7f17e814f0d0 2024-09-17T02:47:05.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17df7fe640 1 -- 172.21.15.131:0/2144495983 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17d00116c0 con 0x7f17e814f0d0 2024-09-17T02:47:05.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17ed77e640 1 -- 172.21.15.131:0/2144495983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e80c4ba0 con 0x7f17e814f0d0 2024-09-17T02:47:05.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17ed77e640 1 -- 172.21.15.131:0/2144495983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e80b50d0 con 0x7f17e814f0d0 2024-09-17T02:47:05.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17df7fe640 1 -- 172.21.15.131:0/2144495983 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f17d0018070 con 0x7f17e814f0d0 2024-09-17T02:47:05.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17df7fe640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17c8073200 0x7f17c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17df7fe640 1 -- 172.21.15.131:0/2144495983 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f17d0094c40 con 0x7f17e814f0d0 2024-09-17T02:47:05.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.493+0000 7f17e6ffd640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17c8073200 0x7f17c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.497+0000 7f17e6ffd640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17c8073200 0x7f17c8075710 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f17e00016c0 tx=0x7f17e00195a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:05.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.501+0000 7f17df7fe640 1 -- 172.21.15.131:0/2144495983 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f17d005fa80 con 0x7f17e814f0d0 2024-09-17T02:47:05.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.661+0000 7f17ed77e640 1 -- 172.21.15.131:0/2144495983 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f17e8073430 con 0x7f17e814f0d0 2024-09-17T02:47:05.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.661+0000 7f17df7fe640 1 -- 172.21.15.131:0/2144495983 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+17122 (secure 0 0 0) 0x7f17d005f380 con 0x7f17e814f0d0 2024-09-17T02:47:05.660 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:05.660 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":80,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:45:41.796170+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:45:17.161688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-17T02:45:18.453383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6835/655402879":"2024-09-18T02:45:41.796095+0000","172.21.15.131:6833/655402879":"2024-09-18T02:45:41.796095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:47:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 -- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17c8073200 msgr2=0x7f17c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17c8073200 0x7f17c8075710 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f17e00016c0 tx=0x7f17e00195a0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 -- 172.21.15.131:0/2144495983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e814f0d0 msgr2=0x7f17e80ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e814f0d0 0x7f17e80ceaf0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f17d0002090 tx=0x7f17d000ed90 comp rx=0 tx=0).stop 2024-09-17T02:47:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 -- 172.21.15.131:0/2144495983 shutdown_connections 2024-09-17T02:47:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f17c8073200 0x7f17c8075710 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17e80c62b0 0x7f17e80c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17e814f0d0 0x7f17e80ceaf0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 --2- 172.21.15.131:0/2144495983 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17d4003a40 0x7f17e80ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 -- 172.21.15.131:0/2144495983 >> 172.21.15.131:0/2144495983 conn(0x7f17e806eea0 msgr2=0x7f17e80b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 -- 172.21.15.131:0/2144495983 shutdown_connections 2024-09-17T02:47:05.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.665+0000 7f17ed77e640 1 -- 172.21.15.131:0/2144495983 wait complete. 2024-09-17T02:47:05.682 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T02:47:05.682 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:47:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.821+0000 7f5a24c0d640 1 -- 172.21.15.131:0/2714045038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a200cd470 msgr2=0x7f5a2006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.821+0000 7f5a24c0d640 1 --2- 172.21.15.131:0/2714045038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a200cd470 0x7f5a2006efb0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5a200b0b40 tx=0x7f5a1801cb10 comp rx=0 tx=0).stop 2024-09-17T02:47:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.821+0000 7f5a24c0d640 1 -- 172.21.15.131:0/2714045038 shutdown_connections 2024-09-17T02:47:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.821+0000 7f5a24c0d640 1 --2- 172.21.15.131:0/2714045038 >> v2:172.21.15.189:3300/0 conn(0x7f5a2015bf80 0x7f5a20151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.821+0000 7f5a24c0d640 1 --2- 172.21.15.131:0/2714045038 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a200cd470 0x7f5a2006efb0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.821+0000 7f5a24c0d640 1 -- 172.21.15.131:0/2714045038 >> 172.21.15.131:0/2714045038 conn(0x7f5a20073d10 msgr2=0x7f5a20074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:05.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.821+0000 7f5a24c0d640 1 -- 172.21.15.131:0/2714045038 shutdown_connections 2024-09-17T02:47:05.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.821+0000 7f5a24c0d640 1 -- 172.21.15.131:0/2714045038 wait complete. 2024-09-17T02:47:05.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.821+0000 7f5a24c0d640 1 Processor -- start 2024-09-17T02:47:05.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.821+0000 7f5a24c0d640 1 -- start start 2024-09-17T02:47:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a24c0d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a2006f4f0 0x7f5a201f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a24c0d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a200cd470 0x7f5a2015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a24c0d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a2015bf80 0x7f5a20148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a24c0d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20169000 con 0x7f5a200cd470 2024-09-17T02:47:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a24c0d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a200cc330 con 0x7f5a2015bf80 2024-09-17T02:47:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a24c0d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5a20169180 con 0x7f5a2006f4f0 2024-09-17T02:47:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1dd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a200cd470 0x7f5a2015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1dd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a200cd470 0x7f5a2015ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39380/0 (socket says 172.21.15.131:39380) 2024-09-17T02:47:05.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1dd9b640 1 -- 172.21.15.131:0/1838652448 learned_addr learned my addr 172.21.15.131:0/1838652448 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1e59c640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a2006f4f0 0x7f5a201f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1ed9d640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a2015bf80 0x7f5a20148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1dd9b640 1 -- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a2006f4f0 msgr2=0x7f5a201f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1dd9b640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a2006f4f0 0x7f5a201f97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1dd9b640 1 -- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a2015bf80 msgr2=0x7f5a20148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1dd9b640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a2015bf80 0x7f5a20148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1dd9b640 1 -- 172.21.15.131:0/1838652448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a200c7450 con 0x7f5a200cd470 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1ed9d640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a2015bf80 0x7f5a20148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a1dd9b640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a200cd470 0x7f5a2015ac30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400b3e0 tx=0x7f5a1400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a0effd640 1 -- 172.21.15.131:0/1838652448 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14015020 con 0x7f5a200cd470 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a0effd640 1 -- 172.21.15.131:0/1838652448 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a14011940 con 0x7f5a200cd470 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a0effd640 1 -- 172.21.15.131:0/1838652448 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14011d90 con 0x7f5a200cd470 2024-09-17T02:47:05.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a24c0d640 1 -- 172.21.15.131:0/1838652448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a200c7730 con 0x7f5a200cd470 2024-09-17T02:47:05.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a24c0d640 1 -- 172.21.15.131:0/1838652448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a2014bad0 con 0x7f5a200cd470 2024-09-17T02:47:05.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.825+0000 7f5a24c0d640 1 -- 172.21.15.131:0/1838652448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a2006efb0 con 0x7f5a200cd470 2024-09-17T02:47:05.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.829+0000 7f5a0effd640 1 -- 172.21.15.131:0/1838652448 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a14018020 con 0x7f5a200cd470 2024-09-17T02:47:05.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.829+0000 7f5a0effd640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a00073200 0x7f5a00075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:05.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.829+0000 7f5a0effd640 1 -- 172.21.15.131:0/1838652448 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f5a14094260 con 0x7f5a200cd470 2024-09-17T02:47:05.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.829+0000 7f5a1e59c640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a00073200 0x7f5a00075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:05.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.829+0000 7f5a1e59c640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a00073200 0x7f5a00075710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5a180af000 tx=0x7f5a1801b350 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:05.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:05.833+0000 7f5a0effd640 1 -- 172.21.15.131:0/1838652448 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a1405f080 con 0x7f5a200cd470 2024-09-17T02:47:06.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.021+0000 7f5a24c0d640 1 -- 172.21.15.131:0/1838652448 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a200d0d80 con 0x7f5a200cd470 2024-09-17T02:47:06.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.021+0000 7f5a0effd640 1 -- 172.21.15.131:0/1838652448 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5230 (secure 0 0 0) 0x7f5a1405e980 con 0x7f5a200cd470 2024-09-17T02:47:06.021 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:06.021 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":54,"btime":"2024-09-17T02:46:32:884315+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:46:32.834031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":16,"addr":"172.21.15.131:6835/962976667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":962976667},{"type":"v1","addr":"172.21.15.131:6835","nonce":962976667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6609,"qdb_cluster":[6609]},"id":5}]} 2024-09-17T02:47:06.021 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 54 2024-09-17T02:47:06.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 -- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a00073200 msgr2=0x7f5a00075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a00073200 0x7f5a00075710 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f5a180af000 tx=0x7f5a1801b350 comp rx=0 tx=0).stop 2024-09-17T02:47:06.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 -- 172.21.15.131:0/1838652448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a200cd470 msgr2=0x7f5a2015ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a200cd470 0x7f5a2015ac30 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5a1400b3e0 tx=0x7f5a1400b8a0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 -- 172.21.15.131:0/1838652448 shutdown_connections 2024-09-17T02:47:06.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a00073200 0x7f5a00075710 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a2015bf80 0x7f5a20148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a200cd470 0x7f5a2015ac30 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 --2- 172.21.15.131:0/1838652448 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a2006f4f0 0x7f5a201f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 -- 172.21.15.131:0/1838652448 >> 172.21.15.131:0/1838652448 conn(0x7f5a20073d10 msgr2=0x7f5a2014c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:06.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 -- 172.21.15.131:0/1838652448 shutdown_connections 2024-09-17T02:47:06.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.025+0000 7f5a24c0d640 1 -- 172.21.15.131:0/1838652448 wait complete. 2024-09-17T02:47:06.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:47:06.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.185+0000 7faf9bca4640 1 -- 172.21.15.131:0/4221708898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9406e3e0 msgr2=0x7faf9415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.185+0000 7faf9bca4640 1 --2- 172.21.15.131:0/4221708898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9406e3e0 0x7faf9415bea0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7faf90009840 tx=0x7faf9001c820 comp rx=0 tx=0).stop 2024-09-17T02:47:06.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.185+0000 7faf9bca4640 1 -- 172.21.15.131:0/4221708898 shutdown_connections 2024-09-17T02:47:06.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.185+0000 7faf9bca4640 1 --2- 172.21.15.131:0/4221708898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9406e3e0 0x7faf9415bea0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.185+0000 7faf9bca4640 1 --2- 172.21.15.131:0/4221708898 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf940ced00 0x7faf9406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.185+0000 7faf9bca4640 1 -- 172.21.15.131:0/4221708898 >> 172.21.15.131:0/4221708898 conn(0x7faf9406eea0 msgr2=0x7faf9406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:06.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.185+0000 7faf9bca4640 1 -- 172.21.15.131:0/4221708898 shutdown_connections 2024-09-17T02:47:06.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.185+0000 7faf9bca4640 1 -- 172.21.15.131:0/4221708898 wait complete. 2024-09-17T02:47:06.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.185+0000 7faf9bca4640 1 Processor -- start 2024-09-17T02:47:06.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf9bca4640 1 -- start start 2024-09-17T02:47:06.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf9bca4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9406e3e0 0x7faf940ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf9bca4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf940ced00 0x7faf94150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf9bca4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf9415c3e0 0x7faf940b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf9bca4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faf94168b60 con 0x7faf9406e3e0 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf9bca4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf941689e0 con 0x7faf940ced00 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf9bca4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7faf94168ce0 con 0x7faf9415c3e0 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf99a40640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9406e3e0 0x7faf940ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf99a40640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9406e3e0 0x7faf940ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39402/0 (socket says 172.21.15.131:39402) 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf99a40640 1 -- 172.21.15.131:0/4048852109 learned_addr learned my addr 172.21.15.131:0/4048852109 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf99a40640 1 -- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf9415c3e0 msgr2=0x7faf940b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf99a40640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf9415c3e0 0x7faf940b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf99a40640 1 -- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf940ced00 msgr2=0x7faf94150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf9923f640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf940ced00 0x7faf94150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:06.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf99a40640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf940ced00 0x7faf94150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf99a40640 1 -- 172.21.15.131:0/4048852109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf940b5f70 con 0x7faf9406e3e0 2024-09-17T02:47:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf99a40640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9406e3e0 0x7faf940ae9f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7faf8400d720 tx=0x7faf8400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf8a7fc640 1 -- 172.21.15.131:0/4048852109 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84017070 con 0x7faf9406e3e0 2024-09-17T02:47:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf8a7fc640 1 -- 172.21.15.131:0/4048852109 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf84009cc0 con 0x7faf9406e3e0 2024-09-17T02:47:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf8a7fc640 1 -- 172.21.15.131:0/4048852109 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84005280 con 0x7faf9406e3e0 2024-09-17T02:47:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf9bca4640 1 -- 172.21.15.131:0/4048852109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf94151520 con 0x7faf9406e3e0 2024-09-17T02:47:06.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf9bca4640 1 -- 172.21.15.131:0/4048852109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf941fc0d0 con 0x7faf9406e3e0 2024-09-17T02:47:06.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.189+0000 7faf9bca4640 1 -- 172.21.15.131:0/4048852109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf941fbec0 con 0x7faf9406e3e0 2024-09-17T02:47:06.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.193+0000 7faf8a7fc640 1 -- 172.21.15.131:0/4048852109 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faf84005420 con 0x7faf9406e3e0 2024-09-17T02:47:06.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.193+0000 7faf8a7fc640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf78073200 0x7faf78075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.193+0000 7faf8a7fc640 1 -- 172.21.15.131:0/4048852109 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7faf84094590 con 0x7faf9406e3e0 2024-09-17T02:47:06.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.193+0000 7faf9923f640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf78073200 0x7faf78075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:06.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.193+0000 7faf9923f640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf78073200 0x7faf78075710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faf900af000 tx=0x7faf9001b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:06.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.197+0000 7faf8a7fc640 1 -- 172.21.15.131:0/4048852109 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf8405f3c0 con 0x7faf9406e3e0 2024-09-17T02:47:06.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.385+0000 7faf9bca4640 1 -- 172.21.15.131:0/4048852109 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf9415bea0 con 0x7faf9406e3e0 2024-09-17T02:47:06.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.385+0000 7faf8a7fc640 1 -- 172.21.15.131:0/4048852109 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5230 (secure 0 0 0) 0x7faf8405ecc0 con 0x7faf9406e3e0 2024-09-17T02:47:06.385 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:06.385 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":54,"btime":"2024-09-17T02:46:32:884315+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:45:19.460749+0000","modified":"2024-09-17T02:46:32.834031+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6609},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6609":{"gid":6609,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":16,"addr":"172.21.15.131:6835/962976667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":962976667},{"type":"v1","addr":"172.21.15.131:6835","nonce":962976667}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6609,"qdb_cluster":[6609]},"id":5}]} 2024-09-17T02:47:06.385 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 54 2024-09-17T02:47:06.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 -- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf78073200 msgr2=0x7faf78075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf78073200 0x7faf78075710 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faf900af000 tx=0x7faf9001b060 comp rx=0 tx=0).stop 2024-09-17T02:47:06.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 -- 172.21.15.131:0/4048852109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9406e3e0 msgr2=0x7faf940ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9406e3e0 0x7faf940ae9f0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7faf8400d720 tx=0x7faf8400dbe0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 -- 172.21.15.131:0/4048852109 shutdown_connections 2024-09-17T02:47:06.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf78073200 0x7faf78075710 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf9415c3e0 0x7faf940b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf940ced00 0x7faf94150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 --2- 172.21.15.131:0/4048852109 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf9406e3e0 0x7faf940ae9f0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 -- 172.21.15.131:0/4048852109 >> 172.21.15.131:0/4048852109 conn(0x7faf9406eea0 msgr2=0x7faf940b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:06.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 -- 172.21.15.131:0/4048852109 shutdown_connections 2024-09-17T02:47:06.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.389+0000 7faf9bca4640 1 -- 172.21.15.131:0/4048852109 wait complete. 2024-09-17T02:47:06.406 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:47:06.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 -- 172.21.15.131:0/1110333852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f989c15b2f0 msgr2=0x7f989c0cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 --2- 172.21.15.131:0/1110333852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f989c15b2f0 0x7f989c0cae70 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f989c069b60 tx=0x7f988401c890 comp rx=0 tx=0).stop 2024-09-17T02:47:06.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 -- 172.21.15.131:0/1110333852 shutdown_connections 2024-09-17T02:47:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 --2- 172.21.15.131:0/1110333852 >> v2:172.21.15.189:3300/0 conn(0x7f989c070c90 0x7f989c0710a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 --2- 172.21.15.131:0/1110333852 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f989c15b2f0 0x7f989c0cae70 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 -- 172.21.15.131:0/1110333852 >> 172.21.15.131:0/1110333852 conn(0x7f989c0cd6a0 msgr2=0x7f989c0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 -- 172.21.15.131:0/1110333852 shutdown_connections 2024-09-17T02:47:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 -- 172.21.15.131:0/1110333852 wait complete. 2024-09-17T02:47:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 Processor -- start 2024-09-17T02:47:06.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 -- start start 2024-09-17T02:47:06.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f989c070c90 0x7f989c156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.549+0000 7f98a1a08640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f989c0cf4e0 0x7f989c156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f98a1a08640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f989c15b2f0 0x7f989c1fa180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f98a1a08640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f989c168b00 con 0x7f989c15b2f0 2024-09-17T02:47:06.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f98a1a08640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f989c071be0 con 0x7f989c070c90 2024-09-17T02:47:06.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f98a1a08640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f989c168c80 con 0x7f989c0cf4e0 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f989c15b2f0 0x7f989c1fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f989c15b2f0 0x7f989c1fa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39418/0 (socket says 172.21.15.131:39418) 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989b7fe640 1 -- 172.21.15.131:0/1586236529 learned_addr learned my addr 172.21.15.131:0/1586236529 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989a7fc640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f989c0cf4e0 0x7f989c156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989affd640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f989c070c90 0x7f989c156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989b7fe640 1 -- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f989c0cf4e0 msgr2=0x7f989c156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989b7fe640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f989c0cf4e0 0x7f989c156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989b7fe640 1 -- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f989c070c90 msgr2=0x7f989c156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989b7fe640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f989c070c90 0x7f989c156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989b7fe640 1 -- 172.21.15.131:0/1586236529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989c1517e0 con 0x7f989c15b2f0 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989b7fe640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f989c15b2f0 0x7f989c1fa180 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f988c00c770 tx=0x7f988c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f9893fff640 1 -- 172.21.15.131:0/1586236529 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c007bb0 con 0x7f989c15b2f0 2024-09-17T02:47:06.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f9893fff640 1 -- 172.21.15.131:0/1586236529 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f988c007e40 con 0x7f989c15b2f0 2024-09-17T02:47:06.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f9893fff640 1 -- 172.21.15.131:0/1586236529 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c005280 con 0x7f989c15b2f0 2024-09-17T02:47:06.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f98a1a08640 1 -- 172.21.15.131:0/1586236529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989c15cf10 con 0x7f989c15b2f0 2024-09-17T02:47:06.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f98a1a08640 1 -- 172.21.15.131:0/1586236529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f989c15d4d0 con 0x7f989c15b2f0 2024-09-17T02:47:06.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f98a1a08640 1 -- 172.21.15.131:0/1586236529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f989c0710a0 con 0x7f989c15b2f0 2024-09-17T02:47:06.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f9893fff640 1 -- 172.21.15.131:0/1586236529 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f988c005420 con 0x7f989c15b2f0 2024-09-17T02:47:06.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f9893fff640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f987c073200 0x7f987c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f9893fff640 1 -- 172.21.15.131:0/1586236529 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f988c094810 con 0x7f989c15b2f0 2024-09-17T02:47:06.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.553+0000 7f989affd640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f987c073200 0x7f987c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:06.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.557+0000 7f989affd640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f987c073200 0x7f987c075710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f98840af000 tx=0x7f988401b0d0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:06.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.557+0000 7f9893fff640 1 -- 172.21.15.131:0/1586236529 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f988c014030 con 0x7f989c15b2f0 2024-09-17T02:47:06.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.725+0000 7f98a1a08640 1 -- 172.21.15.131:0/1586236529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f989c1fadc0 con 0x7f989c15b2f0 2024-09-17T02:47:06.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f9893fff640 1 -- 172.21.15.131:0/1586236529 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+17122 (secure 0 0 0) 0x7f988c05f650 con 0x7f989c15b2f0 2024-09-17T02:47:06.725 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:06.726 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":80,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:45:41.796170+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:45:17.161688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-17T02:45:18.453383+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6835/655402879":"2024-09-18T02:45:41.796095+0000","172.21.15.131:6833/655402879":"2024-09-18T02:45:41.796095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:47:06.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 -- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f987c073200 msgr2=0x7f987c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f987c073200 0x7f987c075710 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f98840af000 tx=0x7f988401b0d0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 -- 172.21.15.131:0/1586236529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f989c15b2f0 msgr2=0x7f989c1fa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f989c15b2f0 0x7f989c1fa180 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f988c00c770 tx=0x7f988c00cc30 comp rx=0 tx=0).stop 2024-09-17T02:47:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 -- 172.21.15.131:0/1586236529 shutdown_connections 2024-09-17T02:47:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f987c073200 0x7f987c075710 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f989c15b2f0 0x7f989c1fa180 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f989c0cf4e0 0x7f989c156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 --2- 172.21.15.131:0/1586236529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f989c070c90 0x7f989c156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 -- 172.21.15.131:0/1586236529 >> 172.21.15.131:0/1586236529 conn(0x7f989c0cd6a0 msgr2=0x7f989c155160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 -- 172.21.15.131:0/1586236529 shutdown_connections 2024-09-17T02:47:06.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.729+0000 7f98a1a08640 1 -- 172.21.15.131:0/1586236529 wait complete. 2024-09-17T02:47:06.746 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:47:06.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.905+0000 7f4b3dd82640 1 -- 172.21.15.131:0/1298143227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b3806e5b0 msgr2=0x7f4b3815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.905+0000 7f4b3dd82640 1 --2- 172.21.15.131:0/1298143227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b3806e5b0 0x7f4b3815bea0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009900 tx=0x7f4b2c01c8e0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.905+0000 7f4b3dd82640 1 -- 172.21.15.131:0/1298143227 shutdown_connections 2024-09-17T02:47:06.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.905+0000 7f4b3dd82640 1 --2- 172.21.15.131:0/1298143227 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b3815c3e0 0x7f4b380cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.905+0000 7f4b3dd82640 1 --2- 172.21.15.131:0/1298143227 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b3806e5b0 0x7f4b3815bea0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.905+0000 7f4b3dd82640 1 -- 172.21.15.131:0/1298143227 >> 172.21.15.131:0/1298143227 conn(0x7f4b3806eea0 msgr2=0x7f4b3806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:06.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.905+0000 7f4b3dd82640 1 -- 172.21.15.131:0/1298143227 shutdown_connections 2024-09-17T02:47:06.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.905+0000 7f4b3dd82640 1 -- 172.21.15.131:0/1298143227 wait complete. 2024-09-17T02:47:06.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.905+0000 7f4b3dd82640 1 Processor -- start 2024-09-17T02:47:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.905+0000 7f4b3dd82640 1 -- start start 2024-09-17T02:47:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b3dd82640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b380ced00 0x7f4b3815e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b3dd82640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b3815c3e0 0x7f4b38154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b3dd82640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b381566c0 0x7f4b38155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b3dd82640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4b38168890 con 0x7f4b380ced00 2024-09-17T02:47:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b3dd82640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4b38168710 con 0x7f4b381566c0 2024-09-17T02:47:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b3dd82640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4b38168a10 con 0x7f4b3815c3e0 2024-09-17T02:47:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b377fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b380ced00 0x7f4b3815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b377fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b380ced00 0x7f4b3815e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39442/0 (socket says 172.21.15.131:39442) 2024-09-17T02:47:06.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b377fe640 1 -- 172.21.15.131:0/3300746442 learned_addr learned my addr 172.21.15.131:0/3300746442 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b36ffd640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b3815c3e0 0x7f4b38154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b37fff640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b381566c0 0x7f4b38155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b377fe640 1 -- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b3815c3e0 msgr2=0x7f4b38154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b377fe640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b3815c3e0 0x7f4b38154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b377fe640 1 -- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b381566c0 msgr2=0x7f4b38155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b377fe640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b381566c0 0x7f4b38155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b377fe640 1 -- 172.21.15.131:0/3300746442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b381f83e0 con 0x7f4b380ced00 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b37fff640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b381566c0 0x7f4b38155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b377fe640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b380ced00 0x7f4b3815e030 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e690 tx=0x7f4b2800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b1ffff640 1 -- 172.21.15.131:0/3300746442 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b28015070 con 0x7f4b380ced00 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b1ffff640 1 -- 172.21.15.131:0/3300746442 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b28005380 con 0x7f4b380ced00 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b3dd82640 1 -- 172.21.15.131:0/3300746442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b381f88e0 con 0x7f4b380ced00 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b1ffff640 1 -- 172.21.15.131:0/3300746442 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280018d0 con 0x7f4b380ced00 2024-09-17T02:47:06.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b3dd82640 1 -- 172.21.15.131:0/3300746442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b381f8ea0 con 0x7f4b380ced00 2024-09-17T02:47:06.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b3dd82640 1 -- 172.21.15.131:0/3300746442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b3815bea0 con 0x7f4b380ced00 2024-09-17T02:47:06.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b1ffff640 1 -- 172.21.15.131:0/3300746442 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4b28018070 con 0x7f4b380ced00 2024-09-17T02:47:06.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b1ffff640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b24073230 0x7f4b24075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:06.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b1ffff640 1 -- 172.21.15.131:0/3300746442 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6187+0+0 (secure 0 0 0) 0x7f4b28096670 con 0x7f4b380ced00 2024-09-17T02:47:06.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.909+0000 7f4b36ffd640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b24073230 0x7f4b24075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:06.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.913+0000 7f4b36ffd640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b24073230 0x7f4b24075740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0af000 tx=0x7f4b2c01b120 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:06.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:06.913+0000 7f4b1ffff640 1 -- 172.21.15.131:0/3300746442 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4b280614a0 con 0x7f4b380ced00 2024-09-17T02:47:07.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.113+0000 7f4b3dd82640 1 -- 172.21.15.131:0/3300746442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4b3814aa60 con 0x7f4b380ced00 2024-09-17T02:47:07.112 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:47:07.113+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:47:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.117+0000 7f4b1ffff640 1 -- 172.21.15.131:0/3300746442 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) ==== 161+0+0 (secure 0 0 0) 0x7f4b28060da0 con 0x7f4b380ced00 2024-09-17T02:47:07.114 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:47:07.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 -- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b24073230 msgr2=0x7f4b24075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:07.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b24073230 0x7f4b24075740 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c0af000 tx=0x7f4b2c01b120 comp rx=0 tx=0).stop 2024-09-17T02:47:07.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 -- 172.21.15.131:0/3300746442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b380ced00 msgr2=0x7f4b3815e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:07.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b380ced00 0x7f4b3815e030 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800e690 tx=0x7f4b2800eb50 comp rx=0 tx=0).stop 2024-09-17T02:47:07.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 -- 172.21.15.131:0/3300746442 shutdown_connections 2024-09-17T02:47:07.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4b24073230 0x7f4b24075740 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:07.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4b381566c0 0x7f4b38155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:07.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4b3815c3e0 0x7f4b38154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:07.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 --2- 172.21.15.131:0/3300746442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4b380ced00 0x7f4b3815e030 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:07.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 -- 172.21.15.131:0/3300746442 >> 172.21.15.131:0/3300746442 conn(0x7f4b3806eea0 msgr2=0x7f4b3806dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:07.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 -- 172.21.15.131:0/3300746442 shutdown_connections 2024-09-17T02:47:07.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.121+0000 7f4b3dd82640 1 -- 172.21.15.131:0/3300746442 wait complete. 2024-09-17T02:47:07.122 INFO:tasks.ceph.mds.c.smithi131.stderr: -4745> 2024-09-17T02:45:47.814+0000 7f5131508640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:47:07.143 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T02:47:07.186 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:47:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.293+0000 7f1722c96640 1 -- 172.21.15.131:0/969250930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1704004d50 msgr2=0x7f17040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.293+0000 7f1722c96640 1 --2- 172.21.15.131:0/969250930 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1704004d50 0x7f17040051c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f171c072ad0 tx=0x7f171401d150 comp rx=0 tx=0).stop 2024-09-17T02:47:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.293+0000 7f1721233640 1 -- 172.21.15.131:0/969250930 reap_dead start 2024-09-17T02:47:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.293+0000 7f1722c96640 1 -- 172.21.15.131:0/969250930 shutdown_connections 2024-09-17T02:47:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.293+0000 7f1722c96640 1 -- 172.21.15.131:0/969250930 >> 172.21.15.131:0/969250930 conn(0x7f171c06eea0 msgr2=0x7f171c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.293+0000 7f1722c96640 1 -- 172.21.15.131:0/969250930 shutdown_connections 2024-09-17T02:47:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.293+0000 7f1722c96640 1 -- 172.21.15.131:0/969250930 wait complete. 2024-09-17T02:47:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.293+0000 7f1722c96640 1 Processor -- start 2024-09-17T02:47:07.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.293+0000 7f1722c96640 1 -- start start 2024-09-17T02:47:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1722c96640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c153a80 0x7f171c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1722c96640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f171c0c8ff0 0x7f171c166b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1722c96640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f171c0c9400 0x7f171c1670b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1722c96640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f171c06f9c0 con 0x7f171c153a80 2024-09-17T02:47:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1722c96640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f171c06e570 con 0x7f171c0c9400 2024-09-17T02:47:07.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1722c96640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f171c06fb40 con 0x7f171c0c8ff0 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1720a32640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c153a80 0x7f171c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1720a32640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c153a80 0x7f171c159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39462/0 (socket says 172.21.15.131:39462) 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1720a32640 1 -- 172.21.15.131:0/2112414447 learned_addr learned my addr 172.21.15.131:0/2112414447 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1721233640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f171c0c9400 0x7f171c1670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1720a32640 1 -- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f171c0c8ff0 msgr2=0x7f171c166b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1720a32640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f171c0c8ff0 0x7f171c166b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1720a32640 1 -- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f171c0c9400 msgr2=0x7f171c1670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1720a32640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f171c0c9400 0x7f171c1670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1720a32640 1 -- 172.21.15.131:0/2112414447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c153e90 con 0x7f171c153a80 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1721233640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f171c0c9400 0x7f171c1670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1720a32640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c153a80 0x7f171c159c90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f1718001020 tx=0x7f1718004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:07.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f17117fa640 1 -- 172.21.15.131:0/2112414447 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718015020 con 0x7f171c153a80 2024-09-17T02:47:07.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f17117fa640 1 -- 172.21.15.131:0/2112414447 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17180109d0 con 0x7f171c153a80 2024-09-17T02:47:07.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1722c96640 1 -- 172.21.15.131:0/2112414447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171c15a3a0 con 0x7f171c153a80 2024-09-17T02:47:07.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f17117fa640 1 -- 172.21.15.131:0/2112414447 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1718010e20 con 0x7f171c153a80 2024-09-17T02:47:07.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1722c96640 1 -- 172.21.15.131:0/2112414447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171c15a960 con 0x7f171c153a80 2024-09-17T02:47:07.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f1722c96640 1 -- 172.21.15.131:0/2112414447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171c06e9d0 con 0x7f171c153a80 2024-09-17T02:47:07.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.297+0000 7f17117fa640 1 -- 172.21.15.131:0/2112414447 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1718018020 con 0x7f171c153a80 2024-09-17T02:47:07.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.301+0000 7f17117fa640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1708073200 0x7f1708075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:07.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.301+0000 7f17117fa640 1 -- 172.21.15.131:0/2112414447 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6273+0+0 (secure 0 0 0) 0x7f1718095070 con 0x7f171c153a80 2024-09-17T02:47:07.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.301+0000 7f1713fff640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1708073200 0x7f1708075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:07.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.301+0000 7f1713fff640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1708073200 0x7f1708075710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f171c0af5d0 tx=0x7f170c00d3f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:07.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.305+0000 7f17117fa640 1 -- 172.21.15.131:0/2112414447 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f171801f020 con 0x7f171c153a80 2024-09-17T02:47:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:07.493+0000 7f1722c96640 1 -- 172.21.15.131:0/2112414447 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f171c06f570 con 0x7f171c153a80 2024-09-17T02:47:08.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f17117fa640 1 -- 172.21.15.131:0/2112414447 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) ==== 104+0+0 (secure 0 0 0) 0x7f171c06e7f0 con 0x7f171c153a80 2024-09-17T02:47:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 -- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1708073200 msgr2=0x7f1708075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1708073200 0x7f1708075710 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f171c0af5d0 tx=0x7f170c00d3f0 comp rx=0 tx=0).stop 2024-09-17T02:47:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 -- 172.21.15.131:0/2112414447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c153a80 msgr2=0x7f171c159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c153a80 0x7f171c159c90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f1718001020 tx=0x7f1718004ac0 comp rx=0 tx=0).stop 2024-09-17T02:47:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 -- 172.21.15.131:0/2112414447 shutdown_connections 2024-09-17T02:47:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1708073200 0x7f1708075710 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f171c0c9400 0x7f171c1670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f171c0c8ff0 0x7f171c166b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:08.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 --2- 172.21.15.131:0/2112414447 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c153a80 0x7f171c159c90 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:08.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 -- 172.21.15.131:0/2112414447 >> 172.21.15.131:0/2112414447 conn(0x7f171c06eea0 msgr2=0x7f171c15ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:08.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 -- 172.21.15.131:0/2112414447 shutdown_connections 2024-09-17T02:47:08.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.129+0000 7f1722c96640 1 -- 172.21.15.131:0/2112414447 wait complete. 2024-09-17T02:47:08.154 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T02:47:08.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.357+0000 7f825112b640 1 -- 172.21.15.131:0/4080411307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8234004b20 msgr2=0x7f8234004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:08.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.357+0000 7f825112b640 1 --2- 172.21.15.131:0/4080411307 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8234004b20 0x7f8234004f90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f823c00c2e0 tx=0x7f823c01cf60 comp rx=0 tx=0).stop 2024-09-17T02:47:08.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.357+0000 7f824b59e640 1 -- 172.21.15.131:0/4080411307 reap_dead start 2024-09-17T02:47:08.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.357+0000 7f825112b640 1 -- 172.21.15.131:0/4080411307 shutdown_connections 2024-09-17T02:47:08.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.357+0000 7f825112b640 1 -- 172.21.15.131:0/4080411307 >> 172.21.15.131:0/4080411307 conn(0x7f824c0734f0 msgr2=0x7f824c073900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:08.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.357+0000 7f825112b640 1 -- 172.21.15.131:0/4080411307 shutdown_connections 2024-09-17T02:47:08.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.357+0000 7f825112b640 1 -- 172.21.15.131:0/4080411307 wait complete. 2024-09-17T02:47:08.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f825112b640 1 Processor -- start 2024-09-17T02:47:08.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f825112b640 1 -- start start 2024-09-17T02:47:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f825112b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234004b20 0x7f824c149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:08.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f825112b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f824c14db70 0x7f824c14e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f825112b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f824c15cf20 0x7f824c15d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f825112b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f824c0cc8c0 con 0x7f824c14db70 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f825112b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f824c0cc740 con 0x7f8234004b20 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f825112b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f824c0cca40 con 0x7f824c15cf20 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f824a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f824c14db70 0x7f824c14e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f824a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f824c14db70 0x7f824c14e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39478/0 (socket says 172.21.15.131:39478) 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f824a59c640 1 -- 172.21.15.131:0/2104344586 learned_addr learned my addr 172.21.15.131:0/2104344586 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f824ad9d640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234004b20 0x7f824c149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f824b59e640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f824c15cf20 0x7f824c15d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f824a59c640 1 -- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f824c15cf20 msgr2=0x7f824c15d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f824a59c640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f824c15cf20 0x7f824c15d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f824a59c640 1 -- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234004b20 msgr2=0x7f824c149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:08.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f824a59c640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234004b20 0x7f824c149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:08.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f824a59c640 1 -- 172.21.15.131:0/2104344586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f824c15d870 con 0x7f824c14db70 2024-09-17T02:47:08.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f824a59c640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f824c14db70 0x7f824c14e2c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f824000e790 tx=0x7f824000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:08.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f82337fe640 1 -- 172.21.15.131:0/2104344586 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824000eea0 con 0x7f824c14db70 2024-09-17T02:47:08.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f82337fe640 1 -- 172.21.15.131:0/2104344586 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f824000cd50 con 0x7f824c14db70 2024-09-17T02:47:08.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f825112b640 1 -- 172.21.15.131:0/2104344586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f824c14d0d0 con 0x7f824c14db70 2024-09-17T02:47:08.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f82337fe640 1 -- 172.21.15.131:0/2104344586 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82400116b0 con 0x7f824c14db70 2024-09-17T02:47:08.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f825112b640 1 -- 172.21.15.131:0/2104344586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f824c14ed10 con 0x7f824c14db70 2024-09-17T02:47:08.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.361+0000 7f825112b640 1 -- 172.21.15.131:0/2104344586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f824c14ea80 con 0x7f824c14db70 2024-09-17T02:47:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.365+0000 7f82337fe640 1 -- 172.21.15.131:0/2104344586 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8240011850 con 0x7f824c14db70 2024-09-17T02:47:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.365+0000 7f82337fe640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8238073200 0x7f8238075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.365+0000 7f82337fe640 1 -- 172.21.15.131:0/2104344586 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6273+0+0 (secure 0 0 0) 0x7f8240094ff0 con 0x7f824c14db70 2024-09-17T02:47:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.365+0000 7f824ad9d640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8238073200 0x7f8238075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:08.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.365+0000 7f824ad9d640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8238073200 0x7f8238075710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f823c0ac000 tx=0x7f823c0af040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:08.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.369+0000 7f82337fe640 1 -- 172.21.15.131:0/2104344586 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f824c15d870 con 0x7f824c14db70 2024-09-17T02:47:08.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:08.529+0000 7f825112b640 1 -- 172.21.15.131:0/2104344586 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f824c1551b0 con 0x7f824c14db70 2024-09-17T02:47:09.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.349+0000 7f82337fe640 1 -- 172.21.15.131:0/2104344586 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) ==== 181+0+0 (secure 0 0 0) 0x7f8240016030 con 0x7f824c14db70 2024-09-17T02:47:09.347 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-09-17T02:47:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 -- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8238073200 msgr2=0x7f8238075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8238073200 0x7f8238075710 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f823c0ac000 tx=0x7f823c0af040 comp rx=0 tx=0).stop 2024-09-17T02:47:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 -- 172.21.15.131:0/2104344586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f824c14db70 msgr2=0x7f824c14e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f824c14db70 0x7f824c14e2c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f824000e790 tx=0x7f824000ec50 comp rx=0 tx=0).stop 2024-09-17T02:47:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 -- 172.21.15.131:0/2104344586 shutdown_connections 2024-09-17T02:47:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8238073200 0x7f8238075710 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:09.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f824c15cf20 0x7f824c15d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f824c14db70 0x7f824c14e2c0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 --2- 172.21.15.131:0/2104344586 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8234004b20 0x7f824c149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 -- 172.21.15.131:0/2104344586 >> 172.21.15.131:0/2104344586 conn(0x7f824c0734f0 msgr2=0x7f824c1504b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 -- 172.21.15.131:0/2104344586 shutdown_connections 2024-09-17T02:47:09.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.353+0000 7f825112b640 1 -- 172.21.15.131:0/2104344586 wait complete. 2024-09-17T02:47:09.377 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T02:47:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.541+0000 7fc7c66dc640 1 -- 172.21.15.131:0/325611587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c015c9d0 msgr2=0x7fc7c015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.541+0000 7fc7c66dc640 1 --2- 172.21.15.131:0/325611587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c015c9d0 0x7fc7c015cde0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00726b0 tx=0x7fc7b801ccc0 comp rx=0 tx=0).stop 2024-09-17T02:47:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.541+0000 7fc7c66dc640 1 -- 172.21.15.131:0/325611587 shutdown_connections 2024-09-17T02:47:09.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.541+0000 7fc7c66dc640 1 --2- 172.21.15.131:0/325611587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c015c9d0 0x7fc7c015cde0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.541+0000 7fc7c66dc640 1 --2- 172.21.15.131:0/325611587 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7c015ffc0 0x7fc7c01603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.541+0000 7fc7c66dc640 1 --2- 172.21.15.131:0/325611587 >> v2:172.21.15.189:3300/0 conn(0x7fc7c00cc870 0x7fc7c00b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.541+0000 7fc7c66dc640 1 -- 172.21.15.131:0/325611587 >> 172.21.15.131:0/325611587 conn(0x7fc7c00d05a0 msgr2=0x7fc7c00d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.541+0000 7fc7c66dc640 1 -- 172.21.15.131:0/325611587 shutdown_connections 2024-09-17T02:47:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.541+0000 7fc7c66dc640 1 -- 172.21.15.131:0/325611587 wait complete. 2024-09-17T02:47:09.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.541+0000 7fc7c66dc640 1 Processor -- start 2024-09-17T02:47:09.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.541+0000 7fc7c66dc640 1 -- start start 2024-09-17T02:47:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7c66dc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cc870 0x7fc7c0165260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7c66dc640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7c015c9d0 0x7fc7c0155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7c66dc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c015ffc0 0x7fc7c0160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7c66dc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c0070040 con 0x7fc7c00cc870 2024-09-17T02:47:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7c66dc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c006fec0 con 0x7fc7c015ffc0 2024-09-17T02:47:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7c66dc640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c00701c0 con 0x7fc7c015c9d0 2024-09-17T02:47:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cc870 0x7fc7c0165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cc870 0x7fc7c0165260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39498/0 (socket says 172.21.15.131:39498) 2024-09-17T02:47:09.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bffff640 1 -- 172.21.15.131:0/1753063576 learned_addr learned my addr 172.21.15.131:0/1753063576 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7c4c79640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c015ffc0 0x7fc7c0160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bf7fe640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7c015c9d0 0x7fc7c0155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:09.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bffff640 1 -- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7c015c9d0 msgr2=0x7fc7c0155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bffff640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7c015c9d0 0x7fc7c0155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bffff640 1 -- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c015ffc0 msgr2=0x7fc7c0160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bffff640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c015ffc0 0x7fc7c0160dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bffff640 1 -- 172.21.15.131:0/1753063576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c0165c40 con 0x7fc7c00cc870 2024-09-17T02:47:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bffff640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cc870 0x7fc7c0165260 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000c940 tx=0x7fc7b000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bcff9640 1 -- 172.21.15.131:0/1753063576 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0004220 con 0x7fc7c00cc870 2024-09-17T02:47:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bcff9640 1 -- 172.21.15.131:0/1753063576 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7b00079b0 con 0x7fc7c00cc870 2024-09-17T02:47:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7bcff9640 1 -- 172.21.15.131:0/1753063576 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00044b0 con 0x7fc7c00cc870 2024-09-17T02:47:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7c66dc640 1 -- 172.21.15.131:0/1753063576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c0165f20 con 0x7fc7c00cc870 2024-09-17T02:47:09.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.545+0000 7fc7c66dc640 1 -- 172.21.15.131:0/1753063576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c01664e0 con 0x7fc7c00cc870 2024-09-17T02:47:09.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.549+0000 7fc7c66dc640 1 -- 172.21.15.131:0/1753063576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c0070710 con 0x7fc7c00cc870 2024-09-17T02:47:09.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.549+0000 7fc7bcff9640 1 -- 172.21.15.131:0/1753063576 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc7b0005020 con 0x7fc7c00cc870 2024-09-17T02:47:09.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.549+0000 7fc7bcff9640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073250 0x7fc7ac075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:09.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.549+0000 7fc7bcff9640 1 -- 172.21.15.131:0/1753063576 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5835+0+0 (secure 0 0 0) 0x7fc7b00947d0 con 0x7fc7c00cc870 2024-09-17T02:47:09.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.549+0000 7fc7bf7fe640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073250 0x7fc7ac075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:09.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.553+0000 7fc7bf7fe640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073250 0x7fc7ac075760 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4005d90 tx=0x7fc7b4005d00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:09.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.553+0000 7fc7bcff9640 1 -- 172.21.15.131:0/1753063576 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc7b0014030 con 0x7fc7c00cc870 2024-09-17T02:47:09.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:09.725+0000 7fc7c66dc640 1 -- 172.21.15.131:0/1753063576 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc7c00d0b50 con 0x7fc7c00cc870 2024-09-17T02:47:10.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.353+0000 7fc7bcff9640 1 -- 172.21.15.131:0/1753063576 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) ==== 169+0+0 (secure 0 0 0) 0x7fc7b005f7d0 con 0x7fc7c00cc870 2024-09-17T02:47:10.350 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-09-17T02:47:10.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.357+0000 7fc7c66dc640 1 -- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073250 msgr2=0x7fc7ac075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:10.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.357+0000 7fc7c66dc640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073250 0x7fc7ac075760 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4005d90 tx=0x7fc7b4005d00 comp rx=0 tx=0).stop 2024-09-17T02:47:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.357+0000 7fc7c66dc640 1 -- 172.21.15.131:0/1753063576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cc870 msgr2=0x7fc7c0165260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.357+0000 7fc7c66dc640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cc870 0x7fc7c0165260 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000c940 tx=0x7fc7b000ce00 comp rx=0 tx=0).stop 2024-09-17T02:47:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.357+0000 7fc7c66dc640 1 -- 172.21.15.131:0/1753063576 shutdown_connections 2024-09-17T02:47:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.357+0000 7fc7c66dc640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ac073250 0x7fc7ac075760 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.357+0000 7fc7c66dc640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7c015ffc0 0x7fc7c0160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.357+0000 7fc7c66dc640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7c015c9d0 0x7fc7c0155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.357+0000 7fc7c66dc640 1 --2- 172.21.15.131:0/1753063576 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7c00cc870 0x7fc7c0165260 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.357+0000 7fc7c66dc640 1 -- 172.21.15.131:0/1753063576 >> 172.21.15.131:0/1753063576 conn(0x7fc7c00d05a0 msgr2=0x7fc7c0143800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:10.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.357+0000 7fc7c66dc640 1 -- 172.21.15.131:0/1753063576 shutdown_connections 2024-09-17T02:47:10.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.361+0000 7fc7c66dc640 1 -- 172.21.15.131:0/1753063576 wait complete. 2024-09-17T02:47:10.381 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T02:47:10.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 -- 172.21.15.131:0/654218493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb7180c4ac0 msgr2=0x7fb7180c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:10.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 --2- 172.21.15.131:0/654218493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb7180c4ac0 0x7fb7180c4ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009900 tx=0x7fb70c01c850 comp rx=0 tx=0).stop 2024-09-17T02:47:10.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 -- 172.21.15.131:0/654218493 shutdown_connections 2024-09-17T02:47:10.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 --2- 172.21.15.131:0/654218493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb7180c4ac0 0x7fb7180c4ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 --2- 172.21.15.131:0/654218493 >> v2:172.21.15.189:3300/0 conn(0x7fb71806e2c0 0x7fb7180c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 -- 172.21.15.131:0/654218493 >> 172.21.15.131:0/654218493 conn(0x7fb71806eea0 msgr2=0x7fb71806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:10.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 -- 172.21.15.131:0/654218493 shutdown_connections 2024-09-17T02:47:10.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 -- 172.21.15.131:0/654218493 wait complete. 2024-09-17T02:47:10.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 Processor -- start 2024-09-17T02:47:10.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 -- start start 2024-09-17T02:47:10.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71806e2c0 0x7fb7181f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:10.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7180c4ac0 0x7fb71815aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:10.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb7180cdbf0 0x7fb71815b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:10.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb7180cf0a0 con 0x7fb71806e2c0 2024-09-17T02:47:10.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb7180cef20 con 0x7fb7180c4ac0 2024-09-17T02:47:10.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb71d6ec640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb7180cf220 con 0x7fb7180cdbf0 2024-09-17T02:47:10.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb716ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71806e2c0 0x7fb7181f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:10.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb716ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71806e2c0 0x7fb7181f87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39514/0 (socket says 172.21.15.131:39514) 2024-09-17T02:47:10.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb716ffd640 1 -- 172.21.15.131:0/3030502041 learned_addr learned my addr 172.21.15.131:0/3030502041 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:10.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb7177fe640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb7180cdbf0 0x7fb71815b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:10.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.537+0000 7fb7167fc640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7180c4ac0 0x7fb71815aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:10.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb716ffd640 1 -- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb7180cdbf0 msgr2=0x7fb71815b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:10.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb716ffd640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb7180cdbf0 0x7fb71815b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb716ffd640 1 -- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7180c4ac0 msgr2=0x7fb71815aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:10.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb716ffd640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7180c4ac0 0x7fb71815aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb716ffd640 1 -- 172.21.15.131:0/3030502041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7180ca9a0 con 0x7fb71806e2c0 2024-09-17T02:47:10.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb7167fc640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7180c4ac0 0x7fb71815aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:10.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb7177fe640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb7180cdbf0 0x7fb71815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:47:10.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb716ffd640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71806e2c0 0x7fb7181f87b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb71000ebf0 tx=0x7fb71000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:10.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb6ff7fe640 1 -- 172.21.15.131:0/3030502041 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710010630 con 0x7fb71806e2c0 2024-09-17T02:47:10.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb6ff7fe640 1 -- 172.21.15.131:0/3030502041 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7100108c0 con 0x7fb71806e2c0 2024-09-17T02:47:10.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb71d6ec640 1 -- 172.21.15.131:0/3030502041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7180cac80 con 0x7fb71806e2c0 2024-09-17T02:47:10.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb6ff7fe640 1 -- 172.21.15.131:0/3030502041 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb710010cf0 con 0x7fb71806e2c0 2024-09-17T02:47:10.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb71d6ec640 1 -- 172.21.15.131:0/3030502041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7180cb240 con 0x7fb71806e2c0 2024-09-17T02:47:10.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb71d6ec640 1 -- 172.21.15.131:0/3030502041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7180c4580 con 0x7fb71806e2c0 2024-09-17T02:47:10.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb6ff7fe640 1 -- 172.21.15.131:0/3030502041 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb710018420 con 0x7fb71806e2c0 2024-09-17T02:47:10.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb6ff7fe640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb704073200 0x7fb704075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:10.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb6ff7fe640 1 -- 172.21.15.131:0/3030502041 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5421+0+0 (secure 0 0 0) 0x7fb710094e90 con 0x7fb71806e2c0 2024-09-17T02:47:10.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb7167fc640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb704073200 0x7fb704075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:10.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.541+0000 7fb7167fc640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb704073200 0x7fb704075710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0af000 tx=0x7fb70c01b090 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:10.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.545+0000 7fb6ff7fe640 1 -- 172.21.15.131:0/3030502041 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb71005ffe0 con 0x7fb71806e2c0 2024-09-17T02:47:10.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.713+0000 7fb71d6ec640 1 -- 172.21.15.131:0/3030502041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fb7180b19d0 con 0x7fb71806e2c0 2024-09-17T02:47:10.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb6ff7fe640 1 -- 172.21.15.131:0/3030502041 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7fb71005f8e0 con 0x7fb71806e2c0 2024-09-17T02:47:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 -- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb704073200 msgr2=0x7fb704075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb704073200 0x7fb704075710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0af000 tx=0x7fb70c01b090 comp rx=0 tx=0).stop 2024-09-17T02:47:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 -- 172.21.15.131:0/3030502041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71806e2c0 msgr2=0x7fb7181f87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71806e2c0 0x7fb7181f87b0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb71000ebf0 tx=0x7fb71000c6a0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 -- 172.21.15.131:0/3030502041 shutdown_connections 2024-09-17T02:47:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb704073200 0x7fb704075710 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb7180cdbf0 0x7fb71815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb7180c4ac0 0x7fb71815aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 --2- 172.21.15.131:0/3030502041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb71806e2c0 0x7fb7181f87b0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 -- 172.21.15.131:0/3030502041 >> 172.21.15.131:0/3030502041 conn(0x7fb71806eea0 msgr2=0x7fb7180b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:10.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 -- 172.21.15.131:0/3030502041 shutdown_connections 2024-09-17T02:47:10.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.721+0000 7fb71d6ec640 1 -- 172.21.15.131:0/3030502041 wait complete. 2024-09-17T02:47:10.740 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-09-17T02:47:10.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 -- 172.21.15.131:0/2948101648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14c0003a60 msgr2=0x7f14c0003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:10.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 --2- 172.21.15.131:0/2948101648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14c0003a60 0x7f14c0003ed0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009b40 tx=0x7f14d001d270 comp rx=0 tx=0).stop 2024-09-17T02:47:10.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 -- 172.21.15.131:0/2948101648 shutdown_connections 2024-09-17T02:47:10.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 --2- 172.21.15.131:0/2948101648 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14c0003a60 0x7f14c0003ed0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 -- 172.21.15.131:0/2948101648 >> 172.21.15.131:0/2948101648 conn(0x7f14d406eea0 msgr2=0x7f14d406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:10.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 -- 172.21.15.131:0/2948101648 shutdown_connections 2024-09-17T02:47:10.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 -- 172.21.15.131:0/2948101648 wait complete. 2024-09-17T02:47:10.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 Processor -- start 2024-09-17T02:47:10.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 -- start start 2024-09-17T02:47:10.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14c0003a60 0x7f14d41f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:10.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d41570f0 0x7f14d41586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:10.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14d41f7520 0x7f14d41f7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:10.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f14d40cc480 con 0x7f14c0003a60 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f14d40cc300 con 0x7f14d41570f0 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14db396640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f14d40cc600 con 0x7f14d41f7520 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d9132640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14c0003a60 0x7f14d41f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d9132640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14c0003a60 0x7f14d41f0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39522/0 (socket says 172.21.15.131:39522) 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d9132640 1 -- 172.21.15.131:0/2864598168 learned_addr learned my addr 172.21.15.131:0/2864598168 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d9933640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14d41f7520 0x7f14d41f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d8931640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d41570f0 0x7f14d41586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d9132640 1 -- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14d41f7520 msgr2=0x7f14d41f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d9132640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14d41f7520 0x7f14d41f7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d9132640 1 -- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d41570f0 msgr2=0x7f14d41586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d9132640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d41570f0 0x7f14d41586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d9132640 1 -- 172.21.15.131:0/2864598168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d41f7070 con 0x7f14c0003a60 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d8931640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d41570f0 0x7f14d41586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:10.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.893+0000 7f14d9132640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14c0003a60 0x7f14d41f0f40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f14d001d0a0 tx=0x7f14d0005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:10.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.897+0000 7f14c9ffb640 1 -- 172.21.15.131:0/2864598168 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d00bb070 con 0x7f14c0003a60 2024-09-17T02:47:10.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.897+0000 7f14c9ffb640 1 -- 172.21.15.131:0/2864598168 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14d0004ac0 con 0x7f14c0003a60 2024-09-17T02:47:10.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.897+0000 7f14c9ffb640 1 -- 172.21.15.131:0/2864598168 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d00b7440 con 0x7f14c0003a60 2024-09-17T02:47:10.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.897+0000 7f14db396640 1 -- 172.21.15.131:0/2864598168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d41f7e70 con 0x7f14c0003a60 2024-09-17T02:47:10.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.897+0000 7f14db396640 1 -- 172.21.15.131:0/2864598168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d41fad80 con 0x7f14c0003a60 2024-09-17T02:47:10.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.897+0000 7f14db396640 1 -- 172.21.15.131:0/2864598168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d4075b80 con 0x7f14c0003a60 2024-09-17T02:47:10.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.897+0000 7f14c9ffb640 1 -- 172.21.15.131:0/2864598168 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f14d00b5040 con 0x7f14c0003a60 2024-09-17T02:47:10.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.897+0000 7f14c9ffb640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f14b8073200 0x7f14b8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:10.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.897+0000 7f14c9ffb640 1 -- 172.21.15.131:0/2864598168 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5421+0+0 (secure 0 0 0) 0x7f14d012f690 con 0x7f14c0003a60 2024-09-17T02:47:10.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.897+0000 7f14d8931640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f14b8073200 0x7f14b8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:10.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.897+0000 7f14d8931640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f14b8073200 0x7f14b8075710 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f14d415e0e0 tx=0x7f14c4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:10.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:10.901+0000 7f14c9ffb640 1 -- 172.21.15.131:0/2864598168 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f14d0016f80 con 0x7f14c0003a60 2024-09-17T02:47:11.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.121+0000 7f14db396640 1 -- 172.21.15.131:0/2864598168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f14d40ca960 con 0x7f14c0003a60 2024-09-17T02:47:11.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.353+0000 7f14c9ffb640 1 -- 172.21.15.131:0/2864598168 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v297) ==== 166+0+0 (secure 0 0 0) 0x7f14d00fa7d0 con 0x7f14c0003a60 2024-09-17T02:47:11.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 -- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f14b8073200 msgr2=0x7f14b8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:11.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f14b8073200 0x7f14b8075710 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f14d415e0e0 tx=0x7f14c4009040 comp rx=0 tx=0).stop 2024-09-17T02:47:11.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 -- 172.21.15.131:0/2864598168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14c0003a60 msgr2=0x7f14d41f0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:11.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14c0003a60 0x7f14d41f0f40 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f14d001d0a0 tx=0x7f14d0005130 comp rx=0 tx=0).stop 2024-09-17T02:47:11.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 -- 172.21.15.131:0/2864598168 shutdown_connections 2024-09-17T02:47:11.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f14b8073200 0x7f14b8075710 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f14d41f7520 0x7f14d41f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f14d41570f0 0x7f14d41586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 --2- 172.21.15.131:0/2864598168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f14c0003a60 0x7f14d41f0f40 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 -- 172.21.15.131:0/2864598168 >> 172.21.15.131:0/2864598168 conn(0x7f14d406eea0 msgr2=0x7f14d4155650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:11.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 -- 172.21.15.131:0/2864598168 shutdown_connections 2024-09-17T02:47:11.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.357+0000 7f14db396640 1 -- 172.21.15.131:0/2864598168 wait complete. 2024-09-17T02:47:11.377 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-09-17T02:47:11.377 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-09-17T02:47:11.378 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-17T02:47:11.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:47:11.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.537+0000 7f26425ce640 1 -- 172.21.15.131:0/3726338453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f263c1604d0 msgr2=0x7f263c152be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.537+0000 7f26425ce640 1 --2- 172.21.15.131:0/3726338453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f263c1604d0 0x7f263c152be0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f263c160a90 tx=0x7f263401c7e0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.537+0000 7f26425ce640 1 -- 172.21.15.131:0/3726338453 shutdown_connections 2024-09-17T02:47:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.537+0000 7f26425ce640 1 --2- 172.21.15.131:0/3726338453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f263c1604d0 0x7f263c152be0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.537+0000 7f26425ce640 1 --2- 172.21.15.131:0/3726338453 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f263c0cd250 0x7f263c15ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.537+0000 7f26425ce640 1 --2- 172.21.15.131:0/3726338453 >> v2:172.21.15.189:3300/0 conn(0x7f263c0cce40 0x7f263c06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.537+0000 7f26425ce640 1 -- 172.21.15.131:0/3726338453 >> 172.21.15.131:0/3726338453 conn(0x7f263c06fbe0 msgr2=0x7f263c06fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.537+0000 7f26425ce640 1 -- 172.21.15.131:0/3726338453 shutdown_connections 2024-09-17T02:47:11.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.537+0000 7f26425ce640 1 -- 172.21.15.131:0/3726338453 wait complete. 2024-09-17T02:47:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.537+0000 7f26425ce640 1 Processor -- start 2024-09-17T02:47:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f26425ce640 1 -- start start 2024-09-17T02:47:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f26425ce640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f263c0cce40 0x7f263c14fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f26425ce640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f263c0cd250 0x7f263c15b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f26425ce640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f263c1604d0 0x7f263c15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f26425ce640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f263c15d2c0 con 0x7f263c0cd250 2024-09-17T02:47:11.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f26425ce640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f263c15d140 con 0x7f263c1604d0 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f26425ce640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f263c15d440 con 0x7f263c0cce40 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f263b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f263c0cd250 0x7f263c15b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f263b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f263c0cd250 0x7f263c15b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54442/0 (socket says 172.21.15.131:54442) 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f263b7fe640 1 -- 172.21.15.131:0/2833696620 learned_addr learned my addr 172.21.15.131:0/2833696620 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f2640b6b640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f263c1604d0 0x7f263c15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f263bfff640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f263c0cce40 0x7f263c14fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f263b7fe640 1 -- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f263c0cce40 msgr2=0x7f263c14fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f263b7fe640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f263c0cce40 0x7f263c14fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f263b7fe640 1 -- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f263c1604d0 msgr2=0x7f263c15b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f263b7fe640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f263c1604d0 0x7f263c15b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f263b7fe640 1 -- 172.21.15.131:0/2833696620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c0b69c0 con 0x7f263c0cd250 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f2640b6b640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f263c1604d0 0x7f263c15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:47:11.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f263b7fe640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f263c0cd250 0x7f263c15b3a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f263000d720 tx=0x7f263000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f2638ff9640 1 -- 172.21.15.131:0/2833696620 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630017070 con 0x7f263c0cd250 2024-09-17T02:47:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f2638ff9640 1 -- 172.21.15.131:0/2833696620 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2630009b10 con 0x7f263c0cd250 2024-09-17T02:47:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f2638ff9640 1 -- 172.21.15.131:0/2833696620 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630010460 con 0x7f263c0cd250 2024-09-17T02:47:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f26425ce640 1 -- 172.21.15.131:0/2833696620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263c1f8db0 con 0x7f263c0cd250 2024-09-17T02:47:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f26425ce640 1 -- 172.21.15.131:0/2833696620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263c1514e0 con 0x7f263c0cd250 2024-09-17T02:47:11.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.541+0000 7f26425ce640 1 -- 172.21.15.131:0/2833696620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f263c0b6c30 con 0x7f263c0cd250 2024-09-17T02:47:11.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.545+0000 7f2638ff9640 1 -- 172.21.15.131:0/2833696620 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f263001a070 con 0x7f263c0cd250 2024-09-17T02:47:11.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.545+0000 7f2638ff9640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2628073200 0x7f2628075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:11.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.545+0000 7f2638ff9640 1 -- 172.21.15.131:0/2833696620 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5421+0+0 (secure 0 0 0) 0x7f2630094260 con 0x7f263c0cd250 2024-09-17T02:47:11.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.545+0000 7f263bfff640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2628073200 0x7f2628075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:11.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.545+0000 7f263bfff640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2628073200 0x7f2628075710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f262c009d30 tx=0x7f262c0092e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:11.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.549+0000 7f2638ff9640 1 -- 172.21.15.131:0/2833696620 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f263005f390 con 0x7f263c0cd250 2024-09-17T02:47:11.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.705+0000 7f26425ce640 1 -- 172.21.15.131:0/2833696620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f263c15ff90 con 0x7f263c0cd250 2024-09-17T02:47:11.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f2638ff9640 1 -- 172.21.15.131:0/2833696620 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+13484 (secure 0 0 0) 0x7f263005ec90 con 0x7f263c0cd250 2024-09-17T02:47:11.706 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:11.706 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":83,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:47:10.352769+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6833/962976667":"2024-09-18T02:47:07.117591+0000","172.21.15.131:6835/655402879":"2024-09-18T02:45:41.796095+0000","172.21.15.131:6835/962976667":"2024-09-18T02:47:07.117591+0000","172.21.15.131:6833/655402879":"2024-09-18T02:45:41.796095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:47:11.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 -- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2628073200 msgr2=0x7f2628075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:11.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2628073200 0x7f2628075710 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f262c009d30 tx=0x7f262c0092e0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 -- 172.21.15.131:0/2833696620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f263c0cd250 msgr2=0x7f263c15b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f263c0cd250 0x7f263c15b3a0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f263000d720 tx=0x7f263000dbe0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 -- 172.21.15.131:0/2833696620 shutdown_connections 2024-09-17T02:47:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2628073200 0x7f2628075710 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f263c1604d0 0x7f263c15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f263c0cd250 0x7f263c15b3a0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 --2- 172.21.15.131:0/2833696620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f263c0cce40 0x7f263c14fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 -- 172.21.15.131:0/2833696620 >> 172.21.15.131:0/2833696620 conn(0x7f263c06fbe0 msgr2=0x7f263c15e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 -- 172.21.15.131:0/2833696620 shutdown_connections 2024-09-17T02:47:11.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.709+0000 7f26425ce640 1 -- 172.21.15.131:0/2833696620 wait complete. 2024-09-17T02:47:11.730 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T02:40:41.163614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T02:47:11.730 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T02:47:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.877+0000 7f0c20ba4640 1 -- 172.21.15.131:0/229867229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1c0d3470 msgr2=0x7f0c1c162b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.877+0000 7f0c20ba4640 1 --2- 172.21.15.131:0/229867229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1c0d3470 0x7f0c1c162b20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c06ecf0 tx=0x7f0c1401cb60 comp rx=0 tx=0).stop 2024-09-17T02:47:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.877+0000 7f0c20ba4640 1 -- 172.21.15.131:0/229867229 shutdown_connections 2024-09-17T02:47:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.877+0000 7f0c20ba4640 1 --2- 172.21.15.131:0/229867229 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1c0d3470 0x7f0c1c162b20 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.877+0000 7f0c20ba4640 1 --2- 172.21.15.131:0/229867229 >> v2:172.21.15.189:3300/0 conn(0x7f0c1c0cf710 0x7f0c1c15f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.877+0000 7f0c20ba4640 1 -- 172.21.15.131:0/229867229 >> 172.21.15.131:0/229867229 conn(0x7f0c1c0d1b10 msgr2=0x7f0c1c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.877+0000 7f0c20ba4640 1 -- 172.21.15.131:0/229867229 shutdown_connections 2024-09-17T02:47:11.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.877+0000 7f0c20ba4640 1 -- 172.21.15.131:0/229867229 wait complete. 2024-09-17T02:47:11.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.877+0000 7f0c20ba4640 1 Processor -- start 2024-09-17T02:47:11.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.877+0000 7f0c20ba4640 1 -- start start 2024-09-17T02:47:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c20ba4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1c0cf710 0x7f0c1c0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c20ba4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c1c0d3470 0x7f0c1c1fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c20ba4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0c1c15f7d0 0x7f0c1c1fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c20ba4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c16bdb0 con 0x7f0c1c0cf710 2024-09-17T02:47:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c20ba4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c15fdf0 con 0x7f0c1c0d3470 2024-09-17T02:47:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c20ba4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1c16bf30 con 0x7f0c1c15f7d0 2024-09-17T02:47:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c1a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1c0cf710 0x7f0c1c0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c1a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1c0cf710 0x7f0c1c0cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54466/0 (socket says 172.21.15.131:54466) 2024-09-17T02:47:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c1a59c640 1 -- 172.21.15.131:0/2041885191 learned_addr learned my addr 172.21.15.131:0/2041885191 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:11.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c13fff640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c1c0d3470 0x7f0c1c1fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c1ad9d640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0c1c15f7d0 0x7f0c1c1fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c1a59c640 1 -- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0c1c15f7d0 msgr2=0x7f0c1c1fc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c1a59c640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0c1c15f7d0 0x7f0c1c1fc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c1a59c640 1 -- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c1c0d3470 msgr2=0x7f0c1c1fbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c1a59c640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c1c0d3470 0x7f0c1c1fbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c1a59c640 1 -- 172.21.15.131:0/2041885191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c158260 con 0x7f0c1c0cf710 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c1a59c640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1c0cf710 0x7f0c1c0cca50 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c970 tx=0x7f0c0c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c12ffd640 1 -- 172.21.15.131:0/2041885191 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c007bb0 con 0x7f0c1c0cf710 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c12ffd640 1 -- 172.21.15.131:0/2041885191 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c0c007e40 con 0x7f0c1c0cf710 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c20ba4640 1 -- 172.21.15.131:0/2041885191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1c158540 con 0x7f0c1c0cf710 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c12ffd640 1 -- 172.21.15.131:0/2041885191 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00f6b0 con 0x7f0c1c0cf710 2024-09-17T02:47:11.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c20ba4640 1 -- 172.21.15.131:0/2041885191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1c158b00 con 0x7f0c1c0cf710 2024-09-17T02:47:11.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.881+0000 7f0c20ba4640 1 -- 172.21.15.131:0/2041885191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1c15f290 con 0x7f0c1c0cf710 2024-09-17T02:47:11.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.885+0000 7f0c12ffd640 1 -- 172.21.15.131:0/2041885191 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f0c0c00f850 con 0x7f0c1c0cf710 2024-09-17T02:47:11.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.885+0000 7f0c12ffd640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073230 0x7f0bfc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:11.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.885+0000 7f0c12ffd640 1 -- 172.21.15.131:0/2041885191 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5421+0+0 (secure 0 0 0) 0x7f0c0c0947d0 con 0x7f0c1c0cf710 2024-09-17T02:47:11.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.885+0000 7f0c13fff640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073230 0x7f0bfc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:11.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.885+0000 7f0c13fff640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073230 0x7f0bfc075740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c157340 tx=0x7f0c04009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:11.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:11.889+0000 7f0c12ffd640 1 -- 172.21.15.131:0/2041885191 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0c0c014030 con 0x7f0c1c0cf710 2024-09-17T02:47:12.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 -- 172.21.15.131:0/2041885191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0c1c0b8f80 con 0x7f0c1c0cf710 2024-09-17T02:47:12.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c12ffd640 1 -- 172.21.15.131:0/2041885191 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f0c0c05fa30 con 0x7f0c1c0cf710 2024-09-17T02:47:12.054 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-09-17T02:47:12.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 -- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073230 msgr2=0x7f0bfc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:12.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073230 0x7f0bfc075740 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c157340 tx=0x7f0c04009040 comp rx=0 tx=0).stop 2024-09-17T02:47:12.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 -- 172.21.15.131:0/2041885191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1c0cf710 msgr2=0x7f0c1c0cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:12.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1c0cf710 0x7f0c1c0cca50 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c970 tx=0x7f0c0c00ce30 comp rx=0 tx=0).stop 2024-09-17T02:47:12.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 -- 172.21.15.131:0/2041885191 shutdown_connections 2024-09-17T02:47:12.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0bfc073230 0x7f0bfc075740 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:12.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0c1c15f7d0 0x7f0c1c1fc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:12.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f0c1c0d3470 0x7f0c1c1fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:12.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 --2- 172.21.15.131:0/2041885191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0c1c0cf710 0x7f0c1c0cca50 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:12.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 -- 172.21.15.131:0/2041885191 >> 172.21.15.131:0/2041885191 conn(0x7f0c1c0d1b10 msgr2=0x7f0c1c0b7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:12.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 -- 172.21.15.131:0/2041885191 shutdown_connections 2024-09-17T02:47:12.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.057+0000 7f0c20ba4640 1 -- 172.21.15.131:0/2041885191 wait complete. 2024-09-17T02:47:12.076 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-09-17T02:47:12.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 -- 172.21.15.131:0/3133523572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1738004810 msgr2=0x7f1738004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:12.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 --2- 172.21.15.131:0/3133523572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1738004810 0x7f1738004c20 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f173c009b40 tx=0x7f173c01d250 comp rx=0 tx=0).stop 2024-09-17T02:47:12.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 -- 172.21.15.131:0/3133523572 shutdown_connections 2024-09-17T02:47:12.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 --2- 172.21.15.131:0/3133523572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1738004810 0x7f1738004c20 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:12.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 -- 172.21.15.131:0/3133523572 >> 172.21.15.131:0/3133523572 conn(0x7f174c06eea0 msgr2=0x7f174c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:12.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 -- 172.21.15.131:0/3133523572 shutdown_connections 2024-09-17T02:47:12.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 -- 172.21.15.131:0/3133523572 wait complete. 2024-09-17T02:47:12.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 Processor -- start 2024-09-17T02:47:12.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 -- start start 2024-09-17T02:47:12.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1738004810 0x7f174c0afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:12.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f174c14eae0 0x7f174c0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:12.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f174c1504b0 0x7f174c166b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f174c06fb40 con 0x7f1738004810 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f174c06f9c0 con 0x7f174c14eae0 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f1751277640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f174c06fcc0 con 0x7f174c1504b0 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f174affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1738004810 0x7f174c0afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f174affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1738004810 0x7f174c0afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54490/0 (socket says 172.21.15.131:54490) 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f174affd640 1 -- 172.21.15.131:0/4160865850 learned_addr learned my addr 172.21.15.131:0/4160865850 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f174a7fc640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f174c14eae0 0x7f174c0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f174b7fe640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f174c1504b0 0x7f174c166b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f174affd640 1 -- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f174c1504b0 msgr2=0x7f174c166b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f174affd640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f174c1504b0 0x7f174c166b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f174affd640 1 -- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f174c14eae0 msgr2=0x7f174c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f174affd640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f174c14eae0 0x7f174c0b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f174affd640 1 -- 172.21.15.131:0/4160865850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c15a540 con 0x7f1738004810 2024-09-17T02:47:12.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.221+0000 7f174affd640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1738004810 0x7f174c0afb40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f17340116b0 tx=0x7f1734011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:12.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.225+0000 7f17437fe640 1 -- 172.21.15.131:0/4160865850 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734017070 con 0x7f1738004810 2024-09-17T02:47:12.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.225+0000 7f17437fe640 1 -- 172.21.15.131:0/4160865850 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f173400fc00 con 0x7f1738004810 2024-09-17T02:47:12.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.225+0000 7f17437fe640 1 -- 172.21.15.131:0/4160865850 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1734013450 con 0x7f1738004810 2024-09-17T02:47:12.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.225+0000 7f1751277640 1 -- 172.21.15.131:0/4160865850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c15a820 con 0x7f1738004810 2024-09-17T02:47:12.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.225+0000 7f1751277640 1 -- 172.21.15.131:0/4160865850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c15ade0 con 0x7f1738004810 2024-09-17T02:47:12.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.225+0000 7f1751277640 1 -- 172.21.15.131:0/4160865850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174c073430 con 0x7f1738004810 2024-09-17T02:47:12.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.225+0000 7f17437fe640 1 -- 172.21.15.131:0/4160865850 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f17340135f0 con 0x7f1738004810 2024-09-17T02:47:12.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.225+0000 7f17437fe640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f172c073230 0x7f172c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:12.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.225+0000 7f17437fe640 1 -- 172.21.15.131:0/4160865850 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5421+0+0 (secure 0 0 0) 0x7f1734098130 con 0x7f1738004810 2024-09-17T02:47:12.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.225+0000 7f174a7fc640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f172c073230 0x7f172c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:12.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.225+0000 7f174a7fc640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f172c073230 0x7f172c075740 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f173c0ad000 tx=0x7f173c019ff0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.229+0000 7f17437fe640 1 -- 172.21.15.131:0/4160865850 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f174c06e7f0 con 0x7f1738004810 2024-09-17T02:47:12.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:12.445+0000 7f1751277640 1 -- 172.21.15.131:0/4160865850 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f174c06e9d0 con 0x7f1738004810 2024-09-17T02:47:13.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.365+0000 7f17437fe640 1 -- 172.21.15.131:0/4160865850 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v299) ==== 170+0+0 (secure 0 0 0) 0x7f1734063300 con 0x7f1738004810 2024-09-17T02:47:13.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 -- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f172c073230 msgr2=0x7f172c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:13.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f172c073230 0x7f172c075740 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f173c0ad000 tx=0x7f173c019ff0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 -- 172.21.15.131:0/4160865850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1738004810 msgr2=0x7f174c0afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:13.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1738004810 0x7f174c0afb40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f17340116b0 tx=0x7f1734011b70 comp rx=0 tx=0).stop 2024-09-17T02:47:13.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 -- 172.21.15.131:0/4160865850 shutdown_connections 2024-09-17T02:47:13.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f172c073230 0x7f172c075740 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f174c1504b0 0x7f174c166b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f174c14eae0 0x7f174c0b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 --2- 172.21.15.131:0/4160865850 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1738004810 0x7f174c0afb40 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 -- 172.21.15.131:0/4160865850 >> 172.21.15.131:0/4160865850 conn(0x7f174c06eea0 msgr2=0x7f174c0ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:13.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 -- 172.21.15.131:0/4160865850 shutdown_connections 2024-09-17T02:47:13.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.369+0000 7f1751277640 1 -- 172.21.15.131:0/4160865850 wait complete. 2024-09-17T02:47:13.394 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T02:47:13.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.549+0000 7f99bd1ac640 1 -- 172.21.15.131:0/3327004545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b815cba0 msgr2=0x7f99b80ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:13.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.549+0000 7f99bd1ac640 1 --2- 172.21.15.131:0/3327004545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b815cba0 0x7f99b80ccd70 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f99b815dcc0 tx=0x7f99a801c690 comp rx=0 tx=0).stop 2024-09-17T02:47:13.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.549+0000 7f99bd1ac640 1 -- 172.21.15.131:0/3327004545 shutdown_connections 2024-09-17T02:47:13.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.549+0000 7f99bd1ac640 1 --2- 172.21.15.131:0/3327004545 >> v2:172.21.15.189:3300/0 conn(0x7f99b80d0160 0x7f99b80d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.549+0000 7f99bd1ac640 1 --2- 172.21.15.131:0/3327004545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b815cba0 0x7f99b80ccd70 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.549+0000 7f99bd1ac640 1 -- 172.21.15.131:0/3327004545 >> 172.21.15.131:0/3327004545 conn(0x7f99b80cf110 msgr2=0x7f99b80cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:13.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.549+0000 7f99bd1ac640 1 -- 172.21.15.131:0/3327004545 shutdown_connections 2024-09-17T02:47:13.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.549+0000 7f99bd1ac640 1 -- 172.21.15.131:0/3327004545 wait complete. 2024-09-17T02:47:13.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99bd1ac640 1 Processor -- start 2024-09-17T02:47:13.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99bd1ac640 1 -- start start 2024-09-17T02:47:13.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99bd1ac640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f99b80b2210 0x7f99b8152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:13.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99bd1ac640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b80d0160 0x7f99b8070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:13.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99bd1ac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99b815cba0 0x7f99b80b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:13.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99bd1ac640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f99b81537e0 con 0x7f99b80d0160 2024-09-17T02:47:13.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99bd1ac640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f99b8153660 con 0x7f99b815cba0 2024-09-17T02:47:13.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99bd1ac640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f99b8153960 con 0x7f99b80b2210 2024-09-17T02:47:13.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99b659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b80d0160 0x7f99b8070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:13.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99b659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b80d0160 0x7f99b8070ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54506/0 (socket says 172.21.15.131:54506) 2024-09-17T02:47:13.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99b659c640 1 -- 172.21.15.131:0/218616107 learned_addr learned my addr 172.21.15.131:0/218616107 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:13.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99b6d9d640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f99b80b2210 0x7f99b8152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99b659c640 1 -- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f99b80b2210 msgr2=0x7f99b8152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99b659c640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f99b80b2210 0x7f99b8152700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99b659c640 1 -- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99b815cba0 msgr2=0x7f99b80b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99b659c640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99b815cba0 0x7f99b80b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99b659c640 1 -- 172.21.15.131:0/218616107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b8152c40 con 0x7f99b80d0160 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99b6d9d640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f99b80b2210 0x7f99b8152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99b659c640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b80d0160 0x7f99b8070ea0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f99ac005d10 tx=0x7f99ac00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f999f7fe640 1 -- 172.21.15.131:0/218616107 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac017070 con 0x7f99b80d0160 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f999f7fe640 1 -- 172.21.15.131:0/218616107 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99ac004110 con 0x7f99b80d0160 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99bd1ac640 1 -- 172.21.15.131:0/218616107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b80c6210 con 0x7f99b80d0160 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f999f7fe640 1 -- 172.21.15.131:0/218616107 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac0099d0 con 0x7f99b80d0160 2024-09-17T02:47:13.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99bd1ac640 1 -- 172.21.15.131:0/218616107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b80c67d0 con 0x7f99b80d0160 2024-09-17T02:47:13.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.553+0000 7f99bd1ac640 1 -- 172.21.15.131:0/218616107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b80d0570 con 0x7f99b80d0160 2024-09-17T02:47:13.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.557+0000 7f999f7fe640 1 -- 172.21.15.131:0/218616107 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f99ac009b70 con 0x7f99b80d0160 2024-09-17T02:47:13.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.557+0000 7f999f7fe640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f99a4073230 0x7f99a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:13.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.557+0000 7f999f7fe640 1 -- 172.21.15.131:0/218616107 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5421+0+0 (secure 0 0 0) 0x7f99ac094120 con 0x7f99b80d0160 2024-09-17T02:47:13.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.557+0000 7f99b6d9d640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f99a4073230 0x7f99a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:13.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.557+0000 7f99b6d9d640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f99a4073230 0x7f99a4075740 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f99a80af000 tx=0x7f99a801aed0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:13.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.561+0000 7f999f7fe640 1 -- 172.21.15.131:0/218616107 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f99ac05f250 con 0x7f99b80d0160 2024-09-17T02:47:13.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.733+0000 7f99bd1ac640 1 -- 172.21.15.131:0/218616107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f99b806a170 con 0x7f99b80d0160 2024-09-17T02:47:13.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.733+0000 7f999f7fe640 1 -- 172.21.15.131:0/218616107 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+2294 (secure 0 0 0) 0x7f99ac05eb50 con 0x7f99b80d0160 2024-09-17T02:47:13.731 INFO:teuthology.orchestra.run.smithi131.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T02:47:13.731 INFO:teuthology.orchestra.run.smithi131.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T02:47:13.731 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced client_mount_timeout 30 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_client 20/20 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_mgr 20/20 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_ms 1/1 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3855.598857 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3598.912856 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3977.024117 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3823.557728 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4643.710284 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4452.598518 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6659.946233 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4539.778076 2024-09-17T02:47:13.732 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds 20/20 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_ms 2/2 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_frag true 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_scatterstat true 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_verify_scatter true 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced client_mount_timeout 600 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_client 20/20 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_ms 1/1 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T02:47:13.733 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T02:47:13.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 -- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f99a4073230 msgr2=0x7f99a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:13.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f99a4073230 0x7f99a4075740 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f99a80af000 tx=0x7f99a801aed0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 -- 172.21.15.131:0/218616107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b80d0160 msgr2=0x7f99b8070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:13.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b80d0160 0x7f99b8070ea0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f99ac005d10 tx=0x7f99ac00dbb0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 -- 172.21.15.131:0/218616107 shutdown_connections 2024-09-17T02:47:13.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f99a4073230 0x7f99a4075740 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f99b815cba0 0x7f99b80b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f99b80d0160 0x7f99b8070ea0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 --2- 172.21.15.131:0/218616107 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f99b80b2210 0x7f99b8152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 -- 172.21.15.131:0/218616107 >> 172.21.15.131:0/218616107 conn(0x7f99b80cf110 msgr2=0x7f99b815dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:13.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 -- 172.21.15.131:0/218616107 shutdown_connections 2024-09-17T02:47:13.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.737+0000 7f99bd1ac640 1 -- 172.21.15.131:0/218616107 wait complete. 2024-09-17T02:47:13.754 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T02:47:13.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.913+0000 7f9be3c29640 1 -- 172.21.15.131:0/817011129 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bdc0743b0 msgr2=0x7f9bdc0b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:13.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.913+0000 7f9be3c29640 1 --2- 172.21.15.131:0/817011129 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bdc0743b0 0x7f9bdc0b4940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009c80 tx=0x7f9bd801cda0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.913+0000 7f9be3c29640 1 -- 172.21.15.131:0/817011129 shutdown_connections 2024-09-17T02:47:13.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.913+0000 7f9be3c29640 1 --2- 172.21.15.131:0/817011129 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bdc0743b0 0x7f9bdc0b4940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.913+0000 7f9be3c29640 1 --2- 172.21.15.131:0/817011129 >> v2:172.21.15.189:3300/0 conn(0x7f9bdc0ad870 0x7f9bdc1622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.913+0000 7f9be3c29640 1 -- 172.21.15.131:0/817011129 >> 172.21.15.131:0/817011129 conn(0x7f9bdc070720 msgr2=0x7f9bdc0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:13.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.913+0000 7f9be3c29640 1 -- 172.21.15.131:0/817011129 shutdown_connections 2024-09-17T02:47:13.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.913+0000 7f9be3c29640 1 -- 172.21.15.131:0/817011129 wait complete. 2024-09-17T02:47:13.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be3c29640 1 Processor -- start 2024-09-17T02:47:13.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be3c29640 1 -- start start 2024-09-17T02:47:13.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be3c29640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bdc0ad870 0x7f9bdc143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:13.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be3c29640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bdc162830 0x7f9bdc155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:13.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be3c29640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bdc161630 0x7f9bdc1550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be3c29640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc16bd30 con 0x7f9bdc162830 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be3c29640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9bdc16bbb0 con 0x7f9bdc0ad870 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be3c29640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9bdc16beb0 con 0x7f9bdc161630 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be11c4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bdc162830 0x7f9bdc155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be11c4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bdc162830 0x7f9bdc155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54520/0 (socket says 172.21.15.131:54520) 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be11c4640 1 -- 172.21.15.131:0/3932774692 learned_addr learned my addr 172.21.15.131:0/3932774692 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be19c5640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bdc0ad870 0x7f9bdc143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be21c6640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bdc161630 0x7f9bdc1550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be11c4640 1 -- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bdc161630 msgr2=0x7f9bdc1550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be11c4640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bdc161630 0x7f9bdc1550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be11c4640 1 -- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bdc0ad870 msgr2=0x7f9bdc143c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be11c4640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bdc0ad870 0x7f9bdc143c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be11c4640 1 -- 172.21.15.131:0/3932774692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bdc1441a0 con 0x7f9bdc162830 2024-09-17T02:47:13.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be11c4640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bdc162830 0x7f9bdc155e70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00c3d0 tx=0x7f9bcc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9bd27fc640 1 -- 172.21.15.131:0/3932774692 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc017070 con 0x7f9bdc162830 2024-09-17T02:47:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9bd27fc640 1 -- 172.21.15.131:0/3932774692 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bcc0053c0 con 0x7f9bdc162830 2024-09-17T02:47:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be3c29640 1 -- 172.21.15.131:0/3932774692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bdc160590 con 0x7f9bdc162830 2024-09-17T02:47:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9bd27fc640 1 -- 172.21.15.131:0/3932774692 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc003880 con 0x7f9bdc162830 2024-09-17T02:47:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.917+0000 7f9be3c29640 1 -- 172.21.15.131:0/3932774692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bdc160b50 con 0x7f9bdc162830 2024-09-17T02:47:13.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.921+0000 7f9be3c29640 1 -- 172.21.15.131:0/3932774692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bdc0b4940 con 0x7f9bdc162830 2024-09-17T02:47:13.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.921+0000 7f9bd27fc640 1 -- 172.21.15.131:0/3932774692 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9bcc01a070 con 0x7f9bdc162830 2024-09-17T02:47:13.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.921+0000 7f9bd27fc640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bc0073200 0x7f9bc0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:13.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.921+0000 7f9bd27fc640 1 -- 172.21.15.131:0/3932774692 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5421+0+0 (secure 0 0 0) 0x7f9bcc093430 con 0x7f9bdc162830 2024-09-17T02:47:13.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.921+0000 7f9be19c5640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bc0073200 0x7f9bc0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:13.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.921+0000 7f9be19c5640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bc0073200 0x7f9bc0075710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009ad0 tx=0x7f9bd40091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:13.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:13.925+0000 7f9bd27fc640 1 -- 172.21.15.131:0/3932774692 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9bcc05e570 con 0x7f9bdc162830 2024-09-17T02:47:14.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.081+0000 7f9be3c29640 1 -- 172.21.15.131:0/3932774692 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f9bdc160320 con 0x7f9bdc162830 2024-09-17T02:47:14.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.085+0000 7f9bd27fc640 1 -- 172.21.15.131:0/3932774692 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f9bcc05de70 con 0x7f9bdc162830 2024-09-17T02:47:14.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 -- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bc0073200 msgr2=0x7f9bc0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:14.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bc0073200 0x7f9bc0075710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009ad0 tx=0x7f9bd40091e0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 -- 172.21.15.131:0/3932774692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bdc162830 msgr2=0x7f9bdc155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:14.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bdc162830 0x7f9bdc155e70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00c3d0 tx=0x7f9bcc00c890 comp rx=0 tx=0).stop 2024-09-17T02:47:14.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 -- 172.21.15.131:0/3932774692 shutdown_connections 2024-09-17T02:47:14.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bc0073200 0x7f9bc0075710 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bdc161630 0x7f9bdc1550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bdc162830 0x7f9bdc155e70 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 --2- 172.21.15.131:0/3932774692 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bdc0ad870 0x7f9bdc143c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 -- 172.21.15.131:0/3932774692 >> 172.21.15.131:0/3932774692 conn(0x7f9bdc070720 msgr2=0x7f9bdc0af700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:14.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 -- 172.21.15.131:0/3932774692 shutdown_connections 2024-09-17T02:47:14.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.089+0000 7f9be3c29640 1 -- 172.21.15.131:0/3932774692 wait complete. 2024-09-17T02:47:14.106 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:47:14.106 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:47:14.189 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:47:14.248 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:47:14.248 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:47:14.248 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:47:14.254 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:47:14.254 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:47:14.254 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:47:14.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:47:14.260 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:47:14.260 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:47:14.302 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:47:14.302 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:47:14.302 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:47:14.308 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:47:14.309 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T02:47:14.309 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:47:14.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.509+0000 7fbc05f08640 1 -- 172.21.15.131:0/1352773493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc000cd470 msgr2=0x7fbc0006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:14.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.509+0000 7fbc05f08640 1 --2- 172.21.15.131:0/1352773493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc000cd470 0x7fbc0006efb0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fbc000b0b40 tx=0x7fbbf001c770 comp rx=0 tx=0).stop 2024-09-17T02:47:14.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.509+0000 7fbc05f08640 1 -- 172.21.15.131:0/1352773493 shutdown_connections 2024-09-17T02:47:14.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.509+0000 7fbc05f08640 1 --2- 172.21.15.131:0/1352773493 >> v2:172.21.15.189:3300/0 conn(0x7fbc0015bf80 0x7fbc00151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.509+0000 7fbc05f08640 1 --2- 172.21.15.131:0/1352773493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc000cd470 0x7fbc0006efb0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.509+0000 7fbc05f08640 1 -- 172.21.15.131:0/1352773493 >> 172.21.15.131:0/1352773493 conn(0x7fbc00073d10 msgr2=0x7fbc00074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:14.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.509+0000 7fbc05f08640 1 -- 172.21.15.131:0/1352773493 shutdown_connections 2024-09-17T02:47:14.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.509+0000 7fbc05f08640 1 -- 172.21.15.131:0/1352773493 wait complete. 2024-09-17T02:47:14.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.509+0000 7fbc05f08640 1 Processor -- start 2024-09-17T02:47:14.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.509+0000 7fbc05f08640 1 -- start start 2024-09-17T02:47:14.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbc05f08640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc0006f4f0 0x7fbc000b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:14.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbc05f08640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbc000cd470 0x7fbc000b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:14.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbc05f08640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0015bf80 0x7fbc00151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:14.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbc05f08640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00169000 con 0x7fbc0015bf80 2024-09-17T02:47:14.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbc05f08640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbc000cc330 con 0x7fbc0006f4f0 2024-09-17T02:47:14.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbc05f08640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fbc00169180 con 0x7fbc000cd470 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbfffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0015bf80 0x7fbc00151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbfffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0015bf80 0x7fbc00151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54526/0 (socket says 172.21.15.131:54526) 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbfffff640 1 -- 172.21.15.131:0/1553719418 learned_addr learned my addr 172.21.15.131:0/1553719418 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbff7fe640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc0006f4f0 0x7fbc000b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbfeffd640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbc000cd470 0x7fbc000b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbfffff640 1 -- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbc000cd470 msgr2=0x7fbc000b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbfffff640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbc000cd470 0x7fbc000b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbfffff640 1 -- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc0006f4f0 msgr2=0x7fbc000b67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbfffff640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc0006f4f0 0x7fbc000b67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbfffff640 1 -- 172.21.15.131:0/1553719418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc001fc3b0 con 0x7fbc0015bf80 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbfeffd640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbc000cd470 0x7fbc000b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbff7fe640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc0006f4f0 0x7fbc000b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbfffff640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0015bf80 0x7fbc00151120 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800e9a0 tx=0x7fbbf800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:14.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbe7fff640 1 -- 172.21.15.131:0/1553719418 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8004d30 con 0x7fbc0015bf80 2024-09-17T02:47:14.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbe7fff640 1 -- 172.21.15.131:0/1553719418 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbf800cac0 con 0x7fbc0015bf80 2024-09-17T02:47:14.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbc05f08640 1 -- 172.21.15.131:0/1553719418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc001fc690 con 0x7fbc0015bf80 2024-09-17T02:47:14.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbe7fff640 1 -- 172.21.15.131:0/1553719418 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8005020 con 0x7fbc0015bf80 2024-09-17T02:47:14.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbc05f08640 1 -- 172.21.15.131:0/1553719418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc001fcc50 con 0x7fbc0015bf80 2024-09-17T02:47:14.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbc05f08640 1 -- 172.21.15.131:0/1553719418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0006efb0 con 0x7fbc0015bf80 2024-09-17T02:47:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbe7fff640 1 -- 172.21.15.131:0/1553719418 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbbf80051c0 con 0x7fbc0015bf80 2024-09-17T02:47:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbe7fff640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbec073230 0x7fbbec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbe7fff640 1 -- 172.21.15.131:0/1553719418 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5421+0+0 (secure 0 0 0) 0x7fbbf8094950 con 0x7fbc0015bf80 2024-09-17T02:47:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.513+0000 7fbbff7fe640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbec073230 0x7fbbec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:14.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.517+0000 7fbbff7fe640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbec073230 0x7fbbec075740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00af000 tx=0x7fbbf001afb0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:14.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.517+0000 7fbbe7fff640 1 -- 172.21.15.131:0/1553719418 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbbf8016030 con 0x7fbc0015bf80 2024-09-17T02:47:14.681 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:47:14.683 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:47:14.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 -- 172.21.15.131:0/1553719418 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc000d0d80 con 0x7fbc0015bf80 2024-09-17T02:47:14.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbbe7fff640 1 -- 172.21.15.131:0/1553719418 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+3917 (secure 0 0 0) 0x7fbbf805fa90 con 0x7fbc0015bf80 2024-09-17T02:47:14.722 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:14.722 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":56,"btime":"2024-09-17T02:47:08:121045+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5192,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/2131411028","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":2131411028},{"type":"v1","addr":"172.21.15.189:6837","nonce":2131411028}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5581,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/152179737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":152179737},{"type":"v1","addr":"172.21.15.189:6835","nonce":152179737}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6396,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/591387669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":591387669},{"type":"v1","addr":"172.21.15.131:6834","nonce":591387669}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6954,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3380746984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":3380746984},{"type":"v1","addr":"172.21.15.131:6835","nonce":3380746984}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[]} 2024-09-17T02:47:14.723 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 56 2024-09-17T02:47:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 -- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbec073230 msgr2=0x7fbbec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbec073230 0x7fbbec075740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00af000 tx=0x7fbbf001afb0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 -- 172.21.15.131:0/1553719418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0015bf80 msgr2=0x7fbc00151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0015bf80 0x7fbc00151120 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800e9a0 tx=0x7fbbf800ee60 comp rx=0 tx=0).stop 2024-09-17T02:47:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 -- 172.21.15.131:0/1553719418 shutdown_connections 2024-09-17T02:47:14.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbec073230 0x7fbbec075740 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbc0015bf80 0x7fbc00151120 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbc000cd470 0x7fbc000b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 --2- 172.21.15.131:0/1553719418 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbc0006f4f0 0x7fbc000b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 -- 172.21.15.131:0/1553719418 >> 172.21.15.131:0/1553719418 conn(0x7fbc00073d10 msgr2=0x7fbc000b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 -- 172.21.15.131:0/1553719418 shutdown_connections 2024-09-17T02:47:14.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:14.725+0000 7fbc05f08640 1 -- 172.21.15.131:0/1553719418 wait complete. 2024-09-17T02:47:14.745 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T02:47:14.745 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T02:47:14.745 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T02:47:14.745 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:47:14.745 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T02:47:14.745 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T02:47:14.746 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T02:47:14.746 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:47:14.746 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T02:47:14.746 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T02:47:14.746 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T02:47:14.746 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:47:14.746 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T02:47:14.746 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T02:47:14.747 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T02:47:14.747 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:47:14.785 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T02:47:14.785 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T02:47:14.791 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T02:47:14.791 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T02:47:14.814 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T02:47:14.814 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T02:47:14.837 INFO:tasks.ceph.mds.d:Started 2024-09-17T02:47:14.837 DEBUG:teuthology.parallel:result is None 2024-09-17T02:47:14.839 INFO:tasks.ceph.mds.b:Started 2024-09-17T02:47:14.839 DEBUG:teuthology.parallel:result is None 2024-09-17T02:47:14.858 INFO:tasks.ceph.mds.a:Started 2024-09-17T02:47:14.858 DEBUG:teuthology.parallel:result is None 2024-09-17T02:47:14.934 INFO:tasks.ceph.mds.b.smithi189.stdout:starting mds.b at 2024-09-17T02:47:14.940 INFO:tasks.ceph.mds.d.smithi189.stdout:starting mds.d at 2024-09-17T02:47:14.959 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:47:18.851 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-17T02:47:20.814 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T02:47:20.814 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T02:47:20.817 INFO:tasks.ceph.mds.c:Started 2024-09-17T02:47:20.818 DEBUG:teuthology.parallel:result is None 2024-09-17T02:47:20.818 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T02:47:20.922 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:47:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.973+0000 7fa04c786640 1 -- 172.21.15.131:0/3582308162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa030004000 msgr2=0x7fa030004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.973+0000 7fa04c786640 1 --2- 172.21.15.131:0/3582308162 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa030004000 0x7fa030004810 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fa040009b60 tx=0x7fa04001d2b0 comp rx=0 tx=0).stop 2024-09-17T02:47:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.973+0000 7fa04ad23640 1 -- 172.21.15.131:0/3582308162 reap_dead start 2024-09-17T02:47:20.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.973+0000 7fa04c786640 1 -- 172.21.15.131:0/3582308162 shutdown_connections 2024-09-17T02:47:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.973+0000 7fa04c786640 1 -- 172.21.15.131:0/3582308162 >> 172.21.15.131:0/3582308162 conn(0x7fa04406fbe0 msgr2=0x7fa04406fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04c786640 1 -- 172.21.15.131:0/3582308162 shutdown_connections 2024-09-17T02:47:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04c786640 1 -- 172.21.15.131:0/3582308162 wait complete. 2024-09-17T02:47:20.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04c786640 1 Processor -- start 2024-09-17T02:47:20.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04c786640 1 -- start start 2024-09-17T02:47:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04c786640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa030004000 0x7fa0440ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04c786640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0440c4dc0 0x7fa0440c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04c786640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa044153200 0x7fa0440c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04c786640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa044169760 con 0x7fa044153200 2024-09-17T02:47:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04c786640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0441695e0 con 0x7fa0440c4dc0 2024-09-17T02:47:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04c786640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa0441698e0 con 0x7fa030004000 2024-09-17T02:47:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04ad23640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa044153200 0x7fa0440c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04ad23640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa044153200 0x7fa0440c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54560/0 (socket says 172.21.15.131:54560) 2024-09-17T02:47:20.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04ad23640 1 -- 172.21.15.131:0/3041428169 learned_addr learned my addr 172.21.15.131:0/3041428169 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04a522640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa030004000 0x7fa0440ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04ad23640 1 -- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa030004000 msgr2=0x7fa0440ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04ad23640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa030004000 0x7fa0440ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa049d21640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0440c4dc0 0x7fa0440c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04ad23640 1 -- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0440c4dc0 msgr2=0x7fa0440c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04ad23640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0440c4dc0 0x7fa0440c5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04ad23640 1 -- 172.21.15.131:0/3041428169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0440c9b20 con 0x7fa044153200 2024-09-17T02:47:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.977+0000 7fa04ad23640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa044153200 0x7fa0440c80c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa0440d0ef0 tx=0x7fa03c00ee10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.981+0000 7fa03affd640 1 -- 172.21.15.131:0/3041428169 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c004b30 con 0x7fa044153200 2024-09-17T02:47:20.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.981+0000 7fa03affd640 1 -- 172.21.15.131:0/3041428169 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa03c004dc0 con 0x7fa044153200 2024-09-17T02:47:20.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.981+0000 7fa03affd640 1 -- 172.21.15.131:0/3041428169 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c00cd30 con 0x7fa044153200 2024-09-17T02:47:20.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.981+0000 7fa04c786640 1 -- 172.21.15.131:0/3041428169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0440c9e00 con 0x7fa044153200 2024-09-17T02:47:20.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.981+0000 7fa04c786640 1 -- 172.21.15.131:0/3041428169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0440c8b20 con 0x7fa044153200 2024-09-17T02:47:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.981+0000 7fa04c786640 1 -- 172.21.15.131:0/3041428169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0440cc0f0 con 0x7fa044153200 2024-09-17T02:47:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.981+0000 7fa03affd640 1 -- 172.21.15.131:0/3041428169 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa03c01b030 con 0x7fa044153200 2024-09-17T02:47:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.981+0000 7fa03affd640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa028073200 0x7fa028075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.981+0000 7fa03affd640 1 -- 172.21.15.131:0/3041428169 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5421+0+0 (secure 0 0 0) 0x7fa03c093c30 con 0x7fa044153200 2024-09-17T02:47:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.981+0000 7fa04a522640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa028073200 0x7fa028075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:20.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.985+0000 7fa04a522640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa028073200 0x7fa028075710 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa034001870 tx=0x7fa0340015e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:20.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:20.985+0000 7fa03affd640 1 -- 172.21.15.131:0/3041428169 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa03c022060 con 0x7fa044153200 2024-09-17T02:47:21.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:21.157+0000 7fa04c786640 1 -- 172.21.15.131:0/3041428169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa0440b14b0 con 0x7fa044153200 2024-09-17T02:47:22.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.329+0000 7fa03affd640 1 -- 172.21.15.131:0/3041428169 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) ==== 97+0+0 (secure 0 0 0) 0x7fa03c05ed60 con 0x7fa044153200 2024-09-17T02:47:22.329 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-09-17T02:47:22.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 -- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa028073200 msgr2=0x7fa028075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:22.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa028073200 0x7fa028075710 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa034001870 tx=0x7fa0340015e0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 -- 172.21.15.131:0/3041428169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa044153200 msgr2=0x7fa0440c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:22.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa044153200 0x7fa0440c80c0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa0440d0ef0 tx=0x7fa03c00ee10 comp rx=0 tx=0).stop 2024-09-17T02:47:22.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 -- 172.21.15.131:0/3041428169 shutdown_connections 2024-09-17T02:47:22.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa028073200 0x7fa028075710 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa044153200 0x7fa0440c80c0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa0440c4dc0 0x7fa0440c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 --2- 172.21.15.131:0/3041428169 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa030004000 0x7fa0440ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 -- 172.21.15.131:0/3041428169 >> 172.21.15.131:0/3041428169 conn(0x7fa04406fbe0 msgr2=0x7fa044071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:22.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 -- 172.21.15.131:0/3041428169 shutdown_connections 2024-09-17T02:47:22.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.337+0000 7fa04c786640 1 -- 172.21.15.131:0/3041428169 wait complete. 2024-09-17T02:47:22.357 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T02:47:22.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.517+0000 7fd91065e640 1 -- 172.21.15.131:0/324659319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd908155240 msgr2=0x7fd908073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:22.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.517+0000 7fd91065e640 1 --2- 172.21.15.131:0/324659319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd908155240 0x7fd908073430 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fd9080afaf0 tx=0x7fd8fc01c7d0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.517+0000 7fd91065e640 1 -- 172.21.15.131:0/324659319 shutdown_connections 2024-09-17T02:47:22.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.517+0000 7fd91065e640 1 --2- 172.21.15.131:0/324659319 >> v2:172.21.15.189:3300/0 conn(0x7fd908073970 0x7fd90806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.517+0000 7fd91065e640 1 --2- 172.21.15.131:0/324659319 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd908155240 0x7fd908073430 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.517+0000 7fd91065e640 1 -- 172.21.15.131:0/324659319 >> 172.21.15.131:0/324659319 conn(0x7fd90806eea0 msgr2=0x7fd90806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:22.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.517+0000 7fd91065e640 1 -- 172.21.15.131:0/324659319 shutdown_connections 2024-09-17T02:47:22.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.517+0000 7fd91065e640 1 -- 172.21.15.131:0/324659319 wait complete. 2024-09-17T02:47:22.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.517+0000 7fd91065e640 1 Processor -- start 2024-09-17T02:47:22.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.517+0000 7fd91065e640 1 -- start start 2024-09-17T02:47:22.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd91065e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd90806e3e0 0x7fd90814d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:22.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd91065e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd908073970 0x7fd90814f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:22.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd91065e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd908155240 0x7fd9081fc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:22.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd91065e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd908168e00 con 0x7fd908073970 2024-09-17T02:47:22.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd91065e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd908168c80 con 0x7fd90806e3e0 2024-09-17T02:47:22.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd91065e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd908168f80 con 0x7fd908155240 2024-09-17T02:47:22.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90dbf9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd908073970 0x7fd90814f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:22.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90dbf9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd908073970 0x7fd90814f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50308/0 (socket says 172.21.15.131:50308) 2024-09-17T02:47:22.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90dbf9640 1 -- 172.21.15.131:0/3871876811 learned_addr learned my addr 172.21.15.131:0/3871876811 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:22.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90e3fa640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd90806e3e0 0x7fd90814d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90dbf9640 1 -- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd908155240 msgr2=0x7fd9081fc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90dbf9640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd908155240 0x7fd9081fc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90dbf9640 1 -- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd90806e3e0 msgr2=0x7fd90814d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90dbf9640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd90806e3e0 0x7fd90814d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90dbf9640 1 -- 172.21.15.131:0/3871876811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90814db50 con 0x7fd908073970 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90e3fa640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd90806e3e0 0x7fd90814d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90dbf9640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd908073970 0x7fd90814f0a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd9040058d0 tx=0x7fd90400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd8faffd640 1 -- 172.21.15.131:0/3871876811 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904017070 con 0x7fd908073970 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd8faffd640 1 -- 172.21.15.131:0/3871876811 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd904009d20 con 0x7fd908073970 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd91065e640 1 -- 172.21.15.131:0/3871876811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90814fb60 con 0x7fd908073970 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd8faffd640 1 -- 172.21.15.131:0/3871876811 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9040106c0 con 0x7fd908073970 2024-09-17T02:47:22.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd91065e640 1 -- 172.21.15.131:0/3871876811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd908150120 con 0x7fd908073970 2024-09-17T02:47:22.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd91065e640 1 -- 172.21.15.131:0/3871876811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90814f8b0 con 0x7fd908073970 2024-09-17T02:47:22.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd8faffd640 1 -- 172.21.15.131:0/3871876811 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd90401a070 con 0x7fd908073970 2024-09-17T02:47:22.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd8faffd640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8ec073200 0x7fd8ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:22.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd8faffd640 1 -- 172.21.15.131:0/3871876811 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5249+0+0 (secure 0 0 0) 0x7fd904094200 con 0x7fd908073970 2024-09-17T02:47:22.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.521+0000 7fd90e3fa640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8ec073200 0x7fd8ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:22.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.525+0000 7fd90e3fa640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8ec073200 0x7fd8ec075710 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0af000 tx=0x7fd8fc01b010 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:22.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.529+0000 7fd8faffd640 1 -- 172.21.15.131:0/3871876811 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd90405f3e0 con 0x7fd908073970 2024-09-17T02:47:22.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 -- 172.21.15.131:0/3871876811 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd90806dea0 con 0x7fd908073970 2024-09-17T02:47:22.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd8faffd640 1 -- 172.21.15.131:0/3871876811 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v32) ==== 80+0+5304 (secure 0 0 0) 0x7fd90405ece0 con 0x7fd908073970 2024-09-17T02:47:22.754 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:22.754 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:47:22.754 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-09-17T02:47:22.754 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.754 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-09-17T02:47:22.754 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm4U1cHhAATu86VrwT2vMjL/cW2SICMw==", 2024-09-17T02:47:22.754 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:47:22.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:47:22.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmVpUSKxAA4l5zGHXg0VRasiyBI/j4ig==", 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.755 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmwHuCJRAAlOj+za8ZmqMI55PsShjKDw==", 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-09-17T02:47:22.756 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm9f+9MhAAS8z9jZFYon3nrmu9/4JMPg==", 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD16+hmV9xkBxAAT+IDf2J4awJfqYcx2xDoNA==", 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.757 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD46+hmOhJsGBAAdlA4BSdOPk8BySUTfo1ghg==", 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.758 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD76+hm3mSoKBAAQPoGFy816yWZQW6vPIK6bA==", 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-09-17T02:47:22.759 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD+6+hmpZaPNxAAp0EV6bn5SugDMwA4RnnJwg==", 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAF7OhmgDfVAxAALACK3bk+PK3UhG20b498HQ==", 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:47:22.760 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAI7Ohm6w44FBAA90GkelUAbiNjAwn+6IJWcg==", 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.761 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.762 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-09-17T02:47:22.762 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAL7Ohm6S6TIxAArISJW//rmEONG2bC/sfkXA==", 2024-09-17T02:47:22.762 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.762 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:47:22.762 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:47:22.762 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.762 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.762 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.762 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.762 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAO7OhmICwTMxAArAr7DhcO8A0FDSu7VEBehw==", 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA==", 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.763 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:47:22.764 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-09-17T02:47:22.764 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T02:47:22.764 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.764 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.764 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.764 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-09-17T02:47:22.764 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmduVoBRAAwr50jSXw6sCyo+5t+3iT7A==", 2024-09-17T02:47:22.764 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.764 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:47:22.764 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:47:22.764 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:47:22.765 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:47:22.765 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.765 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.765 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.765 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-09-17T02:47:22.765 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmBBCDABAAMw9581QyaOxJk5fs35o+ng==", 2024-09-17T02:47:22.765 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.765 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:47:22.765 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:47:22.765 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:47:22.765 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmgznRCRAA1Z8d32ADoSG+7RHtdPGmTQ==", 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.766 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmpUfvABAAzclxwjR4E+UxvtkbwVqDXw==", 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmnWmJGBAA6fzewrm8ELqUAz/MczTOGA==", 2024-09-17T02:47:22.767 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmrTq3ExAAuR8+2bItlpHqqFzlvPxASA==", 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:47:22.768 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.769 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:22.769 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-09-17T02:47:22.769 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:47:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 -- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8ec073200 msgr2=0x7fd8ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8ec073200 0x7fd8ec075710 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0af000 tx=0x7fd8fc01b010 comp rx=0 tx=0).stop 2024-09-17T02:47:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 -- 172.21.15.131:0/3871876811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd908073970 msgr2=0x7fd90814f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd908073970 0x7fd90814f0a0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fd9040058d0 tx=0x7fd90400dde0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 -- 172.21.15.131:0/3871876811 shutdown_connections 2024-09-17T02:47:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8ec073200 0x7fd8ec075710 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd908155240 0x7fd9081fc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd908073970 0x7fd90814f0a0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 --2- 172.21.15.131:0/3871876811 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd90806e3e0 0x7fd90814d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 -- 172.21.15.131:0/3871876811 >> 172.21.15.131:0/3871876811 conn(0x7fd90806eea0 msgr2=0x7fd90806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:22.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 -- 172.21.15.131:0/3871876811 shutdown_connections 2024-09-17T02:47:22.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.757+0000 7fd91065e640 1 -- 172.21.15.131:0/3871876811 wait complete. 2024-09-17T02:47:22.775 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T02:47:22.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.929+0000 7f8e9cba5640 1 -- 172.21.15.131:0/2472704885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e980ced00 msgr2=0x7f8e9806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:22.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.929+0000 7f8e9cba5640 1 --2- 172.21.15.131:0/2472704885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e980ced00 0x7f8e9806dea0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f8e88009900 tx=0x7f8e8801c700 comp rx=0 tx=0).stop 2024-09-17T02:47:22.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.929+0000 7f8e9cba5640 1 -- 172.21.15.131:0/2472704885 shutdown_connections 2024-09-17T02:47:22.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.929+0000 7f8e9cba5640 1 --2- 172.21.15.131:0/2472704885 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e9815c5b0 0x7f8e980cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.929+0000 7f8e9cba5640 1 --2- 172.21.15.131:0/2472704885 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e980ced00 0x7f8e9806dea0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.929+0000 7f8e9cba5640 1 -- 172.21.15.131:0/2472704885 >> 172.21.15.131:0/2472704885 conn(0x7f8e9806eea0 msgr2=0x7f8e9806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:22.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.929+0000 7f8e9cba5640 1 -- 172.21.15.131:0/2472704885 shutdown_connections 2024-09-17T02:47:22.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.929+0000 7f8e9cba5640 1 -- 172.21.15.131:0/2472704885 wait complete. 2024-09-17T02:47:22.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.929+0000 7f8e9cba5640 1 Processor -- start 2024-09-17T02:47:22.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.929+0000 7f8e9cba5640 1 -- start start 2024-09-17T02:47:22.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e9cba5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e9806e3e0 0x7f8e981f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:22.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e9cba5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e980ced00 0x7f8e981f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:22.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e9cba5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e9815c5b0 0x7f8e9815a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e9cba5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8e98168720 con 0x7f8e9815c5b0 2024-09-17T02:47:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e9cba5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8e981685a0 con 0x7f8e9806e3e0 2024-09-17T02:47:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e9cba5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8e981688a0 con 0x7f8e980ced00 2024-09-17T02:47:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e96d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e9815c5b0 0x7f8e9815a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e96d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e9815c5b0 0x7f8e9815a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50324/0 (socket says 172.21.15.131:50324) 2024-09-17T02:47:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e96d9d640 1 -- 172.21.15.131:0/4272585700 learned_addr learned my addr 172.21.15.131:0/4272585700 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e95d9b640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e980ced00 0x7f8e981f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e9659c640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e9806e3e0 0x7f8e981f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e96d9d640 1 -- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e980ced00 msgr2=0x7f8e981f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:22.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e96d9d640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e980ced00 0x7f8e981f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e96d9d640 1 -- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e9806e3e0 msgr2=0x7f8e981f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:22.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e96d9d640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e9806e3e0 0x7f8e981f93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:22.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e96d9d640 1 -- 172.21.15.131:0/4272585700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e981f98e0 con 0x7f8e9815c5b0 2024-09-17T02:47:22.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e96d9d640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e9815c5b0 0x7f8e9815a0b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000cab0 tx=0x7f8e9000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:22.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e86ffd640 1 -- 172.21.15.131:0/4272585700 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90007d90 con 0x7f8e9815c5b0 2024-09-17T02:47:22.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e86ffd640 1 -- 172.21.15.131:0/4272585700 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e9000f450 con 0x7f8e9815c5b0 2024-09-17T02:47:22.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e86ffd640 1 -- 172.21.15.131:0/4272585700 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9000f8c0 con 0x7f8e9815c5b0 2024-09-17T02:47:22.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e9cba5640 1 -- 172.21.15.131:0/4272585700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e98164bc0 con 0x7f8e9815c5b0 2024-09-17T02:47:22.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e9cba5640 1 -- 172.21.15.131:0/4272585700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e98165480 con 0x7f8e9815c5b0 2024-09-17T02:47:22.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e9cba5640 1 -- 172.21.15.131:0/4272585700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e9806dea0 con 0x7f8e9815c5b0 2024-09-17T02:47:22.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e86ffd640 1 -- 172.21.15.131:0/4272585700 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8e9000fa60 con 0x7f8e9815c5b0 2024-09-17T02:47:22.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e86ffd640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e78073200 0x7f8e78075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:22.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e86ffd640 1 -- 172.21.15.131:0/4272585700 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5249+0+0 (secure 0 0 0) 0x7f8e90094b50 con 0x7f8e9815c5b0 2024-09-17T02:47:22.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.933+0000 7f8e9659c640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e78073200 0x7f8e78075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:22.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.937+0000 7f8e9659c640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e78073200 0x7f8e78075710 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8e880ae000 tx=0x7f8e8801af40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:22.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:22.937+0000 7f8e86ffd640 1 -- 172.21.15.131:0/4272585700 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8e981f98e0 con 0x7f8e9815c5b0 2024-09-17T02:47:23.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.105+0000 7f8e9cba5640 1 -- 172.21.15.131:0/4272585700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8e980cbc90 con 0x7f8e9815c5b0 2024-09-17T02:47:23.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e86ffd640 1 -- 172.21.15.131:0/4272585700 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) ==== 84+0+65 (secure 0 0 0) 0x7f8e90014030 con 0x7f8e9815c5b0 2024-09-17T02:47:23.105 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:23.105 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:47:23.105 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:47:23.105 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-09-17T02:47:23.105 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-09-17T02:47:23.106 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:23.106 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:47:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 -- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e78073200 msgr2=0x7f8e78075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e78073200 0x7f8e78075710 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8e880ae000 tx=0x7f8e8801af40 comp rx=0 tx=0).stop 2024-09-17T02:47:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 -- 172.21.15.131:0/4272585700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e9815c5b0 msgr2=0x7f8e9815a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e9815c5b0 0x7f8e9815a0b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000cab0 tx=0x7f8e9000cf70 comp rx=0 tx=0).stop 2024-09-17T02:47:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 -- 172.21.15.131:0/4272585700 shutdown_connections 2024-09-17T02:47:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8e78073200 0x7f8e78075710 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8e9815c5b0 0x7f8e9815a0b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8e980ced00 0x7f8e981f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 --2- 172.21.15.131:0/4272585700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8e9806e3e0 0x7f8e981f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:23.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 -- 172.21.15.131:0/4272585700 >> 172.21.15.131:0/4272585700 conn(0x7f8e9806eea0 msgr2=0x7f8e9815d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:23.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 -- 172.21.15.131:0/4272585700 shutdown_connections 2024-09-17T02:47:23.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.109+0000 7f8e9cba5640 1 -- 172.21.15.131:0/4272585700 wait complete. 2024-09-17T02:47:23.126 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T02:47:23.126 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T02:47:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.289+0000 7f3fc528c640 1 -- 172.21.15.131:0/725508383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa8004810 msgr2=0x7f3fa8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.289+0000 7f3fc528c640 1 --2- 172.21.15.131:0/725508383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa8004810 0x7f3fa8004c20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400bb10 tx=0x7f3fb401cda0 comp rx=0 tx=0).stop 2024-09-17T02:47:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.289+0000 7f3fc528c640 1 -- 172.21.15.131:0/725508383 shutdown_connections 2024-09-17T02:47:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.289+0000 7f3fc528c640 1 --2- 172.21.15.131:0/725508383 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fa8004810 0x7f3fa8004c20 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.289+0000 7f3fc528c640 1 -- 172.21.15.131:0/725508383 >> 172.21.15.131:0/725508383 conn(0x7f3fc00734f0 msgr2=0x7f3fc0073900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.289+0000 7f3fc528c640 1 -- 172.21.15.131:0/725508383 shutdown_connections 2024-09-17T02:47:23.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.289+0000 7f3fc528c640 1 -- 172.21.15.131:0/725508383 wait complete. 2024-09-17T02:47:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fc528c640 1 Processor -- start 2024-09-17T02:47:23.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fc528c640 1 -- start start 2024-09-17T02:47:23.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fc528c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3fa8004810 0x7f3fc0148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:23.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fc528c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3fc015e8a0 0x7f3fc0160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:23.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fc528c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fc0160780 0x7f3fc01f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:23.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fc528c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc00cb1b0 con 0x7f3fc0160780 2024-09-17T02:47:23.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fc528c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc00cb030 con 0x7f3fc015e8a0 2024-09-17T02:47:23.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fc528c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc00cb330 con 0x7f3fa8004810 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fc0160780 0x7f3fc01f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fc0160780 0x7f3fc01f38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50334/0 (socket says 172.21.15.131:50334) 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbf7fe640 1 -- 172.21.15.131:0/1581207774 learned_addr learned my addr 172.21.15.131:0/1581207774 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbe7fc640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3fc015e8a0 0x7f3fc0160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbf7fe640 1 -- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3fa8004810 msgr2=0x7f3fc0148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbeffd640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3fa8004810 0x7f3fc0148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbf7fe640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3fa8004810 0x7f3fc0148350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbf7fe640 1 -- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3fc015e8a0 msgr2=0x7f3fc0160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbf7fe640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3fc015e8a0 0x7f3fc0160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbf7fe640 1 -- 172.21.15.131:0/1581207774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc0160b90 con 0x7f3fc0160780 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbe7fc640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3fc015e8a0 0x7f3fc0160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fbf7fe640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fc0160780 0x7f3fc01f38c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0051ff0 tx=0x7f3fb8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fa77fe640 1 -- 172.21.15.131:0/1581207774 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8017070 con 0x7f3fc0160780 2024-09-17T02:47:23.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fa77fe640 1 -- 172.21.15.131:0/1581207774 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fb8013af0 con 0x7f3fc0160780 2024-09-17T02:47:23.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fc528c640 1 -- 172.21.15.131:0/1581207774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc0151600 con 0x7f3fc0160780 2024-09-17T02:47:23.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fa77fe640 1 -- 172.21.15.131:0/1581207774 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8012440 con 0x7f3fc0160780 2024-09-17T02:47:23.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.293+0000 7f3fc528c640 1 -- 172.21.15.131:0/1581207774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc01f7120 con 0x7f3fc0160780 2024-09-17T02:47:23.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.297+0000 7f3fc528c640 1 -- 172.21.15.131:0/1581207774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc00c69a0 con 0x7f3fc0160780 2024-09-17T02:47:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.297+0000 7f3fa77fe640 1 -- 172.21.15.131:0/1581207774 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3fb80125e0 con 0x7f3fc0160780 2024-09-17T02:47:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.297+0000 7f3fa77fe640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fac073230 0x7f3fac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.297+0000 7f3fa77fe640 1 -- 172.21.15.131:0/1581207774 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5249+0+0 (secure 0 0 0) 0x7f3fb8097d20 con 0x7f3fc0160780 2024-09-17T02:47:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.297+0000 7f3fbeffd640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fac073230 0x7f3fac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:23.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.297+0000 7f3fbeffd640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fac073230 0x7f3fac075740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00069a0 tx=0x7f3fb0006910 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:23.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.301+0000 7f3fa77fe640 1 -- 172.21.15.131:0/1581207774 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3fc0160b90 con 0x7f3fc0160780 2024-09-17T02:47:23.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:23.469+0000 7f3fc528c640 1 -- 172.21.15.131:0/1581207774 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3fc0159210 con 0x7f3fc0160780 2024-09-17T02:47:24.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.361+0000 7f3fa77fe640 1 -- 172.21.15.131:0/1581207774 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) ==== 138+0+0 (secure 0 0 0) 0x7f3fb8063020 con 0x7f3fc0160780 2024-09-17T02:47:24.358 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-09-17T02:47:24.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 -- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fac073230 msgr2=0x7f3fac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:24.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fac073230 0x7f3fac075740 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00069a0 tx=0x7f3fb0006910 comp rx=0 tx=0).stop 2024-09-17T02:47:24.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 -- 172.21.15.131:0/1581207774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fc0160780 msgr2=0x7f3fc01f38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:24.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fc0160780 0x7f3fc01f38c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0051ff0 tx=0x7f3fb8011ab0 comp rx=0 tx=0).stop 2024-09-17T02:47:24.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 -- 172.21.15.131:0/1581207774 shutdown_connections 2024-09-17T02:47:24.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fac073230 0x7f3fac075740 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:24.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fc0160780 0x7f3fc01f38c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:24.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3fc015e8a0 0x7f3fc0160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:24.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 --2- 172.21.15.131:0/1581207774 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3fa8004810 0x7f3fc0148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:24.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 -- 172.21.15.131:0/1581207774 >> 172.21.15.131:0/1581207774 conn(0x7f3fc00734f0 msgr2=0x7f3fc015fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:24.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 -- 172.21.15.131:0/1581207774 shutdown_connections 2024-09-17T02:47:24.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.365+0000 7f3fc528c640 1 -- 172.21.15.131:0/1581207774 wait complete. 2024-09-17T02:47:24.390 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T02:47:24.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.557+0000 7f052ada9640 1 -- 172.21.15.131:0/322287181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f052415b870 msgr2=0x7f052415bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:24.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.557+0000 7f052ada9640 1 --2- 172.21.15.131:0/322287181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f052415b870 0x7f052415bc80 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f05240d08f0 tx=0x7f051c01ccf0 comp rx=0 tx=0).stop 2024-09-17T02:47:24.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.557+0000 7f052ada9640 1 -- 172.21.15.131:0/322287181 shutdown_connections 2024-09-17T02:47:24.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.557+0000 7f052ada9640 1 --2- 172.21.15.131:0/322287181 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f052415b870 0x7f052415bc80 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:24.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.557+0000 7f052ada9640 1 --2- 172.21.15.131:0/322287181 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f052415ee60 0x7f052415f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:24.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.557+0000 7f052ada9640 1 --2- 172.21.15.131:0/322287181 >> v2:172.21.15.189:3300/0 conn(0x7f0524152300 0x7f05240748c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:24.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.557+0000 7f052ada9640 1 -- 172.21.15.131:0/322287181 >> 172.21.15.131:0/322287181 conn(0x7f05240d0490 msgr2=0x7f052406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:24.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.557+0000 7f052ada9640 1 -- 172.21.15.131:0/322287181 shutdown_connections 2024-09-17T02:47:24.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.561+0000 7f052ada9640 1 -- 172.21.15.131:0/322287181 wait complete. 2024-09-17T02:47:24.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.561+0000 7f052ada9640 1 Processor -- start 2024-09-17T02:47:24.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.561+0000 7f052ada9640 1 -- start start 2024-09-17T02:47:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.561+0000 7f052ada9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0524152300 0x7f0524143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f052ada9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f052415b870 0x7f05241581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f052ada9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f052415ee60 0x7f0524158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f052ada9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f05240701c0 con 0x7f0524152300 2024-09-17T02:47:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f052ada9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0524070040 con 0x7f052415ee60 2024-09-17T02:47:24.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f052ada9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0524070340 con 0x7f052415b870 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f0528b45640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0524152300 0x7f0524143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f0528b45640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0524152300 0x7f0524143090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50360/0 (socket says 172.21.15.131:50360) 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f0528b45640 1 -- 172.21.15.131:0/3185209325 learned_addr learned my addr 172.21.15.131:0/3185209325 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f0523fff640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f052415b870 0x7f05241581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f0529346640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f052415ee60 0x7f0524158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f0528b45640 1 -- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f052415b870 msgr2=0x7f05241581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f0528b45640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f052415b870 0x7f05241581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f0528b45640 1 -- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f052415ee60 msgr2=0x7f0524158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f0528b45640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f052415ee60 0x7f0524158730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f0528b45640 1 -- 172.21.15.131:0/3185209325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0524159270 con 0x7f0524152300 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f0528b45640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0524152300 0x7f0524143090 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f051400d6d0 tx=0x7f051400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f05217fa640 1 -- 172.21.15.131:0/3185209325 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0514017070 con 0x7f0524152300 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f05217fa640 1 -- 172.21.15.131:0/3185209325 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0514009ab0 con 0x7f0524152300 2024-09-17T02:47:24.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f052ada9640 1 -- 172.21.15.131:0/3185209325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05241435d0 con 0x7f0524152300 2024-09-17T02:47:24.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f05217fa640 1 -- 172.21.15.131:0/3185209325 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0514010460 con 0x7f0524152300 2024-09-17T02:47:24.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f052ada9640 1 -- 172.21.15.131:0/3185209325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0524157710 con 0x7f0524152300 2024-09-17T02:47:24.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f052ada9640 1 -- 172.21.15.131:0/3185209325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f052415bc80 con 0x7f0524152300 2024-09-17T02:47:24.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f05217fa640 1 -- 172.21.15.131:0/3185209325 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f051401a070 con 0x7f0524152300 2024-09-17T02:47:24.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.565+0000 7f05217fa640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0510073230 0x7f0510075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:24.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.569+0000 7f05217fa640 1 -- 172.21.15.131:0/3185209325 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5619+0+0 (secure 0 0 0) 0x7f0514094230 con 0x7f0524152300 2024-09-17T02:47:24.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.569+0000 7f0523fff640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0510073230 0x7f0510075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:24.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.569+0000 7f0523fff640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0510073230 0x7f0510075740 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f05180058d0 tx=0x7f0518005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:24.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.573+0000 7f05217fa640 1 -- 172.21.15.131:0/3185209325 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f051405f2a0 con 0x7f0524152300 2024-09-17T02:47:24.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:24.745+0000 7f052ada9640 1 -- 172.21.15.131:0/3185209325 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0524157500 con 0x7f0524152300 2024-09-17T02:47:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.373+0000 7f05217fa640 1 -- 172.21.15.131:0/3185209325 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) ==== 170+0+0 (secure 0 0 0) 0x7f051405eba0 con 0x7f0524152300 2024-09-17T02:47:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-09-17T02:47:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 -- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0510073230 msgr2=0x7f0510075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0510073230 0x7f0510075740 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f05180058d0 tx=0x7f0518005860 comp rx=0 tx=0).stop 2024-09-17T02:47:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 -- 172.21.15.131:0/3185209325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0524152300 msgr2=0x7f0524143090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:25.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0524152300 0x7f0524143090 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f051400d6d0 tx=0x7f051400db90 comp rx=0 tx=0).stop 2024-09-17T02:47:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 -- 172.21.15.131:0/3185209325 shutdown_connections 2024-09-17T02:47:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0510073230 0x7f0510075740 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f052415ee60 0x7f0524158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f052415b870 0x7f05241581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 --2- 172.21.15.131:0/3185209325 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0524152300 0x7f0524143090 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 -- 172.21.15.131:0/3185209325 >> 172.21.15.131:0/3185209325 conn(0x7f05240d0490 msgr2=0x7f052415ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 -- 172.21.15.131:0/3185209325 shutdown_connections 2024-09-17T02:47:25.376 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.377+0000 7f052ada9640 1 -- 172.21.15.131:0/3185209325 wait complete. 2024-09-17T02:47:25.416 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T02:47:25.564 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-17T02:47:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.573+0000 7fb51c607640 1 -- 172.21.15.131:0/989628352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5140cf710 msgr2=0x7fb51415f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.573+0000 7fb51c607640 1 --2- 172.21.15.131:0/989628352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5140cf710 0x7fb51415f290 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fb5140ce990 tx=0x7fb51001c8a0 comp rx=0 tx=0).stop 2024-09-17T02:47:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.573+0000 7fb51c607640 1 -- 172.21.15.131:0/989628352 shutdown_connections 2024-09-17T02:47:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.573+0000 7fb51c607640 1 --2- 172.21.15.131:0/989628352 >> v2:172.21.15.189:3300/0 conn(0x7fb514162950 0x7fb514162d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:25.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.573+0000 7fb51c607640 1 --2- 172.21.15.131:0/989628352 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb51415f7d0 0x7fb514162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:25.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.573+0000 7fb51c607640 1 --2- 172.21.15.131:0/989628352 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb5140cf710 0x7fb51415f290 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:25.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.573+0000 7fb51c607640 1 -- 172.21.15.131:0/989628352 >> 172.21.15.131:0/989628352 conn(0x7fb5140d1b10 msgr2=0x7fb5140d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:25.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.573+0000 7fb51c607640 1 -- 172.21.15.131:0/989628352 shutdown_connections 2024-09-17T02:47:25.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.573+0000 7fb51c607640 1 -- 172.21.15.131:0/989628352 wait complete. 2024-09-17T02:47:25.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51c607640 1 Processor -- start 2024-09-17T02:47:25.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51c607640 1 -- start start 2024-09-17T02:47:25.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51c607640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb5140cf710 0x7fb51415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:25.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51c607640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb51415f7d0 0x7fb514144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:25.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51c607640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb514162950 0x7fb5141588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:25.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51c607640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140d2df0 con 0x7fb51415f7d0 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51c607640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb5140d2c70 con 0x7fb514162950 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51c607640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb5140d2f70 con 0x7fb5140cf710 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb519ba2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb51415f7d0 0x7fb514144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb519ba2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb51415f7d0 0x7fb514144770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50376/0 (socket says 172.21.15.131:50376) 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb519ba2640 1 -- 172.21.15.131:0/3220116918 learned_addr learned my addr 172.21.15.131:0/3220116918 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51a3a3640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb5140cf710 0x7fb51415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51aba4640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb514162950 0x7fb5141588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb519ba2640 1 -- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb5140cf710 msgr2=0x7fb51415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb519ba2640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb5140cf710 0x7fb51415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb519ba2640 1 -- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb514162950 msgr2=0x7fb5141588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb519ba2640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb514162950 0x7fb5141588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb519ba2640 1 -- 172.21.15.131:0/3220116918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51415b0a0 con 0x7fb51415f7d0 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb519ba2640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb51415f7d0 0x7fb514144770 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb50400d700 tx=0x7fb50400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:25.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb50affd640 1 -- 172.21.15.131:0/3220116918 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504017070 con 0x7fb51415f7d0 2024-09-17T02:47:25.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb50affd640 1 -- 172.21.15.131:0/3220116918 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb504009b10 con 0x7fb51415f7d0 2024-09-17T02:47:25.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51c607640 1 -- 172.21.15.131:0/3220116918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51415b340 con 0x7fb51415f7d0 2024-09-17T02:47:25.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb50affd640 1 -- 172.21.15.131:0/3220116918 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb504010460 con 0x7fb51415f7d0 2024-09-17T02:47:25.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51c607640 1 -- 172.21.15.131:0/3220116918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb51415b900 con 0x7fb51415f7d0 2024-09-17T02:47:25.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.577+0000 7fb51c607640 1 -- 172.21.15.131:0/3220116918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5140d3440 con 0x7fb51415f7d0 2024-09-17T02:47:25.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.581+0000 7fb50affd640 1 -- 172.21.15.131:0/3220116918 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb50401a070 con 0x7fb51415f7d0 2024-09-17T02:47:25.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.581+0000 7fb50affd640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb4f8073200 0x7fb4f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:25.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.581+0000 7fb50affd640 1 -- 172.21.15.131:0/3220116918 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6001+0+0 (secure 0 0 0) 0x7fb5040947f0 con 0x7fb51415f7d0 2024-09-17T02:47:25.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.581+0000 7fb51a3a3640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb4f8073200 0x7fb4f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:25.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.581+0000 7fb51a3a3640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb4f8073200 0x7fb4f8075710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb5100af000 tx=0x7fb51001b0e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:25.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.585+0000 7fb50affd640 1 -- 172.21.15.131:0/3220116918 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb50405f7c0 con 0x7fb51415f7d0 2024-09-17T02:47:25.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:25.793+0000 7fb51c607640 1 -- 172.21.15.131:0/3220116918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb5140d38a0 con 0x7fb51415f7d0 2024-09-17T02:47:26.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.401+0000 7fb50affd640 1 -- 172.21.15.131:0/3220116918 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v60) ==== 174+0+0 (secure 0 0 0) 0x7fb50405f0c0 con 0x7fb51415f7d0 2024-09-17T02:47:26.398 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 13 and data pool 14 2024-09-17T02:47:26.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.409+0000 7fb51c607640 1 -- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb4f8073200 msgr2=0x7fb4f8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:26.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.409+0000 7fb51c607640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb4f8073200 0x7fb4f8075710 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb5100af000 tx=0x7fb51001b0e0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.409+0000 7fb51c607640 1 -- 172.21.15.131:0/3220116918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb51415f7d0 msgr2=0x7fb514144770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:26.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.409+0000 7fb51c607640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb51415f7d0 0x7fb514144770 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb50400d700 tx=0x7fb50400dbc0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.409+0000 7fb51c607640 1 -- 172.21.15.131:0/3220116918 shutdown_connections 2024-09-17T02:47:26.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.409+0000 7fb51c607640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb4f8073200 0x7fb4f8075710 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb5100af000 tx=0x7fb51001b0e0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.409+0000 7fb51c607640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb514162950 0x7fb5141588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.409+0000 7fb51c607640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb51415f7d0 0x7fb514144770 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.409+0000 7fb51c607640 1 --2- 172.21.15.131:0/3220116918 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb5140cf710 0x7fb51415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.409+0000 7fb51c607640 1 -- 172.21.15.131:0/3220116918 >> 172.21.15.131:0/3220116918 conn(0x7fb5140d1b10 msgr2=0x7fb5141617f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:26.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.413+0000 7fb51c607640 1 -- 172.21.15.131:0/3220116918 shutdown_connections 2024-09-17T02:47:26.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.413+0000 7fb51c607640 1 -- 172.21.15.131:0/3220116918 wait complete. 2024-09-17T02:47:26.446 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:47:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.597+0000 7f24cca94640 1 -- 172.21.15.131:0/3278879969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24c806e2c0 msgr2=0x7f24c80c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.597+0000 7f24cca94640 1 --2- 172.21.15.131:0/3278879969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24c806e2c0 0x7f24c80c4580 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f24c806a1c0 tx=0x7f24b801c830 comp rx=0 tx=0).stop 2024-09-17T02:47:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.597+0000 7f24cca94640 1 -- 172.21.15.131:0/3278879969 shutdown_connections 2024-09-17T02:47:26.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.597+0000 7f24cca94640 1 --2- 172.21.15.131:0/3278879969 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f24c80cec00 0x7f24c80cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.597+0000 7f24cca94640 1 --2- 172.21.15.131:0/3278879969 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24c806e2c0 0x7f24c80c4580 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.597+0000 7f24cca94640 1 -- 172.21.15.131:0/3278879969 >> 172.21.15.131:0/3278879969 conn(0x7f24c806eea0 msgr2=0x7f24c806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.597+0000 7f24cca94640 1 -- 172.21.15.131:0/3278879969 shutdown_connections 2024-09-17T02:47:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.597+0000 7f24cca94640 1 -- 172.21.15.131:0/3278879969 wait complete. 2024-09-17T02:47:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.597+0000 7f24cca94640 1 Processor -- start 2024-09-17T02:47:26.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.597+0000 7f24cca94640 1 -- start start 2024-09-17T02:47:26.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24cca94640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24c806e2c0 0x7f24c81f8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:26.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24cca94640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f24c80c4ac0 0x7f24c8153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:26.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24cca94640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24c80cec00 0x7f24c81539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:26.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24cca94640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8169c60 con 0x7f24c80cec00 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24cca94640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f24c8169ae0 con 0x7f24c806e2c0 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24cca94640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f24c8169de0 con 0x7f24c80c4ac0 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24c6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24c80cec00 0x7f24c81539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24c6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24c80cec00 0x7f24c81539a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50390/0 (socket says 172.21.15.131:50390) 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24c6d9d640 1 -- 172.21.15.131:0/318406529 learned_addr learned my addr 172.21.15.131:0/318406529 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24c659c640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24c806e2c0 0x7f24c81f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24bdd9b640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f24c80c4ac0 0x7f24c8153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24c6d9d640 1 -- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f24c80c4ac0 msgr2=0x7f24c8153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24c6d9d640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f24c80c4ac0 0x7f24c8153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24c6d9d640 1 -- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24c806e2c0 msgr2=0x7f24c81f8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24c6d9d640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24c806e2c0 0x7f24c81f8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24c6d9d640 1 -- 172.21.15.131:0/318406529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c81f8c40 con 0x7f24c80cec00 2024-09-17T02:47:26.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24c6d9d640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24c80cec00 0x7f24c81539a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f24c000cd10 tx=0x7f24c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:26.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24bf7fe640 1 -- 172.21.15.131:0/318406529 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c000f450 con 0x7f24c80cec00 2024-09-17T02:47:26.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24bf7fe640 1 -- 172.21.15.131:0/318406529 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f24c000f700 con 0x7f24c80cec00 2024-09-17T02:47:26.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24cca94640 1 -- 172.21.15.131:0/318406529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c81fcf80 con 0x7f24c80cec00 2024-09-17T02:47:26.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24bf7fe640 1 -- 172.21.15.131:0/318406529 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c000fb50 con 0x7f24c80cec00 2024-09-17T02:47:26.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24cca94640 1 -- 172.21.15.131:0/318406529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c8165f40 con 0x7f24c80cec00 2024-09-17T02:47:26.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.601+0000 7f24cca94640 1 -- 172.21.15.131:0/318406529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c80c4580 con 0x7f24c80cec00 2024-09-17T02:47:26.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.605+0000 7f24bf7fe640 1 -- 172.21.15.131:0/318406529 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f24c000fcf0 con 0x7f24c80cec00 2024-09-17T02:47:26.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.605+0000 7f24bf7fe640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24a8073200 0x7f24a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:26.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.605+0000 7f24bf7fe640 1 -- 172.21.15.131:0/318406529 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f24c0094e50 con 0x7f24c80cec00 2024-09-17T02:47:26.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.605+0000 7f24c659c640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24a8073200 0x7f24a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:26.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.605+0000 7f24c659c640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24a8073200 0x7f24a8075710 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f24b80ae000 tx=0x7f24b801b070 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:26.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.609+0000 7f24bf7fe640 1 -- 172.21.15.131:0/318406529 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f24c0014030 con 0x7f24c80cec00 2024-09-17T02:47:26.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.777+0000 7f24cca94640 1 -- 172.21.15.131:0/318406529 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24c8073310 con 0x7f24c80cec00 2024-09-17T02:47:26.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24bf7fe640 1 -- 172.21.15.131:0/318406529 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+17054 (secure 0 0 0) 0x7f24c005fd70 con 0x7f24c80cec00 2024-09-17T02:47:26.778 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:26.778 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":91,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:47:25.797806+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:47:23.485820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-17T02:47:24.759592+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:47:26.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24cca94640 1 -- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24a8073200 msgr2=0x7f24a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:26.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24cca94640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24a8073200 0x7f24a8075710 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f24b80ae000 tx=0x7f24b801b070 comp rx=0 tx=0).stop 2024-09-17T02:47:26.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24cca94640 1 -- 172.21.15.131:0/318406529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24c80cec00 msgr2=0x7f24c81539a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:26.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24cca94640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24c80cec00 0x7f24c81539a0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f24c000cd10 tx=0x7f24c000a620 comp rx=0 tx=0).stop 2024-09-17T02:47:26.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24cca94640 1 -- 172.21.15.131:0/318406529 shutdown_connections 2024-09-17T02:47:26.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24cca94640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f24a8073200 0x7f24a8075710 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24cca94640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f24c80cec00 0x7f24c81539a0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24cca94640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f24c80c4ac0 0x7f24c8153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24cca94640 1 --2- 172.21.15.131:0/318406529 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f24c806e2c0 0x7f24c81f8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24cca94640 1 -- 172.21.15.131:0/318406529 >> 172.21.15.131:0/318406529 conn(0x7f24c806eea0 msgr2=0x7f24c8156120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:26.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.781+0000 7f24cca94640 1 -- 172.21.15.131:0/318406529 shutdown_connections 2024-09-17T02:47:26.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.785+0000 7f24cca94640 1 -- 172.21.15.131:0/318406529 wait complete. 2024-09-17T02:47:26.802 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:47:26.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.953+0000 7f3a850ee640 1 -- 172.21.15.131:0/42283755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8015f7d0 msgr2=0x7f3a80162410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:26.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.953+0000 7f3a850ee640 1 --2- 172.21.15.131:0/42283755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8015f7d0 0x7f3a80162410 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009900 tx=0x7f3a7401c8e0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.953+0000 7f3a850ee640 1 -- 172.21.15.131:0/42283755 shutdown_connections 2024-09-17T02:47:26.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.953+0000 7f3a850ee640 1 --2- 172.21.15.131:0/42283755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a8015f7d0 0x7f3a80162410 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.953+0000 7f3a850ee640 1 --2- 172.21.15.131:0/42283755 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a800cf710 0x7f3a8015f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.953+0000 7f3a850ee640 1 -- 172.21.15.131:0/42283755 >> 172.21.15.131:0/42283755 conn(0x7f3a800d1b10 msgr2=0x7f3a800d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:26.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.953+0000 7f3a850ee640 1 -- 172.21.15.131:0/42283755 shutdown_connections 2024-09-17T02:47:26.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.953+0000 7f3a850ee640 1 -- 172.21.15.131:0/42283755 wait complete. 2024-09-17T02:47:26.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.953+0000 7f3a850ee640 1 Processor -- start 2024-09-17T02:47:26.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.953+0000 7f3a850ee640 1 -- start start 2024-09-17T02:47:26.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a850ee640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a800cf710 0x7f3a8014cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:26.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a850ee640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a8015f7d0 0x7f3a80155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:26.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a850ee640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a80162950 0x7f3a801563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:26.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a850ee640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a800d3500 con 0x7f3a80162950 2024-09-17T02:47:26.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a850ee640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a800d3380 con 0x7f3a800cf710 2024-09-17T02:47:26.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a850ee640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3a800d3680 con 0x7f3a8015f7d0 2024-09-17T02:47:26.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7f59e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a80162950 0x7f3a801563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:26.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7f59e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a80162950 0x7f3a801563e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50404/0 (socket says 172.21.15.131:50404) 2024-09-17T02:47:26.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7f59e640 1 -- 172.21.15.131:0/3596877938 learned_addr learned my addr 172.21.15.131:0/3596877938 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:26.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7ed9d640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a800cf710 0x7f3a8014cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7e59c640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a8015f7d0 0x7f3a80155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7f59e640 1 -- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a8015f7d0 msgr2=0x7f3a80155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7f59e640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a8015f7d0 0x7f3a80155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7f59e640 1 -- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a800cf710 msgr2=0x7f3a8014cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7f59e640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a800cf710 0x7f3a8014cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7f59e640 1 -- 172.21.15.131:0/3596877938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8014d4e0 con 0x7f3a80162950 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7e59c640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a8015f7d0 0x7f3a80155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a7f59e640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a80162950 0x7f3a801563e0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800b970 tx=0x7f3a7800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a677fe640 1 -- 172.21.15.131:0/3596877938 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78019070 con 0x7f3a80162950 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a677fe640 1 -- 172.21.15.131:0/3596877938 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a7800cc30 con 0x7f3a80162950 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a850ee640 1 -- 172.21.15.131:0/3596877938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a801fce80 con 0x7f3a80162950 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a677fe640 1 -- 172.21.15.131:0/3596877938 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7800ce60 con 0x7f3a80162950 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a850ee640 1 -- 172.21.15.131:0/3596877938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a801fd440 con 0x7f3a80162950 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a850ee640 1 -- 172.21.15.131:0/3596877938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8015f290 con 0x7f3a80162950 2024-09-17T02:47:26.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.957+0000 7f3a677fe640 1 -- 172.21.15.131:0/3596877938 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3a7801c070 con 0x7f3a80162950 2024-09-17T02:47:26.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.961+0000 7f3a677fe640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a6c073230 0x7f3a6c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:26.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.961+0000 7f3a677fe640 1 -- 172.21.15.131:0/3596877938 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f3a78093f30 con 0x7f3a80162950 2024-09-17T02:47:26.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.961+0000 7f3a7ed9d640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a6c073230 0x7f3a6c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:26.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.961+0000 7f3a7ed9d640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a6c073230 0x7f3a6c075740 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3a700058d0 tx=0x7f3a70005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:26.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:26.965+0000 7f3a677fe640 1 -- 172.21.15.131:0/3596877938 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3a7805ee50 con 0x7f3a80162950 2024-09-17T02:47:27.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.137+0000 7f3a850ee640 1 -- 172.21.15.131:0/3596877938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a80160e50 con 0x7f3a80162950 2024-09-17T02:47:27.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.137+0000 7f3a677fe640 1 -- 172.21.15.131:0/3596877938 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+17054 (secure 0 0 0) 0x7f3a7805e750 con 0x7f3a80162950 2024-09-17T02:47:27.137 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:27.137 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":91,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:47:25.797806+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:47:23.485820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-17T02:47:24.759592+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:47:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 -- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a6c073230 msgr2=0x7f3a6c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a6c073230 0x7f3a6c075740 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3a700058d0 tx=0x7f3a70005860 comp rx=0 tx=0).stop 2024-09-17T02:47:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 -- 172.21.15.131:0/3596877938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a80162950 msgr2=0x7f3a801563e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a80162950 0x7f3a801563e0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800b970 tx=0x7f3a7800be30 comp rx=0 tx=0).stop 2024-09-17T02:47:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 -- 172.21.15.131:0/3596877938 shutdown_connections 2024-09-17T02:47:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a6c073230 0x7f3a6c075740 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a80162950 0x7f3a801563e0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:27.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a8015f7d0 0x7f3a80155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 --2- 172.21.15.131:0/3596877938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a800cf710 0x7f3a8014cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 -- 172.21.15.131:0/3596877938 >> 172.21.15.131:0/3596877938 conn(0x7f3a800d1b10 msgr2=0x7f3a8015ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 -- 172.21.15.131:0/3596877938 shutdown_connections 2024-09-17T02:47:27.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.141+0000 7f3a850ee640 1 -- 172.21.15.131:0/3596877938 wait complete. 2024-09-17T02:47:27.161 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T02:47:27.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.309+0000 7f3b10ae0640 1 -- 172.21.15.131:0/2941317549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b0c0c4ac0 msgr2=0x7f3b0c0c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:27.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.309+0000 7f3b10ae0640 1 --2- 172.21.15.131:0/2941317549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b0c0c4ac0 0x7f3b0c0c4ed0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0727f0 tx=0x7f3af401c830 comp rx=0 tx=0).stop 2024-09-17T02:47:27.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.309+0000 7f3b10ae0640 1 -- 172.21.15.131:0/2941317549 shutdown_connections 2024-09-17T02:47:27.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.309+0000 7f3b10ae0640 1 --2- 172.21.15.131:0/2941317549 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b0c0c4ac0 0x7f3b0c0c4ed0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.309+0000 7f3b10ae0640 1 --2- 172.21.15.131:0/2941317549 >> v2:172.21.15.189:3300/0 conn(0x7f3b0c0cdbf0 0x7f3b0c0ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.309+0000 7f3b10ae0640 1 --2- 172.21.15.131:0/2941317549 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3b0c06e2c0 0x7f3b0c0c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.309+0000 7f3b10ae0640 1 -- 172.21.15.131:0/2941317549 >> 172.21.15.131:0/2941317549 conn(0x7f3b0c06eea0 msgr2=0x7f3b0c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.309+0000 7f3b10ae0640 1 -- 172.21.15.131:0/2941317549 shutdown_connections 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.309+0000 7f3b10ae0640 1 -- 172.21.15.131:0/2941317549 wait complete. 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b10ae0640 1 Processor -- start 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b10ae0640 1 -- start start 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b10ae0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b0c06e2c0 0x7f3b0c0c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b10ae0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b0c0c4ac0 0x7f3b0c154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b10ae0640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3b0c0cdbf0 0x7f3b0c1547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b10ae0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c0ceed0 con 0x7f3b0c06e2c0 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b10ae0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c0ced50 con 0x7f3b0c0c4ac0 2024-09-17T02:47:27.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b10ae0640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0c0cf050 con 0x7f3b0c0cdbf0 2024-09-17T02:47:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b0a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b0c06e2c0 0x7f3b0c0c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b0a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b0c06e2c0 0x7f3b0c0c9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50434/0 (socket says 172.21.15.131:50434) 2024-09-17T02:47:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b0a59c640 1 -- 172.21.15.131:0/1528097660 learned_addr learned my addr 172.21.15.131:0/1528097660 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b03fff640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b0c0c4ac0 0x7f3b0c154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b0ad9d640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3b0c0cdbf0 0x7f3b0c1547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b0a59c640 1 -- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3b0c0cdbf0 msgr2=0x7f3b0c1547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b0a59c640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3b0c0cdbf0 0x7f3b0c1547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b0a59c640 1 -- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b0c0c4ac0 msgr2=0x7f3b0c154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b0a59c640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b0c0c4ac0 0x7f3b0c154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b0a59c640 1 -- 172.21.15.131:0/1528097660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c156600 con 0x7f3b0c06e2c0 2024-09-17T02:47:27.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b03fff640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b0c0c4ac0 0x7f3b0c154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b0a59c640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b0c06e2c0 0x7f3b0c0c9ab0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00d660 tx=0x7f3afc00db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1528097660 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc017070 con 0x7f3b0c06e2c0 2024-09-17T02:47:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1528097660 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3afc005310 con 0x7f3b0c06e2c0 2024-09-17T02:47:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b10ae0640 1 -- 172.21.15.131:0/1528097660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0c156b00 con 0x7f3b0c06e2c0 2024-09-17T02:47:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b10ae0640 1 -- 172.21.15.131:0/1528097660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0c1570c0 con 0x7f3b0c06e2c0 2024-09-17T02:47:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.313+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1528097660 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc009820 con 0x7f3b0c06e2c0 2024-09-17T02:47:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.317+0000 7f3b10ae0640 1 -- 172.21.15.131:0/1528097660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b0c0c4ed0 con 0x7f3b0c06e2c0 2024-09-17T02:47:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.317+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1528097660 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3afc0099c0 con 0x7f3b0c06e2c0 2024-09-17T02:47:27.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.317+0000 7f3b02ffd640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3aec073200 0x7f3aec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:27.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.317+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1528097660 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f3afc0947c0 con 0x7f3b0c06e2c0 2024-09-17T02:47:27.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.317+0000 7f3b03fff640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3aec073200 0x7f3aec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:27.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.317+0000 7f3b03fff640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3aec073200 0x7f3aec075710 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3af40af000 tx=0x7f3af40a8040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:27.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.321+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1528097660 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3afc05f7c0 con 0x7f3b0c06e2c0 2024-09-17T02:47:27.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:27.513+0000 7f3b10ae0640 1 -- 172.21.15.131:0/1528097660 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3b0c073310 con 0x7f3b0c06e2c0 2024-09-17T02:47:28.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1528097660 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) ==== 118+0+0 (secure 0 0 0) 0x7f3afc05f0c0 con 0x7f3b0c06e2c0 2024-09-17T02:47:28.412 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 -- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3aec073200 msgr2=0x7f3aec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3aec073200 0x7f3aec075710 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3af40af000 tx=0x7f3af40a8040 comp rx=0 tx=0).stop 2024-09-17T02:47:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 -- 172.21.15.131:0/1528097660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b0c06e2c0 msgr2=0x7f3b0c0c9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b0c06e2c0 0x7f3b0c0c9ab0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f3afc00d660 tx=0x7f3afc00db20 comp rx=0 tx=0).stop 2024-09-17T02:47:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 -- 172.21.15.131:0/1528097660 shutdown_connections 2024-09-17T02:47:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3aec073200 0x7f3aec075710 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3b0c0cdbf0 0x7f3b0c1547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3b0c0c4ac0 0x7f3b0c154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 --2- 172.21.15.131:0/1528097660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3b0c06e2c0 0x7f3b0c0c9ab0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 -- 172.21.15.131:0/1528097660 >> 172.21.15.131:0/1528097660 conn(0x7f3b0c06eea0 msgr2=0x7f3b0c14ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 -- 172.21.15.131:0/1528097660 shutdown_connections 2024-09-17T02:47:28.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.413+0000 7f3b10ae0640 1 -- 172.21.15.131:0/1528097660 wait complete. 2024-09-17T02:47:28.439 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T02:47:28.439 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T02:47:28.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.605+0000 7f183c3b9640 1 -- 172.21.15.131:0/2385998144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1820005260 msgr2=0x7f182000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:28.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.605+0000 7f183c3b9640 1 --2- 172.21.15.131:0/2385998144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1820005260 0x7f182000b9d0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1834160070 tx=0x7f183001cf00 comp rx=0 tx=0).stop 2024-09-17T02:47:28.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.605+0000 7f183c3b9640 1 -- 172.21.15.131:0/2385998144 shutdown_connections 2024-09-17T02:47:28.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.605+0000 7f183c3b9640 1 --2- 172.21.15.131:0/2385998144 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1820005260 0x7f182000b9d0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:28.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.605+0000 7f183c3b9640 1 -- 172.21.15.131:0/2385998144 >> 172.21.15.131:0/2385998144 conn(0x7f183406eea0 msgr2=0x7f183406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:28.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.605+0000 7f183c3b9640 1 -- 172.21.15.131:0/2385998144 shutdown_connections 2024-09-17T02:47:28.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.605+0000 7f183c3b9640 1 -- 172.21.15.131:0/2385998144 wait complete. 2024-09-17T02:47:28.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183c3b9640 1 Processor -- start 2024-09-17T02:47:28.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183c3b9640 1 -- start start 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183c3b9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1820005260 0x7f18341e96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183c3b9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1834159990 0x7f1834142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183c3b9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1834142960 0x7f18341f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183c3b9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f183415f270 con 0x7f1834142960 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183c3b9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f18340cf320 con 0x7f1820005260 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183c3b9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f183406dea0 con 0x7f1834159990 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183a956640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1834142960 0x7f18341f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183a956640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1834142960 0x7f18341f6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50444/0 (socket says 172.21.15.131:50444) 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183a956640 1 -- 172.21.15.131:0/90651179 learned_addr learned my addr 172.21.15.131:0/90651179 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183a155640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1820005260 0x7f18341e96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f1839954640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1834159990 0x7f1834142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:28.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183a956640 1 -- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1834159990 msgr2=0x7f1834142420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:28.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183a956640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1834159990 0x7f1834142420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:28.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183a956640 1 -- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1820005260 msgr2=0x7f18341e96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:28.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183a956640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1820005260 0x7f18341e96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:28.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183a956640 1 -- 172.21.15.131:0/90651179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18341e8610 con 0x7f1834142960 2024-09-17T02:47:28.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183a956640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1834142960 0x7f18341f6c50 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f183001d7a0 tx=0x7f18300011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:28.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f182affd640 1 -- 172.21.15.131:0/90651179 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830006660 con 0x7f1834142960 2024-09-17T02:47:28.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f182affd640 1 -- 172.21.15.131:0/90651179 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18300016d0 con 0x7f1834142960 2024-09-17T02:47:28.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183c3b9640 1 -- 172.21.15.131:0/90651179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18341e88f0 con 0x7f1834142960 2024-09-17T02:47:28.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f182affd640 1 -- 172.21.15.131:0/90651179 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830003600 con 0x7f1834142960 2024-09-17T02:47:28.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.609+0000 7f183c3b9640 1 -- 172.21.15.131:0/90651179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18341e8eb0 con 0x7f1834142960 2024-09-17T02:47:28.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.613+0000 7f183c3b9640 1 -- 172.21.15.131:0/90651179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1834075b80 con 0x7f1834142960 2024-09-17T02:47:28.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.613+0000 7f182affd640 1 -- 172.21.15.131:0/90651179 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f18300b8030 con 0x7f1834142960 2024-09-17T02:47:28.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.613+0000 7f182affd640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1818073200 0x7f1818075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:28.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.613+0000 7f182affd640 1 -- 172.21.15.131:0/90651179 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f183012e180 con 0x7f1834142960 2024-09-17T02:47:28.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.613+0000 7f183a155640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1818073200 0x7f1818075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:28.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.613+0000 7f183a155640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1818073200 0x7f1818075710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f182c009270 tx=0x7f182c0091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:28.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.617+0000 7f182affd640 1 -- 172.21.15.131:0/90651179 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f183406e7f0 con 0x7f1834142960 2024-09-17T02:47:28.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:28.821+0000 7f183c3b9640 1 -- 172.21.15.131:0/90651179 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f18341e8360 con 0x7f1834142960 2024-09-17T02:47:29.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.421+0000 7f182affd640 1 -- 172.21.15.131:0/90651179 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) ==== 122+0+0 (secure 0 0 0) 0x7f18300f8b10 con 0x7f1834142960 2024-09-17T02:47:29.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 -- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1818073200 msgr2=0x7f1818075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:29.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1818073200 0x7f1818075710 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f182c009270 tx=0x7f182c0091e0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 -- 172.21.15.131:0/90651179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1834142960 msgr2=0x7f18341f6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:29.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1834142960 0x7f18341f6c50 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f183001d7a0 tx=0x7f18300011a0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 -- 172.21.15.131:0/90651179 shutdown_connections 2024-09-17T02:47:29.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1818073200 0x7f1818075710 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1834142960 0x7f18341f6c50 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1834159990 0x7f1834142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 --2- 172.21.15.131:0/90651179 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1820005260 0x7f18341e96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 -- 172.21.15.131:0/90651179 >> 172.21.15.131:0/90651179 conn(0x7f183406eea0 msgr2=0x7f18340afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:29.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 -- 172.21.15.131:0/90651179 shutdown_connections 2024-09-17T02:47:29.424 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.425+0000 7f183c3b9640 1 -- 172.21.15.131:0/90651179 wait complete. 2024-09-17T02:47:29.449 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T02:47:29.449 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:47:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.601+0000 7f7d7c597640 1 -- 172.21.15.131:0/3344005564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d740c53a0 msgr2=0x7f7d740cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.601+0000 7f7d7c597640 1 --2- 172.21.15.131:0/3344005564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d740c53a0 0x7f7d740cf010 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009900 tx=0x7f7d7001c850 comp rx=0 tx=0).stop 2024-09-17T02:47:29.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.601+0000 7f7d7c597640 1 -- 172.21.15.131:0/3344005564 shutdown_connections 2024-09-17T02:47:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.601+0000 7f7d7c597640 1 --2- 172.21.15.131:0/3344005564 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d740c53a0 0x7f7d740cf010 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.601+0000 7f7d7c597640 1 -- 172.21.15.131:0/3344005564 >> 172.21.15.131:0/3344005564 conn(0x7f7d7406eea0 msgr2=0x7f7d7406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.601+0000 7f7d7c597640 1 -- 172.21.15.131:0/3344005564 shutdown_connections 2024-09-17T02:47:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.601+0000 7f7d7c597640 1 -- 172.21.15.131:0/3344005564 wait complete. 2024-09-17T02:47:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7c597640 1 Processor -- start 2024-09-17T02:47:29.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7c597640 1 -- start start 2024-09-17T02:47:29.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7c597640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d7406e2c0 0x7f7d740c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7c597640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d740c53a0 0x7f7d7414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7c597640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d740cec00 0x7f7d7415b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7c597640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7416ae80 con 0x7f7d7406e2c0 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7c597640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7416ad00 con 0x7f7d740c53a0 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7c597640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7416b000 con 0x7f7d740cec00 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7a333640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d7406e2c0 0x7f7d740c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7a333640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d7406e2c0 0x7f7d740c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50462/0 (socket says 172.21.15.131:50462) 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7a333640 1 -- 172.21.15.131:0/2832459112 learned_addr learned my addr 172.21.15.131:0/2832459112 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7ab34640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d740cec00 0x7f7d7415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d79b32640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d740c53a0 0x7f7d7414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7a333640 1 -- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d740cec00 msgr2=0x7f7d7415b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:29.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7a333640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d740cec00 0x7f7d7415b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7a333640 1 -- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d740c53a0 msgr2=0x7f7d7414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:29.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7a333640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d740c53a0 0x7f7d7414c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7a333640 1 -- 172.21.15.131:0/2832459112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7415a8b0 con 0x7f7d7406e2c0 2024-09-17T02:47:29.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d79b32640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d740c53a0 0x7f7d7414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:29.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7ab34640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d740cec00 0x7f7d7415b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:29.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7a333640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d7406e2c0 0x7f7d740c98a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00b440 tx=0x7f7d6c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:29.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d66ffd640 1 -- 172.21.15.131:0/2832459112 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c015020 con 0x7f7d7406e2c0 2024-09-17T02:47:29.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d66ffd640 1 -- 172.21.15.131:0/2832459112 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d6c011980 con 0x7f7d7406e2c0 2024-09-17T02:47:29.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d66ffd640 1 -- 172.21.15.131:0/2832459112 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c011dd0 con 0x7f7d7406e2c0 2024-09-17T02:47:29.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7c597640 1 -- 172.21.15.131:0/2832459112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d74155bf0 con 0x7f7d7406e2c0 2024-09-17T02:47:29.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.605+0000 7f7d7c597640 1 -- 172.21.15.131:0/2832459112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7415ab50 con 0x7f7d7406e2c0 2024-09-17T02:47:29.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.609+0000 7f7d7c597640 1 -- 172.21.15.131:0/2832459112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d74156110 con 0x7f7d7406e2c0 2024-09-17T02:47:29.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.609+0000 7f7d66ffd640 1 -- 172.21.15.131:0/2832459112 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7d6c018020 con 0x7f7d7406e2c0 2024-09-17T02:47:29.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.609+0000 7f7d66ffd640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d58073230 0x7f7d58075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:29.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.609+0000 7f7d66ffd640 1 -- 172.21.15.131:0/2832459112 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f7d6c094020 con 0x7f7d7406e2c0 2024-09-17T02:47:29.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.609+0000 7f7d79b32640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d58073230 0x7f7d58075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:29.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.609+0000 7f7d79b32640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d58073230 0x7f7d58075740 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7d68007fd0 tx=0x7f7d68009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:29.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.613+0000 7f7d66ffd640 1 -- 172.21.15.131:0/2832459112 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7d74156110 con 0x7f7d7406e2c0 2024-09-17T02:47:29.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.801+0000 7f7d7c597640 1 -- 172.21.15.131:0/2832459112 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d740cf010 con 0x7f7d7406e2c0 2024-09-17T02:47:29.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.801+0000 7f7d66ffd640 1 -- 172.21.15.131:0/2832459112 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5237 (secure 0 0 0) 0x7f7d6c01f070 con 0x7f7d7406e2c0 2024-09-17T02:47:29.801 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:29.801 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":64,"btime":"2024-09-17T02:47:29:415704+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:47:28.826817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2766162522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2766162522},{"type":"v1","addr":"172.21.15.131:6835","nonce":2766162522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7038,"qdb_cluster":[7038]},"id":6}]} 2024-09-17T02:47:29.801 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 64 2024-09-17T02:47:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 -- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d58073230 msgr2=0x7f7d58075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d58073230 0x7f7d58075740 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7d68007fd0 tx=0x7f7d68009040 comp rx=0 tx=0).stop 2024-09-17T02:47:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 -- 172.21.15.131:0/2832459112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d7406e2c0 msgr2=0x7f7d740c98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d7406e2c0 0x7f7d740c98a0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00b440 tx=0x7f7d6c00b900 comp rx=0 tx=0).stop 2024-09-17T02:47:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 -- 172.21.15.131:0/2832459112 shutdown_connections 2024-09-17T02:47:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d58073230 0x7f7d58075740 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d740cec00 0x7f7d7415b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d740c53a0 0x7f7d7414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 --2- 172.21.15.131:0/2832459112 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d7406e2c0 0x7f7d740c98a0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 -- 172.21.15.131:0/2832459112 >> 172.21.15.131:0/2832459112 conn(0x7f7d7406eea0 msgr2=0x7f7d740b3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:29.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 -- 172.21.15.131:0/2832459112 shutdown_connections 2024-09-17T02:47:29.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.805+0000 7f7d7c597640 1 -- 172.21.15.131:0/2832459112 wait complete. 2024-09-17T02:47:29.822 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:47:29.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.977+0000 7fc79af5c640 1 -- 172.21.15.131:0/3927064571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c003a10 msgr2=0x7fc77c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:29.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.977+0000 7fc79af5c640 1 --2- 172.21.15.131:0/3927064571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc77c003a10 0x7fc77c003e80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c2e0 tx=0x7fc78401cf80 comp rx=0 tx=0).stop 2024-09-17T02:47:29.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.977+0000 7fc7994f9640 1 -- 172.21.15.131:0/3927064571 reap_dead start 2024-09-17T02:47:29.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.977+0000 7fc79af5c640 1 -- 172.21.15.131:0/3927064571 shutdown_connections 2024-09-17T02:47:29.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.977+0000 7fc79af5c640 1 -- 172.21.15.131:0/3927064571 >> 172.21.15.131:0/3927064571 conn(0x7fc7940d1b10 msgr2=0x7fc7940d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:29.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.977+0000 7fc79af5c640 1 -- 172.21.15.131:0/3927064571 shutdown_connections 2024-09-17T02:47:29.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.977+0000 7fc79af5c640 1 -- 172.21.15.131:0/3927064571 wait complete. 2024-09-17T02:47:29.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc79af5c640 1 Processor -- start 2024-09-17T02:47:29.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc79af5c640 1 -- start start 2024-09-17T02:47:29.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc79af5c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc77c003a10 0x7fc79414e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc79af5c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7941fb970 0x7fc7941fbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc79af5c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7941fc2c0 0x7fc7941f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc79af5c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc794169a30 con 0x7fc7941fb970 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc79af5c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7941698b0 con 0x7fc7941fc2c0 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc79af5c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc794169bb0 con 0x7fc77c003a10 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc793fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7941fb970 0x7fc7941fbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc793fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7941fb970 0x7fc7941fbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50472/0 (socket says 172.21.15.131:50472) 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc793fff640 1 -- 172.21.15.131:0/4083450489 learned_addr learned my addr 172.21.15.131:0/4083450489 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc798cf8640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc77c003a10 0x7fc79414e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc7994f9640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7941fc2c0 0x7fc7941f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc793fff640 1 -- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc77c003a10 msgr2=0x7fc79414e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:29.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc793fff640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc77c003a10 0x7fc79414e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc793fff640 1 -- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7941fc2c0 msgr2=0x7fc7941f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:29.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc793fff640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7941fc2c0 0x7fc7941f8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:29.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc793fff640 1 -- 172.21.15.131:0/4083450489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79414e840 con 0x7fc7941fb970 2024-09-17T02:47:29.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc7994f9640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7941fc2c0 0x7fc7941f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:29.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc793fff640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7941fb970 0x7fc7941fbd80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc788010990 tx=0x7fc788010e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:29.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc7917fa640 1 -- 172.21.15.131:0/4083450489 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78800ed80 con 0x7fc7941fb970 2024-09-17T02:47:29.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc7917fa640 1 -- 172.21.15.131:0/4083450489 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc788005020 con 0x7fc7941fb970 2024-09-17T02:47:29.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc79af5c640 1 -- 172.21.15.131:0/4083450489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7941514a0 con 0x7fc7941fb970 2024-09-17T02:47:29.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc7917fa640 1 -- 172.21.15.131:0/4083450489 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788005450 con 0x7fc7941fb970 2024-09-17T02:47:29.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.981+0000 7fc79af5c640 1 -- 172.21.15.131:0/4083450489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc794151a60 con 0x7fc7941fb970 2024-09-17T02:47:29.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.985+0000 7fc79af5c640 1 -- 172.21.15.131:0/4083450489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7940d2460 con 0x7fc7941fb970 2024-09-17T02:47:29.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.985+0000 7fc7917fa640 1 -- 172.21.15.131:0/4083450489 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc7880047b0 con 0x7fc7941fb970 2024-09-17T02:47:29.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.985+0000 7fc7917fa640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc780073200 0x7fc780075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:29.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.985+0000 7fc7917fa640 1 -- 172.21.15.131:0/4083450489 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7fc7880950e0 con 0x7fc7941fb970 2024-09-17T02:47:29.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.985+0000 7fc798cf8640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc780073200 0x7fc780075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:29.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.985+0000 7fc798cf8640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc780073200 0x7fc780075710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c2e0 tx=0x7fc7840af040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:29.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:29.989+0000 7fc7917fa640 1 -- 172.21.15.131:0/4083450489 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc788060000 con 0x7fc7941fb970 2024-09-17T02:47:30.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.161+0000 7fc79af5c640 1 -- 172.21.15.131:0/4083450489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc79406abc0 con 0x7fc7941fb970 2024-09-17T02:47:30.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc7917fa640 1 -- 172.21.15.131:0/4083450489 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+17041 (secure 0 0 0) 0x7fc78805f900 con 0x7fc7941fb970 2024-09-17T02:47:30.162 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:30.162 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":92,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:47:27.403719+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:47:23.485820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-17T02:47:24.759592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:47:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 -- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc780073200 msgr2=0x7fc780075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc780073200 0x7fc780075710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c2e0 tx=0x7fc7840af040 comp rx=0 tx=0).stop 2024-09-17T02:47:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 -- 172.21.15.131:0/4083450489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7941fb970 msgr2=0x7fc7941fbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7941fb970 0x7fc7941fbd80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc788010990 tx=0x7fc788010e50 comp rx=0 tx=0).stop 2024-09-17T02:47:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 -- 172.21.15.131:0/4083450489 shutdown_connections 2024-09-17T02:47:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc780073200 0x7fc780075710 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7941fc2c0 0x7fc7941f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7941fb970 0x7fc7941fbd80 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 --2- 172.21.15.131:0/4083450489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc77c003a10 0x7fc79414e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 -- 172.21.15.131:0/4083450489 >> 172.21.15.131:0/4083450489 conn(0x7fc7940d1b10 msgr2=0x7fc7940b8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 -- 172.21.15.131:0/4083450489 shutdown_connections 2024-09-17T02:47:30.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.165+0000 7fc79af5c640 1 -- 172.21.15.131:0/4083450489 wait complete. 2024-09-17T02:47:30.184 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T02:47:30.185 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:47:30.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.325+0000 7f37ef79d640 1 -- 172.21.15.131:0/4007048970 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f37e8071210 msgr2=0x7f37e8071620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.325+0000 7f37ef79d640 1 --2- 172.21.15.131:0/4007048970 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f37e8071210 0x7f37e8071620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009840 tx=0x7f37d801c820 comp rx=0 tx=0).stop 2024-09-17T02:47:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.325+0000 7f37ef79d640 1 -- 172.21.15.131:0/4007048970 shutdown_connections 2024-09-17T02:47:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.325+0000 7f37ef79d640 1 --2- 172.21.15.131:0/4007048970 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f37e8071210 0x7f37e8071620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.325+0000 7f37ef79d640 1 --2- 172.21.15.131:0/4007048970 >> v2:172.21.15.189:3300/0 conn(0x7f37e80ada50 0x7f37e815e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.325+0000 7f37ef79d640 1 -- 172.21.15.131:0/4007048970 >> 172.21.15.131:0/4007048970 conn(0x7f37e80d1130 msgr2=0x7f37e80d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.325+0000 7f37ef79d640 1 -- 172.21.15.131:0/4007048970 shutdown_connections 2024-09-17T02:47:30.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.325+0000 7f37ef79d640 1 -- 172.21.15.131:0/4007048970 wait complete. 2024-09-17T02:47:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ef79d640 1 Processor -- start 2024-09-17T02:47:30.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ef79d640 1 -- start start 2024-09-17T02:47:30.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ef79d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37e806db30 0x7f37e8149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:30.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ef79d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f37e8071210 0x7f37e8156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:30.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ef79d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37e80ada50 0x7f37e8156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ef79d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f37e816bc20 con 0x7f37e80ada50 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ef79d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f37e80ae070 con 0x7f37e806db30 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ef79d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f37e816bda0 con 0x7f37e8071210 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37edd3a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37e80ada50 0x7f37e8156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37edd3a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37e80ada50 0x7f37e8156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50486/0 (socket says 172.21.15.131:50486) 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37edd3a640 1 -- 172.21.15.131:0/4210919439 learned_addr learned my addr 172.21.15.131:0/4210919439 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ecd38640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f37e8071210 0x7f37e8156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ed539640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37e806db30 0x7f37e8149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37edd3a640 1 -- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f37e8071210 msgr2=0x7f37e8156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37edd3a640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f37e8071210 0x7f37e8156120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37edd3a640 1 -- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37e806db30 msgr2=0x7f37e8149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37edd3a640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37e806db30 0x7f37e8149f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37edd3a640 1 -- 172.21.15.131:0/4210919439 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e814a4a0 con 0x7f37e80ada50 2024-09-17T02:47:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37edd3a640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37e80ada50 0x7f37e8156660 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f37e400c3c0 tx=0x7f37e400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ddffb640 1 -- 172.21.15.131:0/4210919439 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4005220 con 0x7f37e80ada50 2024-09-17T02:47:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ef79d640 1 -- 172.21.15.131:0/4210919439 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e815d5c0 con 0x7f37e80ada50 2024-09-17T02:47:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ddffb640 1 -- 172.21.15.131:0/4210919439 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37e40054b0 con 0x7f37e80ada50 2024-09-17T02:47:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ddffb640 1 -- 172.21.15.131:0/4210919439 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4006a20 con 0x7f37e80ada50 2024-09-17T02:47:30.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.329+0000 7f37ef79d640 1 -- 172.21.15.131:0/4210919439 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e814bec0 con 0x7f37e80ada50 2024-09-17T02:47:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.333+0000 7f37ef79d640 1 -- 172.21.15.131:0/4210919439 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e8071620 con 0x7f37e80ada50 2024-09-17T02:47:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.333+0000 7f37ddffb640 1 -- 172.21.15.131:0/4210919439 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f37e4019020 con 0x7f37e80ada50 2024-09-17T02:47:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.333+0000 7f37ddffb640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f37cc073200 0x7f37cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.333+0000 7f37ddffb640 1 -- 172.21.15.131:0/4210919439 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f37e4093c30 con 0x7f37e80ada50 2024-09-17T02:47:30.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.333+0000 7f37ed539640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f37cc073200 0x7f37cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:30.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.333+0000 7f37ed539640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f37cc073200 0x7f37cc075710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009e70 tx=0x7f37e00095c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:30.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.337+0000 7f37ddffb640 1 -- 172.21.15.131:0/4210919439 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f37e4020020 con 0x7f37e80ada50 2024-09-17T02:47:30.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.497+0000 7f37ef79d640 1 -- 172.21.15.131:0/4210919439 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f37e815d370 con 0x7f37cc073200 2024-09-17T02:47:30.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.513+0000 7f37ddffb640 1 -- 172.21.15.131:0/4210919439 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247465 (secure 0 0 0) 0x7f37e815d370 con 0x7f37cc073200 2024-09-17T02:47:30.512 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:30.514 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:47:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 -- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f37cc073200 msgr2=0x7f37cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f37cc073200 0x7f37cc075710 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009e70 tx=0x7f37e00095c0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 -- 172.21.15.131:0/4210919439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37e80ada50 msgr2=0x7f37e8156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37e80ada50 0x7f37e8156660 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f37e400c3c0 tx=0x7f37e400c7c0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 -- 172.21.15.131:0/4210919439 shutdown_connections 2024-09-17T02:47:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f37cc073200 0x7f37cc075710 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f37e80ada50 0x7f37e8156660 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f37e8071210 0x7f37e8156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 --2- 172.21.15.131:0/4210919439 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37e806db30 0x7f37e8149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 -- 172.21.15.131:0/4210919439 >> 172.21.15.131:0/4210919439 conn(0x7f37e80d1130 msgr2=0x7f37e806df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:30.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 -- 172.21.15.131:0/4210919439 shutdown_connections 2024-09-17T02:47:30.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.517+0000 7f37ef79d640 1 -- 172.21.15.131:0/4210919439 wait complete. 2024-09-17T02:47:30.533 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-17T02:47:29.275711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259328,"kb_used_data":9788,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715392,"statfs":{"total":773094113280,"available":772828561408,"internally_reserved":0,"allocated":10022912,"data_stored":5214549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.910845"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422667+0000","last_change":"2024-09-17T02:47:26.451633+0000","last_active":"2024-09-17T02:47:27.422667+0000","last_peered":"2024-09-17T02:47:27.422667+0000","last_clean":"2024-09-17T02:47:27.422667+0000","last_became_active":"2024-09-17T02:47:26.450812+0000","last_became_peered":"2024-09-17T02:47:26.450812+0000","last_unstale":"2024-09-17T02:47:27.422667+0000","last_undegraded":"2024-09-17T02:47:27.422667+0000","last_fullsized":"2024-09-17T02:47:27.422667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:06.130998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.404861+0000","last_change":"2024-09-17T02:47:25.404861+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.404861+0000","last_undegraded":"2024-09-17T02:47:25.404861+0000","last_fullsized":"2024-09-17T02:47:25.404861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.392716+0000","last_change":"2024-09-17T02:47:25.392716+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.392716+0000","last_undegraded":"2024-09-17T02:47:25.392716+0000","last_fullsized":"2024-09-17T02:47:25.392716+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.394859+0000","last_change":"2024-09-17T02:47:25.394859+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.394859+0000","last_undegraded":"2024-09-17T02:47:25.394859+0000","last_fullsized":"2024-09-17T02:47:25.394859+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.404469+0000","last_change":"2024-09-17T02:47:25.404469+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.404469+0000","last_undegraded":"2024-09-17T02:47:25.404469+0000","last_fullsized":"2024-09-17T02:47:25.404469+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.405928+0000","last_change":"2024-09-17T02:47:25.405928+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.405928+0000","last_undegraded":"2024-09-17T02:47:25.405928+0000","last_fullsized":"2024-09-17T02:47:25.405928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.390435+0000","last_change":"2024-09-17T02:47:25.390435+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.390435+0000","last_undegraded":"2024-09-17T02:47:25.390435+0000","last_fullsized":"2024-09-17T02:47:25.390435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.402210+0000","last_change":"2024-09-17T02:47:25.402210+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.402210+0000","last_undegraded":"2024-09-17T02:47:25.402210+0000","last_fullsized":"2024-09-17T02:47:25.402210+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.386493+0000","last_change":"2024-09-17T02:47:25.386493+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.386493+0000","last_undegraded":"2024-09-17T02:47:25.386493+0000","last_fullsized":"2024-09-17T02:47:25.386493+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.392764+0000","last_change":"2024-09-17T02:47:25.392764+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.392764+0000","last_undegraded":"2024-09-17T02:47:25.392764+0000","last_fullsized":"2024-09-17T02:47:25.392764+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.388466+0000","last_change":"2024-09-17T02:47:25.388466+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.388466+0000","last_undegraded":"2024-09-17T02:47:25.388466+0000","last_fullsized":"2024-09-17T02:47:25.388466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.393605+0000","last_change":"2024-09-17T02:47:25.393605+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.393605+0000","last_undegraded":"2024-09-17T02:47:25.393605+0000","last_fullsized":"2024-09-17T02:47:25.393605+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.410168+0000","last_change":"2024-09-17T02:47:25.410168+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.410168+0000","last_undegraded":"2024-09-17T02:47:25.410168+0000","last_fullsized":"2024-09-17T02:47:25.410168+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.391149+0000","last_change":"2024-09-17T02:47:25.391149+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.391149+0000","last_undegraded":"2024-09-17T02:47:25.391149+0000","last_fullsized":"2024-09-17T02:47:25.391149+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.392004+0000","last_change":"2024-09-17T02:47:25.392004+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.392004+0000","last_undegraded":"2024-09-17T02:47:25.392004+0000","last_fullsized":"2024-09-17T02:47:25.392004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.400977+0000","last_change":"2024-09-17T02:47:25.400977+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.400977+0000","last_undegraded":"2024-09-17T02:47:25.400977+0000","last_fullsized":"2024-09-17T02:47:25.400977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.406030+0000","last_change":"2024-09-17T02:47:25.406030+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.406030+0000","last_undegraded":"2024-09-17T02:47:25.406030+0000","last_fullsized":"2024-09-17T02:47:25.406030+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.391330+0000","last_change":"2024-09-17T02:47:25.391330+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.391330+0000","last_undegraded":"2024-09-17T02:47:25.391330+0000","last_fullsized":"2024-09-17T02:47:25.391330+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.392835+0000","last_change":"2024-09-17T02:47:25.392835+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.392835+0000","last_undegraded":"2024-09-17T02:47:25.392835+0000","last_fullsized":"2024-09-17T02:47:25.392835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.384257+0000","last_change":"2024-09-17T02:47:25.384257+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.384257+0000","last_undegraded":"2024-09-17T02:47:25.384257+0000","last_fullsized":"2024-09-17T02:47:25.384257+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.403781+0000","last_change":"2024-09-17T02:47:25.403781+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.403781+0000","last_undegraded":"2024-09-17T02:47:25.403781+0000","last_fullsized":"2024-09-17T02:47:25.403781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.390040+0000","last_change":"2024-09-17T02:47:25.390040+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.390040+0000","last_undegraded":"2024-09-17T02:47:25.390040+0000","last_fullsized":"2024-09-17T02:47:25.390040+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.403689+0000","last_change":"2024-09-17T02:47:25.403689+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.403689+0000","last_undegraded":"2024-09-17T02:47:25.403689+0000","last_fullsized":"2024-09-17T02:47:25.403689+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.418593+0000","last_change":"2024-09-17T02:47:26.438730+0000","last_active":"2024-09-17T02:47:27.418593+0000","last_peered":"2024-09-17T02:47:27.418593+0000","last_clean":"2024-09-17T02:47:27.418593+0000","last_became_active":"2024-09-17T02:47:26.420076+0000","last_became_peered":"2024-09-17T02:47:26.420076+0000","last_unstale":"2024-09-17T02:47:27.418593+0000","last_undegraded":"2024-09-17T02:47:27.418593+0000","last_fullsized":"2024-09-17T02:47:27.418593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:12.606293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.403690+0000","last_change":"2024-09-17T02:47:25.403690+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.403690+0000","last_undegraded":"2024-09-17T02:47:25.403690+0000","last_fullsized":"2024-09-17T02:47:25.403690+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.402945+0000","last_change":"2024-09-17T02:47:25.402945+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.402945+0000","last_undegraded":"2024-09-17T02:47:25.402945+0000","last_fullsized":"2024-09-17T02:47:25.402945+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.391318+0000","last_change":"2024-09-17T02:47:25.391318+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.391318+0000","last_undegraded":"2024-09-17T02:47:25.391318+0000","last_fullsized":"2024-09-17T02:47:25.391318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.409435+0000","last_change":"2024-09-17T02:47:25.409435+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.409435+0000","last_undegraded":"2024-09-17T02:47:25.409435+0000","last_fullsized":"2024-09-17T02:47:25.409435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.402326+0000","last_change":"2024-09-17T02:47:25.402326+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.402326+0000","last_undegraded":"2024-09-17T02:47:25.402326+0000","last_fullsized":"2024-09-17T02:47:25.402326+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.401754+0000","last_change":"2024-09-17T02:47:25.401754+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.401754+0000","last_undegraded":"2024-09-17T02:47:25.401754+0000","last_fullsized":"2024-09-17T02:47:25.401754+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.392184+0000","last_change":"2024-09-17T02:47:25.392184+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.392184+0000","last_undegraded":"2024-09-17T02:47:25.392184+0000","last_fullsized":"2024-09-17T02:47:25.392184+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.399408+0000","last_change":"2024-09-17T02:47:25.399408+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.399408+0000","last_undegraded":"2024-09-17T02:47:25.399408+0000","last_fullsized":"2024-09-17T02:47:25.399408+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421125+0000","last_change":"2024-09-17T02:47:26.419153+0000","last_active":"2024-09-17T02:47:27.421125+0000","last_peered":"2024-09-17T02:47:27.421125+0000","last_clean":"2024-09-17T02:47:27.421125+0000","last_became_active":"2024-09-17T02:47:26.418280+0000","last_became_peered":"2024-09-17T02:47:26.418280+0000","last_unstale":"2024-09-17T02:47:27.421125+0000","last_undegraded":"2024-09-17T02:47:27.421125+0000","last_fullsized":"2024-09-17T02:47:27.421125+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:17.220457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.409473+0000","last_change":"2024-09-17T02:47:25.404962+0000","last_active":"2024-09-17T02:47:25.409473+0000","last_peered":"2024-09-17T02:47:25.409473+0000","last_clean":"2024-09-17T02:47:25.409473+0000","last_became_active":"2024-09-17T02:47:25.404574+0000","last_became_peered":"2024-09-17T02:47:25.404574+0000","last_unstale":"2024-09-17T02:47:25.409473+0000","last_undegraded":"2024-09-17T02:47:25.409473+0000","last_fullsized":"2024-09-17T02:47:25.409473+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:00.198104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.382792+0000","last_change":"2024-09-17T02:47:25.382792+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.382792+0000","last_undegraded":"2024-09-17T02:47:25.382792+0000","last_fullsized":"2024-09-17T02:47:25.382792+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.398031+0000","last_change":"2024-09-17T02:47:25.394418+0000","last_active":"2024-09-17T02:47:25.398031+0000","last_peered":"2024-09-17T02:47:25.398031+0000","last_clean":"2024-09-17T02:47:25.398031+0000","last_became_active":"2024-09-17T02:47:25.393974+0000","last_became_peered":"2024-09-17T02:47:25.393974+0000","last_unstale":"2024-09-17T02:47:25.398031+0000","last_undegraded":"2024-09-17T02:47:25.398031+0000","last_fullsized":"2024-09-17T02:47:25.398031+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:37.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.385646+0000","last_change":"2024-09-17T02:47:25.385646+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.385646+0000","last_undegraded":"2024-09-17T02:47:25.385646+0000","last_fullsized":"2024-09-17T02:47:25.385646+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.420057+0000","last_change":"2024-09-17T02:47:25.418775+0000","last_active":"2024-09-17T02:47:25.420057+0000","last_peered":"2024-09-17T02:47:25.420057+0000","last_clean":"2024-09-17T02:47:25.420057+0000","last_became_active":"2024-09-17T02:47:25.418458+0000","last_became_peered":"2024-09-17T02:47:25.418458+0000","last_unstale":"2024-09-17T02:47:25.420057+0000","last_undegraded":"2024-09-17T02:47:25.420057+0000","last_fullsized":"2024-09-17T02:47:25.420057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:53.327706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.390407+0000","last_change":"2024-09-17T02:47:25.390407+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.390407+0000","last_undegraded":"2024-09-17T02:47:25.390407+0000","last_fullsized":"2024-09-17T02:47:25.390407+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.413571+0000","last_change":"2024-09-17T02:47:25.412944+0000","last_active":"2024-09-17T02:47:25.413571+0000","last_peered":"2024-09-17T02:47:25.413571+0000","last_clean":"2024-09-17T02:47:25.413571+0000","last_became_active":"2024-09-17T02:47:25.412515+0000","last_became_peered":"2024-09-17T02:47:25.412515+0000","last_unstale":"2024-09-17T02:47:25.413571+0000","last_undegraded":"2024-09-17T02:47:25.413571+0000","last_fullsized":"2024-09-17T02:47:25.413571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:11.200621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.398647+0000","last_change":"2024-09-17T02:47:25.398647+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.398647+0000","last_undegraded":"2024-09-17T02:47:25.398647+0000","last_fullsized":"2024-09-17T02:47:25.398647+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.412709+0000","last_change":"2024-09-17T02:47:25.411938+0000","last_active":"2024-09-17T02:47:25.412709+0000","last_peered":"2024-09-17T02:47:25.412709+0000","last_clean":"2024-09-17T02:47:25.412709+0000","last_became_active":"2024-09-17T02:47:25.411486+0000","last_became_peered":"2024-09-17T02:47:25.411486+0000","last_unstale":"2024-09-17T02:47:25.412709+0000","last_undegraded":"2024-09-17T02:47:25.412709+0000","last_fullsized":"2024-09-17T02:47:25.412709+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:34.937753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.385519+0000","last_change":"2024-09-17T02:47:25.385519+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.385519+0000","last_undegraded":"2024-09-17T02:47:25.385519+0000","last_fullsized":"2024-09-17T02:47:25.385519+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.424202+0000","last_change":"2024-09-17T02:47:25.422380+0000","last_active":"2024-09-17T02:47:25.424202+0000","last_peered":"2024-09-17T02:47:25.424202+0000","last_clean":"2024-09-17T02:47:25.424202+0000","last_became_active":"2024-09-17T02:47:25.421906+0000","last_became_peered":"2024-09-17T02:47:25.421906+0000","last_unstale":"2024-09-17T02:47:25.424202+0000","last_undegraded":"2024-09-17T02:47:25.424202+0000","last_fullsized":"2024-09-17T02:47:25.424202+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:53.113825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.383087+0000","last_change":"2024-09-17T02:47:25.383087+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.383087+0000","last_undegraded":"2024-09-17T02:47:25.383087+0000","last_fullsized":"2024-09-17T02:47:25.383087+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.414536+0000","last_change":"2024-09-17T02:47:25.410268+0000","last_active":"2024-09-17T02:47:25.414536+0000","last_peered":"2024-09-17T02:47:25.414536+0000","last_clean":"2024-09-17T02:47:25.414536+0000","last_became_active":"2024-09-17T02:47:25.409433+0000","last_became_peered":"2024-09-17T02:47:25.409433+0000","last_unstale":"2024-09-17T02:47:25.414536+0000","last_undegraded":"2024-09-17T02:47:25.414536+0000","last_fullsized":"2024-09-17T02:47:25.414536+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:07.402409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.420382+0000","last_change":"2024-09-17T02:47:26.410976+0000","last_active":"2024-09-17T02:47:27.420382+0000","last_peered":"2024-09-17T02:47:27.420382+0000","last_clean":"2024-09-17T02:47:27.420382+0000","last_became_active":"2024-09-17T02:47:26.407305+0000","last_became_peered":"2024-09-17T02:47:26.407305+0000","last_unstale":"2024-09-17T02:47:27.420382+0000","last_undegraded":"2024-09-17T02:47:27.420382+0000","last_fullsized":"2024-09-17T02:47:27.420382+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:53.755378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.423195+0000","last_change":"2024-09-17T02:47:25.420193+0000","last_active":"2024-09-17T02:47:25.423195+0000","last_peered":"2024-09-17T02:47:25.423195+0000","last_clean":"2024-09-17T02:47:25.423195+0000","last_became_active":"2024-09-17T02:47:25.419700+0000","last_became_peered":"2024-09-17T02:47:25.419700+0000","last_unstale":"2024-09-17T02:47:25.423195+0000","last_undegraded":"2024-09-17T02:47:25.423195+0000","last_fullsized":"2024-09-17T02:47:25.423195+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:49:30.791406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.402350+0000","last_change":"2024-09-17T02:47:25.402350+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.402350+0000","last_undegraded":"2024-09-17T02:47:25.402350+0000","last_fullsized":"2024-09-17T02:47:25.402350+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.420772+0000","last_change":"2024-09-17T02:47:25.419435+0000","last_active":"2024-09-17T02:47:25.420772+0000","last_peered":"2024-09-17T02:47:25.420772+0000","last_clean":"2024-09-17T02:47:25.420772+0000","last_became_active":"2024-09-17T02:47:25.418955+0000","last_became_peered":"2024-09-17T02:47:25.418955+0000","last_unstale":"2024-09-17T02:47:25.420772+0000","last_undegraded":"2024-09-17T02:47:25.420772+0000","last_fullsized":"2024-09-17T02:47:25.420772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:13.708227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.387848+0000","last_change":"2024-09-17T02:47:25.387848+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.387848+0000","last_undegraded":"2024-09-17T02:47:25.387848+0000","last_fullsized":"2024-09-17T02:47:25.387848+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.416359+0000","last_change":"2024-09-17T02:47:25.414624+0000","last_active":"2024-09-17T02:47:25.416359+0000","last_peered":"2024-09-17T02:47:25.416359+0000","last_clean":"2024-09-17T02:47:25.416359+0000","last_became_active":"2024-09-17T02:47:25.413560+0000","last_became_peered":"2024-09-17T02:47:25.413560+0000","last_unstale":"2024-09-17T02:47:25.416359+0000","last_undegraded":"2024-09-17T02:47:25.416359+0000","last_fullsized":"2024-09-17T02:47:25.416359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:20.187685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422254+0000","last_change":"2024-09-17T02:47:26.441675+0000","last_active":"2024-09-17T02:47:27.422254+0000","last_peered":"2024-09-17T02:47:27.422254+0000","last_clean":"2024-09-17T02:47:27.422254+0000","last_became_active":"2024-09-17T02:47:26.437413+0000","last_became_peered":"2024-09-17T02:47:26.437413+0000","last_unstale":"2024-09-17T02:47:27.422254+0000","last_undegraded":"2024-09-17T02:47:27.422254+0000","last_fullsized":"2024-09-17T02:47:27.422254+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:50.459729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.420604+0000","last_change":"2024-09-17T02:47:25.419022+0000","last_active":"2024-09-17T02:47:25.420604+0000","last_peered":"2024-09-17T02:47:25.420604+0000","last_clean":"2024-09-17T02:47:25.420604+0000","last_became_active":"2024-09-17T02:47:25.418540+0000","last_became_peered":"2024-09-17T02:47:25.418540+0000","last_unstale":"2024-09-17T02:47:25.420604+0000","last_undegraded":"2024-09-17T02:47:25.420604+0000","last_fullsized":"2024-09-17T02:47:25.420604+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:26.753403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.398901+0000","last_change":"2024-09-17T02:47:25.398901+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.398901+0000","last_undegraded":"2024-09-17T02:47:25.398901+0000","last_fullsized":"2024-09-17T02:47:25.398901+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"91'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421338+0000","last_change":"2024-09-17T02:47:25.418647+0000","last_active":"2024-09-17T02:47:27.421338+0000","last_peered":"2024-09-17T02:47:27.421338+0000","last_clean":"2024-09-17T02:47:27.421338+0000","last_became_active":"2024-09-17T02:47:25.416994+0000","last_became_peered":"2024-09-17T02:47:25.416994+0000","last_unstale":"2024-09-17T02:47:27.421338+0000","last_undegraded":"2024-09-17T02:47:27.421338+0000","last_fullsized":"2024-09-17T02:47:27.421338+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:58.768722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.418667+0000","last_change":"2024-09-17T02:47:26.419193+0000","last_active":"2024-09-17T02:47:27.418667+0000","last_peered":"2024-09-17T02:47:27.418667+0000","last_clean":"2024-09-17T02:47:27.418667+0000","last_became_active":"2024-09-17T02:47:26.418643+0000","last_became_peered":"2024-09-17T02:47:26.418643+0000","last_unstale":"2024-09-17T02:47:27.418667+0000","last_undegraded":"2024-09-17T02:47:27.418667+0000","last_fullsized":"2024-09-17T02:47:27.418667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:46.698904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.420735+0000","last_change":"2024-09-17T02:47:25.418902+0000","last_active":"2024-09-17T02:47:25.420735+0000","last_peered":"2024-09-17T02:47:25.420735+0000","last_clean":"2024-09-17T02:47:25.420735+0000","last_became_active":"2024-09-17T02:47:25.418568+0000","last_became_peered":"2024-09-17T02:47:25.418568+0000","last_unstale":"2024-09-17T02:47:25.420735+0000","last_undegraded":"2024-09-17T02:47:25.420735+0000","last_fullsized":"2024-09-17T02:47:25.420735+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:10.152523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.401220+0000","last_change":"2024-09-17T02:47:25.401220+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.401220+0000","last_undegraded":"2024-09-17T02:47:25.401220+0000","last_fullsized":"2024-09-17T02:47:25.401220+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419403+0000","last_change":"2024-09-17T02:47:25.418948+0000","last_active":"2024-09-17T02:47:27.419403+0000","last_peered":"2024-09-17T02:47:27.419403+0000","last_clean":"2024-09-17T02:47:27.419403+0000","last_became_active":"2024-09-17T02:47:25.417436+0000","last_became_peered":"2024-09-17T02:47:25.417436+0000","last_unstale":"2024-09-17T02:47:27.419403+0000","last_undegraded":"2024-09-17T02:47:27.419403+0000","last_fullsized":"2024-09-17T02:47:27.419403+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:25.386796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419176+0000","last_change":"2024-09-17T02:47:26.438873+0000","last_active":"2024-09-17T02:47:27.419176+0000","last_peered":"2024-09-17T02:47:27.419176+0000","last_clean":"2024-09-17T02:47:27.419176+0000","last_became_active":"2024-09-17T02:47:26.436291+0000","last_became_peered":"2024-09-17T02:47:26.436291+0000","last_unstale":"2024-09-17T02:47:27.419176+0000","last_undegraded":"2024-09-17T02:47:27.419176+0000","last_fullsized":"2024-09-17T02:47:27.419176+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:56.037102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.423108+0000","last_change":"2024-09-17T02:47:25.411837+0000","last_active":"2024-09-17T02:47:27.423108+0000","last_peered":"2024-09-17T02:47:27.423108+0000","last_clean":"2024-09-17T02:47:27.423108+0000","last_became_active":"2024-09-17T02:47:25.411299+0000","last_became_peered":"2024-09-17T02:47:25.411299+0000","last_unstale":"2024-09-17T02:47:27.423108+0000","last_undegraded":"2024-09-17T02:47:27.423108+0000","last_fullsized":"2024-09-17T02:47:27.423108+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:23.951343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.397868+0000","last_change":"2024-09-17T02:47:25.397868+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.397868+0000","last_undegraded":"2024-09-17T02:47:25.397868+0000","last_fullsized":"2024-09-17T02:47:25.397868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.423510+0000","last_change":"2024-09-17T02:47:25.421873+0000","last_active":"2024-09-17T02:47:25.423510+0000","last_peered":"2024-09-17T02:47:25.423510+0000","last_clean":"2024-09-17T02:47:25.423510+0000","last_became_active":"2024-09-17T02:47:25.421196+0000","last_became_peered":"2024-09-17T02:47:25.421196+0000","last_unstale":"2024-09-17T02:47:25.423510+0000","last_undegraded":"2024-09-17T02:47:25.423510+0000","last_fullsized":"2024-09-17T02:47:25.423510+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:34.197992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.400106+0000","last_change":"2024-09-17T02:47:25.400106+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.400106+0000","last_undegraded":"2024-09-17T02:47:25.400106+0000","last_fullsized":"2024-09-17T02:47:25.400106+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.402421+0000","last_change":"2024-09-17T02:47:25.401532+0000","last_active":"2024-09-17T02:47:25.402421+0000","last_peered":"2024-09-17T02:47:25.402421+0000","last_clean":"2024-09-17T02:47:25.402421+0000","last_became_active":"2024-09-17T02:47:25.400932+0000","last_became_peered":"2024-09-17T02:47:25.400932+0000","last_unstale":"2024-09-17T02:47:25.402421+0000","last_undegraded":"2024-09-17T02:47:25.402421+0000","last_fullsized":"2024-09-17T02:47:25.402421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:20.461693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.387166+0000","last_change":"2024-09-17T02:47:25.387166+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.387166+0000","last_undegraded":"2024-09-17T02:47:25.387166+0000","last_fullsized":"2024-09-17T02:47:25.387166+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.398576+0000","last_change":"2024-09-17T02:47:25.397773+0000","last_active":"2024-09-17T02:47:25.398576+0000","last_peered":"2024-09-17T02:47:25.398576+0000","last_clean":"2024-09-17T02:47:25.398576+0000","last_became_active":"2024-09-17T02:47:25.397452+0000","last_became_peered":"2024-09-17T02:47:25.397452+0000","last_unstale":"2024-09-17T02:47:25.398576+0000","last_undegraded":"2024-09-17T02:47:25.398576+0000","last_fullsized":"2024-09-17T02:47:25.398576+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:52.989426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.400455+0000","last_change":"2024-09-17T02:47:25.400455+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.400455+0000","last_undegraded":"2024-09-17T02:47:25.400455+0000","last_fullsized":"2024-09-17T02:47:25.400455+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.397993+0000","last_change":"2024-09-17T02:47:25.396451+0000","last_active":"2024-09-17T02:47:25.397993+0000","last_peered":"2024-09-17T02:47:25.397993+0000","last_clean":"2024-09-17T02:47:25.397993+0000","last_became_active":"2024-09-17T02:47:25.396146+0000","last_became_peered":"2024-09-17T02:47:25.396146+0000","last_unstale":"2024-09-17T02:47:25.397993+0000","last_undegraded":"2024-09-17T02:47:25.397993+0000","last_fullsized":"2024-09-17T02:47:25.397993+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:13.234448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.389828+0000","last_change":"2024-09-17T02:47:25.389828+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.389828+0000","last_undegraded":"2024-09-17T02:47:25.389828+0000","last_fullsized":"2024-09-17T02:47:25.389828+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.409288+0000","last_change":"2024-09-17T02:47:25.407194+0000","last_active":"2024-09-17T02:47:25.409288+0000","last_peered":"2024-09-17T02:47:25.409288+0000","last_clean":"2024-09-17T02:47:25.409288+0000","last_became_active":"2024-09-17T02:47:25.406881+0000","last_became_peered":"2024-09-17T02:47:25.406881+0000","last_unstale":"2024-09-17T02:47:25.409288+0000","last_undegraded":"2024-09-17T02:47:25.409288+0000","last_fullsized":"2024-09-17T02:47:25.409288+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:25.248822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.391110+0000","last_change":"2024-09-17T02:47:25.391110+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.391110+0000","last_undegraded":"2024-09-17T02:47:25.391110+0000","last_fullsized":"2024-09-17T02:47:25.391110+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.424283+0000","last_change":"2024-09-17T02:47:25.419788+0000","last_active":"2024-09-17T02:47:25.424283+0000","last_peered":"2024-09-17T02:47:25.424283+0000","last_clean":"2024-09-17T02:47:25.424283+0000","last_became_active":"2024-09-17T02:47:25.419102+0000","last_became_peered":"2024-09-17T02:47:25.419102+0000","last_unstale":"2024-09-17T02:47:25.424283+0000","last_undegraded":"2024-09-17T02:47:25.424283+0000","last_fullsized":"2024-09-17T02:47:25.424283+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:59.071106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.389311+0000","last_change":"2024-09-17T02:47:25.389311+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.389311+0000","last_undegraded":"2024-09-17T02:47:25.389311+0000","last_fullsized":"2024-09-17T02:47:25.389311+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.423642+0000","last_change":"2024-09-17T02:47:25.421863+0000","last_active":"2024-09-17T02:47:25.423642+0000","last_peered":"2024-09-17T02:47:25.423642+0000","last_clean":"2024-09-17T02:47:25.423642+0000","last_became_active":"2024-09-17T02:47:25.421355+0000","last_became_peered":"2024-09-17T02:47:25.421355+0000","last_unstale":"2024-09-17T02:47:25.423642+0000","last_undegraded":"2024-09-17T02:47:25.423642+0000","last_fullsized":"2024-09-17T02:47:25.423642+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:51.949665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421042+0000","last_change":"2024-09-17T02:47:26.417573+0000","last_active":"2024-09-17T02:47:27.421042+0000","last_peered":"2024-09-17T02:47:27.421042+0000","last_clean":"2024-09-17T02:47:27.421042+0000","last_became_active":"2024-09-17T02:47:26.414446+0000","last_became_peered":"2024-09-17T02:47:26.414446+0000","last_unstale":"2024-09-17T02:47:27.421042+0000","last_undegraded":"2024-09-17T02:47:27.421042+0000","last_fullsized":"2024-09-17T02:47:27.421042+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:50.565801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.423519+0000","last_change":"2024-09-17T02:47:25.413072+0000","last_active":"2024-09-17T02:47:25.423519+0000","last_peered":"2024-09-17T02:47:25.423519+0000","last_clean":"2024-09-17T02:47:25.423519+0000","last_became_active":"2024-09-17T02:47:25.412033+0000","last_became_peered":"2024-09-17T02:47:25.412033+0000","last_unstale":"2024-09-17T02:47:25.423519+0000","last_undegraded":"2024-09-17T02:47:25.423519+0000","last_fullsized":"2024-09-17T02:47:25.423519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:17.484218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.391626+0000","last_change":"2024-09-17T02:47:25.391626+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.391626+0000","last_undegraded":"2024-09-17T02:47:25.391626+0000","last_fullsized":"2024-09-17T02:47:25.391626+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.394445+0000","last_change":"2024-09-17T02:47:25.390686+0000","last_active":"2024-09-17T02:47:25.394445+0000","last_peered":"2024-09-17T02:47:25.394445+0000","last_clean":"2024-09-17T02:47:25.394445+0000","last_became_active":"2024-09-17T02:47:25.390236+0000","last_became_peered":"2024-09-17T02:47:25.390236+0000","last_unstale":"2024-09-17T02:47:25.394445+0000","last_undegraded":"2024-09-17T02:47:25.394445+0000","last_fullsized":"2024-09-17T02:47:25.394445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:11.661929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.397227+0000","last_change":"2024-09-17T02:47:25.397227+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.397227+0000","last_undegraded":"2024-09-17T02:47:25.397227+0000","last_fullsized":"2024-09-17T02:47:25.397227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.399013+0000","last_change":"2024-09-17T02:47:25.396097+0000","last_active":"2024-09-17T02:47:25.399013+0000","last_peered":"2024-09-17T02:47:25.399013+0000","last_clean":"2024-09-17T02:47:25.399013+0000","last_became_active":"2024-09-17T02:47:25.395670+0000","last_became_peered":"2024-09-17T02:47:25.395670+0000","last_unstale":"2024-09-17T02:47:25.399013+0000","last_undegraded":"2024-09-17T02:47:25.399013+0000","last_fullsized":"2024-09-17T02:47:25.399013+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:19.832862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.408383+0000","last_change":"2024-09-17T02:47:25.408383+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.408383+0000","last_undegraded":"2024-09-17T02:47:25.408383+0000","last_fullsized":"2024-09-17T02:47:25.408383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.414973+0000","last_change":"2024-09-17T02:47:25.414356+0000","last_active":"2024-09-17T02:47:25.414973+0000","last_peered":"2024-09-17T02:47:25.414973+0000","last_clean":"2024-09-17T02:47:25.414973+0000","last_became_active":"2024-09-17T02:47:25.413979+0000","last_became_peered":"2024-09-17T02:47:25.413979+0000","last_unstale":"2024-09-17T02:47:25.414973+0000","last_undegraded":"2024-09-17T02:47:25.414973+0000","last_fullsized":"2024-09-17T02:47:25.414973+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:45.715018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.400257+0000","last_change":"2024-09-17T02:47:25.400257+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.400257+0000","last_undegraded":"2024-09-17T02:47:25.400257+0000","last_fullsized":"2024-09-17T02:47:25.400257+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.395262+0000","last_change":"2024-09-17T02:47:25.392813+0000","last_active":"2024-09-17T02:47:25.395262+0000","last_peered":"2024-09-17T02:47:25.395262+0000","last_clean":"2024-09-17T02:47:25.395262+0000","last_became_active":"2024-09-17T02:47:25.392472+0000","last_became_peered":"2024-09-17T02:47:25.392472+0000","last_unstale":"2024-09-17T02:47:25.395262+0000","last_undegraded":"2024-09-17T02:47:25.395262+0000","last_fullsized":"2024-09-17T02:47:25.395262+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:34.852780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.382701+0000","last_change":"2024-09-17T02:47:25.382701+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.382701+0000","last_undegraded":"2024-09-17T02:47:25.382701+0000","last_fullsized":"2024-09-17T02:47:25.382701+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421520+0000","last_change":"2024-09-17T02:47:25.404863+0000","last_active":"2024-09-17T02:47:27.421520+0000","last_peered":"2024-09-17T02:47:27.421520+0000","last_clean":"2024-09-17T02:47:27.421520+0000","last_became_active":"2024-09-17T02:47:25.403960+0000","last_became_peered":"2024-09-17T02:47:25.403960+0000","last_unstale":"2024-09-17T02:47:27.421520+0000","last_undegraded":"2024-09-17T02:47:27.421520+0000","last_fullsized":"2024-09-17T02:47:27.421520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:38.479422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.390390+0000","last_change":"2024-09-17T02:47:25.390390+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.390390+0000","last_undegraded":"2024-09-17T02:47:25.390390+0000","last_fullsized":"2024-09-17T02:47:25.390390+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.409356+0000","last_change":"2024-09-17T02:47:25.403962+0000","last_active":"2024-09-17T02:47:25.409356+0000","last_peered":"2024-09-17T02:47:25.409356+0000","last_clean":"2024-09-17T02:47:25.409356+0000","last_became_active":"2024-09-17T02:47:25.403605+0000","last_became_peered":"2024-09-17T02:47:25.403605+0000","last_unstale":"2024-09-17T02:47:25.409356+0000","last_undegraded":"2024-09-17T02:47:25.409356+0000","last_fullsized":"2024-09-17T02:47:25.409356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:46.106338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.388513+0000","last_change":"2024-09-17T02:47:25.388513+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.388513+0000","last_undegraded":"2024-09-17T02:47:25.388513+0000","last_fullsized":"2024-09-17T02:47:25.388513+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.411901+0000","last_change":"2024-09-17T02:47:25.410837+0000","last_active":"2024-09-17T02:47:25.411901+0000","last_peered":"2024-09-17T02:47:25.411901+0000","last_clean":"2024-09-17T02:47:25.411901+0000","last_became_active":"2024-09-17T02:47:25.410644+0000","last_became_peered":"2024-09-17T02:47:25.410644+0000","last_unstale":"2024-09-17T02:47:25.411901+0000","last_undegraded":"2024-09-17T02:47:25.411901+0000","last_fullsized":"2024-09-17T02:47:25.411901+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:04.657138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.389815+0000","last_change":"2024-09-17T02:47:25.389815+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.389815+0000","last_undegraded":"2024-09-17T02:47:25.389815+0000","last_fullsized":"2024-09-17T02:47:25.389815+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.398501+0000","last_change":"2024-09-17T02:47:25.396522+0000","last_active":"2024-09-17T02:47:25.398501+0000","last_peered":"2024-09-17T02:47:25.398501+0000","last_clean":"2024-09-17T02:47:25.398501+0000","last_became_active":"2024-09-17T02:47:25.395729+0000","last_became_peered":"2024-09-17T02:47:25.395729+0000","last_unstale":"2024-09-17T02:47:25.398501+0000","last_undegraded":"2024-09-17T02:47:25.398501+0000","last_fullsized":"2024-09-17T02:47:25.398501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:54.515348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.401128+0000","last_change":"2024-09-17T02:47:25.401128+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.401128+0000","last_undegraded":"2024-09-17T02:47:25.401128+0000","last_fullsized":"2024-09-17T02:47:25.401128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421501+0000","last_change":"2024-09-17T02:47:25.404744+0000","last_active":"2024-09-17T02:47:27.421501+0000","last_peered":"2024-09-17T02:47:27.421501+0000","last_clean":"2024-09-17T02:47:27.421501+0000","last_became_active":"2024-09-17T02:47:25.403842+0000","last_became_peered":"2024-09-17T02:47:25.403842+0000","last_unstale":"2024-09-17T02:47:27.421501+0000","last_undegraded":"2024-09-17T02:47:27.421501+0000","last_fullsized":"2024-09-17T02:47:27.421501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:02.237396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":242,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.386269+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:47:25.386269+0000","last_peered":"2024-09-17T02:47:25.386269+0000","last_clean":"2024-09-17T02:47:25.386269+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:47:25.386269+0000","last_undegraded":"2024-09-17T02:47:25.386269+0000","last_fullsized":"2024-09-17T02:47:25.386269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1101824,"data_stored":503944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1683456,"data_stored":1087645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1101824,"data_stored":503944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":497277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1101824,"data_stored":503944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509525,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1695744,"data_stored":1094312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1101824,"data_stored":503944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1146880,"data_stored":519539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:47:30.537 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:47:30.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.729+0000 7f9fe872c640 1 -- 172.21.15.131:0/1600959174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe0156bd0 msgr2=0x7f9fe00cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.729+0000 7f9fe872c640 1 --2- 172.21.15.131:0/1600959174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe0156bd0 0x7f9fe00cca70 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f9fe00b1da0 tx=0x7f9fd801cd00 comp rx=0 tx=0).stop 2024-09-17T02:47:30.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.729+0000 7f9fe872c640 1 -- 172.21.15.131:0/1600959174 shutdown_connections 2024-09-17T02:47:30.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.729+0000 7f9fe872c640 1 --2- 172.21.15.131:0/1600959174 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9fe01601a0 0x7f9fe01605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.729+0000 7f9fe872c640 1 --2- 172.21.15.131:0/1600959174 >> v2:172.21.15.189:3300/0 conn(0x7f9fe00b41a0 0x7f9fe00b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.729+0000 7f9fe872c640 1 --2- 172.21.15.131:0/1600959174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe0156bd0 0x7f9fe00cca70 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.729+0000 7f9fe872c640 1 -- 172.21.15.131:0/1600959174 >> 172.21.15.131:0/1600959174 conn(0x7f9fe00cf150 msgr2=0x7f9fe00cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:30.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.729+0000 7f9fe872c640 1 -- 172.21.15.131:0/1600959174 shutdown_connections 2024-09-17T02:47:30.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.729+0000 7f9fe872c640 1 -- 172.21.15.131:0/1600959174 wait complete. 2024-09-17T02:47:30.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe872c640 1 Processor -- start 2024-09-17T02:47:30.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe872c640 1 -- start start 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe872c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe00b41a0 0x7f9fe00b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe872c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9fe0156bd0 0x7f9fe00b7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe872c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe01601a0 0x7f9fe00cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe872c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe0168b40 con 0x7f9fe01601a0 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe872c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fe00b2b60 con 0x7f9fe00b41a0 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe872c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9fe0168cc0 con 0x7f9fe0156bd0 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe6cc9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe01601a0 0x7f9fe00cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe6cc9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe01601a0 0x7f9fe00cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50508/0 (socket says 172.21.15.131:50508) 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe6cc9640 1 -- 172.21.15.131:0/2008565174 learned_addr learned my addr 172.21.15.131:0/2008565174 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe64c8640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe00b41a0 0x7f9fe00b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe5cc7640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9fe0156bd0 0x7f9fe00b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:30.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe6cc9640 1 -- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9fe0156bd0 msgr2=0x7f9fe00b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe6cc9640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9fe0156bd0 0x7f9fe00b7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe6cc9640 1 -- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe00b41a0 msgr2=0x7f9fe00b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe6cc9640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe00b41a0 0x7f9fe00b5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe6cc9640 1 -- 172.21.15.131:0/2008565174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fe0155750 con 0x7f9fe01601a0 2024-09-17T02:47:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe64c8640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe00b41a0 0x7f9fe00b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:47:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe6cc9640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe01601a0 0x7f9fe00cb9e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000d660 tx=0x7f9fd000db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fdeffd640 1 -- 172.21.15.131:0/2008565174 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0017070 con 0x7f9fe01601a0 2024-09-17T02:47:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fdeffd640 1 -- 172.21.15.131:0/2008565174 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fd0005310 con 0x7f9fe01601a0 2024-09-17T02:47:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe872c640 1 -- 172.21.15.131:0/2008565174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fe0155290 con 0x7f9fe01601a0 2024-09-17T02:47:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fdeffd640 1 -- 172.21.15.131:0/2008565174 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0009820 con 0x7f9fe01601a0 2024-09-17T02:47:30.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.733+0000 7f9fe872c640 1 -- 172.21.15.131:0/2008565174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fe0154fb0 con 0x7f9fe01601a0 2024-09-17T02:47:30.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.737+0000 7f9fe872c640 1 -- 172.21.15.131:0/2008565174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fe00cca70 con 0x7f9fe01601a0 2024-09-17T02:47:30.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.737+0000 7f9fdeffd640 1 -- 172.21.15.131:0/2008565174 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9fd00099c0 con 0x7f9fe01601a0 2024-09-17T02:47:30.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.737+0000 7f9fdeffd640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fcc073200 0x7f9fcc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:30.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.737+0000 7f9fdeffd640 1 -- 172.21.15.131:0/2008565174 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f9fd0094390 con 0x7f9fe01601a0 2024-09-17T02:47:30.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.737+0000 7f9fe64c8640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fcc073200 0x7f9fcc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:30.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.737+0000 7f9fe64c8640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fcc073200 0x7f9fcc075710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80af000 tx=0x7f9fd801b5a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:30.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.741+0000 7f9fdeffd640 1 -- 172.21.15.131:0/2008565174 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9fd005f340 con 0x7f9fe01601a0 2024-09-17T02:47:30.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.909+0000 7f9fe872c640 1 -- 172.21.15.131:0/2008565174 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9fe01554e0 con 0x7f9fcc073200 2024-09-17T02:47:30.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.921+0000 7f9fdeffd640 1 -- 172.21.15.131:0/2008565174 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247465 (secure 0 0 0) 0x7f9fe01554e0 con 0x7f9fcc073200 2024-09-17T02:47:30.920 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:30.921 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:47:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 -- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fcc073200 msgr2=0x7f9fcc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fcc073200 0x7f9fcc075710 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9fd80af000 tx=0x7f9fd801b5a0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 -- 172.21.15.131:0/2008565174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe01601a0 msgr2=0x7f9fe00cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe01601a0 0x7f9fe00cb9e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f9fd000d660 tx=0x7f9fd000db20 comp rx=0 tx=0).stop 2024-09-17T02:47:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 -- 172.21.15.131:0/2008565174 shutdown_connections 2024-09-17T02:47:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fcc073200 0x7f9fcc075710 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fe01601a0 0x7f9fe00cb9e0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9fe0156bd0 0x7f9fe00b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 --2- 172.21.15.131:0/2008565174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fe00b41a0 0x7f9fe00b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:30.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 -- 172.21.15.131:0/2008565174 >> 172.21.15.131:0/2008565174 conn(0x7f9fe00cf150 msgr2=0x7f9fe00c8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:30.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 -- 172.21.15.131:0/2008565174 shutdown_connections 2024-09-17T02:47:30.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:30.925+0000 7f9fe872c640 1 -- 172.21.15.131:0/2008565174 wait complete. 2024-09-17T02:47:30.943 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-17T02:47:29.275711+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591980,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":107,"num_write_kb":1631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259328,"kb_used_data":9788,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715392,"statfs":{"total":773094113280,"available":772828561408,"internally_reserved":0,"allocated":10022912,"data_stored":5214549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.910845"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422667+0000","last_change":"2024-09-17T02:47:26.451633+0000","last_active":"2024-09-17T02:47:27.422667+0000","last_peered":"2024-09-17T02:47:27.422667+0000","last_clean":"2024-09-17T02:47:27.422667+0000","last_became_active":"2024-09-17T02:47:26.450812+0000","last_became_peered":"2024-09-17T02:47:26.450812+0000","last_unstale":"2024-09-17T02:47:27.422667+0000","last_undegraded":"2024-09-17T02:47:27.422667+0000","last_fullsized":"2024-09-17T02:47:27.422667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:06.130998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.404861+0000","last_change":"2024-09-17T02:47:25.404861+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.404861+0000","last_undegraded":"2024-09-17T02:47:25.404861+0000","last_fullsized":"2024-09-17T02:47:25.404861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.392716+0000","last_change":"2024-09-17T02:47:25.392716+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.392716+0000","last_undegraded":"2024-09-17T02:47:25.392716+0000","last_fullsized":"2024-09-17T02:47:25.392716+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.394859+0000","last_change":"2024-09-17T02:47:25.394859+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.394859+0000","last_undegraded":"2024-09-17T02:47:25.394859+0000","last_fullsized":"2024-09-17T02:47:25.394859+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.404469+0000","last_change":"2024-09-17T02:47:25.404469+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.404469+0000","last_undegraded":"2024-09-17T02:47:25.404469+0000","last_fullsized":"2024-09-17T02:47:25.404469+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.405928+0000","last_change":"2024-09-17T02:47:25.405928+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.405928+0000","last_undegraded":"2024-09-17T02:47:25.405928+0000","last_fullsized":"2024-09-17T02:47:25.405928+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.390435+0000","last_change":"2024-09-17T02:47:25.390435+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.390435+0000","last_undegraded":"2024-09-17T02:47:25.390435+0000","last_fullsized":"2024-09-17T02:47:25.390435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.402210+0000","last_change":"2024-09-17T02:47:25.402210+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.402210+0000","last_undegraded":"2024-09-17T02:47:25.402210+0000","last_fullsized":"2024-09-17T02:47:25.402210+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.386493+0000","last_change":"2024-09-17T02:47:25.386493+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.386493+0000","last_undegraded":"2024-09-17T02:47:25.386493+0000","last_fullsized":"2024-09-17T02:47:25.386493+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.392764+0000","last_change":"2024-09-17T02:47:25.392764+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.392764+0000","last_undegraded":"2024-09-17T02:47:25.392764+0000","last_fullsized":"2024-09-17T02:47:25.392764+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.388466+0000","last_change":"2024-09-17T02:47:25.388466+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.388466+0000","last_undegraded":"2024-09-17T02:47:25.388466+0000","last_fullsized":"2024-09-17T02:47:25.388466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.393605+0000","last_change":"2024-09-17T02:47:25.393605+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.393605+0000","last_undegraded":"2024-09-17T02:47:25.393605+0000","last_fullsized":"2024-09-17T02:47:25.393605+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.410168+0000","last_change":"2024-09-17T02:47:25.410168+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.410168+0000","last_undegraded":"2024-09-17T02:47:25.410168+0000","last_fullsized":"2024-09-17T02:47:25.410168+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.391149+0000","last_change":"2024-09-17T02:47:25.391149+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.391149+0000","last_undegraded":"2024-09-17T02:47:25.391149+0000","last_fullsized":"2024-09-17T02:47:25.391149+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.392004+0000","last_change":"2024-09-17T02:47:25.392004+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.392004+0000","last_undegraded":"2024-09-17T02:47:25.392004+0000","last_fullsized":"2024-09-17T02:47:25.392004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.400977+0000","last_change":"2024-09-17T02:47:25.400977+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.400977+0000","last_undegraded":"2024-09-17T02:47:25.400977+0000","last_fullsized":"2024-09-17T02:47:25.400977+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.406030+0000","last_change":"2024-09-17T02:47:25.406030+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.406030+0000","last_undegraded":"2024-09-17T02:47:25.406030+0000","last_fullsized":"2024-09-17T02:47:25.406030+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.391330+0000","last_change":"2024-09-17T02:47:25.391330+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.391330+0000","last_undegraded":"2024-09-17T02:47:25.391330+0000","last_fullsized":"2024-09-17T02:47:25.391330+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.392835+0000","last_change":"2024-09-17T02:47:25.392835+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.392835+0000","last_undegraded":"2024-09-17T02:47:25.392835+0000","last_fullsized":"2024-09-17T02:47:25.392835+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.384257+0000","last_change":"2024-09-17T02:47:25.384257+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.384257+0000","last_undegraded":"2024-09-17T02:47:25.384257+0000","last_fullsized":"2024-09-17T02:47:25.384257+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.403781+0000","last_change":"2024-09-17T02:47:25.403781+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.403781+0000","last_undegraded":"2024-09-17T02:47:25.403781+0000","last_fullsized":"2024-09-17T02:47:25.403781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.390040+0000","last_change":"2024-09-17T02:47:25.390040+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.390040+0000","last_undegraded":"2024-09-17T02:47:25.390040+0000","last_fullsized":"2024-09-17T02:47:25.390040+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.403689+0000","last_change":"2024-09-17T02:47:25.403689+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.403689+0000","last_undegraded":"2024-09-17T02:47:25.403689+0000","last_fullsized":"2024-09-17T02:47:25.403689+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.418593+0000","last_change":"2024-09-17T02:47:26.438730+0000","last_active":"2024-09-17T02:47:27.418593+0000","last_peered":"2024-09-17T02:47:27.418593+0000","last_clean":"2024-09-17T02:47:27.418593+0000","last_became_active":"2024-09-17T02:47:26.420076+0000","last_became_peered":"2024-09-17T02:47:26.420076+0000","last_unstale":"2024-09-17T02:47:27.418593+0000","last_undegraded":"2024-09-17T02:47:27.418593+0000","last_fullsized":"2024-09-17T02:47:27.418593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:12.606293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.403690+0000","last_change":"2024-09-17T02:47:25.403690+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.403690+0000","last_undegraded":"2024-09-17T02:47:25.403690+0000","last_fullsized":"2024-09-17T02:47:25.403690+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.402945+0000","last_change":"2024-09-17T02:47:25.402945+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.402945+0000","last_undegraded":"2024-09-17T02:47:25.402945+0000","last_fullsized":"2024-09-17T02:47:25.402945+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.391318+0000","last_change":"2024-09-17T02:47:25.391318+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.391318+0000","last_undegraded":"2024-09-17T02:47:25.391318+0000","last_fullsized":"2024-09-17T02:47:25.391318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.409435+0000","last_change":"2024-09-17T02:47:25.409435+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.409435+0000","last_undegraded":"2024-09-17T02:47:25.409435+0000","last_fullsized":"2024-09-17T02:47:25.409435+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.402326+0000","last_change":"2024-09-17T02:47:25.402326+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.402326+0000","last_undegraded":"2024-09-17T02:47:25.402326+0000","last_fullsized":"2024-09-17T02:47:25.402326+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.401754+0000","last_change":"2024-09-17T02:47:25.401754+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.401754+0000","last_undegraded":"2024-09-17T02:47:25.401754+0000","last_fullsized":"2024-09-17T02:47:25.401754+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.392184+0000","last_change":"2024-09-17T02:47:25.392184+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.392184+0000","last_undegraded":"2024-09-17T02:47:25.392184+0000","last_fullsized":"2024-09-17T02:47:25.392184+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.399408+0000","last_change":"2024-09-17T02:47:25.399408+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.399408+0000","last_undegraded":"2024-09-17T02:47:25.399408+0000","last_fullsized":"2024-09-17T02:47:25.399408+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421125+0000","last_change":"2024-09-17T02:47:26.419153+0000","last_active":"2024-09-17T02:47:27.421125+0000","last_peered":"2024-09-17T02:47:27.421125+0000","last_clean":"2024-09-17T02:47:27.421125+0000","last_became_active":"2024-09-17T02:47:26.418280+0000","last_became_peered":"2024-09-17T02:47:26.418280+0000","last_unstale":"2024-09-17T02:47:27.421125+0000","last_undegraded":"2024-09-17T02:47:27.421125+0000","last_fullsized":"2024-09-17T02:47:27.421125+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:17.220457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.409473+0000","last_change":"2024-09-17T02:47:25.404962+0000","last_active":"2024-09-17T02:47:25.409473+0000","last_peered":"2024-09-17T02:47:25.409473+0000","last_clean":"2024-09-17T02:47:25.409473+0000","last_became_active":"2024-09-17T02:47:25.404574+0000","last_became_peered":"2024-09-17T02:47:25.404574+0000","last_unstale":"2024-09-17T02:47:25.409473+0000","last_undegraded":"2024-09-17T02:47:25.409473+0000","last_fullsized":"2024-09-17T02:47:25.409473+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:00.198104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.382792+0000","last_change":"2024-09-17T02:47:25.382792+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.382792+0000","last_undegraded":"2024-09-17T02:47:25.382792+0000","last_fullsized":"2024-09-17T02:47:25.382792+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.398031+0000","last_change":"2024-09-17T02:47:25.394418+0000","last_active":"2024-09-17T02:47:25.398031+0000","last_peered":"2024-09-17T02:47:25.398031+0000","last_clean":"2024-09-17T02:47:25.398031+0000","last_became_active":"2024-09-17T02:47:25.393974+0000","last_became_peered":"2024-09-17T02:47:25.393974+0000","last_unstale":"2024-09-17T02:47:25.398031+0000","last_undegraded":"2024-09-17T02:47:25.398031+0000","last_fullsized":"2024-09-17T02:47:25.398031+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:37.931308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.385646+0000","last_change":"2024-09-17T02:47:25.385646+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.385646+0000","last_undegraded":"2024-09-17T02:47:25.385646+0000","last_fullsized":"2024-09-17T02:47:25.385646+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.420057+0000","last_change":"2024-09-17T02:47:25.418775+0000","last_active":"2024-09-17T02:47:25.420057+0000","last_peered":"2024-09-17T02:47:25.420057+0000","last_clean":"2024-09-17T02:47:25.420057+0000","last_became_active":"2024-09-17T02:47:25.418458+0000","last_became_peered":"2024-09-17T02:47:25.418458+0000","last_unstale":"2024-09-17T02:47:25.420057+0000","last_undegraded":"2024-09-17T02:47:25.420057+0000","last_fullsized":"2024-09-17T02:47:25.420057+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:53.327706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.390407+0000","last_change":"2024-09-17T02:47:25.390407+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.390407+0000","last_undegraded":"2024-09-17T02:47:25.390407+0000","last_fullsized":"2024-09-17T02:47:25.390407+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.413571+0000","last_change":"2024-09-17T02:47:25.412944+0000","last_active":"2024-09-17T02:47:25.413571+0000","last_peered":"2024-09-17T02:47:25.413571+0000","last_clean":"2024-09-17T02:47:25.413571+0000","last_became_active":"2024-09-17T02:47:25.412515+0000","last_became_peered":"2024-09-17T02:47:25.412515+0000","last_unstale":"2024-09-17T02:47:25.413571+0000","last_undegraded":"2024-09-17T02:47:25.413571+0000","last_fullsized":"2024-09-17T02:47:25.413571+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:12:11.200621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.398647+0000","last_change":"2024-09-17T02:47:25.398647+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.398647+0000","last_undegraded":"2024-09-17T02:47:25.398647+0000","last_fullsized":"2024-09-17T02:47:25.398647+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.412709+0000","last_change":"2024-09-17T02:47:25.411938+0000","last_active":"2024-09-17T02:47:25.412709+0000","last_peered":"2024-09-17T02:47:25.412709+0000","last_clean":"2024-09-17T02:47:25.412709+0000","last_became_active":"2024-09-17T02:47:25.411486+0000","last_became_peered":"2024-09-17T02:47:25.411486+0000","last_unstale":"2024-09-17T02:47:25.412709+0000","last_undegraded":"2024-09-17T02:47:25.412709+0000","last_fullsized":"2024-09-17T02:47:25.412709+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:34.937753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.385519+0000","last_change":"2024-09-17T02:47:25.385519+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.385519+0000","last_undegraded":"2024-09-17T02:47:25.385519+0000","last_fullsized":"2024-09-17T02:47:25.385519+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.424202+0000","last_change":"2024-09-17T02:47:25.422380+0000","last_active":"2024-09-17T02:47:25.424202+0000","last_peered":"2024-09-17T02:47:25.424202+0000","last_clean":"2024-09-17T02:47:25.424202+0000","last_became_active":"2024-09-17T02:47:25.421906+0000","last_became_peered":"2024-09-17T02:47:25.421906+0000","last_unstale":"2024-09-17T02:47:25.424202+0000","last_undegraded":"2024-09-17T02:47:25.424202+0000","last_fullsized":"2024-09-17T02:47:25.424202+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:01:53.113825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.383087+0000","last_change":"2024-09-17T02:47:25.383087+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.383087+0000","last_undegraded":"2024-09-17T02:47:25.383087+0000","last_fullsized":"2024-09-17T02:47:25.383087+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.414536+0000","last_change":"2024-09-17T02:47:25.410268+0000","last_active":"2024-09-17T02:47:25.414536+0000","last_peered":"2024-09-17T02:47:25.414536+0000","last_clean":"2024-09-17T02:47:25.414536+0000","last_became_active":"2024-09-17T02:47:25.409433+0000","last_became_peered":"2024-09-17T02:47:25.409433+0000","last_unstale":"2024-09-17T02:47:25.414536+0000","last_undegraded":"2024-09-17T02:47:25.414536+0000","last_fullsized":"2024-09-17T02:47:25.414536+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:05:07.402409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.420382+0000","last_change":"2024-09-17T02:47:26.410976+0000","last_active":"2024-09-17T02:47:27.420382+0000","last_peered":"2024-09-17T02:47:27.420382+0000","last_clean":"2024-09-17T02:47:27.420382+0000","last_became_active":"2024-09-17T02:47:26.407305+0000","last_became_peered":"2024-09-17T02:47:26.407305+0000","last_unstale":"2024-09-17T02:47:27.420382+0000","last_undegraded":"2024-09-17T02:47:27.420382+0000","last_fullsized":"2024-09-17T02:47:27.420382+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:53.755378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.423195+0000","last_change":"2024-09-17T02:47:25.420193+0000","last_active":"2024-09-17T02:47:25.423195+0000","last_peered":"2024-09-17T02:47:25.423195+0000","last_clean":"2024-09-17T02:47:25.423195+0000","last_became_active":"2024-09-17T02:47:25.419700+0000","last_became_peered":"2024-09-17T02:47:25.419700+0000","last_unstale":"2024-09-17T02:47:25.423195+0000","last_undegraded":"2024-09-17T02:47:25.423195+0000","last_fullsized":"2024-09-17T02:47:25.423195+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:49:30.791406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.402350+0000","last_change":"2024-09-17T02:47:25.402350+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.402350+0000","last_undegraded":"2024-09-17T02:47:25.402350+0000","last_fullsized":"2024-09-17T02:47:25.402350+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.420772+0000","last_change":"2024-09-17T02:47:25.419435+0000","last_active":"2024-09-17T02:47:25.420772+0000","last_peered":"2024-09-17T02:47:25.420772+0000","last_clean":"2024-09-17T02:47:25.420772+0000","last_became_active":"2024-09-17T02:47:25.418955+0000","last_became_peered":"2024-09-17T02:47:25.418955+0000","last_unstale":"2024-09-17T02:47:25.420772+0000","last_undegraded":"2024-09-17T02:47:25.420772+0000","last_fullsized":"2024-09-17T02:47:25.420772+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:01:13.708227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.387848+0000","last_change":"2024-09-17T02:47:25.387848+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.387848+0000","last_undegraded":"2024-09-17T02:47:25.387848+0000","last_fullsized":"2024-09-17T02:47:25.387848+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.416359+0000","last_change":"2024-09-17T02:47:25.414624+0000","last_active":"2024-09-17T02:47:25.416359+0000","last_peered":"2024-09-17T02:47:25.416359+0000","last_clean":"2024-09-17T02:47:25.416359+0000","last_became_active":"2024-09-17T02:47:25.413560+0000","last_became_peered":"2024-09-17T02:47:25.413560+0000","last_unstale":"2024-09-17T02:47:25.416359+0000","last_undegraded":"2024-09-17T02:47:25.416359+0000","last_fullsized":"2024-09-17T02:47:25.416359+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:23:20.187685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422254+0000","last_change":"2024-09-17T02:47:26.441675+0000","last_active":"2024-09-17T02:47:27.422254+0000","last_peered":"2024-09-17T02:47:27.422254+0000","last_clean":"2024-09-17T02:47:27.422254+0000","last_became_active":"2024-09-17T02:47:26.437413+0000","last_became_peered":"2024-09-17T02:47:26.437413+0000","last_unstale":"2024-09-17T02:47:27.422254+0000","last_undegraded":"2024-09-17T02:47:27.422254+0000","last_fullsized":"2024-09-17T02:47:27.422254+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:50.459729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.420604+0000","last_change":"2024-09-17T02:47:25.419022+0000","last_active":"2024-09-17T02:47:25.420604+0000","last_peered":"2024-09-17T02:47:25.420604+0000","last_clean":"2024-09-17T02:47:25.420604+0000","last_became_active":"2024-09-17T02:47:25.418540+0000","last_became_peered":"2024-09-17T02:47:25.418540+0000","last_unstale":"2024-09-17T02:47:25.420604+0000","last_undegraded":"2024-09-17T02:47:25.420604+0000","last_fullsized":"2024-09-17T02:47:25.420604+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:26.753403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.398901+0000","last_change":"2024-09-17T02:47:25.398901+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.398901+0000","last_undegraded":"2024-09-17T02:47:25.398901+0000","last_fullsized":"2024-09-17T02:47:25.398901+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"91'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421338+0000","last_change":"2024-09-17T02:47:25.418647+0000","last_active":"2024-09-17T02:47:27.421338+0000","last_peered":"2024-09-17T02:47:27.421338+0000","last_clean":"2024-09-17T02:47:27.421338+0000","last_became_active":"2024-09-17T02:47:25.416994+0000","last_became_peered":"2024-09-17T02:47:25.416994+0000","last_unstale":"2024-09-17T02:47:27.421338+0000","last_undegraded":"2024-09-17T02:47:27.421338+0000","last_fullsized":"2024-09-17T02:47:27.421338+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:58.768722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.418667+0000","last_change":"2024-09-17T02:47:26.419193+0000","last_active":"2024-09-17T02:47:27.418667+0000","last_peered":"2024-09-17T02:47:27.418667+0000","last_clean":"2024-09-17T02:47:27.418667+0000","last_became_active":"2024-09-17T02:47:26.418643+0000","last_became_peered":"2024-09-17T02:47:26.418643+0000","last_unstale":"2024-09-17T02:47:27.418667+0000","last_undegraded":"2024-09-17T02:47:27.418667+0000","last_fullsized":"2024-09-17T02:47:27.418667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:46.698904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.420735+0000","last_change":"2024-09-17T02:47:25.418902+0000","last_active":"2024-09-17T02:47:25.420735+0000","last_peered":"2024-09-17T02:47:25.420735+0000","last_clean":"2024-09-17T02:47:25.420735+0000","last_became_active":"2024-09-17T02:47:25.418568+0000","last_became_peered":"2024-09-17T02:47:25.418568+0000","last_unstale":"2024-09-17T02:47:25.420735+0000","last_undegraded":"2024-09-17T02:47:25.420735+0000","last_fullsized":"2024-09-17T02:47:25.420735+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:10.152523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.401220+0000","last_change":"2024-09-17T02:47:25.401220+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.401220+0000","last_undegraded":"2024-09-17T02:47:25.401220+0000","last_fullsized":"2024-09-17T02:47:25.401220+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419403+0000","last_change":"2024-09-17T02:47:25.418948+0000","last_active":"2024-09-17T02:47:27.419403+0000","last_peered":"2024-09-17T02:47:27.419403+0000","last_clean":"2024-09-17T02:47:27.419403+0000","last_became_active":"2024-09-17T02:47:25.417436+0000","last_became_peered":"2024-09-17T02:47:25.417436+0000","last_unstale":"2024-09-17T02:47:27.419403+0000","last_undegraded":"2024-09-17T02:47:27.419403+0000","last_fullsized":"2024-09-17T02:47:27.419403+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:25.386796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419176+0000","last_change":"2024-09-17T02:47:26.438873+0000","last_active":"2024-09-17T02:47:27.419176+0000","last_peered":"2024-09-17T02:47:27.419176+0000","last_clean":"2024-09-17T02:47:27.419176+0000","last_became_active":"2024-09-17T02:47:26.436291+0000","last_became_peered":"2024-09-17T02:47:26.436291+0000","last_unstale":"2024-09-17T02:47:27.419176+0000","last_undegraded":"2024-09-17T02:47:27.419176+0000","last_fullsized":"2024-09-17T02:47:27.419176+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:56.037102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.423108+0000","last_change":"2024-09-17T02:47:25.411837+0000","last_active":"2024-09-17T02:47:27.423108+0000","last_peered":"2024-09-17T02:47:27.423108+0000","last_clean":"2024-09-17T02:47:27.423108+0000","last_became_active":"2024-09-17T02:47:25.411299+0000","last_became_peered":"2024-09-17T02:47:25.411299+0000","last_unstale":"2024-09-17T02:47:27.423108+0000","last_undegraded":"2024-09-17T02:47:27.423108+0000","last_fullsized":"2024-09-17T02:47:27.423108+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:23.951343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.397868+0000","last_change":"2024-09-17T02:47:25.397868+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.397868+0000","last_undegraded":"2024-09-17T02:47:25.397868+0000","last_fullsized":"2024-09-17T02:47:25.397868+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.423510+0000","last_change":"2024-09-17T02:47:25.421873+0000","last_active":"2024-09-17T02:47:25.423510+0000","last_peered":"2024-09-17T02:47:25.423510+0000","last_clean":"2024-09-17T02:47:25.423510+0000","last_became_active":"2024-09-17T02:47:25.421196+0000","last_became_peered":"2024-09-17T02:47:25.421196+0000","last_unstale":"2024-09-17T02:47:25.423510+0000","last_undegraded":"2024-09-17T02:47:25.423510+0000","last_fullsized":"2024-09-17T02:47:25.423510+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:34.197992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.400106+0000","last_change":"2024-09-17T02:47:25.400106+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.400106+0000","last_undegraded":"2024-09-17T02:47:25.400106+0000","last_fullsized":"2024-09-17T02:47:25.400106+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.402421+0000","last_change":"2024-09-17T02:47:25.401532+0000","last_active":"2024-09-17T02:47:25.402421+0000","last_peered":"2024-09-17T02:47:25.402421+0000","last_clean":"2024-09-17T02:47:25.402421+0000","last_became_active":"2024-09-17T02:47:25.400932+0000","last_became_peered":"2024-09-17T02:47:25.400932+0000","last_unstale":"2024-09-17T02:47:25.402421+0000","last_undegraded":"2024-09-17T02:47:25.402421+0000","last_fullsized":"2024-09-17T02:47:25.402421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:20.461693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.387166+0000","last_change":"2024-09-17T02:47:25.387166+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.387166+0000","last_undegraded":"2024-09-17T02:47:25.387166+0000","last_fullsized":"2024-09-17T02:47:25.387166+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.398576+0000","last_change":"2024-09-17T02:47:25.397773+0000","last_active":"2024-09-17T02:47:25.398576+0000","last_peered":"2024-09-17T02:47:25.398576+0000","last_clean":"2024-09-17T02:47:25.398576+0000","last_became_active":"2024-09-17T02:47:25.397452+0000","last_became_peered":"2024-09-17T02:47:25.397452+0000","last_unstale":"2024-09-17T02:47:25.398576+0000","last_undegraded":"2024-09-17T02:47:25.398576+0000","last_fullsized":"2024-09-17T02:47:25.398576+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:52.989426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.400455+0000","last_change":"2024-09-17T02:47:25.400455+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.400455+0000","last_undegraded":"2024-09-17T02:47:25.400455+0000","last_fullsized":"2024-09-17T02:47:25.400455+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.397993+0000","last_change":"2024-09-17T02:47:25.396451+0000","last_active":"2024-09-17T02:47:25.397993+0000","last_peered":"2024-09-17T02:47:25.397993+0000","last_clean":"2024-09-17T02:47:25.397993+0000","last_became_active":"2024-09-17T02:47:25.396146+0000","last_became_peered":"2024-09-17T02:47:25.396146+0000","last_unstale":"2024-09-17T02:47:25.397993+0000","last_undegraded":"2024-09-17T02:47:25.397993+0000","last_fullsized":"2024-09-17T02:47:25.397993+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:13.234448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.389828+0000","last_change":"2024-09-17T02:47:25.389828+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.389828+0000","last_undegraded":"2024-09-17T02:47:25.389828+0000","last_fullsized":"2024-09-17T02:47:25.389828+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.409288+0000","last_change":"2024-09-17T02:47:25.407194+0000","last_active":"2024-09-17T02:47:25.409288+0000","last_peered":"2024-09-17T02:47:25.409288+0000","last_clean":"2024-09-17T02:47:25.409288+0000","last_became_active":"2024-09-17T02:47:25.406881+0000","last_became_peered":"2024-09-17T02:47:25.406881+0000","last_unstale":"2024-09-17T02:47:25.409288+0000","last_undegraded":"2024-09-17T02:47:25.409288+0000","last_fullsized":"2024-09-17T02:47:25.409288+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:25.248822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.391110+0000","last_change":"2024-09-17T02:47:25.391110+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.391110+0000","last_undegraded":"2024-09-17T02:47:25.391110+0000","last_fullsized":"2024-09-17T02:47:25.391110+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.424283+0000","last_change":"2024-09-17T02:47:25.419788+0000","last_active":"2024-09-17T02:47:25.424283+0000","last_peered":"2024-09-17T02:47:25.424283+0000","last_clean":"2024-09-17T02:47:25.424283+0000","last_became_active":"2024-09-17T02:47:25.419102+0000","last_became_peered":"2024-09-17T02:47:25.419102+0000","last_unstale":"2024-09-17T02:47:25.424283+0000","last_undegraded":"2024-09-17T02:47:25.424283+0000","last_fullsized":"2024-09-17T02:47:25.424283+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:59.071106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.389311+0000","last_change":"2024-09-17T02:47:25.389311+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.389311+0000","last_undegraded":"2024-09-17T02:47:25.389311+0000","last_fullsized":"2024-09-17T02:47:25.389311+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.423642+0000","last_change":"2024-09-17T02:47:25.421863+0000","last_active":"2024-09-17T02:47:25.423642+0000","last_peered":"2024-09-17T02:47:25.423642+0000","last_clean":"2024-09-17T02:47:25.423642+0000","last_became_active":"2024-09-17T02:47:25.421355+0000","last_became_peered":"2024-09-17T02:47:25.421355+0000","last_unstale":"2024-09-17T02:47:25.423642+0000","last_undegraded":"2024-09-17T02:47:25.423642+0000","last_fullsized":"2024-09-17T02:47:25.423642+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:51.949665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421042+0000","last_change":"2024-09-17T02:47:26.417573+0000","last_active":"2024-09-17T02:47:27.421042+0000","last_peered":"2024-09-17T02:47:27.421042+0000","last_clean":"2024-09-17T02:47:27.421042+0000","last_became_active":"2024-09-17T02:47:26.414446+0000","last_became_peered":"2024-09-17T02:47:26.414446+0000","last_unstale":"2024-09-17T02:47:27.421042+0000","last_undegraded":"2024-09-17T02:47:27.421042+0000","last_fullsized":"2024-09-17T02:47:27.421042+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:50.565801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.423519+0000","last_change":"2024-09-17T02:47:25.413072+0000","last_active":"2024-09-17T02:47:25.423519+0000","last_peered":"2024-09-17T02:47:25.423519+0000","last_clean":"2024-09-17T02:47:25.423519+0000","last_became_active":"2024-09-17T02:47:25.412033+0000","last_became_peered":"2024-09-17T02:47:25.412033+0000","last_unstale":"2024-09-17T02:47:25.423519+0000","last_undegraded":"2024-09-17T02:47:25.423519+0000","last_fullsized":"2024-09-17T02:47:25.423519+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:17.484218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.391626+0000","last_change":"2024-09-17T02:47:25.391626+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.391626+0000","last_undegraded":"2024-09-17T02:47:25.391626+0000","last_fullsized":"2024-09-17T02:47:25.391626+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.394445+0000","last_change":"2024-09-17T02:47:25.390686+0000","last_active":"2024-09-17T02:47:25.394445+0000","last_peered":"2024-09-17T02:47:25.394445+0000","last_clean":"2024-09-17T02:47:25.394445+0000","last_became_active":"2024-09-17T02:47:25.390236+0000","last_became_peered":"2024-09-17T02:47:25.390236+0000","last_unstale":"2024-09-17T02:47:25.394445+0000","last_undegraded":"2024-09-17T02:47:25.394445+0000","last_fullsized":"2024-09-17T02:47:25.394445+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:11.661929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.397227+0000","last_change":"2024-09-17T02:47:25.397227+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.397227+0000","last_undegraded":"2024-09-17T02:47:25.397227+0000","last_fullsized":"2024-09-17T02:47:25.397227+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.399013+0000","last_change":"2024-09-17T02:47:25.396097+0000","last_active":"2024-09-17T02:47:25.399013+0000","last_peered":"2024-09-17T02:47:25.399013+0000","last_clean":"2024-09-17T02:47:25.399013+0000","last_became_active":"2024-09-17T02:47:25.395670+0000","last_became_peered":"2024-09-17T02:47:25.395670+0000","last_unstale":"2024-09-17T02:47:25.399013+0000","last_undegraded":"2024-09-17T02:47:25.399013+0000","last_fullsized":"2024-09-17T02:47:25.399013+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:19.832862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.408383+0000","last_change":"2024-09-17T02:47:25.408383+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.408383+0000","last_undegraded":"2024-09-17T02:47:25.408383+0000","last_fullsized":"2024-09-17T02:47:25.408383+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.414973+0000","last_change":"2024-09-17T02:47:25.414356+0000","last_active":"2024-09-17T02:47:25.414973+0000","last_peered":"2024-09-17T02:47:25.414973+0000","last_clean":"2024-09-17T02:47:25.414973+0000","last_became_active":"2024-09-17T02:47:25.413979+0000","last_became_peered":"2024-09-17T02:47:25.413979+0000","last_unstale":"2024-09-17T02:47:25.414973+0000","last_undegraded":"2024-09-17T02:47:25.414973+0000","last_fullsized":"2024-09-17T02:47:25.414973+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:45.715018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.400257+0000","last_change":"2024-09-17T02:47:25.400257+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.400257+0000","last_undegraded":"2024-09-17T02:47:25.400257+0000","last_fullsized":"2024-09-17T02:47:25.400257+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.395262+0000","last_change":"2024-09-17T02:47:25.392813+0000","last_active":"2024-09-17T02:47:25.395262+0000","last_peered":"2024-09-17T02:47:25.395262+0000","last_clean":"2024-09-17T02:47:25.395262+0000","last_became_active":"2024-09-17T02:47:25.392472+0000","last_became_peered":"2024-09-17T02:47:25.392472+0000","last_unstale":"2024-09-17T02:47:25.395262+0000","last_undegraded":"2024-09-17T02:47:25.395262+0000","last_fullsized":"2024-09-17T02:47:25.395262+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:34.852780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.382701+0000","last_change":"2024-09-17T02:47:25.382701+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.382701+0000","last_undegraded":"2024-09-17T02:47:25.382701+0000","last_fullsized":"2024-09-17T02:47:25.382701+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421520+0000","last_change":"2024-09-17T02:47:25.404863+0000","last_active":"2024-09-17T02:47:27.421520+0000","last_peered":"2024-09-17T02:47:27.421520+0000","last_clean":"2024-09-17T02:47:27.421520+0000","last_became_active":"2024-09-17T02:47:25.403960+0000","last_became_peered":"2024-09-17T02:47:25.403960+0000","last_unstale":"2024-09-17T02:47:27.421520+0000","last_undegraded":"2024-09-17T02:47:27.421520+0000","last_fullsized":"2024-09-17T02:47:27.421520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:38.479422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.390390+0000","last_change":"2024-09-17T02:47:25.390390+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.390390+0000","last_undegraded":"2024-09-17T02:47:25.390390+0000","last_fullsized":"2024-09-17T02:47:25.390390+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.409356+0000","last_change":"2024-09-17T02:47:25.403962+0000","last_active":"2024-09-17T02:47:25.409356+0000","last_peered":"2024-09-17T02:47:25.409356+0000","last_clean":"2024-09-17T02:47:25.409356+0000","last_became_active":"2024-09-17T02:47:25.403605+0000","last_became_peered":"2024-09-17T02:47:25.403605+0000","last_unstale":"2024-09-17T02:47:25.409356+0000","last_undegraded":"2024-09-17T02:47:25.409356+0000","last_fullsized":"2024-09-17T02:47:25.409356+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:46.106338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.388513+0000","last_change":"2024-09-17T02:47:25.388513+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.388513+0000","last_undegraded":"2024-09-17T02:47:25.388513+0000","last_fullsized":"2024-09-17T02:47:25.388513+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.411901+0000","last_change":"2024-09-17T02:47:25.410837+0000","last_active":"2024-09-17T02:47:25.411901+0000","last_peered":"2024-09-17T02:47:25.411901+0000","last_clean":"2024-09-17T02:47:25.411901+0000","last_became_active":"2024-09-17T02:47:25.410644+0000","last_became_peered":"2024-09-17T02:47:25.410644+0000","last_unstale":"2024-09-17T02:47:25.411901+0000","last_undegraded":"2024-09-17T02:47:25.411901+0000","last_fullsized":"2024-09-17T02:47:25.411901+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:04.657138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.389815+0000","last_change":"2024-09-17T02:47:25.389815+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.389815+0000","last_undegraded":"2024-09-17T02:47:25.389815+0000","last_fullsized":"2024-09-17T02:47:25.389815+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.398501+0000","last_change":"2024-09-17T02:47:25.396522+0000","last_active":"2024-09-17T02:47:25.398501+0000","last_peered":"2024-09-17T02:47:25.398501+0000","last_clean":"2024-09-17T02:47:25.398501+0000","last_became_active":"2024-09-17T02:47:25.395729+0000","last_became_peered":"2024-09-17T02:47:25.395729+0000","last_unstale":"2024-09-17T02:47:25.398501+0000","last_undegraded":"2024-09-17T02:47:25.398501+0000","last_fullsized":"2024-09-17T02:47:25.398501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:54.515348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":90,"state":"creating+peering","last_fresh":"2024-09-17T02:47:25.401128+0000","last_change":"2024-09-17T02:47:25.401128+0000","last_active":"2024-09-17T02:47:25.364440+0000","last_peered":"2024-09-17T02:47:25.364440+0000","last_clean":"2024-09-17T02:47:25.364440+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:47:25.401128+0000","last_undegraded":"2024-09-17T02:47:25.401128+0000","last_fullsized":"2024-09-17T02:47:25.401128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421501+0000","last_change":"2024-09-17T02:47:25.404744+0000","last_active":"2024-09-17T02:47:27.421501+0000","last_peered":"2024-09-17T02:47:27.421501+0000","last_clean":"2024-09-17T02:47:27.421501+0000","last_became_active":"2024-09-17T02:47:25.403842+0000","last_became_peered":"2024-09-17T02:47:25.403842+0000","last_unstale":"2024-09-17T02:47:27.421501+0000","last_undegraded":"2024-09-17T02:47:27.421501+0000","last_fullsized":"2024-09-17T02:47:27.421501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:02.237396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":242,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-17T02:47:25.386269+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:47:25.386269+0000","last_peered":"2024-09-17T02:47:25.386269+0000","last_clean":"2024-09-17T02:47:25.386269+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:47:25.386269+0000","last_undegraded":"2024-09-17T02:47:25.386269+0000","last_fullsized":"2024-09-17T02:47:25.386269+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1101824,"data_stored":503944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32836,"kb_used_data":1644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1683456,"data_stored":1087645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1101824,"data_stored":503944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1089536,"data_stored":497277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1101824,"data_stored":503944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509525,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1695744,"data_stored":1094312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509526,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1101824,"data_stored":503944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1146880,"data_stored":519539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:47:30.947 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:47:31.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.141+0000 7f6beefc9640 1 -- 172.21.15.131:0/1528738682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6be815c5b0 msgr2=0x7f6be80cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.141+0000 7f6beefc9640 1 --2- 172.21.15.131:0/1528738682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6be815c5b0 0x7f6be80cbaf0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f6be806a1c0 tx=0x7f6be001cba0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.141+0000 7f6beefc9640 1 -- 172.21.15.131:0/1528738682 shutdown_connections 2024-09-17T02:47:31.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.141+0000 7f6beefc9640 1 --2- 172.21.15.131:0/1528738682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6be815c5b0 0x7f6be80cbaf0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.141+0000 7f6beefc9640 1 --2- 172.21.15.131:0/1528738682 >> v2:172.21.15.189:3300/0 conn(0x7f6be80ced00 0x7f6be806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.141+0000 7f6beefc9640 1 -- 172.21.15.131:0/1528738682 >> 172.21.15.131:0/1528738682 conn(0x7f6be806eea0 msgr2=0x7f6be806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:31.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.141+0000 7f6beefc9640 1 -- 172.21.15.131:0/1528738682 shutdown_connections 2024-09-17T02:47:31.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.141+0000 7f6beefc9640 1 -- 172.21.15.131:0/1528738682 wait complete. 2024-09-17T02:47:31.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6beefc9640 1 Processor -- start 2024-09-17T02:47:31.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6beefc9640 1 -- start start 2024-09-17T02:47:31.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6beefc9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6be806e3e0 0x7f6be81f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6beefc9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6be80ced00 0x7f6be8151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6beefc9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6be815c5b0 0x7f6be81521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6beefc9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6be8168720 con 0x7f6be806e3e0 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6beefc9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6be81685a0 con 0x7f6be80ced00 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6beefc9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6be81688a0 con 0x7f6be815c5b0 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6becd65640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6be806e3e0 0x7f6be81f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6becd65640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6be806e3e0 0x7f6be81f8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43422/0 (socket says 172.21.15.131:43422) 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6becd65640 1 -- 172.21.15.131:0/3359034968 learned_addr learned my addr 172.21.15.131:0/3359034968 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6bed566640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6be815c5b0 0x7f6be81521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6be7fff640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6be80ced00 0x7f6be8151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6becd65640 1 -- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6be815c5b0 msgr2=0x7f6be81521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6becd65640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6be815c5b0 0x7f6be81521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6becd65640 1 -- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6be80ced00 msgr2=0x7f6be8151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6becd65640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6be80ced00 0x7f6be8151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6becd65640 1 -- 172.21.15.131:0/3359034968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6be81f8850 con 0x7f6be806e3e0 2024-09-17T02:47:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6bed566640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6be815c5b0 0x7f6be81521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:47:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6be7fff640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6be80ced00 0x7f6be8151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:47:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6becd65640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6be806e3e0 0x7f6be81f8310 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000d720 tx=0x7f6bd000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6be57fa640 1 -- 172.21.15.131:0/3359034968 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0017070 con 0x7f6be806e3e0 2024-09-17T02:47:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6be57fa640 1 -- 172.21.15.131:0/3359034968 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bd0005310 con 0x7f6be806e3e0 2024-09-17T02:47:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6beefc9640 1 -- 172.21.15.131:0/3359034968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6be8155a80 con 0x7f6be806e3e0 2024-09-17T02:47:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6be57fa640 1 -- 172.21.15.131:0/3359034968 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd00097c0 con 0x7f6be806e3e0 2024-09-17T02:47:31.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6beefc9640 1 -- 172.21.15.131:0/3359034968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6be81554c0 con 0x7f6be806e3e0 2024-09-17T02:47:31.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.145+0000 7f6beefc9640 1 -- 172.21.15.131:0/3359034968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6be806dea0 con 0x7f6be806e3e0 2024-09-17T02:47:31.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.149+0000 7f6be57fa640 1 -- 172.21.15.131:0/3359034968 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6bd0009960 con 0x7f6be806e3e0 2024-09-17T02:47:31.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.149+0000 7f6be57fa640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bc8073200 0x7f6bc8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.149+0000 7f6be57fa640 1 -- 172.21.15.131:0/3359034968 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f6bd0094480 con 0x7f6be806e3e0 2024-09-17T02:47:31.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.149+0000 7f6be7fff640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bc8073200 0x7f6bc8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.149+0000 7f6be7fff640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bc8073200 0x7f6bc8075710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6be8153fd0 tx=0x7f6bd8008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:31.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.153+0000 7f6be57fa640 1 -- 172.21.15.131:0/3359034968 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6be81f8850 con 0x7f6be806e3e0 2024-09-17T02:47:31.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.309+0000 7f6beefc9640 1 -- 172.21.15.131:0/3359034968 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6be806f590 con 0x7f6bc8073200 2024-09-17T02:47:31.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.321+0000 7f6be57fa640 1 -- 172.21.15.131:0/3359034968 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252075 (secure 0 0 0) 0x7f6be806f590 con 0x7f6bc8073200 2024-09-17T02:47:31.322 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:31.324 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:47:31.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 -- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bc8073200 msgr2=0x7f6bc8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bc8073200 0x7f6bc8075710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6be8153fd0 tx=0x7f6bd8008040 comp rx=0 tx=0).stop 2024-09-17T02:47:31.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 -- 172.21.15.131:0/3359034968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6be806e3e0 msgr2=0x7f6be81f8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6be806e3e0 0x7f6be81f8310 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000d720 tx=0x7f6bd000dbe0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 -- 172.21.15.131:0/3359034968 shutdown_connections 2024-09-17T02:47:31.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6bc8073200 0x7f6bc8075710 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6be815c5b0 0x7f6be81521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6be80ced00 0x7f6be8151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 --2- 172.21.15.131:0/3359034968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6be806e3e0 0x7f6be81f8310 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 -- 172.21.15.131:0/3359034968 >> 172.21.15.131:0/3359034968 conn(0x7f6be806eea0 msgr2=0x7f6be806ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:31.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 -- 172.21.15.131:0/3359034968 shutdown_connections 2024-09-17T02:47:31.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.325+0000 7f6beefc9640 1 -- 172.21.15.131:0/3359034968 wait complete. 2024-09-17T02:47:31.341 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-17T02:47:31.276426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259572,"kb_used_data":10024,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715148,"statfs":{"total":773094113280,"available":772828311552,"internally_reserved":0,"allocated":10264576,"data_stored":5327980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001964"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422667+0000","last_change":"2024-09-17T02:47:26.451633+0000","last_active":"2024-09-17T02:47:27.422667+0000","last_peered":"2024-09-17T02:47:27.422667+0000","last_clean":"2024-09-17T02:47:27.422667+0000","last_became_active":"2024-09-17T02:47:26.450812+0000","last_became_peered":"2024-09-17T02:47:26.450812+0000","last_unstale":"2024-09-17T02:47:27.422667+0000","last_undegraded":"2024-09-17T02:47:27.422667+0000","last_fullsized":"2024-09-17T02:47:27.422667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:06.130998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768309+0000","last_change":"2024-09-17T02:47:26.435817+0000","last_active":"2024-09-17T02:47:27.768309+0000","last_peered":"2024-09-17T02:47:27.768309+0000","last_clean":"2024-09-17T02:47:27.768309+0000","last_became_active":"2024-09-17T02:47:26.435429+0000","last_became_peered":"2024-09-17T02:47:26.435429+0000","last_unstale":"2024-09-17T02:47:27.768309+0000","last_undegraded":"2024-09-17T02:47:27.768309+0000","last_fullsized":"2024-09-17T02:47:27.768309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:03.221935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422426+0000","last_change":"2024-09-17T02:47:26.422687+0000","last_active":"2024-09-17T02:47:27.422426+0000","last_peered":"2024-09-17T02:47:27.422426+0000","last_clean":"2024-09-17T02:47:27.422426+0000","last_became_active":"2024-09-17T02:47:26.421534+0000","last_became_peered":"2024-09-17T02:47:26.421534+0000","last_unstale":"2024-09-17T02:47:27.422426+0000","last_undegraded":"2024-09-17T02:47:27.422426+0000","last_fullsized":"2024-09-17T02:47:27.422426+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:28.446385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419818+0000","last_change":"2024-09-17T02:47:26.419919+0000","last_active":"2024-09-17T02:47:27.419818+0000","last_peered":"2024-09-17T02:47:27.419818+0000","last_clean":"2024-09-17T02:47:27.419818+0000","last_became_active":"2024-09-17T02:47:26.419062+0000","last_became_peered":"2024-09-17T02:47:26.419062+0000","last_unstale":"2024-09-17T02:47:27.419818+0000","last_undegraded":"2024-09-17T02:47:27.419818+0000","last_fullsized":"2024-09-17T02:47:27.419818+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:17.965654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766901+0000","last_change":"2024-09-17T02:47:26.425433+0000","last_active":"2024-09-17T02:47:27.766901+0000","last_peered":"2024-09-17T02:47:27.766901+0000","last_clean":"2024-09-17T02:47:27.766901+0000","last_became_active":"2024-09-17T02:47:26.420685+0000","last_became_peered":"2024-09-17T02:47:26.420685+0000","last_unstale":"2024-09-17T02:47:27.766901+0000","last_undegraded":"2024-09-17T02:47:27.766901+0000","last_fullsized":"2024-09-17T02:47:27.766901+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:15.124007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766590+0000","last_change":"2024-09-17T02:47:26.454339+0000","last_active":"2024-09-17T02:47:27.766590+0000","last_peered":"2024-09-17T02:47:27.766590+0000","last_clean":"2024-09-17T02:47:27.766590+0000","last_became_active":"2024-09-17T02:47:26.453723+0000","last_became_peered":"2024-09-17T02:47:26.453723+0000","last_unstale":"2024-09-17T02:47:27.766590+0000","last_undegraded":"2024-09-17T02:47:27.766590+0000","last_fullsized":"2024-09-17T02:47:27.766590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:30.811217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.415989+0000","last_change":"2024-09-17T02:47:26.431390+0000","last_active":"2024-09-17T02:47:27.415989+0000","last_peered":"2024-09-17T02:47:27.415989+0000","last_clean":"2024-09-17T02:47:27.415989+0000","last_became_active":"2024-09-17T02:47:26.431015+0000","last_became_peered":"2024-09-17T02:47:26.431015+0000","last_unstale":"2024-09-17T02:47:27.415989+0000","last_undegraded":"2024-09-17T02:47:27.415989+0000","last_fullsized":"2024-09-17T02:47:27.415989+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:51.096970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766358+0000","last_change":"2024-09-17T02:47:26.432290+0000","last_active":"2024-09-17T02:47:27.766358+0000","last_peered":"2024-09-17T02:47:27.766358+0000","last_clean":"2024-09-17T02:47:27.766358+0000","last_became_active":"2024-09-17T02:47:26.431479+0000","last_became_peered":"2024-09-17T02:47:26.431479+0000","last_unstale":"2024-09-17T02:47:27.766358+0000","last_undegraded":"2024-09-17T02:47:27.766358+0000","last_fullsized":"2024-09-17T02:47:27.766358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:39.578242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416330+0000","last_change":"2024-09-17T02:47:26.428950+0000","last_active":"2024-09-17T02:47:27.416330+0000","last_peered":"2024-09-17T02:47:27.416330+0000","last_clean":"2024-09-17T02:47:27.416330+0000","last_became_active":"2024-09-17T02:47:26.428599+0000","last_became_peered":"2024-09-17T02:47:26.428599+0000","last_unstale":"2024-09-17T02:47:27.416330+0000","last_undegraded":"2024-09-17T02:47:27.416330+0000","last_fullsized":"2024-09-17T02:47:27.416330+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:07.335822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419341+0000","last_change":"2024-09-17T02:47:26.433543+0000","last_active":"2024-09-17T02:47:27.419341+0000","last_peered":"2024-09-17T02:47:27.419341+0000","last_clean":"2024-09-17T02:47:27.419341+0000","last_became_active":"2024-09-17T02:47:26.432850+0000","last_became_peered":"2024-09-17T02:47:26.432850+0000","last_unstale":"2024-09-17T02:47:27.419341+0000","last_undegraded":"2024-09-17T02:47:27.419341+0000","last_fullsized":"2024-09-17T02:47:27.419341+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:23.938528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416194+0000","last_change":"2024-09-17T02:47:26.429547+0000","last_active":"2024-09-17T02:47:27.416194+0000","last_peered":"2024-09-17T02:47:27.416194+0000","last_clean":"2024-09-17T02:47:27.416194+0000","last_became_active":"2024-09-17T02:47:26.429189+0000","last_became_peered":"2024-09-17T02:47:26.429189+0000","last_unstale":"2024-09-17T02:47:27.416194+0000","last_undegraded":"2024-09-17T02:47:27.416194+0000","last_fullsized":"2024-09-17T02:47:27.416194+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:35.392361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.420510+0000","last_change":"2024-09-17T02:47:26.453163+0000","last_active":"2024-09-17T02:47:27.420510+0000","last_peered":"2024-09-17T02:47:27.420510+0000","last_clean":"2024-09-17T02:47:27.420510+0000","last_became_active":"2024-09-17T02:47:26.452601+0000","last_became_peered":"2024-09-17T02:47:26.452601+0000","last_unstale":"2024-09-17T02:47:27.420510+0000","last_undegraded":"2024-09-17T02:47:27.420510+0000","last_fullsized":"2024-09-17T02:47:27.420510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:34.534013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768784+0000","last_change":"2024-09-17T02:47:26.443086+0000","last_active":"2024-09-17T02:47:27.768784+0000","last_peered":"2024-09-17T02:47:27.768784+0000","last_clean":"2024-09-17T02:47:27.768784+0000","last_became_active":"2024-09-17T02:47:26.442205+0000","last_became_peered":"2024-09-17T02:47:26.442205+0000","last_unstale":"2024-09-17T02:47:27.768784+0000","last_undegraded":"2024-09-17T02:47:27.768784+0000","last_fullsized":"2024-09-17T02:47:27.768784+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:25.973524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746489+0000","last_change":"2024-09-17T02:47:26.423413+0000","last_active":"2024-09-17T02:47:27.746489+0000","last_peered":"2024-09-17T02:47:27.746489+0000","last_clean":"2024-09-17T02:47:27.746489+0000","last_became_active":"2024-09-17T02:47:26.423062+0000","last_became_peered":"2024-09-17T02:47:26.423062+0000","last_unstale":"2024-09-17T02:47:27.746489+0000","last_undegraded":"2024-09-17T02:47:27.746489+0000","last_fullsized":"2024-09-17T02:47:27.746489+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:16.597414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421280+0000","last_change":"2024-09-17T02:47:26.428868+0000","last_active":"2024-09-17T02:47:27.421280+0000","last_peered":"2024-09-17T02:47:27.421280+0000","last_clean":"2024-09-17T02:47:27.421280+0000","last_became_active":"2024-09-17T02:47:26.426622+0000","last_became_peered":"2024-09-17T02:47:26.426622+0000","last_unstale":"2024-09-17T02:47:27.421280+0000","last_undegraded":"2024-09-17T02:47:27.421280+0000","last_fullsized":"2024-09-17T02:47:27.421280+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:15.955660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766351+0000","last_change":"2024-09-17T02:47:26.434274+0000","last_active":"2024-09-17T02:47:27.766351+0000","last_peered":"2024-09-17T02:47:27.766351+0000","last_clean":"2024-09-17T02:47:27.766351+0000","last_became_active":"2024-09-17T02:47:26.433517+0000","last_became_peered":"2024-09-17T02:47:26.433517+0000","last_unstale":"2024-09-17T02:47:27.766351+0000","last_undegraded":"2024-09-17T02:47:27.766351+0000","last_fullsized":"2024-09-17T02:47:27.766351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:00.041629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769558+0000","last_change":"2024-09-17T02:47:26.453403+0000","last_active":"2024-09-17T02:47:27.769558+0000","last_peered":"2024-09-17T02:47:27.769558+0000","last_clean":"2024-09-17T02:47:27.769558+0000","last_became_active":"2024-09-17T02:47:26.452810+0000","last_became_peered":"2024-09-17T02:47:26.452810+0000","last_unstale":"2024-09-17T02:47:27.769558+0000","last_undegraded":"2024-09-17T02:47:27.769558+0000","last_fullsized":"2024-09-17T02:47:27.769558+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:06.058284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419516+0000","last_change":"2024-09-17T02:47:26.406401+0000","last_active":"2024-09-17T02:47:27.419516+0000","last_peered":"2024-09-17T02:47:27.419516+0000","last_clean":"2024-09-17T02:47:27.419516+0000","last_became_active":"2024-09-17T02:47:26.405810+0000","last_became_peered":"2024-09-17T02:47:26.405810+0000","last_unstale":"2024-09-17T02:47:27.419516+0000","last_undegraded":"2024-09-17T02:47:27.419516+0000","last_fullsized":"2024-09-17T02:47:27.419516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:17.427456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768538+0000","last_change":"2024-09-17T02:47:26.421898+0000","last_active":"2024-09-17T02:47:27.768538+0000","last_peered":"2024-09-17T02:47:27.768538+0000","last_clean":"2024-09-17T02:47:27.768538+0000","last_became_active":"2024-09-17T02:47:26.421595+0000","last_became_peered":"2024-09-17T02:47:26.421595+0000","last_unstale":"2024-09-17T02:47:27.768538+0000","last_undegraded":"2024-09-17T02:47:27.768538+0000","last_fullsized":"2024-09-17T02:47:27.768538+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:10.282061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768163+0000","last_change":"2024-09-17T02:47:26.456961+0000","last_active":"2024-09-17T02:47:27.768163+0000","last_peered":"2024-09-17T02:47:27.768163+0000","last_clean":"2024-09-17T02:47:27.768163+0000","last_became_active":"2024-09-17T02:47:26.456375+0000","last_became_peered":"2024-09-17T02:47:26.456375+0000","last_unstale":"2024-09-17T02:47:27.768163+0000","last_undegraded":"2024-09-17T02:47:27.768163+0000","last_fullsized":"2024-09-17T02:47:27.768163+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:22.010481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769004+0000","last_change":"2024-09-17T02:47:26.425659+0000","last_active":"2024-09-17T02:47:27.769004+0000","last_peered":"2024-09-17T02:47:27.769004+0000","last_clean":"2024-09-17T02:47:27.769004+0000","last_became_active":"2024-09-17T02:47:26.418728+0000","last_became_peered":"2024-09-17T02:47:26.418728+0000","last_unstale":"2024-09-17T02:47:27.769004+0000","last_undegraded":"2024-09-17T02:47:27.769004+0000","last_fullsized":"2024-09-17T02:47:27.769004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:12.211104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767567+0000","last_change":"2024-09-17T02:47:26.420752+0000","last_active":"2024-09-17T02:47:27.767567+0000","last_peered":"2024-09-17T02:47:27.767567+0000","last_clean":"2024-09-17T02:47:27.767567+0000","last_became_active":"2024-09-17T02:47:26.420423+0000","last_became_peered":"2024-09-17T02:47:26.420423+0000","last_unstale":"2024-09-17T02:47:27.767567+0000","last_undegraded":"2024-09-17T02:47:27.767567+0000","last_fullsized":"2024-09-17T02:47:27.767567+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:50.371938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766752+0000","last_change":"2024-09-17T02:47:26.449510+0000","last_active":"2024-09-17T02:47:27.766752+0000","last_peered":"2024-09-17T02:47:27.766752+0000","last_clean":"2024-09-17T02:47:27.766752+0000","last_became_active":"2024-09-17T02:47:26.449136+0000","last_became_peered":"2024-09-17T02:47:26.449136+0000","last_unstale":"2024-09-17T02:47:27.766752+0000","last_undegraded":"2024-09-17T02:47:27.766752+0000","last_fullsized":"2024-09-17T02:47:27.766752+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:46.727764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.418593+0000","last_change":"2024-09-17T02:47:26.438730+0000","last_active":"2024-09-17T02:47:27.418593+0000","last_peered":"2024-09-17T02:47:27.418593+0000","last_clean":"2024-09-17T02:47:27.418593+0000","last_became_active":"2024-09-17T02:47:26.420076+0000","last_became_peered":"2024-09-17T02:47:26.420076+0000","last_unstale":"2024-09-17T02:47:27.418593+0000","last_undegraded":"2024-09-17T02:47:27.418593+0000","last_fullsized":"2024-09-17T02:47:27.418593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:12.606293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768851+0000","last_change":"2024-09-17T02:47:26.428029+0000","last_active":"2024-09-17T02:47:27.768851+0000","last_peered":"2024-09-17T02:47:27.768851+0000","last_clean":"2024-09-17T02:47:27.768851+0000","last_became_active":"2024-09-17T02:47:26.427694+0000","last_became_peered":"2024-09-17T02:47:26.427694+0000","last_unstale":"2024-09-17T02:47:27.768851+0000","last_undegraded":"2024-09-17T02:47:27.768851+0000","last_fullsized":"2024-09-17T02:47:27.768851+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:10.261716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768992+0000","last_change":"2024-09-17T02:47:26.427423+0000","last_active":"2024-09-17T02:47:27.768992+0000","last_peered":"2024-09-17T02:47:27.768992+0000","last_clean":"2024-09-17T02:47:27.768992+0000","last_became_active":"2024-09-17T02:47:26.427019+0000","last_became_peered":"2024-09-17T02:47:26.427019+0000","last_unstale":"2024-09-17T02:47:27.768992+0000","last_undegraded":"2024-09-17T02:47:27.768992+0000","last_fullsized":"2024-09-17T02:47:27.768992+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:10.586243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422781+0000","last_change":"2024-09-17T02:47:26.411901+0000","last_active":"2024-09-17T02:47:27.422781+0000","last_peered":"2024-09-17T02:47:27.422781+0000","last_clean":"2024-09-17T02:47:27.422781+0000","last_became_active":"2024-09-17T02:47:26.409711+0000","last_became_peered":"2024-09-17T02:47:26.409711+0000","last_unstale":"2024-09-17T02:47:27.422781+0000","last_undegraded":"2024-09-17T02:47:27.422781+0000","last_fullsized":"2024-09-17T02:47:27.422781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:04.645639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767833+0000","last_change":"2024-09-17T02:47:26.451152+0000","last_active":"2024-09-17T02:47:27.767833+0000","last_peered":"2024-09-17T02:47:27.767833+0000","last_clean":"2024-09-17T02:47:27.767833+0000","last_became_active":"2024-09-17T02:47:26.445335+0000","last_became_peered":"2024-09-17T02:47:26.445335+0000","last_unstale":"2024-09-17T02:47:27.767833+0000","last_undegraded":"2024-09-17T02:47:27.767833+0000","last_fullsized":"2024-09-17T02:47:27.767833+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:39.809055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769745+0000","last_change":"2024-09-17T02:47:26.453695+0000","last_active":"2024-09-17T02:47:27.769745+0000","last_peered":"2024-09-17T02:47:27.769745+0000","last_clean":"2024-09-17T02:47:27.769745+0000","last_became_active":"2024-09-17T02:47:26.453394+0000","last_became_peered":"2024-09-17T02:47:26.453394+0000","last_unstale":"2024-09-17T02:47:27.769745+0000","last_undegraded":"2024-09-17T02:47:27.769745+0000","last_fullsized":"2024-09-17T02:47:27.769745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:50.371938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766880+0000","last_change":"2024-09-17T02:47:26.424922+0000","last_active":"2024-09-17T02:47:27.766880+0000","last_peered":"2024-09-17T02:47:27.766880+0000","last_clean":"2024-09-17T02:47:27.766880+0000","last_became_active":"2024-09-17T02:47:26.418098+0000","last_became_peered":"2024-09-17T02:47:26.418098+0000","last_unstale":"2024-09-17T02:47:27.766880+0000","last_undegraded":"2024-09-17T02:47:27.766880+0000","last_fullsized":"2024-09-17T02:47:27.766880+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:18.176172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767618+0000","last_change":"2024-09-17T02:47:26.411672+0000","last_active":"2024-09-17T02:47:27.767618+0000","last_peered":"2024-09-17T02:47:27.767618+0000","last_clean":"2024-09-17T02:47:27.767618+0000","last_became_active":"2024-09-17T02:47:26.411280+0000","last_became_peered":"2024-09-17T02:47:26.411280+0000","last_unstale":"2024-09-17T02:47:27.767618+0000","last_undegraded":"2024-09-17T02:47:27.767618+0000","last_fullsized":"2024-09-17T02:47:27.767618+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:43.278442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767318+0000","last_change":"2024-09-17T02:47:26.433522+0000","last_active":"2024-09-17T02:47:27.767318+0000","last_peered":"2024-09-17T02:47:27.767318+0000","last_clean":"2024-09-17T02:47:27.767318+0000","last_became_active":"2024-09-17T02:47:26.433017+0000","last_became_peered":"2024-09-17T02:47:26.433017+0000","last_unstale":"2024-09-17T02:47:27.767318+0000","last_undegraded":"2024-09-17T02:47:27.767318+0000","last_fullsized":"2024-09-17T02:47:27.767318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:35.865526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421125+0000","last_change":"2024-09-17T02:47:26.419153+0000","last_active":"2024-09-17T02:47:27.421125+0000","last_peered":"2024-09-17T02:47:27.421125+0000","last_clean":"2024-09-17T02:47:27.421125+0000","last_became_active":"2024-09-17T02:47:26.418280+0000","last_became_peered":"2024-09-17T02:47:26.418280+0000","last_unstale":"2024-09-17T02:47:27.421125+0000","last_undegraded":"2024-09-17T02:47:27.421125+0000","last_fullsized":"2024-09-17T02:47:27.421125+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:17.220457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"91'4","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769913+0000","last_change":"2024-09-17T02:47:25.404962+0000","last_active":"2024-09-17T02:47:27.769913+0000","last_peered":"2024-09-17T02:47:27.769913+0000","last_clean":"2024-09-17T02:47:27.769913+0000","last_became_active":"2024-09-17T02:47:25.404574+0000","last_became_peered":"2024-09-17T02:47:25.404574+0000","last_unstale":"2024-09-17T02:47:27.769913+0000","last_undegraded":"2024-09-17T02:47:27.769913+0000","last_fullsized":"2024-09-17T02:47:27.769913+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:27.433639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768770+0000","last_change":"2024-09-17T02:47:26.448672+0000","last_active":"2024-09-17T02:47:27.768770+0000","last_peered":"2024-09-17T02:47:27.768770+0000","last_clean":"2024-09-17T02:47:27.768770+0000","last_became_active":"2024-09-17T02:47:26.447991+0000","last_became_peered":"2024-09-17T02:47:26.447991+0000","last_unstale":"2024-09-17T02:47:27.768770+0000","last_undegraded":"2024-09-17T02:47:27.768770+0000","last_fullsized":"2024-09-17T02:47:27.768770+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:05.713702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746189+0000","last_change":"2024-09-17T02:47:25.394418+0000","last_active":"2024-09-17T02:47:27.746189+0000","last_peered":"2024-09-17T02:47:27.746189+0000","last_clean":"2024-09-17T02:47:27.746189+0000","last_became_active":"2024-09-17T02:47:25.393974+0000","last_became_peered":"2024-09-17T02:47:25.393974+0000","last_unstale":"2024-09-17T02:47:27.746189+0000","last_undegraded":"2024-09-17T02:47:27.746189+0000","last_fullsized":"2024-09-17T02:47:27.746189+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:10.358930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416397+0000","last_change":"2024-09-17T02:47:26.444143+0000","last_active":"2024-09-17T02:47:27.416397+0000","last_peered":"2024-09-17T02:47:27.416397+0000","last_clean":"2024-09-17T02:47:27.416397+0000","last_became_active":"2024-09-17T02:47:26.443779+0000","last_became_peered":"2024-09-17T02:47:26.443779+0000","last_unstale":"2024-09-17T02:47:27.416397+0000","last_undegraded":"2024-09-17T02:47:27.416397+0000","last_fullsized":"2024-09-17T02:47:27.416397+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:28.224146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768881+0000","last_change":"2024-09-17T02:47:25.418775+0000","last_active":"2024-09-17T02:47:27.768881+0000","last_peered":"2024-09-17T02:47:27.768881+0000","last_clean":"2024-09-17T02:47:27.768881+0000","last_became_active":"2024-09-17T02:47:25.418458+0000","last_became_peered":"2024-09-17T02:47:25.418458+0000","last_unstale":"2024-09-17T02:47:27.768881+0000","last_undegraded":"2024-09-17T02:47:27.768881+0000","last_fullsized":"2024-09-17T02:47:27.768881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:16.952908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419487+0000","last_change":"2024-09-17T02:47:26.438283+0000","last_active":"2024-09-17T02:47:27.419487+0000","last_peered":"2024-09-17T02:47:27.419487+0000","last_clean":"2024-09-17T02:47:27.419487+0000","last_became_active":"2024-09-17T02:47:26.437598+0000","last_became_peered":"2024-09-17T02:47:26.437598+0000","last_unstale":"2024-09-17T02:47:27.419487+0000","last_undegraded":"2024-09-17T02:47:27.419487+0000","last_fullsized":"2024-09-17T02:47:27.419487+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:42.817995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.745987+0000","last_change":"2024-09-17T02:47:25.412944+0000","last_active":"2024-09-17T02:47:27.745987+0000","last_peered":"2024-09-17T02:47:27.745987+0000","last_clean":"2024-09-17T02:47:27.745987+0000","last_became_active":"2024-09-17T02:47:25.412515+0000","last_became_peered":"2024-09-17T02:47:25.412515+0000","last_unstale":"2024-09-17T02:47:27.745987+0000","last_undegraded":"2024-09-17T02:47:27.745987+0000","last_fullsized":"2024-09-17T02:47:27.745987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:06.364211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767322+0000","last_change":"2024-09-17T02:47:26.457572+0000","last_active":"2024-09-17T02:47:27.767322+0000","last_peered":"2024-09-17T02:47:27.767322+0000","last_clean":"2024-09-17T02:47:27.767322+0000","last_became_active":"2024-09-17T02:47:26.451939+0000","last_became_peered":"2024-09-17T02:47:26.451939+0000","last_unstale":"2024-09-17T02:47:27.767322+0000","last_undegraded":"2024-09-17T02:47:27.767322+0000","last_fullsized":"2024-09-17T02:47:27.767322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:57.435169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767430+0000","last_change":"2024-09-17T02:47:25.411938+0000","last_active":"2024-09-17T02:47:27.767430+0000","last_peered":"2024-09-17T02:47:27.767430+0000","last_clean":"2024-09-17T02:47:27.767430+0000","last_became_active":"2024-09-17T02:47:25.411486+0000","last_became_peered":"2024-09-17T02:47:25.411486+0000","last_unstale":"2024-09-17T02:47:27.767430+0000","last_undegraded":"2024-09-17T02:47:27.767430+0000","last_fullsized":"2024-09-17T02:47:27.767430+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:08.479886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416053+0000","last_change":"2024-09-17T02:47:26.424905+0000","last_active":"2024-09-17T02:47:27.416053+0000","last_peered":"2024-09-17T02:47:27.416053+0000","last_clean":"2024-09-17T02:47:27.416053+0000","last_became_active":"2024-09-17T02:47:26.424551+0000","last_became_peered":"2024-09-17T02:47:26.424551+0000","last_unstale":"2024-09-17T02:47:27.416053+0000","last_undegraded":"2024-09-17T02:47:27.416053+0000","last_fullsized":"2024-09-17T02:47:27.416053+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:03.042595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768416+0000","last_change":"2024-09-17T02:47:25.422380+0000","last_active":"2024-09-17T02:47:27.768416+0000","last_peered":"2024-09-17T02:47:27.768416+0000","last_clean":"2024-09-17T02:47:27.768416+0000","last_became_active":"2024-09-17T02:47:25.421906+0000","last_became_peered":"2024-09-17T02:47:25.421906+0000","last_unstale":"2024-09-17T02:47:27.768416+0000","last_undegraded":"2024-09-17T02:47:27.768416+0000","last_fullsized":"2024-09-17T02:47:27.768416+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:04.941668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767297+0000","last_change":"2024-09-17T02:47:26.435576+0000","last_active":"2024-09-17T02:47:27.767297+0000","last_peered":"2024-09-17T02:47:27.767297+0000","last_clean":"2024-09-17T02:47:27.767297+0000","last_became_active":"2024-09-17T02:47:26.434782+0000","last_became_peered":"2024-09-17T02:47:26.434782+0000","last_unstale":"2024-09-17T02:47:27.767297+0000","last_undegraded":"2024-09-17T02:47:27.767297+0000","last_fullsized":"2024-09-17T02:47:27.767297+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:12.674675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422144+0000","last_change":"2024-09-17T02:47:25.410268+0000","last_active":"2024-09-17T02:47:27.422144+0000","last_peered":"2024-09-17T02:47:27.422144+0000","last_clean":"2024-09-17T02:47:27.422144+0000","last_became_active":"2024-09-17T02:47:25.409433+0000","last_became_peered":"2024-09-17T02:47:25.409433+0000","last_unstale":"2024-09-17T02:47:27.422144+0000","last_undegraded":"2024-09-17T02:47:27.422144+0000","last_fullsized":"2024-09-17T02:47:27.422144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:46.106338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.420382+0000","last_change":"2024-09-17T02:47:26.410976+0000","last_active":"2024-09-17T02:47:27.420382+0000","last_peered":"2024-09-17T02:47:27.420382+0000","last_clean":"2024-09-17T02:47:27.420382+0000","last_became_active":"2024-09-17T02:47:26.407305+0000","last_became_peered":"2024-09-17T02:47:26.407305+0000","last_unstale":"2024-09-17T02:47:27.420382+0000","last_undegraded":"2024-09-17T02:47:27.420382+0000","last_fullsized":"2024-09-17T02:47:27.420382+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:53.755378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766722+0000","last_change":"2024-09-17T02:47:25.420193+0000","last_active":"2024-09-17T02:47:27.766722+0000","last_peered":"2024-09-17T02:47:27.766722+0000","last_clean":"2024-09-17T02:47:27.766722+0000","last_became_active":"2024-09-17T02:47:25.419700+0000","last_became_peered":"2024-09-17T02:47:25.419700+0000","last_unstale":"2024-09-17T02:47:27.766722+0000","last_undegraded":"2024-09-17T02:47:27.766722+0000","last_fullsized":"2024-09-17T02:47:27.766722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:41.533605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768164+0000","last_change":"2024-09-17T02:47:26.453517+0000","last_active":"2024-09-17T02:47:27.768164+0000","last_peered":"2024-09-17T02:47:27.768164+0000","last_clean":"2024-09-17T02:47:27.768164+0000","last_became_active":"2024-09-17T02:47:26.453175+0000","last_became_peered":"2024-09-17T02:47:26.453175+0000","last_unstale":"2024-09-17T02:47:27.768164+0000","last_undegraded":"2024-09-17T02:47:27.768164+0000","last_fullsized":"2024-09-17T02:47:27.768164+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:16.597414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769913+0000","last_change":"2024-09-17T02:47:25.419435+0000","last_active":"2024-09-17T02:47:27.769913+0000","last_peered":"2024-09-17T02:47:27.769913+0000","last_clean":"2024-09-17T02:47:27.769913+0000","last_became_active":"2024-09-17T02:47:25.418955+0000","last_became_peered":"2024-09-17T02:47:25.418955+0000","last_unstale":"2024-09-17T02:47:27.769913+0000","last_undegraded":"2024-09-17T02:47:27.769913+0000","last_fullsized":"2024-09-17T02:47:27.769913+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:03.632893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416447+0000","last_change":"2024-09-17T02:47:26.421172+0000","last_active":"2024-09-17T02:47:27.416447+0000","last_peered":"2024-09-17T02:47:27.416447+0000","last_clean":"2024-09-17T02:47:27.416447+0000","last_became_active":"2024-09-17T02:47:26.420916+0000","last_became_peered":"2024-09-17T02:47:26.420916+0000","last_unstale":"2024-09-17T02:47:27.416447+0000","last_undegraded":"2024-09-17T02:47:27.416447+0000","last_fullsized":"2024-09-17T02:47:27.416447+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:29.293488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.420010+0000","last_change":"2024-09-17T02:47:25.414624+0000","last_active":"2024-09-17T02:47:27.420010+0000","last_peered":"2024-09-17T02:47:27.420010+0000","last_clean":"2024-09-17T02:47:27.420010+0000","last_became_active":"2024-09-17T02:47:25.413560+0000","last_became_peered":"2024-09-17T02:47:25.413560+0000","last_unstale":"2024-09-17T02:47:27.420010+0000","last_undegraded":"2024-09-17T02:47:27.420010+0000","last_fullsized":"2024-09-17T02:47:27.420010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:00.198104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422254+0000","last_change":"2024-09-17T02:47:26.441675+0000","last_active":"2024-09-17T02:47:27.422254+0000","last_peered":"2024-09-17T02:47:27.422254+0000","last_clean":"2024-09-17T02:47:27.422254+0000","last_became_active":"2024-09-17T02:47:26.437413+0000","last_became_peered":"2024-09-17T02:47:26.437413+0000","last_unstale":"2024-09-17T02:47:27.422254+0000","last_undegraded":"2024-09-17T02:47:27.422254+0000","last_fullsized":"2024-09-17T02:47:27.422254+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:50.459729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"91'3","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416794+0000","last_change":"2024-09-17T02:47:25.419022+0000","last_active":"2024-09-17T02:47:27.416794+0000","last_peered":"2024-09-17T02:47:27.416794+0000","last_clean":"2024-09-17T02:47:27.416794+0000","last_became_active":"2024-09-17T02:47:25.418540+0000","last_became_peered":"2024-09-17T02:47:25.418540+0000","last_unstale":"2024-09-17T02:47:27.416794+0000","last_undegraded":"2024-09-17T02:47:27.416794+0000","last_fullsized":"2024-09-17T02:47:27.416794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:04.700956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769873+0000","last_change":"2024-09-17T02:47:26.429054+0000","last_active":"2024-09-17T02:47:27.769873+0000","last_peered":"2024-09-17T02:47:27.769873+0000","last_clean":"2024-09-17T02:47:27.769873+0000","last_became_active":"2024-09-17T02:47:26.428270+0000","last_became_peered":"2024-09-17T02:47:26.428270+0000","last_unstale":"2024-09-17T02:47:27.769873+0000","last_undegraded":"2024-09-17T02:47:27.769873+0000","last_fullsized":"2024-09-17T02:47:27.769873+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:16.361134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"91'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421338+0000","last_change":"2024-09-17T02:47:25.418647+0000","last_active":"2024-09-17T02:47:27.421338+0000","last_peered":"2024-09-17T02:47:27.421338+0000","last_clean":"2024-09-17T02:47:27.421338+0000","last_became_active":"2024-09-17T02:47:25.416994+0000","last_became_peered":"2024-09-17T02:47:25.416994+0000","last_unstale":"2024-09-17T02:47:27.421338+0000","last_undegraded":"2024-09-17T02:47:27.421338+0000","last_fullsized":"2024-09-17T02:47:27.421338+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:58.768722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.418667+0000","last_change":"2024-09-17T02:47:26.419193+0000","last_active":"2024-09-17T02:47:27.418667+0000","last_peered":"2024-09-17T02:47:27.418667+0000","last_clean":"2024-09-17T02:47:27.418667+0000","last_became_active":"2024-09-17T02:47:26.418643+0000","last_became_peered":"2024-09-17T02:47:26.418643+0000","last_unstale":"2024-09-17T02:47:27.418667+0000","last_undegraded":"2024-09-17T02:47:27.418667+0000","last_fullsized":"2024-09-17T02:47:27.418667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:46.698904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.770213+0000","last_change":"2024-09-17T02:47:25.418902+0000","last_active":"2024-09-17T02:47:27.770213+0000","last_peered":"2024-09-17T02:47:27.770213+0000","last_clean":"2024-09-17T02:47:27.770213+0000","last_became_active":"2024-09-17T02:47:25.418568+0000","last_became_peered":"2024-09-17T02:47:25.418568+0000","last_unstale":"2024-09-17T02:47:27.770213+0000","last_undegraded":"2024-09-17T02:47:27.770213+0000","last_fullsized":"2024-09-17T02:47:27.770213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:22.925782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767126+0000","last_change":"2024-09-17T02:47:26.414277+0000","last_active":"2024-09-17T02:47:27.767126+0000","last_peered":"2024-09-17T02:47:27.767126+0000","last_clean":"2024-09-17T02:47:27.767126+0000","last_became_active":"2024-09-17T02:47:26.413010+0000","last_became_peered":"2024-09-17T02:47:26.413010+0000","last_unstale":"2024-09-17T02:47:27.767126+0000","last_undegraded":"2024-09-17T02:47:27.767126+0000","last_fullsized":"2024-09-17T02:47:27.767126+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:00.083852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419403+0000","last_change":"2024-09-17T02:47:25.418948+0000","last_active":"2024-09-17T02:47:27.419403+0000","last_peered":"2024-09-17T02:47:27.419403+0000","last_clean":"2024-09-17T02:47:27.419403+0000","last_became_active":"2024-09-17T02:47:25.417436+0000","last_became_peered":"2024-09-17T02:47:25.417436+0000","last_unstale":"2024-09-17T02:47:27.419403+0000","last_undegraded":"2024-09-17T02:47:27.419403+0000","last_fullsized":"2024-09-17T02:47:27.419403+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:25.386796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419176+0000","last_change":"2024-09-17T02:47:26.438873+0000","last_active":"2024-09-17T02:47:27.419176+0000","last_peered":"2024-09-17T02:47:27.419176+0000","last_clean":"2024-09-17T02:47:27.419176+0000","last_became_active":"2024-09-17T02:47:26.436291+0000","last_became_peered":"2024-09-17T02:47:26.436291+0000","last_unstale":"2024-09-17T02:47:27.419176+0000","last_undegraded":"2024-09-17T02:47:27.419176+0000","last_fullsized":"2024-09-17T02:47:27.419176+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:56.037102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.423108+0000","last_change":"2024-09-17T02:47:25.411837+0000","last_active":"2024-09-17T02:47:27.423108+0000","last_peered":"2024-09-17T02:47:27.423108+0000","last_clean":"2024-09-17T02:47:27.423108+0000","last_became_active":"2024-09-17T02:47:25.411299+0000","last_became_peered":"2024-09-17T02:47:25.411299+0000","last_unstale":"2024-09-17T02:47:27.423108+0000","last_undegraded":"2024-09-17T02:47:27.423108+0000","last_fullsized":"2024-09-17T02:47:27.423108+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:23.951343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769489+0000","last_change":"2024-09-17T02:47:26.449804+0000","last_active":"2024-09-17T02:47:27.769489+0000","last_peered":"2024-09-17T02:47:27.769489+0000","last_clean":"2024-09-17T02:47:27.769489+0000","last_became_active":"2024-09-17T02:47:26.445395+0000","last_became_peered":"2024-09-17T02:47:26.445395+0000","last_unstale":"2024-09-17T02:47:27.769489+0000","last_undegraded":"2024-09-17T02:47:27.769489+0000","last_fullsized":"2024-09-17T02:47:27.769489+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:39.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"91'8","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419774+0000","last_change":"2024-09-17T02:47:25.421873+0000","last_active":"2024-09-17T02:47:27.419774+0000","last_peered":"2024-09-17T02:47:27.419774+0000","last_clean":"2024-09-17T02:47:27.419774+0000","last_became_active":"2024-09-17T02:47:25.421196+0000","last_became_peered":"2024-09-17T02:47:25.421196+0000","last_unstale":"2024-09-17T02:47:27.419774+0000","last_undegraded":"2024-09-17T02:47:27.419774+0000","last_fullsized":"2024-09-17T02:47:27.419774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:54.515348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769031+0000","last_change":"2024-09-17T02:47:26.458333+0000","last_active":"2024-09-17T02:47:27.769031+0000","last_peered":"2024-09-17T02:47:27.769031+0000","last_clean":"2024-09-17T02:47:27.769031+0000","last_became_active":"2024-09-17T02:47:26.450921+0000","last_became_peered":"2024-09-17T02:47:26.450921+0000","last_unstale":"2024-09-17T02:47:27.769031+0000","last_undegraded":"2024-09-17T02:47:27.769031+0000","last_fullsized":"2024-09-17T02:47:27.769031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:36.379174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767855+0000","last_change":"2024-09-17T02:47:25.401532+0000","last_active":"2024-09-17T02:47:27.767855+0000","last_peered":"2024-09-17T02:47:27.767855+0000","last_clean":"2024-09-17T02:47:27.767855+0000","last_became_active":"2024-09-17T02:47:25.400932+0000","last_became_peered":"2024-09-17T02:47:25.400932+0000","last_unstale":"2024-09-17T02:47:27.767855+0000","last_undegraded":"2024-09-17T02:47:27.767855+0000","last_fullsized":"2024-09-17T02:47:27.767855+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:30.981067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416578+0000","last_change":"2024-09-17T02:47:26.407118+0000","last_active":"2024-09-17T02:47:27.416578+0000","last_peered":"2024-09-17T02:47:27.416578+0000","last_clean":"2024-09-17T02:47:27.416578+0000","last_became_active":"2024-09-17T02:47:26.406456+0000","last_became_peered":"2024-09-17T02:47:26.406456+0000","last_unstale":"2024-09-17T02:47:27.416578+0000","last_undegraded":"2024-09-17T02:47:27.416578+0000","last_fullsized":"2024-09-17T02:47:27.416578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:39.049060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746340+0000","last_change":"2024-09-17T02:47:25.397773+0000","last_active":"2024-09-17T02:47:27.746340+0000","last_peered":"2024-09-17T02:47:27.746340+0000","last_clean":"2024-09-17T02:47:27.746340+0000","last_became_active":"2024-09-17T02:47:25.397452+0000","last_became_peered":"2024-09-17T02:47:25.397452+0000","last_unstale":"2024-09-17T02:47:27.746340+0000","last_undegraded":"2024-09-17T02:47:27.746340+0000","last_fullsized":"2024-09-17T02:47:27.746340+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:34.937753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769006+0000","last_change":"2024-09-17T02:47:26.443993+0000","last_active":"2024-09-17T02:47:27.769006+0000","last_peered":"2024-09-17T02:47:27.769006+0000","last_clean":"2024-09-17T02:47:27.769006+0000","last_became_active":"2024-09-17T02:47:26.443651+0000","last_became_peered":"2024-09-17T02:47:26.443651+0000","last_unstale":"2024-09-17T02:47:27.769006+0000","last_undegraded":"2024-09-17T02:47:27.769006+0000","last_fullsized":"2024-09-17T02:47:27.769006+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:43.278442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746255+0000","last_change":"2024-09-17T02:47:25.396451+0000","last_active":"2024-09-17T02:47:27.746255+0000","last_peered":"2024-09-17T02:47:27.746255+0000","last_clean":"2024-09-17T02:47:27.746255+0000","last_became_active":"2024-09-17T02:47:25.396146+0000","last_became_peered":"2024-09-17T02:47:25.396146+0000","last_unstale":"2024-09-17T02:47:27.746255+0000","last_undegraded":"2024-09-17T02:47:27.746255+0000","last_fullsized":"2024-09-17T02:47:27.746255+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:28:55.866634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767876+0000","last_change":"2024-09-17T02:47:26.444819+0000","last_active":"2024-09-17T02:47:27.767876+0000","last_peered":"2024-09-17T02:47:27.767876+0000","last_clean":"2024-09-17T02:47:27.767876+0000","last_became_active":"2024-09-17T02:47:26.443794+0000","last_became_peered":"2024-09-17T02:47:26.443794+0000","last_unstale":"2024-09-17T02:47:27.767876+0000","last_undegraded":"2024-09-17T02:47:27.767876+0000","last_fullsized":"2024-09-17T02:47:27.767876+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:26.261568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416035+0000","last_change":"2024-09-17T02:47:25.407194+0000","last_active":"2024-09-17T02:47:27.416035+0000","last_peered":"2024-09-17T02:47:27.416035+0000","last_clean":"2024-09-17T02:47:27.416035+0000","last_became_active":"2024-09-17T02:47:25.406881+0000","last_became_peered":"2024-09-17T02:47:25.406881+0000","last_unstale":"2024-09-17T02:47:27.416035+0000","last_undegraded":"2024-09-17T02:47:27.416035+0000","last_fullsized":"2024-09-17T02:47:27.416035+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:20.997735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.745904+0000","last_change":"2024-09-17T02:47:26.424417+0000","last_active":"2024-09-17T02:47:27.745904+0000","last_peered":"2024-09-17T02:47:27.745904+0000","last_clean":"2024-09-17T02:47:27.745904+0000","last_became_active":"2024-09-17T02:47:26.424080+0000","last_became_peered":"2024-09-17T02:47:26.424080+0000","last_unstale":"2024-09-17T02:47:27.745904+0000","last_undegraded":"2024-09-17T02:47:27.745904+0000","last_fullsized":"2024-09-17T02:47:27.745904+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:00.083852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767755+0000","last_change":"2024-09-17T02:47:25.419788+0000","last_active":"2024-09-17T02:47:27.767755+0000","last_peered":"2024-09-17T02:47:27.767755+0000","last_clean":"2024-09-17T02:47:27.767755+0000","last_became_active":"2024-09-17T02:47:25.419102+0000","last_became_peered":"2024-09-17T02:47:25.419102+0000","last_unstale":"2024-09-17T02:47:27.767755+0000","last_undegraded":"2024-09-17T02:47:27.767755+0000","last_fullsized":"2024-09-17T02:47:27.767755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:15.584669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767923+0000","last_change":"2024-09-17T02:47:26.401986+0000","last_active":"2024-09-17T02:47:27.767923+0000","last_peered":"2024-09-17T02:47:27.767923+0000","last_clean":"2024-09-17T02:47:27.767923+0000","last_became_active":"2024-09-17T02:47:26.400697+0000","last_became_peered":"2024-09-17T02:47:26.400697+0000","last_unstale":"2024-09-17T02:47:27.767923+0000","last_undegraded":"2024-09-17T02:47:27.767923+0000","last_fullsized":"2024-09-17T02:47:27.767923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:19.106029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.745766+0000","last_change":"2024-09-17T02:47:25.421863+0000","last_active":"2024-09-17T02:47:27.745766+0000","last_peered":"2024-09-17T02:47:27.745766+0000","last_clean":"2024-09-17T02:47:27.745766+0000","last_became_active":"2024-09-17T02:47:25.421355+0000","last_became_peered":"2024-09-17T02:47:25.421355+0000","last_unstale":"2024-09-17T02:47:27.745766+0000","last_undegraded":"2024-09-17T02:47:27.745766+0000","last_fullsized":"2024-09-17T02:47:27.745766+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:20.461693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421042+0000","last_change":"2024-09-17T02:47:26.417573+0000","last_active":"2024-09-17T02:47:27.421042+0000","last_peered":"2024-09-17T02:47:27.421042+0000","last_clean":"2024-09-17T02:47:27.421042+0000","last_became_active":"2024-09-17T02:47:26.414446+0000","last_became_peered":"2024-09-17T02:47:26.414446+0000","last_unstale":"2024-09-17T02:47:27.421042+0000","last_undegraded":"2024-09-17T02:47:27.421042+0000","last_fullsized":"2024-09-17T02:47:27.421042+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:50.565801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767807+0000","last_change":"2024-09-17T02:47:25.413072+0000","last_active":"2024-09-17T02:47:27.767807+0000","last_peered":"2024-09-17T02:47:27.767807+0000","last_clean":"2024-09-17T02:47:27.767807+0000","last_became_active":"2024-09-17T02:47:25.412033+0000","last_became_peered":"2024-09-17T02:47:25.412033+0000","last_unstale":"2024-09-17T02:47:27.767807+0000","last_undegraded":"2024-09-17T02:47:27.767807+0000","last_fullsized":"2024-09-17T02:47:27.767807+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:25.100410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768080+0000","last_change":"2024-09-17T02:47:26.420721+0000","last_active":"2024-09-17T02:47:27.768080+0000","last_peered":"2024-09-17T02:47:27.768080+0000","last_clean":"2024-09-17T02:47:27.768080+0000","last_became_active":"2024-09-17T02:47:26.420354+0000","last_became_peered":"2024-09-17T02:47:26.420354+0000","last_unstale":"2024-09-17T02:47:27.768080+0000","last_undegraded":"2024-09-17T02:47:27.768080+0000","last_fullsized":"2024-09-17T02:47:27.768080+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:30:05.141305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416520+0000","last_change":"2024-09-17T02:47:25.390686+0000","last_active":"2024-09-17T02:47:27.416520+0000","last_peered":"2024-09-17T02:47:27.416520+0000","last_clean":"2024-09-17T02:47:27.416520+0000","last_became_active":"2024-09-17T02:47:25.390236+0000","last_became_peered":"2024-09-17T02:47:25.390236+0000","last_unstale":"2024-09-17T02:47:27.416520+0000","last_undegraded":"2024-09-17T02:47:27.416520+0000","last_fullsized":"2024-09-17T02:47:27.416520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:55.492515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768596+0000","last_change":"2024-09-17T02:47:26.450126+0000","last_active":"2024-09-17T02:47:27.768596+0000","last_peered":"2024-09-17T02:47:27.768596+0000","last_clean":"2024-09-17T02:47:27.768596+0000","last_became_active":"2024-09-17T02:47:26.445482+0000","last_became_peered":"2024-09-17T02:47:26.445482+0000","last_unstale":"2024-09-17T02:47:27.768596+0000","last_undegraded":"2024-09-17T02:47:27.768596+0000","last_fullsized":"2024-09-17T02:47:27.768596+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:45.742572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416576+0000","last_change":"2024-09-17T02:47:25.396097+0000","last_active":"2024-09-17T02:47:27.416576+0000","last_peered":"2024-09-17T02:47:27.416576+0000","last_clean":"2024-09-17T02:47:27.416576+0000","last_became_active":"2024-09-17T02:47:25.395670+0000","last_became_peered":"2024-09-17T02:47:25.395670+0000","last_unstale":"2024-09-17T02:47:27.416576+0000","last_undegraded":"2024-09-17T02:47:27.416576+0000","last_fullsized":"2024-09-17T02:47:27.416576+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:15.348388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769902+0000","last_change":"2024-09-17T02:47:26.449859+0000","last_active":"2024-09-17T02:47:27.769902+0000","last_peered":"2024-09-17T02:47:27.769902+0000","last_clean":"2024-09-17T02:47:27.769902+0000","last_became_active":"2024-09-17T02:47:26.449290+0000","last_became_peered":"2024-09-17T02:47:26.449290+0000","last_unstale":"2024-09-17T02:47:27.769902+0000","last_undegraded":"2024-09-17T02:47:27.769902+0000","last_fullsized":"2024-09-17T02:47:27.769902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:29.692087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769004+0000","last_change":"2024-09-17T02:47:25.414356+0000","last_active":"2024-09-17T02:47:27.769004+0000","last_peered":"2024-09-17T02:47:27.769004+0000","last_clean":"2024-09-17T02:47:27.769004+0000","last_became_active":"2024-09-17T02:47:25.413979+0000","last_became_peered":"2024-09-17T02:47:25.413979+0000","last_unstale":"2024-09-17T02:47:27.769004+0000","last_undegraded":"2024-09-17T02:47:27.769004+0000","last_fullsized":"2024-09-17T02:47:27.769004+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:59.127265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769424+0000","last_change":"2024-09-17T02:47:26.425824+0000","last_active":"2024-09-17T02:47:27.769424+0000","last_peered":"2024-09-17T02:47:27.769424+0000","last_clean":"2024-09-17T02:47:27.769424+0000","last_became_active":"2024-09-17T02:47:26.414310+0000","last_became_peered":"2024-09-17T02:47:26.414310+0000","last_unstale":"2024-09-17T02:47:27.769424+0000","last_undegraded":"2024-09-17T02:47:27.769424+0000","last_fullsized":"2024-09-17T02:47:27.769424+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:03.730055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416631+0000","last_change":"2024-09-17T02:47:25.392813+0000","last_active":"2024-09-17T02:47:27.416631+0000","last_peered":"2024-09-17T02:47:27.416631+0000","last_clean":"2024-09-17T02:47:27.416631+0000","last_became_active":"2024-09-17T02:47:25.392472+0000","last_became_peered":"2024-09-17T02:47:25.392472+0000","last_unstale":"2024-09-17T02:47:27.416631+0000","last_undegraded":"2024-09-17T02:47:27.416631+0000","last_fullsized":"2024-09-17T02:47:27.416631+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:05.045538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767992+0000","last_change":"2024-09-17T02:47:26.444405+0000","last_active":"2024-09-17T02:47:27.767992+0000","last_peered":"2024-09-17T02:47:27.767992+0000","last_clean":"2024-09-17T02:47:27.767992+0000","last_became_active":"2024-09-17T02:47:26.443000+0000","last_became_peered":"2024-09-17T02:47:26.443000+0000","last_unstale":"2024-09-17T02:47:27.767992+0000","last_undegraded":"2024-09-17T02:47:27.767992+0000","last_fullsized":"2024-09-17T02:47:27.767992+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:27.766149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421520+0000","last_change":"2024-09-17T02:47:25.404863+0000","last_active":"2024-09-17T02:47:27.421520+0000","last_peered":"2024-09-17T02:47:27.421520+0000","last_clean":"2024-09-17T02:47:27.421520+0000","last_became_active":"2024-09-17T02:47:25.403960+0000","last_became_peered":"2024-09-17T02:47:25.403960+0000","last_unstale":"2024-09-17T02:47:27.421520+0000","last_undegraded":"2024-09-17T02:47:27.421520+0000","last_fullsized":"2024-09-17T02:47:27.421520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:38.479422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421658+0000","last_change":"2024-09-17T02:47:26.402827+0000","last_active":"2024-09-17T02:47:27.421658+0000","last_peered":"2024-09-17T02:47:27.421658+0000","last_clean":"2024-09-17T02:47:27.421658+0000","last_became_active":"2024-09-17T02:47:26.402001+0000","last_became_peered":"2024-09-17T02:47:26.402001+0000","last_unstale":"2024-09-17T02:47:27.421658+0000","last_undegraded":"2024-09-17T02:47:27.421658+0000","last_fullsized":"2024-09-17T02:47:27.421658+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:09.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769530+0000","last_change":"2024-09-17T02:47:25.403962+0000","last_active":"2024-09-17T02:47:27.769530+0000","last_peered":"2024-09-17T02:47:27.769530+0000","last_clean":"2024-09-17T02:47:27.769530+0000","last_became_active":"2024-09-17T02:47:25.403605+0000","last_became_peered":"2024-09-17T02:47:25.403605+0000","last_unstale":"2024-09-17T02:47:27.769530+0000","last_undegraded":"2024-09-17T02:47:27.769530+0000","last_fullsized":"2024-09-17T02:47:27.769530+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:31.843272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768230+0000","last_change":"2024-09-17T02:47:26.432557+0000","last_active":"2024-09-17T02:47:27.768230+0000","last_peered":"2024-09-17T02:47:27.768230+0000","last_clean":"2024-09-17T02:47:27.768230+0000","last_became_active":"2024-09-17T02:47:26.432209+0000","last_became_peered":"2024-09-17T02:47:26.432209+0000","last_unstale":"2024-09-17T02:47:27.768230+0000","last_undegraded":"2024-09-17T02:47:27.768230+0000","last_fullsized":"2024-09-17T02:47:27.768230+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:40.137137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767271+0000","last_change":"2024-09-17T02:47:25.410837+0000","last_active":"2024-09-17T02:47:27.767271+0000","last_peered":"2024-09-17T02:47:27.767271+0000","last_clean":"2024-09-17T02:47:27.767271+0000","last_became_active":"2024-09-17T02:47:25.410644+0000","last_became_peered":"2024-09-17T02:47:25.410644+0000","last_unstale":"2024-09-17T02:47:27.767271+0000","last_undegraded":"2024-09-17T02:47:27.767271+0000","last_fullsized":"2024-09-17T02:47:27.767271+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:41.805249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767390+0000","last_change":"2024-09-17T02:47:26.448485+0000","last_active":"2024-09-17T02:47:27.767390+0000","last_peered":"2024-09-17T02:47:27.767390+0000","last_clean":"2024-09-17T02:47:27.767390+0000","last_became_active":"2024-09-17T02:47:26.447870+0000","last_became_peered":"2024-09-17T02:47:26.447870+0000","last_unstale":"2024-09-17T02:47:27.767390+0000","last_undegraded":"2024-09-17T02:47:27.767390+0000","last_fullsized":"2024-09-17T02:47:27.767390+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:20.845608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766663+0000","last_change":"2024-09-17T02:47:25.396522+0000","last_active":"2024-09-17T02:47:27.766663+0000","last_peered":"2024-09-17T02:47:27.766663+0000","last_clean":"2024-09-17T02:47:27.766663+0000","last_became_active":"2024-09-17T02:47:25.395729+0000","last_became_peered":"2024-09-17T02:47:25.395729+0000","last_unstale":"2024-09-17T02:47:27.766663+0000","last_undegraded":"2024-09-17T02:47:27.766663+0000","last_fullsized":"2024-09-17T02:47:27.766663+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:58.843240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769403+0000","last_change":"2024-09-17T02:47:26.451744+0000","last_active":"2024-09-17T02:47:27.769403+0000","last_peered":"2024-09-17T02:47:27.769403+0000","last_clean":"2024-09-17T02:47:27.769403+0000","last_became_active":"2024-09-17T02:47:26.451445+0000","last_became_peered":"2024-09-17T02:47:26.451445+0000","last_unstale":"2024-09-17T02:47:27.769403+0000","last_undegraded":"2024-09-17T02:47:27.769403+0000","last_fullsized":"2024-09-17T02:47:27.769403+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:54.458584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421501+0000","last_change":"2024-09-17T02:47:25.404744+0000","last_active":"2024-09-17T02:47:27.421501+0000","last_peered":"2024-09-17T02:47:27.421501+0000","last_clean":"2024-09-17T02:47:27.421501+0000","last_became_active":"2024-09-17T02:47:25.403842+0000","last_became_peered":"2024-09-17T02:47:25.403842+0000","last_unstale":"2024-09-17T02:47:27.421501+0000","last_undegraded":"2024-09-17T02:47:27.421501+0000","last_fullsized":"2024-09-17T02:47:27.421501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:02.237396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":246,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767020+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:47:27.767020+0000","last_peered":"2024-09-17T02:47:27.767020+0000","last_clean":"2024-09-17T02:47:27.767020+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:47:27.767020+0000","last_undegraded":"2024-09-17T02:47:27.767020+0000","last_fullsized":"2024-09-17T02:47:27.767020+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509526,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":517803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1720320,"data_stored":1108171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":517803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509525,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1146880,"data_stored":519527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509526,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1134592,"data_stored":518407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509526,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1732608,"data_stored":1108857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509527,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1130496,"data_stored":517873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1146880,"data_stored":519539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:47:31.345 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T02:47:31.349 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-09-17T02:47:31.353+0000 7fa75fc2c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:47:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 -- 172.21.15.131:0/644283938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8080ced00 msgr2=0x7fd80806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 --2- 172.21.15.131:0/644283938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8080ced00 0x7fd80806dea0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009c80 tx=0x7fd7f801cbc0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 -- 172.21.15.131:0/644283938 shutdown_connections 2024-09-17T02:47:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 --2- 172.21.15.131:0/644283938 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd80806e3e0 0x7fd80815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 --2- 172.21.15.131:0/644283938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8080ced00 0x7fd80806dea0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 -- 172.21.15.131:0/644283938 >> 172.21.15.131:0/644283938 conn(0x7fd80806eea0 msgr2=0x7fd80806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 -- 172.21.15.131:0/644283938 shutdown_connections 2024-09-17T02:47:31.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 -- 172.21.15.131:0/644283938 wait complete. 2024-09-17T02:47:31.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 Processor -- start 2024-09-17T02:47:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 -- start start 2024-09-17T02:47:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd80806e3e0 0x7fd8081549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8080ced00 0x7fd808160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd80815c3e0 0x7fd8081f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.505 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd808168d70 con 0x7fd80815c3e0 2024-09-17T02:47:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd808168bf0 con 0x7fd8080ced00 2024-09-17T02:47:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.501+0000 7fd80fcc7640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd808168ef0 con 0x7fd80806e3e0 2024-09-17T02:47:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80e264640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd80815c3e0 0x7fd8081f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80e264640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd80815c3e0 0x7fd8081f6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43434/0 (socket says 172.21.15.131:43434) 2024-09-17T02:47:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80e264640 1 -- 172.21.15.131:0/4250085543 learned_addr learned my addr 172.21.15.131:0/4250085543 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:31.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80d262640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8080ced00 0x7fd808160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80da63640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd80806e3e0 0x7fd8081549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80e264640 1 -- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd80806e3e0 msgr2=0x7fd8081549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80e264640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd80806e3e0 0x7fd8081549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80e264640 1 -- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8080ced00 msgr2=0x7fd808160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80e264640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8080ced00 0x7fd808160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80e264640 1 -- 172.21.15.131:0/4250085543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8081fa050 con 0x7fd80815c3e0 2024-09-17T02:47:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80d262640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8080ced00 0x7fd808160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80e264640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd80815c3e0 0x7fd8081f6a70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd80000b410 tx=0x7fd80000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd8067fc640 1 -- 172.21.15.131:0/4250085543 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800015020 con 0x7fd80815c3e0 2024-09-17T02:47:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd8067fc640 1 -- 172.21.15.131:0/4250085543 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd800011920 con 0x7fd80815c3e0 2024-09-17T02:47:31.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd8067fc640 1 -- 172.21.15.131:0/4250085543 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800011d70 con 0x7fd80815c3e0 2024-09-17T02:47:31.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80fcc7640 1 -- 172.21.15.131:0/4250085543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8081fa4c0 con 0x7fd80815c3e0 2024-09-17T02:47:31.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80fcc7640 1 -- 172.21.15.131:0/4250085543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8081faa80 con 0x7fd80815c3e0 2024-09-17T02:47:31.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80fcc7640 1 -- 172.21.15.131:0/4250085543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80815bea0 con 0x7fd80815c3e0 2024-09-17T02:47:31.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd8067fc640 1 -- 172.21.15.131:0/4250085543 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd800018020 con 0x7fd80815c3e0 2024-09-17T02:47:31.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd8067fc640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd7f4073200 0x7fd7f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd8067fc640 1 -- 172.21.15.131:0/4250085543 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7fd800094190 con 0x7fd80815c3e0 2024-09-17T02:47:31.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.505+0000 7fd80da63640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd7f4073200 0x7fd7f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.509+0000 7fd80da63640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd7f4073200 0x7fd7f4075710 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80af000 tx=0x7fd7f801b400 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:31.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.509+0000 7fd8067fc640 1 -- 172.21.15.131:0/4250085543 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd80001f020 con 0x7fd80815c3e0 2024-09-17T02:47:31.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.733+0000 7fd80fcc7640 1 -- 172.21.15.131:0/4250085543 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fd8081fa220 con 0x7fd80815c3e0 2024-09-17T02:47:31.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.733+0000 7fd8067fc640 1 -- 172.21.15.131:0/4250085543 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1056 (secure 0 0 0) 0x7fd80005f140 con 0x7fd80815c3e0 2024-09-17T02:47:31.732 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:31.732 INFO:teuthology.orchestra.run.smithi131.stdout:{"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":425,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":92,"num_osds":8,"num_up_osds":8,"osd_up_since":1726540840,"num_in_osds":8,"osd_in_since":1726540830,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":265801728,"bytes_avail":772828311552,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":64,"btime":"2024-09-17T02:47:29:415704+0000","id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7038}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-09-17T02:47:07.269046+0000","services":{}},"progress_events":{}} 2024-09-17T02:47:31.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 -- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd7f4073200 msgr2=0x7fd7f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd7f4073200 0x7fd7f4075710 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80af000 tx=0x7fd7f801b400 comp rx=0 tx=0).stop 2024-09-17T02:47:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 -- 172.21.15.131:0/4250085543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd80815c3e0 msgr2=0x7fd8081f6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd80815c3e0 0x7fd8081f6a70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd80000b410 tx=0x7fd80000b8d0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 -- 172.21.15.131:0/4250085543 shutdown_connections 2024-09-17T02:47:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd7f4073200 0x7fd7f4075710 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd80815c3e0 0x7fd8081f6a70 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd8080ced00 0x7fd808160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 --2- 172.21.15.131:0/4250085543 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd80806e3e0 0x7fd8081549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 -- 172.21.15.131:0/4250085543 >> 172.21.15.131:0/4250085543 conn(0x7fd80806eea0 msgr2=0x7fd8080b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 -- 172.21.15.131:0/4250085543 shutdown_connections 2024-09-17T02:47:31.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.737+0000 7fd80fcc7640 1 -- 172.21.15.131:0/4250085543 wait complete. 2024-09-17T02:47:31.753 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T02:47:31.753 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:47:31.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.897+0000 7f76ec6cc640 1 -- 172.21.15.131:0/890708568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76e415c5b0 msgr2=0x7f76e40cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.897+0000 7f76ec6cc640 1 --2- 172.21.15.131:0/890708568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76e415c5b0 0x7f76e40cbaf0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f76dc009900 tx=0x7f76dc01c8e0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.897+0000 7f76ec6cc640 1 -- 172.21.15.131:0/890708568 shutdown_connections 2024-09-17T02:47:31.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.897+0000 7f76ec6cc640 1 --2- 172.21.15.131:0/890708568 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76e415c5b0 0x7f76e40cbaf0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.897+0000 7f76ec6cc640 1 --2- 172.21.15.131:0/890708568 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f76e40ced00 0x7f76e406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.897+0000 7f76ec6cc640 1 -- 172.21.15.131:0/890708568 >> 172.21.15.131:0/890708568 conn(0x7f76e406eea0 msgr2=0x7f76e406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:31.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.897+0000 7f76ec6cc640 1 -- 172.21.15.131:0/890708568 shutdown_connections 2024-09-17T02:47:31.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.897+0000 7f76ec6cc640 1 -- 172.21.15.131:0/890708568 wait complete. 2024-09-17T02:47:31.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.897+0000 7f76ec6cc640 1 Processor -- start 2024-09-17T02:47:31.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.897+0000 7f76ec6cc640 1 -- start start 2024-09-17T02:47:31.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76ec6cc640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f76e406e3e0 0x7f76e4154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76ec6cc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76e40ced00 0x7f76e4154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76ec6cc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76e415c5b0 0x7f76e41fbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76ec6cc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f76e4168720 con 0x7f76e40ced00 2024-09-17T02:47:31.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76ec6cc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f76e41685a0 con 0x7f76e415c5b0 2024-09-17T02:47:31.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76ec6cc640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f76e41688a0 con 0x7f76e406e3e0 2024-09-17T02:47:31.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76e9c67640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76e40ced00 0x7f76e4154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76e9c67640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76e40ced00 0x7f76e4154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43464/0 (socket says 172.21.15.131:43464) 2024-09-17T02:47:31.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76e9c67640 1 -- 172.21.15.131:0/2829962833 learned_addr learned my addr 172.21.15.131:0/2829962833 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:31.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76eac69640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76e415c5b0 0x7f76e41fbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76ea468640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f76e406e3e0 0x7f76e4154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76e9c67640 1 -- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f76e406e3e0 msgr2=0x7f76e4154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76e9c67640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f76e406e3e0 0x7f76e4154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76e9c67640 1 -- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76e415c5b0 msgr2=0x7f76e41fbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:31.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76e9c67640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76e415c5b0 0x7f76e41fbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:31.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76e9c67640 1 -- 172.21.15.131:0/2829962833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e4154bc0 con 0x7f76e40ced00 2024-09-17T02:47:31.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76e9c67640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76e40ced00 0x7f76e4154680 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f76d400c3c0 tx=0x7f76d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:31.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76daffd640 1 -- 172.21.15.131:0/2829962833 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4017070 con 0x7f76e40ced00 2024-09-17T02:47:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76daffd640 1 -- 172.21.15.131:0/2829962833 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76d40053c0 con 0x7f76e40ced00 2024-09-17T02:47:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76daffd640 1 -- 172.21.15.131:0/2829962833 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4003860 con 0x7f76e40ced00 2024-09-17T02:47:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76ec6cc640 1 -- 172.21.15.131:0/2829962833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e40af5c0 con 0x7f76e40ced00 2024-09-17T02:47:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76ec6cc640 1 -- 172.21.15.131:0/2829962833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e415e240 con 0x7f76e40ced00 2024-09-17T02:47:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.901+0000 7f76ec6cc640 1 -- 172.21.15.131:0/2829962833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e406dea0 con 0x7f76e40ced00 2024-09-17T02:47:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.905+0000 7f76daffd640 1 -- 172.21.15.131:0/2829962833 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f76d401a070 con 0x7f76e40ced00 2024-09-17T02:47:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.905+0000 7f76daffd640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f76c8073200 0x7f76c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.905+0000 7f76daffd640 1 -- 172.21.15.131:0/2829962833 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 6101+0+0 (secure 0 0 0) 0x7f76d4093710 con 0x7f76e40ced00 2024-09-17T02:47:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.905+0000 7f76ea468640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f76c8073200 0x7f76c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.905+0000 7f76ea468640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f76c8073200 0x7f76c8075710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f76e0005c50 tx=0x7f76e0005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:31.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:31.909+0000 7f76daffd640 1 -- 172.21.15.131:0/2829962833 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f76e406e7f0 con 0x7f76e40ced00 2024-09-17T02:47:32.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.069+0000 7f76ec6cc640 1 -- 172.21.15.131:0/2829962833 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76e40cbb90 con 0x7f76c8073200 2024-09-17T02:47:32.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.081+0000 7f76daffd640 1 -- 172.21.15.131:0/2829962833 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252075 (secure 0 0 0) 0x7f76e40cbb90 con 0x7f76c8073200 2024-09-17T02:47:32.080 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:32.081 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:47:32.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 -- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f76c8073200 msgr2=0x7f76c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:32.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f76c8073200 0x7f76c8075710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f76e0005c50 tx=0x7f76e0005be0 comp rx=0 tx=0).stop 2024-09-17T02:47:32.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 -- 172.21.15.131:0/2829962833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76e40ced00 msgr2=0x7f76e4154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:32.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76e40ced00 0x7f76e4154680 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f76d400c3c0 tx=0x7f76d400c880 comp rx=0 tx=0).stop 2024-09-17T02:47:32.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 -- 172.21.15.131:0/2829962833 shutdown_connections 2024-09-17T02:47:32.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f76c8073200 0x7f76c8075710 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:32.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f76e415c5b0 0x7f76e41fbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:32.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f76e40ced00 0x7f76e4154680 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:32.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 --2- 172.21.15.131:0/2829962833 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f76e406e3e0 0x7f76e4154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:32.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 -- 172.21.15.131:0/2829962833 >> 172.21.15.131:0/2829962833 conn(0x7f76e406eea0 msgr2=0x7f76e406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:32.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 -- 172.21.15.131:0/2829962833 shutdown_connections 2024-09-17T02:47:32.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:32.085+0000 7f76ec6cc640 1 -- 172.21.15.131:0/2829962833 wait complete. 2024-09-17T02:47:32.099 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-17T02:47:31.276426+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259572,"kb_used_data":10024,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715148,"statfs":{"total":773094113280,"available":772828311552,"internally_reserved":0,"allocated":10264576,"data_stored":5327980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":54,"apply_latency_ms":54,"commit_latency_ns":54000000,"apply_latency_ns":54000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001964"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422667+0000","last_change":"2024-09-17T02:47:26.451633+0000","last_active":"2024-09-17T02:47:27.422667+0000","last_peered":"2024-09-17T02:47:27.422667+0000","last_clean":"2024-09-17T02:47:27.422667+0000","last_became_active":"2024-09-17T02:47:26.450812+0000","last_became_peered":"2024-09-17T02:47:26.450812+0000","last_unstale":"2024-09-17T02:47:27.422667+0000","last_undegraded":"2024-09-17T02:47:27.422667+0000","last_fullsized":"2024-09-17T02:47:27.422667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:06.130998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768309+0000","last_change":"2024-09-17T02:47:26.435817+0000","last_active":"2024-09-17T02:47:27.768309+0000","last_peered":"2024-09-17T02:47:27.768309+0000","last_clean":"2024-09-17T02:47:27.768309+0000","last_became_active":"2024-09-17T02:47:26.435429+0000","last_became_peered":"2024-09-17T02:47:26.435429+0000","last_unstale":"2024-09-17T02:47:27.768309+0000","last_undegraded":"2024-09-17T02:47:27.768309+0000","last_fullsized":"2024-09-17T02:47:27.768309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:03.221935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422426+0000","last_change":"2024-09-17T02:47:26.422687+0000","last_active":"2024-09-17T02:47:27.422426+0000","last_peered":"2024-09-17T02:47:27.422426+0000","last_clean":"2024-09-17T02:47:27.422426+0000","last_became_active":"2024-09-17T02:47:26.421534+0000","last_became_peered":"2024-09-17T02:47:26.421534+0000","last_unstale":"2024-09-17T02:47:27.422426+0000","last_undegraded":"2024-09-17T02:47:27.422426+0000","last_fullsized":"2024-09-17T02:47:27.422426+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:28.446385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419818+0000","last_change":"2024-09-17T02:47:26.419919+0000","last_active":"2024-09-17T02:47:27.419818+0000","last_peered":"2024-09-17T02:47:27.419818+0000","last_clean":"2024-09-17T02:47:27.419818+0000","last_became_active":"2024-09-17T02:47:26.419062+0000","last_became_peered":"2024-09-17T02:47:26.419062+0000","last_unstale":"2024-09-17T02:47:27.419818+0000","last_undegraded":"2024-09-17T02:47:27.419818+0000","last_fullsized":"2024-09-17T02:47:27.419818+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:17.965654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766901+0000","last_change":"2024-09-17T02:47:26.425433+0000","last_active":"2024-09-17T02:47:27.766901+0000","last_peered":"2024-09-17T02:47:27.766901+0000","last_clean":"2024-09-17T02:47:27.766901+0000","last_became_active":"2024-09-17T02:47:26.420685+0000","last_became_peered":"2024-09-17T02:47:26.420685+0000","last_unstale":"2024-09-17T02:47:27.766901+0000","last_undegraded":"2024-09-17T02:47:27.766901+0000","last_fullsized":"2024-09-17T02:47:27.766901+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:15.124007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766590+0000","last_change":"2024-09-17T02:47:26.454339+0000","last_active":"2024-09-17T02:47:27.766590+0000","last_peered":"2024-09-17T02:47:27.766590+0000","last_clean":"2024-09-17T02:47:27.766590+0000","last_became_active":"2024-09-17T02:47:26.453723+0000","last_became_peered":"2024-09-17T02:47:26.453723+0000","last_unstale":"2024-09-17T02:47:27.766590+0000","last_undegraded":"2024-09-17T02:47:27.766590+0000","last_fullsized":"2024-09-17T02:47:27.766590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:30.811217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.415989+0000","last_change":"2024-09-17T02:47:26.431390+0000","last_active":"2024-09-17T02:47:27.415989+0000","last_peered":"2024-09-17T02:47:27.415989+0000","last_clean":"2024-09-17T02:47:27.415989+0000","last_became_active":"2024-09-17T02:47:26.431015+0000","last_became_peered":"2024-09-17T02:47:26.431015+0000","last_unstale":"2024-09-17T02:47:27.415989+0000","last_undegraded":"2024-09-17T02:47:27.415989+0000","last_fullsized":"2024-09-17T02:47:27.415989+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:51.096970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766358+0000","last_change":"2024-09-17T02:47:26.432290+0000","last_active":"2024-09-17T02:47:27.766358+0000","last_peered":"2024-09-17T02:47:27.766358+0000","last_clean":"2024-09-17T02:47:27.766358+0000","last_became_active":"2024-09-17T02:47:26.431479+0000","last_became_peered":"2024-09-17T02:47:26.431479+0000","last_unstale":"2024-09-17T02:47:27.766358+0000","last_undegraded":"2024-09-17T02:47:27.766358+0000","last_fullsized":"2024-09-17T02:47:27.766358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:39.578242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416330+0000","last_change":"2024-09-17T02:47:26.428950+0000","last_active":"2024-09-17T02:47:27.416330+0000","last_peered":"2024-09-17T02:47:27.416330+0000","last_clean":"2024-09-17T02:47:27.416330+0000","last_became_active":"2024-09-17T02:47:26.428599+0000","last_became_peered":"2024-09-17T02:47:26.428599+0000","last_unstale":"2024-09-17T02:47:27.416330+0000","last_undegraded":"2024-09-17T02:47:27.416330+0000","last_fullsized":"2024-09-17T02:47:27.416330+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:07.335822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419341+0000","last_change":"2024-09-17T02:47:26.433543+0000","last_active":"2024-09-17T02:47:27.419341+0000","last_peered":"2024-09-17T02:47:27.419341+0000","last_clean":"2024-09-17T02:47:27.419341+0000","last_became_active":"2024-09-17T02:47:26.432850+0000","last_became_peered":"2024-09-17T02:47:26.432850+0000","last_unstale":"2024-09-17T02:47:27.419341+0000","last_undegraded":"2024-09-17T02:47:27.419341+0000","last_fullsized":"2024-09-17T02:47:27.419341+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:23.938528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416194+0000","last_change":"2024-09-17T02:47:26.429547+0000","last_active":"2024-09-17T02:47:27.416194+0000","last_peered":"2024-09-17T02:47:27.416194+0000","last_clean":"2024-09-17T02:47:27.416194+0000","last_became_active":"2024-09-17T02:47:26.429189+0000","last_became_peered":"2024-09-17T02:47:26.429189+0000","last_unstale":"2024-09-17T02:47:27.416194+0000","last_undegraded":"2024-09-17T02:47:27.416194+0000","last_fullsized":"2024-09-17T02:47:27.416194+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:35.392361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.420510+0000","last_change":"2024-09-17T02:47:26.453163+0000","last_active":"2024-09-17T02:47:27.420510+0000","last_peered":"2024-09-17T02:47:27.420510+0000","last_clean":"2024-09-17T02:47:27.420510+0000","last_became_active":"2024-09-17T02:47:26.452601+0000","last_became_peered":"2024-09-17T02:47:26.452601+0000","last_unstale":"2024-09-17T02:47:27.420510+0000","last_undegraded":"2024-09-17T02:47:27.420510+0000","last_fullsized":"2024-09-17T02:47:27.420510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:34.534013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768784+0000","last_change":"2024-09-17T02:47:26.443086+0000","last_active":"2024-09-17T02:47:27.768784+0000","last_peered":"2024-09-17T02:47:27.768784+0000","last_clean":"2024-09-17T02:47:27.768784+0000","last_became_active":"2024-09-17T02:47:26.442205+0000","last_became_peered":"2024-09-17T02:47:26.442205+0000","last_unstale":"2024-09-17T02:47:27.768784+0000","last_undegraded":"2024-09-17T02:47:27.768784+0000","last_fullsized":"2024-09-17T02:47:27.768784+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:25.973524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746489+0000","last_change":"2024-09-17T02:47:26.423413+0000","last_active":"2024-09-17T02:47:27.746489+0000","last_peered":"2024-09-17T02:47:27.746489+0000","last_clean":"2024-09-17T02:47:27.746489+0000","last_became_active":"2024-09-17T02:47:26.423062+0000","last_became_peered":"2024-09-17T02:47:26.423062+0000","last_unstale":"2024-09-17T02:47:27.746489+0000","last_undegraded":"2024-09-17T02:47:27.746489+0000","last_fullsized":"2024-09-17T02:47:27.746489+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:16.597414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421280+0000","last_change":"2024-09-17T02:47:26.428868+0000","last_active":"2024-09-17T02:47:27.421280+0000","last_peered":"2024-09-17T02:47:27.421280+0000","last_clean":"2024-09-17T02:47:27.421280+0000","last_became_active":"2024-09-17T02:47:26.426622+0000","last_became_peered":"2024-09-17T02:47:26.426622+0000","last_unstale":"2024-09-17T02:47:27.421280+0000","last_undegraded":"2024-09-17T02:47:27.421280+0000","last_fullsized":"2024-09-17T02:47:27.421280+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:15.955660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766351+0000","last_change":"2024-09-17T02:47:26.434274+0000","last_active":"2024-09-17T02:47:27.766351+0000","last_peered":"2024-09-17T02:47:27.766351+0000","last_clean":"2024-09-17T02:47:27.766351+0000","last_became_active":"2024-09-17T02:47:26.433517+0000","last_became_peered":"2024-09-17T02:47:26.433517+0000","last_unstale":"2024-09-17T02:47:27.766351+0000","last_undegraded":"2024-09-17T02:47:27.766351+0000","last_fullsized":"2024-09-17T02:47:27.766351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:00.041629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769558+0000","last_change":"2024-09-17T02:47:26.453403+0000","last_active":"2024-09-17T02:47:27.769558+0000","last_peered":"2024-09-17T02:47:27.769558+0000","last_clean":"2024-09-17T02:47:27.769558+0000","last_became_active":"2024-09-17T02:47:26.452810+0000","last_became_peered":"2024-09-17T02:47:26.452810+0000","last_unstale":"2024-09-17T02:47:27.769558+0000","last_undegraded":"2024-09-17T02:47:27.769558+0000","last_fullsized":"2024-09-17T02:47:27.769558+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:06.058284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419516+0000","last_change":"2024-09-17T02:47:26.406401+0000","last_active":"2024-09-17T02:47:27.419516+0000","last_peered":"2024-09-17T02:47:27.419516+0000","last_clean":"2024-09-17T02:47:27.419516+0000","last_became_active":"2024-09-17T02:47:26.405810+0000","last_became_peered":"2024-09-17T02:47:26.405810+0000","last_unstale":"2024-09-17T02:47:27.419516+0000","last_undegraded":"2024-09-17T02:47:27.419516+0000","last_fullsized":"2024-09-17T02:47:27.419516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:17.427456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768538+0000","last_change":"2024-09-17T02:47:26.421898+0000","last_active":"2024-09-17T02:47:27.768538+0000","last_peered":"2024-09-17T02:47:27.768538+0000","last_clean":"2024-09-17T02:47:27.768538+0000","last_became_active":"2024-09-17T02:47:26.421595+0000","last_became_peered":"2024-09-17T02:47:26.421595+0000","last_unstale":"2024-09-17T02:47:27.768538+0000","last_undegraded":"2024-09-17T02:47:27.768538+0000","last_fullsized":"2024-09-17T02:47:27.768538+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:10.282061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768163+0000","last_change":"2024-09-17T02:47:26.456961+0000","last_active":"2024-09-17T02:47:27.768163+0000","last_peered":"2024-09-17T02:47:27.768163+0000","last_clean":"2024-09-17T02:47:27.768163+0000","last_became_active":"2024-09-17T02:47:26.456375+0000","last_became_peered":"2024-09-17T02:47:26.456375+0000","last_unstale":"2024-09-17T02:47:27.768163+0000","last_undegraded":"2024-09-17T02:47:27.768163+0000","last_fullsized":"2024-09-17T02:47:27.768163+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:22.010481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769004+0000","last_change":"2024-09-17T02:47:26.425659+0000","last_active":"2024-09-17T02:47:27.769004+0000","last_peered":"2024-09-17T02:47:27.769004+0000","last_clean":"2024-09-17T02:47:27.769004+0000","last_became_active":"2024-09-17T02:47:26.418728+0000","last_became_peered":"2024-09-17T02:47:26.418728+0000","last_unstale":"2024-09-17T02:47:27.769004+0000","last_undegraded":"2024-09-17T02:47:27.769004+0000","last_fullsized":"2024-09-17T02:47:27.769004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:12.211104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767567+0000","last_change":"2024-09-17T02:47:26.420752+0000","last_active":"2024-09-17T02:47:27.767567+0000","last_peered":"2024-09-17T02:47:27.767567+0000","last_clean":"2024-09-17T02:47:27.767567+0000","last_became_active":"2024-09-17T02:47:26.420423+0000","last_became_peered":"2024-09-17T02:47:26.420423+0000","last_unstale":"2024-09-17T02:47:27.767567+0000","last_undegraded":"2024-09-17T02:47:27.767567+0000","last_fullsized":"2024-09-17T02:47:27.767567+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:50.371938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766752+0000","last_change":"2024-09-17T02:47:26.449510+0000","last_active":"2024-09-17T02:47:27.766752+0000","last_peered":"2024-09-17T02:47:27.766752+0000","last_clean":"2024-09-17T02:47:27.766752+0000","last_became_active":"2024-09-17T02:47:26.449136+0000","last_became_peered":"2024-09-17T02:47:26.449136+0000","last_unstale":"2024-09-17T02:47:27.766752+0000","last_undegraded":"2024-09-17T02:47:27.766752+0000","last_fullsized":"2024-09-17T02:47:27.766752+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:46.727764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.418593+0000","last_change":"2024-09-17T02:47:26.438730+0000","last_active":"2024-09-17T02:47:27.418593+0000","last_peered":"2024-09-17T02:47:27.418593+0000","last_clean":"2024-09-17T02:47:27.418593+0000","last_became_active":"2024-09-17T02:47:26.420076+0000","last_became_peered":"2024-09-17T02:47:26.420076+0000","last_unstale":"2024-09-17T02:47:27.418593+0000","last_undegraded":"2024-09-17T02:47:27.418593+0000","last_fullsized":"2024-09-17T02:47:27.418593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:12.606293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768851+0000","last_change":"2024-09-17T02:47:26.428029+0000","last_active":"2024-09-17T02:47:27.768851+0000","last_peered":"2024-09-17T02:47:27.768851+0000","last_clean":"2024-09-17T02:47:27.768851+0000","last_became_active":"2024-09-17T02:47:26.427694+0000","last_became_peered":"2024-09-17T02:47:26.427694+0000","last_unstale":"2024-09-17T02:47:27.768851+0000","last_undegraded":"2024-09-17T02:47:27.768851+0000","last_fullsized":"2024-09-17T02:47:27.768851+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:10.261716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768992+0000","last_change":"2024-09-17T02:47:26.427423+0000","last_active":"2024-09-17T02:47:27.768992+0000","last_peered":"2024-09-17T02:47:27.768992+0000","last_clean":"2024-09-17T02:47:27.768992+0000","last_became_active":"2024-09-17T02:47:26.427019+0000","last_became_peered":"2024-09-17T02:47:26.427019+0000","last_unstale":"2024-09-17T02:47:27.768992+0000","last_undegraded":"2024-09-17T02:47:27.768992+0000","last_fullsized":"2024-09-17T02:47:27.768992+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:10.586243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422781+0000","last_change":"2024-09-17T02:47:26.411901+0000","last_active":"2024-09-17T02:47:27.422781+0000","last_peered":"2024-09-17T02:47:27.422781+0000","last_clean":"2024-09-17T02:47:27.422781+0000","last_became_active":"2024-09-17T02:47:26.409711+0000","last_became_peered":"2024-09-17T02:47:26.409711+0000","last_unstale":"2024-09-17T02:47:27.422781+0000","last_undegraded":"2024-09-17T02:47:27.422781+0000","last_fullsized":"2024-09-17T02:47:27.422781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:04.645639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767833+0000","last_change":"2024-09-17T02:47:26.451152+0000","last_active":"2024-09-17T02:47:27.767833+0000","last_peered":"2024-09-17T02:47:27.767833+0000","last_clean":"2024-09-17T02:47:27.767833+0000","last_became_active":"2024-09-17T02:47:26.445335+0000","last_became_peered":"2024-09-17T02:47:26.445335+0000","last_unstale":"2024-09-17T02:47:27.767833+0000","last_undegraded":"2024-09-17T02:47:27.767833+0000","last_fullsized":"2024-09-17T02:47:27.767833+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:39.809055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769745+0000","last_change":"2024-09-17T02:47:26.453695+0000","last_active":"2024-09-17T02:47:27.769745+0000","last_peered":"2024-09-17T02:47:27.769745+0000","last_clean":"2024-09-17T02:47:27.769745+0000","last_became_active":"2024-09-17T02:47:26.453394+0000","last_became_peered":"2024-09-17T02:47:26.453394+0000","last_unstale":"2024-09-17T02:47:27.769745+0000","last_undegraded":"2024-09-17T02:47:27.769745+0000","last_fullsized":"2024-09-17T02:47:27.769745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:50.371938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766880+0000","last_change":"2024-09-17T02:47:26.424922+0000","last_active":"2024-09-17T02:47:27.766880+0000","last_peered":"2024-09-17T02:47:27.766880+0000","last_clean":"2024-09-17T02:47:27.766880+0000","last_became_active":"2024-09-17T02:47:26.418098+0000","last_became_peered":"2024-09-17T02:47:26.418098+0000","last_unstale":"2024-09-17T02:47:27.766880+0000","last_undegraded":"2024-09-17T02:47:27.766880+0000","last_fullsized":"2024-09-17T02:47:27.766880+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:18.176172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767618+0000","last_change":"2024-09-17T02:47:26.411672+0000","last_active":"2024-09-17T02:47:27.767618+0000","last_peered":"2024-09-17T02:47:27.767618+0000","last_clean":"2024-09-17T02:47:27.767618+0000","last_became_active":"2024-09-17T02:47:26.411280+0000","last_became_peered":"2024-09-17T02:47:26.411280+0000","last_unstale":"2024-09-17T02:47:27.767618+0000","last_undegraded":"2024-09-17T02:47:27.767618+0000","last_fullsized":"2024-09-17T02:47:27.767618+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:43.278442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767318+0000","last_change":"2024-09-17T02:47:26.433522+0000","last_active":"2024-09-17T02:47:27.767318+0000","last_peered":"2024-09-17T02:47:27.767318+0000","last_clean":"2024-09-17T02:47:27.767318+0000","last_became_active":"2024-09-17T02:47:26.433017+0000","last_became_peered":"2024-09-17T02:47:26.433017+0000","last_unstale":"2024-09-17T02:47:27.767318+0000","last_undegraded":"2024-09-17T02:47:27.767318+0000","last_fullsized":"2024-09-17T02:47:27.767318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:35.865526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421125+0000","last_change":"2024-09-17T02:47:26.419153+0000","last_active":"2024-09-17T02:47:27.421125+0000","last_peered":"2024-09-17T02:47:27.421125+0000","last_clean":"2024-09-17T02:47:27.421125+0000","last_became_active":"2024-09-17T02:47:26.418280+0000","last_became_peered":"2024-09-17T02:47:26.418280+0000","last_unstale":"2024-09-17T02:47:27.421125+0000","last_undegraded":"2024-09-17T02:47:27.421125+0000","last_fullsized":"2024-09-17T02:47:27.421125+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:17.220457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"91'4","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769913+0000","last_change":"2024-09-17T02:47:25.404962+0000","last_active":"2024-09-17T02:47:27.769913+0000","last_peered":"2024-09-17T02:47:27.769913+0000","last_clean":"2024-09-17T02:47:27.769913+0000","last_became_active":"2024-09-17T02:47:25.404574+0000","last_became_peered":"2024-09-17T02:47:25.404574+0000","last_unstale":"2024-09-17T02:47:27.769913+0000","last_undegraded":"2024-09-17T02:47:27.769913+0000","last_fullsized":"2024-09-17T02:47:27.769913+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:27.433639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768770+0000","last_change":"2024-09-17T02:47:26.448672+0000","last_active":"2024-09-17T02:47:27.768770+0000","last_peered":"2024-09-17T02:47:27.768770+0000","last_clean":"2024-09-17T02:47:27.768770+0000","last_became_active":"2024-09-17T02:47:26.447991+0000","last_became_peered":"2024-09-17T02:47:26.447991+0000","last_unstale":"2024-09-17T02:47:27.768770+0000","last_undegraded":"2024-09-17T02:47:27.768770+0000","last_fullsized":"2024-09-17T02:47:27.768770+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:05.713702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746189+0000","last_change":"2024-09-17T02:47:25.394418+0000","last_active":"2024-09-17T02:47:27.746189+0000","last_peered":"2024-09-17T02:47:27.746189+0000","last_clean":"2024-09-17T02:47:27.746189+0000","last_became_active":"2024-09-17T02:47:25.393974+0000","last_became_peered":"2024-09-17T02:47:25.393974+0000","last_unstale":"2024-09-17T02:47:27.746189+0000","last_undegraded":"2024-09-17T02:47:27.746189+0000","last_fullsized":"2024-09-17T02:47:27.746189+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:10.358930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416397+0000","last_change":"2024-09-17T02:47:26.444143+0000","last_active":"2024-09-17T02:47:27.416397+0000","last_peered":"2024-09-17T02:47:27.416397+0000","last_clean":"2024-09-17T02:47:27.416397+0000","last_became_active":"2024-09-17T02:47:26.443779+0000","last_became_peered":"2024-09-17T02:47:26.443779+0000","last_unstale":"2024-09-17T02:47:27.416397+0000","last_undegraded":"2024-09-17T02:47:27.416397+0000","last_fullsized":"2024-09-17T02:47:27.416397+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:28.224146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768881+0000","last_change":"2024-09-17T02:47:25.418775+0000","last_active":"2024-09-17T02:47:27.768881+0000","last_peered":"2024-09-17T02:47:27.768881+0000","last_clean":"2024-09-17T02:47:27.768881+0000","last_became_active":"2024-09-17T02:47:25.418458+0000","last_became_peered":"2024-09-17T02:47:25.418458+0000","last_unstale":"2024-09-17T02:47:27.768881+0000","last_undegraded":"2024-09-17T02:47:27.768881+0000","last_fullsized":"2024-09-17T02:47:27.768881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:16.952908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419487+0000","last_change":"2024-09-17T02:47:26.438283+0000","last_active":"2024-09-17T02:47:27.419487+0000","last_peered":"2024-09-17T02:47:27.419487+0000","last_clean":"2024-09-17T02:47:27.419487+0000","last_became_active":"2024-09-17T02:47:26.437598+0000","last_became_peered":"2024-09-17T02:47:26.437598+0000","last_unstale":"2024-09-17T02:47:27.419487+0000","last_undegraded":"2024-09-17T02:47:27.419487+0000","last_fullsized":"2024-09-17T02:47:27.419487+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:42.817995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.745987+0000","last_change":"2024-09-17T02:47:25.412944+0000","last_active":"2024-09-17T02:47:27.745987+0000","last_peered":"2024-09-17T02:47:27.745987+0000","last_clean":"2024-09-17T02:47:27.745987+0000","last_became_active":"2024-09-17T02:47:25.412515+0000","last_became_peered":"2024-09-17T02:47:25.412515+0000","last_unstale":"2024-09-17T02:47:27.745987+0000","last_undegraded":"2024-09-17T02:47:27.745987+0000","last_fullsized":"2024-09-17T02:47:27.745987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:06.364211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767322+0000","last_change":"2024-09-17T02:47:26.457572+0000","last_active":"2024-09-17T02:47:27.767322+0000","last_peered":"2024-09-17T02:47:27.767322+0000","last_clean":"2024-09-17T02:47:27.767322+0000","last_became_active":"2024-09-17T02:47:26.451939+0000","last_became_peered":"2024-09-17T02:47:26.451939+0000","last_unstale":"2024-09-17T02:47:27.767322+0000","last_undegraded":"2024-09-17T02:47:27.767322+0000","last_fullsized":"2024-09-17T02:47:27.767322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:57.435169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767430+0000","last_change":"2024-09-17T02:47:25.411938+0000","last_active":"2024-09-17T02:47:27.767430+0000","last_peered":"2024-09-17T02:47:27.767430+0000","last_clean":"2024-09-17T02:47:27.767430+0000","last_became_active":"2024-09-17T02:47:25.411486+0000","last_became_peered":"2024-09-17T02:47:25.411486+0000","last_unstale":"2024-09-17T02:47:27.767430+0000","last_undegraded":"2024-09-17T02:47:27.767430+0000","last_fullsized":"2024-09-17T02:47:27.767430+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:08.479886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416053+0000","last_change":"2024-09-17T02:47:26.424905+0000","last_active":"2024-09-17T02:47:27.416053+0000","last_peered":"2024-09-17T02:47:27.416053+0000","last_clean":"2024-09-17T02:47:27.416053+0000","last_became_active":"2024-09-17T02:47:26.424551+0000","last_became_peered":"2024-09-17T02:47:26.424551+0000","last_unstale":"2024-09-17T02:47:27.416053+0000","last_undegraded":"2024-09-17T02:47:27.416053+0000","last_fullsized":"2024-09-17T02:47:27.416053+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:03.042595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768416+0000","last_change":"2024-09-17T02:47:25.422380+0000","last_active":"2024-09-17T02:47:27.768416+0000","last_peered":"2024-09-17T02:47:27.768416+0000","last_clean":"2024-09-17T02:47:27.768416+0000","last_became_active":"2024-09-17T02:47:25.421906+0000","last_became_peered":"2024-09-17T02:47:25.421906+0000","last_unstale":"2024-09-17T02:47:27.768416+0000","last_undegraded":"2024-09-17T02:47:27.768416+0000","last_fullsized":"2024-09-17T02:47:27.768416+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:04.941668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767297+0000","last_change":"2024-09-17T02:47:26.435576+0000","last_active":"2024-09-17T02:47:27.767297+0000","last_peered":"2024-09-17T02:47:27.767297+0000","last_clean":"2024-09-17T02:47:27.767297+0000","last_became_active":"2024-09-17T02:47:26.434782+0000","last_became_peered":"2024-09-17T02:47:26.434782+0000","last_unstale":"2024-09-17T02:47:27.767297+0000","last_undegraded":"2024-09-17T02:47:27.767297+0000","last_fullsized":"2024-09-17T02:47:27.767297+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:12.674675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422144+0000","last_change":"2024-09-17T02:47:25.410268+0000","last_active":"2024-09-17T02:47:27.422144+0000","last_peered":"2024-09-17T02:47:27.422144+0000","last_clean":"2024-09-17T02:47:27.422144+0000","last_became_active":"2024-09-17T02:47:25.409433+0000","last_became_peered":"2024-09-17T02:47:25.409433+0000","last_unstale":"2024-09-17T02:47:27.422144+0000","last_undegraded":"2024-09-17T02:47:27.422144+0000","last_fullsized":"2024-09-17T02:47:27.422144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:46.106338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.420382+0000","last_change":"2024-09-17T02:47:26.410976+0000","last_active":"2024-09-17T02:47:27.420382+0000","last_peered":"2024-09-17T02:47:27.420382+0000","last_clean":"2024-09-17T02:47:27.420382+0000","last_became_active":"2024-09-17T02:47:26.407305+0000","last_became_peered":"2024-09-17T02:47:26.407305+0000","last_unstale":"2024-09-17T02:47:27.420382+0000","last_undegraded":"2024-09-17T02:47:27.420382+0000","last_fullsized":"2024-09-17T02:47:27.420382+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:53.755378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766722+0000","last_change":"2024-09-17T02:47:25.420193+0000","last_active":"2024-09-17T02:47:27.766722+0000","last_peered":"2024-09-17T02:47:27.766722+0000","last_clean":"2024-09-17T02:47:27.766722+0000","last_became_active":"2024-09-17T02:47:25.419700+0000","last_became_peered":"2024-09-17T02:47:25.419700+0000","last_unstale":"2024-09-17T02:47:27.766722+0000","last_undegraded":"2024-09-17T02:47:27.766722+0000","last_fullsized":"2024-09-17T02:47:27.766722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:41.533605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768164+0000","last_change":"2024-09-17T02:47:26.453517+0000","last_active":"2024-09-17T02:47:27.768164+0000","last_peered":"2024-09-17T02:47:27.768164+0000","last_clean":"2024-09-17T02:47:27.768164+0000","last_became_active":"2024-09-17T02:47:26.453175+0000","last_became_peered":"2024-09-17T02:47:26.453175+0000","last_unstale":"2024-09-17T02:47:27.768164+0000","last_undegraded":"2024-09-17T02:47:27.768164+0000","last_fullsized":"2024-09-17T02:47:27.768164+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:16.597414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769913+0000","last_change":"2024-09-17T02:47:25.419435+0000","last_active":"2024-09-17T02:47:27.769913+0000","last_peered":"2024-09-17T02:47:27.769913+0000","last_clean":"2024-09-17T02:47:27.769913+0000","last_became_active":"2024-09-17T02:47:25.418955+0000","last_became_peered":"2024-09-17T02:47:25.418955+0000","last_unstale":"2024-09-17T02:47:27.769913+0000","last_undegraded":"2024-09-17T02:47:27.769913+0000","last_fullsized":"2024-09-17T02:47:27.769913+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:03.632893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416447+0000","last_change":"2024-09-17T02:47:26.421172+0000","last_active":"2024-09-17T02:47:27.416447+0000","last_peered":"2024-09-17T02:47:27.416447+0000","last_clean":"2024-09-17T02:47:27.416447+0000","last_became_active":"2024-09-17T02:47:26.420916+0000","last_became_peered":"2024-09-17T02:47:26.420916+0000","last_unstale":"2024-09-17T02:47:27.416447+0000","last_undegraded":"2024-09-17T02:47:27.416447+0000","last_fullsized":"2024-09-17T02:47:27.416447+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:29.293488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.420010+0000","last_change":"2024-09-17T02:47:25.414624+0000","last_active":"2024-09-17T02:47:27.420010+0000","last_peered":"2024-09-17T02:47:27.420010+0000","last_clean":"2024-09-17T02:47:27.420010+0000","last_became_active":"2024-09-17T02:47:25.413560+0000","last_became_peered":"2024-09-17T02:47:25.413560+0000","last_unstale":"2024-09-17T02:47:27.420010+0000","last_undegraded":"2024-09-17T02:47:27.420010+0000","last_fullsized":"2024-09-17T02:47:27.420010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:00.198104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422254+0000","last_change":"2024-09-17T02:47:26.441675+0000","last_active":"2024-09-17T02:47:27.422254+0000","last_peered":"2024-09-17T02:47:27.422254+0000","last_clean":"2024-09-17T02:47:27.422254+0000","last_became_active":"2024-09-17T02:47:26.437413+0000","last_became_peered":"2024-09-17T02:47:26.437413+0000","last_unstale":"2024-09-17T02:47:27.422254+0000","last_undegraded":"2024-09-17T02:47:27.422254+0000","last_fullsized":"2024-09-17T02:47:27.422254+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:50.459729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"91'3","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416794+0000","last_change":"2024-09-17T02:47:25.419022+0000","last_active":"2024-09-17T02:47:27.416794+0000","last_peered":"2024-09-17T02:47:27.416794+0000","last_clean":"2024-09-17T02:47:27.416794+0000","last_became_active":"2024-09-17T02:47:25.418540+0000","last_became_peered":"2024-09-17T02:47:25.418540+0000","last_unstale":"2024-09-17T02:47:27.416794+0000","last_undegraded":"2024-09-17T02:47:27.416794+0000","last_fullsized":"2024-09-17T02:47:27.416794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:04.700956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769873+0000","last_change":"2024-09-17T02:47:26.429054+0000","last_active":"2024-09-17T02:47:27.769873+0000","last_peered":"2024-09-17T02:47:27.769873+0000","last_clean":"2024-09-17T02:47:27.769873+0000","last_became_active":"2024-09-17T02:47:26.428270+0000","last_became_peered":"2024-09-17T02:47:26.428270+0000","last_unstale":"2024-09-17T02:47:27.769873+0000","last_undegraded":"2024-09-17T02:47:27.769873+0000","last_fullsized":"2024-09-17T02:47:27.769873+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:16.361134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"91'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421338+0000","last_change":"2024-09-17T02:47:25.418647+0000","last_active":"2024-09-17T02:47:27.421338+0000","last_peered":"2024-09-17T02:47:27.421338+0000","last_clean":"2024-09-17T02:47:27.421338+0000","last_became_active":"2024-09-17T02:47:25.416994+0000","last_became_peered":"2024-09-17T02:47:25.416994+0000","last_unstale":"2024-09-17T02:47:27.421338+0000","last_undegraded":"2024-09-17T02:47:27.421338+0000","last_fullsized":"2024-09-17T02:47:27.421338+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:58.768722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.418667+0000","last_change":"2024-09-17T02:47:26.419193+0000","last_active":"2024-09-17T02:47:27.418667+0000","last_peered":"2024-09-17T02:47:27.418667+0000","last_clean":"2024-09-17T02:47:27.418667+0000","last_became_active":"2024-09-17T02:47:26.418643+0000","last_became_peered":"2024-09-17T02:47:26.418643+0000","last_unstale":"2024-09-17T02:47:27.418667+0000","last_undegraded":"2024-09-17T02:47:27.418667+0000","last_fullsized":"2024-09-17T02:47:27.418667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:46.698904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.770213+0000","last_change":"2024-09-17T02:47:25.418902+0000","last_active":"2024-09-17T02:47:27.770213+0000","last_peered":"2024-09-17T02:47:27.770213+0000","last_clean":"2024-09-17T02:47:27.770213+0000","last_became_active":"2024-09-17T02:47:25.418568+0000","last_became_peered":"2024-09-17T02:47:25.418568+0000","last_unstale":"2024-09-17T02:47:27.770213+0000","last_undegraded":"2024-09-17T02:47:27.770213+0000","last_fullsized":"2024-09-17T02:47:27.770213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:22.925782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767126+0000","last_change":"2024-09-17T02:47:26.414277+0000","last_active":"2024-09-17T02:47:27.767126+0000","last_peered":"2024-09-17T02:47:27.767126+0000","last_clean":"2024-09-17T02:47:27.767126+0000","last_became_active":"2024-09-17T02:47:26.413010+0000","last_became_peered":"2024-09-17T02:47:26.413010+0000","last_unstale":"2024-09-17T02:47:27.767126+0000","last_undegraded":"2024-09-17T02:47:27.767126+0000","last_fullsized":"2024-09-17T02:47:27.767126+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:00.083852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419403+0000","last_change":"2024-09-17T02:47:25.418948+0000","last_active":"2024-09-17T02:47:27.419403+0000","last_peered":"2024-09-17T02:47:27.419403+0000","last_clean":"2024-09-17T02:47:27.419403+0000","last_became_active":"2024-09-17T02:47:25.417436+0000","last_became_peered":"2024-09-17T02:47:25.417436+0000","last_unstale":"2024-09-17T02:47:27.419403+0000","last_undegraded":"2024-09-17T02:47:27.419403+0000","last_fullsized":"2024-09-17T02:47:27.419403+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:25.386796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419176+0000","last_change":"2024-09-17T02:47:26.438873+0000","last_active":"2024-09-17T02:47:27.419176+0000","last_peered":"2024-09-17T02:47:27.419176+0000","last_clean":"2024-09-17T02:47:27.419176+0000","last_became_active":"2024-09-17T02:47:26.436291+0000","last_became_peered":"2024-09-17T02:47:26.436291+0000","last_unstale":"2024-09-17T02:47:27.419176+0000","last_undegraded":"2024-09-17T02:47:27.419176+0000","last_fullsized":"2024-09-17T02:47:27.419176+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:56.037102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.423108+0000","last_change":"2024-09-17T02:47:25.411837+0000","last_active":"2024-09-17T02:47:27.423108+0000","last_peered":"2024-09-17T02:47:27.423108+0000","last_clean":"2024-09-17T02:47:27.423108+0000","last_became_active":"2024-09-17T02:47:25.411299+0000","last_became_peered":"2024-09-17T02:47:25.411299+0000","last_unstale":"2024-09-17T02:47:27.423108+0000","last_undegraded":"2024-09-17T02:47:27.423108+0000","last_fullsized":"2024-09-17T02:47:27.423108+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:23.951343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769489+0000","last_change":"2024-09-17T02:47:26.449804+0000","last_active":"2024-09-17T02:47:27.769489+0000","last_peered":"2024-09-17T02:47:27.769489+0000","last_clean":"2024-09-17T02:47:27.769489+0000","last_became_active":"2024-09-17T02:47:26.445395+0000","last_became_peered":"2024-09-17T02:47:26.445395+0000","last_unstale":"2024-09-17T02:47:27.769489+0000","last_undegraded":"2024-09-17T02:47:27.769489+0000","last_fullsized":"2024-09-17T02:47:27.769489+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:39.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"91'8","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419774+0000","last_change":"2024-09-17T02:47:25.421873+0000","last_active":"2024-09-17T02:47:27.419774+0000","last_peered":"2024-09-17T02:47:27.419774+0000","last_clean":"2024-09-17T02:47:27.419774+0000","last_became_active":"2024-09-17T02:47:25.421196+0000","last_became_peered":"2024-09-17T02:47:25.421196+0000","last_unstale":"2024-09-17T02:47:27.419774+0000","last_undegraded":"2024-09-17T02:47:27.419774+0000","last_fullsized":"2024-09-17T02:47:27.419774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:54.515348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769031+0000","last_change":"2024-09-17T02:47:26.458333+0000","last_active":"2024-09-17T02:47:27.769031+0000","last_peered":"2024-09-17T02:47:27.769031+0000","last_clean":"2024-09-17T02:47:27.769031+0000","last_became_active":"2024-09-17T02:47:26.450921+0000","last_became_peered":"2024-09-17T02:47:26.450921+0000","last_unstale":"2024-09-17T02:47:27.769031+0000","last_undegraded":"2024-09-17T02:47:27.769031+0000","last_fullsized":"2024-09-17T02:47:27.769031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:36.379174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767855+0000","last_change":"2024-09-17T02:47:25.401532+0000","last_active":"2024-09-17T02:47:27.767855+0000","last_peered":"2024-09-17T02:47:27.767855+0000","last_clean":"2024-09-17T02:47:27.767855+0000","last_became_active":"2024-09-17T02:47:25.400932+0000","last_became_peered":"2024-09-17T02:47:25.400932+0000","last_unstale":"2024-09-17T02:47:27.767855+0000","last_undegraded":"2024-09-17T02:47:27.767855+0000","last_fullsized":"2024-09-17T02:47:27.767855+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:30.981067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416578+0000","last_change":"2024-09-17T02:47:26.407118+0000","last_active":"2024-09-17T02:47:27.416578+0000","last_peered":"2024-09-17T02:47:27.416578+0000","last_clean":"2024-09-17T02:47:27.416578+0000","last_became_active":"2024-09-17T02:47:26.406456+0000","last_became_peered":"2024-09-17T02:47:26.406456+0000","last_unstale":"2024-09-17T02:47:27.416578+0000","last_undegraded":"2024-09-17T02:47:27.416578+0000","last_fullsized":"2024-09-17T02:47:27.416578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:39.049060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746340+0000","last_change":"2024-09-17T02:47:25.397773+0000","last_active":"2024-09-17T02:47:27.746340+0000","last_peered":"2024-09-17T02:47:27.746340+0000","last_clean":"2024-09-17T02:47:27.746340+0000","last_became_active":"2024-09-17T02:47:25.397452+0000","last_became_peered":"2024-09-17T02:47:25.397452+0000","last_unstale":"2024-09-17T02:47:27.746340+0000","last_undegraded":"2024-09-17T02:47:27.746340+0000","last_fullsized":"2024-09-17T02:47:27.746340+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:34.937753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769006+0000","last_change":"2024-09-17T02:47:26.443993+0000","last_active":"2024-09-17T02:47:27.769006+0000","last_peered":"2024-09-17T02:47:27.769006+0000","last_clean":"2024-09-17T02:47:27.769006+0000","last_became_active":"2024-09-17T02:47:26.443651+0000","last_became_peered":"2024-09-17T02:47:26.443651+0000","last_unstale":"2024-09-17T02:47:27.769006+0000","last_undegraded":"2024-09-17T02:47:27.769006+0000","last_fullsized":"2024-09-17T02:47:27.769006+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:43.278442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746255+0000","last_change":"2024-09-17T02:47:25.396451+0000","last_active":"2024-09-17T02:47:27.746255+0000","last_peered":"2024-09-17T02:47:27.746255+0000","last_clean":"2024-09-17T02:47:27.746255+0000","last_became_active":"2024-09-17T02:47:25.396146+0000","last_became_peered":"2024-09-17T02:47:25.396146+0000","last_unstale":"2024-09-17T02:47:27.746255+0000","last_undegraded":"2024-09-17T02:47:27.746255+0000","last_fullsized":"2024-09-17T02:47:27.746255+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:28:55.866634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767876+0000","last_change":"2024-09-17T02:47:26.444819+0000","last_active":"2024-09-17T02:47:27.767876+0000","last_peered":"2024-09-17T02:47:27.767876+0000","last_clean":"2024-09-17T02:47:27.767876+0000","last_became_active":"2024-09-17T02:47:26.443794+0000","last_became_peered":"2024-09-17T02:47:26.443794+0000","last_unstale":"2024-09-17T02:47:27.767876+0000","last_undegraded":"2024-09-17T02:47:27.767876+0000","last_fullsized":"2024-09-17T02:47:27.767876+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:26.261568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416035+0000","last_change":"2024-09-17T02:47:25.407194+0000","last_active":"2024-09-17T02:47:27.416035+0000","last_peered":"2024-09-17T02:47:27.416035+0000","last_clean":"2024-09-17T02:47:27.416035+0000","last_became_active":"2024-09-17T02:47:25.406881+0000","last_became_peered":"2024-09-17T02:47:25.406881+0000","last_unstale":"2024-09-17T02:47:27.416035+0000","last_undegraded":"2024-09-17T02:47:27.416035+0000","last_fullsized":"2024-09-17T02:47:27.416035+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:20.997735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.745904+0000","last_change":"2024-09-17T02:47:26.424417+0000","last_active":"2024-09-17T02:47:27.745904+0000","last_peered":"2024-09-17T02:47:27.745904+0000","last_clean":"2024-09-17T02:47:27.745904+0000","last_became_active":"2024-09-17T02:47:26.424080+0000","last_became_peered":"2024-09-17T02:47:26.424080+0000","last_unstale":"2024-09-17T02:47:27.745904+0000","last_undegraded":"2024-09-17T02:47:27.745904+0000","last_fullsized":"2024-09-17T02:47:27.745904+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:00.083852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767755+0000","last_change":"2024-09-17T02:47:25.419788+0000","last_active":"2024-09-17T02:47:27.767755+0000","last_peered":"2024-09-17T02:47:27.767755+0000","last_clean":"2024-09-17T02:47:27.767755+0000","last_became_active":"2024-09-17T02:47:25.419102+0000","last_became_peered":"2024-09-17T02:47:25.419102+0000","last_unstale":"2024-09-17T02:47:27.767755+0000","last_undegraded":"2024-09-17T02:47:27.767755+0000","last_fullsized":"2024-09-17T02:47:27.767755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:15.584669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767923+0000","last_change":"2024-09-17T02:47:26.401986+0000","last_active":"2024-09-17T02:47:27.767923+0000","last_peered":"2024-09-17T02:47:27.767923+0000","last_clean":"2024-09-17T02:47:27.767923+0000","last_became_active":"2024-09-17T02:47:26.400697+0000","last_became_peered":"2024-09-17T02:47:26.400697+0000","last_unstale":"2024-09-17T02:47:27.767923+0000","last_undegraded":"2024-09-17T02:47:27.767923+0000","last_fullsized":"2024-09-17T02:47:27.767923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:19.106029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.745766+0000","last_change":"2024-09-17T02:47:25.421863+0000","last_active":"2024-09-17T02:47:27.745766+0000","last_peered":"2024-09-17T02:47:27.745766+0000","last_clean":"2024-09-17T02:47:27.745766+0000","last_became_active":"2024-09-17T02:47:25.421355+0000","last_became_peered":"2024-09-17T02:47:25.421355+0000","last_unstale":"2024-09-17T02:47:27.745766+0000","last_undegraded":"2024-09-17T02:47:27.745766+0000","last_fullsized":"2024-09-17T02:47:27.745766+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:20.461693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421042+0000","last_change":"2024-09-17T02:47:26.417573+0000","last_active":"2024-09-17T02:47:27.421042+0000","last_peered":"2024-09-17T02:47:27.421042+0000","last_clean":"2024-09-17T02:47:27.421042+0000","last_became_active":"2024-09-17T02:47:26.414446+0000","last_became_peered":"2024-09-17T02:47:26.414446+0000","last_unstale":"2024-09-17T02:47:27.421042+0000","last_undegraded":"2024-09-17T02:47:27.421042+0000","last_fullsized":"2024-09-17T02:47:27.421042+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:50.565801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767807+0000","last_change":"2024-09-17T02:47:25.413072+0000","last_active":"2024-09-17T02:47:27.767807+0000","last_peered":"2024-09-17T02:47:27.767807+0000","last_clean":"2024-09-17T02:47:27.767807+0000","last_became_active":"2024-09-17T02:47:25.412033+0000","last_became_peered":"2024-09-17T02:47:25.412033+0000","last_unstale":"2024-09-17T02:47:27.767807+0000","last_undegraded":"2024-09-17T02:47:27.767807+0000","last_fullsized":"2024-09-17T02:47:27.767807+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:25.100410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768080+0000","last_change":"2024-09-17T02:47:26.420721+0000","last_active":"2024-09-17T02:47:27.768080+0000","last_peered":"2024-09-17T02:47:27.768080+0000","last_clean":"2024-09-17T02:47:27.768080+0000","last_became_active":"2024-09-17T02:47:26.420354+0000","last_became_peered":"2024-09-17T02:47:26.420354+0000","last_unstale":"2024-09-17T02:47:27.768080+0000","last_undegraded":"2024-09-17T02:47:27.768080+0000","last_fullsized":"2024-09-17T02:47:27.768080+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:30:05.141305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416520+0000","last_change":"2024-09-17T02:47:25.390686+0000","last_active":"2024-09-17T02:47:27.416520+0000","last_peered":"2024-09-17T02:47:27.416520+0000","last_clean":"2024-09-17T02:47:27.416520+0000","last_became_active":"2024-09-17T02:47:25.390236+0000","last_became_peered":"2024-09-17T02:47:25.390236+0000","last_unstale":"2024-09-17T02:47:27.416520+0000","last_undegraded":"2024-09-17T02:47:27.416520+0000","last_fullsized":"2024-09-17T02:47:27.416520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:55.492515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768596+0000","last_change":"2024-09-17T02:47:26.450126+0000","last_active":"2024-09-17T02:47:27.768596+0000","last_peered":"2024-09-17T02:47:27.768596+0000","last_clean":"2024-09-17T02:47:27.768596+0000","last_became_active":"2024-09-17T02:47:26.445482+0000","last_became_peered":"2024-09-17T02:47:26.445482+0000","last_unstale":"2024-09-17T02:47:27.768596+0000","last_undegraded":"2024-09-17T02:47:27.768596+0000","last_fullsized":"2024-09-17T02:47:27.768596+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:45.742572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416576+0000","last_change":"2024-09-17T02:47:25.396097+0000","last_active":"2024-09-17T02:47:27.416576+0000","last_peered":"2024-09-17T02:47:27.416576+0000","last_clean":"2024-09-17T02:47:27.416576+0000","last_became_active":"2024-09-17T02:47:25.395670+0000","last_became_peered":"2024-09-17T02:47:25.395670+0000","last_unstale":"2024-09-17T02:47:27.416576+0000","last_undegraded":"2024-09-17T02:47:27.416576+0000","last_fullsized":"2024-09-17T02:47:27.416576+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:15.348388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769902+0000","last_change":"2024-09-17T02:47:26.449859+0000","last_active":"2024-09-17T02:47:27.769902+0000","last_peered":"2024-09-17T02:47:27.769902+0000","last_clean":"2024-09-17T02:47:27.769902+0000","last_became_active":"2024-09-17T02:47:26.449290+0000","last_became_peered":"2024-09-17T02:47:26.449290+0000","last_unstale":"2024-09-17T02:47:27.769902+0000","last_undegraded":"2024-09-17T02:47:27.769902+0000","last_fullsized":"2024-09-17T02:47:27.769902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:29.692087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769004+0000","last_change":"2024-09-17T02:47:25.414356+0000","last_active":"2024-09-17T02:47:27.769004+0000","last_peered":"2024-09-17T02:47:27.769004+0000","last_clean":"2024-09-17T02:47:27.769004+0000","last_became_active":"2024-09-17T02:47:25.413979+0000","last_became_peered":"2024-09-17T02:47:25.413979+0000","last_unstale":"2024-09-17T02:47:27.769004+0000","last_undegraded":"2024-09-17T02:47:27.769004+0000","last_fullsized":"2024-09-17T02:47:27.769004+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:59.127265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769424+0000","last_change":"2024-09-17T02:47:26.425824+0000","last_active":"2024-09-17T02:47:27.769424+0000","last_peered":"2024-09-17T02:47:27.769424+0000","last_clean":"2024-09-17T02:47:27.769424+0000","last_became_active":"2024-09-17T02:47:26.414310+0000","last_became_peered":"2024-09-17T02:47:26.414310+0000","last_unstale":"2024-09-17T02:47:27.769424+0000","last_undegraded":"2024-09-17T02:47:27.769424+0000","last_fullsized":"2024-09-17T02:47:27.769424+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:03.730055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416631+0000","last_change":"2024-09-17T02:47:25.392813+0000","last_active":"2024-09-17T02:47:27.416631+0000","last_peered":"2024-09-17T02:47:27.416631+0000","last_clean":"2024-09-17T02:47:27.416631+0000","last_became_active":"2024-09-17T02:47:25.392472+0000","last_became_peered":"2024-09-17T02:47:25.392472+0000","last_unstale":"2024-09-17T02:47:27.416631+0000","last_undegraded":"2024-09-17T02:47:27.416631+0000","last_fullsized":"2024-09-17T02:47:27.416631+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:05.045538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767992+0000","last_change":"2024-09-17T02:47:26.444405+0000","last_active":"2024-09-17T02:47:27.767992+0000","last_peered":"2024-09-17T02:47:27.767992+0000","last_clean":"2024-09-17T02:47:27.767992+0000","last_became_active":"2024-09-17T02:47:26.443000+0000","last_became_peered":"2024-09-17T02:47:26.443000+0000","last_unstale":"2024-09-17T02:47:27.767992+0000","last_undegraded":"2024-09-17T02:47:27.767992+0000","last_fullsized":"2024-09-17T02:47:27.767992+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:27.766149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421520+0000","last_change":"2024-09-17T02:47:25.404863+0000","last_active":"2024-09-17T02:47:27.421520+0000","last_peered":"2024-09-17T02:47:27.421520+0000","last_clean":"2024-09-17T02:47:27.421520+0000","last_became_active":"2024-09-17T02:47:25.403960+0000","last_became_peered":"2024-09-17T02:47:25.403960+0000","last_unstale":"2024-09-17T02:47:27.421520+0000","last_undegraded":"2024-09-17T02:47:27.421520+0000","last_fullsized":"2024-09-17T02:47:27.421520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:38.479422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421658+0000","last_change":"2024-09-17T02:47:26.402827+0000","last_active":"2024-09-17T02:47:27.421658+0000","last_peered":"2024-09-17T02:47:27.421658+0000","last_clean":"2024-09-17T02:47:27.421658+0000","last_became_active":"2024-09-17T02:47:26.402001+0000","last_became_peered":"2024-09-17T02:47:26.402001+0000","last_unstale":"2024-09-17T02:47:27.421658+0000","last_undegraded":"2024-09-17T02:47:27.421658+0000","last_fullsized":"2024-09-17T02:47:27.421658+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:09.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769530+0000","last_change":"2024-09-17T02:47:25.403962+0000","last_active":"2024-09-17T02:47:27.769530+0000","last_peered":"2024-09-17T02:47:27.769530+0000","last_clean":"2024-09-17T02:47:27.769530+0000","last_became_active":"2024-09-17T02:47:25.403605+0000","last_became_peered":"2024-09-17T02:47:25.403605+0000","last_unstale":"2024-09-17T02:47:27.769530+0000","last_undegraded":"2024-09-17T02:47:27.769530+0000","last_fullsized":"2024-09-17T02:47:27.769530+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:31.843272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768230+0000","last_change":"2024-09-17T02:47:26.432557+0000","last_active":"2024-09-17T02:47:27.768230+0000","last_peered":"2024-09-17T02:47:27.768230+0000","last_clean":"2024-09-17T02:47:27.768230+0000","last_became_active":"2024-09-17T02:47:26.432209+0000","last_became_peered":"2024-09-17T02:47:26.432209+0000","last_unstale":"2024-09-17T02:47:27.768230+0000","last_undegraded":"2024-09-17T02:47:27.768230+0000","last_fullsized":"2024-09-17T02:47:27.768230+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:40.137137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767271+0000","last_change":"2024-09-17T02:47:25.410837+0000","last_active":"2024-09-17T02:47:27.767271+0000","last_peered":"2024-09-17T02:47:27.767271+0000","last_clean":"2024-09-17T02:47:27.767271+0000","last_became_active":"2024-09-17T02:47:25.410644+0000","last_became_peered":"2024-09-17T02:47:25.410644+0000","last_unstale":"2024-09-17T02:47:27.767271+0000","last_undegraded":"2024-09-17T02:47:27.767271+0000","last_fullsized":"2024-09-17T02:47:27.767271+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:41.805249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767390+0000","last_change":"2024-09-17T02:47:26.448485+0000","last_active":"2024-09-17T02:47:27.767390+0000","last_peered":"2024-09-17T02:47:27.767390+0000","last_clean":"2024-09-17T02:47:27.767390+0000","last_became_active":"2024-09-17T02:47:26.447870+0000","last_became_peered":"2024-09-17T02:47:26.447870+0000","last_unstale":"2024-09-17T02:47:27.767390+0000","last_undegraded":"2024-09-17T02:47:27.767390+0000","last_fullsized":"2024-09-17T02:47:27.767390+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:20.845608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766663+0000","last_change":"2024-09-17T02:47:25.396522+0000","last_active":"2024-09-17T02:47:27.766663+0000","last_peered":"2024-09-17T02:47:27.766663+0000","last_clean":"2024-09-17T02:47:27.766663+0000","last_became_active":"2024-09-17T02:47:25.395729+0000","last_became_peered":"2024-09-17T02:47:25.395729+0000","last_unstale":"2024-09-17T02:47:27.766663+0000","last_undegraded":"2024-09-17T02:47:27.766663+0000","last_fullsized":"2024-09-17T02:47:27.766663+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:58.843240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769403+0000","last_change":"2024-09-17T02:47:26.451744+0000","last_active":"2024-09-17T02:47:27.769403+0000","last_peered":"2024-09-17T02:47:27.769403+0000","last_clean":"2024-09-17T02:47:27.769403+0000","last_became_active":"2024-09-17T02:47:26.451445+0000","last_became_peered":"2024-09-17T02:47:26.451445+0000","last_unstale":"2024-09-17T02:47:27.769403+0000","last_undegraded":"2024-09-17T02:47:27.769403+0000","last_fullsized":"2024-09-17T02:47:27.769403+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:54.458584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421501+0000","last_change":"2024-09-17T02:47:25.404744+0000","last_active":"2024-09-17T02:47:27.421501+0000","last_peered":"2024-09-17T02:47:27.421501+0000","last_clean":"2024-09-17T02:47:27.421501+0000","last_became_active":"2024-09-17T02:47:25.403842+0000","last_became_peered":"2024-09-17T02:47:25.403842+0000","last_unstale":"2024-09-17T02:47:27.421501+0000","last_undegraded":"2024-09-17T02:47:27.421501+0000","last_fullsized":"2024-09-17T02:47:27.421501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:02.237396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":246,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767020+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:47:27.767020+0000","last_peered":"2024-09-17T02:47:27.767020+0000","last_clean":"2024-09-17T02:47:27.767020+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:47:27.767020+0000","last_undegraded":"2024-09-17T02:47:27.767020+0000","last_fullsized":"2024-09-17T02:47:27.767020+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509526,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":517803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1720320,"data_stored":1108171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":517803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509525,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1146880,"data_stored":519527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509526,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1134592,"data_stored":518407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509526,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1732608,"data_stored":1108857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509527,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1130496,"data_stored":517873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574934,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1146880,"data_stored":519539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:47:35.105 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:47:35.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.277+0000 7f5586775640 1 -- 172.21.15.131:0/2356560304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55800cf480 msgr2=0x7f5580169700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:35.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.277+0000 7f5586775640 1 --2- 172.21.15.131:0/2356560304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55800cf480 0x7f5580169700 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5570009840 tx=0x7f557001c790 comp rx=0 tx=0).stop 2024-09-17T02:47:35.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.277+0000 7f5586775640 1 -- 172.21.15.131:0/2356560304 shutdown_connections 2024-09-17T02:47:35.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.277+0000 7f5586775640 1 --2- 172.21.15.131:0/2356560304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55800cf480 0x7f5580169700 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.277+0000 7f5586775640 1 -- 172.21.15.131:0/2356560304 >> 172.21.15.131:0/2356560304 conn(0x7f558006eea0 msgr2=0x7f558006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:35.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.277+0000 7f5586775640 1 -- 172.21.15.131:0/2356560304 shutdown_connections 2024-09-17T02:47:35.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.277+0000 7f5586775640 1 -- 172.21.15.131:0/2356560304 wait complete. 2024-09-17T02:47:35.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.277+0000 7f5586775640 1 Processor -- start 2024-09-17T02:47:35.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.277+0000 7f5586775640 1 -- start start 2024-09-17T02:47:35.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f5586775640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558006dea0 0x7f5580152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:35.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f5586775640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55800cac80 0x7f55801545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:35.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f5586775640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f55800cf480 0x7f55801498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:35.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f5586775640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f558016cf40 con 0x7f55800cac80 2024-09-17T02:47:35.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f5586775640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f558016cdc0 con 0x7f558006dea0 2024-09-17T02:47:35.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f5586775640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f558016d0c0 con 0x7f55800cf480 2024-09-17T02:47:35.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55800cac80 0x7f55801545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:35.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55800cac80 0x7f55801545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43482/0 (socket says 172.21.15.131:43482) 2024-09-17T02:47:35.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557f7fe640 1 -- 172.21.15.131:0/1093347240 learned_addr learned my addr 172.21.15.131:0/1093347240 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557ffff640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558006dea0 0x7f5580152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f5584d12640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f55800cf480 0x7f55801498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557f7fe640 1 -- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f55800cf480 msgr2=0x7f55801498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557f7fe640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f55800cf480 0x7f55801498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557f7fe640 1 -- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558006dea0 msgr2=0x7f5580152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557f7fe640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558006dea0 0x7f5580152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557f7fe640 1 -- 172.21.15.131:0/1093347240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5580152e90 con 0x7f55800cac80 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557ffff640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558006dea0 0x7f5580152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557f7fe640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55800cac80 0x7f55801545b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f556800c390 tx=0x7f556800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557cff9640 1 -- 172.21.15.131:0/1093347240 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568011790 con 0x7f55800cac80 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557cff9640 1 -- 172.21.15.131:0/1093347240 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5568017070 con 0x7f55800cac80 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f5586775640 1 -- 172.21.15.131:0/1093347240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5580148830 con 0x7f55800cac80 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f557cff9640 1 -- 172.21.15.131:0/1093347240 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568011d00 con 0x7f55800cac80 2024-09-17T02:47:35.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f5586775640 1 -- 172.21.15.131:0/1093347240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5580148df0 con 0x7f55800cac80 2024-09-17T02:47:35.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.281+0000 7f5586775640 1 -- 172.21.15.131:0/1093347240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55800cb090 con 0x7f55800cac80 2024-09-17T02:47:35.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.285+0000 7f557cff9640 1 -- 172.21.15.131:0/1093347240 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f556801c460 con 0x7f55800cac80 2024-09-17T02:47:35.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.285+0000 7f557cff9640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5560073200 0x7f5560075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:35.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.285+0000 7f557cff9640 1 -- 172.21.15.131:0/1093347240 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6283+0+0 (secure 0 0 0) 0x7f5568094230 con 0x7f55800cac80 2024-09-17T02:47:35.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.285+0000 7f557ffff640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5560073200 0x7f5560075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:35.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.285+0000 7f557ffff640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5560073200 0x7f5560075710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f55780012f0 tx=0x7f5578009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:35.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.289+0000 7f557cff9640 1 -- 172.21.15.131:0/1093347240 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f556805f010 con 0x7f55800cac80 2024-09-17T02:47:35.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.461+0000 7f5586775640 1 -- 172.21.15.131:0/1093347240 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5580070c70 con 0x7f5560073200 2024-09-17T02:47:35.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.477+0000 7f557cff9640 1 -- 172.21.15.131:0/1093347240 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252059 (secure 0 0 0) 0x7f5580070c70 con 0x7f5560073200 2024-09-17T02:47:35.479 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:35.480 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:47:35.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.481+0000 7f5586775640 1 -- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5560073200 msgr2=0x7f5560075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.481+0000 7f5586775640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5560073200 0x7f5560075710 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f55780012f0 tx=0x7f5578009040 comp rx=0 tx=0).stop 2024-09-17T02:47:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.481+0000 7f5586775640 1 -- 172.21.15.131:0/1093347240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55800cac80 msgr2=0x7f55801545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.481+0000 7f5586775640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55800cac80 0x7f55801545b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f556800c390 tx=0x7f556800c850 comp rx=0 tx=0).stop 2024-09-17T02:47:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.485+0000 7f5586775640 1 -- 172.21.15.131:0/1093347240 shutdown_connections 2024-09-17T02:47:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.485+0000 7f5586775640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5560073200 0x7f5560075710 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.485+0000 7f5586775640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f55800cf480 0x7f55801498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.485+0000 7f5586775640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f55800cac80 0x7f55801545b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.485+0000 7f5586775640 1 --2- 172.21.15.131:0/1093347240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f558006dea0 0x7f5580152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.485+0000 7f5586775640 1 -- 172.21.15.131:0/1093347240 >> 172.21.15.131:0/1093347240 conn(0x7f558006eea0 msgr2=0x7f5580073b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.485+0000 7f5586775640 1 -- 172.21.15.131:0/1093347240 shutdown_connections 2024-09-17T02:47:35.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.485+0000 7f5586775640 1 -- 172.21.15.131:0/1093347240 wait complete. 2024-09-17T02:47:35.502 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-17T02:47:35.452692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259572,"kb_used_data":10024,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715148,"statfs":{"total":773094113280,"available":772828311552,"internally_reserved":0,"allocated":10264576,"data_stored":5327980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.037834"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422667+0000","last_change":"2024-09-17T02:47:26.451633+0000","last_active":"2024-09-17T02:47:27.422667+0000","last_peered":"2024-09-17T02:47:27.422667+0000","last_clean":"2024-09-17T02:47:27.422667+0000","last_became_active":"2024-09-17T02:47:26.450812+0000","last_became_peered":"2024-09-17T02:47:26.450812+0000","last_unstale":"2024-09-17T02:47:27.422667+0000","last_undegraded":"2024-09-17T02:47:27.422667+0000","last_fullsized":"2024-09-17T02:47:27.422667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:06.130998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768309+0000","last_change":"2024-09-17T02:47:26.435817+0000","last_active":"2024-09-17T02:47:27.768309+0000","last_peered":"2024-09-17T02:47:27.768309+0000","last_clean":"2024-09-17T02:47:27.768309+0000","last_became_active":"2024-09-17T02:47:26.435429+0000","last_became_peered":"2024-09-17T02:47:26.435429+0000","last_unstale":"2024-09-17T02:47:27.768309+0000","last_undegraded":"2024-09-17T02:47:27.768309+0000","last_fullsized":"2024-09-17T02:47:27.768309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:03.221935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422426+0000","last_change":"2024-09-17T02:47:26.422687+0000","last_active":"2024-09-17T02:47:27.422426+0000","last_peered":"2024-09-17T02:47:27.422426+0000","last_clean":"2024-09-17T02:47:27.422426+0000","last_became_active":"2024-09-17T02:47:26.421534+0000","last_became_peered":"2024-09-17T02:47:26.421534+0000","last_unstale":"2024-09-17T02:47:27.422426+0000","last_undegraded":"2024-09-17T02:47:27.422426+0000","last_fullsized":"2024-09-17T02:47:27.422426+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:28.446385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419818+0000","last_change":"2024-09-17T02:47:26.419919+0000","last_active":"2024-09-17T02:47:27.419818+0000","last_peered":"2024-09-17T02:47:27.419818+0000","last_clean":"2024-09-17T02:47:27.419818+0000","last_became_active":"2024-09-17T02:47:26.419062+0000","last_became_peered":"2024-09-17T02:47:26.419062+0000","last_unstale":"2024-09-17T02:47:27.419818+0000","last_undegraded":"2024-09-17T02:47:27.419818+0000","last_fullsized":"2024-09-17T02:47:27.419818+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:17.965654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766901+0000","last_change":"2024-09-17T02:47:26.425433+0000","last_active":"2024-09-17T02:47:27.766901+0000","last_peered":"2024-09-17T02:47:27.766901+0000","last_clean":"2024-09-17T02:47:27.766901+0000","last_became_active":"2024-09-17T02:47:26.420685+0000","last_became_peered":"2024-09-17T02:47:26.420685+0000","last_unstale":"2024-09-17T02:47:27.766901+0000","last_undegraded":"2024-09-17T02:47:27.766901+0000","last_fullsized":"2024-09-17T02:47:27.766901+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:15.124007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766590+0000","last_change":"2024-09-17T02:47:26.454339+0000","last_active":"2024-09-17T02:47:27.766590+0000","last_peered":"2024-09-17T02:47:27.766590+0000","last_clean":"2024-09-17T02:47:27.766590+0000","last_became_active":"2024-09-17T02:47:26.453723+0000","last_became_peered":"2024-09-17T02:47:26.453723+0000","last_unstale":"2024-09-17T02:47:27.766590+0000","last_undegraded":"2024-09-17T02:47:27.766590+0000","last_fullsized":"2024-09-17T02:47:27.766590+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:30.811217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.415989+0000","last_change":"2024-09-17T02:47:26.431390+0000","last_active":"2024-09-17T02:47:27.415989+0000","last_peered":"2024-09-17T02:47:27.415989+0000","last_clean":"2024-09-17T02:47:27.415989+0000","last_became_active":"2024-09-17T02:47:26.431015+0000","last_became_peered":"2024-09-17T02:47:26.431015+0000","last_unstale":"2024-09-17T02:47:27.415989+0000","last_undegraded":"2024-09-17T02:47:27.415989+0000","last_fullsized":"2024-09-17T02:47:27.415989+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:20:51.096970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766358+0000","last_change":"2024-09-17T02:47:26.432290+0000","last_active":"2024-09-17T02:47:27.766358+0000","last_peered":"2024-09-17T02:47:27.766358+0000","last_clean":"2024-09-17T02:47:27.766358+0000","last_became_active":"2024-09-17T02:47:26.431479+0000","last_became_peered":"2024-09-17T02:47:26.431479+0000","last_unstale":"2024-09-17T02:47:27.766358+0000","last_undegraded":"2024-09-17T02:47:27.766358+0000","last_fullsized":"2024-09-17T02:47:27.766358+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:53:39.578242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416330+0000","last_change":"2024-09-17T02:47:26.428950+0000","last_active":"2024-09-17T02:47:27.416330+0000","last_peered":"2024-09-17T02:47:27.416330+0000","last_clean":"2024-09-17T02:47:27.416330+0000","last_became_active":"2024-09-17T02:47:26.428599+0000","last_became_peered":"2024-09-17T02:47:26.428599+0000","last_unstale":"2024-09-17T02:47:27.416330+0000","last_undegraded":"2024-09-17T02:47:27.416330+0000","last_fullsized":"2024-09-17T02:47:27.416330+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:07.335822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419341+0000","last_change":"2024-09-17T02:47:26.433543+0000","last_active":"2024-09-17T02:47:27.419341+0000","last_peered":"2024-09-17T02:47:27.419341+0000","last_clean":"2024-09-17T02:47:27.419341+0000","last_became_active":"2024-09-17T02:47:26.432850+0000","last_became_peered":"2024-09-17T02:47:26.432850+0000","last_unstale":"2024-09-17T02:47:27.419341+0000","last_undegraded":"2024-09-17T02:47:27.419341+0000","last_fullsized":"2024-09-17T02:47:27.419341+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:23.938528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416194+0000","last_change":"2024-09-17T02:47:26.429547+0000","last_active":"2024-09-17T02:47:27.416194+0000","last_peered":"2024-09-17T02:47:27.416194+0000","last_clean":"2024-09-17T02:47:27.416194+0000","last_became_active":"2024-09-17T02:47:26.429189+0000","last_became_peered":"2024-09-17T02:47:26.429189+0000","last_unstale":"2024-09-17T02:47:27.416194+0000","last_undegraded":"2024-09-17T02:47:27.416194+0000","last_fullsized":"2024-09-17T02:47:27.416194+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:35.392361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.420510+0000","last_change":"2024-09-17T02:47:26.453163+0000","last_active":"2024-09-17T02:47:27.420510+0000","last_peered":"2024-09-17T02:47:27.420510+0000","last_clean":"2024-09-17T02:47:27.420510+0000","last_became_active":"2024-09-17T02:47:26.452601+0000","last_became_peered":"2024-09-17T02:47:26.452601+0000","last_unstale":"2024-09-17T02:47:27.420510+0000","last_undegraded":"2024-09-17T02:47:27.420510+0000","last_fullsized":"2024-09-17T02:47:27.420510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:35:34.534013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768784+0000","last_change":"2024-09-17T02:47:26.443086+0000","last_active":"2024-09-17T02:47:27.768784+0000","last_peered":"2024-09-17T02:47:27.768784+0000","last_clean":"2024-09-17T02:47:27.768784+0000","last_became_active":"2024-09-17T02:47:26.442205+0000","last_became_peered":"2024-09-17T02:47:26.442205+0000","last_unstale":"2024-09-17T02:47:27.768784+0000","last_undegraded":"2024-09-17T02:47:27.768784+0000","last_fullsized":"2024-09-17T02:47:27.768784+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:25.973524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746489+0000","last_change":"2024-09-17T02:47:26.423413+0000","last_active":"2024-09-17T02:47:27.746489+0000","last_peered":"2024-09-17T02:47:27.746489+0000","last_clean":"2024-09-17T02:47:27.746489+0000","last_became_active":"2024-09-17T02:47:26.423062+0000","last_became_peered":"2024-09-17T02:47:26.423062+0000","last_unstale":"2024-09-17T02:47:27.746489+0000","last_undegraded":"2024-09-17T02:47:27.746489+0000","last_fullsized":"2024-09-17T02:47:27.746489+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:16.597414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421280+0000","last_change":"2024-09-17T02:47:26.428868+0000","last_active":"2024-09-17T02:47:27.421280+0000","last_peered":"2024-09-17T02:47:27.421280+0000","last_clean":"2024-09-17T02:47:27.421280+0000","last_became_active":"2024-09-17T02:47:26.426622+0000","last_became_peered":"2024-09-17T02:47:26.426622+0000","last_unstale":"2024-09-17T02:47:27.421280+0000","last_undegraded":"2024-09-17T02:47:27.421280+0000","last_fullsized":"2024-09-17T02:47:27.421280+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:26:15.955660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766351+0000","last_change":"2024-09-17T02:47:26.434274+0000","last_active":"2024-09-17T02:47:27.766351+0000","last_peered":"2024-09-17T02:47:27.766351+0000","last_clean":"2024-09-17T02:47:27.766351+0000","last_became_active":"2024-09-17T02:47:26.433517+0000","last_became_peered":"2024-09-17T02:47:26.433517+0000","last_unstale":"2024-09-17T02:47:27.766351+0000","last_undegraded":"2024-09-17T02:47:27.766351+0000","last_fullsized":"2024-09-17T02:47:27.766351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:00.041629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769558+0000","last_change":"2024-09-17T02:47:26.453403+0000","last_active":"2024-09-17T02:47:27.769558+0000","last_peered":"2024-09-17T02:47:27.769558+0000","last_clean":"2024-09-17T02:47:27.769558+0000","last_became_active":"2024-09-17T02:47:26.452810+0000","last_became_peered":"2024-09-17T02:47:26.452810+0000","last_unstale":"2024-09-17T02:47:27.769558+0000","last_undegraded":"2024-09-17T02:47:27.769558+0000","last_fullsized":"2024-09-17T02:47:27.769558+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:06.058284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419516+0000","last_change":"2024-09-17T02:47:26.406401+0000","last_active":"2024-09-17T02:47:27.419516+0000","last_peered":"2024-09-17T02:47:27.419516+0000","last_clean":"2024-09-17T02:47:27.419516+0000","last_became_active":"2024-09-17T02:47:26.405810+0000","last_became_peered":"2024-09-17T02:47:26.405810+0000","last_unstale":"2024-09-17T02:47:27.419516+0000","last_undegraded":"2024-09-17T02:47:27.419516+0000","last_fullsized":"2024-09-17T02:47:27.419516+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:17.427456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768538+0000","last_change":"2024-09-17T02:47:26.421898+0000","last_active":"2024-09-17T02:47:27.768538+0000","last_peered":"2024-09-17T02:47:27.768538+0000","last_clean":"2024-09-17T02:47:27.768538+0000","last_became_active":"2024-09-17T02:47:26.421595+0000","last_became_peered":"2024-09-17T02:47:26.421595+0000","last_unstale":"2024-09-17T02:47:27.768538+0000","last_undegraded":"2024-09-17T02:47:27.768538+0000","last_fullsized":"2024-09-17T02:47:27.768538+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:10.282061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768163+0000","last_change":"2024-09-17T02:47:26.456961+0000","last_active":"2024-09-17T02:47:27.768163+0000","last_peered":"2024-09-17T02:47:27.768163+0000","last_clean":"2024-09-17T02:47:27.768163+0000","last_became_active":"2024-09-17T02:47:26.456375+0000","last_became_peered":"2024-09-17T02:47:26.456375+0000","last_unstale":"2024-09-17T02:47:27.768163+0000","last_undegraded":"2024-09-17T02:47:27.768163+0000","last_fullsized":"2024-09-17T02:47:27.768163+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:22.010481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769004+0000","last_change":"2024-09-17T02:47:26.425659+0000","last_active":"2024-09-17T02:47:27.769004+0000","last_peered":"2024-09-17T02:47:27.769004+0000","last_clean":"2024-09-17T02:47:27.769004+0000","last_became_active":"2024-09-17T02:47:26.418728+0000","last_became_peered":"2024-09-17T02:47:26.418728+0000","last_unstale":"2024-09-17T02:47:27.769004+0000","last_undegraded":"2024-09-17T02:47:27.769004+0000","last_fullsized":"2024-09-17T02:47:27.769004+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:01:12.211104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767567+0000","last_change":"2024-09-17T02:47:26.420752+0000","last_active":"2024-09-17T02:47:27.767567+0000","last_peered":"2024-09-17T02:47:27.767567+0000","last_clean":"2024-09-17T02:47:27.767567+0000","last_became_active":"2024-09-17T02:47:26.420423+0000","last_became_peered":"2024-09-17T02:47:26.420423+0000","last_unstale":"2024-09-17T02:47:27.767567+0000","last_undegraded":"2024-09-17T02:47:27.767567+0000","last_fullsized":"2024-09-17T02:47:27.767567+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:50.371938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766752+0000","last_change":"2024-09-17T02:47:26.449510+0000","last_active":"2024-09-17T02:47:27.766752+0000","last_peered":"2024-09-17T02:47:27.766752+0000","last_clean":"2024-09-17T02:47:27.766752+0000","last_became_active":"2024-09-17T02:47:26.449136+0000","last_became_peered":"2024-09-17T02:47:26.449136+0000","last_unstale":"2024-09-17T02:47:27.766752+0000","last_undegraded":"2024-09-17T02:47:27.766752+0000","last_fullsized":"2024-09-17T02:47:27.766752+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:46.727764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.418593+0000","last_change":"2024-09-17T02:47:26.438730+0000","last_active":"2024-09-17T02:47:27.418593+0000","last_peered":"2024-09-17T02:47:27.418593+0000","last_clean":"2024-09-17T02:47:27.418593+0000","last_became_active":"2024-09-17T02:47:26.420076+0000","last_became_peered":"2024-09-17T02:47:26.420076+0000","last_unstale":"2024-09-17T02:47:27.418593+0000","last_undegraded":"2024-09-17T02:47:27.418593+0000","last_fullsized":"2024-09-17T02:47:27.418593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:12.606293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768851+0000","last_change":"2024-09-17T02:47:26.428029+0000","last_active":"2024-09-17T02:47:27.768851+0000","last_peered":"2024-09-17T02:47:27.768851+0000","last_clean":"2024-09-17T02:47:27.768851+0000","last_became_active":"2024-09-17T02:47:26.427694+0000","last_became_peered":"2024-09-17T02:47:26.427694+0000","last_unstale":"2024-09-17T02:47:27.768851+0000","last_undegraded":"2024-09-17T02:47:27.768851+0000","last_fullsized":"2024-09-17T02:47:27.768851+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:10.261716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768992+0000","last_change":"2024-09-17T02:47:26.427423+0000","last_active":"2024-09-17T02:47:27.768992+0000","last_peered":"2024-09-17T02:47:27.768992+0000","last_clean":"2024-09-17T02:47:27.768992+0000","last_became_active":"2024-09-17T02:47:26.427019+0000","last_became_peered":"2024-09-17T02:47:26.427019+0000","last_unstale":"2024-09-17T02:47:27.768992+0000","last_undegraded":"2024-09-17T02:47:27.768992+0000","last_fullsized":"2024-09-17T02:47:27.768992+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:10.586243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422781+0000","last_change":"2024-09-17T02:47:26.411901+0000","last_active":"2024-09-17T02:47:27.422781+0000","last_peered":"2024-09-17T02:47:27.422781+0000","last_clean":"2024-09-17T02:47:27.422781+0000","last_became_active":"2024-09-17T02:47:26.409711+0000","last_became_peered":"2024-09-17T02:47:26.409711+0000","last_unstale":"2024-09-17T02:47:27.422781+0000","last_undegraded":"2024-09-17T02:47:27.422781+0000","last_fullsized":"2024-09-17T02:47:27.422781+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:04.645639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767833+0000","last_change":"2024-09-17T02:47:26.451152+0000","last_active":"2024-09-17T02:47:27.767833+0000","last_peered":"2024-09-17T02:47:27.767833+0000","last_clean":"2024-09-17T02:47:27.767833+0000","last_became_active":"2024-09-17T02:47:26.445335+0000","last_became_peered":"2024-09-17T02:47:26.445335+0000","last_unstale":"2024-09-17T02:47:27.767833+0000","last_undegraded":"2024-09-17T02:47:27.767833+0000","last_fullsized":"2024-09-17T02:47:27.767833+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:39.809055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769745+0000","last_change":"2024-09-17T02:47:26.453695+0000","last_active":"2024-09-17T02:47:27.769745+0000","last_peered":"2024-09-17T02:47:27.769745+0000","last_clean":"2024-09-17T02:47:27.769745+0000","last_became_active":"2024-09-17T02:47:26.453394+0000","last_became_peered":"2024-09-17T02:47:26.453394+0000","last_unstale":"2024-09-17T02:47:27.769745+0000","last_undegraded":"2024-09-17T02:47:27.769745+0000","last_fullsized":"2024-09-17T02:47:27.769745+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:22:50.371938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766880+0000","last_change":"2024-09-17T02:47:26.424922+0000","last_active":"2024-09-17T02:47:27.766880+0000","last_peered":"2024-09-17T02:47:27.766880+0000","last_clean":"2024-09-17T02:47:27.766880+0000","last_became_active":"2024-09-17T02:47:26.418098+0000","last_became_peered":"2024-09-17T02:47:26.418098+0000","last_unstale":"2024-09-17T02:47:27.766880+0000","last_undegraded":"2024-09-17T02:47:27.766880+0000","last_fullsized":"2024-09-17T02:47:27.766880+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:19:18.176172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767618+0000","last_change":"2024-09-17T02:47:26.411672+0000","last_active":"2024-09-17T02:47:27.767618+0000","last_peered":"2024-09-17T02:47:27.767618+0000","last_clean":"2024-09-17T02:47:27.767618+0000","last_became_active":"2024-09-17T02:47:26.411280+0000","last_became_peered":"2024-09-17T02:47:26.411280+0000","last_unstale":"2024-09-17T02:47:27.767618+0000","last_undegraded":"2024-09-17T02:47:27.767618+0000","last_fullsized":"2024-09-17T02:47:27.767618+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:43.278442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767318+0000","last_change":"2024-09-17T02:47:26.433522+0000","last_active":"2024-09-17T02:47:27.767318+0000","last_peered":"2024-09-17T02:47:27.767318+0000","last_clean":"2024-09-17T02:47:27.767318+0000","last_became_active":"2024-09-17T02:47:26.433017+0000","last_became_peered":"2024-09-17T02:47:26.433017+0000","last_unstale":"2024-09-17T02:47:27.767318+0000","last_undegraded":"2024-09-17T02:47:27.767318+0000","last_fullsized":"2024-09-17T02:47:27.767318+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:39:35.865526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421125+0000","last_change":"2024-09-17T02:47:26.419153+0000","last_active":"2024-09-17T02:47:27.421125+0000","last_peered":"2024-09-17T02:47:27.421125+0000","last_clean":"2024-09-17T02:47:27.421125+0000","last_became_active":"2024-09-17T02:47:26.418280+0000","last_became_peered":"2024-09-17T02:47:26.418280+0000","last_unstale":"2024-09-17T02:47:27.421125+0000","last_undegraded":"2024-09-17T02:47:27.421125+0000","last_fullsized":"2024-09-17T02:47:27.421125+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:31:17.220457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"91'4","reported_seq":19,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769913+0000","last_change":"2024-09-17T02:47:25.404962+0000","last_active":"2024-09-17T02:47:27.769913+0000","last_peered":"2024-09-17T02:47:27.769913+0000","last_clean":"2024-09-17T02:47:27.769913+0000","last_became_active":"2024-09-17T02:47:25.404574+0000","last_became_peered":"2024-09-17T02:47:25.404574+0000","last_unstale":"2024-09-17T02:47:27.769913+0000","last_undegraded":"2024-09-17T02:47:27.769913+0000","last_fullsized":"2024-09-17T02:47:27.769913+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:27.433639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768770+0000","last_change":"2024-09-17T02:47:26.448672+0000","last_active":"2024-09-17T02:47:27.768770+0000","last_peered":"2024-09-17T02:47:27.768770+0000","last_clean":"2024-09-17T02:47:27.768770+0000","last_became_active":"2024-09-17T02:47:26.447991+0000","last_became_peered":"2024-09-17T02:47:26.447991+0000","last_unstale":"2024-09-17T02:47:27.768770+0000","last_undegraded":"2024-09-17T02:47:27.768770+0000","last_fullsized":"2024-09-17T02:47:27.768770+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:05.713702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1e","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746189+0000","last_change":"2024-09-17T02:47:25.394418+0000","last_active":"2024-09-17T02:47:27.746189+0000","last_peered":"2024-09-17T02:47:27.746189+0000","last_clean":"2024-09-17T02:47:27.746189+0000","last_became_active":"2024-09-17T02:47:25.393974+0000","last_became_peered":"2024-09-17T02:47:25.393974+0000","last_unstale":"2024-09-17T02:47:27.746189+0000","last_undegraded":"2024-09-17T02:47:27.746189+0000","last_fullsized":"2024-09-17T02:47:27.746189+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:10.358930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416397+0000","last_change":"2024-09-17T02:47:26.444143+0000","last_active":"2024-09-17T02:47:27.416397+0000","last_peered":"2024-09-17T02:47:27.416397+0000","last_clean":"2024-09-17T02:47:27.416397+0000","last_became_active":"2024-09-17T02:47:26.443779+0000","last_became_peered":"2024-09-17T02:47:26.443779+0000","last_unstale":"2024-09-17T02:47:27.416397+0000","last_undegraded":"2024-09-17T02:47:27.416397+0000","last_fullsized":"2024-09-17T02:47:27.416397+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:28.224146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768881+0000","last_change":"2024-09-17T02:47:25.418775+0000","last_active":"2024-09-17T02:47:27.768881+0000","last_peered":"2024-09-17T02:47:27.768881+0000","last_clean":"2024-09-17T02:47:27.768881+0000","last_became_active":"2024-09-17T02:47:25.418458+0000","last_became_peered":"2024-09-17T02:47:25.418458+0000","last_unstale":"2024-09-17T02:47:27.768881+0000","last_undegraded":"2024-09-17T02:47:27.768881+0000","last_fullsized":"2024-09-17T02:47:27.768881+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:16.952908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419487+0000","last_change":"2024-09-17T02:47:26.438283+0000","last_active":"2024-09-17T02:47:27.419487+0000","last_peered":"2024-09-17T02:47:27.419487+0000","last_clean":"2024-09-17T02:47:27.419487+0000","last_became_active":"2024-09-17T02:47:26.437598+0000","last_became_peered":"2024-09-17T02:47:26.437598+0000","last_unstale":"2024-09-17T02:47:27.419487+0000","last_undegraded":"2024-09-17T02:47:27.419487+0000","last_fullsized":"2024-09-17T02:47:27.419487+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:42.817995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.745987+0000","last_change":"2024-09-17T02:47:25.412944+0000","last_active":"2024-09-17T02:47:27.745987+0000","last_peered":"2024-09-17T02:47:27.745987+0000","last_clean":"2024-09-17T02:47:27.745987+0000","last_became_active":"2024-09-17T02:47:25.412515+0000","last_became_peered":"2024-09-17T02:47:25.412515+0000","last_unstale":"2024-09-17T02:47:27.745987+0000","last_undegraded":"2024-09-17T02:47:27.745987+0000","last_fullsized":"2024-09-17T02:47:27.745987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:24:06.364211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767322+0000","last_change":"2024-09-17T02:47:26.457572+0000","last_active":"2024-09-17T02:47:27.767322+0000","last_peered":"2024-09-17T02:47:27.767322+0000","last_clean":"2024-09-17T02:47:27.767322+0000","last_became_active":"2024-09-17T02:47:26.451939+0000","last_became_peered":"2024-09-17T02:47:26.451939+0000","last_unstale":"2024-09-17T02:47:27.767322+0000","last_undegraded":"2024-09-17T02:47:27.767322+0000","last_fullsized":"2024-09-17T02:47:27.767322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:11:57.435169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1b","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767430+0000","last_change":"2024-09-17T02:47:25.411938+0000","last_active":"2024-09-17T02:47:27.767430+0000","last_peered":"2024-09-17T02:47:27.767430+0000","last_clean":"2024-09-17T02:47:27.767430+0000","last_became_active":"2024-09-17T02:47:25.411486+0000","last_became_peered":"2024-09-17T02:47:25.411486+0000","last_unstale":"2024-09-17T02:47:27.767430+0000","last_undegraded":"2024-09-17T02:47:27.767430+0000","last_fullsized":"2024-09-17T02:47:27.767430+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:07:08.479886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416053+0000","last_change":"2024-09-17T02:47:26.424905+0000","last_active":"2024-09-17T02:47:27.416053+0000","last_peered":"2024-09-17T02:47:27.416053+0000","last_clean":"2024-09-17T02:47:27.416053+0000","last_became_active":"2024-09-17T02:47:26.424551+0000","last_became_peered":"2024-09-17T02:47:26.424551+0000","last_unstale":"2024-09-17T02:47:27.416053+0000","last_undegraded":"2024-09-17T02:47:27.416053+0000","last_fullsized":"2024-09-17T02:47:27.416053+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:22:03.042595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768416+0000","last_change":"2024-09-17T02:47:25.422380+0000","last_active":"2024-09-17T02:47:27.768416+0000","last_peered":"2024-09-17T02:47:27.768416+0000","last_clean":"2024-09-17T02:47:27.768416+0000","last_became_active":"2024-09-17T02:47:25.421906+0000","last_became_peered":"2024-09-17T02:47:25.421906+0000","last_unstale":"2024-09-17T02:47:27.768416+0000","last_undegraded":"2024-09-17T02:47:27.768416+0000","last_fullsized":"2024-09-17T02:47:27.768416+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:17:04.941668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767297+0000","last_change":"2024-09-17T02:47:26.435576+0000","last_active":"2024-09-17T02:47:27.767297+0000","last_peered":"2024-09-17T02:47:27.767297+0000","last_clean":"2024-09-17T02:47:27.767297+0000","last_became_active":"2024-09-17T02:47:26.434782+0000","last_became_peered":"2024-09-17T02:47:26.434782+0000","last_unstale":"2024-09-17T02:47:27.767297+0000","last_undegraded":"2024-09-17T02:47:27.767297+0000","last_fullsized":"2024-09-17T02:47:27.767297+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:12.674675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422144+0000","last_change":"2024-09-17T02:47:25.410268+0000","last_active":"2024-09-17T02:47:27.422144+0000","last_peered":"2024-09-17T02:47:27.422144+0000","last_clean":"2024-09-17T02:47:27.422144+0000","last_became_active":"2024-09-17T02:47:25.409433+0000","last_became_peered":"2024-09-17T02:47:25.409433+0000","last_unstale":"2024-09-17T02:47:27.422144+0000","last_undegraded":"2024-09-17T02:47:27.422144+0000","last_fullsized":"2024-09-17T02:47:27.422144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:46.106338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.420382+0000","last_change":"2024-09-17T02:47:26.410976+0000","last_active":"2024-09-17T02:47:27.420382+0000","last_peered":"2024-09-17T02:47:27.420382+0000","last_clean":"2024-09-17T02:47:27.420382+0000","last_became_active":"2024-09-17T02:47:26.407305+0000","last_became_peered":"2024-09-17T02:47:26.407305+0000","last_unstale":"2024-09-17T02:47:27.420382+0000","last_undegraded":"2024-09-17T02:47:27.420382+0000","last_fullsized":"2024-09-17T02:47:27.420382+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:53.755378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766722+0000","last_change":"2024-09-17T02:47:25.420193+0000","last_active":"2024-09-17T02:47:27.766722+0000","last_peered":"2024-09-17T02:47:27.766722+0000","last_clean":"2024-09-17T02:47:27.766722+0000","last_became_active":"2024-09-17T02:47:25.419700+0000","last_became_peered":"2024-09-17T02:47:25.419700+0000","last_unstale":"2024-09-17T02:47:27.766722+0000","last_undegraded":"2024-09-17T02:47:27.766722+0000","last_fullsized":"2024-09-17T02:47:27.766722+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:05:41.533605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768164+0000","last_change":"2024-09-17T02:47:26.453517+0000","last_active":"2024-09-17T02:47:27.768164+0000","last_peered":"2024-09-17T02:47:27.768164+0000","last_clean":"2024-09-17T02:47:27.768164+0000","last_became_active":"2024-09-17T02:47:26.453175+0000","last_became_peered":"2024-09-17T02:47:26.453175+0000","last_unstale":"2024-09-17T02:47:27.768164+0000","last_undegraded":"2024-09-17T02:47:27.768164+0000","last_fullsized":"2024-09-17T02:47:27.768164+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:16.597414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769913+0000","last_change":"2024-09-17T02:47:25.419435+0000","last_active":"2024-09-17T02:47:27.769913+0000","last_peered":"2024-09-17T02:47:27.769913+0000","last_clean":"2024-09-17T02:47:27.769913+0000","last_became_active":"2024-09-17T02:47:25.418955+0000","last_became_peered":"2024-09-17T02:47:25.418955+0000","last_unstale":"2024-09-17T02:47:27.769913+0000","last_undegraded":"2024-09-17T02:47:27.769913+0000","last_fullsized":"2024-09-17T02:47:27.769913+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:04:03.632893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416447+0000","last_change":"2024-09-17T02:47:26.421172+0000","last_active":"2024-09-17T02:47:27.416447+0000","last_peered":"2024-09-17T02:47:27.416447+0000","last_clean":"2024-09-17T02:47:27.416447+0000","last_became_active":"2024-09-17T02:47:26.420916+0000","last_became_peered":"2024-09-17T02:47:26.420916+0000","last_unstale":"2024-09-17T02:47:27.416447+0000","last_undegraded":"2024-09-17T02:47:27.416447+0000","last_fullsized":"2024-09-17T02:47:27.416447+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:53:29.293488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.420010+0000","last_change":"2024-09-17T02:47:25.414624+0000","last_active":"2024-09-17T02:47:27.420010+0000","last_peered":"2024-09-17T02:47:27.420010+0000","last_clean":"2024-09-17T02:47:27.420010+0000","last_became_active":"2024-09-17T02:47:25.413560+0000","last_became_peered":"2024-09-17T02:47:25.413560+0000","last_unstale":"2024-09-17T02:47:27.420010+0000","last_undegraded":"2024-09-17T02:47:27.420010+0000","last_fullsized":"2024-09-17T02:47:27.420010+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:00.198104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.422254+0000","last_change":"2024-09-17T02:47:26.441675+0000","last_active":"2024-09-17T02:47:27.422254+0000","last_peered":"2024-09-17T02:47:27.422254+0000","last_clean":"2024-09-17T02:47:27.422254+0000","last_became_active":"2024-09-17T02:47:26.437413+0000","last_became_peered":"2024-09-17T02:47:26.437413+0000","last_unstale":"2024-09-17T02:47:27.422254+0000","last_undegraded":"2024-09-17T02:47:27.422254+0000","last_fullsized":"2024-09-17T02:47:27.422254+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:50.459729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"91'3","reported_seq":18,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416794+0000","last_change":"2024-09-17T02:47:25.419022+0000","last_active":"2024-09-17T02:47:27.416794+0000","last_peered":"2024-09-17T02:47:27.416794+0000","last_clean":"2024-09-17T02:47:27.416794+0000","last_became_active":"2024-09-17T02:47:25.418540+0000","last_became_peered":"2024-09-17T02:47:25.418540+0000","last_unstale":"2024-09-17T02:47:27.416794+0000","last_undegraded":"2024-09-17T02:47:27.416794+0000","last_fullsized":"2024-09-17T02:47:27.416794+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:04.700956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769873+0000","last_change":"2024-09-17T02:47:26.429054+0000","last_active":"2024-09-17T02:47:27.769873+0000","last_peered":"2024-09-17T02:47:27.769873+0000","last_clean":"2024-09-17T02:47:27.769873+0000","last_became_active":"2024-09-17T02:47:26.428270+0000","last_became_peered":"2024-09-17T02:47:26.428270+0000","last_unstale":"2024-09-17T02:47:27.769873+0000","last_undegraded":"2024-09-17T02:47:27.769873+0000","last_fullsized":"2024-09-17T02:47:27.769873+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:16.361134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.14","version":"91'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421338+0000","last_change":"2024-09-17T02:47:25.418647+0000","last_active":"2024-09-17T02:47:27.421338+0000","last_peered":"2024-09-17T02:47:27.421338+0000","last_clean":"2024-09-17T02:47:27.421338+0000","last_became_active":"2024-09-17T02:47:25.416994+0000","last_became_peered":"2024-09-17T02:47:25.416994+0000","last_unstale":"2024-09-17T02:47:27.421338+0000","last_undegraded":"2024-09-17T02:47:27.421338+0000","last_fullsized":"2024-09-17T02:47:27.421338+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:58.768722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.418667+0000","last_change":"2024-09-17T02:47:26.419193+0000","last_active":"2024-09-17T02:47:27.418667+0000","last_peered":"2024-09-17T02:47:27.418667+0000","last_clean":"2024-09-17T02:47:27.418667+0000","last_became_active":"2024-09-17T02:47:26.418643+0000","last_became_peered":"2024-09-17T02:47:26.418643+0000","last_unstale":"2024-09-17T02:47:27.418667+0000","last_undegraded":"2024-09-17T02:47:27.418667+0000","last_fullsized":"2024-09-17T02:47:27.418667+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:46.698904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.13","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.770213+0000","last_change":"2024-09-17T02:47:25.418902+0000","last_active":"2024-09-17T02:47:27.770213+0000","last_peered":"2024-09-17T02:47:27.770213+0000","last_clean":"2024-09-17T02:47:27.770213+0000","last_became_active":"2024-09-17T02:47:25.418568+0000","last_became_peered":"2024-09-17T02:47:25.418568+0000","last_unstale":"2024-09-17T02:47:27.770213+0000","last_undegraded":"2024-09-17T02:47:27.770213+0000","last_fullsized":"2024-09-17T02:47:27.770213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:22.925782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767126+0000","last_change":"2024-09-17T02:47:26.414277+0000","last_active":"2024-09-17T02:47:27.767126+0000","last_peered":"2024-09-17T02:47:27.767126+0000","last_clean":"2024-09-17T02:47:27.767126+0000","last_became_active":"2024-09-17T02:47:26.413010+0000","last_became_peered":"2024-09-17T02:47:26.413010+0000","last_unstale":"2024-09-17T02:47:27.767126+0000","last_undegraded":"2024-09-17T02:47:27.767126+0000","last_fullsized":"2024-09-17T02:47:27.767126+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:00.083852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419403+0000","last_change":"2024-09-17T02:47:25.418948+0000","last_active":"2024-09-17T02:47:27.419403+0000","last_peered":"2024-09-17T02:47:27.419403+0000","last_clean":"2024-09-17T02:47:27.419403+0000","last_became_active":"2024-09-17T02:47:25.417436+0000","last_became_peered":"2024-09-17T02:47:25.417436+0000","last_unstale":"2024-09-17T02:47:27.419403+0000","last_undegraded":"2024-09-17T02:47:27.419403+0000","last_fullsized":"2024-09-17T02:47:27.419403+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:25.386796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419176+0000","last_change":"2024-09-17T02:47:26.438873+0000","last_active":"2024-09-17T02:47:27.419176+0000","last_peered":"2024-09-17T02:47:27.419176+0000","last_clean":"2024-09-17T02:47:27.419176+0000","last_became_active":"2024-09-17T02:47:26.436291+0000","last_became_peered":"2024-09-17T02:47:26.436291+0000","last_unstale":"2024-09-17T02:47:27.419176+0000","last_undegraded":"2024-09-17T02:47:27.419176+0000","last_fullsized":"2024-09-17T02:47:27.419176+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:56.037102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.423108+0000","last_change":"2024-09-17T02:47:25.411837+0000","last_active":"2024-09-17T02:47:27.423108+0000","last_peered":"2024-09-17T02:47:27.423108+0000","last_clean":"2024-09-17T02:47:27.423108+0000","last_became_active":"2024-09-17T02:47:25.411299+0000","last_became_peered":"2024-09-17T02:47:25.411299+0000","last_unstale":"2024-09-17T02:47:27.423108+0000","last_undegraded":"2024-09-17T02:47:27.423108+0000","last_fullsized":"2024-09-17T02:47:27.423108+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:43:23.951343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769489+0000","last_change":"2024-09-17T02:47:26.449804+0000","last_active":"2024-09-17T02:47:27.769489+0000","last_peered":"2024-09-17T02:47:27.769489+0000","last_clean":"2024-09-17T02:47:27.769489+0000","last_became_active":"2024-09-17T02:47:26.445395+0000","last_became_peered":"2024-09-17T02:47:26.445395+0000","last_unstale":"2024-09-17T02:47:27.769489+0000","last_undegraded":"2024-09-17T02:47:27.769489+0000","last_fullsized":"2024-09-17T02:47:27.769489+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:24:39.388421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.10","version":"91'8","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.419774+0000","last_change":"2024-09-17T02:47:25.421873+0000","last_active":"2024-09-17T02:47:27.419774+0000","last_peered":"2024-09-17T02:47:27.419774+0000","last_clean":"2024-09-17T02:47:27.419774+0000","last_became_active":"2024-09-17T02:47:25.421196+0000","last_became_peered":"2024-09-17T02:47:25.421196+0000","last_unstale":"2024-09-17T02:47:27.419774+0000","last_undegraded":"2024-09-17T02:47:27.419774+0000","last_fullsized":"2024-09-17T02:47:27.419774+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:44:54.515348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769031+0000","last_change":"2024-09-17T02:47:26.458333+0000","last_active":"2024-09-17T02:47:27.769031+0000","last_peered":"2024-09-17T02:47:27.769031+0000","last_clean":"2024-09-17T02:47:27.769031+0000","last_became_active":"2024-09-17T02:47:26.450921+0000","last_became_peered":"2024-09-17T02:47:26.450921+0000","last_unstale":"2024-09-17T02:47:27.769031+0000","last_undegraded":"2024-09-17T02:47:27.769031+0000","last_fullsized":"2024-09-17T02:47:27.769031+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:15:36.379174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.f","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767855+0000","last_change":"2024-09-17T02:47:25.401532+0000","last_active":"2024-09-17T02:47:27.767855+0000","last_peered":"2024-09-17T02:47:27.767855+0000","last_clean":"2024-09-17T02:47:27.767855+0000","last_became_active":"2024-09-17T02:47:25.400932+0000","last_became_peered":"2024-09-17T02:47:25.400932+0000","last_unstale":"2024-09-17T02:47:27.767855+0000","last_undegraded":"2024-09-17T02:47:27.767855+0000","last_fullsized":"2024-09-17T02:47:27.767855+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:30.981067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416578+0000","last_change":"2024-09-17T02:47:26.407118+0000","last_active":"2024-09-17T02:47:27.416578+0000","last_peered":"2024-09-17T02:47:27.416578+0000","last_clean":"2024-09-17T02:47:27.416578+0000","last_became_active":"2024-09-17T02:47:26.406456+0000","last_became_peered":"2024-09-17T02:47:26.406456+0000","last_unstale":"2024-09-17T02:47:27.416578+0000","last_undegraded":"2024-09-17T02:47:27.416578+0000","last_fullsized":"2024-09-17T02:47:27.416578+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:22:39.049060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.e","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746340+0000","last_change":"2024-09-17T02:47:25.397773+0000","last_active":"2024-09-17T02:47:27.746340+0000","last_peered":"2024-09-17T02:47:27.746340+0000","last_clean":"2024-09-17T02:47:27.746340+0000","last_became_active":"2024-09-17T02:47:25.397452+0000","last_became_peered":"2024-09-17T02:47:25.397452+0000","last_unstale":"2024-09-17T02:47:27.746340+0000","last_undegraded":"2024-09-17T02:47:27.746340+0000","last_fullsized":"2024-09-17T02:47:27.746340+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:34.937753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769006+0000","last_change":"2024-09-17T02:47:26.443993+0000","last_active":"2024-09-17T02:47:27.769006+0000","last_peered":"2024-09-17T02:47:27.769006+0000","last_clean":"2024-09-17T02:47:27.769006+0000","last_became_active":"2024-09-17T02:47:26.443651+0000","last_became_peered":"2024-09-17T02:47:26.443651+0000","last_unstale":"2024-09-17T02:47:27.769006+0000","last_undegraded":"2024-09-17T02:47:27.769006+0000","last_fullsized":"2024-09-17T02:47:27.769006+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:43.278442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.d","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.746255+0000","last_change":"2024-09-17T02:47:25.396451+0000","last_active":"2024-09-17T02:47:27.746255+0000","last_peered":"2024-09-17T02:47:27.746255+0000","last_clean":"2024-09-17T02:47:27.746255+0000","last_became_active":"2024-09-17T02:47:25.396146+0000","last_became_peered":"2024-09-17T02:47:25.396146+0000","last_unstale":"2024-09-17T02:47:27.746255+0000","last_undegraded":"2024-09-17T02:47:27.746255+0000","last_fullsized":"2024-09-17T02:47:27.746255+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:28:55.866634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767876+0000","last_change":"2024-09-17T02:47:26.444819+0000","last_active":"2024-09-17T02:47:27.767876+0000","last_peered":"2024-09-17T02:47:27.767876+0000","last_clean":"2024-09-17T02:47:27.767876+0000","last_became_active":"2024-09-17T02:47:26.443794+0000","last_became_peered":"2024-09-17T02:47:26.443794+0000","last_unstale":"2024-09-17T02:47:27.767876+0000","last_undegraded":"2024-09-17T02:47:27.767876+0000","last_fullsized":"2024-09-17T02:47:27.767876+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:39:26.261568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.b","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416035+0000","last_change":"2024-09-17T02:47:25.407194+0000","last_active":"2024-09-17T02:47:27.416035+0000","last_peered":"2024-09-17T02:47:27.416035+0000","last_clean":"2024-09-17T02:47:27.416035+0000","last_became_active":"2024-09-17T02:47:25.406881+0000","last_became_peered":"2024-09-17T02:47:25.406881+0000","last_unstale":"2024-09-17T02:47:27.416035+0000","last_undegraded":"2024-09-17T02:47:27.416035+0000","last_fullsized":"2024-09-17T02:47:27.416035+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:33:20.997735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.745904+0000","last_change":"2024-09-17T02:47:26.424417+0000","last_active":"2024-09-17T02:47:27.745904+0000","last_peered":"2024-09-17T02:47:27.745904+0000","last_clean":"2024-09-17T02:47:27.745904+0000","last_became_active":"2024-09-17T02:47:26.424080+0000","last_became_peered":"2024-09-17T02:47:26.424080+0000","last_unstale":"2024-09-17T02:47:27.745904+0000","last_undegraded":"2024-09-17T02:47:27.745904+0000","last_fullsized":"2024-09-17T02:47:27.745904+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:00.083852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767755+0000","last_change":"2024-09-17T02:47:25.419788+0000","last_active":"2024-09-17T02:47:27.767755+0000","last_peered":"2024-09-17T02:47:27.767755+0000","last_clean":"2024-09-17T02:47:27.767755+0000","last_became_active":"2024-09-17T02:47:25.419102+0000","last_became_peered":"2024-09-17T02:47:25.419102+0000","last_unstale":"2024-09-17T02:47:27.767755+0000","last_undegraded":"2024-09-17T02:47:27.767755+0000","last_fullsized":"2024-09-17T02:47:27.767755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:25:15.584669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767923+0000","last_change":"2024-09-17T02:47:26.401986+0000","last_active":"2024-09-17T02:47:27.767923+0000","last_peered":"2024-09-17T02:47:27.767923+0000","last_clean":"2024-09-17T02:47:27.767923+0000","last_became_active":"2024-09-17T02:47:26.400697+0000","last_became_peered":"2024-09-17T02:47:26.400697+0000","last_unstale":"2024-09-17T02:47:27.767923+0000","last_undegraded":"2024-09-17T02:47:27.767923+0000","last_fullsized":"2024-09-17T02:47:27.767923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:19.106029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.745766+0000","last_change":"2024-09-17T02:47:25.421863+0000","last_active":"2024-09-17T02:47:27.745766+0000","last_peered":"2024-09-17T02:47:27.745766+0000","last_clean":"2024-09-17T02:47:27.745766+0000","last_became_active":"2024-09-17T02:47:25.421355+0000","last_became_peered":"2024-09-17T02:47:25.421355+0000","last_unstale":"2024-09-17T02:47:27.745766+0000","last_undegraded":"2024-09-17T02:47:27.745766+0000","last_fullsized":"2024-09-17T02:47:27.745766+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:20.461693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421042+0000","last_change":"2024-09-17T02:47:26.417573+0000","last_active":"2024-09-17T02:47:27.421042+0000","last_peered":"2024-09-17T02:47:27.421042+0000","last_clean":"2024-09-17T02:47:27.421042+0000","last_became_active":"2024-09-17T02:47:26.414446+0000","last_became_peered":"2024-09-17T02:47:26.414446+0000","last_unstale":"2024-09-17T02:47:27.421042+0000","last_undegraded":"2024-09-17T02:47:27.421042+0000","last_fullsized":"2024-09-17T02:47:27.421042+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:50.565801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767807+0000","last_change":"2024-09-17T02:47:25.413072+0000","last_active":"2024-09-17T02:47:27.767807+0000","last_peered":"2024-09-17T02:47:27.767807+0000","last_clean":"2024-09-17T02:47:27.767807+0000","last_became_active":"2024-09-17T02:47:25.412033+0000","last_became_peered":"2024-09-17T02:47:25.412033+0000","last_unstale":"2024-09-17T02:47:27.767807+0000","last_undegraded":"2024-09-17T02:47:27.767807+0000","last_fullsized":"2024-09-17T02:47:27.767807+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:25.100410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768080+0000","last_change":"2024-09-17T02:47:26.420721+0000","last_active":"2024-09-17T02:47:27.768080+0000","last_peered":"2024-09-17T02:47:27.768080+0000","last_clean":"2024-09-17T02:47:27.768080+0000","last_became_active":"2024-09-17T02:47:26.420354+0000","last_became_peered":"2024-09-17T02:47:26.420354+0000","last_unstale":"2024-09-17T02:47:27.768080+0000","last_undegraded":"2024-09-17T02:47:27.768080+0000","last_fullsized":"2024-09-17T02:47:27.768080+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:30:05.141305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.7","version":"91'2","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416520+0000","last_change":"2024-09-17T02:47:25.390686+0000","last_active":"2024-09-17T02:47:27.416520+0000","last_peered":"2024-09-17T02:47:27.416520+0000","last_clean":"2024-09-17T02:47:27.416520+0000","last_became_active":"2024-09-17T02:47:25.390236+0000","last_became_peered":"2024-09-17T02:47:25.390236+0000","last_unstale":"2024-09-17T02:47:27.416520+0000","last_undegraded":"2024-09-17T02:47:27.416520+0000","last_fullsized":"2024-09-17T02:47:27.416520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:03:55.492515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768596+0000","last_change":"2024-09-17T02:47:26.450126+0000","last_active":"2024-09-17T02:47:27.768596+0000","last_peered":"2024-09-17T02:47:27.768596+0000","last_clean":"2024-09-17T02:47:27.768596+0000","last_became_active":"2024-09-17T02:47:26.445482+0000","last_became_peered":"2024-09-17T02:47:26.445482+0000","last_unstale":"2024-09-17T02:47:27.768596+0000","last_undegraded":"2024-09-17T02:47:27.768596+0000","last_fullsized":"2024-09-17T02:47:27.768596+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:52:45.742572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"91'1","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416576+0000","last_change":"2024-09-17T02:47:25.396097+0000","last_active":"2024-09-17T02:47:27.416576+0000","last_peered":"2024-09-17T02:47:27.416576+0000","last_clean":"2024-09-17T02:47:27.416576+0000","last_became_active":"2024-09-17T02:47:25.395670+0000","last_became_peered":"2024-09-17T02:47:25.395670+0000","last_unstale":"2024-09-17T02:47:27.416576+0000","last_undegraded":"2024-09-17T02:47:27.416576+0000","last_fullsized":"2024-09-17T02:47:27.416576+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:47:15.348388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769902+0000","last_change":"2024-09-17T02:47:26.449859+0000","last_active":"2024-09-17T02:47:27.769902+0000","last_peered":"2024-09-17T02:47:27.769902+0000","last_clean":"2024-09-17T02:47:27.769902+0000","last_became_active":"2024-09-17T02:47:26.449290+0000","last_became_peered":"2024-09-17T02:47:26.449290+0000","last_unstale":"2024-09-17T02:47:27.769902+0000","last_undegraded":"2024-09-17T02:47:27.769902+0000","last_fullsized":"2024-09-17T02:47:27.769902+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:29.692087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769004+0000","last_change":"2024-09-17T02:47:25.414356+0000","last_active":"2024-09-17T02:47:27.769004+0000","last_peered":"2024-09-17T02:47:27.769004+0000","last_clean":"2024-09-17T02:47:27.769004+0000","last_became_active":"2024-09-17T02:47:25.413979+0000","last_became_peered":"2024-09-17T02:47:25.413979+0000","last_unstale":"2024-09-17T02:47:27.769004+0000","last_undegraded":"2024-09-17T02:47:27.769004+0000","last_fullsized":"2024-09-17T02:47:27.769004+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:59.127265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769424+0000","last_change":"2024-09-17T02:47:26.425824+0000","last_active":"2024-09-17T02:47:27.769424+0000","last_peered":"2024-09-17T02:47:27.769424+0000","last_clean":"2024-09-17T02:47:27.769424+0000","last_became_active":"2024-09-17T02:47:26.414310+0000","last_became_peered":"2024-09-17T02:47:26.414310+0000","last_unstale":"2024-09-17T02:47:27.769424+0000","last_undegraded":"2024-09-17T02:47:27.769424+0000","last_fullsized":"2024-09-17T02:47:27.769424+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:03.730055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.416631+0000","last_change":"2024-09-17T02:47:25.392813+0000","last_active":"2024-09-17T02:47:27.416631+0000","last_peered":"2024-09-17T02:47:27.416631+0000","last_clean":"2024-09-17T02:47:27.416631+0000","last_became_active":"2024-09-17T02:47:25.392472+0000","last_became_peered":"2024-09-17T02:47:25.392472+0000","last_unstale":"2024-09-17T02:47:27.416631+0000","last_undegraded":"2024-09-17T02:47:27.416631+0000","last_fullsized":"2024-09-17T02:47:27.416631+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:05.045538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767992+0000","last_change":"2024-09-17T02:47:26.444405+0000","last_active":"2024-09-17T02:47:27.767992+0000","last_peered":"2024-09-17T02:47:27.767992+0000","last_clean":"2024-09-17T02:47:27.767992+0000","last_became_active":"2024-09-17T02:47:26.443000+0000","last_became_peered":"2024-09-17T02:47:26.443000+0000","last_unstale":"2024-09-17T02:47:27.767992+0000","last_undegraded":"2024-09-17T02:47:27.767992+0000","last_fullsized":"2024-09-17T02:47:27.767992+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:27.766149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421520+0000","last_change":"2024-09-17T02:47:25.404863+0000","last_active":"2024-09-17T02:47:27.421520+0000","last_peered":"2024-09-17T02:47:27.421520+0000","last_clean":"2024-09-17T02:47:27.421520+0000","last_became_active":"2024-09-17T02:47:25.403960+0000","last_became_peered":"2024-09-17T02:47:25.403960+0000","last_unstale":"2024-09-17T02:47:27.421520+0000","last_undegraded":"2024-09-17T02:47:27.421520+0000","last_fullsized":"2024-09-17T02:47:27.421520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:58:38.479422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421658+0000","last_change":"2024-09-17T02:47:26.402827+0000","last_active":"2024-09-17T02:47:27.421658+0000","last_peered":"2024-09-17T02:47:27.421658+0000","last_clean":"2024-09-17T02:47:27.421658+0000","last_became_active":"2024-09-17T02:47:26.402001+0000","last_became_peered":"2024-09-17T02:47:26.402001+0000","last_unstale":"2024-09-17T02:47:27.421658+0000","last_undegraded":"2024-09-17T02:47:27.421658+0000","last_fullsized":"2024-09-17T02:47:27.421658+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:14:09.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769530+0000","last_change":"2024-09-17T02:47:25.403962+0000","last_active":"2024-09-17T02:47:27.769530+0000","last_peered":"2024-09-17T02:47:27.769530+0000","last_clean":"2024-09-17T02:47:27.769530+0000","last_became_active":"2024-09-17T02:47:25.403605+0000","last_became_peered":"2024-09-17T02:47:25.403605+0000","last_unstale":"2024-09-17T02:47:27.769530+0000","last_undegraded":"2024-09-17T02:47:27.769530+0000","last_fullsized":"2024-09-17T02:47:27.769530+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:03:31.843272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.768230+0000","last_change":"2024-09-17T02:47:26.432557+0000","last_active":"2024-09-17T02:47:27.768230+0000","last_peered":"2024-09-17T02:47:27.768230+0000","last_clean":"2024-09-17T02:47:27.768230+0000","last_became_active":"2024-09-17T02:47:26.432209+0000","last_became_peered":"2024-09-17T02:47:26.432209+0000","last_unstale":"2024-09-17T02:47:27.768230+0000","last_undegraded":"2024-09-17T02:47:27.768230+0000","last_fullsized":"2024-09-17T02:47:27.768230+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:10:40.137137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767271+0000","last_change":"2024-09-17T02:47:25.410837+0000","last_active":"2024-09-17T02:47:27.767271+0000","last_peered":"2024-09-17T02:47:27.767271+0000","last_clean":"2024-09-17T02:47:27.767271+0000","last_became_active":"2024-09-17T02:47:25.410644+0000","last_became_peered":"2024-09-17T02:47:25.410644+0000","last_unstale":"2024-09-17T02:47:27.767271+0000","last_undegraded":"2024-09-17T02:47:27.767271+0000","last_fullsized":"2024-09-17T02:47:27.767271+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:41.805249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767390+0000","last_change":"2024-09-17T02:47:26.448485+0000","last_active":"2024-09-17T02:47:27.767390+0000","last_peered":"2024-09-17T02:47:27.767390+0000","last_clean":"2024-09-17T02:47:27.767390+0000","last_became_active":"2024-09-17T02:47:26.447870+0000","last_became_peered":"2024-09-17T02:47:26.447870+0000","last_unstale":"2024-09-17T02:47:27.767390+0000","last_undegraded":"2024-09-17T02:47:27.767390+0000","last_fullsized":"2024-09-17T02:47:27.767390+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:20.845608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.766663+0000","last_change":"2024-09-17T02:47:25.396522+0000","last_active":"2024-09-17T02:47:27.766663+0000","last_peered":"2024-09-17T02:47:27.766663+0000","last_clean":"2024-09-17T02:47:27.766663+0000","last_became_active":"2024-09-17T02:47:25.395729+0000","last_became_peered":"2024-09-17T02:47:25.395729+0000","last_unstale":"2024-09-17T02:47:27.766663+0000","last_undegraded":"2024-09-17T02:47:27.766663+0000","last_fullsized":"2024-09-17T02:47:27.766663+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:52:58.843240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.769403+0000","last_change":"2024-09-17T02:47:26.451744+0000","last_active":"2024-09-17T02:47:27.769403+0000","last_peered":"2024-09-17T02:47:27.769403+0000","last_clean":"2024-09-17T02:47:27.769403+0000","last_became_active":"2024-09-17T02:47:26.451445+0000","last_became_peered":"2024-09-17T02:47:26.451445+0000","last_unstale":"2024-09-17T02:47:27.769403+0000","last_undegraded":"2024-09-17T02:47:27.769403+0000","last_fullsized":"2024-09-17T02:47:27.769403+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:25.364440+0000","last_clean_scrub_stamp":"2024-09-17T02:47:25.364440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:54.458584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.421501+0000","last_change":"2024-09-17T02:47:25.404744+0000","last_active":"2024-09-17T02:47:27.421501+0000","last_peered":"2024-09-17T02:47:27.421501+0000","last_clean":"2024-09-17T02:47:27.421501+0000","last_became_active":"2024-09-17T02:47:25.403842+0000","last_became_peered":"2024-09-17T02:47:25.403842+0000","last_unstale":"2024-09-17T02:47:27.421501+0000","last_undegraded":"2024-09-17T02:47:27.421501+0000","last_fullsized":"2024-09-17T02:47:27.421501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:47:24.351694+0000","last_clean_scrub_stamp":"2024-09-17T02:47:24.351694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:53:02.237396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":246,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-17T02:47:27.767020+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:47:27.767020+0000","last_peered":"2024-09-17T02:47:27.767020+0000","last_clean":"2024-09-17T02:47:27.767020+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:47:27.767020+0000","last_undegraded":"2024-09-17T02:47:27.767020+0000","last_fullsized":"2024-09-17T02:47:27.767020+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509526,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":517803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1720320,"data_stored":1108171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32292,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1126400,"data_stored":517803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509525,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1146880,"data_stored":519527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509526,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1134592,"data_stored":518407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509526,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1732608,"data_stored":1108857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509527,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1130496,"data_stored":517873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32316,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339524,"statfs":{"total":96636764160,"available":96603672576,"internally_reserved":0,"allocated":1146880,"data_stored":519539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:47:35.506 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T02:47:35.506 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T02:47:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.721+0000 7f616e261640 1 -- 172.21.15.131:0/3859894689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6150004d50 msgr2=0x7f6150005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.721+0000 7f616e261640 1 --2- 172.21.15.131:0/3859894689 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6150004d50 0x7f6150005200 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f616000c610 tx=0x7f616001fa40 comp rx=0 tx=0).stop 2024-09-17T02:47:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.721+0000 7f6167fff640 1 -- 172.21.15.131:0/3859894689 reap_dead start 2024-09-17T02:47:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.721+0000 7f616e261640 1 -- 172.21.15.131:0/3859894689 shutdown_connections 2024-09-17T02:47:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.721+0000 7f616e261640 1 -- 172.21.15.131:0/3859894689 >> 172.21.15.131:0/3859894689 conn(0x7f61680cefd0 msgr2=0x7f61680cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:35.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.721+0000 7f616e261640 1 -- 172.21.15.131:0/3859894689 shutdown_connections 2024-09-17T02:47:35.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.721+0000 7f616e261640 1 -- 172.21.15.131:0/3859894689 wait complete. 2024-09-17T02:47:35.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f616e261640 1 Processor -- start 2024-09-17T02:47:35.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f616e261640 1 -- start start 2024-09-17T02:47:35.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f616e261640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61680c5c40 0x7f61680c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:35.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f616e261640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61681f6b10 0x7f61681f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:35.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f616e261640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f61680b2df0 0x7f61680b3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f616e261640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f61680cc920 con 0x7f61681f6b10 2024-09-17T02:47:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f616e261640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f61680cc7a0 con 0x7f61680c5c40 2024-09-17T02:47:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f616e261640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f61680ccaa0 con 0x7f61680b2df0 2024-09-17T02:47:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f6166ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61681f6b10 0x7f61681f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f6166ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61681f6b10 0x7f61681f6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43500/0 (socket says 172.21.15.131:43500) 2024-09-17T02:47:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f6166ffd640 1 -- 172.21.15.131:0/2262328311 learned_addr learned my addr 172.21.15.131:0/2262328311 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:35.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f61677fe640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61680c5c40 0x7f61680c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:35.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f6167fff640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f61680b2df0 0x7f61680b3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:35.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f6166ffd640 1 -- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f61680b2df0 msgr2=0x7f61680b3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:35.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f6166ffd640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f61680b2df0 0x7f61680b3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f6166ffd640 1 -- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61680c5c40 msgr2=0x7f61680c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:35.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f6166ffd640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61680c5c40 0x7f61680c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f6166ffd640 1 -- 172.21.15.131:0/2262328311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61681f7460 con 0x7f61681f6b10 2024-09-17T02:47:35.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f61677fe640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61680c5c40 0x7f61680c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:35.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f6167fff640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f61680b2df0 0x7f61680b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:35.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f6166ffd640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61681f6b10 0x7f61681f6f20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f615c011a50 tx=0x7f615c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:35.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f614ffff640 1 -- 172.21.15.131:0/2262328311 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c01f070 con 0x7f61681f6b10 2024-09-17T02:47:35.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f614ffff640 1 -- 172.21.15.131:0/2262328311 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f615c00fe80 con 0x7f61681f6b10 2024-09-17T02:47:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f614ffff640 1 -- 172.21.15.131:0/2262328311 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615c012690 con 0x7f61681f6b10 2024-09-17T02:47:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f616e261640 1 -- 172.21.15.131:0/2262328311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61680c6690 con 0x7f61681f6b10 2024-09-17T02:47:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.725+0000 7f616e261640 1 -- 172.21.15.131:0/2262328311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61680c6c50 con 0x7f61681f6b10 2024-09-17T02:47:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.729+0000 7f616e261640 1 -- 172.21.15.131:0/2262328311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61680cd030 con 0x7f61681f6b10 2024-09-17T02:47:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.729+0000 7f614ffff640 1 -- 172.21.15.131:0/2262328311 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f615c022070 con 0x7f61681f6b10 2024-09-17T02:47:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.729+0000 7f614ffff640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6154073200 0x7f6154075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.729+0000 7f614ffff640 1 -- 172.21.15.131:0/2262328311 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6283+0+0 (secure 0 0 0) 0x7f615c094e40 con 0x7f61681f6b10 2024-09-17T02:47:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.729+0000 7f61677fe640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6154073200 0x7f6154075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:35.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.729+0000 7f61677fe640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6154073200 0x7f6154075710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c10 tx=0x7f6158006b80 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:35.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.733+0000 7f614ffff640 1 -- 172.21.15.131:0/2262328311 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f61681f7460 con 0x7f61681f6b10 2024-09-17T02:47:35.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.969+0000 7f616e261640 1 -- 172.21.15.131:0/2262328311 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f616815c910 con 0x7f61681f6b10 2024-09-17T02:47:35.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.973+0000 7f614ffff640 1 -- 172.21.15.131:0/2262328311 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v33) ==== 190+0+147 (secure 0 0 0) 0x7f615c05f6f0 con 0x7f61681f6b10 2024-09-17T02:47:35.972 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-09-17T02:47:35.972 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA== 2024-09-17T02:47:35.972 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-09-17T02:47:35.972 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-09-17T02:47:35.973 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T02:47:35.973 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-09-17T02:47:35.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 -- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6154073200 msgr2=0x7f6154075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:35.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6154073200 0x7f6154075710 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6158000c10 tx=0x7f6158006b80 comp rx=0 tx=0).stop 2024-09-17T02:47:35.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 -- 172.21.15.131:0/2262328311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61681f6b10 msgr2=0x7f61681f6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:35.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61681f6b10 0x7f61681f6f20 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f615c011a50 tx=0x7f615c011f10 comp rx=0 tx=0).stop 2024-09-17T02:47:35.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 -- 172.21.15.131:0/2262328311 shutdown_connections 2024-09-17T02:47:35.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6154073200 0x7f6154075710 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f61680b2df0 0x7f61680b3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f61681f6b10 0x7f61681f6f20 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 --2- 172.21.15.131:0/2262328311 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f61680c5c40 0x7f61680c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:35.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 -- 172.21.15.131:0/2262328311 >> 172.21.15.131:0/2262328311 conn(0x7f61680cefd0 msgr2=0x7f6168070990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:35.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 -- 172.21.15.131:0/2262328311 shutdown_connections 2024-09-17T02:47:35.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:35.977+0000 7f616e261640 1 -- 172.21.15.131:0/2262328311 wait complete. 2024-09-17T02:47:35.993 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:47:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.153+0000 7fa6735b5640 1 -- 172.21.15.131:0/3118508362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa66c162370 msgr2=0x7fa66c162780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.153+0000 7fa6735b5640 1 --2- 172.21.15.131:0/3118508362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa66c162370 0x7fa66c162780 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa66c06e9a0 tx=0x7fa66801c810 comp rx=0 tx=0).stop 2024-09-17T02:47:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.153+0000 7fa6735b5640 1 -- 172.21.15.131:0/3118508362 shutdown_connections 2024-09-17T02:47:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.153+0000 7fa6735b5640 1 --2- 172.21.15.131:0/3118508362 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa66c162370 0x7fa66c162780 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.153+0000 7fa6735b5640 1 --2- 172.21.15.131:0/3118508362 >> v2:172.21.15.189:3300/0 conn(0x7fa66c0cac80 0x7fa66c0cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.153+0000 7fa6735b5640 1 --2- 172.21.15.131:0/3118508362 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa66c06dea0 0x7fa66c06e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.153+0000 7fa6735b5640 1 -- 172.21.15.131:0/3118508362 >> 172.21.15.131:0/3118508362 conn(0x7fa66c06eea0 msgr2=0x7fa66c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:36.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.153+0000 7fa6735b5640 1 -- 172.21.15.131:0/3118508362 shutdown_connections 2024-09-17T02:47:36.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.153+0000 7fa6735b5640 1 -- 172.21.15.131:0/3118508362 wait complete. 2024-09-17T02:47:36.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.153+0000 7fa6735b5640 1 Processor -- start 2024-09-17T02:47:36.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.153+0000 7fa6735b5640 1 -- start start 2024-09-17T02:47:36.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa6735b5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa66c06dea0 0x7fa66c14af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa6735b5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa66c0cac80 0x7fa66c156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa6735b5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa66c162370 0x7fa66c15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa6735b5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c0cf150 con 0x7fa66c06dea0 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa6735b5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c0cefd0 con 0x7fa66c0cac80 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa6735b5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa66c0cf2d0 con 0x7fa66c162370 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa671351640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa66c06dea0 0x7fa66c14af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa671351640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa66c06dea0 0x7fa66c14af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43512/0 (socket says 172.21.15.131:43512) 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa671351640 1 -- 172.21.15.131:0/1585744892 learned_addr learned my addr 172.21.15.131:0/1585744892 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa670b50640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa66c0cac80 0x7fa66c156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa671351640 1 -- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa66c162370 msgr2=0x7fa66c15b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa671351640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa66c162370 0x7fa66c15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa671351640 1 -- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa66c0cac80 msgr2=0x7fa66c156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa671351640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa66c0cac80 0x7fa66c156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa671351640 1 -- 172.21.15.131:0/1585744892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c0c7590 con 0x7fa66c06dea0 2024-09-17T02:47:36.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa671351640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa66c06dea0 0x7fa66c14af80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa66400d960 tx=0x7fa66400de20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:36.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa661ffb640 1 -- 172.21.15.131:0/1585744892 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664016070 con 0x7fa66c06dea0 2024-09-17T02:47:36.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa661ffb640 1 -- 172.21.15.131:0/1585744892 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa664004e20 con 0x7fa66c06dea0 2024-09-17T02:47:36.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa661ffb640 1 -- 172.21.15.131:0/1585744892 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66400bd70 con 0x7fa66c06dea0 2024-09-17T02:47:36.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa6735b5640 1 -- 172.21.15.131:0/1585744892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c0b4a80 con 0x7fa66c06dea0 2024-09-17T02:47:36.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa6735b5640 1 -- 172.21.15.131:0/1585744892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66c14b4c0 con 0x7fa66c06dea0 2024-09-17T02:47:36.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa6735b5640 1 -- 172.21.15.131:0/1585744892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa66c0cb090 con 0x7fa66c06dea0 2024-09-17T02:47:36.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.157+0000 7fa661ffb640 1 -- 172.21.15.131:0/1585744892 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa664003980 con 0x7fa66c06dea0 2024-09-17T02:47:36.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.161+0000 7fa661ffb640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa650073200 0x7fa650075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.161+0000 7fa661ffb640 1 -- 172.21.15.131:0/1585744892 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6283+0+0 (secure 0 0 0) 0x7fa664095300 con 0x7fa66c06dea0 2024-09-17T02:47:36.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.161+0000 7fa670b50640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa650073200 0x7fa650075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.161+0000 7fa670b50640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa650073200 0x7fa650075710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa65c005d90 tx=0x7fa65c005ca0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:36.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.165+0000 7fa661ffb640 1 -- 172.21.15.131:0/1585744892 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa66401f020 con 0x7fa66c06dea0 2024-09-17T02:47:36.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.369+0000 7fa6735b5640 1 -- 172.21.15.131:0/1585744892 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa66c070c70 con 0x7fa66c06dea0 2024-09-17T02:47:36.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.369+0000 7fa661ffb640 1 -- 172.21.15.131:0/1585744892 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5237 (secure 0 0 0) 0x7fa664003c70 con 0x7fa66c06dea0 2024-09-17T02:47:36.367 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:36.367 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":64,"btime":"2024-09-17T02:47:29:415704+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:47:28.826817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2766162522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2766162522},{"type":"v1","addr":"172.21.15.131:6835","nonce":2766162522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7038,"qdb_cluster":[7038]},"id":6}]} 2024-09-17T02:47:36.367 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 64 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 -- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa650073200 msgr2=0x7fa650075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa650073200 0x7fa650075710 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa65c005d90 tx=0x7fa65c005ca0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 -- 172.21.15.131:0/1585744892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa66c06dea0 msgr2=0x7fa66c14af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa66c06dea0 0x7fa66c14af80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa66400d960 tx=0x7fa66400de20 comp rx=0 tx=0).stop 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 -- 172.21.15.131:0/1585744892 shutdown_connections 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa650073200 0x7fa650075710 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa66c162370 0x7fa66c15b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa66c0cac80 0x7fa66c156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 --2- 172.21.15.131:0/1585744892 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa66c06dea0 0x7fa66c14af80 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 -- 172.21.15.131:0/1585744892 >> 172.21.15.131:0/1585744892 conn(0x7fa66c06eea0 msgr2=0x7fa66c154e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 -- 172.21.15.131:0/1585744892 shutdown_connections 2024-09-17T02:47:36.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.373+0000 7fa6735b5640 1 -- 172.21.15.131:0/1585744892 wait complete. 2024-09-17T02:47:36.792 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:47:25.797713+0000', 'modified': '2024-09-17T02:47:28.826817+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7038}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7038': {'gid': 7038, 'name': 'c', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6835/2766162522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 2766162522}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 2766162522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7038, 'qdb_cluster': [7038]} 2024-09-17T02:47:36.792 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:47:36.792 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T02:47:36.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 -- 172.21.15.131:0/4195191086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee00c53a0 msgr2=0x7fcee00cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:36.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 --2- 172.21.15.131:0/4195191086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee00c53a0 0x7fcee00cf010 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fced8009bc0 tx=0x7fced801cb10 comp rx=0 tx=0).stop 2024-09-17T02:47:36.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 -- 172.21.15.131:0/4195191086 shutdown_connections 2024-09-17T02:47:36.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 --2- 172.21.15.131:0/4195191086 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee00c53a0 0x7fcee00cf010 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 -- 172.21.15.131:0/4195191086 >> 172.21.15.131:0/4195191086 conn(0x7fcee006eea0 msgr2=0x7fcee006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:36.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 -- 172.21.15.131:0/4195191086 shutdown_connections 2024-09-17T02:47:36.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 -- 172.21.15.131:0/4195191086 wait complete. 2024-09-17T02:47:36.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 Processor -- start 2024-09-17T02:47:36.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 -- start start 2024-09-17T02:47:36.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcee006e2c0 0x7fcee014dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee00c53a0 0x7fcee00c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcee00cec00 0x7fcee00c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcee016aed0 con 0x7fcee00c53a0 2024-09-17T02:47:36.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcee016ad50 con 0x7fcee00cec00 2024-09-17T02:47:36.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcee61ac640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcee016b050 con 0x7fcee006e2c0 2024-09-17T02:47:36.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcedeffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee00c53a0 0x7fcee00c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcedeffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee00c53a0 0x7fcee00c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43544/0 (socket says 172.21.15.131:43544) 2024-09-17T02:47:36.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.961+0000 7fcedeffd640 1 -- 172.21.15.131:0/3593018061 learned_addr learned my addr 172.21.15.131:0/3593018061 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:36.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcedffff640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcee00cec00 0x7fcee00c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcee006e2c0 0x7fcee014dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcedeffd640 1 -- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcee006e2c0 msgr2=0x7fcee014dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:36.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcedeffd640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcee006e2c0 0x7fcee014dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcedeffd640 1 -- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcee00cec00 msgr2=0x7fcee00c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:36.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcedeffd640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcee00cec00 0x7fcee00c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcedeffd640 1 -- 172.21.15.131:0/3593018061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee00b13a0 con 0x7fcee00c53a0 2024-09-17T02:47:36.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcee006e2c0 0x7fcee014dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:36.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcedeffd640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee00c53a0 0x7fcee00c9190 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fced400c3c0 tx=0x7fced400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:36.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcec7fff640 1 -- 172.21.15.131:0/3593018061 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4017070 con 0x7fcee00c53a0 2024-09-17T02:47:36.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcec7fff640 1 -- 172.21.15.131:0/3593018061 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fced40053c0 con 0x7fcee00c53a0 2024-09-17T02:47:36.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcee61ac640 1 -- 172.21.15.131:0/3593018061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee00c86b0 con 0x7fcee00c53a0 2024-09-17T02:47:36.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcec7fff640 1 -- 172.21.15.131:0/3593018061 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4003860 con 0x7fcee00c53a0 2024-09-17T02:47:36.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcee61ac640 1 -- 172.21.15.131:0/3593018061 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee00c8c70 con 0x7fcee00c53a0 2024-09-17T02:47:36.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcec7fff640 1 -- 172.21.15.131:0/3593018061 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fced401a070 con 0x7fcee00c53a0 2024-09-17T02:47:36.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcec7fff640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcecc073200 0x7fcecc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcec7fff640 1 -- 172.21.15.131:0/3593018061 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6283+0+0 (secure 0 0 0) 0x7fced40937c0 con 0x7fcee00c53a0 2024-09-17T02:47:36.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcecc073200 0x7fcecc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fceed477000 1 Processor -- start 2024-09-17T02:47:36.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.965+0000 7fceed477000 1 -- start start 2024-09-17T02:47:36.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcecc073200 0x7fcecc075710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fced0001230 tx=0x7fced0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:36.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93b7b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a2a950 0x557c93b7bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a13ca0 0x557c93b7c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557c93a50410 con 0x557c9363a460 2024-09-17T02:47:36.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557c9398db40 con 0x557c93a13ca0 2024-09-17T02:47:36.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557c9397cf70 con 0x557c93a2a950 2024-09-17T02:47:36.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93b7b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93b7b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43560/0 (socket says 172.21.15.131:43560) 2024-09-17T02:47:36.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedf7fe640 1 -- 172.21.15.131:0/241381661 learned_addr learned my addr 172.21.15.131:0/241381661 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:36.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedeffd640 1 --2- 172.21.15.131:0/241381661 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a2a950 0x557c93b7bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedffff640 1 --2- 172.21.15.131:0/241381661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a13ca0 0x557c93b7c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedf7fe640 1 -- 172.21.15.131:0/241381661 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a2a950 msgr2=0x557c93b7bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/241381661 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a2a950 0x557c93b7bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedf7fe640 1 -- 172.21.15.131:0/241381661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a13ca0 msgr2=0x557c93b7c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/241381661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a13ca0 0x557c93b7c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedf7fe640 1 -- 172.21.15.131:0/241381661 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c939445b0 con 0x557c9363a460 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/241381661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93b7b6a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fced0007bd0 tx=0x7fced0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e906708ba8141a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcec57fa640 1 -- 172.21.15.131:0/241381661 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0024070 con 0x557c9363a460 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcec57fa640 1 -- 172.21.15.131:0/241381661 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fced0014db0 con 0x557c9363a460 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fcec57fa640 1 -- 172.21.15.131:0/241381661 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced001f640 con 0x557c9363a460 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 -- 172.21.15.131:0/241381661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 msgr2=0x557c93b7b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 --2- 172.21.15.131:0/241381661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93b7b6a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fced0007bd0 tx=0x7fced0010000 comp rx=0 tx=0).stop 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 -- 172.21.15.131:0/241381661 shutdown_connections 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 --2- 172.21.15.131:0/241381661 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a13ca0 0x557c93b7c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 --2- 172.21.15.131:0/241381661 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a2a950 0x557c93b7bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 --2- 172.21.15.131:0/241381661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93b7b6a0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 -- 172.21.15.131:0/241381661 >> 172.21.15.131:0/241381661 conn(0x557c93b00d90 msgr2=0x557c93b30690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 -- 172.21.15.131:0/241381661 shutdown_connections 2024-09-17T02:47:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 -- 172.21.15.131:0/241381661 wait complete. 2024-09-17T02:47:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 -- start start 2024-09-17T02:47:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.969+0000 7fceed477000 1 Processor -- start 2024-09-17T02:47:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c9363a460 0x557c93b82920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a13ca0 0x557c93b82e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a2a950 0x557c93b833a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557c939175f0 con 0x557c93a13ca0 2024-09-17T02:47:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557c93980d80 con 0x557c93a2a950 2024-09-17T02:47:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557c93923520 con 0x557c9363a460 2024-09-17T02:47:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 10 client.0 _ll_register_callbacks cb 0x557c93927830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:47:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedeffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a13ca0 0x557c93b82e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedeffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a13ca0 0x557c93b82e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43572/0 (socket says 172.21.15.131:43572) 2024-09-17T02:47:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedeffd640 1 -- 172.21.15.131:0/473878007 learned_addr learned my addr 172.21.15.131:0/473878007 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedffff640 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a2a950 0x557c93b833a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c9363a460 0x557c93b82920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedeffd640 1 -- 172.21.15.131:0/473878007 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c9363a460 msgr2=0x557c93b82920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedeffd640 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c9363a460 0x557c93b82920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedeffd640 1 -- 172.21.15.131:0/473878007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a2a950 msgr2=0x557c93b833a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedeffd640 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a2a950 0x557c93b833a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedeffd640 1 -- 172.21.15.131:0/473878007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c9391b190 con 0x557c93a13ca0 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedeffd640 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a13ca0 0x557c93b82e60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fced400a040 tx=0x7fced405d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda228764b982c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcebe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcebe7fc640 1 -- 172.21.15.131:0/473878007 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced4062980 con 0x557c93a13ca0 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcebe7fc640 1 -- 172.21.15.131:0/473878007 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6283+0+0 (secure 0 0 0) 0x7fced405d310 con 0x557c93a13ca0 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 1 -- 172.21.15.131:0/473878007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557c93915110 con 0x557c93a13ca0 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcebe7fc640 1 -- 172.21.15.131:0/473878007 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fced40678c0 con 0x557c93a13ca0 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 10 client.7179 fetch_fsmap learned FSMap version 64 2024-09-17T02:47:36.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 1 -- 172.21.15.131:0/473878007 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x557c93a01f10 con 0x557c93a13ca0 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcebe7fc640 1 -- 172.21.15.131:0/473878007 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557c93915110 con 0x557c93a13ca0 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcebe7fc640 1 -- 172.21.15.131:0/473878007 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 64) ==== 3502+0+0 (secure 0 0 0) 0x7fced40597f0 con 0x557c93a13ca0 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 10 client.7179 fetch_fsmap finished waiting for FSMap version 64 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 10 client.7179 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 20 client.7179 populate_metadata read hostname 'smithi131' 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.131:6834/2766162522,v1:172.21.15.131:6835/2766162522] conn(0x557c9361f7d0 0x557c93b8cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 4 client.7179 mds_command: new command op to 7038 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 1 -- 172.21.15.131:0/473878007 --> [v2:172.21.15.131:6834/2766162522,v1:172.21.15.131:6835/2766162522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c9398fa90 con 0x557c9361f7d0 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.131:6834/2766162522,v1:172.21.15.131:6835/2766162522] conn(0x557c9361f7d0 0x557c93b8cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcebe7fc640 10 client.7179 ms_handle_connect on v2:172.21.15.131:6834/2766162522 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.131:6834/2766162522,v1:172.21.15.131:6835/2766162522] conn(0x557c9361f7d0 0x557c93b8cd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fbadf5d8c537670 server_cookie=7a5380ca170c9fc2 in_seq=0 out_seq=0 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcebe7fc640 1 -- 172.21.15.131:0/473878007 <== mds.0 v2:172.21.15.131:6834/2766162522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557c9398fa90 con 0x557c9361f7d0 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fcebe7fc640 10 client.7179 handle_command_reply: tid=0 2024-09-17T02:47:36.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.973+0000 7fceed477000 1 client.7179 shutdown 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fceed477000 1 -- 172.21.15.131:0/473878007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a13ca0 msgr2=0x557c93b82e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fceed477000 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a13ca0 0x557c93b82e60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fced400a040 tx=0x7fced405d740 comp rx=0 tx=0).stop 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fceed477000 1 -- 172.21.15.131:0/473878007 shutdown_connections 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fceed477000 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.131:6834/2766162522,v1:172.21.15.131:6835/2766162522] conn(0x557c9361f7d0 0x557c93b8cd00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fceed477000 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a2a950 0x557c93b833a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fceed477000 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a13ca0 0x557c93b82e60 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fceed477000 1 --2- 172.21.15.131:0/473878007 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c9363a460 0x557c93b82920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fceed477000 1 -- 172.21.15.131:0/473878007 >> 172.21.15.131:0/473878007 conn(0x557c93b00d90 msgr2=0x557c93b37440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fcebe7fc640 0 client.7179 ms_handle_reset on v2:172.21.15.131:6834/2766162522 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fceed477000 1 -- 172.21.15.131:0/473878007 shutdown_connections 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fceed477000 1 -- 172.21.15.131:0/473878007 wait complete. 2024-09-17T02:47:36.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.977+0000 7fceed477000 20 client.7179 trim_cache size 0 max 16384 2024-09-17T02:47:36.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.993+0000 7fceed477000 1 Processor -- start 2024-09-17T02:47:36.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.993+0000 7fceed477000 1 -- start start 2024-09-17T02:47:36.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93bed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a13ca0 0x557c93bedb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a2a950 0x557c93bee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0205960 con 0x557c9363a460 2024-09-17T02:47:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcee0204940 con 0x557c93a13ca0 2024-09-17T02:47:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcee01fca20 con 0x557c93a2a950 2024-09-17T02:47:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcedf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93bed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcedf7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93bed620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43578/0 (socket says 172.21.15.131:43578) 2024-09-17T02:47:36.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcedf7fe640 1 -- 172.21.15.131:0/1394070304 learned_addr learned my addr 172.21.15.131:0/1394070304 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcedeffd640 1 --2- 172.21.15.131:0/1394070304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a13ca0 0x557c93bedb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcedf7fe640 1 -- 172.21.15.131:0/1394070304 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a2a950 msgr2=0x557c93bee0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/1394070304 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a2a950 0x557c93bee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcedf7fe640 1 -- 172.21.15.131:0/1394070304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a13ca0 msgr2=0x557c93bedb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/1394070304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a13ca0 0x557c93bedb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcedf7fe640 1 -- 172.21.15.131:0/1394070304 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c939d8db0 con 0x557c9363a460 2024-09-17T02:47:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/1394070304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93bed620 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fced000d520 tx=0x7fced00011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92716850dd2bfc2c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:36.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcec4ff9640 1 -- 172.21.15.131:0/1394070304 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0024040 con 0x557c9363a460 2024-09-17T02:47:36.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcec4ff9640 1 -- 172.21.15.131:0/1394070304 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fced0005020 con 0x557c9363a460 2024-09-17T02:47:36.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fcec4ff9640 1 -- 172.21.15.131:0/1394070304 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced0005410 con 0x557c9363a460 2024-09-17T02:47:36.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 -- 172.21.15.131:0/1394070304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 msgr2=0x557c93bed620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:36.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 --2- 172.21.15.131:0/1394070304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93bed620 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fced000d520 tx=0x7fced00011a0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 -- 172.21.15.131:0/1394070304 shutdown_connections 2024-09-17T02:47:36.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 --2- 172.21.15.131:0/1394070304 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a2a950 0x557c93bee0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 --2- 172.21.15.131:0/1394070304 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c93a13ca0 0x557c93bedb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.998 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 --2- 172.21.15.131:0/1394070304 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c9363a460 0x557c93bed620 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:36.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 -- 172.21.15.131:0/1394070304 >> 172.21.15.131:0/1394070304 conn(0x557c93b00d90 msgr2=0x557c93b79620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:36.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 -- 172.21.15.131:0/1394070304 shutdown_connections 2024-09-17T02:47:36.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 -- 172.21.15.131:0/1394070304 wait complete. 2024-09-17T02:47:36.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 -- start start 2024-09-17T02:47:36.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:36.997+0000 7fceed477000 1 Processor -- start 2024-09-17T02:47:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c9363a460 0x557c93bf1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a13ca0 0x557c93bf18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a2a950 0x557c93bf1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557c93a01a80 con 0x557c93a2a950 2024-09-17T02:47:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557c9398c910 con 0x557c9363a460 2024-09-17T02:47:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557c93a3c750 con 0x557c93a13ca0 2024-09-17T02:47:37.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 10 client.0 _ll_register_callbacks cb 0x7fced405a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a2a950 0x557c93bf1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a2a950 0x557c93bf1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43594/0 (socket says 172.21.15.131:43594) 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedffff640 1 -- 172.21.15.131:0/2992687536 learned_addr learned my addr 172.21.15.131:0/2992687536 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedeffd640 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a13ca0 0x557c93bf18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c9363a460 0x557c93bf1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedffff640 1 -- 172.21.15.131:0/2992687536 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a13ca0 msgr2=0x557c93bf18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedffff640 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a13ca0 0x557c93bf18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedffff640 1 -- 172.21.15.131:0/2992687536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c9363a460 msgr2=0x557c93bf1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedffff640 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c9363a460 0x557c93bf1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedffff640 1 -- 172.21.15.131:0/2992687536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c93943ab0 con 0x557c93a2a950 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedeffd640 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a13ca0 0x557c93bf18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:37.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c9363a460 0x557c93bf1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedffff640 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a2a950 0x557c93bf1e10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fced8009b10 tx=0x7fced8003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490327cdb848fb64 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcebe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:47:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcebe7fc640 1 -- 172.21.15.131:0/2992687536 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced80b2070 con 0x557c93a2a950 2024-09-17T02:47:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcebe7fc640 1 -- 172.21.15.131:0/2992687536 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 6283+0+0 (secure 0 0 0) 0x7fced8009200 con 0x557c93a2a950 2024-09-17T02:47:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 -- 172.21.15.131:0/2992687536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fced0005020 con 0x557c93a2a950 2024-09-17T02:47:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcebe7fc640 1 -- 172.21.15.131:0/2992687536 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fced8004a50 con 0x557c93a2a950 2024-09-17T02:47:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 10 client.7185 fetch_fsmap learned FSMap version 64 2024-09-17T02:47:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 -- 172.21.15.131:0/2992687536 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) -- 0x557c93922070 con 0x557c93a2a950 2024-09-17T02:47:37.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcebe7fc640 1 -- 172.21.15.131:0/2992687536 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fced0005020 con 0x557c93a2a950 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcebe7fc640 1 -- 172.21.15.131:0/2992687536 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 64) ==== 3502+0+0 (secure 0 0 0) 0x7fced8004bf0 con 0x557c93a2a950 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 10 client.7185 fetch_fsmap finished waiting for FSMap version 64 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 10 client.7185 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 20 client.7185 populate_metadata read hostname 'smithi131' 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.131:6834/2766162522,v1:172.21.15.131:6835/2766162522] conn(0x557c9360d830 0x557c93bad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 4 client.7185 mds_command: new command op to 7038 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 -- 172.21.15.131:0/2992687536 --> [v2:172.21.15.131:6834/2766162522,v1:172.21.15.131:6835/2766162522] -- command(tid 0: {"prefix": "status"}) -- 0x557c9398b540 con 0x557c9360d830 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.131:6834/2766162522,v1:172.21.15.131:6835/2766162522] conn(0x557c9360d830 0x557c93bad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcebe7fc640 10 client.7185 ms_handle_connect on v2:172.21.15.131:6834/2766162522 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcedf7fe640 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.131:6834/2766162522,v1:172.21.15.131:6835/2766162522] conn(0x557c9360d830 0x557c93bad730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8748e7bfe9ac93d0 server_cookie=9a4b12ec8464e7ac in_seq=0 out_seq=0 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcebe7fc640 1 -- 172.21.15.131:0/2992687536 <== mds.0 v2:172.21.15.131:6834/2766162522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x557c9398b540 con 0x557c9360d830 2024-09-17T02:47:37.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcebe7fc640 10 client.7185 handle_command_reply: tid=0 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 client.7185 shutdown 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 -- 172.21.15.131:0/2992687536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a2a950 msgr2=0x557c93bf1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a2a950 0x557c93bf1e10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fced8009b10 tx=0x7fced8003e80 comp rx=0 tx=0).stop 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 -- 172.21.15.131:0/2992687536 shutdown_connections 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.131:6834/2766162522,v1:172.21.15.131:6835/2766162522] conn(0x557c9360d830 0x557c93bad730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557c93a2a950 0x557c93bf1e10 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557c93a13ca0 0x557c93bf18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcebe7fc640 0 client.7185 ms_handle_reset on v2:172.21.15.131:6834/2766162522 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 --2- 172.21.15.131:0/2992687536 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557c9363a460 0x557c93bf1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 -- 172.21.15.131:0/2992687536 >> 172.21.15.131:0/2992687536 conn(0x557c93b00d90 msgr2=0x557c93b8c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 -- 172.21.15.131:0/2992687536 shutdown_connections 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 1 -- 172.21.15.131:0/2992687536 wait complete. 2024-09-17T02:47:37.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fceed477000 20 client.7185 trim_cache size 0 max 16384 2024-09-17T02:47:37.010 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:47:37.010 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:47:37.010 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:47:37.010 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7038, 2024-09-17T02:47:37.010 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:47:37.010 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:47:37.010 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:47:37.010 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 10.600268227999999, 2024-09-17T02:47:37.010 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 64, 2024-09-17T02:47:37.010 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 94, 2024-09-17T02:47:37.010 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 92, 2024-09-17T02:47:37.011 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 16.07604744 2024-09-17T02:47:37.011 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:47:37.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 -- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcecc073200 msgr2=0x7fcecc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:37.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcecc073200 0x7fcecc075710 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fced0001230 tx=0x7fced0009040 comp rx=0 tx=0).stop 2024-09-17T02:47:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 -- 172.21.15.131:0/3593018061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee00c53a0 msgr2=0x7fcee00c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee00c53a0 0x7fcee00c9190 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fced400c3c0 tx=0x7fced400c880 comp rx=0 tx=0).stop 2024-09-17T02:47:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 -- 172.21.15.131:0/3593018061 shutdown_connections 2024-09-17T02:47:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcecc073200 0x7fcecc075710 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcee00cec00 0x7fcee00c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcee00c53a0 0x7fcee00c9190 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 --2- 172.21.15.131:0/3593018061 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcee006e2c0 0x7fcee014dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 -- 172.21.15.131:0/3593018061 >> 172.21.15.131:0/3593018061 conn(0x7fcee006eea0 msgr2=0x7fcee00cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 -- 172.21.15.131:0/3593018061 shutdown_connections 2024-09-17T02:47:37.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.001+0000 7fcee61ac640 1 -- 172.21.15.131:0/3593018061 wait complete. 2024-09-17T02:47:37.019 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:47:37.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.165+0000 7f8bc5a53640 1 -- 172.21.15.131:0/2625817533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc015b7d0 msgr2=0x7f8bc0071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:37.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.165+0000 7f8bc5a53640 1 --2- 172.21.15.131:0/2625817533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc015b7d0 0x7f8bc0071210 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009900 tx=0x7f8bb001c850 comp rx=0 tx=0).stop 2024-09-17T02:47:37.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.165+0000 7f8bc5a53640 1 -- 172.21.15.131:0/2625817533 shutdown_connections 2024-09-17T02:47:37.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.165+0000 7f8bc5a53640 1 --2- 172.21.15.131:0/2625817533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc015b7d0 0x7f8bc0071210 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.165+0000 7f8bc5a53640 1 -- 172.21.15.131:0/2625817533 >> 172.21.15.131:0/2625817533 conn(0x7f8bc006fdb0 msgr2=0x7f8bc00701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:37.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.165+0000 7f8bc5a53640 1 -- 172.21.15.131:0/2625817533 shutdown_connections 2024-09-17T02:47:37.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.165+0000 7f8bc5a53640 1 -- 172.21.15.131:0/2625817533 wait complete. 2024-09-17T02:47:37.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.169+0000 7f8bc5a53640 1 Processor -- start 2024-09-17T02:47:37.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.169+0000 7f8bc5a53640 1 -- start start 2024-09-17T02:47:37.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.169+0000 7f8bc5a53640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8bc00d0620 0x7f8bc0140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.169+0000 7f8bc5a53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc015b7d0 0x7f8bc0155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.169+0000 7f8bc5a53640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc015bbe0 0x7f8bc01fa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.169+0000 7f8bc5a53640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0169ee0 con 0x7f8bc015b7d0 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.169+0000 7f8bc5a53640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0169d60 con 0x7f8bc015bbe0 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.169+0000 7f8bc5a53640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc016a060 con 0x7f8bc00d0620 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbe7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc015b7d0 0x7f8bc0155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbe7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc015b7d0 0x7f8bc0155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43608/0 (socket says 172.21.15.131:43608) 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbe7fc640 1 -- 172.21.15.131:0/3251720523 learned_addr learned my addr 172.21.15.131:0/3251720523 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbf7fe640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc015bbe0 0x7f8bc01fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbeffd640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8bc00d0620 0x7f8bc0140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbe7fc640 1 -- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8bc00d0620 msgr2=0x7f8bc0140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:37.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbe7fc640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8bc00d0620 0x7f8bc0140300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbe7fc640 1 -- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc015bbe0 msgr2=0x7f8bc01fa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:37.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbe7fc640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc015bbe0 0x7f8bc01fa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbe7fc640 1 -- 172.21.15.131:0/3251720523 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc0140b20 con 0x7f8bc015b7d0 2024-09-17T02:47:37.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbe7fc640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc015b7d0 0x7f8bc0155170 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400b410 tx=0x7f8bb400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:37.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8ba7fff640 1 -- 172.21.15.131:0/3251720523 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4015030 con 0x7f8bc015b7d0 2024-09-17T02:47:37.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8ba7fff640 1 -- 172.21.15.131:0/3251720523 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bb4004020 con 0x7f8bc015b7d0 2024-09-17T02:47:37.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8ba7fff640 1 -- 172.21.15.131:0/3251720523 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40044b0 con 0x7f8bc015b7d0 2024-09-17T02:47:37.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bc5a53640 1 -- 172.21.15.131:0/3251720523 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc00b3f70 con 0x7f8bc015b7d0 2024-09-17T02:47:37.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bc5a53640 1 -- 172.21.15.131:0/3251720523 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc0154970 con 0x7f8bc015b7d0 2024-09-17T02:47:37.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bc5a53640 1 -- 172.21.15.131:0/3251720523 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc0071210 con 0x7f8bc015b7d0 2024-09-17T02:47:37.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8ba7fff640 1 -- 172.21.15.131:0/3251720523 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8bb4018070 con 0x7f8bc015b7d0 2024-09-17T02:47:37.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8ba7fff640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bac073230 0x7f8bac075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:37.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8ba7fff640 1 -- 172.21.15.131:0/3251720523 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6283+0+0 (secure 0 0 0) 0x7f8bb4093ff0 con 0x7f8bc015b7d0 2024-09-17T02:47:37.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.173+0000 7f8bbeffd640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bac073230 0x7f8bac075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:37.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.177+0000 7f8bbeffd640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bac073230 0x7f8bac075740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00af000 tx=0x7f8bb001b090 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:37.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.177+0000 7f8ba7fff640 1 -- 172.21.15.131:0/3251720523 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8bb405edd0 con 0x7f8bc015b7d0 2024-09-17T02:47:37.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.377+0000 7f8bc5a53640 1 -- 172.21.15.131:0/3251720523 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8bc014b650 con 0x7f8bc015b7d0 2024-09-17T02:47:37.375 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.377+0000 7f8ba7fff640 1 -- 172.21.15.131:0/3251720523 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) ==== 53+0+73 (secure 0 0 0) 0x7f8bb405e6d0 con 0x7f8bc015b7d0 2024-09-17T02:47:37.376 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:47:37.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 -- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bac073230 msgr2=0x7f8bac075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:37.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bac073230 0x7f8bac075740 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00af000 tx=0x7f8bb001b090 comp rx=0 tx=0).stop 2024-09-17T02:47:37.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 -- 172.21.15.131:0/3251720523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc015b7d0 msgr2=0x7f8bc0155170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:37.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc015b7d0 0x7f8bc0155170 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400b410 tx=0x7f8bb400b8d0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 -- 172.21.15.131:0/3251720523 shutdown_connections 2024-09-17T02:47:37.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bac073230 0x7f8bac075740 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8bc015bbe0 0x7f8bc01fa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bc015b7d0 0x7f8bc0155170 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 --2- 172.21.15.131:0/3251720523 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8bc00d0620 0x7f8bc0140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:37.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 -- 172.21.15.131:0/3251720523 >> 172.21.15.131:0/3251720523 conn(0x7f8bc006fdb0 msgr2=0x7f8bc014e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:37.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 -- 172.21.15.131:0/3251720523 shutdown_connections 2024-09-17T02:47:37.379 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37.381+0000 7f8bc5a53640 1 -- 172.21.15.131:0/3251720523 wait complete. 2024-09-17T02:47:37.396 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:47:37.396 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:47:37.396 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:47:37.396 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:47:37.396 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:37.396 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:47:37.396 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:47:37.396 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:47:37.396 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:47:37.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:37.397 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:47:37.403 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:47:37.403 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:47:37.403 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:47:37.403 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:37.403 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:47:37.403 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:37.403 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:47:37.403 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:37.403 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:47:37.403 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42049sec preferred_lft 42049sec 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:47:37.404 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:37.405 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:47:37.406 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:37.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:37.406 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:47:37.457 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:47:37.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:37.458 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:47:37.509 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:47:37.509 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:47:37.509 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:47:37.509 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:47:37.509 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:47:37.509 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:47:37.510 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:47:37.510 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:47:37.510 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:47:37.510 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:47:37.510 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:47:37.511 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:47:37.511 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:47:37.511 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:47:37.511 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:47:37.511 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:47:37.511 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:47:37.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:47:37.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:47:37.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:37.630 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:47:37.678 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:47:37.678 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:47:37.678 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:47:37.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:37.678 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:47:37.734 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:47:37.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:47:37.734 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:37.782 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:47:37.782 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:47:37.782 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:37.829 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:47:37.887 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:47:37.936 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:47:37.936 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:47:37.936 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:47:37.936 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:47:37.936 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:47:37.936 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:47:37.936 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:47:37.936 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:47:37.936 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:47:37.936 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:47:37.937 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:47:37.937 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:47:37.937 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:47:37.937 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:47:37.937 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:47:37.937 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:47:37.937 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:47:37.937 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:47:37.938 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:47:37.938 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:47:37.938 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:47:37.938 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:47:37.938 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:47:37.938 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:47:37.938 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:47:37.938 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:47:37.938 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:47:37.939 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:47:37.939 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:47:37.939 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:37.939 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:37.939 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:37.939 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:37.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:37.940 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:47:37.985 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:47:37.986 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:47:38.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:47:38.042 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:47:38.090 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:47:38.090 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:47:38.091 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:47:38.092 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:47:38.093 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:47:38.093 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:38.093 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:38.093 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:38.093 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:38.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:38.093 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:47:38.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:47:38.141+0000 7f3bd3b68580 -1 init, newargv = 0x7f3bc40ad020 newargc=15 2024-09-17T02:47:38.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[46241]: starting ceph client 2024-09-17T02:47:38.143 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[46241]: starting fuse 2024-09-17T02:47:39.237 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:47:39.254 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:47:39.303 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:47:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:47:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:47:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:47:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:47:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:47:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:47:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:47:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:47:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:47:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:47:39.304 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:47:39.305 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:47:39.306 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:47:39.306 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:47:39.306 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:47:39.306 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:39.306 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:39.306 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:39.306 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:39.306 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:39.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:39.307 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:47:39.354 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:47:39.354 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:47:39.354 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:47:39.354 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:47:39.354 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:47:39.354 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:39.355 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:47:39.356 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:47:39.356 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:47:39.356 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:47:39.356 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:47:39.356 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:47:39.356 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:47:39.356 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:47:39.356 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:39.356 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:47:39.356 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:47:39.479 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.46241.asok 2024-09-17T02:47:39.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:47:39.491 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46241.asok 2024-09-17T02:47:39.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:39.491 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46241.asok status 2024-09-17T02:47:39.666 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:47:39.666 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:47:39.666 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:47:39.666 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:47:39.666 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:47:39.666 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "46241", 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7200, 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 7200 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:47:39.667 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 135919593 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 135919593 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.7200 192.168.0.1:0/135919593", 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/135919593", 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 64, 2024-09-17T02:47:39.668 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 94, 2024-09-17T02:47:39.669 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:47:39.669 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:47:39.669 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:47:39.669 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:47:39.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:39.684 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:47:39.684 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:39.690 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:47:39.691 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:39.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:39.691 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:39.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:39.745 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-09-17T02:47:39.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:39.802 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-09-17T02:47:39.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:39.860 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-09-17T02:47:39.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:39.916 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-09-17T02:47:39.974 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:47:39.974 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:47:39.974 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:47:39.974 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:39.974 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-09-17T02:47:39.974 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:47:40.080 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627776 2024-09-17T02:47:40.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:47:40.089 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:47:40.089 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:47:40.138 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:47:40.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.341+0000 7f1fec970640 1 -- 172.21.15.131:0/3082412493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8164310 msgr2=0x7f1fe8166730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:40.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.341+0000 7f1fec970640 1 --2- 172.21.15.131:0/3082412493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8164310 0x7f1fe8166730 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1fe815a780 tx=0x7f1fd801c850 comp rx=0 tx=0).stop 2024-09-17T02:47:40.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 -- 172.21.15.131:0/3082412493 shutdown_connections 2024-09-17T02:47:40.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 --2- 172.21.15.131:0/3082412493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8164310 0x7f1fe8166730 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 --2- 172.21.15.131:0/3082412493 >> v2:172.21.15.189:3300/0 conn(0x7f1fe80b48b0 0x7f1fe80cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 -- 172.21.15.131:0/3082412493 >> 172.21.15.131:0/3082412493 conn(0x7f1fe80cf150 msgr2=0x7f1fe80cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:40.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 -- 172.21.15.131:0/3082412493 shutdown_connections 2024-09-17T02:47:40.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 -- 172.21.15.131:0/3082412493 wait complete. 2024-09-17T02:47:40.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 Processor -- start 2024-09-17T02:47:40.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 -- start start 2024-09-17T02:47:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8156bd0 0x7f1fe81519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fe8164310 0x7f1fe8151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1fe8150f20 0x7f1fe81f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe8169820 con 0x7f1fe8156bd0 2024-09-17T02:47:40.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe81696a0 con 0x7f1fe8164310 2024-09-17T02:47:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fec970640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe81699a0 con 0x7f1fe8150f20 2024-09-17T02:47:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fe659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8156bd0 0x7f1fe81519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fe659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8156bd0 0x7f1fe81519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43626/0 (socket says 172.21.15.131:43626) 2024-09-17T02:47:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fe659c640 1 -- 172.21.15.131:0/1239486273 learned_addr learned my addr 172.21.15.131:0/1239486273 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fe6d9d640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1fe8150f20 0x7f1fe81f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fe5d9b640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fe8164310 0x7f1fe8151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:40.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fe659c640 1 -- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1fe8150f20 msgr2=0x7f1fe81f9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:40.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fe659c640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1fe8150f20 0x7f1fe81f9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fe659c640 1 -- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fe8164310 msgr2=0x7f1fe8151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:40.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.345+0000 7f1fe659c640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fe8164310 0x7f1fe8151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fe659c640 1 -- 172.21.15.131:0/1239486273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe815dd50 con 0x7f1fe8156bd0 2024-09-17T02:47:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fe659c640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8156bd0 0x7f1fe81519a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000ba80 tx=0x7f1fe000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fdeffd640 1 -- 172.21.15.131:0/1239486273 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe000cc00 con 0x7f1fe8156bd0 2024-09-17T02:47:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fdeffd640 1 -- 172.21.15.131:0/1239486273 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fe0003980 con 0x7f1fe8156bd0 2024-09-17T02:47:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fec970640 1 -- 172.21.15.131:0/1239486273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe81f74c0 con 0x7f1fe8156bd0 2024-09-17T02:47:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fdeffd640 1 -- 172.21.15.131:0/1239486273 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0003dd0 con 0x7f1fe8156bd0 2024-09-17T02:47:40.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fec970640 1 -- 172.21.15.131:0/1239486273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe81509a0 con 0x7f1fe8156bd0 2024-09-17T02:47:40.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fec970640 1 -- 172.21.15.131:0/1239486273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe8150790 con 0x7f1fe8156bd0 2024-09-17T02:47:40.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fdeffd640 1 -- 172.21.15.131:0/1239486273 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1fe0006d10 con 0x7f1fe8156bd0 2024-09-17T02:47:40.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fdeffd640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1fc8073230 0x7f1fc8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:40.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fdeffd640 1 -- 172.21.15.131:0/1239486273 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6283+0+0 (secure 0 0 0) 0x7f1fe00940d0 con 0x7f1fe8156bd0 2024-09-17T02:47:40.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fe5d9b640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1fc8073230 0x7f1fc8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:40.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.349+0000 7f1fe5d9b640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1fc8073230 0x7f1fc8075740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00058d0 tx=0x7f1fd0005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:40.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.353+0000 7f1fdeffd640 1 -- 172.21.15.131:0/1239486273 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1fe005eec0 con 0x7f1fe8156bd0 2024-09-17T02:47:40.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.533+0000 7f1fec970640 1 -- 172.21.15.131:0/1239486273 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1fe80cca70 con 0x7f1fe8156bd0 2024-09-17T02:47:40.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.533+0000 7f1fdeffd640 1 -- 172.21.15.131:0/1239486273 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v94) ==== 80+0+0 (secure 0 0 0) 0x7f1fe005e7c0 con 0x7f1fe8156bd0 2024-09-17T02:47:40.532 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:47:40.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 -- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1fc8073230 msgr2=0x7f1fc8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:40.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1fc8073230 0x7f1fc8075740 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1fd00058d0 tx=0x7f1fd0005860 comp rx=0 tx=0).stop 2024-09-17T02:47:40.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 -- 172.21.15.131:0/1239486273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8156bd0 msgr2=0x7f1fe81519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:40.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8156bd0 0x7f1fe81519a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1fe000ba80 tx=0x7f1fe000bf40 comp rx=0 tx=0).stop 2024-09-17T02:47:40.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 -- 172.21.15.131:0/1239486273 shutdown_connections 2024-09-17T02:47:40.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1fc8073230 0x7f1fc8075740 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1fe8150f20 0x7f1fe81f9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1fe8164310 0x7f1fe8151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 --2- 172.21.15.131:0/1239486273 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1fe8156bd0 0x7f1fe81519a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 -- 172.21.15.131:0/1239486273 >> 172.21.15.131:0/1239486273 conn(0x7f1fe80cf150 msgr2=0x7f1fe80701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:40.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 -- 172.21.15.131:0/1239486273 shutdown_connections 2024-09-17T02:47:40.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.537+0000 7f1fec970640 1 -- 172.21.15.131:0/1239486273 wait complete. 2024-09-17T02:47:40.553 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:47:40.553 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:47:40.559 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:47:40.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.757+0000 7fe650ff8640 1 -- 172.21.15.131:0/1941793848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64c06e3e0 msgr2=0x7fe64c15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:40.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.757+0000 7fe650ff8640 1 --2- 172.21.15.131:0/1941793848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64c06e3e0 0x7fe64c15bea0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fe64c06a1c0 tx=0x7fe64001c6e0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.757+0000 7fe650ff8640 1 -- 172.21.15.131:0/1941793848 shutdown_connections 2024-09-17T02:47:40.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.757+0000 7fe650ff8640 1 --2- 172.21.15.131:0/1941793848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64c06e3e0 0x7fe64c15bea0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.757+0000 7fe650ff8640 1 --2- 172.21.15.131:0/1941793848 >> v2:172.21.15.189:3300/0 conn(0x7fe64c0ced00 0x7fe64c06dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.757+0000 7fe650ff8640 1 -- 172.21.15.131:0/1941793848 >> 172.21.15.131:0/1941793848 conn(0x7fe64c06eea0 msgr2=0x7fe64c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:40.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.757+0000 7fe650ff8640 1 -- 172.21.15.131:0/1941793848 shutdown_connections 2024-09-17T02:47:40.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.757+0000 7fe650ff8640 1 -- 172.21.15.131:0/1941793848 wait complete. 2024-09-17T02:47:40.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe650ff8640 1 Processor -- start 2024-09-17T02:47:40.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe650ff8640 1 -- start start 2024-09-17T02:47:40.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe650ff8640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe64c06e3e0 0x7fe64c151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:40.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe650ff8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64c0ced00 0x7fe64c1517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:40.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe650ff8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c15c3e0 0x7fe64c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:40.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe650ff8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c168bf0 con 0x7fe64c0ced00 2024-09-17T02:47:40.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe650ff8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c0cf300 con 0x7fe64c15c3e0 2024-09-17T02:47:40.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe650ff8640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe64c168d70 con 0x7fe64c06e3e0 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe649d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64c0ced00 0x7fe64c1517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe649d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64c0ced00 0x7fe64c1517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43644/0 (socket says 172.21.15.131:43644) 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe649d9b640 1 -- 172.21.15.131:0/4005349335 learned_addr learned my addr 172.21.15.131:0/4005349335 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe64a59c640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe64c06e3e0 0x7fe64c151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe64ad9d640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c15c3e0 0x7fe64c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe649d9b640 1 -- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe64c06e3e0 msgr2=0x7fe64c151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe649d9b640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe64c06e3e0 0x7fe64c151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe649d9b640 1 -- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c15c3e0 msgr2=0x7fe64c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe649d9b640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c15c3e0 0x7fe64c0cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe649d9b640 1 -- 172.21.15.131:0/4005349335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c14f240 con 0x7fe64c0ced00 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe649d9b640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64c0ced00 0x7fe64c1517a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe640009850 tx=0x7fe6400a6590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe63affd640 1 -- 172.21.15.131:0/4005349335 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6400ac070 con 0x7fe64c0ced00 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe63affd640 1 -- 172.21.15.131:0/4005349335 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6400054b0 con 0x7fe64c0ced00 2024-09-17T02:47:40.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe650ff8640 1 -- 172.21.15.131:0/4005349335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c146df0 con 0x7fe64c0ced00 2024-09-17T02:47:40.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe63affd640 1 -- 172.21.15.131:0/4005349335 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe640004290 con 0x7fe64c0ced00 2024-09-17T02:47:40.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.761+0000 7fe650ff8640 1 -- 172.21.15.131:0/4005349335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c14f7b0 con 0x7fe64c0ced00 2024-09-17T02:47:40.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.765+0000 7fe650ff8640 1 -- 172.21.15.131:0/4005349335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64c075b80 con 0x7fe64c0ced00 2024-09-17T02:47:40.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.765+0000 7fe63affd640 1 -- 172.21.15.131:0/4005349335 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe6400af070 con 0x7fe64c0ced00 2024-09-17T02:47:40.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.765+0000 7fe63affd640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe62c073200 0x7fe62c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:40.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.765+0000 7fe63affd640 1 -- 172.21.15.131:0/4005349335 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6283+0+0 (secure 0 0 0) 0x7fe64012e5c0 con 0x7fe64c0ced00 2024-09-17T02:47:40.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.765+0000 7fe64a59c640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe62c073200 0x7fe62c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:40.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.765+0000 7fe64a59c640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe62c073200 0x7fe62c075710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe63c006fd0 tx=0x7fe63c008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:40.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.769+0000 7fe63affd640 1 -- 172.21.15.131:0/4005349335 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe64c06e7f0 con 0x7fe64c0ced00 2024-09-17T02:47:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.945+0000 7fe650ff8640 1 -- 172.21.15.131:0/4005349335 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe64c146ff0 con 0x7fe64c0ced00 2024-09-17T02:47:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.945+0000 7fe63affd640 1 -- 172.21.15.131:0/4005349335 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v94) ==== 80+0+0 (secure 0 0 0) 0x7fe6400f93b0 con 0x7fe64c0ced00 2024-09-17T02:47:40.943 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:47:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 -- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe62c073200 msgr2=0x7fe62c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe62c073200 0x7fe62c075710 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe63c006fd0 tx=0x7fe63c008040 comp rx=0 tx=0).stop 2024-09-17T02:47:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 -- 172.21.15.131:0/4005349335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64c0ced00 msgr2=0x7fe64c1517a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64c0ced00 0x7fe64c1517a0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe640009850 tx=0x7fe6400a6590 comp rx=0 tx=0).stop 2024-09-17T02:47:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 -- 172.21.15.131:0/4005349335 shutdown_connections 2024-09-17T02:47:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe62c073200 0x7fe62c075710 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe64c15c3e0 0x7fe64c0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe64c0ced00 0x7fe64c1517a0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 --2- 172.21.15.131:0/4005349335 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe64c06e3e0 0x7fe64c151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 -- 172.21.15.131:0/4005349335 >> 172.21.15.131:0/4005349335 conn(0x7fe64c06eea0 msgr2=0x7fe64c155370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 -- 172.21.15.131:0/4005349335 shutdown_connections 2024-09-17T02:47:40.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:40.949+0000 7fe650ff8640 1 -- 172.21.15.131:0/4005349335 wait complete. 2024-09-17T02:47:40.965 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:47:40.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:40.965 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:40.976 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[46241]: fuse finished with error 0 and tester_r 0 2024-09-17T02:47:40.978 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:47:44.794 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:47:44.797 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:47:47.080 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:47:47.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:47.080 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:47:47.087 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:47:47.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.297+0000 7fba857a1640 1 -- 172.21.15.131:0/3606191082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba68004810 msgr2=0x7fba68004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:47.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.297+0000 7fba857a1640 1 --2- 172.21.15.131:0/3606191082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba68004810 0x7fba68004c20 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fba74006fd0 tx=0x7fba7401cf00 comp rx=0 tx=0).stop 2024-09-17T02:47:47.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.297+0000 7fba857a1640 1 -- 172.21.15.131:0/3606191082 shutdown_connections 2024-09-17T02:47:47.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.297+0000 7fba857a1640 1 --2- 172.21.15.131:0/3606191082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba68004810 0x7fba68004c20 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.297+0000 7fba857a1640 1 -- 172.21.15.131:0/3606191082 >> 172.21.15.131:0/3606191082 conn(0x7fba8006eea0 msgr2=0x7fba8006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:47.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.297+0000 7fba857a1640 1 -- 172.21.15.131:0/3606191082 shutdown_connections 2024-09-17T02:47:47.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.297+0000 7fba857a1640 1 -- 172.21.15.131:0/3606191082 wait complete. 2024-09-17T02:47:47.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba857a1640 1 Processor -- start 2024-09-17T02:47:47.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba857a1640 1 -- start start 2024-09-17T02:47:47.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba857a1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68004810 0x7fba80149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba857a1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba80145d00 0x7fba801f9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba857a1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba801fa370 0x7fba80156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba857a1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fba800c4f10 con 0x7fba80145d00 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba857a1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fba800c4d90 con 0x7fba68004810 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba857a1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fba800c5090 con 0x7fba801fa370 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba7e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba80145d00 0x7fba801f9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba7e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba80145d00 0x7fba801f9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54772/0 (socket says 172.21.15.131:54772) 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba7e7fc640 1 -- 172.21.15.131:0/2313668087 learned_addr learned my addr 172.21.15.131:0/2313668087 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba7effd640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68004810 0x7fba80149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba7f7fe640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba801fa370 0x7fba80156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba7e7fc640 1 -- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba801fa370 msgr2=0x7fba80156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:47.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba7e7fc640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba801fa370 0x7fba80156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba7e7fc640 1 -- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68004810 msgr2=0x7fba80149030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba7e7fc640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68004810 0x7fba80149030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba7e7fc640 1 -- 172.21.15.131:0/2313668087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8015da70 con 0x7fba80145d00 2024-09-17T02:47:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba7e7fc640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba80145d00 0x7fba801f9e30 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fba74000c10 tx=0x7fba74006cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba677fe640 1 -- 172.21.15.131:0/2313668087 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba740a5b60 con 0x7fba80145d00 2024-09-17T02:47:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba677fe640 1 -- 172.21.15.131:0/2313668087 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba740014c0 con 0x7fba80145d00 2024-09-17T02:47:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba857a1640 1 -- 172.21.15.131:0/2313668087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba801554a0 con 0x7fba80145d00 2024-09-17T02:47:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba677fe640 1 -- 172.21.15.131:0/2313668087 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba740060d0 con 0x7fba80145d00 2024-09-17T02:47:47.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.301+0000 7fba857a1640 1 -- 172.21.15.131:0/2313668087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8015d4d0 con 0x7fba80145d00 2024-09-17T02:47:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.305+0000 7fba857a1640 1 -- 172.21.15.131:0/2313668087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba800cde70 con 0x7fba80145d00 2024-09-17T02:47:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.305+0000 7fba677fe640 1 -- 172.21.15.131:0/2313668087 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fba74006270 con 0x7fba80145d00 2024-09-17T02:47:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.305+0000 7fba677fe640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba6c073230 0x7fba6c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.305+0000 7fba677fe640 1 -- 172.21.15.131:0/2313668087 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6283+0+0 (secure 0 0 0) 0x7fba7412dfc0 con 0x7fba80145d00 2024-09-17T02:47:47.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.305+0000 7fba7effd640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba6c073230 0x7fba6c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:47.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.305+0000 7fba7effd640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba6c073230 0x7fba6c075740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fba70000c10 tx=0x7fba70009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:47.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.309+0000 7fba677fe640 1 -- 172.21.15.131:0/2313668087 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fba74016cb0 con 0x7fba80145d00 2024-09-17T02:47:47.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.509+0000 7fba857a1640 1 -- 172.21.15.131:0/2313668087 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba80073310 con 0x7fba80145d00 2024-09-17T02:47:47.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba677fe640 1 -- 172.21.15.131:0/2313668087 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+5237 (secure 0 0 0) 0x7fba740f8e30 con 0x7fba80145d00 2024-09-17T02:47:47.510 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:47.510 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":64,"btime":"2024-09-17T02:47:29:415704+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:47:28.826817+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"c","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2766162522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":2766162522},{"type":"v1","addr":"172.21.15.131:6835","nonce":2766162522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7038,"qdb_cluster":[7038]},"id":6}]} 2024-09-17T02:47:47.510 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 64 2024-09-17T02:47:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 -- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba6c073230 msgr2=0x7fba6c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:47.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba6c073230 0x7fba6c075740 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fba70000c10 tx=0x7fba70009040 comp rx=0 tx=0).stop 2024-09-17T02:47:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 -- 172.21.15.131:0/2313668087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba80145d00 msgr2=0x7fba801f9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba80145d00 0x7fba801f9e30 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fba74000c10 tx=0x7fba74006cb0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 -- 172.21.15.131:0/2313668087 shutdown_connections 2024-09-17T02:47:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba6c073230 0x7fba6c075740 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba801fa370 0x7fba80156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba80145d00 0x7fba801f9e30 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 --2- 172.21.15.131:0/2313668087 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba68004810 0x7fba80149030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 -- 172.21.15.131:0/2313668087 >> 172.21.15.131:0/2313668087 conn(0x7fba8006eea0 msgr2=0x7fba800b5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 -- 172.21.15.131:0/2313668087 shutdown_connections 2024-09-17T02:47:47.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.513+0000 7fba857a1640 1 -- 172.21.15.131:0/2313668087 wait complete. 2024-09-17T02:47:47.531 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-17T02:47:47.717 INFO:teuthology.orchestra.run.smithi131.stdout:{"message":"","return_code":0} 2024-09-17T02:47:47.717 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T02:47:47.718 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:47:47.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 -- 172.21.15.131:0/1943065691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6b4005360 msgr2=0x7fd6b400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:47.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 --2- 172.21.15.131:0/1943065691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6b4005360 0x7fd6b400b9d0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80016f0 tx=0x7fd6b801c7b0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 -- 172.21.15.131:0/1943065691 shutdown_connections 2024-09-17T02:47:47.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 --2- 172.21.15.131:0/1943065691 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6b4005360 0x7fd6b400b9d0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 -- 172.21.15.131:0/1943065691 >> 172.21.15.131:0/1943065691 conn(0x7fd6c806eea0 msgr2=0x7fd6c806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:47.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 -- 172.21.15.131:0/1943065691 shutdown_connections 2024-09-17T02:47:47.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 -- 172.21.15.131:0/1943065691 wait complete. 2024-09-17T02:47:47.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 Processor -- start 2024-09-17T02:47:47.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 -- start start 2024-09-17T02:47:47.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6b4005360 0x7fd6c80c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:47.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c81ef0f0 0x7fd6c81f8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:47.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd6c81ef500 0x7fd6c81f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:47.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c815cbb0 con 0x7fd6c81ef0f0 2024-09-17T02:47:47.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c815ca30 con 0x7fd6b4005360 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6ce204640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c80cf010 con 0x7fd6c81ef500 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6c6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c81ef0f0 0x7fd6c81f8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6c6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c81ef0f0 0x7fd6c81f8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54790/0 (socket says 172.21.15.131:54790) 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6c6ffd640 1 -- 172.21.15.131:0/4113924477 learned_addr learned my addr 172.21.15.131:0/4113924477 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.877+0000 7fd6c77fe640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6b4005360 0x7fd6c80c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6c7fff640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd6c81ef500 0x7fd6c81f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6c6ffd640 1 -- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd6c81ef500 msgr2=0x7fd6c81f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6c6ffd640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd6c81ef500 0x7fd6c81f8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6c6ffd640 1 -- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6b4005360 msgr2=0x7fd6c80c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6c6ffd640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6b4005360 0x7fd6c80c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6c6ffd640 1 -- 172.21.15.131:0/4113924477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c80cada0 con 0x7fd6c81ef0f0 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6c77fe640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6b4005360 0x7fd6c80c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:47:47.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6c6ffd640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c81ef0f0 0x7fd6c81f8930 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0010a10 tx=0x7fd6b0010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:47.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6bffff640 1 -- 172.21.15.131:0/4113924477 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b000ece0 con 0x7fd6c81ef0f0 2024-09-17T02:47:47.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6bffff640 1 -- 172.21.15.131:0/4113924477 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6b000ee80 con 0x7fd6c81ef0f0 2024-09-17T02:47:47.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6ce204640 1 -- 172.21.15.131:0/4113924477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c80c79d0 con 0x7fd6c81ef0f0 2024-09-17T02:47:47.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6bffff640 1 -- 172.21.15.131:0/4113924477 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0005260 con 0x7fd6c81ef0f0 2024-09-17T02:47:47.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6ce204640 1 -- 172.21.15.131:0/4113924477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c80c7f90 con 0x7fd6c81ef0f0 2024-09-17T02:47:47.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6ce204640 1 -- 172.21.15.131:0/4113924477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c80c4580 con 0x7fd6c81ef0f0 2024-09-17T02:47:47.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6bffff640 1 -- 172.21.15.131:0/4113924477 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd6b0005400 con 0x7fd6c81ef0f0 2024-09-17T02:47:47.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6bffff640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6a8073200 0x7fd6a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:47.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6bffff640 1 -- 172.21.15.131:0/4113924477 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6283+0+0 (secure 0 0 0) 0x7fd6b0095370 con 0x7fd6c81ef0f0 2024-09-17T02:47:47.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.881+0000 7fd6c77fe640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6a8073200 0x7fd6a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:47.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.885+0000 7fd6c77fe640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6a8073200 0x7fd6a8075710 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0001340 tx=0x7fd6c000c040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:47.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:47.885+0000 7fd6bffff640 1 -- 172.21.15.131:0/4113924477 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd6b0060150 con 0x7fd6c81ef0f0 2024-09-17T02:47:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.089+0000 7fd6ce204640 1 -- 172.21.15.131:0/4113924477 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd6c8073310 con 0x7fd6c81ef0f0 2024-09-17T02:47:48.087 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:47:48.089+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:47:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.093+0000 7fd6bffff640 1 -- 172.21.15.131:0/4113924477 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) ==== 161+0+0 (secure 0 0 0) 0x7fd6b005fa50 con 0x7fd6c81ef0f0 2024-09-17T02:47:48.091 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:47:48.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 -- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6a8073200 msgr2=0x7fd6a8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6a8073200 0x7fd6a8075710 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0001340 tx=0x7fd6c000c040 comp rx=0 tx=0).stop 2024-09-17T02:47:48.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 -- 172.21.15.131:0/4113924477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c81ef0f0 msgr2=0x7fd6c81f8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c81ef0f0 0x7fd6c81f8930 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd6b0010a10 tx=0x7fd6b0010ed0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 -- 172.21.15.131:0/4113924477 shutdown_connections 2024-09-17T02:47:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd6a8073200 0x7fd6a8075710 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd6c81ef500 0x7fd6c81f8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd6c81ef0f0 0x7fd6c81f8930 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 --2- 172.21.15.131:0/4113924477 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd6b4005360 0x7fd6c80c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 -- 172.21.15.131:0/4113924477 >> 172.21.15.131:0/4113924477 conn(0x7fd6c806eea0 msgr2=0x7fd6c80cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:48.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 -- 172.21.15.131:0/4113924477 shutdown_connections 2024-09-17T02:47:48.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.097+0000 7fd6ce204640 1 -- 172.21.15.131:0/4113924477 wait complete. 2024-09-17T02:47:48.099 INFO:tasks.ceph.mds.c.smithi131.stderr: -4657> 2024-09-17T02:47:31.353+0000 7fa75fc2c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:47:48.119 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-09-17T02:47:48.162 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:47:48.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc18fa0b080 1 Processor -- start 2024-09-17T02:47:48.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc18fa0b080 1 -- start start 2024-09-17T02:47:48.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc18fa0b080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc174003c50 0x56521cae1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc18fa0b080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 0x56521ca4c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc18fa0b080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56521ca37230 0x56521ca37640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc18fa0b080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56521c9b9f40 con 0x56521ca4c560 2024-09-17T02:47:48.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc18fa0b080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56521c9c6d00 con 0x56521ca37230 2024-09-17T02:47:48.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc18fa0b080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56521c95ba00 con 0x7fc174003c50 2024-09-17T02:47:48.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc188ce1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 0x56521ca4c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc188ce1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 0x56521ca4c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54826/0 (socket says 172.21.15.131:54826) 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc188ce1640 1 -- 172.21.15.131:0/2788653726 learned_addr learned my addr 172.21.15.131:0/2788653726 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc188ce1640 1 -- 172.21.15.131:0/2788653726 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc174003c50 msgr2=0x56521cae1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc188ce1640 1 --2- 172.21.15.131:0/2788653726 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc174003c50 0x56521cae1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc188ce1640 1 -- 172.21.15.131:0/2788653726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56521ca37230 msgr2=0x56521ca37640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc188ce1640 1 --2- 172.21.15.131:0/2788653726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56521ca37230 0x56521ca37640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc188ce1640 1 -- 172.21.15.131:0/2788653726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521cac4930 con 0x56521ca4c560 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc188ce1640 1 --2- 172.21.15.131:0/2788653726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 0x56521ca4c970 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc18000b300 tx=0x7fc18000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dbc3cec6b9dee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc17b7fe640 1 -- 172.21.15.131:0/2788653726 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180012020 con 0x56521ca4c560 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc17b7fe640 1 -- 172.21.15.131:0/2788653726 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc18000e950 con 0x56521ca4c560 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.181+0000 7fc17b7fe640 1 -- 172.21.15.131:0/2788653726 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18000eda0 con 0x56521ca4c560 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 -- 172.21.15.131:0/2788653726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 msgr2=0x56521ca4c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 --2- 172.21.15.131:0/2788653726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 0x56521ca4c970 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fc18000b300 tx=0x7fc18000b7c0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 -- 172.21.15.131:0/2788653726 shutdown_connections 2024-09-17T02:47:48.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 --2- 172.21.15.131:0/2788653726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56521ca37230 0x56521ca37640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 --2- 172.21.15.131:0/2788653726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 0x56521ca4c970 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 --2- 172.21.15.131:0/2788653726 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc174003c50 0x56521cae1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 -- 172.21.15.131:0/2788653726 >> 172.21.15.131:0/2788653726 conn(0x56521c9f9d00 msgr2=0x56521ca3afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:48.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 -- 172.21.15.131:0/2788653726 shutdown_connections 2024-09-17T02:47:48.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 -- 172.21.15.131:0/2788653726 wait complete. 2024-09-17T02:47:48.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 Processor -- start 2024-09-17T02:47:48.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 -- start start 2024-09-17T02:47:48.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56521ca37230 0x56521cb7e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 0x56521ca365e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc174003c50 0x56521ca37640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56521cae1750 con 0x56521ca4c560 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56521cae14b0 con 0x7fc174003c50 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56521cae19f0 con 0x56521ca37230 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc188ce1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 0x56521ca365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc188ce1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 0x56521ca365e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54832/0 (socket says 172.21.15.131:54832) 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc188ce1640 1 -- 172.21.15.131:0/1081405929 learned_addr learned my addr 172.21.15.131:0/1081405929 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc1894e2640 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56521ca37230 0x56521cb7e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc188ce1640 1 -- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56521ca37230 msgr2=0x56521cb7e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc188ce1640 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56521ca37230 0x56521cb7e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc189ce3640 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc174003c50 0x56521ca37640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc188ce1640 1 -- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc174003c50 msgr2=0x56521ca37640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc188ce1640 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc174003c50 0x56521ca37640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc188ce1640 1 -- 172.21.15.131:0/1081405929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521cb7b070 con 0x56521ca4c560 2024-09-17T02:47:48.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc188ce1640 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 0x56521ca365e0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc180018a80 tx=0x7fc180018ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc17a7fc640 1 -- 172.21.15.131:0/1081405929 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1800165c0 con 0x56521ca4c560 2024-09-17T02:47:48.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc17a7fc640 1 -- 172.21.15.131:0/1081405929 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc180016760 con 0x56521ca4c560 2024-09-17T02:47:48.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 -- 172.21.15.131:0/1081405929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56521cb7daa0 con 0x56521ca4c560 2024-09-17T02:47:48.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc17a7fc640 1 -- 172.21.15.131:0/1081405929 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180016b90 con 0x56521ca4c560 2024-09-17T02:47:48.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.185+0000 7fc18fa0b080 1 -- 172.21.15.131:0/1081405929 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521cb7dfe0 con 0x56521ca4c560 2024-09-17T02:47:48.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.189+0000 7fc17a7fc640 1 -- 172.21.15.131:0/1081405929 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc180015020 con 0x56521ca4c560 2024-09-17T02:47:48.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.189+0000 7fc17a7fc640 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc16c071be0 0x7fc16c0740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.189+0000 7fc17a7fc640 1 -- 172.21.15.131:0/1081405929 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fc180094fb0 con 0x56521ca4c560 2024-09-17T02:47:48.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.189+0000 7fc18fa0b080 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x56521cb7a8b0 0x56521ca4ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.189+0000 7fc18fa0b080 1 -- 172.21.15.131:0/1081405929 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e95) -- 0x56521ca4cfd0 con 0x56521cb7a8b0 2024-09-17T02:47:48.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.189+0000 7fc1894e2640 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc16c071be0 0x7fc16c0740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.189+0000 7fc189ce3640 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x56521cb7a8b0 0x56521ca4ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.189+0000 7fc1894e2640 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc16c071be0 0x7fc16c0740f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc17c004fd0 tx=0x7fc17c004630 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.189+0000 7fc189ce3640 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x56521cb7a8b0 0x56521ca4ca90 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc189ce3640 1 -- 172.21.15.131:0/1081405929 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v95'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc184001880 con 0x56521cb7a8b0 2024-09-17T02:47:48.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc18fa0b080 1 -- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x56521cb7a8b0 msgr2=0x56521ca4ca90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc18fa0b080 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x56521cb7a8b0 0x56521ca4ca90 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc18fa0b080 1 -- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc16c071be0 msgr2=0x7fc16c0740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc18fa0b080 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc16c071be0 0x7fc16c0740f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc17c004fd0 tx=0x7fc17c004630 comp rx=0 tx=0).stop 2024-09-17T02:47:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc18fa0b080 1 -- 172.21.15.131:0/1081405929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 msgr2=0x56521ca365e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc18fa0b080 1 --2- 172.21.15.131:0/1081405929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56521ca4c560 0x56521ca365e0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc180018a80 tx=0x7fc180018ab0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc189ce3640 1 -- 172.21.15.131:0/1081405929 reap_dead start 2024-09-17T02:47:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc18fa0b080 1 -- 172.21.15.131:0/1081405929 shutdown_connections 2024-09-17T02:47:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc18fa0b080 1 -- 172.21.15.131:0/1081405929 >> 172.21.15.131:0/1081405929 conn(0x56521c9f9d00 msgr2=0x56521cac3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc18fa0b080 1 -- 172.21.15.131:0/1081405929 shutdown_connections 2024-09-17T02:47:48.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.197+0000 7fc18fa0b080 1 -- 172.21.15.131:0/1081405929 wait complete. 2024-09-17T02:47:48.202 DEBUG:teuthology.orchestra.run.smithi131:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --yes-i-really-really-mean-it 2024-09-17T02:47:48.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 1 -- start start 2024-09-17T02:47:48.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 1 Processor -- start 2024-09-17T02:47:48.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e3680 0x5600e0578580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e3b00 0x5600e0578000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e4400 0x5600e0578b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5600e131e180 con 0x5600e04e3b00 2024-09-17T02:47:48.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5600e131e300 con 0x5600e04e3680 2024-09-17T02:47:48.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5600e131e480 con 0x5600e04e4400 2024-09-17T02:47:48.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6cc80640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e3b00 0x5600e0578000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6cc80640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e3b00 0x5600e0578000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54854/0 (socket says 172.21.15.131:54854) 2024-09-17T02:47:48.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6cc80640 1 -- 172.21.15.131:0/3073932736 learned_addr learned my addr 172.21.15.131:0/3073932736 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:48.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6d481640 1 --2- 172.21.15.131:0/3073932736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e3680 0x5600e0578580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/3073932736 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e4400 0x5600e0578b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6cc80640 1 -- 172.21.15.131:0/3073932736 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e4400 msgr2=0x5600e0578b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:48.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/3073932736 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e4400 0x5600e0578b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6cc80640 1 -- 172.21.15.131:0/3073932736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e3680 msgr2=0x5600e0578580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:48.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/3073932736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e3680 0x5600e0578580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6cc80640 1 -- 172.21.15.131:0/3073932736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e0518b40 con 0x5600e04e3b00 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6d481640 1 --2- 172.21.15.131:0/3073932736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e3680 0x5600e0578580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/3073932736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e3b00 0x5600e0578000 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x5600e1265d40 tx=0x5600e132a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab18f07f385c387 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ac7c640 1 -- 172.21.15.131:0/3073932736 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5600e0437520 con 0x5600e04e3b00 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ac7c640 1 -- 172.21.15.131:0/3073932736 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5600e0454540 con 0x5600e04e3b00 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/3073932736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e0518d20 con 0x5600e04e3b00 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ac7c640 1 -- 172.21.15.131:0/3073932736 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5600e04371e0 con 0x5600e04e3b00 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/3073932736 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5600e0518f00 con 0x5600e04e3b00 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 4 waiting for MDS map... 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ac7c640 1 -- 172.21.15.131:0/3073932736 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x5600e0581000 con 0x5600e04e3b00 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ac7c640 1 -- 172.21.15.131:0/3073932736 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 65) ==== 2893+0+0 (secure 0 0 0) 0x5600e0457180 con 0x5600e04e3b00 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 4 Got MDS map 65 2024-09-17T02:47:48.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 10 main: JournalTool::main 2024-09-17T02:47:48.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.309+0000 7fcc6ecc2400 4 main: JournalTool: connecting to RADOS... 2024-09-17T02:47:48.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 Processor -- start 2024-09-17T02:47:48.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 -- start start 2024-09-17T02:47:48.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e2d80 0x5600e0579080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e2900 0x5600e0579600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2480 0x5600e0579b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5600e131e600 con 0x5600e04e2480 2024-09-17T02:47:48.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5600e131e780 con 0x5600e04e2900 2024-09-17T02:47:48.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5600e131e900 con 0x5600e04e2d80 2024-09-17T02:47:48.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6cc80640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2480 0x5600e0579b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6cc80640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2480 0x5600e0579b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54868/0 (socket says 172.21.15.131:54868) 2024-09-17T02:47:48.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6cc80640 1 -- 172.21.15.131:0/2318168814 learned_addr learned my addr 172.21.15.131:0/2318168814 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:48.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/2318168814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e2d80 0x5600e0579080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6cc80640 1 -- 172.21.15.131:0/2318168814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e2d80 msgr2=0x5600e0579080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:48.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/2318168814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e2d80 0x5600e0579080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6cc80640 1 -- 172.21.15.131:0/2318168814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e2900 msgr2=0x5600e0579600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:48.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/2318168814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e2900 0x5600e0579600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6cc80640 1 -- 172.21.15.131:0/2318168814 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e0518d20 con 0x5600e04e2480 2024-09-17T02:47:48.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/2318168814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2480 0x5600e0579b80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x5600e133f0e0 tx=0x5600e132a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111d1b1757472e80 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc68c78640 1 -- 172.21.15.131:0/2318168814 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5600e0437040 con 0x5600e04e2480 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc68c78640 1 -- 172.21.15.131:0/2318168814 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5600e0454fc0 con 0x5600e04e2480 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc68c78640 1 -- 172.21.15.131:0/2318168814 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5600e0437520 con 0x5600e04e2480 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/2318168814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2480 msgr2=0x5600e0579b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/2318168814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2480 0x5600e0579b80 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x5600e133f0e0 tx=0x5600e132a420 comp rx=0 tx=0).stop 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/2318168814 shutdown_connections 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/2318168814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2480 0x5600e0579b80 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/2318168814 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e2900 0x5600e0579600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/2318168814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e2d80 0x5600e0579080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/2318168814 >> 172.21.15.131:0/2318168814 conn(0x5600e04e3200 msgr2=0x5600e0736000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/2318168814 shutdown_connections 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.313+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/2318168814 wait complete. 2024-09-17T02:47:48.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6ecc2400 1 Processor -- start 2024-09-17T02:47:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6ecc2400 1 -- start start 2024-09-17T02:47:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e2480 0x5600e0579b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e2900 0x5600e0579600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2d80 0x5600e0579080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5600e131fc80 con 0x5600e04e2d80 2024-09-17T02:47:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5600e131fb00 con 0x5600e04e2480 2024-09-17T02:47:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5600e131f980 con 0x5600e04e2900 2024-09-17T02:47:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6cc80640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2d80 0x5600e0579080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6cc80640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2d80 0x5600e0579080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54876/0 (socket says 172.21.15.131:54876) 2024-09-17T02:47:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6cc80640 1 -- 172.21.15.131:0/629834433 learned_addr learned my addr 172.21.15.131:0/629834433 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:48.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e2480 0x5600e0579b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6d481640 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e2900 0x5600e0579600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6cc80640 1 -- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e2900 msgr2=0x5600e0579600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e2900 0x5600e0579600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6cc80640 1 -- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e2480 msgr2=0x5600e0579b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e2480 0x5600e0579b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6cc80640 1 -- 172.21.15.131:0/629834433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e0519c20 con 0x5600e04e2d80 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6d481640 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e2900 0x5600e0579600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2d80 0x5600e0579080 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x5600e133fda0 tx=0x5600e1350cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc67475640 1 -- 172.21.15.131:0/629834433 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5600e04371e0 con 0x5600e04e2d80 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc67475640 1 -- 172.21.15.131:0/629834433 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5600e0454e00 con 0x5600e04e2d80 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/629834433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5600e0519e00 con 0x5600e04e2d80 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc67475640 1 -- 172.21.15.131:0/629834433 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5600e0436820 con 0x5600e04e2d80 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/629834433 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e0518d20 con 0x5600e04e2d80 2024-09-17T02:47:48.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc6ecc2400 4 main: JournalTool: resolving pool 13 2024-09-17T02:47:48.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc67475640 1 -- 172.21.15.131:0/629834433 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x5600e074cc00 con 0x5600e04e2d80 2024-09-17T02:47:48.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc67475640 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x5600e04e5180 0x5600e057a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.317+0000 7fcc67475640 1 -- 172.21.15.131:0/629834433 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x5600e074e600 con 0x5600e04e2d80 2024-09-17T02:47:48.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 4 main: JournalTool: creating IoCtx.. 2024-09-17T02:47:48.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 4 main: Executing for rank 0 2024-09-17T02:47:48.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x5600e04e5180 0x5600e057a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 -- start start 2024-09-17T02:47:48.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 Processor -- start 2024-09-17T02:47:48.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x5600e04e5180 0x5600e057a100 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x5600e051e6c0 tx=0x5600e051d920 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e4880 0x5600e057a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e4d00 0x5600e057ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e5600 0x5600e057b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5600e131e780 con 0x5600e04e5600 2024-09-17T02:47:48.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5600e131e900 con 0x5600e04e4880 2024-09-17T02:47:48.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5600e131f680 con 0x5600e04e4d00 2024-09-17T02:47:48.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6cc80640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e5600 0x5600e057b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6cc80640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e5600 0x5600e057b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54880/0 (socket says 172.21.15.131:54880) 2024-09-17T02:47:48.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6cc80640 1 -- 172.21.15.131:0/4028772032 learned_addr learned my addr 172.21.15.131:0/4028772032 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:48.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6d481640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e4d00 0x5600e057ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e4880 0x5600e057a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6cc80640 1 -- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e4d00 msgr2=0x5600e057ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:48.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e4d00 0x5600e057ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6cc80640 1 -- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e4880 msgr2=0x5600e057a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:48.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e4880 0x5600e057a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6cc80640 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e08a0780 con 0x5600e04e5600 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e5600 0x5600e057b180 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x5600e08ac000 tx=0x5600e1351740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ff910c431ac256 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6446f640 1 -- 172.21.15.131:0/4028772032 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5600e07e16c0 con 0x5600e04e5600 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6446f640 1 -- 172.21.15.131:0/4028772032 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5600e0454c40 con 0x5600e04e5600 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6446f640 1 -- 172.21.15.131:0/4028772032 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5600e07e1860 con 0x5600e04e5600 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e08a0960 con 0x5600e04e5600 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5600e08a0b40 con 0x5600e04e5600 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 4 waiting for MDS map... 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6446f640 1 -- 172.21.15.131:0/4028772032 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x5600e08aa400 con 0x5600e04e5600 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6446f640 1 -- 172.21.15.131:0/4028772032 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 65) ==== 2893+0+0 (secure 0 0 0) 0x5600e0457500 con 0x5600e04e5600 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 4 Got MDS map 65 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x5600e04e5a80 0x5600e057b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.321+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5600e08a9c00 con 0x5600e04e5a80 2024-09-17T02:47:48.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x5600e04e5a80 0x5600e057b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x5600e04e5a80 0x5600e057b700 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=fdbd8966561d018a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc6dc82640 1 -- 172.21.15.131:0/4028772032 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5600e050a000 con 0x5600e04e5a80 2024-09-17T02:47:48.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x5600e08c6000 0x5600e08ca000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) -- 0x5600e08ce000 con 0x5600e08c6000 2024-09-17T02:47:48.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc6d481640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x5600e08c6000 0x5600e08ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc6d481640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x5600e08c6000 0x5600e08ca000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=fb458b7bb87e99b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.331 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc6d481640 1 -- 172.21.15.131:0/4028772032 <== osd.6 v2:172.21.15.189:6802/952142225 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5600e050a500 con 0x5600e08c6000 2024-09-17T02:47:48.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc62c6c640 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e95) -- 0x5600e08cec00 con 0x5600e08c6000 2024-09-17T02:47:48.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc62c6c640 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e95) -- 0x5600e08cf000 con 0x5600e04e5a80 2024-09-17T02:47:48.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc6dc82640 1 -- 172.21.15.131:0/4028772032 <== osd.2 v2:172.21.15.131:6800/163801004 2 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5600e050a000 con 0x5600e04e5a80 2024-09-17T02:47:48.332 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc6d481640 1 -- 172.21.15.131:0/4028772032 <== osd.6 v2:172.21.15.189:6802/952142225 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x5600e050a500 con 0x5600e08c6000 2024-09-17T02:47:48.332 INFO:teuthology.orchestra.run.smithi131.stdout:old journal was 4209834~892 2024-09-17T02:47:48.333 INFO:teuthology.orchestra.run.smithi131.stdout:new journal start will be 8388608 (4177882 bytes past old end) 2024-09-17T02:47:48.333 INFO:teuthology.orchestra.run.smithi131.stdout:writing journal head 2024-09-17T02:47:48.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.325+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5600e08ce400 con 0x5600e08c6000 2024-09-17T02:47:48.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6d481640 1 -- 172.21.15.131:0/4028772032 <== osd.6 v2:172.21.15.189:6802/952142225 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v95'20 uv20 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5600e050a500 con 0x5600e08c6000 2024-09-17T02:47:48.334 INFO:teuthology.orchestra.run.smithi131.stdout:writing EResetJournal entry 2024-09-17T02:47:48.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5600e08ce800 con 0x5600e04e5a80 2024-09-17T02:47:48.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x5600e08c6480 0x5600e08ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5600e08ce000 con 0x5600e08c6480 2024-09-17T02:47:48.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x5600e08c6900 0x5600e08cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5600e08cec00 con 0x5600e08c6900 2024-09-17T02:47:48.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5600e08c6d80 0x5600e08cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5600e08cf800 con 0x5600e08c6d80 2024-09-17T02:47:48.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5600e08c7200 0x5600e08cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5600e08cfc00 con 0x5600e08c7200 2024-09-17T02:47:48.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5600e08e8000 con 0x5600e08c6900 2024-09-17T02:47:48.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x5600e08c6480 0x5600e08ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5600e08c7200 0x5600e08cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6d481640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x5600e08c6900 0x5600e08cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5600e08c6d80 0x5600e08cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x5600e08c6480 0x5600e08ca580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=69ec4f3a8adb8b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6dc82640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5600e08c7200 0x5600e08cb600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=a3306100eda20039 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.329+0000 7fcc6d481640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x5600e08c6900 0x5600e08cab00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=bf1d7c1f550cd0d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.333+0000 7fcc6cc80640 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5600e08c6d80 0x5600e08cb080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=99873d2f1569259c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.333+0000 7fcc6dc82640 1 -- 172.21.15.131:0/4028772032 <== osd.2 v2:172.21.15.131:6800/163801004 3 ==== osd_op_reply(6 200.00000002 [delete] v95'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5600e050a000 con 0x5600e04e5a80 2024-09-17T02:47:48.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.333+0000 7fcc6dc82640 1 -- 172.21.15.131:0/4028772032 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(7 200.00000003 [delete] v95'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5600e050a000 con 0x5600e08c6480 2024-09-17T02:47:48.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.333+0000 7fcc62c6c640 1 -- 172.21.15.131:0/4028772032 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) -- 0x5600e08cf400 con 0x5600e04e5a80 2024-09-17T02:47:48.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.333+0000 7fcc6dc82640 1 -- 172.21.15.131:0/4028772032 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== osd_op_reply(10 200.00000006 [delete] v95'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5600e050a000 con 0x5600e08c7200 2024-09-17T02:47:48.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.337+0000 7fcc6d481640 1 -- 172.21.15.131:0/4028772032 <== osd.1 v2:172.21.15.131:6824/1616383728 1 ==== osd_op_reply(8 200.00000004 [delete] v95'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5600e050a500 con 0x5600e08c6900 2024-09-17T02:47:48.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.337+0000 7fcc6d481640 1 -- 172.21.15.131:0/4028772032 <== osd.1 v2:172.21.15.131:6824/1616383728 2 ==== osd_op_reply(11 200.00000007 [delete] v95'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5600e050a500 con 0x5600e08c6900 2024-09-17T02:47:48.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.337+0000 7fcc6dc82640 1 -- 172.21.15.131:0/4028772032 <== osd.2 v2:172.21.15.131:6800/163801004 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v95'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x5600e050a000 con 0x5600e04e5a80 2024-09-17T02:47:48.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6cc80640 1 -- 172.21.15.131:0/4028772032 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(9 200.00000005 [delete] v95'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5600e050a780 con 0x5600e08c6d80 2024-09-17T02:47:48.339 INFO:teuthology.orchestra.run.smithi131.stdout:done 2024-09-17T02:47:48.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5600e08c7200 msgr2=0x5600e08cb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5600e08c7200 0x5600e08cb600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x5600e08c6900 msgr2=0x5600e08cab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x5600e08c6900 0x5600e08cab00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x5600e04e5a80 msgr2=0x5600e057b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x5600e04e5a80 0x5600e057b700 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x5600e08c6480 msgr2=0x5600e08ca580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x5600e08c6480 0x5600e08ca580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5600e08c6d80 msgr2=0x5600e08cb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5600e08c6d80 0x5600e08cb080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x5600e08c6000 msgr2=0x5600e08ca000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x5600e08c6000 0x5600e08ca000 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6d481640 1 -- 172.21.15.131:0/4028772032 reap_dead start 2024-09-17T02:47:48.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6d481640 1 -- 172.21.15.131:0/4028772032 reap_dead start 2024-09-17T02:47:48.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6d481640 1 -- 172.21.15.131:0/4028772032 reap_dead start 2024-09-17T02:47:48.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e5600 msgr2=0x5600e057b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e5600 0x5600e057b180 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x5600e08ac000 tx=0x5600e1351740 comp rx=0 tx=0).stop 2024-09-17T02:47:48.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 shutdown_connections 2024-09-17T02:47:48.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/4028772032 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e5600 0x5600e057b180 secure :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x5600e08ac000 tx=0x5600e1351740 comp rx=0 tx=0).stop 2024-09-17T02:47:48.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 >> 172.21.15.131:0/4028772032 conn(0x5600e04e2000 msgr2=0x5600e0737000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:48.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 shutdown_connections 2024-09-17T02:47:48.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/4028772032 wait complete. 2024-09-17T02:47:48.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x5600e04e5180 msgr2=0x5600e057a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x5600e04e5180 0x5600e057a100 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x5600e051e6c0 tx=0x5600e051d920 comp rx=0 tx=0).stop 2024-09-17T02:47:48.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/629834433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2d80 msgr2=0x5600e0579080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2d80 0x5600e0579080 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x5600e133fda0 tx=0x5600e1350cc0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/629834433 shutdown_connections 2024-09-17T02:47:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x5600e04e5180 0x5600e057a100 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e2d80 0x5600e0579080 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e2900 0x5600e0579600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/629834433 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e2480 0x5600e0579b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/629834433 >> 172.21.15.131:0/629834433 conn(0x5600e04e3200 msgr2=0x5600e0736000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/629834433 shutdown_connections 2024-09-17T02:47:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/629834433 wait complete. 2024-09-17T02:47:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/3073932736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e3b00 msgr2=0x5600e0578000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/3073932736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e3b00 0x5600e0578000 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x5600e1265d40 tx=0x5600e132a120 comp rx=0 tx=0).stop 2024-09-17T02:47:48.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/3073932736 shutdown_connections 2024-09-17T02:47:48.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/3073932736 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5600e04e4400 0x5600e0578b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/3073932736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5600e04e3b00 0x5600e0578000 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 --2- 172.21.15.131:0/3073932736 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5600e04e3680 0x5600e0578580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/3073932736 >> 172.21.15.131:0/3073932736 conn(0x5600e04e3f80 msgr2=0x5600dc3fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:48.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/3073932736 shutdown_connections 2024-09-17T02:47:48.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.341+0000 7fcc6ecc2400 1 -- 172.21.15.131:0/3073932736 wait complete. 2024-09-17T02:47:48.347 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--yes-i-really-really-mean-it'] in time 0:00:00.144706, result: old journal was 4209834~892 new journal start will be 8388608 (4177882 bytes past old end) writing journal head writing EResetJournal entry done 2024-09-17T02:47:48.347 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T02:47:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.489+0000 7f5a03e3f640 1 -- 172.21.15.131:0/251356155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0cf250 msgr2=0x7f59fc0cf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.489+0000 7f5a03e3f640 1 --2- 172.21.15.131:0/251356155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0cf250 0x7f59fc0cf660 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0b6320 tx=0x7f59f801cd80 comp rx=0 tx=0).stop 2024-09-17T02:47:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.489+0000 7f5a03e3f640 1 -- 172.21.15.131:0/251356155 shutdown_connections 2024-09-17T02:47:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.489+0000 7f5a03e3f640 1 --2- 172.21.15.131:0/251356155 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc0cf250 0x7f59fc0cf660 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.489+0000 7f5a03e3f640 1 --2- 172.21.15.131:0/251356155 >> v2:172.21.15.189:3300/0 conn(0x7f59fc0702a0 0x7f59fc0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.489+0000 7f5a03e3f640 1 -- 172.21.15.131:0/251356155 >> 172.21.15.131:0/251356155 conn(0x7f59fc0cd6a0 msgr2=0x7f59fc0cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.489+0000 7f5a03e3f640 1 -- 172.21.15.131:0/251356155 shutdown_connections 2024-09-17T02:47:48.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.489+0000 7f5a03e3f640 1 -- 172.21.15.131:0/251356155 wait complete. 2024-09-17T02:47:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a03e3f640 1 Processor -- start 2024-09-17T02:47:48.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a03e3f640 1 -- start start 2024-09-17T02:47:48.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a03e3f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59fc0702a0 0x7f59fc158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a03e3f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59fc0cf250 0x7f59fc14bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a03e3f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc15c660 0x7f59fc154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a03e3f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc169f30 con 0x7f59fc15c660 2024-09-17T02:47:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a03e3f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc169db0 con 0x7f59fc0cf250 2024-09-17T02:47:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a03e3f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f59fc16a0b0 con 0x7f59fc0702a0 2024-09-17T02:47:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a023dc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc15c660 0x7f59fc154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a023dc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc15c660 0x7f59fc154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54908/0 (socket says 172.21.15.131:54908) 2024-09-17T02:47:48.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a023dc640 1 -- 172.21.15.131:0/1517096491 learned_addr learned my addr 172.21.15.131:0/1517096491 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a013da640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59fc0cf250 0x7f59fc14bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a023dc640 1 -- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59fc0702a0 msgr2=0x7f59fc158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a023dc640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59fc0702a0 0x7f59fc158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a023dc640 1 -- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59fc0cf250 msgr2=0x7f59fc14bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a023dc640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59fc0cf250 0x7f59fc14bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a023dc640 1 -- 172.21.15.131:0/1517096491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc158ff0 con 0x7f59fc15c660 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a023dc640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc15c660 0x7f59fc154550 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f59f801b430 tx=0x7f59f8003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f59f27fc640 1 -- 172.21.15.131:0/1517096491 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f80bc070 con 0x7f59fc15c660 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f59f27fc640 1 -- 172.21.15.131:0/1517096491 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59f80042f0 con 0x7f59fc15c660 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f59f27fc640 1 -- 172.21.15.131:0/1517096491 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f80b2040 con 0x7f59fc15c660 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a03e3f640 1 -- 172.21.15.131:0/1517096491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc0af0d0 con 0x7f59fc15c660 2024-09-17T02:47:48.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a03e3f640 1 -- 172.21.15.131:0/1517096491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc157000 con 0x7f59fc15c660 2024-09-17T02:47:48.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.493+0000 7f5a03e3f640 1 -- 172.21.15.131:0/1517096491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59fc0cf660 con 0x7f59fc15c660 2024-09-17T02:47:48.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.497+0000 7f59f27fc640 1 -- 172.21.15.131:0/1517096491 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f59f80bf070 con 0x7f59fc15c660 2024-09-17T02:47:48.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.497+0000 7f59f27fc640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59e0073200 0x7f59e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.497+0000 7f59f27fc640 1 -- 172.21.15.131:0/1517096491 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f59f812e3a0 con 0x7f59fc15c660 2024-09-17T02:47:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.497+0000 7f5a01bdb640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59e0073200 0x7f59e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:48.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.497+0000 7f5a01bdb640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59e0073200 0x7f59e0075710 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009650 tx=0x7f59f4006bb0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:48.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.501+0000 7f59f27fc640 1 -- 172.21.15.131:0/1517096491 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f59f8016970 con 0x7f59fc15c660 2024-09-17T02:47:48.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:48.689+0000 7f5a03e3f640 1 -- 172.21.15.131:0/1517096491 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f59fc156290 con 0x7f59fc15c660 2024-09-17T02:47:49.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.097+0000 7f59f27fc640 1 -- 172.21.15.131:0/1517096491 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v66) ==== 162+0+0 (secure 0 0 0) 0x7f59f80f9290 con 0x7f59fc15c660 2024-09-17T02:47:49.094 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T02:47:49.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 -- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59e0073200 msgr2=0x7f59e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:49.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59e0073200 0x7f59e0075710 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f59f4009650 tx=0x7f59f4006bb0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 -- 172.21.15.131:0/1517096491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc15c660 msgr2=0x7f59fc154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:49.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc15c660 0x7f59fc154550 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f59f801b430 tx=0x7f59f8003f80 comp rx=0 tx=0).stop 2024-09-17T02:47:49.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 -- 172.21.15.131:0/1517096491 shutdown_connections 2024-09-17T02:47:49.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59e0073200 0x7f59e0075710 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59fc15c660 0x7f59fc154550 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59fc0cf250 0x7f59fc14bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 --2- 172.21.15.131:0/1517096491 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59fc0702a0 0x7f59fc158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 -- 172.21.15.131:0/1517096491 >> 172.21.15.131:0/1517096491 conn(0x7f59fc0cd6a0 msgr2=0x7f59fc0ca730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:49.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 -- 172.21.15.131:0/1517096491 shutdown_connections 2024-09-17T02:47:49.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.101+0000 7f5a03e3f640 1 -- 172.21.15.131:0/1517096491 wait complete. 2024-09-17T02:47:49.129 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:47:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 -- 172.21.15.131:0/2042358594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff406e2c0 msgr2=0x7fcff40c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 --2- 172.21.15.131:0/2042358594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff406e2c0 0x7fcff40c4580 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcff4071d30 tx=0x7fcff001c770 comp rx=0 tx=0).stop 2024-09-17T02:47:49.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 -- 172.21.15.131:0/2042358594 shutdown_connections 2024-09-17T02:47:49.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 --2- 172.21.15.131:0/2042358594 >> v2:172.21.15.189:3300/0 conn(0x7fcff40cec00 0x7fcff40cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 --2- 172.21.15.131:0/2042358594 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff406e2c0 0x7fcff40c4580 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 -- 172.21.15.131:0/2042358594 >> 172.21.15.131:0/2042358594 conn(0x7fcff406eea0 msgr2=0x7fcff406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:49.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 -- 172.21.15.131:0/2042358594 shutdown_connections 2024-09-17T02:47:49.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 -- 172.21.15.131:0/2042358594 wait complete. 2024-09-17T02:47:49.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 Processor -- start 2024-09-17T02:47:49.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 -- start start 2024-09-17T02:47:49.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcff406e2c0 0x7fcff4149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:49.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff40c4ac0 0x7fcff40cba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:49.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcff40cec00 0x7fcff40c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4169c60 con 0x7fcff40c4ac0 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4169ae0 con 0x7fcff406e2c0 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcffbaff640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcff4169de0 con 0x7fcff40cec00 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcff909a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff40c4ac0 0x7fcff40cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcff909a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff40c4ac0 0x7fcff40cba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:54932/0 (socket says 172.21.15.131:54932) 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcff909a640 1 -- 172.21.15.131:0/4082606581 learned_addr learned my addr 172.21.15.131:0/4082606581 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.285+0000 7fcff989b640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcff406e2c0 0x7fcff4149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcffa09c640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcff40cec00 0x7fcff40c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcff909a640 1 -- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcff40cec00 msgr2=0x7fcff40c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcff909a640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcff40cec00 0x7fcff40c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcff909a640 1 -- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcff406e2c0 msgr2=0x7fcff4149420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcff909a640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcff406e2c0 0x7fcff4149420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcff909a640 1 -- 172.21.15.131:0/4082606581 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff4155480 con 0x7fcff40c4ac0 2024-09-17T02:47:49.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcffa09c640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcff40cec00 0x7fcff40c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:49.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcff909a640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff40c4ac0 0x7fcff40cba20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c430 tx=0x7fcfe400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:49.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcfea7fc640 1 -- 172.21.15.131:0/4082606581 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4011790 con 0x7fcff40c4ac0 2024-09-17T02:47:49.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcfea7fc640 1 -- 172.21.15.131:0/4082606581 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe4017070 con 0x7fcff40c4ac0 2024-09-17T02:47:49.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcffbaff640 1 -- 172.21.15.131:0/4082606581 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff4154ec0 con 0x7fcff40c4ac0 2024-09-17T02:47:49.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcfea7fc640 1 -- 172.21.15.131:0/4082606581 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4011ce0 con 0x7fcff40c4ac0 2024-09-17T02:47:49.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcffbaff640 1 -- 172.21.15.131:0/4082606581 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff40cb220 con 0x7fcff40c4ac0 2024-09-17T02:47:49.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcffbaff640 1 -- 172.21.15.131:0/4082606581 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff4156dc0 con 0x7fcff40c4ac0 2024-09-17T02:47:49.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcfea7fc640 1 -- 172.21.15.131:0/4082606581 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcfe401c460 con 0x7fcff40c4ac0 2024-09-17T02:47:49.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcfea7fc640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 0x7fcfd8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:49.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcfea7fc640 1 -- 172.21.15.131:0/4082606581 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fcfe40942b0 con 0x7fcff40c4ac0 2024-09-17T02:47:49.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.289+0000 7fcff989b640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 0x7fcfd8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:49.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.293+0000 7fcff989b640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 0x7fcfd8075710 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcff00af000 tx=0x7fcff001afb0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:49.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.293+0000 7fcfea7fc640 1 -- 172.21.15.131:0/4082606581 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcfe405f0f0 con 0x7fcff40c4ac0 2024-09-17T02:47:49.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.497+0000 7fcffbaff640 1 -- 172.21.15.131:0/4082606581 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcff40c4580 con 0x7fcff40c4ac0 2024-09-17T02:47:49.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.497+0000 7fcfea7fc640 1 -- 172.21.15.131:0/4082606581 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5231 (secure 0 0 0) 0x7fcfe405e9f0 con 0x7fcff40c4ac0 2024-09-17T02:47:49.496 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:49.496 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":67,"btime":"2024-09-17T02:47:49:104443+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:47:49.104412+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7233":{"gid":7233,"name":"c","rank":0,"incarnation":67,"state":"up:replay","state_seq":1,"addr":"172.21.15.131:6835/3013984008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3013984008},{"type":"v1","addr":"172.21.15.131:6835","nonce":3013984008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T02:47:49.496 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 67 2024-09-17T02:47:49.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 -- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 msgr2=0x7fcfd8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:49.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 0x7fcfd8075710 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcff00af000 tx=0x7fcff001afb0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 -- 172.21.15.131:0/4082606581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff40c4ac0 msgr2=0x7fcff40cba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:49.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff40c4ac0 0x7fcff40cba20 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400c430 tx=0x7fcfe400c8f0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 -- 172.21.15.131:0/4082606581 shutdown_connections 2024-09-17T02:47:49.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 0x7fcfd8075710 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcff40cec00 0x7fcff40c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcff40c4ac0 0x7fcff40cba20 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 --2- 172.21.15.131:0/4082606581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcff406e2c0 0x7fcff4149420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:49.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 -- 172.21.15.131:0/4082606581 >> 172.21.15.131:0/4082606581 conn(0x7fcff406eea0 msgr2=0x7fcff415be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:49.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 -- 172.21.15.131:0/4082606581 shutdown_connections 2024-09-17T02:47:49.499 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:49.501+0000 7fcffbaff640 1 -- 172.21.15.131:0/4082606581 wait complete. 2024-09-17T02:47:49.919 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:47:25.797713+0000', 'modified': '2024-09-17T02:47:49.104412+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7233}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7233': {'gid': 7233, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/3013984008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 3013984008}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 3013984008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:47:49.919 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7233:up:replay 2024-09-17T02:47:50.920 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:47:51.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.089+0000 7fc80f6cd640 1 -- 172.21.15.131:0/377718572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f4004d50 msgr2=0x7fc7f4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:51.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.089+0000 7fc80f6cd640 1 --2- 172.21.15.131:0/377718572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7f4004d50 0x7fc7f4005160 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fc800009ee0 tx=0x7fc80001d630 comp rx=0 tx=0).stop 2024-09-17T02:47:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.089+0000 7fc80dc6a640 1 -- 172.21.15.131:0/377718572 reap_dead start 2024-09-17T02:47:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.089+0000 7fc80f6cd640 1 -- 172.21.15.131:0/377718572 shutdown_connections 2024-09-17T02:47:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.089+0000 7fc80f6cd640 1 -- 172.21.15.131:0/377718572 >> 172.21.15.131:0/377718572 conn(0x7fc80806eea0 msgr2=0x7fc80806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.089+0000 7fc80f6cd640 1 -- 172.21.15.131:0/377718572 shutdown_connections 2024-09-17T02:47:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.089+0000 7fc80f6cd640 1 -- 172.21.15.131:0/377718572 wait complete. 2024-09-17T02:47:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80f6cd640 1 Processor -- start 2024-09-17T02:47:51.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80f6cd640 1 -- start start 2024-09-17T02:47:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80f6cd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f4004d50 0x7fc808157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80f6cd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc808149280 0x7fc808154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80f6cd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc808149690 0x7fc808155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80f6cd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc80815ca30 con 0x7fc808149690 2024-09-17T02:47:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80f6cd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc80815cbb0 con 0x7fc7f4004d50 2024-09-17T02:47:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80f6cd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc8080cf010 con 0x7fc808149280 2024-09-17T02:47:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80dc6a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc808149690 0x7fc808155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:51.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80dc6a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc808149690 0x7fc808155410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42796/0 (socket says 172.21.15.131:42796) 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80dc6a640 1 -- 172.21.15.131:0/1538958396 learned_addr learned my addr 172.21.15.131:0/1538958396 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80d469640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f4004d50 0x7fc808157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80cc68640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc808149280 0x7fc808154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80dc6a640 1 -- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc808149280 msgr2=0x7fc808154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80dc6a640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc808149280 0x7fc808154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80dc6a640 1 -- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f4004d50 msgr2=0x7fc808157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80dc6a640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f4004d50 0x7fc808157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80dc6a640 1 -- 172.21.15.131:0/1538958396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8080c9200 con 0x7fc808149690 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80dc6a640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc808149690 0x7fc808155410 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc80001d0a0 tx=0x7fc80001be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc7fdffb640 1 -- 172.21.15.131:0/1538958396 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000ad050 con 0x7fc808149690 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc7fdffb640 1 -- 172.21.15.131:0/1538958396 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc800003980 con 0x7fc808149690 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80f6cd640 1 -- 172.21.15.131:0/1538958396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8080c94e0 con 0x7fc808149690 2024-09-17T02:47:51.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc7fdffb640 1 -- 172.21.15.131:0/1538958396 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800009200 con 0x7fc808149690 2024-09-17T02:47:51.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.093+0000 7fc80f6cd640 1 -- 172.21.15.131:0/1538958396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8080c9aa0 con 0x7fc808149690 2024-09-17T02:47:51.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.097+0000 7fc80f6cd640 1 -- 172.21.15.131:0/1538958396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8080c8f70 con 0x7fc808149690 2024-09-17T02:47:51.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.097+0000 7fc7fdffb640 1 -- 172.21.15.131:0/1538958396 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc8000b0070 con 0x7fc808149690 2024-09-17T02:47:51.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.097+0000 7fc7fdffb640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ec073230 0x7fc7ec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.097+0000 7fc7fdffb640 1 -- 172.21.15.131:0/1538958396 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fc80012efb0 con 0x7fc808149690 2024-09-17T02:47:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.097+0000 7fc80d469640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ec073230 0x7fc7ec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:51.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.097+0000 7fc80d469640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ec073230 0x7fc7ec075740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80036b0 tx=0x7fc7f800c040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:51.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.101+0000 7fc7fdffb640 1 -- 172.21.15.131:0/1538958396 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc800017340 con 0x7fc808149690 2024-09-17T02:47:51.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.301+0000 7fc80f6cd640 1 -- 172.21.15.131:0/1538958396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8080c50a0 con 0x7fc808149690 2024-09-17T02:47:51.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.301+0000 7fc7fdffb640 1 -- 172.21.15.131:0/1538958396 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5231 (secure 0 0 0) 0x7fc8000f9800 con 0x7fc808149690 2024-09-17T02:47:51.299 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:51.299 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":69,"btime":"2024-09-17T02:47:51:118482+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:47:50.123906+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7233":{"gid":7233,"name":"c","rank":0,"incarnation":67,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.131:6835/3013984008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3013984008},{"type":"v1","addr":"172.21.15.131:6835","nonce":3013984008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T02:47:51.300 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 69 2024-09-17T02:47:51.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 -- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ec073230 msgr2=0x7fc7ec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:51.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ec073230 0x7fc7ec075740 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80036b0 tx=0x7fc7f800c040 comp rx=0 tx=0).stop 2024-09-17T02:47:51.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 -- 172.21.15.131:0/1538958396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc808149690 msgr2=0x7fc808155410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:51.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc808149690 0x7fc808155410 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc80001d0a0 tx=0x7fc80001be70 comp rx=0 tx=0).stop 2024-09-17T02:47:51.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 -- 172.21.15.131:0/1538958396 shutdown_connections 2024-09-17T02:47:51.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc7ec073230 0x7fc7ec075740 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:51.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc808149690 0x7fc808155410 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:51.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc808149280 0x7fc808154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:51.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 --2- 172.21.15.131:0/1538958396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7f4004d50 0x7fc808157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:51.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 -- 172.21.15.131:0/1538958396 >> 172.21.15.131:0/1538958396 conn(0x7fc80806eea0 msgr2=0x7fc80815a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:51.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 -- 172.21.15.131:0/1538958396 shutdown_connections 2024-09-17T02:47:51.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:51.305+0000 7fc80f6cd640 1 -- 172.21.15.131:0/1538958396 wait complete. 2024-09-17T02:47:51.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:47:25.797713+0000', 'modified': '2024-09-17T02:47:50.123906+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7233}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7233': {'gid': 7233, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.131:6835/3013984008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 3013984008}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 3013984008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:47:51.724 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7233:up:rejoin 2024-09-17T02:47:52.725 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:47:52.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.893+0000 7f215588c640 1 -- 172.21.15.131:0/419690787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f213c003a40 msgr2=0x7f213c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:52.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.893+0000 7f215588c640 1 --2- 172.21.15.131:0/419690787 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f213c003a40 0x7f213c003eb0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f21480016c0 tx=0x7f214801c790 comp rx=0 tx=0).stop 2024-09-17T02:47:52.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.893+0000 7f214f7fe640 1 -- 172.21.15.131:0/419690787 reap_dead start 2024-09-17T02:47:52.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.893+0000 7f215588c640 1 -- 172.21.15.131:0/419690787 shutdown_connections 2024-09-17T02:47:52.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.893+0000 7f215588c640 1 -- 172.21.15.131:0/419690787 >> 172.21.15.131:0/419690787 conn(0x7f21500d1b10 msgr2=0x7f21500d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:52.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.893+0000 7f215588c640 1 -- 172.21.15.131:0/419690787 shutdown_connections 2024-09-17T02:47:52.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.893+0000 7f215588c640 1 -- 172.21.15.131:0/419690787 wait complete. 2024-09-17T02:47:52.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f215588c640 1 Processor -- start 2024-09-17T02:47:52.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f215588c640 1 -- start start 2024-09-17T02:47:52.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f215588c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f213c003a40 0x7f21501f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:52.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f215588c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21500c7ee0 0x7f215015a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:52.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f215588c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2150159cc0 0x7f21501576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:52.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f215588c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f215016b1f0 con 0x7f2150159cc0 2024-09-17T02:47:52.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f215588c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f215016b070 con 0x7f213c003a40 2024-09-17T02:47:52.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f215588c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f215016b370 con 0x7f21500c7ee0 2024-09-17T02:47:52.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f214f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2150159cc0 0x7f21501576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:52.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f214f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2150159cc0 0x7f21501576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42818/0 (socket says 172.21.15.131:42818) 2024-09-17T02:47:52.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f214f7fe640 1 -- 172.21.15.131:0/3292036957 learned_addr learned my addr 172.21.15.131:0/3292036957 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f214e7fc640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21500c7ee0 0x7f215015a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f214effd640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f213c003a40 0x7f21501f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f214f7fe640 1 -- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21500c7ee0 msgr2=0x7f215015a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f214f7fe640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21500c7ee0 0x7f215015a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f214f7fe640 1 -- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f213c003a40 msgr2=0x7f21501f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f214f7fe640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f213c003a40 0x7f21501f9e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.897+0000 7f214f7fe640 1 -- 172.21.15.131:0/3292036957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f215015ac00 con 0x7f2150159cc0 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f214f7fe640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2150159cc0 0x7f21501576b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f21500b72b0 tx=0x7f214000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f2147fff640 1 -- 172.21.15.131:0/3292036957 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140005ca0 con 0x7f2150159cc0 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f2147fff640 1 -- 172.21.15.131:0/3292036957 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2140004020 con 0x7f2150159cc0 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f215588c640 1 -- 172.21.15.131:0/3292036957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21501594e0 con 0x7f2150159cc0 2024-09-17T02:47:52.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f2147fff640 1 -- 172.21.15.131:0/3292036957 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2140004470 con 0x7f2150159cc0 2024-09-17T02:47:52.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f215588c640 1 -- 172.21.15.131:0/3292036957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21500c8d90 con 0x7f2150159cc0 2024-09-17T02:47:52.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f215588c640 1 -- 172.21.15.131:0/3292036957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21500c83f0 con 0x7f2150159cc0 2024-09-17T02:47:52.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f2147fff640 1 -- 172.21.15.131:0/3292036957 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2140011070 con 0x7f2150159cc0 2024-09-17T02:47:52.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f2147fff640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2130073200 0x7f2130075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:52.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f2147fff640 1 -- 172.21.15.131:0/3292036957 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f21400936a0 con 0x7f2150159cc0 2024-09-17T02:47:52.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f214effd640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2130073200 0x7f2130075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:52.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.901+0000 7f214effd640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2130073200 0x7f2130075710 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f21500b7610 tx=0x7f21480a8040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:52.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:52.905+0000 7f2147fff640 1 -- 172.21.15.131:0/3292036957 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f214005e4b0 con 0x7f2150159cc0 2024-09-17T02:47:53.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.109+0000 7f215588c640 1 -- 172.21.15.131:0/3292036957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f215015f290 con 0x7f2150159cc0 2024-09-17T02:47:53.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f2147fff640 1 -- 172.21.15.131:0/3292036957 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5238 (secure 0 0 0) 0x7f214005ddb0 con 0x7f2150159cc0 2024-09-17T02:47:53.109 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:47:53.110 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":70,"btime":"2024-09-17T02:47:52:120798+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:47:52.120778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7233":{"gid":7233,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/3013984008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3013984008},{"type":"v1","addr":"172.21.15.131:6835","nonce":3013984008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7233,"qdb_cluster":[7233]},"id":6}]} 2024-09-17T02:47:53.110 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 70 2024-09-17T02:47:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 -- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2130073200 msgr2=0x7f2130075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2130073200 0x7f2130075710 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f21500b7610 tx=0x7f21480a8040 comp rx=0 tx=0).stop 2024-09-17T02:47:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 -- 172.21.15.131:0/3292036957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2150159cc0 msgr2=0x7f21501576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2150159cc0 0x7f21501576b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f21500b72b0 tx=0x7f214000c890 comp rx=0 tx=0).stop 2024-09-17T02:47:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 -- 172.21.15.131:0/3292036957 shutdown_connections 2024-09-17T02:47:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2130073200 0x7f2130075710 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2150159cc0 0x7f21501576b0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21500c7ee0 0x7f215015a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 --2- 172.21.15.131:0/3292036957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f213c003a40 0x7f21501f9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 -- 172.21.15.131:0/3292036957 >> 172.21.15.131:0/3292036957 conn(0x7f21500d1b10 msgr2=0x7f21501617e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:53.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 -- 172.21.15.131:0/3292036957 shutdown_connections 2024-09-17T02:47:53.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.113+0000 7f215588c640 1 -- 172.21.15.131:0/3292036957 wait complete. 2024-09-17T02:47:53.536 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:47:25.797713+0000', 'modified': '2024-09-17T02:47:52.120778+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7233}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7233': {'gid': 7233, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.131:6835/3013984008', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 3013984008}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 3013984008}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7233, 'qdb_cluster': [7233]} 2024-09-17T02:47:53.536 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:47:53.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T02:47:53.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.709+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2753911892 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb9f40ced00 msgr2=0x7fb9f406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.709+0000 7fb9fb5e7640 1 --2- 172.21.15.131:0/2753911892 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb9f40ced00 0x7fb9f406dea0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009840 tx=0x7fb9f001c820 comp rx=0 tx=0).stop 2024-09-17T02:47:53.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.709+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2753911892 shutdown_connections 2024-09-17T02:47:53.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.709+0000 7fb9fb5e7640 1 --2- 172.21.15.131:0/2753911892 >> v2:172.21.15.189:3300/0 conn(0x7fb9f406e3e0 0x7fb9f415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.709+0000 7fb9fb5e7640 1 --2- 172.21.15.131:0/2753911892 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb9f40ced00 0x7fb9f406dea0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.709+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2753911892 >> 172.21.15.131:0/2753911892 conn(0x7fb9f406eea0 msgr2=0x7fb9f406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:53.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.709+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2753911892 shutdown_connections 2024-09-17T02:47:53.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.709+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2753911892 wait complete. 2024-09-17T02:47:53.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.709+0000 7fb9fb5e7640 1 Processor -- start 2024-09-17T02:47:53.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.709+0000 7fb9fb5e7640 1 -- start start 2024-09-17T02:47:53.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9fb5e7640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb9f406e3e0 0x7fb9f414ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9fb5e7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9f40ced00 0x7fb9f414f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9fb5e7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9f415c3e0 0x7fb9f4157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9fb5e7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4168b60 con 0x7fb9f40ced00 2024-09-17T02:47:53.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9fb5e7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f41689e0 con 0x7fb9f415c3e0 2024-09-17T02:47:53.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9fb5e7640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f4168ce0 con 0x7fb9f406e3e0 2024-09-17T02:47:53.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9f8b82640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9f40ced00 0x7fb9f414f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9f8b82640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9f40ced00 0x7fb9f414f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42834/0 (socket says 172.21.15.131:42834) 2024-09-17T02:47:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9f8b82640 1 -- 172.21.15.131:0/2543860986 learned_addr learned my addr 172.21.15.131:0/2543860986 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9f9383640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb9f406e3e0 0x7fb9f414ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9f9b84640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9f415c3e0 0x7fb9f4157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9f8b82640 1 -- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb9f406e3e0 msgr2=0x7fb9f414ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb9f406e3e0 0x7fb9f414ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9f8b82640 1 -- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9f415c3e0 msgr2=0x7fb9f4157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9f415c3e0 0x7fb9f4157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9f8b82640 1 -- 172.21.15.131:0/2543860986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f4150160 con 0x7fb9f40ced00 2024-09-17T02:47:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9f40ced00 0x7fb9f414f550 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009a50 tx=0x7fb9e4009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:53.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9e9ffb640 1 -- 172.21.15.131:0/2543860986 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e401c070 con 0x7fb9f40ced00 2024-09-17T02:47:53.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9e9ffb640 1 -- 172.21.15.131:0/2543860986 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9e400baf0 con 0x7fb9f40ced00 2024-09-17T02:47:53.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2543860986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f4150440 con 0x7fb9f40ced00 2024-09-17T02:47:53.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9e9ffb640 1 -- 172.21.15.131:0/2543860986 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4017410 con 0x7fb9f40ced00 2024-09-17T02:47:53.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.713+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2543860986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f4159e70 con 0x7fb9f40ced00 2024-09-17T02:47:53.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9e9ffb640 1 -- 172.21.15.131:0/2543860986 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb9e401f070 con 0x7fb9f40ced00 2024-09-17T02:47:53.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9e9ffb640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9d8073200 0x7fb9d8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9e9ffb640 1 -- 172.21.15.131:0/2543860986 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fb9e4095fe0 con 0x7fb9f40ced00 2024-09-17T02:47:53.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f9383640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9d8073200 0x7fb9d8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fba028b2000 1 Processor -- start 2024-09-17T02:47:53.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fba028b2000 1 -- start start 2024-09-17T02:47:53.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f9383640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9d8073200 0x7fb9d8075710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00af000 tx=0x7fb9f001b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:53.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fba028b2000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde059460 0x559cde59a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fba028b2000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde59abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fba028b2000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde432ca0 0x559cde59b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fba028b2000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559cde46f410 con 0x559cde449950 2024-09-17T02:47:53.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fba028b2000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559cde3acb40 con 0x559cde432ca0 2024-09-17T02:47:53.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fba028b2000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559cde39bf70 con 0x559cde059460 2024-09-17T02:47:53.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f8b82640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde59abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f8b82640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde59abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42836/0 (socket says 172.21.15.131:42836) 2024-09-17T02:47:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f8b82640 1 -- 172.21.15.131:0/980054925 learned_addr learned my addr 172.21.15.131:0/980054925 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f9383640 1 --2- 172.21.15.131:0/980054925 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde059460 0x559cde59a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f9b84640 1 --2- 172.21.15.131:0/980054925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde432ca0 0x559cde59b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f8b82640 1 -- 172.21.15.131:0/980054925 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde059460 msgr2=0x559cde59a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/980054925 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde059460 0x559cde59a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f8b82640 1 -- 172.21.15.131:0/980054925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde432ca0 msgr2=0x559cde59b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/980054925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde432ca0 0x559cde59b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f8b82640 1 -- 172.21.15.131:0/980054925 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cde3635b0 con 0x559cde449950 2024-09-17T02:47:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/980054925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde59abe0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4094210 tx=0x7fb9e40606f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcdb02db2a25d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.717+0000 7fb9deffd640 1 -- 172.21.15.131:0/980054925 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4060180 con 0x559cde449950 2024-09-17T02:47:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9deffd640 1 -- 172.21.15.131:0/980054925 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9e40673e0 con 0x559cde449950 2024-09-17T02:47:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9deffd640 1 -- 172.21.15.131:0/980054925 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e405c180 con 0x559cde449950 2024-09-17T02:47:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 -- 172.21.15.131:0/980054925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 msgr2=0x559cde59abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 --2- 172.21.15.131:0/980054925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde59abe0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4094210 tx=0x7fb9e40606f0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 -- 172.21.15.131:0/980054925 shutdown_connections 2024-09-17T02:47:53.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 --2- 172.21.15.131:0/980054925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde432ca0 0x559cde59b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 --2- 172.21.15.131:0/980054925 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde59abe0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 --2- 172.21.15.131:0/980054925 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde059460 0x559cde59a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 -- 172.21.15.131:0/980054925 >> 172.21.15.131:0/980054925 conn(0x559cde51fd90 msgr2=0x559cde54f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 -- 172.21.15.131:0/980054925 shutdown_connections 2024-09-17T02:47:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 -- 172.21.15.131:0/980054925 wait complete. 2024-09-17T02:47:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 -- start start 2024-09-17T02:47:53.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 Processor -- start 2024-09-17T02:47:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde059460 0x559cde5a1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde449950 0x559cde5a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559cde3365f0 con 0x559cde432ca0 2024-09-17T02:47:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559cde39fd80 con 0x559cde449950 2024-09-17T02:47:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559cde342520 con 0x559cde059460 2024-09-17T02:47:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fba028b2000 10 client.0 _ll_register_callbacks cb 0x559cde346830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:47:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f8b82640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f8b82640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5a1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42852/0 (socket says 172.21.15.131:42852) 2024-09-17T02:47:53.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f8b82640 1 -- 172.21.15.131:0/4293232286 learned_addr learned my addr 172.21.15.131:0/4293232286 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:53.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f9b84640 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde449950 0x559cde5a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f9383640 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde059460 0x559cde5a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f8b82640 1 -- 172.21.15.131:0/4293232286 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde059460 msgr2=0x559cde5a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:53.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde059460 0x559cde5a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f8b82640 1 -- 172.21.15.131:0/4293232286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde449950 msgr2=0x559cde5a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:53.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde449950 0x559cde5a23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f8b82640 1 -- 172.21.15.131:0/4293232286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cde33a190 con 0x559cde432ca0 2024-09-17T02:47:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f9383640 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde059460 0x559cde5a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5a1e60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009a20 tx=0x7fb9e405cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86c5e03e372973b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.721+0000 7fb9d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:47:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fb9d3fff640 1 -- 172.21.15.131:0/4293232286 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e407f060 con 0x559cde432ca0 2024-09-17T02:47:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fb9d3fff640 1 -- 172.21.15.131:0/4293232286 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fb9e407adb0 con 0x559cde432ca0 2024-09-17T02:47:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 -- 172.21.15.131:0/4293232286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559cde334110 con 0x559cde432ca0 2024-09-17T02:47:53.738 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fb9d3fff640 1 -- 172.21.15.131:0/4293232286 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fb9e40673e0 con 0x559cde432ca0 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 10 client.7293 fetch_fsmap learned FSMap version 70 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 -- 172.21.15.131:0/4293232286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x559cde420f10 con 0x559cde432ca0 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fb9d3fff640 1 -- 172.21.15.131:0/4293232286 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559cde334110 con 0x559cde432ca0 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fb9d3fff640 1 -- 172.21.15.131:0/4293232286 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7fb9e404d070 con 0x559cde432ca0 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 10 client.7293 fetch_fsmap finished waiting for FSMap version 70 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 10 client.7293 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 20 client.7293 populate_metadata read hostname 'smithi131' 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cde03e7d0 0x559cde5abd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 4 client.7293 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 -- 172.21.15.131:0/4293232286 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cde3aea90 con 0x559cde03e7d0 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fb9f9383640 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cde03e7d0 0x559cde5abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fb9d3fff640 10 client.7293 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fb9f9383640 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cde03e7d0 0x559cde5abd00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787d4fef74226d91 server_cookie=71a7b234aaac1d80 in_seq=0 out_seq=0 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fb9d3fff640 1 -- 172.21.15.131:0/4293232286 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559cde3aea90 con 0x559cde03e7d0 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fb9d3fff640 10 client.7293 handle_command_reply: tid=0 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 client.7293 shutdown 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 -- 172.21.15.131:0/4293232286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 msgr2=0x559cde5a1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5a1e60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009a20 tx=0x7fb9e405cd30 comp rx=0 tx=0).stop 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 -- 172.21.15.131:0/4293232286 shutdown_connections 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde449950 0x559cde5a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cde03e7d0 0x559cde5abd00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5a1e60 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 --2- 172.21.15.131:0/4293232286 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde059460 0x559cde5a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fb9d3fff640 0 client.7293 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:53.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 -- 172.21.15.131:0/4293232286 >> 172.21.15.131:0/4293232286 conn(0x559cde51fd90 msgr2=0x559cde556440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 -- 172.21.15.131:0/4293232286 shutdown_connections 2024-09-17T02:47:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 1 -- 172.21.15.131:0/4293232286 wait complete. 2024-09-17T02:47:53.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.725+0000 7fba028b2000 20 client.7293 trim_cache size 0 max 16384 2024-09-17T02:47:53.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fba028b2000 1 Processor -- start 2024-09-17T02:47:53.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fba028b2000 1 -- start start 2024-09-17T02:47:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fba028b2000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde059460 0x559cde60c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fba028b2000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde432ca0 0x559cde60cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fba028b2000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde60d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fba028b2000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4201a20 con 0x559cde449950 2024-09-17T02:47:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fba028b2000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f40ce8d0 con 0x559cde059460 2024-09-17T02:47:53.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fba028b2000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb9f40cdbd0 con 0x559cde432ca0 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9f9b84640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde60d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9f9b84640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde60d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42860/0 (socket says 172.21.15.131:42860) 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9f9b84640 1 -- 172.21.15.131:0/975672855 learned_addr learned my addr 172.21.15.131:0/975672855 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9f9383640 1 --2- 172.21.15.131:0/975672855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde059460 0x559cde60c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/975672855 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde432ca0 0x559cde60cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9f9b84640 1 -- 172.21.15.131:0/975672855 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde432ca0 msgr2=0x559cde60cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9f9b84640 1 --2- 172.21.15.131:0/975672855 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde432ca0 0x559cde60cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9f9b84640 1 -- 172.21.15.131:0/975672855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde059460 msgr2=0x559cde60c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9f9b84640 1 --2- 172.21.15.131:0/975672855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde059460 0x559cde60c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9f9b84640 1 -- 172.21.15.131:0/975672855 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cde3f7db0 con 0x559cde449950 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9f9b84640 1 --2- 172.21.15.131:0/975672855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde60d0a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec001dd0 tx=0x7fb9ec001ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f41727b669e9c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9de7fc640 1 -- 172.21.15.131:0/975672855 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec019020 con 0x559cde449950 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9de7fc640 1 -- 172.21.15.131:0/975672855 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9ec010a50 con 0x559cde449950 2024-09-17T02:47:53.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.745+0000 7fb9de7fc640 1 -- 172.21.15.131:0/975672855 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec010e80 con 0x559cde449950 2024-09-17T02:47:53.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 -- 172.21.15.131:0/975672855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 msgr2=0x559cde60d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 --2- 172.21.15.131:0/975672855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde60d0a0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec001dd0 tx=0x7fb9ec001ec0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 -- 172.21.15.131:0/975672855 shutdown_connections 2024-09-17T02:47:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 --2- 172.21.15.131:0/975672855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde449950 0x559cde60d0a0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 --2- 172.21.15.131:0/975672855 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde432ca0 0x559cde60cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 --2- 172.21.15.131:0/975672855 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde059460 0x559cde60c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 -- 172.21.15.131:0/975672855 >> 172.21.15.131:0/975672855 conn(0x559cde51fd90 msgr2=0x559cde598620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 -- 172.21.15.131:0/975672855 shutdown_connections 2024-09-17T02:47:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 -- 172.21.15.131:0/975672855 wait complete. 2024-09-17T02:47:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 -- start start 2024-09-17T02:47:53.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 Processor -- start 2024-09-17T02:47:53.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde059460 0x559cde5cc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde449950 0x559cde5ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559cde39fd80 con 0x559cde432ca0 2024-09-17T02:47:53.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f40ce220 con 0x559cde059460 2024-09-17T02:47:53.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559cde3365f0 con 0x559cde449950 2024-09-17T02:47:53.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 10 client.0 _ll_register_callbacks cb 0x7fb9e40651d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9f8b82640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9f8b82640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5cc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42874/0 (socket says 172.21.15.131:42874) 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9f8b82640 1 -- 172.21.15.131:0/3013284024 learned_addr learned my addr 172.21.15.131:0/3013284024 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9f9383640 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde059460 0x559cde5cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9f8b82640 1 -- 172.21.15.131:0/3013284024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde449950 msgr2=0x559cde5ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde449950 0x559cde5ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9f8b82640 1 -- 172.21.15.131:0/3013284024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde059460 msgr2=0x559cde5cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde059460 0x559cde5cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9f8b82640 1 -- 172.21.15.131:0/3013284024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cde362ab0 con 0x559cde432ca0 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9f9383640 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde059460 0x559cde5cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9f8b82640 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5cc760 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400b690 tx=0x7fb9e4069270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d5913eb70ff6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:47:53.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9d3fff640 1 -- 172.21.15.131:0/3013284024 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e405ce80 con 0x559cde432ca0 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9d3fff640 1 -- 172.21.15.131:0/3013284024 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fb9e405c390 con 0x559cde432ca0 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 -- 172.21.15.131:0/3013284024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9ec010a50 con 0x559cde432ca0 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fb9d3fff640 1 -- 172.21.15.131:0/3013284024 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fb9e405f500 con 0x559cde432ca0 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 10 client.7299 fetch_fsmap learned FSMap version 70 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.749+0000 7fba028b2000 1 -- 172.21.15.131:0/3013284024 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x559cde341070 con 0x559cde432ca0 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9d3fff640 1 -- 172.21.15.131:0/3013284024 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9ec010a50 con 0x559cde432ca0 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9d3fff640 1 -- 172.21.15.131:0/3013284024 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7fb9e404d040 con 0x559cde432ca0 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 10 client.7299 fetch_fsmap finished waiting for FSMap version 70 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 10 client.7299 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 20 client.7299 populate_metadata read hostname 'smithi131' 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cde02c830 0x559cde5cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 4 client.7299 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 -- 172.21.15.131:0/3013284024 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "status"}) -- 0x559cde3aa540 con 0x559cde02c830 2024-09-17T02:47:53.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9f9383640 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cde02c830 0x559cde5cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9d3fff640 10 client.7299 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9f9383640 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cde02c830 0x559cde5cd660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54f83df297765d5a server_cookie=cca20cb8935d7939 in_seq=0 out_seq=0 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9d3fff640 1 -- 172.21.15.131:0/3013284024 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x559cde3aa540 con 0x559cde02c830 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9d3fff640 10 client.7299 handle_command_reply: tid=0 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 client.7299 shutdown 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 -- 172.21.15.131:0/3013284024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 msgr2=0x559cde5cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5cc760 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400b690 tx=0x7fb9e4069270 comp rx=0 tx=0).stop 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 -- 172.21.15.131:0/3013284024 shutdown_connections 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cde449950 0x559cde5ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cde02c830 0x559cde5cd660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cde432ca0 0x559cde5cc760 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 --2- 172.21.15.131:0/3013284024 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cde059460 0x559cde5cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9d3fff640 0 client.7299 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:53.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 -- 172.21.15.131:0/3013284024 >> 172.21.15.131:0/3013284024 conn(0x559cde51fd90 msgr2=0x559cde598fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 -- 172.21.15.131:0/3013284024 shutdown_connections 2024-09-17T02:47:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 1 -- 172.21.15.131:0/3013284024 wait complete. 2024-09-17T02:47:53.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fba028b2000 20 client.7299 trim_cache size 0 max 16384 2024-09-17T02:47:53.755 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:47:53.755 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:47:53.755 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:47:53.755 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7233, 2024-09-17T02:47:53.755 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:47:53.755 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:47:53.756 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:47:53.756 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 4.6460069500000003, 2024-09-17T02:47:53.756 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 70, 2024-09-17T02:47:53.756 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 95, 2024-09-17T02:47:53.756 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 95, 2024-09-17T02:47:53.756 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 5.5854066219999998 2024-09-17T02:47:53.756 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:47:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9d8073200 msgr2=0x7fb9d8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9d8073200 0x7fb9d8075710 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00af000 tx=0x7fb9f001b060 comp rx=0 tx=0).stop 2024-09-17T02:47:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2543860986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9f40ced00 msgr2=0x7fb9f414f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9f40ced00 0x7fb9f414f550 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009a50 tx=0x7fb9e4009f10 comp rx=0 tx=0).stop 2024-09-17T02:47:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2543860986 shutdown_connections 2024-09-17T02:47:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9d8073200 0x7fb9d8075710 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb9f415c3e0 0x7fb9f4157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9f40ced00 0x7fb9f414f550 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 --2- 172.21.15.131:0/2543860986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb9f406e3e0 0x7fb9f414ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2543860986 >> 172.21.15.131:0/2543860986 conn(0x7fb9f406eea0 msgr2=0x7fb9f40cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2543860986 shutdown_connections 2024-09-17T02:47:53.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.753+0000 7fb9fb5e7640 1 -- 172.21.15.131:0/2543860986 wait complete. 2024-09-17T02:47:53.770 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:47:53.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.917+0000 7fd7a1aad640 1 -- 172.21.15.131:0/2471478399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c15c3e0 msgr2=0x7fd79c0cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.917+0000 7fd7a1aad640 1 --2- 172.21.15.131:0/2471478399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c15c3e0 0x7fd79c0cbaf0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd794009c80 tx=0x7fd79401cda0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.917+0000 7fd7a1aad640 1 -- 172.21.15.131:0/2471478399 shutdown_connections 2024-09-17T02:47:53.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.917+0000 7fd7a1aad640 1 --2- 172.21.15.131:0/2471478399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c15c3e0 0x7fd79c0cbaf0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.917+0000 7fd7a1aad640 1 --2- 172.21.15.131:0/2471478399 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd79c06e5b0 0x7fd79c15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.917+0000 7fd7a1aad640 1 -- 172.21.15.131:0/2471478399 >> 172.21.15.131:0/2471478399 conn(0x7fd79c06eea0 msgr2=0x7fd79c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:53.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.917+0000 7fd7a1aad640 1 -- 172.21.15.131:0/2471478399 shutdown_connections 2024-09-17T02:47:53.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.917+0000 7fd7a1aad640 1 -- 172.21.15.131:0/2471478399 wait complete. 2024-09-17T02:47:53.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd7a1aad640 1 Processor -- start 2024-09-17T02:47:53.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd7a1aad640 1 -- start start 2024-09-17T02:47:53.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd7a1aad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0ced00 0x7fd79c14dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd7a1aad640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd79c15c3e0 0x7fd79c0b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd7a1aad640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd79c14f890 0x7fd79c0b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd7a1aad640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c1687d0 con 0x7fd79c0ced00 2024-09-17T02:47:53.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd7a1aad640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd79c168650 con 0x7fd79c15c3e0 2024-09-17T02:47:53.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd7a1aad640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd79c168950 con 0x7fd79c14f890 2024-09-17T02:47:53.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0ced00 0x7fd79c14dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd79b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0ced00 0x7fd79c14dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42892/0 (socket says 172.21.15.131:42892) 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd79b7fe640 1 -- 172.21.15.131:0/932083707 learned_addr learned my addr 172.21.15.131:0/932083707 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd79affd640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd79c15c3e0 0x7fd79c0b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd79b7fe640 1 -- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd79c14f890 msgr2=0x7fd79c0b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd79b7fe640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd79c14f890 0x7fd79c0b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd79b7fe640 1 -- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd79c15c3e0 msgr2=0x7fd79c0b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd79b7fe640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd79c15c3e0 0x7fd79c0b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd79b7fe640 1 -- 172.21.15.131:0/932083707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd79c1fa180 con 0x7fd79c0ced00 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd79b7fe640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0ced00 0x7fd79c14dbf0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd78c001ec0 tx=0x7fd78c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd783fff640 1 -- 172.21.15.131:0/932083707 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c017070 con 0x7fd79c0ced00 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd783fff640 1 -- 172.21.15.131:0/932083707 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd78c003600 con 0x7fd79c0ced00 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd783fff640 1 -- 172.21.15.131:0/932083707 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd78c003a50 con 0x7fd79c0ced00 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd7a1aad640 1 -- 172.21.15.131:0/932083707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd79c1f93c0 con 0x7fd79c0ced00 2024-09-17T02:47:53.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.921+0000 7fd7a1aad640 1 -- 172.21.15.131:0/932083707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd79c1f9980 con 0x7fd79c0ced00 2024-09-17T02:47:53.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.925+0000 7fd7a1aad640 1 -- 172.21.15.131:0/932083707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd79c15bea0 con 0x7fd79c0ced00 2024-09-17T02:47:53.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.925+0000 7fd783fff640 1 -- 172.21.15.131:0/932083707 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd78c01a070 con 0x7fd79c0ced00 2024-09-17T02:47:53.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.925+0000 7fd783fff640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd788073200 0x7fd788075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:53.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.925+0000 7fd783fff640 1 -- 172.21.15.131:0/932083707 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fd78c093cc0 con 0x7fd79c0ced00 2024-09-17T02:47:53.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.925+0000 7fd79affd640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd788073200 0x7fd788075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:53.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.925+0000 7fd79affd640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd788073200 0x7fd788075710 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd79c1566a0 tx=0x7fd790005bf0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:53.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:53.929+0000 7fd783fff640 1 -- 172.21.15.131:0/932083707 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd78c021020 con 0x7fd79c0ced00 2024-09-17T02:47:54.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.117+0000 7fd7a1aad640 1 -- 172.21.15.131:0/932083707 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd79c0b0470 con 0x7fd79c0ced00 2024-09-17T02:47:54.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.117+0000 7fd783fff640 1 -- 172.21.15.131:0/932083707 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7fd78c05ea40 con 0x7fd79c0ced00 2024-09-17T02:47:54.116 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:47:54.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 -- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd788073200 msgr2=0x7fd788075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:54.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd788073200 0x7fd788075710 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd79c1566a0 tx=0x7fd790005bf0 comp rx=0 tx=0).stop 2024-09-17T02:47:54.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 -- 172.21.15.131:0/932083707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0ced00 msgr2=0x7fd79c14dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:54.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0ced00 0x7fd79c14dbf0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd78c001ec0 tx=0x7fd78c00c890 comp rx=0 tx=0).stop 2024-09-17T02:47:54.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 -- 172.21.15.131:0/932083707 shutdown_connections 2024-09-17T02:47:54.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd788073200 0x7fd788075710 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:54.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd79c14f890 0x7fd79c0b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:54.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd79c15c3e0 0x7fd79c0b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:54.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 --2- 172.21.15.131:0/932083707 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd79c0ced00 0x7fd79c14dbf0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:54.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 -- 172.21.15.131:0/932083707 >> 172.21.15.131:0/932083707 conn(0x7fd79c06eea0 msgr2=0x7fd79c0cb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:54.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 -- 172.21.15.131:0/932083707 shutdown_connections 2024-09-17T02:47:54.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54.121+0000 7fd7a1aad640 1 -- 172.21.15.131:0/932083707 wait complete. 2024-09-17T02:47:54.139 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:47:54.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:47:54.140 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:47:54.140 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:47:54.140 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:54.140 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:47:54.140 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:47:54.140 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:47:54.140 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:47:54.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:54.140 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:47:54.146 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:47:54.147 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 42032sec preferred_lft 42032sec 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:47:54.148 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:54.149 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:47:54.149 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:54.149 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:47:54.149 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:54.149 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:47:54.149 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:54.149 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:47:54.149 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:54.149 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:47:54.149 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:47:54.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:54.149 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:47:54.199 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:47:54.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:54.200 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:47:54.253 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:47:54.253 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:47:54.253 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:47:54.253 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:47:54.253 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:47:54.253 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:47:54.253 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:47:54.253 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:47:54.253 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:47:54.253 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:47:54.253 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:47:54.254 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:47:54.254 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:47:54.254 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:47:54.254 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:47:54.254 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:47:54.254 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:47:54.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:47:54.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:47:54.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:54.373 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:47:54.420 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:47:54.421 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:47:54.421 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:47:54.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:54.421 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:47:54.477 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:47:54.477 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:47:54.477 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:54.525 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:47:54.525 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:47:54.525 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:54.573 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:47:54.631 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:47:54.679 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:47:54.680 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:47:54.681 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:47:54.682 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:47:54.682 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:47:54.682 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:47:54.682 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:47:54.682 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:54.682 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:54.682 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:54.682 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:54.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:54.683 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:47:54.733 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:47:54.733 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:47:54.778 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:47:54.792 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:47:54.841 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:47:54.841 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:47:54.841 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:47:54.842 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:47:54.843 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:54.844 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:54.844 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:54.844 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:54.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:54.845 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:47:54.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:47:54.893+0000 7efc06e8e580 -1 init, newargv = 0x7efbe8001f10 newargc=15 2024-09-17T02:47:54.893 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[46721]: starting ceph client 2024-09-17T02:47:54.899 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[46721]: starting fuse 2024-09-17T02:47:55.105 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-09-17T02:47:55.109+0000 7f9a5b4bc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:47:56.000 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:47:56.017 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:47:56.067 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:47:56.068 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:47:56.069 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:47:56.070 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:47:56.070 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:47:56.070 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:47:56.070 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:56.070 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:47:56.070 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:56.070 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:47:56.070 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:56.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:56.071 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:47:56.117 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:47:56.118 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:47:56.118 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:47:56.118 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:47:56.118 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:47:56.118 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:47:56.118 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:47:56.118 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:56.118 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:47:56.118 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:47:56.118 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:47:56.119 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:47:56.120 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:47:56.120 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:47:56.120 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:47:56.120 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:47:56.246 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.46721.asok 2024-09-17T02:47:56.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:47:56.259 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46721.asok 2024-09-17T02:47:56.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:56.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46721.asok status 2024-09-17T02:47:56.436 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:47:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:47:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:47:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:47:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:47:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:47:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:47:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "46721", 2024-09-17T02:47:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:47:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:47:56.437 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7317, 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 7317 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2800268708 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:47:56.438 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2800268708 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.7317 192.168.0.1:0/2800268708", 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/2800268708", 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 70, 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 95, 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:47:56.439 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:47:56.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:56.453 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:47:56.453 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:56.459 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:47:56.459 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:56.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:56.459 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:47:56.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:56.512 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-09-17T02:47:56.568 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-09-17T02:47:56.569+0000 7f9a574b4640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-09-17T02:47:56.569 INFO:teuthology.orchestra.run.smithi131.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-09-17T02:47:56.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:47:56.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:56.570 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-09-17T02:47:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.789+0000 7f5cd97b2640 1 -- 172.21.15.131:0/1764950496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd40cc710 msgr2=0x7f5cd40ccb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.789+0000 7f5cd97b2640 1 --2- 172.21.15.131:0/1764950496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd40cc710 0x7f5cd40ccb20 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009900 tx=0x7f5cbc01c8b0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.789+0000 7f5cd97b2640 1 -- 172.21.15.131:0/1764950496 shutdown_connections 2024-09-17T02:47:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.789+0000 7f5cd97b2640 1 --2- 172.21.15.131:0/1764950496 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5cd40d0dd0 0x7f5cd40d11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.789+0000 7f5cd97b2640 1 --2- 172.21.15.131:0/1764950496 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd40cc710 0x7f5cd40ccb20 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.789+0000 7f5cd97b2640 1 -- 172.21.15.131:0/1764950496 >> 172.21.15.131:0/1764950496 conn(0x7f5cd40b4010 msgr2=0x7f5cd40b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.789+0000 7f5cd97b2640 1 -- 172.21.15.131:0/1764950496 shutdown_connections 2024-09-17T02:47:56.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.789+0000 7f5cd97b2640 1 -- 172.21.15.131:0/1764950496 wait complete. 2024-09-17T02:47:56.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd97b2640 1 Processor -- start 2024-09-17T02:47:56.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd97b2640 1 -- start start 2024-09-17T02:47:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd97b2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd40cc710 0x7f5cd4159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd97b2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5cd40d0dd0 0x7f5cd4154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd97b2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd415da00 0x7f5cd4155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd97b2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4169cc0 con 0x7f5cd40cc710 2024-09-17T02:47:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd97b2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4169b40 con 0x7f5cd415da00 2024-09-17T02:47:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd97b2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd4169e40 con 0x7f5cd40d0dd0 2024-09-17T02:47:56.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd40cc710 0x7f5cd4159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd40cc710 0x7f5cd4159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42900/0 (socket says 172.21.15.131:42900) 2024-09-17T02:47:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/3505360068 learned_addr learned my addr 172.21.15.131:0/3505360068 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd37fe640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd415da00 0x7f5cd4155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5cd40d0dd0 0x7f5cd4154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5cd40d0dd0 msgr2=0x7f5cd4154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5cd40d0dd0 0x7f5cd4154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd415da00 msgr2=0x7f5cd4155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd415da00 0x7f5cd4155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/3505360068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd40b4930 con 0x7f5cd40cc710 2024-09-17T02:47:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd37fe640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd415da00 0x7f5cd4155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:47:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd40cc710 0x7f5cd4159650 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c3f0 tx=0x7f5cc400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5ccb7fe640 1 -- 172.21.15.131:0/3505360068 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4017070 con 0x7f5cd40cc710 2024-09-17T02:47:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5ccb7fe640 1 -- 172.21.15.131:0/3505360068 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cc40053c0 con 0x7f5cd40cc710 2024-09-17T02:47:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd97b2640 1 -- 172.21.15.131:0/3505360068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd40c53c0 con 0x7f5cd40cc710 2024-09-17T02:47:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5ccb7fe640 1 -- 172.21.15.131:0/3505360068 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4003860 con 0x7f5cd40cc710 2024-09-17T02:47:56.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.793+0000 7f5cd97b2640 1 -- 172.21.15.131:0/3505360068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd41f7b80 con 0x7f5cd40cc710 2024-09-17T02:47:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ccb7fe640 1 -- 172.21.15.131:0/3505360068 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5cc401a070 con 0x7f5cd40cc710 2024-09-17T02:47:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ccb7fe640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5cb4073200 0x7f5cb4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ccb7fe640 1 -- 172.21.15.131:0/3505360068 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f5cc4093840 con 0x7f5cd40cc710 2024-09-17T02:47:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5cb4073200 0x7f5cb4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ce0a7d000 1 Processor -- start 2024-09-17T02:47:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5cb4073200 0x7f5cb4075710 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0af000 tx=0x7f5cbc01b0f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:56.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ce0a7d000 1 -- start start 2024-09-17T02:47:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc880fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc88102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc8810830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559cc85c4460 con 0x559cc86b0a50 2024-09-17T02:47:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559cc8621ce0 con 0x559cc869d140 2024-09-17T02:47:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559cc8611110 con 0x559cc8696db0 2024-09-17T02:47:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5cd2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc880fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5cd2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc880fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42908/0 (socket says 172.21.15.131:42908) 2024-09-17T02:47:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/1621978148 learned_addr learned my addr 172.21.15.131:0/1621978148 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.797+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/1621978148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc88102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5cd37fe640 1 --2- 172.21.15.131:0/1621978148 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc8810830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/1621978148 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 msgr2=0x559cc8810830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/1621978148 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc8810830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/1621978148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 msgr2=0x559cc88102f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/1621978148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc88102f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/1621978148 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cc85af340 con 0x559cc86b0a50 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/1621978148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc880fdb0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5cc405de40 tx=0x7f5cc405da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1641251d44d32b82 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5cc8ff9640 1 -- 172.21.15.131:0/1621978148 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc405deb0 con 0x559cc86b0a50 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5cc8ff9640 1 -- 172.21.15.131:0/1621978148 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cc4067840 con 0x559cc86b0a50 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5cc8ff9640 1 -- 172.21.15.131:0/1621978148 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc405d110 con 0x559cc86b0a50 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/1621978148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 msgr2=0x559cc880fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/1621978148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc880fdb0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5cc405de40 tx=0x7f5cc405da80 comp rx=0 tx=0).stop 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/1621978148 shutdown_connections 2024-09-17T02:47:56.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/1621978148 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc8810830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/1621978148 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc88102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/1621978148 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc880fdb0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/1621978148 >> 172.21.15.131:0/1621978148 conn(0x559cc82cf590 msgr2=0x559cc87c4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:56.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/1621978148 shutdown_connections 2024-09-17T02:47:56.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/1621978148 wait complete. 2024-09-17T02:47:56.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 -- start start 2024-09-17T02:47:56.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 Processor -- start 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc87cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc869d140 0x559cc87cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc86b0a50 0x559cc87fb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559cc85b6600 con 0x559cc869d140 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559cc8614f20 con 0x559cc86b0a50 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559cc82d6810 con 0x559cc8696db0 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ce0a7d000 10 client.0 _ll_register_callbacks cb 0x559cc87b1cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ccbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc869d140 0x559cc87cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ccbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc869d140 0x559cc87cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42912/0 (socket says 172.21.15.131:42912) 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.801+0000 7f5ccbfff640 1 -- 172.21.15.131:0/3391330577 learned_addr learned my addr 172.21.15.131:0/3391330577 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cd37fe640 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc86b0a50 0x559cc87fb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc87cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ccbfff640 1 -- 172.21.15.131:0/3391330577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 msgr2=0x559cc87cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc87cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ccbfff640 1 -- 172.21.15.131:0/3391330577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc86b0a50 msgr2=0x559cc87fb680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc86b0a50 0x559cc87fb680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ccbfff640 1 -- 172.21.15.131:0/3391330577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cc85961f0 con 0x559cc869d140 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc869d140 0x559cc87cc1d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009230 tx=0x7f5cbc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f1f522a86323e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/3391330577 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0c8060 con 0x559cc869d140 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/3391330577 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f5cbc0a5df0 con 0x559cc869d140 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/3391330577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559cc85a9480 con 0x559cc869d140 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/3391330577 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f5cbc0ce050 con 0x559cc869d140 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 10 client.7329 fetch_fsmap learned FSMap version 70 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/3391330577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x559cc86961f0 con 0x559cc869d140 2024-09-17T02:47:56.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/3391330577 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559cc85a9480 con 0x559cc869d140 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/3391330577 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f5cbc0d1020 con 0x559cc869d140 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 10 client.7329 fetch_fsmap finished waiting for FSMap version 70 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 10 client.7329 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 20 client.7329 populate_metadata read hostname 'smithi131' 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cc82b3ac0 0x559cc8817030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 4 client.7329 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/3391330577 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cc8623c30 con 0x559cc82b3ac0 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cc82b3ac0 0x559cc8817030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cb1ffb640 10 client.7329 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cc82b3ac0 0x559cc8817030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb6b696f2403562b server_cookie=689119d87a2f862 in_seq=0 out_seq=0 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/3391330577 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559cc8623c30 con 0x559cc82b3ac0 2024-09-17T02:47:56.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cb1ffb640 10 client.7329 handle_command_reply: tid=0 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 client.7329 shutdown 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/3391330577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc869d140 msgr2=0x559cc87cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc869d140 0x559cc87cc1d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009230 tx=0x7f5cbc009260 comp rx=0 tx=0).stop 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/3391330577 shutdown_connections 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc86b0a50 0x559cc87fb680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cc82b3ac0 0x559cc8817030 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc869d140 0x559cc87cc1d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/3391330577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc87cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5cb1ffb640 0 client.7329 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/3391330577 >> 172.21.15.131:0/3391330577 conn(0x559cc82cf590 msgr2=0x559cc87c52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/3391330577 shutdown_connections 2024-09-17T02:47:56.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/3391330577 wait complete. 2024-09-17T02:47:56.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.805+0000 7f5ce0a7d000 20 client.7329 trim_cache size 0 max 16384 2024-09-17T02:47:56.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5ce0a7d000 1 Processor -- start 2024-09-17T02:47:56.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5ce0a7d000 1 -- start start 2024-09-17T02:47:56.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc87c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc88279f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc8827f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4203d60 con 0x559cc86b0a50 2024-09-17T02:47:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4202d80 con 0x559cc869d140 2024-09-17T02:47:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd41f8f20 con 0x559cc8696db0 2024-09-17T02:47:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cd37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc8827f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cd37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc8827f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42924/0 (socket says 172.21.15.131:42924) 2024-09-17T02:47:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cd37fe640 1 -- 172.21.15.131:0/2607620215 learned_addr learned my addr 172.21.15.131:0/2607620215 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/2607620215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc88279f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/2607620215 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc87c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cd37fe640 1 -- 172.21.15.131:0/2607620215 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 msgr2=0x559cc87c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cd37fe640 1 --2- 172.21.15.131:0/2607620215 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc87c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cd37fe640 1 -- 172.21.15.131:0/2607620215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 msgr2=0x559cc88279f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cd37fe640 1 --2- 172.21.15.131:0/2607620215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc88279f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cd37fe640 1 -- 172.21.15.131:0/2607620215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cc866d4f0 con 0x559cc86b0a50 2024-09-17T02:47:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cd37fe640 1 --2- 172.21.15.131:0/2607620215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc8827f30 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00aee0 tx=0x7f5ccc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba5dead5ee791da server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cb3fff640 1 -- 172.21.15.131:0/2607620215 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc004b30 con 0x559cc86b0a50 2024-09-17T02:47:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cb3fff640 1 -- 172.21.15.131:0/2607620215 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ccc004cd0 con 0x559cc86b0a50 2024-09-17T02:47:56.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.825+0000 7f5cb3fff640 1 -- 172.21.15.131:0/2607620215 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc00f870 con 0x559cc86b0a50 2024-09-17T02:47:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/2607620215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 msgr2=0x559cc8827f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/2607620215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc8827f30 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00aee0 tx=0x7f5ccc010800 comp rx=0 tx=0).stop 2024-09-17T02:47:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/2607620215 shutdown_connections 2024-09-17T02:47:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/2607620215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc86b0a50 0x559cc8827f30 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/2607620215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc88279f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/2607620215 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc8696db0 0x559cc87c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/2607620215 >> 172.21.15.131:0/2607620215 conn(0x559cc82cf590 msgr2=0x559cc880ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/2607620215 shutdown_connections 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/2607620215 wait complete. 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 -- start start 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 Processor -- start 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc8696db0 0x559cc8889970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc8889eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc86b0a50 0x559cc888a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559cc85dcdf0 con 0x559cc8696db0 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559cc8614230 con 0x559cc869d140 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559cc8619550 con 0x559cc86b0a50 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ce0a7d000 10 client.0 _ll_register_callbacks cb 0x7f5cbc0b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5cd2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc8696db0 0x559cc8889970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5cd2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc8696db0 0x559cc8889970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42934/0 (socket says 172.21.15.131:42934) 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/330114558 learned_addr learned my addr 172.21.15.131:0/330114558 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc8889eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5cd37fe640 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc86b0a50 0x559cc888a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/330114558 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc86b0a50 msgr2=0x559cc888a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc86b0a50 0x559cc888a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/330114558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 msgr2=0x559cc8889eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc8889eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5cd2ffd640 1 -- 172.21.15.131:0/330114558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cc8596a10 con 0x559cc8696db0 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc8889eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5cd2ffd640 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc8696db0 0x559cc8889970 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4091a30 tx=0x7f5cc4091900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd4b7bee5843869 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.829+0000 7f5cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/330114558 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4077070 con 0x559cc8696db0 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/330114558 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f5cc405d110 con 0x559cc8696db0 2024-09-17T02:47:56.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/330114558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ccc004cd0 con 0x559cc8696db0 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/330114558 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f5cc4067840 con 0x559cc8696db0 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 10 client.7335 fetch_fsmap learned FSMap version 70 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/330114558 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x559cc85a7740 con 0x559cc8696db0 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/330114558 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ccc004cd0 con 0x559cc8696db0 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/330114558 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f5cc4063b20 con 0x559cc8696db0 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 10 client.7335 fetch_fsmap finished waiting for FSMap version 70 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 10 client.7335 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 20 client.7335 populate_metadata read hostname 'smithi131' 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cc82a1b20 0x559cc8844f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 4 client.7335 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/330114558 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x559cc87cc480 con 0x559cc82a1b20 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cc82a1b20 0x559cc8844f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:56.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cb1ffb640 10 client.7335 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ccbfff640 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cc82a1b20 0x559cc8844f90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9317433f6c27e1d server_cookie=7ee844eb0a6a38ff in_seq=0 out_seq=0 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cb1ffb640 1 -- 172.21.15.131:0/330114558 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559cc87cc480 con 0x559cc82a1b20 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cb1ffb640 10 client.7335 handle_command_reply: tid=0 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 client.7335 shutdown 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/330114558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc8696db0 msgr2=0x559cc8889970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc8696db0 0x559cc8889970 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4091a30 tx=0x7f5cc4091900 comp rx=0 tx=0).stop 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/330114558 shutdown_connections 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559cc86b0a50 0x559cc888a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559cc869d140 0x559cc8889eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x559cc82a1b20 0x559cc8844f90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 --2- 172.21.15.131:0/330114558 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559cc8696db0 0x559cc8889970 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cb1ffb640 0 client.7335 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/330114558 >> 172.21.15.131:0/330114558 conn(0x559cc82cf590 msgr2=0x559cc880f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:56.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/330114558 shutdown_connections 2024-09-17T02:47:56.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 1 -- 172.21.15.131:0/330114558 wait complete. 2024-09-17T02:47:56.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5ce0a7d000 20 client.7335 trim_cache size 0 max 16384 2024-09-17T02:47:56.836 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f9a574b4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-09-17T02:47:39.862410+0000 1=1+0) n(v0 rc2024-09-17T02:47:39.976065+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55af17c62100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-17T02:47:39.862410+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-17T02:47:39.976065+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-09-17T02:47:56.836 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:47:56.836 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-09-17T02:47:56.836 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "9441c6a2-2616-4df3-82fb-d48465ab6abc", 2024-09-17T02:47:56.836 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-09-17T02:47:56.837 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:47:56.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 -- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5cb4073200 msgr2=0x7f5cb4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:56.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5cb4073200 0x7f5cb4075710 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0af000 tx=0x7f5cbc01b0f0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 -- 172.21.15.131:0/3505360068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd40cc710 msgr2=0x7f5cd4159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:56.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd40cc710 0x7f5cd4159650 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400c3f0 tx=0x7f5cc400c8b0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 -- 172.21.15.131:0/3505360068 shutdown_connections 2024-09-17T02:47:56.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5cb4073200 0x7f5cb4075710 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5cd415da00 0x7f5cd4155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5cd40d0dd0 0x7f5cd4154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 --2- 172.21.15.131:0/3505360068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5cd40cc710 0x7f5cd4159650 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:56.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 -- 172.21.15.131:0/3505360068 >> 172.21.15.131:0/3505360068 conn(0x7f5cd40b4010 msgr2=0x7f5cd40d2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:56.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 -- 172.21.15.131:0/3505360068 shutdown_connections 2024-09-17T02:47:56.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:56.833+0000 7f5cd97b2640 1 -- 172.21.15.131:0/3505360068 wait complete. 2024-09-17T02:47:56.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:47:56.850 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-17T02:47:57.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.005+0000 7fcfa0955640 1 -- 172.21.15.131:0/273565061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf84003a10 msgr2=0x7fcf84003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:57.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.005+0000 7fcfa0955640 1 --2- 172.21.15.131:0/273565061 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf84003a10 0x7fcf84003e80 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009b10 tx=0x7fcf9001d270 comp rx=0 tx=0).stop 2024-09-17T02:47:57.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.005+0000 7fcf9ad9d640 1 -- 172.21.15.131:0/273565061 reap_dead start 2024-09-17T02:47:57.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.005+0000 7fcfa0955640 1 -- 172.21.15.131:0/273565061 shutdown_connections 2024-09-17T02:47:57.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.005+0000 7fcfa0955640 1 -- 172.21.15.131:0/273565061 >> 172.21.15.131:0/273565061 conn(0x7fcf9c0d1b10 msgr2=0x7fcf9c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:57.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.005+0000 7fcfa0955640 1 -- 172.21.15.131:0/273565061 shutdown_connections 2024-09-17T02:47:57.004 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.005+0000 7fcfa0955640 1 -- 172.21.15.131:0/273565061 wait complete. 2024-09-17T02:47:57.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcfa0955640 1 Processor -- start 2024-09-17T02:47:57.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcfa0955640 1 -- start start 2024-09-17T02:47:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcfa0955640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf84003a10 0x7fcf9c0cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcfa0955640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf9c15c3b0 0x7fcf9c159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcfa0955640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c15d180 0x7fcf9c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcfa0955640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c15fd60 con 0x7fcf9c15d180 2024-09-17T02:47:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcfa0955640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c15fbe0 con 0x7fcf9c15c3b0 2024-09-17T02:47:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcfa0955640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9c169da0 con 0x7fcf84003a10 2024-09-17T02:47:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf9ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c15d180 0x7fcf9c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf9ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c15d180 0x7fcf9c15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42962/0 (socket says 172.21.15.131:42962) 2024-09-17T02:47:57.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf9ad9d640 1 -- 172.21.15.131:0/1030221798 learned_addr learned my addr 172.21.15.131:0/1030221798 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf9a59c640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf84003a10 0x7fcf9c0cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf9c15c3b0 0x7fcf9c159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf9ad9d640 1 -- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf84003a10 msgr2=0x7fcf9c0cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf9ad9d640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf84003a10 0x7fcf9c0cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf9ad9d640 1 -- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf9c15c3b0 msgr2=0x7fcf9c159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf9ad9d640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf9c15c3b0 0x7fcf9c159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf9ad9d640 1 -- 172.21.15.131:0/1030221798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c15dad0 con 0x7fcf9c15d180 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf9c15c3b0 0x7fcf9c159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf9a59c640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf84003a10 0x7fcf9c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf9ad9d640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c15d180 0x7fcf9c15d590 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c08d6d0 tx=0x7fcf9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf8affd640 1 -- 172.21.15.131:0/1030221798 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94004d30 con 0x7fcf9c15d180 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf8affd640 1 -- 172.21.15.131:0/1030221798 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf9400cad0 con 0x7fcf9c15d180 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcfa0955640 1 -- 172.21.15.131:0/1030221798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c0cbdb0 con 0x7fcf9c15d180 2024-09-17T02:47:57.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcf8affd640 1 -- 172.21.15.131:0/1030221798 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94003980 con 0x7fcf9c15d180 2024-09-17T02:47:57.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.009+0000 7fcfa0955640 1 -- 172.21.15.131:0/1030221798 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c15b8d0 con 0x7fcf9c15d180 2024-09-17T02:47:57.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.013+0000 7fcf8affd640 1 -- 172.21.15.131:0/1030221798 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcf94010430 con 0x7fcf9c15d180 2024-09-17T02:47:57.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.013+0000 7fcf8affd640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcf7c073200 0x7fcf7c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.013+0000 7fcf8affd640 1 -- 172.21.15.131:0/1030221798 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fcf940949f0 con 0x7fcf9c15d180 2024-09-17T02:47:57.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.013+0000 7fcf9a59c640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcf7c073200 0x7fcf7c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.013+0000 7fcf9a59c640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcf7c073200 0x7fcf7c075710 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009b10 tx=0x7fcf9001a010 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:57.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.013+0000 7fcfa7c20000 1 Processor -- start 2024-09-17T02:47:57.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.013+0000 7fcfa7c20000 1 -- start start 2024-09-17T02:47:57.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787bfa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 0x55c787bfaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bfb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c787a99cd0 con 0x55c7877a9aa0 2024-09-17T02:47:57.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c7879af330 con 0x55c787a8e6d0 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55c787a0cbb0 con 0x55c7876e5890 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf9ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bfb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf9ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bfb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42974/0 (socket says 172.21.15.131:42974) 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf9ad9d640 1 -- 172.21.15.131:0/2530842530 learned_addr learned my addr 172.21.15.131:0/2530842530 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/2530842530 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 0x55c787bfaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf9a59c640 1 --2- 172.21.15.131:0/2530842530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787bfa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf9ad9d640 1 -- 172.21.15.131:0/2530842530 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 msgr2=0x55c787bfaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf9ad9d640 1 --2- 172.21.15.131:0/2530842530 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 0x55c787bfaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf9ad9d640 1 -- 172.21.15.131:0/2530842530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 msgr2=0x55c787bfa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf9ad9d640 1 --2- 172.21.15.131:0/2530842530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787bfa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf9ad9d640 1 -- 172.21.15.131:0/2530842530 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7879b0c80 con 0x55c7877a9aa0 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf9ad9d640 1 --2- 172.21.15.131:0/2530842530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bfb470 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009420 tx=0x7fcf9405eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469883a573750d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:57.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf7bfff640 1 -- 172.21.15.131:0/2530842530 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9401d040 con 0x55c7877a9aa0 2024-09-17T02:47:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf7bfff640 1 -- 172.21.15.131:0/2530842530 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf94063c70 con 0x55c7877a9aa0 2024-09-17T02:47:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcf7bfff640 1 -- 172.21.15.131:0/2530842530 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94068b20 con 0x55c7877a9aa0 2024-09-17T02:47:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 -- 172.21.15.131:0/2530842530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 msgr2=0x55c787bfb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/2530842530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bfb470 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fcf94009420 tx=0x7fcf9405eca0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 -- 172.21.15.131:0/2530842530 shutdown_connections 2024-09-17T02:47:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/2530842530 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bfb470 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/2530842530 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 0x55c787bfaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/2530842530 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787bfa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 -- 172.21.15.131:0/2530842530 >> 172.21.15.131:0/2530842530 conn(0x55c7876b95c0 msgr2=0x55c787bb7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 -- 172.21.15.131:0/2530842530 shutdown_connections 2024-09-17T02:47:57.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 -- 172.21.15.131:0/2530842530 wait complete. 2024-09-17T02:47:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 -- start start 2024-09-17T02:47:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.017+0000 7fcfa7c20000 1 Processor -- start 2024-09-17T02:47:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 0x55c787bb69b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bb6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787bbd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c7879ffdf0 con 0x55c7877a9aa0 2024-09-17T02:47:57.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c7879fbfe0 con 0x55c787a8e6d0 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55c7879a14d0 con 0x55c7876e5890 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 10 client.0 _ll_register_callbacks cb 0x55c78797f7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf99d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bb6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf99d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bb6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42978/0 (socket says 172.21.15.131:42978) 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf99d9b640 1 -- 172.21.15.131:0/3903008261 learned_addr learned my addr 172.21.15.131:0/3903008261 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf9ad9d640 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787bbd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf9a59c640 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 0x55c787bb69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf99d9b640 1 -- 172.21.15.131:0/3903008261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 msgr2=0x55c787bb69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 0x55c787bb69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf99d9b640 1 -- 172.21.15.131:0/3903008261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 msgr2=0x55c787bbd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787bbd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf99d9b640 1 -- 172.21.15.131:0/3903008261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78799a210 con 0x55c7877a9aa0 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bb6ef0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00f480 tx=0x7fcf8c00f940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8360d2e205e8eda2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf797fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf797fa640 1 -- 172.21.15.131:0/3903008261 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0052e0 con 0x55c7877a9aa0 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf797fa640 1 -- 172.21.15.131:0/3903008261 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fcf8c003600 con 0x55c7877a9aa0 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 -- 172.21.15.131:0/3903008261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c787994350 con 0x55c7877a9aa0 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf797fa640 1 -- 172.21.15.131:0/3903008261 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fcf8c020050 con 0x55c7877a9aa0 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 10 client.7347 fetch_fsmap learned FSMap version 70 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 -- 172.21.15.131:0/3903008261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55c787980fc0 con 0x55c7877a9aa0 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf797fa640 1 -- 172.21.15.131:0/3903008261 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c787994350 con 0x55c7877a9aa0 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf797fa640 1 -- 172.21.15.131:0/3903008261 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7fcf8c023020 con 0x55c7877a9aa0 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 10 client.7347 fetch_fsmap finished waiting for FSMap version 70 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 10 client.7347 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 20 client.7347 populate_metadata read hostname 'smithi131' 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55c78769e930 0x55c787c01e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 4 client.7347 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:47:57.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 -- 172.21.15.131:0/3903008261 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c787a0eb00 con 0x55c78769e930 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf9a59c640 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55c78769e930 0x55c787c01e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf797fa640 10 client.7347 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf9a59c640 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55c78769e930 0x55c787c01e80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167587fc228f9b88 server_cookie=c738a4dc92bfda14 in_seq=0 out_seq=0 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf797fa640 1 -- 172.21.15.131:0/3903008261 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c787a0eb00 con 0x55c78769e930 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf797fa640 10 client.7347 handle_command_reply: tid=0 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 client.7347 shutdown 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 -- 172.21.15.131:0/3903008261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 msgr2=0x55c787bb6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bb6ef0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00f480 tx=0x7fcf8c00f940 comp rx=0 tx=0).stop 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 -- 172.21.15.131:0/3903008261 shutdown_connections 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787bbd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55c78769e930 0x55c787c01e80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787bb6ef0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/3903008261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 0x55c787bb69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcf797fa640 0 client.7347 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:57.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 -- 172.21.15.131:0/3903008261 >> 172.21.15.131:0/3903008261 conn(0x55c7876b95c0 msgr2=0x55c787bafd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:57.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 -- 172.21.15.131:0/3903008261 shutdown_connections 2024-09-17T02:47:57.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 1 -- 172.21.15.131:0/3903008261 wait complete. 2024-09-17T02:47:57.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.021+0000 7fcfa7c20000 20 client.7347 trim_cache size 0 max 16384 2024-09-17T02:47:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcfa7c20000 1 Processor -- start 2024-09-17T02:47:57.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcfa7c20000 1 -- start start 2024-09-17T02:47:57.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 0x55c787c6c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c6c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787c6ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c787bb5b20 con 0x55c7877a9aa0 2024-09-17T02:47:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c206580 con 0x55c787a8e6d0 2024-09-17T02:47:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9c205560 con 0x55c7876e5890 2024-09-17T02:47:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf99d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c6c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.043 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf99d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c6c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42986/0 (socket says 172.21.15.131:42986) 2024-09-17T02:47:57.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf99d9b640 1 -- 172.21.15.131:0/1107065343 learned_addr learned my addr 172.21.15.131:0/1107065343 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:57.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf99d9b640 1 -- 172.21.15.131:0/1107065343 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 msgr2=0x55c787c6c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:57.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/1107065343 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 0x55c787c6c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf99d9b640 1 -- 172.21.15.131:0/1107065343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 msgr2=0x55c787c6ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:57.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf9ad9d640 1 --2- 172.21.15.131:0/1107065343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787c6ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/1107065343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787c6ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf99d9b640 1 -- 172.21.15.131:0/1107065343 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c787a59680 con 0x55c7877a9aa0 2024-09-17T02:47:57.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/1107065343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c6c940 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c011010 tx=0x7fcf8c003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f616984ab2853c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:57.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf7b7fe640 1 -- 172.21.15.131:0/1107065343 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0052e0 con 0x55c7877a9aa0 2024-09-17T02:47:57.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf7b7fe640 1 -- 172.21.15.131:0/1107065343 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf8c018070 con 0x55c7877a9aa0 2024-09-17T02:47:57.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.041+0000 7fcf7b7fe640 1 -- 172.21.15.131:0/1107065343 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c003b40 con 0x55c7877a9aa0 2024-09-17T02:47:57.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 -- 172.21.15.131:0/1107065343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 msgr2=0x55c787c6c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:57.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/1107065343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c6c940 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c011010 tx=0x7fcf8c003850 comp rx=0 tx=0).stop 2024-09-17T02:47:57.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 -- 172.21.15.131:0/1107065343 shutdown_connections 2024-09-17T02:47:57.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/1107065343 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c787a8e6d0 0x55c787c6ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/1107065343 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c6c940 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/1107065343 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c7876e5890 0x55c787c6c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 -- 172.21.15.131:0/1107065343 >> 172.21.15.131:0/1107065343 conn(0x55c7876b95c0 msgr2=0x55c787c0cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:57.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 -- 172.21.15.131:0/1107065343 shutdown_connections 2024-09-17T02:47:57.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 -- 172.21.15.131:0/1107065343 wait complete. 2024-09-17T02:47:57.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 -- start start 2024-09-17T02:47:57.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 Processor -- start 2024-09-17T02:47:57.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7876e5890 0x55c787c70650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c70b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c787a8e6d0 0x55c787c710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c204e90 con 0x55c7877a9aa0 2024-09-17T02:47:57.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c1fcba0 con 0x55c7876e5890 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55c7876c1680 con 0x55c787a8e6d0 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 10 client.0 _ll_register_callbacks cb 0x7fcf8c001440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf99d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c70b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf99d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c70b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:42998/0 (socket says 172.21.15.131:42998) 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf99d9b640 1 -- 172.21.15.131:0/929569760 learned_addr learned my addr 172.21.15.131:0/929569760 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf9a59c640 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7876e5890 0x55c787c70650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf99d9b640 1 -- 172.21.15.131:0/929569760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c787a8e6d0 msgr2=0x55c787c710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c787a8e6d0 0x55c787c710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf99d9b640 1 -- 172.21.15.131:0/929569760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7876e5890 msgr2=0x55c787c70650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7876e5890 0x55c787c70650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf99d9b640 1 -- 172.21.15.131:0/929569760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7879c60b0 con 0x55c7877a9aa0 2024-09-17T02:47:57.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf99d9b640 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c70b90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c001d80 tx=0x7fcf8c003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75089c55bc7ef327 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf797fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf797fa640 1 -- 172.21.15.131:0/929569760 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0038f0 con 0x55c7877a9aa0 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcf797fa640 1 -- 172.21.15.131:0/929569760 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fcf8c01b560 con 0x55c7877a9aa0 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.045+0000 7fcfa7c20000 1 -- 172.21.15.131:0/929569760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf8c018070 con 0x55c7877a9aa0 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcf797fa640 1 -- 172.21.15.131:0/929569760 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fcf8c037050 con 0x55c7877a9aa0 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 10 client.7353 fetch_fsmap learned FSMap version 70 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 -- 172.21.15.131:0/929569760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55c787992610 con 0x55c7877a9aa0 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcf797fa640 1 -- 172.21.15.131:0/929569760 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf8c018070 con 0x55c7877a9aa0 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcf797fa640 1 -- 172.21.15.131:0/929569760 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7fcf8c03a020 con 0x55c7877a9aa0 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 10 client.7353 fetch_fsmap finished waiting for FSMap version 70 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 10 client.7353 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 20 client.7353 populate_metadata read hostname 'smithi131' 2024-09-17T02:47:57.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55c78768c990 0x55c787bb1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 4 client.7353 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 -- 172.21.15.131:0/929569760 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c787a0a5b0 con 0x55c78768c990 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcf9a59c640 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55c78768c990 0x55c787bb1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcf797fa640 10 client.7353 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcf9a59c640 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55c78768c990 0x55c787bb1830 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51303725a4297b37 server_cookie=f18620c9f390a245 in_seq=0 out_seq=0 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcf797fa640 1 -- 172.21.15.131:0/929569760 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x55c787a0a5b0 con 0x55c78768c990 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcf797fa640 10 client.7353 handle_command_reply: tid=0 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 client.7353 shutdown 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 -- 172.21.15.131:0/929569760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 msgr2=0x55c787c70b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c70b90 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c001d80 tx=0x7fcf8c003d70 comp rx=0 tx=0).stop 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 -- 172.21.15.131:0/929569760 shutdown_connections 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c787a8e6d0 0x55c787c710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55c78768c990 0x55c787bb1830 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c7877a9aa0 0x55c787c70b90 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 --2- 172.21.15.131:0/929569760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c7876e5890 0x55c787c70650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcf797fa640 0 client.7353 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:47:57.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 -- 172.21.15.131:0/929569760 >> 172.21.15.131:0/929569760 conn(0x55c7876b95c0 msgr2=0x55c787bbc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:57.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 -- 172.21.15.131:0/929569760 shutdown_connections 2024-09-17T02:47:57.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 1 -- 172.21.15.131:0/929569760 wait complete. 2024-09-17T02:47:57.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa7c20000 20 client.7353 trim_cache size 0 max 16384 2024-09-17T02:47:57.054 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:47:57.054 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: "9441c6a2-2616-4df3-82fb-d48465ab6abc": { 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir", 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "9441c6a2-2616-4df3-82fb-d48465ab6abc", 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive" 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: "9441c6a2-2616-4df3-82fb-d48465ab6abc": { 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:47:56.833277+0000", 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir", 2024-09-17T02:47:57.055 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:47:57.056 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:47:57.056 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:47:57.056 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:47:57.056 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:57.056 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:47:57.056 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:47:57.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 -- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcf7c073200 msgr2=0x7fcf7c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:57.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcf7c073200 0x7fcf7c075710 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcf90009b10 tx=0x7fcf9001a010 comp rx=0 tx=0).stop 2024-09-17T02:47:57.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 -- 172.21.15.131:0/1030221798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c15d180 msgr2=0x7fcf9c15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:47:57.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c15d180 0x7fcf9c15d590 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c08d6d0 tx=0x7fcf9400edf0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 -- 172.21.15.131:0/1030221798 shutdown_connections 2024-09-17T02:47:57.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcf7c073200 0x7fcf7c075710 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf9c15d180 0x7fcf9c15d590 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf9c15c3b0 0x7fcf9c159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 --2- 172.21.15.131:0/1030221798 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf84003a10 0x7fcf9c0cb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:47:57.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 -- 172.21.15.131:0/1030221798 >> 172.21.15.131:0/1030221798 conn(0x7fcf9c0d1b10 msgr2=0x7fcf9c1588b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:47:57.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 -- 172.21.15.131:0/1030221798 shutdown_connections 2024-09-17T02:47:57.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:47:57.049+0000 7fcfa0955640 1 -- 172.21.15.131:0/1030221798 wait complete. 2024-09-17T02:47:57.065 INFO:tasks.cephfs.filesystem:scrub status for tag:9441c6a2-2616-4df3-82fb-d48465ab6abc - {'path': '/dir', 'tag': '9441c6a2-2616-4df3-82fb-d48465ab6abc', 'options': 'recursive'} 2024-09-17T02:47:59.352 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:47:59.357+0000 7f3a39041640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-17T02:48:14.916 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:48:14.919 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:48:27.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:48:27.067 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-17T02:48:27.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.241+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/1616385019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f78073970 msgr2=0x7f9f7806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.241+0000 7f9f7d9e9640 1 --2- 172.21.15.131:0/1616385019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f78073970 0x7f9f7806dea0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f9f7806a1c0 tx=0x7f9f6c01c7d0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.241+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/1616385019 shutdown_connections 2024-09-17T02:48:27.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.241+0000 7f9f7d9e9640 1 --2- 172.21.15.131:0/1616385019 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f78073970 0x7f9f7806dea0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.241+0000 7f9f7d9e9640 1 --2- 172.21.15.131:0/1616385019 >> v2:172.21.15.189:3300/0 conn(0x7f9f78155240 0x7f9f78073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.241+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/1616385019 >> 172.21.15.131:0/1616385019 conn(0x7f9f7806eea0 msgr2=0x7f9f7806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.241+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/1616385019 shutdown_connections 2024-09-17T02:48:27.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.241+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/1616385019 wait complete. 2024-09-17T02:48:27.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.241+0000 7f9f7d9e9640 1 Processor -- start 2024-09-17T02:48:27.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.241+0000 7f9f7d9e9640 1 -- start start 2024-09-17T02:48:27.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f7d9e9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9f7806e3e0 0x7f9f78155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f7d9e9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f78073970 0x7f9f781533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f7d9e9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f78155240 0x7f9f780c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f7d9e9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78167d80 con 0x7f9f78155240 2024-09-17T02:48:27.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f7d9e9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7815f640 con 0x7f9f78073970 2024-09-17T02:48:27.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f7d9e9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9f78167f00 con 0x7f9f7806e3e0 2024-09-17T02:48:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f78155240 0x7f9f780c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f78155240 0x7f9f780c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58200/0 (socket says 172.21.15.131:58200) 2024-09-17T02:48:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f777fe640 1 -- 172.21.15.131:0/3829999840 learned_addr learned my addr 172.21.15.131:0/3829999840 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f767fc640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f78073970 0x7f9f781533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9f7806e3e0 0x7f9f78155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f777fe640 1 -- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9f7806e3e0 msgr2=0x7f9f78155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f777fe640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9f7806e3e0 0x7f9f78155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f777fe640 1 -- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f78073970 msgr2=0x7f9f781533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f777fe640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f78073970 0x7f9f781533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f777fe640 1 -- 172.21.15.131:0/3829999840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7815aab0 con 0x7f9f78155240 2024-09-17T02:48:27.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f777fe640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f78155240 0x7f9f780c5d20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800c3c0 tx=0x7f9f6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f5ffff640 1 -- 172.21.15.131:0/3829999840 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68011790 con 0x7f9f78155240 2024-09-17T02:48:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f5ffff640 1 -- 172.21.15.131:0/3829999840 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f68017070 con 0x7f9f78155240 2024-09-17T02:48:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/3829999840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7815ad90 con 0x7f9f78155240 2024-09-17T02:48:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f5ffff640 1 -- 172.21.15.131:0/3829999840 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68011ce0 con 0x7f9f78155240 2024-09-17T02:48:27.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/3829999840 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f780c7b90 con 0x7f9f78155240 2024-09-17T02:48:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f5ffff640 1 -- 172.21.15.131:0/3829999840 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9f6801c460 con 0x7f9f78155240 2024-09-17T02:48:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f5ffff640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9f64073200 0x7f9f64075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.245+0000 7f9f5ffff640 1 -- 172.21.15.131:0/3829999840 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f9f680942b0 con 0x7f9f78155240 2024-09-17T02:48:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9f64073200 0x7f9f64075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 Processor -- start 2024-09-17T02:48:27.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 -- start start 2024-09-17T02:48:27.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9f64073200 0x7f9f64075710 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000ad80 tx=0x7f9f7000a760 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c630a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5df5890 0x55d7c630af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5eb9aa0 0x55d7c630b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d7c61a9cd0 con 0x55d7c619e6d0 2024-09-17T02:48:27.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d7c60bf330 con 0x55d7c5eb9aa0 2024-09-17T02:48:27.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d7c611cbb0 con 0x55d7c5df5890 2024-09-17T02:48:27.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f76ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c630a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f76ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c630a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58212/0 (socket says 172.21.15.131:58212) 2024-09-17T02:48:27.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f76ffd640 1 -- 172.21.15.131:0/2114389973 learned_addr learned my addr 172.21.15.131:0/2114389973 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:27.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f767fc640 1 --2- 172.21.15.131:0/2114389973 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5df5890 0x55d7c630af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f76ffd640 1 -- 172.21.15.131:0/2114389973 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5df5890 msgr2=0x55d7c630af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/2114389973 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5df5890 0x55d7c630af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f777fe640 1 --2- 172.21.15.131:0/2114389973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5eb9aa0 0x55d7c630b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f76ffd640 1 -- 172.21.15.131:0/2114389973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5eb9aa0 msgr2=0x55d7c630b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/2114389973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5eb9aa0 0x55d7c630b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f76ffd640 1 -- 172.21.15.131:0/2114389973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c60c0c80 con 0x55d7c619e6d0 2024-09-17T02:48:27.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/2114389973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c630a9f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f9f70019750 tx=0x7f9f70011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff64d03569b36812 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f5d7fa640 1 -- 172.21.15.131:0/2114389973 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70004b30 con 0x55d7c619e6d0 2024-09-17T02:48:27.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f5d7fa640 1 -- 172.21.15.131:0/2114389973 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f70004dc0 con 0x55d7c619e6d0 2024-09-17T02:48:27.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f5d7fa640 1 -- 172.21.15.131:0/2114389973 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7000c9f0 con 0x55d7c619e6d0 2024-09-17T02:48:27.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2114389973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 msgr2=0x55d7c630a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/2114389973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c630a9f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f9f70019750 tx=0x7f9f70011000 comp rx=0 tx=0).stop 2024-09-17T02:48:27.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2114389973 shutdown_connections 2024-09-17T02:48:27.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/2114389973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5eb9aa0 0x55d7c630b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/2114389973 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5df5890 0x55d7c630af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/2114389973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c630a9f0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2114389973 >> 172.21.15.131:0/2114389973 conn(0x55d7c5dc95c0 msgr2=0x55d7c62c7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2114389973 shutdown_connections 2024-09-17T02:48:27.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.249+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2114389973 wait complete. 2024-09-17T02:48:27.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 -- start start 2024-09-17T02:48:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 Processor -- start 2024-09-17T02:48:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c62c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5eb9aa0 0x55d7c62c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c619e6d0 0x55d7c62cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d7c610fdf0 con 0x55d7c5df5890 2024-09-17T02:48:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d7c610bfe0 con 0x55d7c619e6d0 2024-09-17T02:48:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d7c60b14d0 con 0x55d7c5eb9aa0 2024-09-17T02:48:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 10 client.0 _ll_register_callbacks cb 0x55d7c608f7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f76ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c62c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f76ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c62c6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58218/0 (socket says 172.21.15.131:58218) 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f76ffd640 1 -- 172.21.15.131:0/1720621843 learned_addr learned my addr 172.21.15.131:0/1720621843 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f777fe640 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c619e6d0 0x55d7c62cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f767fc640 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5eb9aa0 0x55d7c62c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f76ffd640 1 -- 172.21.15.131:0/1720621843 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5eb9aa0 msgr2=0x55d7c62c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5eb9aa0 0x55d7c62c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f76ffd640 1 -- 172.21.15.131:0/1720621843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c619e6d0 msgr2=0x55d7c62cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c619e6d0 0x55d7c62cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f76ffd640 1 -- 172.21.15.131:0/1720621843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c60aa210 con 0x55d7c5df5890 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f777fe640 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c619e6d0 0x55d7c62cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c62c6a00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9f70023040 tx=0x7f9f7000cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d62a6e299ebdd6b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f567fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f567fc640 1 -- 172.21.15.131:0/1720621843 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7002f060 con 0x55d7c5df5890 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 -- 172.21.15.131:0/1720621843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7c60a4350 con 0x55d7c5df5890 2024-09-17T02:48:27.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f567fc640 1 -- 172.21.15.131:0/1720621843 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f9f70004d40 con 0x55d7c5df5890 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f567fc640 1 -- 172.21.15.131:0/1720621843 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f9f70035050 con 0x55d7c5df5890 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 10 client.7368 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 -- 172.21.15.131:0/1720621843 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55d7c6090fc0 con 0x55d7c5df5890 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f567fc640 1 -- 172.21.15.131:0/1720621843 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d7c60a4350 con 0x55d7c5df5890 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f567fc640 1 -- 172.21.15.131:0/1720621843 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f9f70038020 con 0x55d7c5df5890 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 10 client.7368 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 10 client.7368 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 20 client.7368 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55d7c5dae930 0x55d7c6311e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 4 client.7368 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f84cb4000 1 -- 172.21.15.131:0/1720621843 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7c611eb00 con 0x55d7c5dae930 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.253+0000 7f9f767fc640 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55d7c5dae930 0x55d7c6311e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f767fc640 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55d7c5dae930 0x55d7c6311e80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8f95603cf1a5835 server_cookie=1390e6684dca696e in_seq=0 out_seq=0 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f567fc640 10 client.7368 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f567fc640 1 -- 172.21.15.131:0/1720621843 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d7c611eb00 con 0x55d7c5dae930 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f567fc640 10 client.7368 handle_command_reply: tid=0 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 1 client.7368 shutdown 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 1 -- 172.21.15.131:0/1720621843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 msgr2=0x55d7c62c6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c62c6a00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f9f70023040 tx=0x7f9f7000cfc0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 1 -- 172.21.15.131:0/1720621843 shutdown_connections 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c619e6d0 0x55d7c62cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5eb9aa0 0x55d7c62c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55d7c5dae930 0x55d7c6311e80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/1720621843 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c62c6a00 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f567fc640 0 client.7368 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 1 -- 172.21.15.131:0/1720621843 >> 172.21.15.131:0/1720621843 conn(0x55d7c5dc95c0 msgr2=0x55d7c62bfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 1 -- 172.21.15.131:0/1720621843 shutdown_connections 2024-09-17T02:48:27.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 1 -- 172.21.15.131:0/1720621843 wait complete. 2024-09-17T02:48:27.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.257+0000 7f9f84cb4000 20 client.7368 trim_cache size 0 max 16384 2024-09-17T02:48:27.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 Processor -- start 2024-09-17T02:48:27.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 -- start start 2024-09-17T02:48:27.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5df5890 0x55d7c637c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5eb9aa0 0x55d7c637c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c637ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d7c62c5b20 con 0x55d7c619e6d0 2024-09-17T02:48:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78205e20 con 0x55d7c5df5890 2024-09-17T02:48:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9f7815e0e0 con 0x55d7c5eb9aa0 2024-09-17T02:48:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c637ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f777fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c637ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58232/0 (socket says 172.21.15.131:58232) 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f777fe640 1 -- 172.21.15.131:0/3317038108 learned_addr learned my addr 172.21.15.131:0/3317038108 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/3317038108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5df5890 0x55d7c637c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f777fe640 1 -- 172.21.15.131:0/3317038108 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5eb9aa0 msgr2=0x55d7c637c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f767fc640 1 --2- 172.21.15.131:0/3317038108 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5eb9aa0 0x55d7c637c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f777fe640 1 --2- 172.21.15.131:0/3317038108 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5eb9aa0 0x55d7c637c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f777fe640 1 -- 172.21.15.131:0/3317038108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5df5890 msgr2=0x55d7c637c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f777fe640 1 --2- 172.21.15.131:0/3317038108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5df5890 0x55d7c637c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f777fe640 1 -- 172.21.15.131:0/3317038108 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c6169680 con 0x55d7c619e6d0 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/3317038108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5df5890 0x55d7c637c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f777fe640 1 --2- 172.21.15.131:0/3317038108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c637ce80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9f680056f0 tx=0x7f9f6805e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3f0a2c166eca8d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f5cff9640 1 -- 172.21.15.131:0/3317038108 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f68094110 con 0x55d7c619e6d0 2024-09-17T02:48:27.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f5cff9640 1 -- 172.21.15.131:0/3317038108 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f68063020 con 0x55d7c619e6d0 2024-09-17T02:48:27.277 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f5cff9640 1 -- 172.21.15.131:0/3317038108 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6805f0c0 con 0x55d7c619e6d0 2024-09-17T02:48:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 -- 172.21.15.131:0/3317038108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 msgr2=0x55d7c637ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/3317038108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c637ce80 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f9f680056f0 tx=0x7f9f6805e8c0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 -- 172.21.15.131:0/3317038108 shutdown_connections 2024-09-17T02:48:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/3317038108 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c619e6d0 0x55d7c637ce80 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/3317038108 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c5eb9aa0 0x55d7c637c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/3317038108 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5df5890 0x55d7c637c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 -- 172.21.15.131:0/3317038108 >> 172.21.15.131:0/3317038108 conn(0x55d7c5dc95c0 msgr2=0x55d7c631cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 -- 172.21.15.131:0/3317038108 shutdown_connections 2024-09-17T02:48:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.277+0000 7f9f84cb4000 1 -- 172.21.15.131:0/3317038108 wait complete. 2024-09-17T02:48:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 -- start start 2024-09-17T02:48:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 Processor -- start 2024-09-17T02:48:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c6380650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5eb9aa0 0x55d7c6380b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c619e6d0 0x55d7c63810d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7815da10 con 0x55d7c5df5890 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9f78158bb0 con 0x55d7c5eb9aa0 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d7c5dd1680 con 0x55d7c619e6d0 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 10 client.0 _ll_register_callbacks cb 0x7f9f70019620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f76ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c6380650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f76ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c6380650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58234/0 (socket says 172.21.15.131:58234) 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f76ffd640 1 -- 172.21.15.131:0/2756726583 learned_addr learned my addr 172.21.15.131:0/2756726583 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f767fc640 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5eb9aa0 0x55d7c6380b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f777fe640 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c619e6d0 0x55d7c63810d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f76ffd640 1 -- 172.21.15.131:0/2756726583 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c619e6d0 msgr2=0x55d7c63810d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c619e6d0 0x55d7c63810d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f76ffd640 1 -- 172.21.15.131:0/2756726583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5eb9aa0 msgr2=0x55d7c6380b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5eb9aa0 0x55d7c6380b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f76ffd640 1 -- 172.21.15.131:0/2756726583 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c60d60b0 con 0x55d7c5df5890 2024-09-17T02:48:27.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f76ffd640 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c6380650 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000ec50 tx=0x7f9f70005120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2daa75ab6db0805 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f567fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:27.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f567fc640 1 -- 172.21.15.131:0/2756726583 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70005920 con 0x55d7c5df5890 2024-09-17T02:48:27.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f567fc640 1 -- 172.21.15.131:0/2756726583 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f9f7002a460 con 0x55d7c5df5890 2024-09-17T02:48:27.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2756726583 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f68063020 con 0x55d7c5df5890 2024-09-17T02:48:27.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f567fc640 1 -- 172.21.15.131:0/2756726583 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f9f70040050 con 0x55d7c5df5890 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 10 client.7374 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2756726583 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55d7c60a2610 con 0x55d7c5df5890 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f567fc640 1 -- 172.21.15.131:0/2756726583 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f68063020 con 0x55d7c5df5890 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f567fc640 1 -- 172.21.15.131:0/2756726583 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f9f70043020 con 0x55d7c5df5890 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 10 client.7374 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 10 client.7374 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 20 client.7374 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55d7c5d9c990 0x55d7c62c1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 4 client.7374 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2756726583 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7c611a5b0 con 0x55d7c5d9c990 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f767fc640 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55d7c5d9c990 0x55d7c62c1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f567fc640 10 client.7374 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:27.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f767fc640 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55d7c5d9c990 0x55d7c62c1890 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7589539d3f7d84d6 server_cookie=d4799b5ea5abaf40 in_seq=0 out_seq=0 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f567fc640 1 -- 172.21.15.131:0/2756726583 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+377 (crc 0 0 0) 0x55d7c611a5b0 con 0x55d7c5d9c990 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f567fc640 10 client.7374 handle_command_reply: tid=0 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 client.7374 shutdown 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2756726583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 msgr2=0x55d7c6380650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c6380650 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000ec50 tx=0x7f9f70005120 comp rx=0 tx=0).stop 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2756726583 shutdown_connections 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d7c619e6d0 0x55d7c63810d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d7c5eb9aa0 0x55d7c6380b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55d7c5d9c990 0x55d7c62c1890 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f567fc640 0 client.7374 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.281+0000 7f9f84cb4000 1 --2- 172.21.15.131:0/2756726583 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d7c5df5890 0x55d7c6380650 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2756726583 >> 172.21.15.131:0/2756726583 conn(0x55d7c5dc95c0 msgr2=0x55d7c62cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2756726583 shutdown_connections 2024-09-17T02:48:27.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f84cb4000 1 -- 172.21.15.131:0/2756726583 wait complete. 2024-09-17T02:48:27.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f84cb4000 20 client.7374 trim_cache size 0 max 16384 2024-09-17T02:48:27.288 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:48:27.288 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-09-17T02:48:27.288 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {}, 2024-09-17T02:48:27.288 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:48:27.288 INFO:teuthology.orchestra.run.smithi131.stdout: "9441c6a2-2616-4df3-82fb-d48465ab6abc": { 2024-09-17T02:48:27.288 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:47:56.833277+0000", 2024-09-17T02:48:27.289 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir", 2024-09-17T02:48:27.289 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:48:27.289 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:48:27.289 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:48:27.289 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:48:27.289 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:48:27.289 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:48:27.289 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:48:27.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9f64073200 msgr2=0x7f9f64075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9f64073200 0x7f9f64075710 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000ad80 tx=0x7f9f7000a760 comp rx=0 tx=0).stop 2024-09-17T02:48:27.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/3829999840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f78155240 msgr2=0x7f9f780c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f78155240 0x7f9f780c5d20 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f9f6800c3c0 tx=0x7f9f6800c880 comp rx=0 tx=0).stop 2024-09-17T02:48:27.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/3829999840 shutdown_connections 2024-09-17T02:48:27.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9f64073200 0x7f9f64075710 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9f78155240 0x7f9f780c5d20 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9f78073970 0x7f9f781533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 --2- 172.21.15.131:0/3829999840 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9f7806e3e0 0x7f9f78155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/3829999840 >> 172.21.15.131:0/3829999840 conn(0x7f9f7806eea0 msgr2=0x7f9f780b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/3829999840 shutdown_connections 2024-09-17T02:48:27.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.285+0000 7f9f7d9e9640 1 -- 172.21.15.131:0/3829999840 wait complete. 2024-09-17T02:48:27.299 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-17T02:48:27.300 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:48:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.453+0000 7fc382e34640 1 -- 172.21.15.131:0/1075854389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37c15bdb0 msgr2=0x7fc37c151bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.453+0000 7fc382e34640 1 --2- 172.21.15.131:0/1075854389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37c15bdb0 0x7fc37c151bd0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fc374009c80 tx=0x7fc37401cda0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 -- 172.21.15.131:0/1075854389 shutdown_connections 2024-09-17T02:48:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 --2- 172.21.15.131:0/1075854389 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37c15bdb0 0x7fc37c151bd0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 --2- 172.21.15.131:0/1075854389 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc37c06f6c0 0x7fc37c15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 -- 172.21.15.131:0/1075854389 >> 172.21.15.131:0/1075854389 conn(0x7fc37c073d10 msgr2=0x7fc37c074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 -- 172.21.15.131:0/1075854389 shutdown_connections 2024-09-17T02:48:27.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 -- 172.21.15.131:0/1075854389 wait complete. 2024-09-17T02:48:27.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 Processor -- start 2024-09-17T02:48:27.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 -- start start 2024-09-17T02:48:27.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc37c0cd470 0x7fc37c0c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc37c15bdb0 0x7fc37c1483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37c149940 0x7fc37c0b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c169000 con 0x7fc37c149940 2024-09-17T02:48:27.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c0cc410 con 0x7fc37c15bdb0 2024-09-17T02:48:27.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc382e34640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc37c169180 con 0x7fc37c0cd470 2024-09-17T02:48:27.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc3813d1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37c149940 0x7fc37c0b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc3813d1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37c149940 0x7fc37c0b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58248/0 (socket says 172.21.15.131:58248) 2024-09-17T02:48:27.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc3813d1640 1 -- 172.21.15.131:0/933806889 learned_addr learned my addr 172.21.15.131:0/933806889 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:27.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc37bfff640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc37c15bdb0 0x7fc37c1483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc380bd0640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc37c0cd470 0x7fc37c0c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc3813d1640 1 -- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc37c0cd470 msgr2=0x7fc37c0c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc3813d1640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc37c0cd470 0x7fc37c0c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc3813d1640 1 -- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc37c15bdb0 msgr2=0x7fc37c1483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc3813d1640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc37c15bdb0 0x7fc37c1483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc3813d1640 1 -- 172.21.15.131:0/933806889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c148930 con 0x7fc37c149940 2024-09-17T02:48:27.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.457+0000 7fc37bfff640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc37c15bdb0 0x7fc37c1483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:27.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc3813d1640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37c149940 0x7fc37c0b4270 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc37401a9d0 tx=0x7fc3740093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc3797fa640 1 -- 172.21.15.131:0/933806889 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3740bc070 con 0x7fc37c149940 2024-09-17T02:48:27.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc3797fa640 1 -- 172.21.15.131:0/933806889 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3740ad8e0 con 0x7fc37c149940 2024-09-17T02:48:27.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc382e34640 1 -- 172.21.15.131:0/933806889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c148cf0 con 0x7fc37c149940 2024-09-17T02:48:27.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc3797fa640 1 -- 172.21.15.131:0/933806889 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3740b2040 con 0x7fc37c149940 2024-09-17T02:48:27.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc382e34640 1 -- 172.21.15.131:0/933806889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c0c5730 con 0x7fc37c149940 2024-09-17T02:48:27.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc382e34640 1 -- 172.21.15.131:0/933806889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc37c14abb0 con 0x7fc37c149940 2024-09-17T02:48:27.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc3797fa640 1 -- 172.21.15.131:0/933806889 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc3740bf070 con 0x7fc37c149940 2024-09-17T02:48:27.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc3797fa640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc360073230 0x7fc360075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc3797fa640 1 -- 172.21.15.131:0/933806889 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fc37412e5e0 con 0x7fc37c149940 2024-09-17T02:48:27.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc380bd0640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc360073230 0x7fc360075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.461+0000 7fc380bd0640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc360073230 0x7fc360075740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc370000f70 tx=0x7fc370009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.465+0000 7fc3797fa640 1 -- 172.21.15.131:0/933806889 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc374016990 con 0x7fc37c149940 2024-09-17T02:48:27.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.657+0000 7fc382e34640 1 -- 172.21.15.131:0/933806889 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc37c15b870 con 0x7fc37c149940 2024-09-17T02:48:27.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.657+0000 7fc3797fa640 1 -- 172.21.15.131:0/933806889 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5238 (secure 0 0 0) 0x7fc3740f9460 con 0x7fc37c149940 2024-09-17T02:48:27.655 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:48:27.655 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":71,"btime":"2024-09-17T02:47:59:359117+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:47:59.122657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7233":{"gid":7233,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6835/3013984008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3013984008},{"type":"v1","addr":"172.21.15.131:6835","nonce":3013984008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7233,"qdb_cluster":[7233]},"id":6}]} 2024-09-17T02:48:27.655 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 71 2024-09-17T02:48:27.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 -- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc360073230 msgr2=0x7fc360075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc360073230 0x7fc360075740 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc370000f70 tx=0x7fc370009040 comp rx=0 tx=0).stop 2024-09-17T02:48:27.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 -- 172.21.15.131:0/933806889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37c149940 msgr2=0x7fc37c0b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37c149940 0x7fc37c0b4270 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc37401a9d0 tx=0x7fc3740093d0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 -- 172.21.15.131:0/933806889 shutdown_connections 2024-09-17T02:48:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc360073230 0x7fc360075740 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc37c149940 0x7fc37c0b4270 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc37c15bdb0 0x7fc37c1483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 --2- 172.21.15.131:0/933806889 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc37c0cd470 0x7fc37c0c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 -- 172.21.15.131:0/933806889 >> 172.21.15.131:0/933806889 conn(0x7fc37c073d10 msgr2=0x7fc37c14a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 -- 172.21.15.131:0/933806889 shutdown_connections 2024-09-17T02:48:27.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.661+0000 7fc382e34640 1 -- 172.21.15.131:0/933806889 wait complete. 2024-09-17T02:48:27.675 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-17T02:48:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.825+0000 7f2a79862640 1 -- 172.21.15.131:0/514225391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a7406f9c0 msgr2=0x7f2a74073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.825+0000 7f2a79862640 1 --2- 172.21.15.131:0/514225391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a7406f9c0 0x7f2a74073430 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f2a74075670 tx=0x7f2a6801c8c0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.825+0000 7f2a79862640 1 -- 172.21.15.131:0/514225391 shutdown_connections 2024-09-17T02:48:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.825+0000 7f2a79862640 1 --2- 172.21.15.131:0/514225391 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a7406f9c0 0x7f2a74073430 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.825+0000 7f2a79862640 1 --2- 172.21.15.131:0/514225391 >> v2:172.21.15.189:3300/0 conn(0x7f2a74073970 0x7f2a7406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.825+0000 7f2a79862640 1 --2- 172.21.15.131:0/514225391 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a7415b2e0 0x7f2a74155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.825+0000 7f2a79862640 1 -- 172.21.15.131:0/514225391 >> 172.21.15.131:0/514225391 conn(0x7f2a7406eea0 msgr2=0x7f2a7406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.825+0000 7f2a79862640 1 -- 172.21.15.131:0/514225391 shutdown_connections 2024-09-17T02:48:27.823 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.825+0000 7f2a79862640 1 -- 172.21.15.131:0/514225391 wait complete. 2024-09-17T02:48:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a79862640 1 Processor -- start 2024-09-17T02:48:27.824 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a79862640 1 -- start start 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a79862640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a7406f9c0 0x7f2a74153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a79862640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a74073970 0x7f2a74159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a79862640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a7415b2e0 0x7f2a740cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a79862640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74167f00 con 0x7f2a7406f9c0 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a79862640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2a74167d80 con 0x7f2a74073970 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a79862640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2a74168080 con 0x7f2a7415b2e0 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a72ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a7406f9c0 0x7f2a74153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a72ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a7406f9c0 0x7f2a74153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58268/0 (socket says 172.21.15.131:58268) 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a72ffd640 1 -- 172.21.15.131:0/2105835306 learned_addr learned my addr 172.21.15.131:0/2105835306 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a737fe640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a7415b2e0 0x7f2a740cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a727fc640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a74073970 0x7f2a74159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a72ffd640 1 -- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a7415b2e0 msgr2=0x7f2a740cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a7415b2e0 0x7f2a740cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a72ffd640 1 -- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a74073970 msgr2=0x7f2a74159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a74073970 0x7f2a74159b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a72ffd640 1 -- 172.21.15.131:0/2105835306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a741f77a0 con 0x7f2a7406f9c0 2024-09-17T02:48:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a737fe640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a7415b2e0 0x7f2a740cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a7406f9c0 0x7f2a74153450 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400c9f0 tx=0x7f2a6400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a5bfff640 1 -- 172.21.15.131:0/2105835306 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64007b50 con 0x7f2a7406f9c0 2024-09-17T02:48:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a5bfff640 1 -- 172.21.15.131:0/2105835306 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a64007de0 con 0x7f2a7406f9c0 2024-09-17T02:48:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a79862640 1 -- 172.21.15.131:0/2105835306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7415a3a0 con 0x7f2a7406f9c0 2024-09-17T02:48:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a5bfff640 1 -- 172.21.15.131:0/2105835306 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6400f6b0 con 0x7f2a7406f9c0 2024-09-17T02:48:27.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.829+0000 7f2a79862640 1 -- 172.21.15.131:0/2105835306 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7415a960 con 0x7f2a7406f9c0 2024-09-17T02:48:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a5bfff640 1 -- 172.21.15.131:0/2105835306 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2a6400f850 con 0x7f2a7406f9c0 2024-09-17T02:48:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a5bfff640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a60073230 0x7f2a60075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a5bfff640 1 -- 172.21.15.131:0/2105835306 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f2a64094bd0 con 0x7f2a7406f9c0 2024-09-17T02:48:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a727fc640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a60073230 0x7f2a60075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a80b2d000 1 Processor -- start 2024-09-17T02:48:27.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a80b2d000 1 -- start start 2024-09-17T02:48:27.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a727fc640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a60073230 0x7f2a60075740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2a680af000 tx=0x7f2a6801b100 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x5626586682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626585181c0 0x562658668820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x562658668d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562658506d00 con 0x562658522b60 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56265847ac90 con 0x5626585181c0 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56265846a0c0 con 0x562658501d90 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a72ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x5626586682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a72ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x5626586682e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58270/0 (socket says 172.21.15.131:58270) 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a72ffd640 1 -- 172.21.15.131:0/2986264842 learned_addr learned my addr 172.21.15.131:0/2986264842 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a737fe640 1 --2- 172.21.15.131:0/2986264842 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x562658668d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a727fc640 1 --2- 172.21.15.131:0/2986264842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626585181c0 0x562658668820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a72ffd640 1 -- 172.21.15.131:0/2986264842 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 msgr2=0x562658668d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/2986264842 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x562658668d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a72ffd640 1 -- 172.21.15.131:0/2986264842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626585181c0 msgr2=0x562658668820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/2986264842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626585181c0 0x562658668820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a72ffd640 1 -- 172.21.15.131:0/2986264842 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562658445290 con 0x562658522b60 2024-09-17T02:48:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a737fe640 1 --2- 172.21.15.131:0/2986264842 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x562658668d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/2986264842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x5626586682e0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2a64007590 tx=0x7f2a6405ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb857567ffaaf59a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a597fa640 1 -- 172.21.15.131:0/2986264842 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6405f2d0 con 0x562658522b60 2024-09-17T02:48:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a597fa640 1 -- 172.21.15.131:0/2986264842 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a64063db0 con 0x562658522b60 2024-09-17T02:48:27.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.833+0000 7f2a597fa640 1 -- 172.21.15.131:0/2986264842 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64068c60 con 0x562658522b60 2024-09-17T02:48:27.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2986264842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 msgr2=0x5626586682e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/2986264842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x5626586682e0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2a64007590 tx=0x7f2a6405ede0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2986264842 shutdown_connections 2024-09-17T02:48:27.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/2986264842 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x562658668d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/2986264842 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5626585181c0 0x562658668820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/2986264842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x5626586682e0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.834 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2986264842 >> 172.21.15.131:0/2986264842 conn(0x562658127760 msgr2=0x56265864b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2986264842 shutdown_connections 2024-09-17T02:48:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2986264842 wait complete. 2024-09-17T02:48:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 -- start start 2024-09-17T02:48:27.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 Processor -- start 2024-09-17T02:48:27.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658501d90 0x5626586240d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5626585181c0 0x562658624610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x56265864a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562658404940 con 0x562658522b60 2024-09-17T02:48:27.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56265846ded0 con 0x562658501d90 2024-09-17T02:48:27.838 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56265812f820 con 0x5626585181c0 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 10 client.0 _ll_register_callbacks cb 0x5626584487f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x56265864a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a737fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x56265864a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58278/0 (socket says 172.21.15.131:58278) 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a737fe640 1 -- 172.21.15.131:0/1787337826 learned_addr learned my addr 172.21.15.131:0/1787337826 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a737fe640 1 -- 172.21.15.131:0/1787337826 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5626585181c0 msgr2=0x562658624610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a737fe640 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5626585181c0 0x562658624610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a737fe640 1 -- 172.21.15.131:0/1787337826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658501d90 msgr2=0x5626586240d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658501d90 0x5626586240d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a737fe640 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658501d90 0x5626586240d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a737fe640 1 -- 172.21.15.131:0/1787337826 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56265841d550 con 0x562658522b60 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a737fe640 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x56265864a8d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009fc0 tx=0x7f2a6c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35646eaad3aa387f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.839 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a527fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a527fc640 1 -- 172.21.15.131:0/1787337826 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c011910 con 0x562658522b60 2024-09-17T02:48:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a527fc640 1 -- 172.21.15.131:0/1787337826 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f2a6c011ab0 con 0x562658522b60 2024-09-17T02:48:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 -- 172.21.15.131:0/1787337826 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562658402400 con 0x562658522b60 2024-09-17T02:48:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a527fc640 1 -- 172.21.15.131:0/1787337826 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f2a6c022050 con 0x562658522b60 2024-09-17T02:48:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 10 client.7392 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a80b2d000 1 -- 172.21.15.131:0/1787337826 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x562658408480 con 0x562658522b60 2024-09-17T02:48:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.837+0000 7f2a527fc640 1 -- 172.21.15.131:0/1787337826 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562658402400 con 0x562658522b60 2024-09-17T02:48:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a527fc640 1 -- 172.21.15.131:0/1787337826 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f2a6c025020 con 0x562658522b60 2024-09-17T02:48:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 10 client.7392 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 10 client.7392 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:48:27.840 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 20 client.7392 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x56265810cad0 0x5626586798a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 4 client.7392 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 -- 172.21.15.131:0/1787337826 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56265847cbe0 con 0x56265810cad0 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x56265810cad0 0x5626586798a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x56265810cad0 0x5626586798a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cea46a43ddef85d server_cookie=54282aa86171eb21 in_seq=0 out_seq=0 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a527fc640 10 client.7392 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a527fc640 1 -- 172.21.15.131:0/1787337826 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56265847cbe0 con 0x56265810cad0 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a527fc640 10 client.7392 handle_command_reply: tid=0 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 client.7392 shutdown 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 -- 172.21.15.131:0/1787337826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 msgr2=0x56265864a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x56265864a8d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009fc0 tx=0x7f2a6c00f7f0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 -- 172.21.15.131:0/1787337826 shutdown_connections 2024-09-17T02:48:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x56265810cad0 0x5626586798a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562658522b60 0x56265864a8d0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5626585181c0 0x562658624610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a527fc640 0 client.7392 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/1787337826 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658501d90 0x5626586240d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 -- 172.21.15.131:0/1787337826 >> 172.21.15.131:0/1787337826 conn(0x562658127760 msgr2=0x56265861c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 -- 172.21.15.131:0/1787337826 shutdown_connections 2024-09-17T02:48:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 1 -- 172.21.15.131:0/1787337826 wait complete. 2024-09-17T02:48:27.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.841+0000 7f2a80b2d000 20 client.7392 trim_cache size 0 max 16384 2024-09-17T02:48:27.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.853+0000 7f2a80b2d000 1 Processor -- start 2024-09-17T02:48:27.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.853+0000 7f2a80b2d000 1 -- start start 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x562658624490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x56265861f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658522b60 0x56265861fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562658529fc0 con 0x5626585181c0 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562658415300 con 0x562658522b60 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562658479a60 con 0x562658501d90 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a727fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x56265861f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a727fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x56265861f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58282/0 (socket says 172.21.15.131:58282) 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a727fc640 1 -- 172.21.15.131:0/660621217 learned_addr learned my addr 172.21.15.131:0/660621217 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a737fe640 1 --2- 172.21.15.131:0/660621217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658522b60 0x56265861fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/660621217 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x562658624490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a727fc640 1 -- 172.21.15.131:0/660621217 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 msgr2=0x562658624490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a727fc640 1 --2- 172.21.15.131:0/660621217 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x562658624490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a727fc640 1 -- 172.21.15.131:0/660621217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658522b60 msgr2=0x56265861fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a727fc640 1 --2- 172.21.15.131:0/660621217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658522b60 0x56265861fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a727fc640 1 -- 172.21.15.131:0/660621217 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626584efd10 con 0x5626585181c0 2024-09-17T02:48:27.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a727fc640 1 --2- 172.21.15.131:0/660621217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x56265861f6a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2a680b0040 tx=0x7f2a6801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4111017702ab9abd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a58ff9640 1 -- 172.21.15.131:0/660621217 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680c8070 con 0x5626585181c0 2024-09-17T02:48:27.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a58ff9640 1 -- 172.21.15.131:0/660621217 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a680a5c50 con 0x5626585181c0 2024-09-17T02:48:27.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a58ff9640 1 -- 172.21.15.131:0/660621217 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680b2040 con 0x5626585181c0 2024-09-17T02:48:27.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 -- 172.21.15.131:0/660621217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 msgr2=0x56265861f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/660621217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x56265861f6a0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2a680b0040 tx=0x7f2a6801c8c0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 -- 172.21.15.131:0/660621217 shutdown_connections 2024-09-17T02:48:27.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/660621217 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658522b60 0x56265861fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/660621217 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x56265861f6a0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/660621217 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x562658624490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 -- 172.21.15.131:0/660621217 >> 172.21.15.131:0/660621217 conn(0x562658127760 msgr2=0x56265867dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 -- 172.21.15.131:0/660621217 shutdown_connections 2024-09-17T02:48:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 -- 172.21.15.131:0/660621217 wait complete. 2024-09-17T02:48:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 -- start start 2024-09-17T02:48:27.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.857+0000 7f2a80b2d000 1 Processor -- start 2024-09-17T02:48:27.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x5626586e1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x5626586e2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658522b60 0x5626586e2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562658436300 con 0x5626585181c0 2024-09-17T02:48:27.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562658427670 con 0x562658522b60 2024-09-17T02:48:27.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56265846d1e0 con 0x562658501d90 2024-09-17T02:48:27.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 10 client.0 _ll_register_callbacks cb 0x7f2a6c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:27.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a727fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x5626586e2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a727fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x5626586e2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58284/0 (socket says 172.21.15.131:58284) 2024-09-17T02:48:27.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a727fc640 1 -- 172.21.15.131:0/2166702175 learned_addr learned my addr 172.21.15.131:0/2166702175 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:27.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a727fc640 1 -- 172.21.15.131:0/2166702175 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 msgr2=0x5626586e1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a727fc640 1 --2- 172.21.15.131:0/2166702175 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x5626586e1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a727fc640 1 -- 172.21.15.131:0/2166702175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658522b60 msgr2=0x5626586e2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:27.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a727fc640 1 --2- 172.21.15.131:0/2166702175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658522b60 0x5626586e2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a727fc640 1 -- 172.21.15.131:0/2166702175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562658420e50 con 0x5626585181c0 2024-09-17T02:48:27.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a727fc640 1 --2- 172.21.15.131:0/2166702175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x5626586e2340 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2a6801cda0 tx=0x7f2a68004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239f64dc5cb0636e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:27.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a527fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:27.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a527fc640 1 -- 172.21.15.131:0/2166702175 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680c8070 con 0x5626585181c0 2024-09-17T02:48:27.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a527fc640 1 -- 172.21.15.131:0/2166702175 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f2a680a5e00 con 0x5626585181c0 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2166702175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a680a5c50 con 0x5626585181c0 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a527fc640 1 -- 172.21.15.131:0/2166702175 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f2a680bc020 con 0x5626585181c0 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 10 client.7398 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2166702175 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x56265840f2d0 con 0x5626585181c0 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a527fc640 1 -- 172.21.15.131:0/2166702175 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a680a5c50 con 0x5626585181c0 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a527fc640 1 -- 172.21.15.131:0/2166702175 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f2a680d6080 con 0x5626585181c0 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 10 client.7398 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 10 client.7398 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 20 client.7398 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/2166702175 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x5626580fab30 0x56265869d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 4 client.7398 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2166702175 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x562658478690 con 0x5626580fab30 2024-09-17T02:48:27.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/2166702175 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x5626580fab30 0x56265869d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a527fc640 10 client.7398 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a72ffd640 1 --2- 172.21.15.131:0/2166702175 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x5626580fab30 0x56265869d2a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfd8f0ceb7f2fef0 server_cookie=c22f1b923cbd05e0 in_seq=0 out_seq=0 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a527fc640 1 -- 172.21.15.131:0/2166702175 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+419 (crc 0 0 0) 0x562658478690 con 0x5626580fab30 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a527fc640 10 client.7398 handle_command_reply: tid=0 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 client.7398 shutdown 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2166702175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 msgr2=0x5626586e2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/2166702175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x5626586e2340 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2a6801cda0 tx=0x7f2a68004550 comp rx=0 tx=0).stop 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2166702175 shutdown_connections 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/2166702175 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562658522b60 0x5626586e2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/2166702175 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x5626580fab30 0x56265869d2a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/2166702175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5626585181c0 0x5626586e2340 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 --2- 172.21.15.131:0/2166702175 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562658501d90 0x5626586e1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a527fc640 0 client.7398 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:27.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2166702175 >> 172.21.15.131:0/2166702175 conn(0x562658127760 msgr2=0x562658674ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2166702175 shutdown_connections 2024-09-17T02:48:27.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 1 -- 172.21.15.131:0/2166702175 wait complete. 2024-09-17T02:48:27.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a80b2d000 20 client.7398 trim_cache size 0 max 16384 2024-09-17T02:48:27.864 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:48:27.864 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:48:27.864 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:48:27.864 INFO:teuthology.orchestra.run.smithi131.stdout: "damage_type": "dir_frag", 2024-09-17T02:48:27.864 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 926450512, 2024-09-17T02:48:27.864 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 1099511627776, 2024-09-17T02:48:27.864 INFO:teuthology.orchestra.run.smithi131.stdout: "frag": "*", 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir" 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: "damage_type": "backtrace", 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 2164657649, 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 1099511627777, 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/testdir/hardlink" 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: "damage_type": "backtrace", 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 3220628231, 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 1099511627776, 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir" 2024-09-17T02:48:27.865 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:48:27.866 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:48:27.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 -- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a60073230 msgr2=0x7f2a60075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a60073230 0x7f2a60075740 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2a680af000 tx=0x7f2a6801b100 comp rx=0 tx=0).stop 2024-09-17T02:48:27.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 -- 172.21.15.131:0/2105835306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a7406f9c0 msgr2=0x7f2a74153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a7406f9c0 0x7f2a74153450 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400c9f0 tx=0x7f2a6400ceb0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 -- 172.21.15.131:0/2105835306 shutdown_connections 2024-09-17T02:48:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2a60073230 0x7f2a60075740 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2a7415b2e0 0x7f2a740cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2a74073970 0x7f2a74159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 --2- 172.21.15.131:0/2105835306 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2a7406f9c0 0x7f2a74153450 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 -- 172.21.15.131:0/2105835306 >> 172.21.15.131:0/2105835306 conn(0x7f2a7406eea0 msgr2=0x7f2a7406d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 -- 172.21.15.131:0/2105835306 shutdown_connections 2024-09-17T02:48:27.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:27.861+0000 7f2a79862640 1 -- 172.21.15.131:0/2105835306 wait complete. 2024-09-17T02:48:27.877 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T02:48:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.025+0000 7f4876c2d640 1 -- 172.21.15.131:0/260757989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487015f7d0 msgr2=0x7f4870162410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.025+0000 7f4876c2d640 1 --2- 172.21.15.131:0/260757989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487015f7d0 0x7f4870162410 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f487006ecf0 tx=0x7f485c01c7e0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.025+0000 7f4876c2d640 1 -- 172.21.15.131:0/260757989 shutdown_connections 2024-09-17T02:48:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.025+0000 7f4876c2d640 1 --2- 172.21.15.131:0/260757989 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487015f7d0 0x7f4870162410 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.025+0000 7f4876c2d640 1 --2- 172.21.15.131:0/260757989 >> v2:172.21.15.189:3300/0 conn(0x7f48700cf710 0x7f487015f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.025+0000 7f4876c2d640 1 -- 172.21.15.131:0/260757989 >> 172.21.15.131:0/260757989 conn(0x7f48700d1b10 msgr2=0x7f48700d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.025+0000 7f4876c2d640 1 -- 172.21.15.131:0/260757989 shutdown_connections 2024-09-17T02:48:28.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.025+0000 7f4876c2d640 1 -- 172.21.15.131:0/260757989 wait complete. 2024-09-17T02:48:28.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.025+0000 7f4876c2d640 1 Processor -- start 2024-09-17T02:48:28.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.025+0000 7f4876c2d640 1 -- start start 2024-09-17T02:48:28.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f4876c2d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48700cf710 0x7f48700cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f4876c2d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f487015f7d0 0x7f4870154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.027 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f4876c2d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4870162950 0x7f48701551f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f4876c2d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48700d3380 con 0x7f4870162950 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f4876c2d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f487015fe20 con 0x7f48700cf710 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f4876c2d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f48700d3500 con 0x7f487015f7d0 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f48751ca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4870162950 0x7f48701551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f48751ca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4870162950 0x7f48701551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58298/0 (socket says 172.21.15.131:58298) 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f48751ca640 1 -- 172.21.15.131:0/1631318776 learned_addr learned my addr 172.21.15.131:0/1631318776 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f48749c9640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48700cf710 0x7f48700cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f486ffff640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f487015f7d0 0x7f4870154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f48751ca640 1 -- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f487015f7d0 msgr2=0x7f4870154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f48751ca640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f487015f7d0 0x7f4870154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f48751ca640 1 -- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48700cf710 msgr2=0x7f48700cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f48751ca640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48700cf710 0x7f48700cc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f48751ca640 1 -- 172.21.15.131:0/1631318776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487014fca0 con 0x7f4870162950 2024-09-17T02:48:28.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f486ffff640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f487015f7d0 0x7f4870154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:28.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f48751ca640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4870162950 0x7f48701551f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f486800c760 tx=0x7f486800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f486d7fa640 1 -- 172.21.15.131:0/1631318776 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4868003db0 con 0x7f4870162950 2024-09-17T02:48:28.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f486d7fa640 1 -- 172.21.15.131:0/1631318776 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f486800ce10 con 0x7f4870162950 2024-09-17T02:48:28.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f4876c2d640 1 -- 172.21.15.131:0/1631318776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48701fcb70 con 0x7f4870162950 2024-09-17T02:48:28.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f486d7fa640 1 -- 172.21.15.131:0/1631318776 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48680052d0 con 0x7f4870162950 2024-09-17T02:48:28.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f4876c2d640 1 -- 172.21.15.131:0/1631318776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487014f9c0 con 0x7f4870162950 2024-09-17T02:48:28.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f4876c2d640 1 -- 172.21.15.131:0/1631318776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487015f290 con 0x7f4870162950 2024-09-17T02:48:28.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.029+0000 7f486d7fa640 1 -- 172.21.15.131:0/1631318776 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4868005470 con 0x7f4870162950 2024-09-17T02:48:28.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.033+0000 7f486d7fa640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4854073200 0x7f4854075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.033+0000 7f486d7fa640 1 -- 172.21.15.131:0/1631318776 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f4868094140 con 0x7f4870162950 2024-09-17T02:48:28.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.033+0000 7f48749c9640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4854073200 0x7f4854075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.030 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.033+0000 7f48749c9640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4854073200 0x7f4854075710 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f48640058d0 tx=0x7f4864005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.037+0000 7f486d7fa640 1 -- 172.21.15.131:0/1631318776 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f487014fca0 con 0x7f4870162950 2024-09-17T02:48:28.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.245+0000 7f4876c2d640 1 -- 172.21.15.131:0/1631318776 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f48701fc890 con 0x7f4870162950 2024-09-17T02:48:28.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.245+0000 7f486d7fa640 1 -- 172.21.15.131:0/1631318776 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f486805efe0 con 0x7f4870162950 2024-09-17T02:48:28.242 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:48:28.242 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-09-17T02:48:28.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 -- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4854073200 msgr2=0x7f4854075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4854073200 0x7f4854075710 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f48640058d0 tx=0x7f4864005860 comp rx=0 tx=0).stop 2024-09-17T02:48:28.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 -- 172.21.15.131:0/1631318776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4870162950 msgr2=0x7f48701551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4870162950 0x7f48701551f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f486800c760 tx=0x7f486800cc20 comp rx=0 tx=0).stop 2024-09-17T02:48:28.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 -- 172.21.15.131:0/1631318776 shutdown_connections 2024-09-17T02:48:28.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4854073200 0x7f4854075710 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4870162950 0x7f48701551f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f487015f7d0 0x7f4870154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 --2- 172.21.15.131:0/1631318776 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48700cf710 0x7f48700cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 -- 172.21.15.131:0/1631318776 >> 172.21.15.131:0/1631318776 conn(0x7f48700d1b10 msgr2=0x7f48700b8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 -- 172.21.15.131:0/1631318776 shutdown_connections 2024-09-17T02:48:28.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.249+0000 7f4876c2d640 1 -- 172.21.15.131:0/1631318776 wait complete. 2024-09-17T02:48:28.264 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T02:48:28.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:48:28.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-09-17T02:48:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.433+0000 7fae364a4640 1 -- 172.21.15.131:0/1355626563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae1c003a10 msgr2=0x7fae1c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.433+0000 7fae364a4640 1 --2- 172.21.15.131:0/1355626563 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae1c003a10 0x7fae1c003e80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fae2000bae0 tx=0x7fae2001cd80 comp rx=0 tx=0).stop 2024-09-17T02:48:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.433+0000 7fae34a41640 1 -- 172.21.15.131:0/1355626563 reap_dead start 2024-09-17T02:48:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.433+0000 7fae364a4640 1 -- 172.21.15.131:0/1355626563 shutdown_connections 2024-09-17T02:48:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.433+0000 7fae364a4640 1 -- 172.21.15.131:0/1355626563 >> 172.21.15.131:0/1355626563 conn(0x7fae300d05a0 msgr2=0x7fae300d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.433+0000 7fae364a4640 1 -- 172.21.15.131:0/1355626563 shutdown_connections 2024-09-17T02:48:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.433+0000 7fae364a4640 1 -- 172.21.15.131:0/1355626563 wait complete. 2024-09-17T02:48:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.433+0000 7fae364a4640 1 Processor -- start 2024-09-17T02:48:28.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.433+0000 7fae364a4640 1 -- start start 2024-09-17T02:48:28.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae364a4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c003a10 0x7fae30166500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae364a4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fae30166a40 0x7fae301321c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae364a4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae30132700 0x7fae30132b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae364a4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fae300731a0 con 0x7fae30132700 2024-09-17T02:48:28.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae364a4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae300cfdc0 con 0x7fae1c003a10 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae364a4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fae30073320 con 0x7fae30166a40 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae34a41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae30132700 0x7fae30132b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae34a41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae30132700 0x7fae30132b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58312/0 (socket says 172.21.15.131:58312) 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae34a41640 1 -- 172.21.15.131:0/860174287 learned_addr learned my addr 172.21.15.131:0/860174287 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae2ffff640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c003a10 0x7fae30166500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae277fe640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fae30166a40 0x7fae301321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae34a41640 1 -- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fae30166a40 msgr2=0x7fae301321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae34a41640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fae30166a40 0x7fae301321c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae34a41640 1 -- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c003a10 msgr2=0x7fae30166500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae34a41640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c003a10 0x7fae30166500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae34a41640 1 -- 172.21.15.131:0/860174287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae30133050 con 0x7fae30132700 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae2ffff640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c003a10 0x7fae30166500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae34a41640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae30132700 0x7fae30132b10 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fae28001340 tx=0x7fae2800df00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae2d7fa640 1 -- 172.21.15.131:0/860174287 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2800b8e0 con 0x7fae30132700 2024-09-17T02:48:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae364a4640 1 -- 172.21.15.131:0/860174287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae30131440 con 0x7fae30132700 2024-09-17T02:48:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae2d7fa640 1 -- 172.21.15.131:0/860174287 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae28018070 con 0x7fae30132700 2024-09-17T02:48:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae2d7fa640 1 -- 172.21.15.131:0/860174287 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2800bd60 con 0x7fae30132700 2024-09-17T02:48:28.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae364a4640 1 -- 172.21.15.131:0/860174287 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae301316e0 con 0x7fae30132700 2024-09-17T02:48:28.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.437+0000 7fae2d7fa640 1 -- 172.21.15.131:0/860174287 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fae2801d460 con 0x7fae30132700 2024-09-17T02:48:28.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2d7fa640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fae10073230 0x7fae10075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2d7fa640 1 -- 172.21.15.131:0/860174287 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fae280942e0 con 0x7fae30132700 2024-09-17T02:48:28.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2ffff640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fae10073230 0x7fae10075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 Processor -- start 2024-09-17T02:48:28.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 -- start start 2024-09-17T02:48:28.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2ffff640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fae10073230 0x7fae10075740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fae200b0000 tx=0x7fae200a7040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a5210eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fb9ae0 0x562a5210efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a5210f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562a51ecfbd0 con 0x562a51fc11a0 2024-09-17T02:48:28.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562a51ec0430 con 0x562a51fa3970 2024-09-17T02:48:28.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562a51f1dce0 con 0x562a51fb9ae0 2024-09-17T02:48:28.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a5210eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a5210eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58322/0 (socket says 172.21.15.131:58322) 2024-09-17T02:48:28.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2ffff640 1 -- 172.21.15.131:0/833646617 learned_addr learned my addr 172.21.15.131:0/833646617 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:28.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2ffff640 1 -- 172.21.15.131:0/833646617 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fb9ae0 msgr2=0x562a5210efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2ffff640 1 --2- 172.21.15.131:0/833646617 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fb9ae0 0x562a5210efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2ffff640 1 -- 172.21.15.131:0/833646617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 msgr2=0x562a5210f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2ffff640 1 --2- 172.21.15.131:0/833646617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a5210f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2ffff640 1 -- 172.21.15.131:0/833646617 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a51ec1d80 con 0x562a51fc11a0 2024-09-17T02:48:28.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae2ffff640 1 --2- 172.21.15.131:0/833646617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a5210eaa0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fae2000ba30 tx=0x7fae2001cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268488c9744f518e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae25ffb640 1 -- 172.21.15.131:0/833646617 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae200ca070 con 0x562a51fc11a0 2024-09-17T02:48:28.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae25ffb640 1 -- 172.21.15.131:0/833646617 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae20005510 con 0x562a51fc11a0 2024-09-17T02:48:28.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae25ffb640 1 -- 172.21.15.131:0/833646617 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae200ab040 con 0x562a51fc11a0 2024-09-17T02:48:28.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 -- 172.21.15.131:0/833646617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 msgr2=0x562a5210eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 --2- 172.21.15.131:0/833646617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a5210eaa0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fae2000ba30 tx=0x7fae2001cd80 comp rx=0 tx=0).stop 2024-09-17T02:48:28.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 -- 172.21.15.131:0/833646617 shutdown_connections 2024-09-17T02:48:28.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 --2- 172.21.15.131:0/833646617 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a5210f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 --2- 172.21.15.131:0/833646617 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fb9ae0 0x562a5210efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 --2- 172.21.15.131:0/833646617 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a5210eaa0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.441+0000 7fae3d76f000 1 -- 172.21.15.131:0/833646617 >> 172.21.15.131:0/833646617 conn(0x562a51fc78c0 msgr2=0x562a520bf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 -- 172.21.15.131:0/833646617 shutdown_connections 2024-09-17T02:48:28.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 -- 172.21.15.131:0/833646617 wait complete. 2024-09-17T02:48:28.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 -- start start 2024-09-17T02:48:28.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 Processor -- start 2024-09-17T02:48:28.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a52114f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fb9ae0 0x562a52115460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a521159a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562a51f10f20 con 0x562a51fc11a0 2024-09-17T02:48:28.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562a51f0d110 con 0x562a51fa3970 2024-09-17T02:48:28.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562a51eb26d0 con 0x562a51fb9ae0 2024-09-17T02:48:28.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 10 client.0 _ll_register_callbacks cb 0x562a51eb1330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:28.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae34a41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a521159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae34a41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a521159a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58326/0 (socket says 172.21.15.131:58326) 2024-09-17T02:48:28.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae34a41640 1 -- 172.21.15.131:0/266017003 learned_addr learned my addr 172.21.15.131:0/266017003 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:28.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae277fe640 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fb9ae0 0x562a52115460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae2ffff640 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a52114f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae34a41640 1 -- 172.21.15.131:0/266017003 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fb9ae0 msgr2=0x562a52115460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae34a41640 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fb9ae0 0x562a52115460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae34a41640 1 -- 172.21.15.131:0/266017003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 msgr2=0x562a52114f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae34a41640 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a52114f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae34a41640 1 -- 172.21.15.131:0/266017003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a51eab310 con 0x562a51fc11a0 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae2ffff640 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a52114f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae34a41640 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a521159a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fae2805e670 tx=0x7fae2805e1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8a11d3d3090028 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae0effd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae0effd640 1 -- 172.21.15.131:0/266017003 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2805e970 con 0x562a51fc11a0 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae0effd640 1 -- 172.21.15.131:0/266017003 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fae28013d70 con 0x562a51fc11a0 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 -- 172.21.15.131:0/266017003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a51ea5450 con 0x562a51fc11a0 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae0effd640 1 -- 172.21.15.131:0/266017003 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fae28063450 con 0x562a51fc11a0 2024-09-17T02:48:28.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 10 client.7419 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 -- 172.21.15.131:0/266017003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x562a51ec8f60 con 0x562a51fc11a0 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae0effd640 1 -- 172.21.15.131:0/266017003 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562a51ea5450 con 0x562a51fc11a0 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae0effd640 1 -- 172.21.15.131:0/266017003 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7fae28063020 con 0x562a51fc11a0 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 10 client.7419 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 10 client.7419 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 20 client.7419 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x562a51bafb80 0x562a5211f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 4 client.7419 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae3d76f000 1 -- 172.21.15.131:0/266017003 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a51f1fc30 con 0x562a51bafb80 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.445+0000 7fae2ffff640 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x562a51bafb80 0x562a5211f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae0effd640 10 client.7419 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae2ffff640 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x562a51bafb80 0x562a5211f300 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=481c2dbb438d399e server_cookie=6c22d1401325f3aa in_seq=0 out_seq=0 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae0effd640 1 -- 172.21.15.131:0/266017003 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a51f1fc30 con 0x562a51bafb80 2024-09-17T02:48:28.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae0effd640 10 client.7419 handle_command_reply: tid=0 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 1 client.7419 shutdown 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 1 -- 172.21.15.131:0/266017003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 msgr2=0x562a521159a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a521159a0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fae2805e670 tx=0x7fae2805e1a0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 1 -- 172.21.15.131:0/266017003 shutdown_connections 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x562a51bafb80 0x562a5211f300 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a521159a0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fb9ae0 0x562a52115460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 1 --2- 172.21.15.131:0/266017003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a52114f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 1 -- 172.21.15.131:0/266017003 >> 172.21.15.131:0/266017003 conn(0x562a51fc78c0 msgr2=0x562a520c6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae0effd640 0 client.7419 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 1 -- 172.21.15.131:0/266017003 shutdown_connections 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 1 -- 172.21.15.131:0/266017003 wait complete. 2024-09-17T02:48:28.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.449+0000 7fae3d76f000 20 client.7419 trim_cache size 0 max 16384 2024-09-17T02:48:28.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.465+0000 7fae3d76f000 1 Processor -- start 2024-09-17T02:48:28.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.465+0000 7fae3d76f000 1 -- start start 2024-09-17T02:48:28.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fa3970 0x562a52181040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fb9ae0 0x562a52181580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a52181ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fae30201ce0 con 0x562a51fc11a0 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fae30200e20 con 0x562a51fb9ae0 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fae3015f9b0 con 0x562a51fa3970 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae34a41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a52181ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae34a41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a52181ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58340/0 (socket says 172.21.15.131:58340) 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae34a41640 1 -- 172.21.15.131:0/2706274996 learned_addr learned my addr 172.21.15.131:0/2706274996 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae34a41640 1 -- 172.21.15.131:0/2706274996 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fa3970 msgr2=0x562a52181040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae34a41640 1 --2- 172.21.15.131:0/2706274996 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fa3970 0x562a52181040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae277fe640 1 --2- 172.21.15.131:0/2706274996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fb9ae0 0x562a52181580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae34a41640 1 -- 172.21.15.131:0/2706274996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fb9ae0 msgr2=0x562a52181580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae34a41640 1 --2- 172.21.15.131:0/2706274996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fb9ae0 0x562a52181580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae34a41640 1 -- 172.21.15.131:0/2706274996 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a51f91f30 con 0x562a51fc11a0 2024-09-17T02:48:28.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae34a41640 1 --2- 172.21.15.131:0/2706274996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a52181ac0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fae28063610 tx=0x7fae2800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b8ea657b67126b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae257fa640 1 -- 172.21.15.131:0/2706274996 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2805e2c0 con 0x562a51fc11a0 2024-09-17T02:48:28.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae257fa640 1 -- 172.21.15.131:0/2706274996 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae2805e970 con 0x562a51fc11a0 2024-09-17T02:48:28.467 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae257fa640 1 -- 172.21.15.131:0/2706274996 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2807ed90 con 0x562a51fc11a0 2024-09-17T02:48:28.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 -- 172.21.15.131:0/2706274996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 msgr2=0x562a52181ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 --2- 172.21.15.131:0/2706274996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a52181ac0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fae28063610 tx=0x7fae2800df70 comp rx=0 tx=0).stop 2024-09-17T02:48:28.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 -- 172.21.15.131:0/2706274996 shutdown_connections 2024-09-17T02:48:28.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 --2- 172.21.15.131:0/2706274996 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fc11a0 0x562a52181ac0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 --2- 172.21.15.131:0/2706274996 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fb9ae0 0x562a52181580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 --2- 172.21.15.131:0/2706274996 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fa3970 0x562a52181040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 -- 172.21.15.131:0/2706274996 >> 172.21.15.131:0/2706274996 conn(0x562a51fc78c0 msgr2=0x562a5210cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 -- 172.21.15.131:0/2706274996 shutdown_connections 2024-09-17T02:48:28.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 -- 172.21.15.131:0/2706274996 wait complete. 2024-09-17T02:48:28.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 -- start start 2024-09-17T02:48:28.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.469+0000 7fae3d76f000 1 Processor -- start 2024-09-17T02:48:28.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a5211a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fb9ae0 0x562a5211a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fc11a0 0x562a52141830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fae30160000 con 0x562a51fb9ae0 2024-09-17T02:48:28.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562a51fcb8e0 con 0x562a51fa3970 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562a520cdf30 con 0x562a51fc11a0 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 10 client.0 _ll_register_callbacks cb 0x7fae2800bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae277fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fb9ae0 0x562a5211a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae277fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fb9ae0 0x562a5211a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58348/0 (socket says 172.21.15.131:58348) 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae277fe640 1 -- 172.21.15.131:0/2334816849 learned_addr learned my addr 172.21.15.131:0/2334816849 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae2ffff640 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a5211a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae34a41640 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fc11a0 0x562a52141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae277fe640 1 -- 172.21.15.131:0/2334816849 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fc11a0 msgr2=0x562a52141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae277fe640 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fc11a0 0x562a52141830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae277fe640 1 -- 172.21.15.131:0/2334816849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 msgr2=0x562a5211a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae277fe640 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a5211a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae277fe640 1 -- 172.21.15.131:0/2334816849 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a51f68dd0 con 0x562a51fb9ae0 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae2ffff640 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a5211a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:28.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae277fe640 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fb9ae0 0x562a5211a900 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fae18005bd0 tx=0x7fae18012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9adb5b6106b0cd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae0effd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae0effd640 1 -- 172.21.15.131:0/2334816849 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1801f060 con 0x562a51fb9ae0 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae0effd640 1 -- 172.21.15.131:0/2334816849 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fae1800dcb0 con 0x562a51fb9ae0 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 -- 172.21.15.131:0/2334816849 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae2805e970 con 0x562a51fb9ae0 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae0effd640 1 -- 172.21.15.131:0/2334816849 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fae18025050 con 0x562a51fb9ae0 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 10 client.7425 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 -- 172.21.15.131:0/2334816849 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x562a51ea3840 con 0x562a51fb9ae0 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae0effd640 1 -- 172.21.15.131:0/2334816849 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae2805e970 con 0x562a51fb9ae0 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae0effd640 1 -- 172.21.15.131:0/2334816849 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7fae18028020 con 0x562a51fb9ae0 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 10 client.7425 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 10 client.7425 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 20 client.7425 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x562a51b9dbe0 0x562a52184e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 4 client.7425 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 -- 172.21.15.131:0/2334816849 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x562a51f1b6e0 con 0x562a51b9dbe0 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae2ffff640 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x562a51b9dbe0 0x562a52184e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae0effd640 10 client.7425 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae2ffff640 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x562a51b9dbe0 0x562a52184e70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97355d28041acc35 server_cookie=665ad81b12d9fa0a in_seq=0 out_seq=0 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae0effd640 1 -- 172.21.15.131:0/2334816849 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562a51f1b6e0 con 0x562a51b9dbe0 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae0effd640 10 client.7425 handle_command_reply: tid=0 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 client.7425 shutdown 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 -- 172.21.15.131:0/2334816849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fb9ae0 msgr2=0x562a5211a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fb9ae0 0x562a5211a900 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fae18005bd0 tx=0x7fae18012b60 comp rx=0 tx=0).stop 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 -- 172.21.15.131:0/2334816849 shutdown_connections 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562a51fc11a0 0x562a52141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x562a51b9dbe0 0x562a52184e70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562a51fb9ae0 0x562a5211a900 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae0effd640 0 client.7425 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 --2- 172.21.15.131:0/2334816849 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562a51fa3970 0x562a5211a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 -- 172.21.15.131:0/2334816849 >> 172.21.15.131:0/2334816849 conn(0x562a51fc78c0 msgr2=0x562a5210d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 -- 172.21.15.131:0/2334816849 shutdown_connections 2024-09-17T02:48:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 1 -- 172.21.15.131:0/2334816849 wait complete. 2024-09-17T02:48:28.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.473+0000 7fae3d76f000 20 client.7425 trim_cache size 0 max 16384 2024-09-17T02:48:28.476 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:48:28.476 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-09-17T02:48:28.476 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "ef0c0578-0779-4ec5-8591-f4a76a58d43d", 2024-09-17T02:48:28.476 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-09-17T02:48:28.477 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:48:28.477 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-09-17T02:48:28.477+0000 7f9a574b4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-09-17T02:47:39.862410+0000 1=1+0)->f(v1 m2024-09-17T02:47:39.862410+0000) n(v0 rc2024-09-17T02:47:39.976065+0000 2=1+1)->n(v0 rc2024-09-17T02:47:39.976065+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55af17c62100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-17T02:47:39.862410+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-17T02:47:39.976065+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-09-17T02:48:28.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 -- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fae10073230 msgr2=0x7fae10075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fae10073230 0x7fae10075740 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fae200b0000 tx=0x7fae200a7040 comp rx=0 tx=0).stop 2024-09-17T02:48:28.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 -- 172.21.15.131:0/860174287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae30132700 msgr2=0x7fae30132b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae30132700 0x7fae30132b10 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fae28001340 tx=0x7fae2800df00 comp rx=0 tx=0).stop 2024-09-17T02:48:28.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 -- 172.21.15.131:0/860174287 shutdown_connections 2024-09-17T02:48:28.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fae10073230 0x7fae10075740 secure :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fae200b0000 tx=0x7fae200a7040 comp rx=0 tx=0).stop 2024-09-17T02:48:28.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fae30132700 0x7fae30132b10 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fae30166a40 0x7fae301321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 --2- 172.21.15.131:0/860174287 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fae1c003a10 0x7fae30166500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 -- 172.21.15.131:0/860174287 >> 172.21.15.131:0/860174287 conn(0x7fae300d05a0 msgr2=0x7fae3006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 -- 172.21.15.131:0/860174287 shutdown_connections 2024-09-17T02:48:28.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.477+0000 7fae364a4640 1 -- 172.21.15.131:0/860174287 wait complete. 2024-09-17T02:48:28.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:48:28.492 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-17T02:48:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.633+0000 7f6e62fa5640 1 -- 172.21.15.131:0/3046427880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c0ced00 msgr2=0x7f6e5c06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.633+0000 7f6e62fa5640 1 --2- 172.21.15.131:0/3046427880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c0ced00 0x7f6e5c06dea0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0cfe00 tx=0x7f6e4c01c6e0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.633+0000 7f6e62fa5640 1 -- 172.21.15.131:0/3046427880 shutdown_connections 2024-09-17T02:48:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.633+0000 7f6e62fa5640 1 --2- 172.21.15.131:0/3046427880 >> v2:172.21.15.189:3300/0 conn(0x7f6e5c15c5b0 0x7f6e5c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.633+0000 7f6e62fa5640 1 --2- 172.21.15.131:0/3046427880 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c0ced00 0x7f6e5c06dea0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.633+0000 7f6e62fa5640 1 -- 172.21.15.131:0/3046427880 >> 172.21.15.131:0/3046427880 conn(0x7f6e5c06eea0 msgr2=0x7f6e5c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 -- 172.21.15.131:0/3046427880 shutdown_connections 2024-09-17T02:48:28.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 -- 172.21.15.131:0/3046427880 wait complete. 2024-09-17T02:48:28.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 Processor -- start 2024-09-17T02:48:28.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 -- start start 2024-09-17T02:48:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c06e3e0 0x7f6e5c156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6e5c0ced00 0x7f6e5c1659e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c15c5b0 0x7f6e5c1f9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c168750 con 0x7f6e5c06e3e0 2024-09-17T02:48:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c1685d0 con 0x7f6e5c15c5b0 2024-09-17T02:48:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6e5c1688d0 con 0x7f6e5c0ced00 2024-09-17T02:48:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e60d41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c06e3e0 0x7f6e5c156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e60d41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c06e3e0 0x7f6e5c156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58362/0 (socket says 172.21.15.131:58362) 2024-09-17T02:48:28.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e60d41640 1 -- 172.21.15.131:0/450551781 learned_addr learned my addr 172.21.15.131:0/450551781 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e61542640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c15c5b0 0x7f6e5c1f9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6e5c0ced00 0x7f6e5c1659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e60d41640 1 -- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6e5c0ced00 msgr2=0x7f6e5c1659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e60d41640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6e5c0ced00 0x7f6e5c1659e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e60d41640 1 -- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c15c5b0 msgr2=0x7f6e5c1f9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e60d41640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c15c5b0 0x7f6e5c1f9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e60d41640 1 -- 172.21.15.131:0/450551781 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5c0b4a60 con 0x7f6e5c06e3e0 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e61542640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c15c5b0 0x7f6e5c1f9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e60d41640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c06e3e0 0x7f6e5c156200 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009850 tx=0x7f6e4c0a6570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e597fa640 1 -- 172.21.15.131:0/450551781 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0ac070 con 0x7f6e5c06e3e0 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e597fa640 1 -- 172.21.15.131:0/450551781 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e4c004320 con 0x7f6e5c06e3e0 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 -- 172.21.15.131:0/450551781 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e5c0b4d40 con 0x7f6e5c06e3e0 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e597fa640 1 -- 172.21.15.131:0/450551781 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0044d0 con 0x7f6e5c06e3e0 2024-09-17T02:48:28.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.637+0000 7f6e62fa5640 1 -- 172.21.15.131:0/450551781 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e5c0b5300 con 0x7f6e5c06e3e0 2024-09-17T02:48:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.641+0000 7f6e597fa640 1 -- 172.21.15.131:0/450551781 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f6e4c0af070 con 0x7f6e5c06e3e0 2024-09-17T02:48:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.641+0000 7f6e597fa640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6e48073200 0x7f6e48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.641+0000 7f6e597fa640 1 -- 172.21.15.131:0/450551781 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f6e4c12e5e0 con 0x7f6e5c06e3e0 2024-09-17T02:48:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.641+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6e48073200 0x7f6e48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.641+0000 7f6e6a270000 1 Processor -- start 2024-09-17T02:48:28.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.641+0000 7f6e6a270000 1 -- start start 2024-09-17T02:48:28.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.641+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6e48073200 0x7f6e48075710 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c151f60 tx=0x7f6e50008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d8599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d344890 0x55931d859f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d85a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55931d6f8cd0 con 0x55931d6ed6d0 2024-09-17T02:48:28.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55931d60e330 con 0x55931d408aa0 2024-09-17T02:48:28.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55931d66bbb0 con 0x55931d344890 2024-09-17T02:48:28.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e60d41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d8599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e60d41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d8599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58378/0 (socket says 172.21.15.131:58378) 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e60d41640 1 -- 172.21.15.131:0/1286561839 learned_addr learned my addr 172.21.15.131:0/1286561839 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e61542640 1 --2- 172.21.15.131:0/1286561839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d85a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/1286561839 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d344890 0x55931d859f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e60d41640 1 -- 172.21.15.131:0/1286561839 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d344890 msgr2=0x55931d859f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e60d41640 1 --2- 172.21.15.131:0/1286561839 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d344890 0x55931d859f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e60d41640 1 -- 172.21.15.131:0/1286561839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 msgr2=0x55931d85a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e60d41640 1 --2- 172.21.15.131:0/1286561839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d85a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e60d41640 1 -- 172.21.15.131:0/1286561839 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55931d60fc80 con 0x55931d6ed6d0 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e61542640 1 --2- 172.21.15.131:0/1286561839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d85a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e60d41640 1 --2- 172.21.15.131:0/1286561839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d8599f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c12c810 tx=0x7f6e4c0f8b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ad09dffe142629 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e427fc640 1 -- 172.21.15.131:0/1286561839 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0f9370 con 0x55931d6ed6d0 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e427fc640 1 -- 172.21.15.131:0/1286561839 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e4c0f8c70 con 0x55931d6ed6d0 2024-09-17T02:48:28.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e427fc640 1 -- 172.21.15.131:0/1286561839 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c1023d0 con 0x55931d6ed6d0 2024-09-17T02:48:28.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 -- 172.21.15.131:0/1286561839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 msgr2=0x55931d8599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1286561839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d8599f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c12c810 tx=0x7f6e4c0f8b70 comp rx=0 tx=0).stop 2024-09-17T02:48:28.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 -- 172.21.15.131:0/1286561839 shutdown_connections 2024-09-17T02:48:28.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1286561839 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d85a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1286561839 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d344890 0x55931d859f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1286561839 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d8599f0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 -- 172.21.15.131:0/1286561839 >> 172.21.15.131:0/1286561839 conn(0x55931d3185c0 msgr2=0x55931d816ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 -- 172.21.15.131:0/1286561839 shutdown_connections 2024-09-17T02:48:28.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 -- 172.21.15.131:0/1286561839 wait complete. 2024-09-17T02:48:28.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 -- start start 2024-09-17T02:48:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.645+0000 7f6e6a270000 1 Processor -- start 2024-09-17T02:48:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d860e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d8613c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d6ed6d0 0x55931d861900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55931d65edf0 con 0x55931d344890 2024-09-17T02:48:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55931d65afe0 con 0x55931d408aa0 2024-09-17T02:48:28.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55931d6004d0 con 0x55931d6ed6d0 2024-09-17T02:48:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 10 client.0 _ll_register_callbacks cb 0x55931d5de7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e60d41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d860e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e60d41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d860e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58382/0 (socket says 172.21.15.131:58382) 2024-09-17T02:48:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e60d41640 1 -- 172.21.15.131:0/1223573743 learned_addr learned my addr 172.21.15.131:0/1223573743 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d8613c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e61542640 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d6ed6d0 0x55931d861900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e60d41640 1 -- 172.21.15.131:0/1223573743 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d6ed6d0 msgr2=0x55931d861900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e60d41640 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d6ed6d0 0x55931d861900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e60d41640 1 -- 172.21.15.131:0/1223573743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 msgr2=0x55931d8613c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e60d41640 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d8613c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e60d41640 1 -- 172.21.15.131:0/1223573743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55931d5f9210 con 0x55931d344890 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e60d41640 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d860e80 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c01cbc0 tx=0x7f6e4c102420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3814a17644970c94 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1223573743 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0fd750 con 0x55931d344890 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1223573743 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f6e4c0bcd90 con 0x55931d344890 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 -- 172.21.15.131:0/1223573743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55931d5f3350 con 0x55931d344890 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1223573743 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f6e4c102600 con 0x55931d344890 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 10 client.7437 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 -- 172.21.15.131:0/1223573743 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55931d5dffc0 con 0x55931d344890 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1223573743 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55931d5f3350 con 0x55931d344890 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1223573743 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f6e4c0b9070 con 0x55931d344890 2024-09-17T02:48:28.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 10 client.7437 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 10 client.7437 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 20 client.7437 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55931d2fd930 0x55931d86b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 4 client.7437 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 -- 172.21.15.131:0/1223573743 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55931d66db00 con 0x55931d2fd930 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55931d2fd930 0x55931d86b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e3b7fe640 10 client.7437 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55931d2fd930 0x55931d86b260 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e788d0a7f068c09 server_cookie=9fbf9add39a408b in_seq=0 out_seq=0 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1223573743 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55931d66db00 con 0x55931d2fd930 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e3b7fe640 10 client.7437 handle_command_reply: tid=0 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 client.7437 shutdown 2024-09-17T02:48:28.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 -- 172.21.15.131:0/1223573743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 msgr2=0x55931d860e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d860e80 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c01cbc0 tx=0x7f6e4c102420 comp rx=0 tx=0).stop 2024-09-17T02:48:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 -- 172.21.15.131:0/1223573743 shutdown_connections 2024-09-17T02:48:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d6ed6d0 0x55931d861900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d8613c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55931d2fd930 0x55931d86b260 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1223573743 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d860e80 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 -- 172.21.15.131:0/1223573743 >> 172.21.15.131:0/1223573743 conn(0x55931d3185c0 msgr2=0x55931d857b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e3b7fe640 0 client.7437 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 -- 172.21.15.131:0/1223573743 shutdown_connections 2024-09-17T02:48:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 1 -- 172.21.15.131:0/1223573743 wait complete. 2024-09-17T02:48:28.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.649+0000 7f6e6a270000 20 client.7437 trim_cache size 0 max 16384 2024-09-17T02:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e6a270000 1 Processor -- start 2024-09-17T02:48:28.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e6a270000 1 -- start start 2024-09-17T02:48:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d344890 0x55931d867500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d813f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d814480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e6a270000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55931d812ae0 con 0x55931d6ed6d0 2024-09-17T02:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e6a270000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c203730 con 0x55931d408aa0 2024-09-17T02:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e6a270000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6e5c202710 con 0x55931d344890 2024-09-17T02:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e61542640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d814480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e61542640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d814480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58384/0 (socket says 172.21.15.131:58384) 2024-09-17T02:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e61542640 1 -- 172.21.15.131:0/2189704317 learned_addr learned my addr 172.21.15.131:0/2189704317 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/2189704317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d813f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e61542640 1 -- 172.21.15.131:0/2189704317 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d344890 msgr2=0x55931d867500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e60d41640 1 --2- 172.21.15.131:0/2189704317 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d344890 0x55931d867500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e61542640 1 --2- 172.21.15.131:0/2189704317 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d344890 0x55931d867500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e61542640 1 -- 172.21.15.131:0/2189704317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 msgr2=0x55931d813f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e61542640 1 --2- 172.21.15.131:0/2189704317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d813f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e61542640 1 -- 172.21.15.131:0/2189704317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55931d6b8680 con 0x55931d6ed6d0 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/2189704317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d813f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e61542640 1 --2- 172.21.15.131:0/2189704317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d814480 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009e80 tx=0x7f6e54009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6899f83506b8101 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e41ffb640 1 -- 172.21.15.131:0/2189704317 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5400f410 con 0x55931d6ed6d0 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e41ffb640 1 -- 172.21.15.131:0/2189704317 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e5400f6c0 con 0x55931d6ed6d0 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.669+0000 7f6e41ffb640 1 -- 172.21.15.131:0/2189704317 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5400fb10 con 0x55931d6ed6d0 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- 172.21.15.131:0/2189704317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 msgr2=0x55931d814480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 --2- 172.21.15.131:0/2189704317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d814480 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009e80 tx=0x7f6e54009f70 comp rx=0 tx=0).stop 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- 172.21.15.131:0/2189704317 shutdown_connections 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 --2- 172.21.15.131:0/2189704317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d6ed6d0 0x55931d814480 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 --2- 172.21.15.131:0/2189704317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d813f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 --2- 172.21.15.131:0/2189704317 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d344890 0x55931d867500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- 172.21.15.131:0/2189704317 >> 172.21.15.131:0/2189704317 conn(0x55931d3185c0 msgr2=0x55931d80cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- 172.21.15.131:0/2189704317 shutdown_connections 2024-09-17T02:48:28.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- 172.21.15.131:0/2189704317 wait complete. 2024-09-17T02:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- start start 2024-09-17T02:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 Processor -- start 2024-09-17T02:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d8664d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d866a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d6ed6d0 0x55931d88b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c202060 con 0x55931d344890 2024-09-17T02:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c1f98a0 con 0x55931d408aa0 2024-09-17T02:48:28.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55931d320680 con 0x55931d6ed6d0 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 10 client.0 _ll_register_callbacks cb 0x7f6e4c0f8a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e60d41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d8664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e60d41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d8664d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58386/0 (socket says 172.21.15.131:58386) 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e60d41640 1 -- 172.21.15.131:0/1389382476 learned_addr learned my addr 172.21.15.131:0/1389382476 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d866a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e60d41640 1 -- 172.21.15.131:0/1389382476 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d6ed6d0 msgr2=0x55931d88b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e61542640 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d6ed6d0 0x55931d88b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e60d41640 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d6ed6d0 0x55931d88b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e60d41640 1 -- 172.21.15.131:0/1389382476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 msgr2=0x55931d866a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e60d41640 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d866a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e60d41640 1 -- 172.21.15.131:0/1389382476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55931d6250b0 con 0x55931d344890 2024-09-17T02:48:28.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d866a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e60d41640 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d8664d0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c01cf80 tx=0x7f6e4c0f8c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2233795f39fd58d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1389382476 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0f83e0 con 0x55931d344890 2024-09-17T02:48:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1389382476 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f6e4c0fd1d0 con 0x55931d344890 2024-09-17T02:48:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- 172.21.15.131:0/1389382476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e5400f6c0 con 0x55931d344890 2024-09-17T02:48:28.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1389382476 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f6e4c0f9370 con 0x55931d344890 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 10 client.7443 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- 172.21.15.131:0/1389382476 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55931d5f1610 con 0x55931d344890 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1389382476 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e5400f6c0 con 0x55931d344890 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1389382476 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f6e4c0b9070 con 0x55931d344890 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 10 client.7443 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 10 client.7443 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 20 client.7443 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55931d2eb990 0x55931d88c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 4 client.7443 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e6a270000 1 -- 172.21.15.131:0/1389382476 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55931d6695b0 con 0x55931d2eb990 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.673+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55931d2eb990 0x55931d88c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e3b7fe640 10 client.7443 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:28.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e5bfff640 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55931d2eb990 0x55931d88c0e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9c4cc594777580e server_cookie=4544db78c23767fb in_seq=0 out_seq=0 2024-09-17T02:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e3b7fe640 1 -- 172.21.15.131:0/1389382476 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+864 (crc 0 0 0) 0x55931d6695b0 con 0x55931d2eb990 2024-09-17T02:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e3b7fe640 10 client.7443 handle_command_reply: tid=0 2024-09-17T02:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 1 client.7443 shutdown 2024-09-17T02:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 1 -- 172.21.15.131:0/1389382476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 msgr2=0x55931d8664d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d8664d0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c01cf80 tx=0x7f6e4c0f8c70 comp rx=0 tx=0).stop 2024-09-17T02:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 1 -- 172.21.15.131:0/1389382476 shutdown_connections 2024-09-17T02:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55931d6ed6d0 0x55931d88b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55931d408aa0 0x55931d866a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55931d2eb990 0x55931d88c0e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 1 --2- 172.21.15.131:0/1389382476 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55931d344890 0x55931d8664d0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e3b7fe640 0 client.7443 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 1 -- 172.21.15.131:0/1389382476 >> 172.21.15.131:0/1389382476 conn(0x55931d3185c0 msgr2=0x55931d813f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 1 -- 172.21.15.131:0/1389382476 shutdown_connections 2024-09-17T02:48:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 1 -- 172.21.15.131:0/1389382476 wait complete. 2024-09-17T02:48:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e6a270000 20 client.7443 trim_cache size 0 max 16384 2024-09-17T02:48:28.680 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "ef0c0578-0779-4ec5-8591-f4a76a58d43d": { 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir", 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "ef0c0578-0779-4ec5-8591-f4a76a58d43d", 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,repair" 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "ef0c0578-0779-4ec5-8591-f4a76a58d43d": { 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:48:28.473004+0000", 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir", 2024-09-17T02:48:28.681 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: "9441c6a2-2616-4df3-82fb-d48465ab6abc": { 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:47:56.833277+0000", 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir", 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:48:28.682 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:48:28.683 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 -- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6e48073200 msgr2=0x7f6e48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6e48073200 0x7f6e48075710 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c151f60 tx=0x7f6e50008040 comp rx=0 tx=0).stop 2024-09-17T02:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 -- 172.21.15.131:0/450551781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c06e3e0 msgr2=0x7f6e5c156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c06e3e0 0x7f6e5c156200 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009850 tx=0x7f6e4c0a6570 comp rx=0 tx=0).stop 2024-09-17T02:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 -- 172.21.15.131:0/450551781 shutdown_connections 2024-09-17T02:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6e48073200 0x7f6e48075710 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6e5c15c5b0 0x7f6e5c1f9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6e5c0ced00 0x7f6e5c1659e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 --2- 172.21.15.131:0/450551781 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6e5c06e3e0 0x7f6e5c156200 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 -- 172.21.15.131:0/450551781 >> 172.21.15.131:0/450551781 conn(0x7f6e5c06eea0 msgr2=0x7f6e5c155650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:28.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 -- 172.21.15.131:0/450551781 shutdown_connections 2024-09-17T02:48:28.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:28.677+0000 7f6e62fa5640 1 -- 172.21.15.131:0/450551781 wait complete. 2024-09-17T02:48:28.691 INFO:tasks.cephfs.filesystem:scrub status for tag:ef0c0578-0779-4ec5-8591-f4a76a58d43d - {'path': '/dir', 'tag': 'ef0c0578-0779-4ec5-8591-f4a76a58d43d', 'options': 'recursive,repair'} 2024-09-17T02:48:45.069 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:48:45.072 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:48:58.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:48:58.693 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-17T02:48:58.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.868+0000 7f26a9a1d640 1 -- 172.21.15.131:0/1000002587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f268c004810 msgr2=0x7f268c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:58.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.868+0000 7f26a9a1d640 1 --2- 172.21.15.131:0/1000002587 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f268c004810 0x7f268c004c20 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f269800c290 tx=0x7f269801f700 comp rx=0 tx=0).stop 2024-09-17T02:48:58.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.868+0000 7f26a37fe640 1 -- 172.21.15.131:0/1000002587 reap_dead start 2024-09-17T02:48:58.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.868+0000 7f26a9a1d640 1 -- 172.21.15.131:0/1000002587 shutdown_connections 2024-09-17T02:48:58.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.868+0000 7f26a9a1d640 1 -- 172.21.15.131:0/1000002587 >> 172.21.15.131:0/1000002587 conn(0x7f26a40d1b10 msgr2=0x7f26a40d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:58.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.868+0000 7f26a9a1d640 1 -- 172.21.15.131:0/1000002587 shutdown_connections 2024-09-17T02:48:58.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.868+0000 7f26a9a1d640 1 -- 172.21.15.131:0/1000002587 wait complete. 2024-09-17T02:48:58.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a9a1d640 1 Processor -- start 2024-09-17T02:48:58.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a9a1d640 1 -- start start 2024-09-17T02:48:58.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a9a1d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268c004810 0x7f26a414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a9a1d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a40b5540 0x7f26a40b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a9a1d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26a40b5e90 0x7f26a41f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a9a1d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41631e0 con 0x7f26a40b5540 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a9a1d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4163060 con 0x7f268c004810 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a9a1d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f26a415fbe0 con 0x7f26a40b5e90 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a27fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a40b5540 0x7f26a40b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a27fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a40b5540 0x7f26a40b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41294/0 (socket says 172.21.15.131:41294) 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a27fc640 1 -- 172.21.15.131:0/2906275599 learned_addr learned my addr 172.21.15.131:0/2906275599 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a37fe640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26a40b5e90 0x7f26a41f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268c004810 0x7f26a414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a27fc640 1 -- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26a40b5e90 msgr2=0x7f26a41f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a27fc640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26a40b5e90 0x7f26a41f56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a27fc640 1 -- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268c004810 msgr2=0x7f26a414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:58.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a27fc640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268c004810 0x7f26a414eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a27fc640 1 -- 172.21.15.131:0/2906275599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a414ff30 con 0x7f26a40b5540 2024-09-17T02:48:58.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a27fc640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a40b5540 0x7f26a40b5950 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2698026410 tx=0x7f2698001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:58.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f268bfff640 1 -- 172.21.15.131:0/2906275599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26980b3030 con 0x7f26a40b5540 2024-09-17T02:48:58.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f268bfff640 1 -- 172.21.15.131:0/2906275599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2698006530 con 0x7f26a40b5540 2024-09-17T02:48:58.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a9a1d640 1 -- 172.21.15.131:0/2906275599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a41522d0 con 0x7f26a40b5540 2024-09-17T02:48:58.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f268bfff640 1 -- 172.21.15.131:0/2906275599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698003860 con 0x7f26a40b5540 2024-09-17T02:48:58.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.872+0000 7f26a9a1d640 1 -- 172.21.15.131:0/2906275599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a414f040 con 0x7f26a40b5540 2024-09-17T02:48:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.876+0000 7f268bfff640 1 -- 172.21.15.131:0/2906275599 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f26980c3060 con 0x7f26a40b5540 2024-09-17T02:48:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.876+0000 7f268bfff640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2690073200 0x7f2690075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.876+0000 7f268bfff640 1 -- 172.21.15.131:0/2906275599 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f2698131610 con 0x7f26a40b5540 2024-09-17T02:48:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.876+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2690073200 0x7f2690075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.876+0000 7f26b0ce8000 1 Processor -- start 2024-09-17T02:48:58.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.876+0000 7f26b0ce8000 1 -- start start 2024-09-17T02:48:58.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.876+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2690073200 0x7f2690075710 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2694005fd0 tx=0x7f2694005ea0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab149f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a5ff890 0x55fa2ab14f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2ab15470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fa2a9b3cd0 con 0x55fa2a9a86d0 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fa2a8c9330 con 0x55fa2a5ff890 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fa2a926bb0 con 0x55fa2a6c3aa0 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab149f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab149f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41308/0 (socket says 172.21.15.131:41308) 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26a2ffd640 1 -- 172.21.15.131:0/123072833 learned_addr learned my addr 172.21.15.131:0/123072833 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26a2ffd640 1 -- 172.21.15.131:0/123072833 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 msgr2=0x55fa2ab15470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/123072833 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2ab15470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26a2ffd640 1 -- 172.21.15.131:0/123072833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a5ff890 msgr2=0x55fa2ab14f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:58.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/123072833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a5ff890 0x55fa2ab14f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26a2ffd640 1 -- 172.21.15.131:0/123072833 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa2a8cac80 con 0x55fa2a9a86d0 2024-09-17T02:48:58.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/123072833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab149f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f269400fdd0 tx=0x7f2694012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b154c059a101db4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:58.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26897fa640 1 -- 172.21.15.131:0/123072833 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694022070 con 0x55fa2a9a86d0 2024-09-17T02:48:58.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26897fa640 1 -- 172.21.15.131:0/123072833 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f269401e450 con 0x55fa2a9a86d0 2024-09-17T02:48:58.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26897fa640 1 -- 172.21.15.131:0/123072833 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269401e800 con 0x55fa2a9a86d0 2024-09-17T02:48:58.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 -- 172.21.15.131:0/123072833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 msgr2=0x55fa2ab149f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:58.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/123072833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab149f0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f269400fdd0 tx=0x7f2694012000 comp rx=0 tx=0).stop 2024-09-17T02:48:58.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 -- 172.21.15.131:0/123072833 shutdown_connections 2024-09-17T02:48:58.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/123072833 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2ab15470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/123072833 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a5ff890 0x55fa2ab14f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/123072833 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab149f0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 -- 172.21.15.131:0/123072833 >> 172.21.15.131:0/123072833 conn(0x55fa2a5d35c0 msgr2=0x55fa2aad1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 -- 172.21.15.131:0/123072833 shutdown_connections 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 -- 172.21.15.131:0/123072833 wait complete. 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 -- start start 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.880+0000 7f26b0ce8000 1 Processor -- start 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2aad09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2aad0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a9a86d0 0x55fa2aad75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fa2a919df0 con 0x55fa2a5ff890 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fa2a915fe0 con 0x55fa2a9a86d0 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fa2a8bb4d0 con 0x55fa2a6c3aa0 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 10 client.0 _ll_register_callbacks cb 0x55fa2a8997f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2aad09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2aad09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41310/0 (socket says 172.21.15.131:41310) 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a2ffd640 1 -- 172.21.15.131:0/3885981895 learned_addr learned my addr 172.21.15.131:0/3885981895 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a37fe640 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a9a86d0 0x55fa2aad75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a27fc640 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2aad0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a2ffd640 1 -- 172.21.15.131:0/3885981895 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 msgr2=0x55fa2aad0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2aad0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a2ffd640 1 -- 172.21.15.131:0/3885981895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a9a86d0 msgr2=0x55fa2aad75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a9a86d0 0x55fa2aad75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a2ffd640 1 -- 172.21.15.131:0/3885981895 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa2a8b4210 con 0x55fa2a5ff890 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a37fe640 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a9a86d0 0x55fa2aad75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a27fc640 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2aad0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2aad09d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x55fa2a9c9760 tx=0x7f26940206a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ff76cf55a647c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26827fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26827fc640 1 -- 172.21.15.131:0/3885981895 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694035060 con 0x55fa2a5ff890 2024-09-17T02:48:58.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26827fc640 1 -- 172.21.15.131:0/3885981895 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f2694020890 con 0x55fa2a5ff890 2024-09-17T02:48:58.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3885981895 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fa2a8ae350 con 0x55fa2a5ff890 2024-09-17T02:48:58.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26827fc640 1 -- 172.21.15.131:0/3885981895 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f269403b050 con 0x55fa2a5ff890 2024-09-17T02:48:58.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 10 client.7455 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:58.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3885981895 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55fa2a89afc0 con 0x55fa2a5ff890 2024-09-17T02:48:58.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26827fc640 1 -- 172.21.15.131:0/3885981895 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55fa2a8ae350 con 0x55fa2a5ff890 2024-09-17T02:48:58.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26827fc640 1 -- 172.21.15.131:0/3885981895 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f269403e020 con 0x55fa2a5ff890 2024-09-17T02:48:58.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 10 client.7455 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:58.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 10 client.7455 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:48:58.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 20 client.7455 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:58.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55fa2a5b8930 0x55fa2ab1be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 4 client.7455 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:48:58.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3885981895 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa2a928b00 con 0x55fa2a5b8930 2024-09-17T02:48:58.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a27fc640 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55fa2a5b8930 0x55fa2ab1be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26827fc640 10 client.7455 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26a27fc640 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55fa2a5b8930 0x55fa2ab1be80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4df20831625c94c server_cookie=973601f4589f14d3 in_seq=0 out_seq=0 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26827fc640 1 -- 172.21.15.131:0/3885981895 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fa2a928b00 con 0x55fa2a5b8930 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26827fc640 10 client.7455 handle_command_reply: tid=0 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 client.7455 shutdown 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3885981895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 msgr2=0x55fa2aad09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2aad09d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x55fa2a9c9760 tx=0x7f26940206a0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3885981895 shutdown_connections 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a9a86d0 0x55fa2aad75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2aad0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55fa2a5b8930 0x55fa2ab1be80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/3885981895 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2aad09d0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26827fc640 0 client.7455 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:58.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3885981895 >> 172.21.15.131:0/3885981895 conn(0x55fa2a5d35c0 msgr2=0x55fa2aac9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:58.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3885981895 shutdown_connections 2024-09-17T02:48:58.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3885981895 wait complete. 2024-09-17T02:48:58.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.884+0000 7f26b0ce8000 20 client.7455 trim_cache size 0 max 16384 2024-09-17T02:48:58.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26b0ce8000 1 Processor -- start 2024-09-17T02:48:58.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26b0ce8000 1 -- start start 2024-09-17T02:48:58.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2ab85bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2ab86110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab86650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4203220 con 0x55fa2a5ff890 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4202200 con 0x55fa2a9a86d0 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f26a41f99d0 con 0x55fa2a6c3aa0 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2ab85bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2ab85bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41314/0 (socket says 172.21.15.131:41314) 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26a2ffd640 1 -- 172.21.15.131:0/3277711870 learned_addr learned my addr 172.21.15.131:0/3277711870 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26a2ffd640 1 -- 172.21.15.131:0/3277711870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 msgr2=0x55fa2ab86110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26a27fc640 1 --2- 172.21.15.131:0/3277711870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2ab86110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26a37fe640 1 --2- 172.21.15.131:0/3277711870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab86650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/3277711870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2ab86110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26a2ffd640 1 -- 172.21.15.131:0/3277711870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a9a86d0 msgr2=0x55fa2ab86650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/3277711870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab86650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26a2ffd640 1 -- 172.21.15.131:0/3277711870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa2a973680 con 0x55fa2a5ff890 2024-09-17T02:48:58.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.904+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/3277711870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2ab85bd0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f2694020300 tx=0x7f269400b4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349a0e7109ecb8eb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:58.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f2688ff9640 1 -- 172.21.15.131:0/3277711870 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694015040 con 0x55fa2a5ff890 2024-09-17T02:48:58.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f2688ff9640 1 -- 172.21.15.131:0/3277711870 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2694020710 con 0x55fa2a5ff890 2024-09-17T02:48:58.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f2688ff9640 1 -- 172.21.15.131:0/3277711870 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694022040 con 0x55fa2a5ff890 2024-09-17T02:48:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3277711870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 msgr2=0x55fa2ab85bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/3277711870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2ab85bd0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f2694020300 tx=0x7f269400b4d0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3277711870 shutdown_connections 2024-09-17T02:48:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/3277711870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab86650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/3277711870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a6c3aa0 0x55fa2ab86110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/3277711870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a5ff890 0x55fa2ab85bd0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3277711870 >> 172.21.15.131:0/3277711870 conn(0x55fa2a5d35c0 msgr2=0x55fa2aad65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3277711870 shutdown_connections 2024-09-17T02:48:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 -- 172.21.15.131:0/3277711870 wait complete. 2024-09-17T02:48:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 -- start start 2024-09-17T02:48:58.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 Processor -- start 2024-09-17T02:48:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a5ff890 0x55fa2ab89d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a6c3aa0 0x55fa2ab8a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab8a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fa2a919100 con 0x55fa2a9a86d0 2024-09-17T02:48:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fa2aad4d60 con 0x55fa2a6c3aa0 2024-09-17T02:48:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fa2a8ad300 con 0x55fa2a5ff890 2024-09-17T02:48:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26b0ce8000 10 client.0 _ll_register_callbacks cb 0x7f269400fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab8a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab8a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41316/0 (socket says 172.21.15.131:41316) 2024-09-17T02:48:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26a37fe640 1 -- 172.21.15.131:0/605969033 learned_addr learned my addr 172.21.15.131:0/605969033 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a5ff890 0x55fa2ab89d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26a27fc640 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a6c3aa0 0x55fa2ab8a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26a37fe640 1 -- 172.21.15.131:0/605969033 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a5ff890 msgr2=0x55fa2ab89d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26a37fe640 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a5ff890 0x55fa2ab89d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26a37fe640 1 -- 172.21.15.131:0/605969033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a6c3aa0 msgr2=0x55fa2ab8a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26a37fe640 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a6c3aa0 0x55fa2ab8a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.908+0000 7f26a37fe640 1 -- 172.21.15.131:0/605969033 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa2a8e00b0 con 0x55fa2a9a86d0 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a37fe640 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab8a7e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f26a408d6d0 tx=0x7f269c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f95def4928790a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26827fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26827fc640 1 -- 172.21.15.131:0/605969033 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c004d30 con 0x55fa2a9a86d0 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26827fc640 1 -- 172.21.15.131:0/605969033 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f269c01ad40 con 0x55fa2a9a86d0 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 -- 172.21.15.131:0/605969033 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2694020710 con 0x55fa2a9a86d0 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26827fc640 1 -- 172.21.15.131:0/605969033 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f269c009650 con 0x55fa2a9a86d0 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 10 client.7461 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 -- 172.21.15.131:0/605969033 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55fa2a8ac610 con 0x55fa2a9a86d0 2024-09-17T02:48:58.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26827fc640 1 -- 172.21.15.131:0/605969033 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2694020710 con 0x55fa2a9a86d0 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26827fc640 1 -- 172.21.15.131:0/605969033 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f269c021050 con 0x55fa2a9a86d0 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 10 client.7461 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 10 client.7461 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 20 client.7461 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55fa2a5a6990 0x55fa2ab45c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 4 client.7461 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 -- 172.21.15.131:0/605969033 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa2a9245b0 con 0x55fa2a5a6990 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55fa2a5a6990 0x55fa2ab45c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26827fc640 10 client.7461 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a2ffd640 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55fa2a5a6990 0x55fa2ab45c10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d387dcba2b3c653b server_cookie=50a3c8181ebd7840 in_seq=0 out_seq=0 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26827fc640 1 -- 172.21.15.131:0/605969033 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+664 (crc 0 0 0) 0x55fa2a9245b0 con 0x55fa2a5a6990 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26827fc640 10 client.7461 handle_command_reply: tid=0 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 client.7461 shutdown 2024-09-17T02:48:58.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 -- 172.21.15.131:0/605969033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 msgr2=0x55fa2ab8a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab8a7e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f26a408d6d0 tx=0x7f269c011810 comp rx=0 tx=0).stop 2024-09-17T02:48:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 -- 172.21.15.131:0/605969033 shutdown_connections 2024-09-17T02:48:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x55fa2a5a6990 0x55fa2ab45c10 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fa2a9a86d0 0x55fa2ab8a7e0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fa2a6c3aa0 0x55fa2ab8a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 --2- 172.21.15.131:0/605969033 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fa2a5ff890 0x55fa2ab89d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 -- 172.21.15.131:0/605969033 >> 172.21.15.131:0/605969033 conn(0x55fa2a5d35c0 msgr2=0x55fa2ab25b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26827fc640 0 client.7461 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 -- 172.21.15.131:0/605969033 shutdown_connections 2024-09-17T02:48:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 1 -- 172.21.15.131:0/605969033 wait complete. 2024-09-17T02:48:58.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26b0ce8000 20 client.7461 trim_cache size 0 max 16384 2024-09-17T02:48:58.915 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:48:58.915 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-09-17T02:48:58.915 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {}, 2024-09-17T02:48:58.915 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:48:58.915 INFO:teuthology.orchestra.run.smithi131.stdout: "ef0c0578-0779-4ec5-8591-f4a76a58d43d": { 2024-09-17T02:48:58.915 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:48:28.473004+0000", 2024-09-17T02:48:58.915 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir", 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: "9441c6a2-2616-4df3-82fb-d48465ab6abc": { 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:47:56.833277+0000", 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir", 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:48:58.916 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:48:58.917 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:48:58.917 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:48:58.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 -- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2690073200 msgr2=0x7f2690075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:58.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2690073200 0x7f2690075710 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2694005fd0 tx=0x7f2694005ea0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 -- 172.21.15.131:0/2906275599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a40b5540 msgr2=0x7f26a40b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:58.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a40b5540 0x7f26a40b5950 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2698026410 tx=0x7f2698001d40 comp rx=0 tx=0).stop 2024-09-17T02:48:58.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 -- 172.21.15.131:0/2906275599 shutdown_connections 2024-09-17T02:48:58.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2690073200 0x7f2690075710 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26a40b5e90 0x7f26a41f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26a40b5540 0x7f26a40b5950 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 --2- 172.21.15.131:0/2906275599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f268c004810 0x7f26a414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:58.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 -- 172.21.15.131:0/2906275599 >> 172.21.15.131:0/2906275599 conn(0x7f26a40d1b10 msgr2=0x7f26a4068fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:58.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 -- 172.21.15.131:0/2906275599 shutdown_connections 2024-09-17T02:48:58.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:58.912+0000 7f26a9a1d640 1 -- 172.21.15.131:0/2906275599 wait complete. 2024-09-17T02:48:58.929 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-17T02:48:58.930 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:48:59.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.080+0000 7f564e151640 1 -- 172.21.15.131:0/1259024592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634004d50 msgr2=0x7f5634005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.080+0000 7f564e151640 1 --2- 172.21.15.131:0/1259024592 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5634004d50 0x7f5634005200 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f56480cdba0 tx=0x7f564001e390 comp rx=0 tx=0).stop 2024-09-17T02:48:59.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.080+0000 7f5647fff640 1 -- 172.21.15.131:0/1259024592 reap_dead start 2024-09-17T02:48:59.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.080+0000 7f564e151640 1 -- 172.21.15.131:0/1259024592 shutdown_connections 2024-09-17T02:48:59.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.080+0000 7f564e151640 1 -- 172.21.15.131:0/1259024592 >> 172.21.15.131:0/1259024592 conn(0x7f564806eea0 msgr2=0x7f564806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:59.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.080+0000 7f564e151640 1 -- 172.21.15.131:0/1259024592 shutdown_connections 2024-09-17T02:48:59.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.080+0000 7f564e151640 1 -- 172.21.15.131:0/1259024592 wait complete. 2024-09-17T02:48:59.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f564e151640 1 Processor -- start 2024-09-17T02:48:59.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f564e151640 1 -- start start 2024-09-17T02:48:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f564e151640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5634004d50 0x7f564815f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f564e151640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56480b0c20 0x7f56480b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f564e151640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f56481605d0 0x7f56481609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f564e151640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f564806fee0 con 0x7f56480b0c20 2024-09-17T02:48:59.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f564e151640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f564806fd60 con 0x7f5634004d50 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f564e151640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5648070060 con 0x7f56481605d0 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f5646ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56480b0c20 0x7f56480b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f5646ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56480b0c20 0x7f56480b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41336/0 (socket says 172.21.15.131:41336) 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f5646ffd640 1 -- 172.21.15.131:0/2052990766 learned_addr learned my addr 172.21.15.131:0/2052990766 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f5647fff640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f56481605d0 0x7f56481609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f56477fe640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5634004d50 0x7f564815f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f5646ffd640 1 -- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f56481605d0 msgr2=0x7f56481609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f5646ffd640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f56481605d0 0x7f56481609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f5646ffd640 1 -- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5634004d50 msgr2=0x7f564815f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f5646ffd640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5634004d50 0x7f564815f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f5646ffd640 1 -- 172.21.15.131:0/2052990766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56481f9040 con 0x7f56480b0c20 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f5646ffd640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56480b0c20 0x7f56480b1030 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f56380018d0 tx=0x7f5638001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:59.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f563ffff640 1 -- 172.21.15.131:0/2052990766 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638015070 con 0x7f56480b0c20 2024-09-17T02:48:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f563ffff640 1 -- 172.21.15.131:0/2052990766 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f563800eb50 con 0x7f56480b0c20 2024-09-17T02:48:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f564e151640 1 -- 172.21.15.131:0/2052990766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56481f9320 con 0x7f56480b0c20 2024-09-17T02:48:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f563ffff640 1 -- 172.21.15.131:0/2052990766 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56380020d0 con 0x7f56480b0c20 2024-09-17T02:48:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.084+0000 7f564e151640 1 -- 172.21.15.131:0/2052990766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5648160f20 con 0x7f56480b0c20 2024-09-17T02:48:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.088+0000 7f564e151640 1 -- 172.21.15.131:0/2052990766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56481f8db0 con 0x7f56480b0c20 2024-09-17T02:48:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.088+0000 7f563ffff640 1 -- 172.21.15.131:0/2052990766 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5638018070 con 0x7f56480b0c20 2024-09-17T02:48:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.088+0000 7f563ffff640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5628073230 0x7f5628075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.088+0000 7f563ffff640 1 -- 172.21.15.131:0/2052990766 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f5638094df0 con 0x7f56480b0c20 2024-09-17T02:48:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.088+0000 7f56477fe640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5628073230 0x7f5628075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.088+0000 7f56477fe640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5628073230 0x7f5628075740 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5630001890 tx=0x7f5630001d20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:59.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.092+0000 7f563ffff640 1 -- 172.21.15.131:0/2052990766 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f563805fc00 con 0x7f56480b0c20 2024-09-17T02:48:59.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f564e151640 1 -- 172.21.15.131:0/2052990766 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56481f9a40 con 0x7f56480b0c20 2024-09-17T02:48:59.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f563ffff640 1 -- 172.21.15.131:0/2052990766 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5238 (secure 0 0 0) 0x7f563805f500 con 0x7f56480b0c20 2024-09-17T02:48:59.285 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:48:59.285 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":71,"btime":"2024-09-17T02:47:59:359117+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:47:59.122657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7233":{"gid":7233,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6835/3013984008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3013984008},{"type":"v1","addr":"172.21.15.131:6835","nonce":3013984008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7233,"qdb_cluster":[7233]},"id":6}]} 2024-09-17T02:48:59.285 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 71 2024-09-17T02:48:59.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f564e151640 1 -- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5628073230 msgr2=0x7f5628075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f564e151640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5628073230 0x7f5628075740 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5630001890 tx=0x7f5630001d20 comp rx=0 tx=0).stop 2024-09-17T02:48:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f564e151640 1 -- 172.21.15.131:0/2052990766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56480b0c20 msgr2=0x7f56480b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f564e151640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56480b0c20 0x7f56480b1030 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f56380018d0 tx=0x7f5638001d90 comp rx=0 tx=0).stop 2024-09-17T02:48:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f564e151640 1 -- 172.21.15.131:0/2052990766 shutdown_connections 2024-09-17T02:48:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f564e151640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5628073230 0x7f5628075740 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f564e151640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f56481605d0 0x7f56481609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f564e151640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f56480b0c20 0x7f56480b1030 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f564e151640 1 --2- 172.21.15.131:0/2052990766 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5634004d50 0x7f564815f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.288+0000 7f564e151640 1 -- 172.21.15.131:0/2052990766 >> 172.21.15.131:0/2052990766 conn(0x7f564806eea0 msgr2=0x7f56480afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.292+0000 7f564e151640 1 -- 172.21.15.131:0/2052990766 shutdown_connections 2024-09-17T02:48:59.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.292+0000 7f564e151640 1 -- 172.21.15.131:0/2052990766 wait complete. 2024-09-17T02:48:59.305 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-17T02:48:59.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.452+0000 7f84cea4a640 1 -- 172.21.15.131:0/2323292309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84b0004810 msgr2=0x7f84b0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.452+0000 7f84cea4a640 1 --2- 172.21.15.131:0/2323292309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84b0004810 0x7f84b0004c20 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0016f0 tx=0x7f84bc01c750 comp rx=0 tx=0).stop 2024-09-17T02:48:59.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.452+0000 7f84cea4a640 1 -- 172.21.15.131:0/2323292309 shutdown_connections 2024-09-17T02:48:59.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.452+0000 7f84cea4a640 1 --2- 172.21.15.131:0/2323292309 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84b0004810 0x7f84b0004c20 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.452+0000 7f84cea4a640 1 -- 172.21.15.131:0/2323292309 >> 172.21.15.131:0/2323292309 conn(0x7f84c806eea0 msgr2=0x7f84c806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:59.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.452+0000 7f84cea4a640 1 -- 172.21.15.131:0/2323292309 shutdown_connections 2024-09-17T02:48:59.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.452+0000 7f84cea4a640 1 -- 172.21.15.131:0/2323292309 wait complete. 2024-09-17T02:48:59.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84cea4a640 1 Processor -- start 2024-09-17T02:48:59.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84cea4a640 1 -- start start 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84cea4a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84b0004810 0x7f84c80aa020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84cea4a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84c81f3ed0 0x7f84c80afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84cea4a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84c80b0270 0x7f84c80b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84cea4a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f84c815c020 con 0x7f84b0004810 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84cea4a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f84c815bea0 con 0x7f84c80b0270 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84cea4a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f84c815c1a0 con 0x7f84c81f3ed0 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84b0004810 0x7f84c80aa020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84b0004810 0x7f84c80aa020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41360/0 (socket says 172.21.15.131:41360) 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c7fff640 1 -- 172.21.15.131:0/2922956891 learned_addr learned my addr 172.21.15.131:0/2922956891 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84ccfe7640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84c80b0270 0x7f84c80b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c7fff640 1 -- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84c81f3ed0 msgr2=0x7f84c80afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c7fff640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84c81f3ed0 0x7f84c80afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c7fff640 1 -- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84c80b0270 msgr2=0x7f84c80b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c7fff640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84c80b0270 0x7f84c80b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c7fff640 1 -- 172.21.15.131:0/2922956891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c81f56e0 con 0x7f84b0004810 2024-09-17T02:48:59.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84ccfe7640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84c80b0270 0x7f84c80b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:59.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c7fff640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84b0004810 0x7f84c80aa020 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f84c0011790 tx=0x7f84c0011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:59.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c4ff9640 1 -- 172.21.15.131:0/2922956891 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0011e40 con 0x7f84b0004810 2024-09-17T02:48:59.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c4ff9640 1 -- 172.21.15.131:0/2922956891 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84c0011320 con 0x7f84b0004810 2024-09-17T02:48:59.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84cea4a640 1 -- 172.21.15.131:0/2922956891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c81f4640 con 0x7f84b0004810 2024-09-17T02:48:59.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84c4ff9640 1 -- 172.21.15.131:0/2922956891 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c000ca80 con 0x7f84b0004810 2024-09-17T02:48:59.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.456+0000 7f84cea4a640 1 -- 172.21.15.131:0/2922956891 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c81f4c00 con 0x7f84b0004810 2024-09-17T02:48:59.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c4ff9640 1 -- 172.21.15.131:0/2922956891 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f84c000cc20 con 0x7f84b0004810 2024-09-17T02:48:59.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c4ff9640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84b4073200 0x7f84b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c4ff9640 1 -- 172.21.15.131:0/2922956891 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f84c00993b0 con 0x7f84b0004810 2024-09-17T02:48:59.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c77fe640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84b4073200 0x7f84b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84d5d15000 1 Processor -- start 2024-09-17T02:48:59.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84d5d15000 1 -- start start 2024-09-17T02:48:59.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c77fe640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84b4073200 0x7f84b4075710 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0016c0 tx=0x7f84b0005840 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:59.460 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f184ef6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 0x560f184efc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 0x560f184f0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84d5d15000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560f18389d00 con 0x560f183a5b60 2024-09-17T02:48:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84d5d15000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560f182fdc90 con 0x560f18384d90 2024-09-17T02:48:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84d5d15000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x560f182ed0c0 con 0x560f1839b1c0 2024-09-17T02:48:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f184ef6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f184ef6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41366/0 (socket says 172.21.15.131:41366) 2024-09-17T02:48:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c7fff640 1 -- 172.21.15.131:0/4251872848 learned_addr learned my addr 172.21.15.131:0/4251872848 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c77fe640 1 --2- 172.21.15.131:0/4251872848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 0x560f184efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84ccfe7640 1 --2- 172.21.15.131:0/4251872848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 0x560f184f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c7fff640 1 -- 172.21.15.131:0/4251872848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 msgr2=0x560f184efc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c7fff640 1 --2- 172.21.15.131:0/4251872848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 0x560f184efc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c7fff640 1 -- 172.21.15.131:0/4251872848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 msgr2=0x560f184f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c7fff640 1 --2- 172.21.15.131:0/4251872848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 0x560f184f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c7fff640 1 -- 172.21.15.131:0/4251872848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f182c8290 con 0x560f183a5b60 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c77fe640 1 --2- 172.21.15.131:0/4251872848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 0x560f184efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84ccfe7640 1 --2- 172.21.15.131:0/4251872848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 0x560f184f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.460+0000 7f84c7fff640 1 --2- 172.21.15.131:0/4251872848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f184ef6d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f84c00975b0 tx=0x7f84c0063690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc6773b5e63a499 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84adffb640 1 -- 172.21.15.131:0/4251872848 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c00685a0 con 0x560f183a5b60 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84adffb640 1 -- 172.21.15.131:0/4251872848 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84c0068030 con 0x560f183a5b60 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84adffb640 1 -- 172.21.15.131:0/4251872848 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c006d450 con 0x560f183a5b60 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 -- 172.21.15.131:0/4251872848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 msgr2=0x560f184ef6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 --2- 172.21.15.131:0/4251872848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f184ef6d0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f84c00975b0 tx=0x7f84c0063690 comp rx=0 tx=0).stop 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 -- 172.21.15.131:0/4251872848 shutdown_connections 2024-09-17T02:48:59.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 --2- 172.21.15.131:0/4251872848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 0x560f184f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 --2- 172.21.15.131:0/4251872848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 0x560f184efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 --2- 172.21.15.131:0/4251872848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f184ef6d0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 -- 172.21.15.131:0/4251872848 >> 172.21.15.131:0/4251872848 conn(0x560f17faa760 msgr2=0x560f184ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 -- 172.21.15.131:0/4251872848 shutdown_connections 2024-09-17T02:48:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 -- 172.21.15.131:0/4251872848 wait complete. 2024-09-17T02:48:59.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 -- start start 2024-09-17T02:48:59.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 Processor -- start 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f18384d90 0x560f184afb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f1839b1c0 0x560f184b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f183a5b60 0x560f184b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560f18287940 con 0x560f18384d90 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560f182f0ed0 con 0x560f1839b1c0 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x560f17fb2820 con 0x560f183a5b60 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84c7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f18384d90 0x560f184afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 10 client.0 _ll_register_callbacks cb 0x560f182cb7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84c7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f18384d90 0x560f184afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41374/0 (socket says 172.21.15.131:41374) 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84c7fff640 1 -- 172.21.15.131:0/421620808 learned_addr learned my addr 172.21.15.131:0/421620808 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84c7fff640 1 -- 172.21.15.131:0/421620808 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f183a5b60 msgr2=0x560f184b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84c7fff640 1 --2- 172.21.15.131:0/421620808 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f183a5b60 0x560f184b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84c7fff640 1 -- 172.21.15.131:0/421620808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f1839b1c0 msgr2=0x560f184b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:59.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84c7fff640 1 --2- 172.21.15.131:0/421620808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f1839b1c0 0x560f184b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84c7fff640 1 -- 172.21.15.131:0/421620808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f182a0550 con 0x560f18384d90 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84c7fff640 1 --2- 172.21.15.131:0/421620808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f18384d90 0x560f184afb00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f84c0097690 tx=0x7f84c00601c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0855ad9e40ce556 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84a6ffd640 1 -- 172.21.15.131:0/421620808 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c0051180 con 0x560f18384d90 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84a6ffd640 1 -- 172.21.15.131:0/421620808 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f84c00681f0 con 0x560f18384d90 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.464+0000 7f84d5d15000 1 -- 172.21.15.131:0/421620808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f18285400 con 0x560f18384d90 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84a6ffd640 1 -- 172.21.15.131:0/421620808 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f84c00685a0 con 0x560f18384d90 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 10 client.7479 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 -- 172.21.15.131:0/421620808 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x560f1828b480 con 0x560f18384d90 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84a6ffd640 1 -- 172.21.15.131:0/421620808 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560f18285400 con 0x560f18384d90 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84a6ffd640 1 -- 172.21.15.131:0/421620808 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f84c0099050 con 0x560f18384d90 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 10 client.7479 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 10 client.7479 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:48:59.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 20 client.7479 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 --2- 172.21.15.131:0/421620808 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x560f17f8fad0 0x560f184ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 4 client.7479 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 -- 172.21.15.131:0/421620808 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f182ffbe0 con 0x560f17f8fad0 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84c77fe640 1 --2- 172.21.15.131:0/421620808 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x560f17f8fad0 0x560f184ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84a6ffd640 10 client.7479 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84c77fe640 1 --2- 172.21.15.131:0/421620808 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x560f17f8fad0 0x560f184ffc80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8d8a4ebab1dbbb server_cookie=e72687c988a31bfc in_seq=0 out_seq=0 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84a6ffd640 1 -- 172.21.15.131:0/421620808 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f182ffbe0 con 0x560f17f8fad0 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84a6ffd640 10 client.7479 handle_command_reply: tid=0 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 client.7479 shutdown 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 -- 172.21.15.131:0/421620808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f18384d90 msgr2=0x560f184afb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 --2- 172.21.15.131:0/421620808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f18384d90 0x560f184afb00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f84c0097690 tx=0x7f84c00601c0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 -- 172.21.15.131:0/421620808 shutdown_connections 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 --2- 172.21.15.131:0/421620808 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f183a5b60 0x560f184b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 --2- 172.21.15.131:0/421620808 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f1839b1c0 0x560f184b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 --2- 172.21.15.131:0/421620808 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x560f17f8fad0 0x560f184ffc80 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 --2- 172.21.15.131:0/421620808 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f18384d90 0x560f184afb00 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84a6ffd640 0 client.7479 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 -- 172.21.15.131:0/421620808 >> 172.21.15.131:0/421620808 conn(0x560f17faa760 msgr2=0x560f1849e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 -- 172.21.15.131:0/421620808 shutdown_connections 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 1 -- 172.21.15.131:0/421620808 wait complete. 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.468+0000 7f84d5d15000 20 client.7479 trim_cache size 0 max 16384 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.480+0000 7f84d5d15000 1 Processor -- start 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.480+0000 7f84d5d15000 1 -- start start 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f18384d90 0x560f184fec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f1839b1c0 0x560f184ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f183a5b60 0x560f184ff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560f18287940 con 0x560f1839b1c0 2024-09-17T02:48:59.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560f182f0ed0 con 0x560f183a5b60 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x560f184ab510 con 0x560f18384d90 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84c77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f1839b1c0 0x560f184ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84c77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f1839b1c0 0x560f184ff1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41390/0 (socket says 172.21.15.131:41390) 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84c77fe640 1 -- 172.21.15.131:0/3543565045 learned_addr learned my addr 172.21.15.131:0/3543565045 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84c77fe640 1 -- 172.21.15.131:0/3543565045 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f18384d90 msgr2=0x560f184fec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84c77fe640 1 --2- 172.21.15.131:0/3543565045 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f18384d90 0x560f184fec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84c77fe640 1 -- 172.21.15.131:0/3543565045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f183a5b60 msgr2=0x560f184ff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84c77fe640 1 --2- 172.21.15.131:0/3543565045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f183a5b60 0x560f184ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84c77fe640 1 -- 172.21.15.131:0/3543565045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f18372d10 con 0x560f1839b1c0 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84c77fe640 1 --2- 172.21.15.131:0/3543565045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f1839b1c0 0x560f184ff1c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f84bc01b020 tx=0x7f84bc0b0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3797b44a331cf95c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84ad7fa640 1 -- 172.21.15.131:0/3543565045 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0a55f0 con 0x560f1839b1c0 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84ad7fa640 1 -- 172.21.15.131:0/3543565045 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84bc0a9040 con 0x560f1839b1c0 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84ad7fa640 1 -- 172.21.15.131:0/3543565045 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84bc0a59b0 con 0x560f1839b1c0 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 -- 172.21.15.131:0/3543565045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f1839b1c0 msgr2=0x560f184ff1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 --2- 172.21.15.131:0/3543565045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f1839b1c0 0x560f184ff1c0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f84bc01b020 tx=0x7f84bc0b0000 comp rx=0 tx=0).stop 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 -- 172.21.15.131:0/3543565045 shutdown_connections 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 --2- 172.21.15.131:0/3543565045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f183a5b60 0x560f184ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 --2- 172.21.15.131:0/3543565045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f1839b1c0 0x560f184ff1c0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 --2- 172.21.15.131:0/3543565045 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f18384d90 0x560f184fec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 -- 172.21.15.131:0/3543565045 >> 172.21.15.131:0/3543565045 conn(0x560f17faa760 msgr2=0x560f1855cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 -- 172.21.15.131:0/3543565045 shutdown_connections 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 -- 172.21.15.131:0/3543565045 wait complete. 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 -- start start 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.484+0000 7f84d5d15000 1 Processor -- start 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 0x560f18565360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 0x560f185658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f18565de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560f18298300 con 0x560f183a5b60 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560f183acfc0 con 0x560f18384d90 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x560f182fca60 con 0x560f1839b1c0 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 10 client.0 _ll_register_callbacks cb 0x7f84c0088070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84ccfe7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f18565de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84ccfe7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f18565de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41394/0 (socket says 172.21.15.131:41394) 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84ccfe7640 1 -- 172.21.15.131:0/2603185240 learned_addr learned my addr 172.21.15.131:0/2603185240 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84c7fff640 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 0x560f18565360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84c77fe640 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 0x560f185658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84ccfe7640 1 -- 172.21.15.131:0/2603185240 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 msgr2=0x560f185658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84ccfe7640 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 0x560f185658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84ccfe7640 1 -- 172.21.15.131:0/2603185240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 msgr2=0x560f18565360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84ccfe7640 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 0x560f18565360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84ccfe7640 1 -- 172.21.15.131:0/2603185240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f182a3e50 con 0x560f183a5b60 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84c77fe640 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 0x560f185658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:48:59.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84c7fff640 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 0x560f18565360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84ccfe7640 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f18565de0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005c50 tx=0x7f84b8011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddec0b83f0235448 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84a6ffd640 1 -- 172.21.15.131:0/2603185240 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b801f060 con 0x560f183a5b60 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84a6ffd640 1 -- 172.21.15.131:0/2603185240 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f84b800fca0 con 0x560f183a5b60 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 -- 172.21.15.131:0/2603185240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84bc0a9040 con 0x560f183a5b60 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84a6ffd640 1 -- 172.21.15.131:0/2603185240 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f84b8025050 con 0x560f183a5b60 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 10 client.7485 fetch_fsmap learned FSMap version 71 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 -- 172.21.15.131:0/2603185240 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x560f182922d0 con 0x560f183a5b60 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84a6ffd640 1 -- 172.21.15.131:0/2603185240 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84bc0a9040 con 0x560f183a5b60 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84a6ffd640 1 -- 172.21.15.131:0/2603185240 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f84b8028020 con 0x560f183a5b60 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 10 client.7485 fetch_fsmap finished waiting for FSMap version 71 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 10 client.7485 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 20 client.7485 populate_metadata read hostname 'smithi131' 2024-09-17T02:48:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x560f17f7db30 0x560f1854a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 4 client.7485 mds_command: new command op to 7233 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 -- 172.21.15.131:0/2603185240 --> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x560f182fb690 con 0x560f17f7db30 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84c7fff640 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x560f17f7db30 0x560f1854a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84a6ffd640 10 client.7485 ms_handle_connect on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84c7fff640 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x560f17f7db30 0x560f1854a370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c09d4b5dab0bf797 server_cookie=f689c985c5f7a1c2 in_seq=0 out_seq=0 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84a6ffd640 1 -- 172.21.15.131:0/2603185240 <== mds.0 v2:172.21.15.131:6834/3013984008 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x560f182fb690 con 0x560f17f7db30 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84a6ffd640 10 client.7485 handle_command_reply: tid=0 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 client.7485 shutdown 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 -- 172.21.15.131:0/2603185240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 msgr2=0x560f18565de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f18565de0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005c50 tx=0x7f84b8011cb0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 -- 172.21.15.131:0/2603185240 shutdown_connections 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.131:6834/3013984008,v1:172.21.15.131:6835/3013984008] conn(0x560f17f7db30 0x560f1854a370 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560f183a5b60 0x560f18565de0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560f1839b1c0 0x560f185658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 --2- 172.21.15.131:0/2603185240 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560f18384d90 0x560f18565360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84a6ffd640 0 client.7485 ms_handle_reset on v2:172.21.15.131:6834/3013984008 2024-09-17T02:48:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 -- 172.21.15.131:0/2603185240 >> 172.21.15.131:0/2603185240 conn(0x560f17faa760 msgr2=0x560f18500640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 -- 172.21.15.131:0/2603185240 shutdown_connections 2024-09-17T02:48:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 1 -- 172.21.15.131:0/2603185240 wait complete. 2024-09-17T02:48:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84d5d15000 20 client.7485 trim_cache size 0 max 16384 2024-09-17T02:48:59.558 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:48:59.558 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:48:59.558 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:48:59.558 INFO:teuthology.orchestra.run.smithi131.stdout: "damage_type": "backtrace", 2024-09-17T02:48:59.559 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 2164657649, 2024-09-17T02:48:59.559 INFO:teuthology.orchestra.run.smithi131.stdout: "ino": 1099511627777, 2024-09-17T02:48:59.559 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/testdir/hardlink" 2024-09-17T02:48:59.559 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:48:59.559 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:48:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84cea4a640 1 -- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84b4073200 msgr2=0x7f84b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84cea4a640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84b4073200 0x7f84b4075710 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0016c0 tx=0x7f84b0005840 comp rx=0 tx=0).stop 2024-09-17T02:48:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84cea4a640 1 -- 172.21.15.131:0/2922956891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84b0004810 msgr2=0x7f84c80aa020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84cea4a640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84b0004810 0x7f84c80aa020 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f84c0011790 tx=0x7f84c0011c50 comp rx=0 tx=0).stop 2024-09-17T02:48:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84cea4a640 1 -- 172.21.15.131:0/2922956891 shutdown_connections 2024-09-17T02:48:59.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84cea4a640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f84b4073200 0x7f84b4075710 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84cea4a640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f84c80b0270 0x7f84c80b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84cea4a640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f84c81f3ed0 0x7f84c80afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84cea4a640 1 --2- 172.21.15.131:0/2922956891 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f84b0004810 0x7f84c80aa020 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84cea4a640 1 -- 172.21.15.131:0/2922956891 >> 172.21.15.131:0/2922956891 conn(0x7f84c806eea0 msgr2=0x7f84c80cab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.488+0000 7f84cea4a640 1 -- 172.21.15.131:0/2922956891 shutdown_connections 2024-09-17T02:48:59.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.492+0000 7f84cea4a640 1 -- 172.21.15.131:0/2922956891 wait complete. 2024-09-17T02:48:59.561 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:48:59.561 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:48:59.566 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:48:59.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.760+0000 7fd0025d4640 1 -- 172.21.15.131:0/1676395416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcffc06e2c0 msgr2=0x7fcffc0c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.760+0000 7fd0025d4640 1 --2- 172.21.15.131:0/1676395416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcffc06e2c0 0x7fcffc0c4580 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009840 tx=0x7fcff401c790 comp rx=0 tx=0).stop 2024-09-17T02:48:59.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.760+0000 7fd0025d4640 1 -- 172.21.15.131:0/1676395416 shutdown_connections 2024-09-17T02:48:59.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.760+0000 7fd0025d4640 1 --2- 172.21.15.131:0/1676395416 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcffc0cec00 0x7fcffc0cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.760+0000 7fd0025d4640 1 --2- 172.21.15.131:0/1676395416 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcffc06e2c0 0x7fcffc0c4580 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.760+0000 7fd0025d4640 1 -- 172.21.15.131:0/1676395416 >> 172.21.15.131:0/1676395416 conn(0x7fcffc06eea0 msgr2=0x7fcffc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:59.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.760+0000 7fd0025d4640 1 -- 172.21.15.131:0/1676395416 shutdown_connections 2024-09-17T02:48:59.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.760+0000 7fd0025d4640 1 -- 172.21.15.131:0/1676395416 wait complete. 2024-09-17T02:48:59.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd0025d4640 1 Processor -- start 2024-09-17T02:48:59.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd0025d4640 1 -- start start 2024-09-17T02:48:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd0025d4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcffc06e2c0 0x7fcffc15fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd0025d4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcffc0c4ac0 0x7fcffc0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd0025d4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcffc0cec00 0x7fcffc14bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd0025d4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc169c60 con 0x7fcffc06e2c0 2024-09-17T02:48:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd0025d4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc169ae0 con 0x7fcffc0cec00 2024-09-17T02:48:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd0025d4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcffc169de0 con 0x7fcffc0c4ac0 2024-09-17T02:48:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcffbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcffc06e2c0 0x7fcffc15fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcffbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcffc06e2c0 0x7fcffc15fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41410/0 (socket says 172.21.15.131:41410) 2024-09-17T02:48:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcffbfff640 1 -- 172.21.15.131:0/2407969593 learned_addr learned my addr 172.21.15.131:0/2407969593 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:48:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd000b71640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcffc0cec00 0x7fcffc14bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcffb7fe640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcffc0c4ac0 0x7fcffc0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcffbfff640 1 -- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcffc0c4ac0 msgr2=0x7fcffc0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcffbfff640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcffc0c4ac0 0x7fcffc0b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcffbfff640 1 -- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcffc0cec00 msgr2=0x7fcffc14bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcffbfff640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcffc0cec00 0x7fcffc14bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcffbfff640 1 -- 172.21.15.131:0/2407969593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc15f9f0 con 0x7fcffc06e2c0 2024-09-17T02:48:59.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcffbfff640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcffc06e2c0 0x7fcffc15fc80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fcff401cc70 tx=0x7fcff4005cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:59.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcff8ff9640 1 -- 172.21.15.131:0/2407969593 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff40ac070 con 0x7fcffc06e2c0 2024-09-17T02:48:59.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcff8ff9640 1 -- 172.21.15.131:0/2407969593 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcff4005240 con 0x7fcffc06e2c0 2024-09-17T02:48:59.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd0025d4640 1 -- 172.21.15.131:0/2407969593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc14c2a0 con 0x7fcffc06e2c0 2024-09-17T02:48:59.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fcff8ff9640 1 -- 172.21.15.131:0/2407969593 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff4005650 con 0x7fcffc06e2c0 2024-09-17T02:48:59.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd0025d4640 1 -- 172.21.15.131:0/2407969593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcffc15b600 con 0x7fcffc06e2c0 2024-09-17T02:48:59.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.764+0000 7fd0025d4640 1 -- 172.21.15.131:0/2407969593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcffc0c4580 con 0x7fcffc06e2c0 2024-09-17T02:48:59.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.768+0000 7fcff8ff9640 1 -- 172.21.15.131:0/2407969593 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcff40af070 con 0x7fcffc06e2c0 2024-09-17T02:48:59.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.768+0000 7fcff8ff9640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfdc0733d0 0x7fcfdc0758e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:48:59.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.768+0000 7fcff8ff9640 1 -- 172.21.15.131:0/2407969593 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fcff412eb90 con 0x7fcffc06e2c0 2024-09-17T02:48:59.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.768+0000 7fcffb7fe640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfdc0733d0 0x7fcfdc0758e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:48:59.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.768+0000 7fcffb7fe640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfdc0733d0 0x7fcfdc0758e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0015d0 tx=0x7fcfec009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:48:59.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.772+0000 7fcff8ff9640 1 -- 172.21.15.131:0/2407969593 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcff4016410 con 0x7fcffc06e2c0 2024-09-17T02:48:59.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.944+0000 7fd0025d4640 1 -- 172.21.15.131:0/2407969593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcffc073310 con 0x7fcffc06e2c0 2024-09-17T02:48:59.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.944+0000 7fcff8ff9640 1 -- 172.21.15.131:0/2407969593 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v95) ==== 80+0+124 (secure 0 0 0) 0x7fcff40f99b0 con 0x7fcffc06e2c0 2024-09-17T02:48:59.943 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6835/2766162522 2024-09-18T02:47:48.093626+0000 2024-09-17T02:48:59.943 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6834/2766162522 2024-09-18T02:47:48.093626+0000 2024-09-17T02:48:59.943 INFO:teuthology.orchestra.run.smithi131.stderr:listed 2 entries 2024-09-17T02:48:59.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 -- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfdc0733d0 msgr2=0x7fcfdc0758e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfdc0733d0 0x7fcfdc0758e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0015d0 tx=0x7fcfec009040 comp rx=0 tx=0).stop 2024-09-17T02:48:59.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 -- 172.21.15.131:0/2407969593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcffc06e2c0 msgr2=0x7fcffc15fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:48:59.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcffc06e2c0 0x7fcffc15fc80 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fcff401cc70 tx=0x7fcff4005cc0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 -- 172.21.15.131:0/2407969593 shutdown_connections 2024-09-17T02:48:59.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfdc0733d0 0x7fcfdc0758e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcffc0cec00 0x7fcffc14bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcffc0c4ac0 0x7fcffc0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 --2- 172.21.15.131:0/2407969593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcffc06e2c0 0x7fcffc15fc80 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:48:59.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 -- 172.21.15.131:0/2407969593 >> 172.21.15.131:0/2407969593 conn(0x7fcffc06eea0 msgr2=0x7fcffc0ad980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:48:59.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 -- 172.21.15.131:0/2407969593 shutdown_connections 2024-09-17T02:48:59.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:48:59.948+0000 7fd0025d4640 1 -- 172.21.15.131:0/2407969593 wait complete. 2024-09-17T02:48:59.963 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:48:59.963 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:48:59.969 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:49:00.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.168+0000 7fea19d1a640 1 -- 172.21.15.131:0/1831053047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1415c5b0 msgr2=0x7fea140cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:00.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.168+0000 7fea19d1a640 1 --2- 172.21.15.131:0/1831053047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1415c5b0 0x7fea140cbaf0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009c80 tx=0x7fea0c01cda0 comp rx=0 tx=0).stop 2024-09-17T02:49:00.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.168+0000 7fea19d1a640 1 -- 172.21.15.131:0/1831053047 shutdown_connections 2024-09-17T02:49:00.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.168+0000 7fea19d1a640 1 --2- 172.21.15.131:0/1831053047 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea1415c5b0 0x7fea140cbaf0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:00.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.168+0000 7fea19d1a640 1 --2- 172.21.15.131:0/1831053047 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea140ced00 0x7fea1406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:00.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.168+0000 7fea19d1a640 1 -- 172.21.15.131:0/1831053047 >> 172.21.15.131:0/1831053047 conn(0x7fea1406eea0 msgr2=0x7fea1406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:00.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.168+0000 7fea19d1a640 1 -- 172.21.15.131:0/1831053047 shutdown_connections 2024-09-17T02:49:00.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.168+0000 7fea19d1a640 1 -- 172.21.15.131:0/1831053047 wait complete. 2024-09-17T02:49:00.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea19d1a640 1 Processor -- start 2024-09-17T02:49:00.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea19d1a640 1 -- start start 2024-09-17T02:49:00.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea19d1a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea1406e3e0 0x7fea1415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea19d1a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea140ced00 0x7fea141541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea19d1a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea1415c5b0 0x7fea141640b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea19d1a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fea14168720 con 0x7fea140ced00 2024-09-17T02:49:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea19d1a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fea141685a0 con 0x7fea1406e3e0 2024-09-17T02:49:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea19d1a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fea141688a0 con 0x7fea1415c5b0 2024-09-17T02:49:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea12ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea140ced00 0x7fea141541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea12ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea140ced00 0x7fea141541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41426/0 (socket says 172.21.15.131:41426) 2024-09-17T02:49:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea12ffd640 1 -- 172.21.15.131:0/575131247 learned_addr learned my addr 172.21.15.131:0/575131247 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea13fff640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea1415c5b0 0x7fea141640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea137fe640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea1406e3e0 0x7fea1415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:00.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea12ffd640 1 -- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea1415c5b0 msgr2=0x7fea141640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea12ffd640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea1415c5b0 0x7fea141640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea12ffd640 1 -- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea1406e3e0 msgr2=0x7fea1415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea12ffd640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea1406e3e0 0x7fea1415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.172+0000 7fea12ffd640 1 -- 172.21.15.131:0/575131247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea14162c50 con 0x7fea140ced00 2024-09-17T02:49:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea12ffd640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea140ced00 0x7fea141541a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00c3c0 tx=0x7fe9fc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea0bfff640 1 -- 172.21.15.131:0/575131247 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc017070 con 0x7fea140ced00 2024-09-17T02:49:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea0bfff640 1 -- 172.21.15.131:0/575131247 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9fc0053c0 con 0x7fea140ced00 2024-09-17T02:49:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea19d1a640 1 -- 172.21.15.131:0/575131247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1415b700 con 0x7fea140ced00 2024-09-17T02:49:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea0bfff640 1 -- 172.21.15.131:0/575131247 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc003860 con 0x7fea140ced00 2024-09-17T02:49:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea19d1a640 1 -- 172.21.15.131:0/575131247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea14162970 con 0x7fea140ced00 2024-09-17T02:49:00.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea19d1a640 1 -- 172.21.15.131:0/575131247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea1406dea0 con 0x7fea140ced00 2024-09-17T02:49:00.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea0bfff640 1 -- 172.21.15.131:0/575131247 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe9fc01a070 con 0x7fea140ced00 2024-09-17T02:49:00.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea0bfff640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe9f4073200 0x7fe9f4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:00.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea0bfff640 1 -- 172.21.15.131:0/575131247 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fe9fc093730 con 0x7fea140ced00 2024-09-17T02:49:00.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea137fe640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe9f4073200 0x7fe9f4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:00.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.176+0000 7fea137fe640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe9f4073200 0x7fe9f4075710 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fea0400ab40 tx=0x7fea0400a250 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:00.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.180+0000 7fea0bfff640 1 -- 172.21.15.131:0/575131247 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fea1415b700 con 0x7fea140ced00 2024-09-17T02:49:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.364+0000 7fea19d1a640 1 -- 172.21.15.131:0/575131247 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fea1406a610 con 0x7fea140ced00 2024-09-17T02:49:00.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.364+0000 7fea0bfff640 1 -- 172.21.15.131:0/575131247 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v95) ==== 80+0+124 (secure 0 0 0) 0x7fea14162c50 con 0x7fea140ced00 2024-09-17T02:49:00.362 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6835/2766162522 2024-09-18T02:47:48.093626+0000 2024-09-17T02:49:00.363 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6834/2766162522 2024-09-18T02:47:48.093626+0000 2024-09-17T02:49:00.363 INFO:teuthology.orchestra.run.smithi131.stderr:listed 2 entries 2024-09-17T02:49:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 -- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe9f4073200 msgr2=0x7fe9f4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe9f4073200 0x7fe9f4075710 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fea0400ab40 tx=0x7fea0400a250 comp rx=0 tx=0).stop 2024-09-17T02:49:00.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 -- 172.21.15.131:0/575131247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea140ced00 msgr2=0x7fea141541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea140ced00 0x7fea141541a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00c3c0 tx=0x7fe9fc00c880 comp rx=0 tx=0).stop 2024-09-17T02:49:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 -- 172.21.15.131:0/575131247 shutdown_connections 2024-09-17T02:49:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe9f4073200 0x7fe9f4075710 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fea1415c5b0 0x7fea141640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fea140ced00 0x7fea141541a0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 --2- 172.21.15.131:0/575131247 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fea1406e3e0 0x7fea1415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 -- 172.21.15.131:0/575131247 >> 172.21.15.131:0/575131247 conn(0x7fea1406eea0 msgr2=0x7fea140b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 -- 172.21.15.131:0/575131247 shutdown_connections 2024-09-17T02:49:00.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:00.368+0000 7fea19d1a640 1 -- 172.21.15.131:0/575131247 wait complete. 2024-09-17T02:49:00.385 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:49:00.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:00.385 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:49:00.396 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[46721]: fuse finished with error 0 and tester_r 0 2024-09-17T02:49:00.397 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:49:06.498 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:49:06.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:06.499 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:49:06.506 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:49:06.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.712+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/3682391640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2a415f7d0 msgr2=0x7fd2a4162410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:06.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.712+0000 7fd2a9dfe640 1 --2- 172.21.15.131:0/3682391640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2a415f7d0 0x7fd2a4162410 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009900 tx=0x7fd28c01c8e0 comp rx=0 tx=0).stop 2024-09-17T02:49:06.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.712+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/3682391640 shutdown_connections 2024-09-17T02:49:06.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.712+0000 7fd2a9dfe640 1 --2- 172.21.15.131:0/3682391640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2a415f7d0 0x7fd2a4162410 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:06.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.712+0000 7fd2a9dfe640 1 --2- 172.21.15.131:0/3682391640 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2a40cf710 0x7fd2a415f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:06.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.712+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/3682391640 >> 172.21.15.131:0/3682391640 conn(0x7fd2a40d1b10 msgr2=0x7fd2a40d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:06.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.712+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/3682391640 shutdown_connections 2024-09-17T02:49:06.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.712+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/3682391640 wait complete. 2024-09-17T02:49:06.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a9dfe640 1 Processor -- start 2024-09-17T02:49:06.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a9dfe640 1 -- start start 2024-09-17T02:49:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a9dfe640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2a40cf710 0x7fd2a40c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a9dfe640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2a415f7d0 0x7fd2a4155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a9dfe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2a4162950 0x7fd2a415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a9dfe640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a40d3500 con 0x7fd2a4162950 2024-09-17T02:49:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a9dfe640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a40d3380 con 0x7fd2a40cf710 2024-09-17T02:49:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a9dfe640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd2a40d3680 con 0x7fd2a415f7d0 2024-09-17T02:49:06.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2a4162950 0x7fd2a415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2a4162950 0x7fd2a415b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44694/0 (socket says 172.21.15.131:44694) 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a3fff640 1 -- 172.21.15.131:0/2794433620 learned_addr learned my addr 172.21.15.131:0/2794433620 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a2ffd640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2a415f7d0 0x7fd2a4155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a37fe640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2a40cf710 0x7fd2a40c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a3fff640 1 -- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2a415f7d0 msgr2=0x7fd2a4155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a3fff640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2a415f7d0 0x7fd2a4155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a3fff640 1 -- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2a40cf710 msgr2=0x7fd2a40c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a3fff640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2a40cf710 0x7fd2a40c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a3fff640 1 -- 172.21.15.131:0/2794433620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a415a530 con 0x7fd2a4162950 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a3fff640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2a4162950 0x7fd2a415b5d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c3c0 tx=0x7fd29400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd29bfff640 1 -- 172.21.15.131:0/2794433620 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294017070 con 0x7fd2a4162950 2024-09-17T02:49:06.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd29bfff640 1 -- 172.21.15.131:0/2794433620 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2940053c0 con 0x7fd2a4162950 2024-09-17T02:49:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/2794433620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a415a810 con 0x7fd2a4162950 2024-09-17T02:49:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd29bfff640 1 -- 172.21.15.131:0/2794433620 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294003860 con 0x7fd2a4162950 2024-09-17T02:49:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.716+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/2794433620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a415add0 con 0x7fd2a4162950 2024-09-17T02:49:06.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.720+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/2794433620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a40b99b0 con 0x7fd2a4162950 2024-09-17T02:49:06.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.720+0000 7fd29bfff640 1 -- 172.21.15.131:0/2794433620 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd29401a070 con 0x7fd2a4162950 2024-09-17T02:49:06.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.720+0000 7fd29bfff640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd284073230 0x7fd284075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:06.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.720+0000 7fd29bfff640 1 -- 172.21.15.131:0/2794433620 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7fd294093810 con 0x7fd2a4162950 2024-09-17T02:49:06.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.720+0000 7fd2a37fe640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd284073230 0x7fd284075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:06.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.720+0000 7fd2a37fe640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd284073230 0x7fd284075740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd29c005c50 tx=0x7fd29c005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:06.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.724+0000 7fd29bfff640 1 -- 172.21.15.131:0/2794433620 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd29405e590 con 0x7fd2a4162950 2024-09-17T02:49:06.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/2794433620 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2a40b6d00 con 0x7fd2a4162950 2024-09-17T02:49:06.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd29bfff640 1 -- 172.21.15.131:0/2794433620 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5238 (secure 0 0 0) 0x7fd29405de90 con 0x7fd2a4162950 2024-09-17T02:49:06.928 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:06.929 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":71,"btime":"2024-09-17T02:47:59:359117+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:47:59.122657+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7233},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7233":{"gid":7233,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":6,"addr":"172.21.15.131:6835/3013984008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3013984008},{"type":"v1","addr":"172.21.15.131:6835","nonce":3013984008}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7233,"qdb_cluster":[7233]},"id":6}]} 2024-09-17T02:49:06.929 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 71 2024-09-17T02:49:06.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd284073230 msgr2=0x7fd284075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:06.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd284073230 0x7fd284075740 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd29c005c50 tx=0x7fd29c005be0 comp rx=0 tx=0).stop 2024-09-17T02:49:06.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/2794433620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2a4162950 msgr2=0x7fd2a415b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:06.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2a4162950 0x7fd2a415b5d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd29400c3c0 tx=0x7fd29400c880 comp rx=0 tx=0).stop 2024-09-17T02:49:06.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/2794433620 shutdown_connections 2024-09-17T02:49:06.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd284073230 0x7fd284075740 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:06.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd2a4162950 0x7fd2a415b5d0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:06.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd2a415f7d0 0x7fd2a4155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:06.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 --2- 172.21.15.131:0/2794433620 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd2a40cf710 0x7fd2a40c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:06.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/2794433620 >> 172.21.15.131:0/2794433620 conn(0x7fd2a40d1b10 msgr2=0x7fd2a41587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:06.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/2794433620 shutdown_connections 2024-09-17T02:49:06.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:06.932+0000 7fd2a9dfe640 1 -- 172.21.15.131:0/2794433620 wait complete. 2024-09-17T02:49:06.952 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-17T02:49:07.128 INFO:teuthology.orchestra.run.smithi131.stdout:{"message":"","return_code":0} 2024-09-17T02:49:07.128 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T02:49:07.129 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:49:07.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 -- 172.21.15.131:0/1204529834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f401806e5b0 msgr2=0x7f401815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 --2- 172.21.15.131:0/1204529834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f401806e5b0 0x7f401815bea0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f400c009840 tx=0x7f400c01c820 comp rx=0 tx=0).stop 2024-09-17T02:49:07.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 -- 172.21.15.131:0/1204529834 shutdown_connections 2024-09-17T02:49:07.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 --2- 172.21.15.131:0/1204529834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f401815c3e0 0x7f40180cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 --2- 172.21.15.131:0/1204529834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f401806e5b0 0x7f401815bea0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 -- 172.21.15.131:0/1204529834 >> 172.21.15.131:0/1204529834 conn(0x7f401806eea0 msgr2=0x7f401806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 -- 172.21.15.131:0/1204529834 shutdown_connections 2024-09-17T02:49:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 -- 172.21.15.131:0/1204529834 wait complete. 2024-09-17T02:49:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 Processor -- start 2024-09-17T02:49:07.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 -- start start 2024-09-17T02:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40180ced00 0x7f4018152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f401815c3e0 0x7f4018156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4018156ff0 0x7f4018153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4018168840 con 0x7f40180ced00 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f40181686c0 con 0x7f4018156ff0 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f401db5b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f40181689c0 con 0x7f401815c3e0 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f40177fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40180ced00 0x7f4018152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f40177fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40180ced00 0x7f4018152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44706/0 (socket says 172.21.15.131:44706) 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f40177fe640 1 -- 172.21.15.131:0/3725079641 learned_addr learned my addr 172.21.15.131:0/3725079641 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.288+0000 7f4016ffd640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f401815c3e0 0x7f4018156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f4017fff640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4018156ff0 0x7f4018153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f40177fe640 1 -- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f401815c3e0 msgr2=0x7f4018156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f40177fe640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f401815c3e0 0x7f4018156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f40177fe640 1 -- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4018156ff0 msgr2=0x7f4018153750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f40177fe640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4018156ff0 0x7f4018153750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f40177fe640 1 -- 172.21.15.131:0/3725079641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4018160c20 con 0x7f40180ced00 2024-09-17T02:49:07.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f4017fff640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4018156ff0 0x7f4018153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f40177fe640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40180ced00 0x7f4018152a00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f400800c430 tx=0x7f400800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f3ffffff640 1 -- 172.21.15.131:0/3725079641 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008011790 con 0x7f40180ced00 2024-09-17T02:49:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f3ffffff640 1 -- 172.21.15.131:0/3725079641 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4008017070 con 0x7f40180ced00 2024-09-17T02:49:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f401db5b640 1 -- 172.21.15.131:0/3725079641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4018158b90 con 0x7f40180ced00 2024-09-17T02:49:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f3ffffff640 1 -- 172.21.15.131:0/3725079641 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4008011cc0 con 0x7f40180ced00 2024-09-17T02:49:07.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f401db5b640 1 -- 172.21.15.131:0/3725079641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40181578c0 con 0x7f40180ced00 2024-09-17T02:49:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f401db5b640 1 -- 172.21.15.131:0/3725079641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f401815bea0 con 0x7f40180ced00 2024-09-17T02:49:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f3ffffff640 1 -- 172.21.15.131:0/3725079641 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f400801c460 con 0x7f40180ced00 2024-09-17T02:49:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f3ffffff640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4004073200 0x7f4004075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f3ffffff640 1 -- 172.21.15.131:0/3725079641 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6369+0+0 (secure 0 0 0) 0x7f4008094250 con 0x7f40180ced00 2024-09-17T02:49:07.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.292+0000 7f4016ffd640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4004073200 0x7f4004075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.296+0000 7f4016ffd640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4004073200 0x7f4004075710 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f400c0af000 tx=0x7f400c01b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.296+0000 7f3ffffff640 1 -- 172.21.15.131:0/3725079641 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f400805f060 con 0x7f40180ced00 2024-09-17T02:49:07.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.488+0000 7f401db5b640 1 -- 172.21.15.131:0/3725079641 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f401815ee40 con 0x7f40180ced00 2024-09-17T02:49:07.487 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:49:07.492+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:49:07.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.492+0000 7f3ffffff640 1 -- 172.21.15.131:0/3725079641 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) ==== 161+0+0 (secure 0 0 0) 0x7f400805e960 con 0x7f40180ced00 2024-09-17T02:49:07.490 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:49:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 -- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4004073200 msgr2=0x7f4004075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4004073200 0x7f4004075710 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f400c0af000 tx=0x7f400c01b060 comp rx=0 tx=0).stop 2024-09-17T02:49:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 -- 172.21.15.131:0/3725079641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40180ced00 msgr2=0x7f4018152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40180ced00 0x7f4018152a00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f400800c430 tx=0x7f400800c8f0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 -- 172.21.15.131:0/3725079641 shutdown_connections 2024-09-17T02:49:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4004073200 0x7f4004075710 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4018156ff0 0x7f4018153750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f401815c3e0 0x7f4018156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 --2- 172.21.15.131:0/3725079641 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f40180ced00 0x7f4018152a00 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 -- 172.21.15.131:0/3725079641 >> 172.21.15.131:0/3725079641 conn(0x7f401806eea0 msgr2=0x7f401806dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 -- 172.21.15.131:0/3725079641 shutdown_connections 2024-09-17T02:49:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.496+0000 7f401db5b640 1 -- 172.21.15.131:0/3725079641 wait complete. 2024-09-17T02:49:07.499 INFO:tasks.ceph.mds.c.smithi131.stderr: -4843> 2024-09-17T02:47:55.109+0000 7f9a5b4bc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:49:07.500 INFO:tasks.ceph.mds.c.smithi131.stderr: -4581> 2024-09-17T02:47:56.569+0000 7f9a574b4640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-09-17T02:49:07.500 INFO:tasks.ceph.mds.c.smithi131.stderr: -4397> 2024-09-17T02:47:56.833+0000 7f9a574b4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-09-17T02:47:39.862410+0000 1=1+0) n(v0 rc2024-09-17T02:47:39.976065+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55af17c62100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-17T02:47:39.862410+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-17T02:47:39.976065+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-09-17T02:49:07.503 INFO:tasks.ceph.mds.c.smithi131.stderr: -2539> 2024-09-17T02:48:28.477+0000 7f9a574b4640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-09-17T02:47:39.862410+0000 1=1+0)->f(v1 m2024-09-17T02:47:39.862410+0000) n(v0 rc2024-09-17T02:47:39.976065+0000 2=1+1)->n(v0 rc2024-09-17T02:47:39.976065+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55af17c62100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-17T02:47:39.862410+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-17T02:47:39.976065+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-09-17T02:49:07.518 DEBUG:teuthology.orchestra.run.smithi131:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-09-17T02:49:07.564 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:49:07.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 -- start start 2024-09-17T02:49:07.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 Processor -- start 2024-09-17T02:49:07.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5691680 0x55bad63e2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5691b00 0x55bad63e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5691f80 0x55bad63e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bad63fc600 con 0x55bad5691f80 2024-09-17T02:49:07.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bad63fc480 con 0x55bad5691680 2024-09-17T02:49:07.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55bad63fc300 con 0x55bad5691b00 2024-09-17T02:49:07.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0ed96640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5691f80 0x55bad63e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0ed96640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5691f80 0x55bad63e2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44740/0 (socket says 172.21.15.131:44740) 2024-09-17T02:49:07.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2743855879 learned_addr learned my addr 172.21.15.131:0/2743855879 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:07.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0e595640 1 --2- 172.21.15.131:0/2743855879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5691680 0x55bad63e2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2743855879 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5691b00 0x55bad63e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2743855879 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5691b00 msgr2=0x55bad63e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0ed96640 1 --2- 172.21.15.131:0/2743855879 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5691b00 0x55bad63e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2743855879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5691680 msgr2=0x55bad63e2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0ed96640 1 --2- 172.21.15.131:0/2743855879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5691680 0x55bad63e2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2743855879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad16041e0 con 0x55bad5691f80 2024-09-17T02:49:07.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2743855879 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5691b00 0x55bad63e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0ed96640 1 --2- 172.21.15.131:0/2743855879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5691f80 0x55bad63e2b00 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x55bad6534150 tx=0x55bad652e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53b514a9dc155b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0bd90640 1 -- 172.21.15.131:0/2743855879 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bad5647520 con 0x55bad5691f80 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0bd90640 1 -- 172.21.15.131:0/2743855879 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55bad5664540 con 0x55bad5691f80 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0bd90640 1 -- 172.21.15.131:0/2743855879 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bad56471e0 con 0x55bad5691f80 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2743855879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad16054a0 con 0x55bad5691f80 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2743855879 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55bad1605a40 con 0x55bad5691f80 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 4 waiting for MDS map... 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0bd90640 1 -- 172.21.15.131:0/2743855879 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x55bad654c600 con 0x55bad5691f80 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0bd90640 1 -- 172.21.15.131:0/2743855879 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 72) ==== 2893+0+0 (secure 0 0 0) 0x55bad5667180 con 0x55bad5691f80 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 4 Got MDS map 72 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 4 datascan.main: Using metadata pool output 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 4 datascan.main: connecting to RADOS... 2024-09-17T02:49:07.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 Processor -- start 2024-09-17T02:49:07.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.580+0000 7fcd0fdd68c0 1 -- start start 2024-09-17T02:49:07.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5690d80 0x55bad63e3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5690900 0x55bad63e3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5692880 0x55bad63e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bad63fc180 con 0x55bad5692880 2024-09-17T02:49:07.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bad5671380 con 0x55bad5690900 2024-09-17T02:49:07.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55bad5671800 con 0x55bad5690d80 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0dd94640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5692880 0x55bad63e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0dd94640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5692880 0x55bad63e3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44750/0 (socket says 172.21.15.131:44750) 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2522114911 learned_addr learned my addr 172.21.15.131:0/2522114911 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0e595640 1 --2- 172.21.15.131:0/2522114911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5690900 0x55bad63e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2522114911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5690d80 0x55bad63e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2522114911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5690d80 msgr2=0x55bad63e3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2522114911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5690d80 0x55bad63e3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2522114911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5690900 msgr2=0x55bad63e3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2522114911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5690900 0x55bad63e3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2522114911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad1605c20 con 0x55bad5692880 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2522114911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5692880 0x55bad63e3b80 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x55bad6535620 tx=0x55bad6538cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b06661bd61b9e88 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2522114911 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bad5647520 con 0x55bad5692880 2024-09-17T02:49:07.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2522114911 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55bad5664e00 con 0x55bad5692880 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2522114911 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bad56471e0 con 0x55bad5692880 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2522114911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5692880 msgr2=0x55bad63e3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2522114911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5692880 0x55bad63e3b80 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x55bad6535620 tx=0x55bad6538cc0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2522114911 shutdown_connections 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2522114911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5692880 0x55bad63e3b80 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2522114911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5690900 0x55bad63e3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2522114911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5690d80 0x55bad63e3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2522114911 >> 172.21.15.131:0/2522114911 conn(0x55bad5691200 msgr2=0x55bad160a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2522114911 shutdown_connections 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2522114911 wait complete. 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 Processor -- start 2024-09-17T02:49:07.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.584+0000 7fcd0fdd68c0 1 -- start start 2024-09-17T02:49:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5690900 0x55bad63e3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5690d80 0x55bad63e3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5692880 0x55bad63e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55bad63fd380 con 0x55bad5690900 2024-09-17T02:49:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55bad63fc180 con 0x55bad5692880 2024-09-17T02:49:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55bad63fd500 con 0x55bad5690d80 2024-09-17T02:49:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0dd94640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5690900 0x55bad63e3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0dd94640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5690900 0x55bad63e3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44760/0 (socket says 172.21.15.131:44760) 2024-09-17T02:49:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 learned_addr learned my addr 172.21.15.131:0/2969968461 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:07.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5692880 0x55bad63e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5690d80 msgr2=0x55bad63e3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0e595640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5690d80 0x55bad63e3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5690d80 0x55bad63e3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5692880 msgr2=0x55bad63e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5692880 0x55bad63e3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad16041e0 con 0x55bad5690900 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5690900 0x55bad63e3600 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x55bad595c6c0 tx=0x55bad6538ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd08589640 1 -- 172.21.15.131:0/2969968461 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bad5646820 con 0x55bad5690900 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd08589640 1 -- 172.21.15.131:0/2969968461 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55bad5664c40 con 0x55bad5690900 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bad16054a0 con 0x55bad5690900 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd08589640 1 -- 172.21.15.131:0/2969968461 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55bad5647520 con 0x55bad5690900 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad1605a40 con 0x55bad5690900 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 4 datascan.init: resolving metadata pool 13 2024-09-17T02:49:07.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd08589640 1 -- 172.21.15.131:0/2969968461 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x55bad595ac00 con 0x55bad5690900 2024-09-17T02:49:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd08589640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55bad5693a80 0x55bad63e4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd08589640 1 -- 172.21.15.131:0/2969968461 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x55bad5952a00 con 0x55bad5690900 2024-09-17T02:49:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-17T02:49:07.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 4 datascan.main: opening data pool 'cephfs_data' 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55bad5692d00 0x55bad63e4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5950800 con 0x55bad5692d00 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0e595640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55bad5693a80 0x55bad63e4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.588+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55bad5692d00 0x55bad63e4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.592+0000 7fcd0e595640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55bad5693a80 0x55bad63e4100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55bad571ecc0 tx=0x55bad572e960 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.592+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55bad5692d00 0x55bad63e4680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.592+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== osd_op_reply(1 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.592+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951000 con 0x55bad5692d00 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.592+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 2 ==== osd_op_reply(2 [pgnls start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.592+0000 7fcd09d8c640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55bad5a6c000 0x55bad5a74000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.592+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951400 con 0x55bad5a6c000 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.592+0000 7fcd0ed96640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55bad5a6c000 0x55bad5a74000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.592+0000 7fcd0ed96640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55bad5a6c000 0x55bad5a74000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.592+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.6 v2:172.21.15.189:6802/952142225 1 ==== osd_op_reply(3 [pgnls start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6c000 2024-09-17T02:49:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.592+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951800 con 0x55bad5692d00 2024-09-17T02:49:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.596+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 3 ==== osd_op_reply(4 [pgnls start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.596+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78000 con 0x55bad5692d00 2024-09-17T02:49:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.596+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 4 ==== osd_op_reply(5 [pgnls start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.596+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951c00 con 0x55bad5a6c000 2024-09-17T02:49:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.596+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.6 v2:172.21.15.189:6802/952142225 2 ==== osd_op_reply(6 [pgnls start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6c000 2024-09-17T02:49:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.596+0000 7fcd09d8c640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55bad5a6c480 0x55bad5a74580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.596+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78400 con 0x55bad5a6c480 2024-09-17T02:49:07.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.596+0000 7fcd0e595640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55bad5a6c480 0x55bad5a74580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.596+0000 7fcd0e595640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55bad5a6c480 0x55bad5a74580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.600+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(7 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6c480 2024-09-17T02:49:07.599 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.600+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5950800 con 0x55bad5692d00 2024-09-17T02:49:07.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.604+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 5 ==== osd_op_reply(8 [pgnls start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.604+0000 7fcd09d8c640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55bad5a6c900 0x55bad5a74b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.604+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78000 con 0x55bad5a6c900 2024-09-17T02:49:07.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.604+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55bad5a6c900 0x55bad5a74b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.604+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55bad5a6c900 0x55bad5a74b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.604+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== osd_op_reply(9 [pgnls start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5a6c900 2024-09-17T02:49:07.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.604+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951000 con 0x55bad5a6c480 2024-09-17T02:49:07.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.604+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.5 v2:172.21.15.189:6826/3175183672 2 ==== osd_op_reply(10 [pgnls start_epoch 96 out=49b] v94'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6c480 2024-09-17T02:49:07.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.604+0000 7fcd09d8c640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55bad5a6cd80 0x55bad5a75080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.604+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78800 con 0x55bad5a6cd80 2024-09-17T02:49:07.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.608+0000 7fcd0ed96640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55bad5a6cd80 0x55bad5a75080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.608+0000 7fcd0ed96640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55bad5a6cd80 0x55bad5a75080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.608+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 1 ==== osd_op_reply(11 [pgnls start_epoch 96 out=49b] v94'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.608+0000 7fcd0958b640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55bad5a6db00 0x55bad5a76100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.608+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951800 con 0x55bad5a6db00 2024-09-17T02:49:07.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.608+0000 7fcd0e595640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55bad5a6db00 0x55bad5a76100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.608+0000 7fcd0e595640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55bad5a6db00 0x55bad5a76100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.616+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== osd_op_reply(12 [pgnls start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6db00 2024-09-17T02:49:07.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.616+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78c00 con 0x55bad5a6c480 2024-09-17T02:49:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.616+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.5 v2:172.21.15.189:6826/3175183672 3 ==== osd_op_reply(13 [pgnls start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6c480 2024-09-17T02:49:07.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.616+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951000 con 0x55bad5692d00 2024-09-17T02:49:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.616+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 6 ==== osd_op_reply(14 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.616+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78400 con 0x55bad5a6c000 2024-09-17T02:49:07.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.616+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.6 v2:172.21.15.189:6802/952142225 3 ==== osd_op_reply(15 [pgnls start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6c000 2024-09-17T02:49:07.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.616+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad654b400 con 0x55bad5a6db00 2024-09-17T02:49:07.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.7 v2:172.21.15.189:6810/3586470400 2 ==== osd_op_reply(16 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6db00 2024-09-17T02:49:07.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951800 con 0x55bad5a6c900 2024-09-17T02:49:07.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.2 v2:172.21.15.131:6800/163801004 2 ==== osd_op_reply(17 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5a6c900 2024-09-17T02:49:07.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad654b000 con 0x55bad5a6cd80 2024-09-17T02:49:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 2 ==== osd_op_reply(18 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951400 con 0x55bad5a6c000 2024-09-17T02:49:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.6 v2:172.21.15.189:6802/952142225 4 ==== osd_op_reply(19 [pgnls start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6c000 2024-09-17T02:49:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0958b640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55bad5a6df80 0x55bad5a76680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad15dfc00 con 0x55bad5a6df80 2024-09-17T02:49:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55bad5a6df80 0x55bad5a76680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0dd94640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55bad5a6df80 0x55bad5a76680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== osd_op_reply(20 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5a6df80 2024-09-17T02:49:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951c00 con 0x55bad5a6c900 2024-09-17T02:49:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.2 v2:172.21.15.131:6800/163801004 3 ==== osd_op_reply(21 [pgnls start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5a6c900 2024-09-17T02:49:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78000 con 0x55bad5a6c900 2024-09-17T02:49:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.2 v2:172.21.15.131:6800/163801004 4 ==== osd_op_reply(22 [pgnls start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5a6c900 2024-09-17T02:49:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.620+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78c00 con 0x55bad5692d00 2024-09-17T02:49:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 7 ==== osd_op_reply(23 [pgnls start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.620 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad15dfc00 con 0x55bad5692d00 2024-09-17T02:49:07.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 8 ==== osd_op_reply(24 [pgnls start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78800 con 0x55bad5a6c480 2024-09-17T02:49:07.621 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.5 v2:172.21.15.189:6826/3175183672 4 ==== osd_op_reply(25 [pgnls start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6c480 2024-09-17T02:49:07.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5950800 con 0x55bad5a6cd80 2024-09-17T02:49:07.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 3 ==== osd_op_reply(26 [pgnls start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79000 con 0x55bad5a6c000 2024-09-17T02:49:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.6 v2:172.21.15.189:6802/952142225 5 ==== osd_op_reply(27 [pgnls start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6c000 2024-09-17T02:49:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951000 con 0x55bad5a6db00 2024-09-17T02:49:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.7 v2:172.21.15.189:6810/3586470400 3 ==== osd_op_reply(28 [pgnls start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6db00 2024-09-17T02:49:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79400 con 0x55bad5a6cd80 2024-09-17T02:49:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 4 ==== osd_op_reply(29 [pgnls start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.624+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951800 con 0x55bad5a6db00 2024-09-17T02:49:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.7 v2:172.21.15.189:6810/3586470400 4 ==== osd_op_reply(30 [pgnls start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6db00 2024-09-17T02:49:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd09d8c640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55bad5a6d200 0x55bad5a75600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79800 con 0x55bad5a6d200 2024-09-17T02:49:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd0ed96640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55bad5a6d200 0x55bad5a75600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.624 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd0ed96640 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55bad5a6d200 0x55bad5a75600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(31 [pgnls start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6d200 2024-09-17T02:49:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78400 con 0x55bad5a6cd80 2024-09-17T02:49:07.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 5 ==== osd_op_reply(32 [pgnls start_epoch 96 out=49b] v94'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79c00 con 0x55bad5a6df80 2024-09-17T02:49:07.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.0 v2:172.21.15.131:6808/1154840320 2 ==== osd_op_reply(33 [pgnls start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5a6df80 2024-09-17T02:49:07.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79000 con 0x55bad5a6c900 2024-09-17T02:49:07.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.2 v2:172.21.15.131:6800/163801004 5 ==== osd_op_reply(34 [pgnls start_epoch 96 out=49b] v94'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5a6c900 2024-09-17T02:49:07.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78000 con 0x55bad5a6cd80 2024-09-17T02:49:07.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 6 ==== osd_op_reply(35 [pgnls start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.628+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79400 con 0x55bad5a6db00 2024-09-17T02:49:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.7 v2:172.21.15.189:6810/3586470400 5 ==== osd_op_reply(36 [pgnls start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6db00 2024-09-17T02:49:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78c00 con 0x55bad5a6df80 2024-09-17T02:49:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.0 v2:172.21.15.131:6808/1154840320 3 ==== osd_op_reply(37 [pgnls start_epoch 96 out=49b] v94'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5a6df80 2024-09-17T02:49:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951400 con 0x55bad5a6cd80 2024-09-17T02:49:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 7 ==== osd_op_reply(38 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79c00 con 0x55bad5a6d200 2024-09-17T02:49:07.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.3 v2:172.21.15.131:6816/4047536388 2 ==== osd_op_reply(39 [pgnls start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6d200 2024-09-17T02:49:07.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5950800 con 0x55bad5a6d200 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.3 v2:172.21.15.131:6816/4047536388 3 ==== osd_op_reply(40 [pgnls start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6d200 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79000 con 0x55bad5a6c480 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.5 v2:172.21.15.189:6826/3175183672 5 ==== osd_op_reply(41 [pgnls start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6c480 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad654b000 con 0x55bad5a6db00 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.7 v2:172.21.15.189:6810/3586470400 6 ==== osd_op_reply(42 [pgnls start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6db00 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad15dfc00 con 0x55bad5a6d200 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.3 v2:172.21.15.131:6816/4047536388 4 ==== osd_op_reply(43 [pgnls start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6d200 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79800 con 0x55bad5a6d200 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.3 v2:172.21.15.131:6816/4047536388 5 ==== osd_op_reply(44 [pgnls start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6d200 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951c00 con 0x55bad5a6d200 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.3 v2:172.21.15.131:6816/4047536388 6 ==== osd_op_reply(45 [pgnls start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6d200 2024-09-17T02:49:07.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.632+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951400 con 0x55bad5a6c900 2024-09-17T02:49:07.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.2 v2:172.21.15.131:6800/163801004 6 ==== osd_op_reply(46 [pgnls start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5a6c900 2024-09-17T02:49:07.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78400 con 0x55bad5692d00 2024-09-17T02:49:07.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 9 ==== osd_op_reply(47 [pgnls start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5950800 con 0x55bad5a6df80 2024-09-17T02:49:07.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.0 v2:172.21.15.131:6808/1154840320 4 ==== osd_op_reply(48 [pgnls start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5a6df80 2024-09-17T02:49:07.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78000 con 0x55bad5692d00 2024-09-17T02:49:07.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 10 ==== osd_op_reply(49 [pgnls start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951000 con 0x55bad5a6cd80 2024-09-17T02:49:07.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 8 ==== osd_op_reply(50 [pgnls start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79c00 con 0x55bad5a6c480 2024-09-17T02:49:07.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.5 v2:172.21.15.189:6826/3175183672 6 ==== osd_op_reply(51 [pgnls start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6c480 2024-09-17T02:49:07.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951800 con 0x55bad5a6c480 2024-09-17T02:49:07.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.5 v2:172.21.15.189:6826/3175183672 7 ==== osd_op_reply(52 [pgnls start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6c480 2024-09-17T02:49:07.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.636+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78800 con 0x55bad5a6c000 2024-09-17T02:49:07.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.6 v2:172.21.15.189:6802/952142225 6 ==== osd_op_reply(53 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6c000 2024-09-17T02:49:07.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad654b400 con 0x55bad5a6cd80 2024-09-17T02:49:07.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 9 ==== osd_op_reply(54 [pgnls start_epoch 96 out=49b] v94'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79400 con 0x55bad5a6c000 2024-09-17T02:49:07.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.6 v2:172.21.15.189:6802/952142225 7 ==== osd_op_reply(55 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6c000 2024-09-17T02:49:07.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad15dfc00 con 0x55bad5a6cd80 2024-09-17T02:49:07.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 10 ==== osd_op_reply(56 [pgnls start_epoch 96 out=49b] v94'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79000 con 0x55bad5a6cd80 2024-09-17T02:49:07.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 11 ==== osd_op_reply(57 [pgnls start_epoch 96 out=81b] v94'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x55bad5950c00 con 0x55bad5a6cd80 2024-09-17T02:49:07.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 12 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0fdd68c0 10 datascan.operator(): handling object 1099511627777.0 2024-09-17T02:49:07.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x55bad654b400 con 0x55bad5a6cd80 2024-09-17T02:49:07.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 13 ==== osd_op_reply(59 10000000001.00000000 [call] v96'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.640+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5951c00 con 0x55bad5a6cd80 2024-09-17T02:49:07.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.1 v2:172.21.15.131:6824/1616383728 14 ==== osd_op_reply(60 [pgnls start_epoch 96 out=49b] v94'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6cd80 2024-09-17T02:49:07.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79800 con 0x55bad5692d00 2024-09-17T02:49:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 11 ==== osd_op_reply(61 [pgnls start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a9c000 con 0x55bad5a6d200 2024-09-17T02:49:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.3 v2:172.21.15.131:6816/4047536388 7 ==== osd_op_reply(62 [pgnls start_epoch 96 out=49b] v94'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6d200 2024-09-17T02:49:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a78800 con 0x55bad5a6df80 2024-09-17T02:49:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.0 v2:172.21.15.131:6808/1154840320 5 ==== osd_op_reply(63 [pgnls start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5a6df80 2024-09-17T02:49:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a9c400 con 0x55bad5692d00 2024-09-17T02:49:07.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0dd94640 1 -- 172.21.15.131:0/2969968461 <== osd.4 v2:172.21.15.189:6818/1965940492 12 ==== osd_op_reply(64 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a000 con 0x55bad5692d00 2024-09-17T02:49:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd09d8c640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a79400 con 0x55bad5a6db00 2024-09-17T02:49:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 <== osd.7 v2:172.21.15.189:6810/3586470400 7 ==== osd_op_reply(65 [pgnls start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a780 con 0x55bad5a6db00 2024-09-17T02:49:07.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0958b640 1 -- 172.21.15.131:0/2969968461 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x55bad5a9c800 con 0x55bad5a6c000 2024-09-17T02:49:07.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0ed96640 1 -- 172.21.15.131:0/2969968461 <== osd.6 v2:172.21.15.189:6802/952142225 8 ==== osd_op_reply(66 [pgnls start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55bad571a500 con 0x55bad5a6c000 2024-09-17T02:49:07.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55bad5a6df80 msgr2=0x55bad5a76680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55bad5a6df80 0x55bad5a76680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55bad5a6cd80 msgr2=0x55bad5a75080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55bad5a6cd80 0x55bad5a75080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55bad5a6c900 msgr2=0x55bad5a74b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55bad5a6c900 0x55bad5a74b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55bad5a6d200 msgr2=0x55bad5a75600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55bad5a6d200 0x55bad5a75600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55bad5692d00 msgr2=0x55bad63e4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55bad5692d00 0x55bad63e4680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55bad5a6c480 msgr2=0x55bad5a74580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55bad5a6c480 0x55bad5a74580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55bad5a6c000 msgr2=0x55bad5a74000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55bad5a6c000 0x55bad5a74000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55bad5a6db00 msgr2=0x55bad5a76100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55bad5a6db00 0x55bad5a76100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 reap_dead start 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 reap_dead start 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55bad5693a80 msgr2=0x55bad63e4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55bad5693a80 0x55bad63e4100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55bad571ecc0 tx=0x55bad572e960 comp rx=0 tx=0).stop 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 reap_dead start 2024-09-17T02:49:07.646 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0e595640 1 -- 172.21.15.131:0/2969968461 reap_dead start 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5690900 msgr2=0x55bad63e3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5690900 0x55bad63e3600 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x55bad595c6c0 tx=0x55bad6538ea0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 shutdown_connections 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2969968461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5690900 0x55bad63e3600 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.644+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 >> 172.21.15.131:0/2969968461 conn(0x55bad5691200 msgr2=0x55bad160a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.648+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 shutdown_connections 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.648+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2969968461 wait complete. 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.648+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2743855879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5691f80 msgr2=0x55bad63e2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.648+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2743855879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5691f80 0x55bad63e2b00 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x55bad6534150 tx=0x55bad652e180 comp rx=0 tx=0).stop 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.648+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2743855879 shutdown_connections 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.648+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2743855879 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55bad5691f80 0x55bad63e2b00 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.648+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2743855879 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55bad5691b00 0x55bad63e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.648+0000 7fcd0fdd68c0 1 --2- 172.21.15.131:0/2743855879 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55bad5691680 0x55bad63e2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.648+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2743855879 >> 172.21.15.131:0/2743855879 conn(0x55bad5692400 msgr2=0x55bad160b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.648+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2743855879 shutdown_connections 2024-09-17T02:49:07.648 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.648+0000 7fcd0fdd68c0 1 -- 172.21.15.131:0/2743855879 wait complete. 2024-09-17T02:49:07.650 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.132587, result: 2024-09-17T02:49:07.650 DEBUG:teuthology.orchestra.run.smithi131:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-09-17T02:49:07.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 -- start start 2024-09-17T02:49:07.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 Processor -- start 2024-09-17T02:49:07.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d09680 0x562713a50580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d09b00 0x562713a50000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d09f80 0x562713a50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562713a68180 con 0x562712d09b00 2024-09-17T02:49:07.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562713a68300 con 0x562712d09680 2024-09-17T02:49:07.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562713a68480 con 0x562712d09f80 2024-09-17T02:49:07.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1e054640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d09b00 0x562713a50000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1e054640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d09b00 0x562713a50000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44782/0 (socket says 172.21.15.131:44782) 2024-09-17T02:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1e054640 1 -- 172.21.15.131:0/1608015145 learned_addr learned my addr 172.21.15.131:0/1608015145 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1e855640 1 --2- 172.21.15.131:0/1608015145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d09680 0x562713a50580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1f056640 1 --2- 172.21.15.131:0/1608015145 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d09f80 0x562713a50b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1e054640 1 -- 172.21.15.131:0/1608015145 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d09f80 msgr2=0x562713a50b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1e054640 1 --2- 172.21.15.131:0/1608015145 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d09f80 0x562713a50b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1e054640 1 -- 172.21.15.131:0/1608015145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d09680 msgr2=0x562713a50580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1e054640 1 --2- 172.21.15.131:0/1608015145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d09680 0x562713a50580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1e054640 1 -- 172.21.15.131:0/1608015145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56270ec7c1e0 con 0x562712d09b00 2024-09-17T02:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1e054640 1 --2- 172.21.15.131:0/1608015145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d09b00 0x562713a50000 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x562713ba2120 tx=0x562713ba41e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78a745b07939079 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1c050640 1 -- 172.21.15.131:0/1608015145 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562712cbf520 con 0x562712d09b00 2024-09-17T02:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1c050640 1 -- 172.21.15.131:0/1608015145 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562712cdc8c0 con 0x562712d09b00 2024-09-17T02:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1c050640 1 -- 172.21.15.131:0/1608015145 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562712cbf1e0 con 0x562712d09b00 2024-09-17T02:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 -- 172.21.15.131:0/1608015145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56270ec7d2c0 con 0x562712d09b00 2024-09-17T02:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 -- 172.21.15.131:0/1608015145 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56270ec7da40 con 0x562712d09b00 2024-09-17T02:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 4 waiting for MDS map... 2024-09-17T02:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1c050640 1 -- 172.21.15.131:0/1608015145 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x562713bb4600 con 0x562712d09b00 2024-09-17T02:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee1c050640 1 -- 172.21.15.131:0/1608015145 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 72) ==== 2893+0+0 (secure 0 0 0) 0x562712cdf180 con 0x562712d09b00 2024-09-17T02:49:07.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 4 Got MDS map 72 2024-09-17T02:49:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 4 datascan.main: Using metadata pool output 2024-09-17T02:49:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 4 datascan.main: connecting to RADOS... 2024-09-17T02:49:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 Processor -- start 2024-09-17T02:49:07.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.712+0000 7fee200968c0 1 -- start start 2024-09-17T02:49:07.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d08900 0x562713a51600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d0a880 0x562713a51b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562713a68600 con 0x562712d08d80 2024-09-17T02:49:07.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562713a68780 con 0x562712d08900 2024-09-17T02:49:07.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562713a68900 con 0x562712d0a880 2024-09-17T02:49:07.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1f056640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1f056640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44796/0 (socket says 172.21.15.131:44796) 2024-09-17T02:49:07.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1f056640 1 -- 172.21.15.131:0/2049110575 learned_addr learned my addr 172.21.15.131:0/2049110575 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:07.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1e054640 1 --2- 172.21.15.131:0/2049110575 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d0a880 0x562713a51b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1e855640 1 --2- 172.21.15.131:0/2049110575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d08900 0x562713a51600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1f056640 1 -- 172.21.15.131:0/2049110575 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d0a880 msgr2=0x562713a51b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1f056640 1 --2- 172.21.15.131:0/2049110575 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d0a880 0x562713a51b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1f056640 1 -- 172.21.15.131:0/2049110575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d08900 msgr2=0x562713a51600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1f056640 1 --2- 172.21.15.131:0/2049110575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d08900 0x562713a51600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1f056640 1 -- 172.21.15.131:0/2049110575 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56270ec7dc20 con 0x562712d08d80 2024-09-17T02:49:07.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1e855640 1 --2- 172.21.15.131:0/2049110575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d08900 0x562713a51600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:07.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1f056640 1 --2- 172.21.15.131:0/2049110575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x562713ba3590 tx=0x562713ba8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b4e4f5c3756d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1a04c640 1 -- 172.21.15.131:0/2049110575 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562712cbf520 con 0x562712d08d80 2024-09-17T02:49:07.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1a04c640 1 -- 172.21.15.131:0/2049110575 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562712cdcfc0 con 0x562712d08d80 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee1a04c640 1 -- 172.21.15.131:0/2049110575 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562712cbf1e0 con 0x562712d08d80 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 -- 172.21.15.131:0/2049110575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 msgr2=0x562713a51080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 --2- 172.21.15.131:0/2049110575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x562713ba3590 tx=0x562713ba8c00 comp rx=0 tx=0).stop 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 -- 172.21.15.131:0/2049110575 shutdown_connections 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 --2- 172.21.15.131:0/2049110575 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d0a880 0x562713a51b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 --2- 172.21.15.131:0/2049110575 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d08900 0x562713a51600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 --2- 172.21.15.131:0/2049110575 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 -- 172.21.15.131:0/2049110575 >> 172.21.15.131:0/2049110575 conn(0x562712d09200 msgr2=0x562712fb6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 -- 172.21.15.131:0/2049110575 shutdown_connections 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 -- 172.21.15.131:0/2049110575 wait complete. 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 Processor -- start 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.716+0000 7fee200968c0 1 -- start start 2024-09-17T02:49:07.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d08900 0x562713a51600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d0a880 0x562713a51b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562713a69c80 con 0x562712d08d80 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562713a69b00 con 0x562712d0a880 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562713a69980 con 0x562712d08900 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e855640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e855640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44800/0 (socket says 172.21.15.131:44800) 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 learned_addr learned my addr 172.21.15.131:0/1412293244 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e054640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d0a880 0x562713a51b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1f056640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d08900 0x562713a51600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d08900 msgr2=0x562713a51600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e855640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d08900 0x562713a51600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d0a880 msgr2=0x562713a51b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e855640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d0a880 0x562713a51b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56270ec7c1e0 con 0x562712d08d80 2024-09-17T02:49:07.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e054640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d0a880 0x562713a51b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:07.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e855640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x562713ad9740 tx=0x562712da7800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee18849640 1 -- 172.21.15.131:0/1412293244 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562712cbf380 con 0x562712d08d80 2024-09-17T02:49:07.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee18849640 1 -- 172.21.15.131:0/1412293244 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562712cdce00 con 0x562712d08d80 2024-09-17T02:49:07.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56270ec7d2c0 con 0x562712d08d80 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee18849640 1 -- 172.21.15.131:0/1412293244 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562712cbf520 con 0x562712d08d80 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56270ec7de00 con 0x562712d08d80 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 4 datascan.init: resolving metadata pool 13 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee18849640 1 -- 172.21.15.131:0/1412293244 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x562712fd6c00 con 0x562712d08d80 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee18849640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x562712d0ba80 0x562713a52100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee18849640 1 -- 172.21.15.131:0/1412293244 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x562712fd0c00 con 0x562712d08d80 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 4 datascan.main: opening data pool 'cephfs_data' 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x562712d0ad00 0x562713a52680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562712fce800 con 0x562712d0ad00 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1f056640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x562712d0ba80 0x562713a52100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.720+0000 7fee1e054640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x562712d0ad00 0x562713a52680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee1f056640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x562712d0ba80 0x562713a52100 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x562713ba3590 tx=0x5627130a2420 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee1e054640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x562712d0ad00 0x562713a52680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv24 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x562712d0b180 0x562713a52c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562712fcec00 con 0x562712d0b180 2024-09-17T02:49:07.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee1f056640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x562712d0b180 0x562713a52c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee1f056640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x562712d0b180 0x562713a52c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x562712d0b600 0x562713a53180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcf000 con 0x562712d0b600 2024-09-17T02:49:07.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee1e855640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x562712d0b600 0x562713a53180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.724+0000 7fee1e855640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x562712d0b600 0x562713a53180 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcfc00 con 0x562712d0b600 2024-09-17T02:49:07.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec000 con 0x562712d0ad00 2024-09-17T02:49:07.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec400 con 0x562712d0b600 2024-09-17T02:49:07.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ecc00 con 0x562712d0b600 2024-09-17T02:49:07.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec800 con 0x562712d0ad00 2024-09-17T02:49:07.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 3 ==== osd_op_reply(8 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1a04c640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5627130f4000 0x5627130fc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.728+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ed000 con 0x5627130f4000 2024-09-17T02:49:07.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.732+0000 7fee1e054640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5627130f4000 0x5627130fc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.732+0000 7fee1e054640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5627130f4000 0x5627130fc000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.732+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.732+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce800 con 0x562712d0b600 2024-09-17T02:49:07.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.732+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 5 ==== osd_op_reply(10 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.732+0000 7fee1a04c640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x5627130f4480 0x5627130fc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.732+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcf000 con 0x5627130f4480 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.732+0000 7fee1f056640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x5627130f4480 0x5627130fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.732+0000 7fee1f056640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x5627130f4480 0x5627130fc580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.732+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== osd_op_reply(11 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.732+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562713bb3400 con 0x5627130f4000 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 2 ==== osd_op_reply(12 [pgnls-filter start_epoch 96 out=49b] v94'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcfc00 con 0x562712d0b180 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 96 out=49b] v94'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1984b640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x5627130f4d80 0x5627130fd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562713bb3000 con 0x5627130f4d80 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1e855640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x5627130f4d80 0x5627130fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1e855640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x5627130f4d80 0x5627130fd080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== osd_op_reply(14 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce800 con 0x5627130f4000 2024-09-17T02:49:07.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce400 con 0x562712d0b600 2024-09-17T02:49:07.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 6 ==== osd_op_reply(16 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.736+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec400 con 0x562712d0ad00 2024-09-17T02:49:07.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcec00 con 0x5627130f4d80 2024-09-17T02:49:07.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ecc00 con 0x5627130f4480 2024-09-17T02:49:07.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcf000 con 0x562712d0b180 2024-09-17T02:49:07.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec000 con 0x562712d0ad00 2024-09-17T02:49:07.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1984b640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5627130f5200 0x5627130fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcfc00 con 0x5627130f5200 2024-09-17T02:49:07.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1e054640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5627130f5200 0x5627130fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.739 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1e054640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5627130f5200 0x5627130fd600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== osd_op_reply(22 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f5200 2024-09-17T02:49:07.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec800 con 0x5627130f4480 2024-09-17T02:49:07.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ed000 con 0x5627130f4480 2024-09-17T02:49:07.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.740 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.740+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec000 con 0x562712d0b600 2024-09-17T02:49:07.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec400 con 0x562712d0b600 2024-09-17T02:49:07.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ed400 con 0x5627130f4000 2024-09-17T02:49:07.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 4 ==== osd_op_reply(27 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.742 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce800 con 0x562712d0b180 2024-09-17T02:49:07.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ed800 con 0x562712d0ad00 2024-09-17T02:49:07.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 6 ==== osd_op_reply(29 [pgnls-filter start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcfc00 con 0x5627130f4d80 2024-09-17T02:49:07.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 3 ==== osd_op_reply(30 [pgnls-filter start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130edc00 con 0x562712d0b180 2024-09-17T02:49:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 5 ==== osd_op_reply(31 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.744+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562713bb3400 con 0x5627130f4d80 2024-09-17T02:49:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 4 ==== osd_op_reply(32 [pgnls-filter start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1a04c640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x5627130f4900 0x5627130fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec000 con 0x5627130f4900 2024-09-17T02:49:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1f056640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x5627130f4900 0x5627130fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1f056640 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x5627130f4900 0x5627130fcb00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce400 con 0x562712d0b180 2024-09-17T02:49:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 96 out=49b] v94'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec400 con 0x5627130f5200 2024-09-17T02:49:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.0 v2:172.21.15.131:6808/1154840320 2 ==== osd_op_reply(35 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f5200 2024-09-17T02:49:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcec00 con 0x5627130f4480 2024-09-17T02:49:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 5 ==== osd_op_reply(36 [pgnls-filter start_epoch 96 out=49b] v94'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ecc00 con 0x562712d0b180 2024-09-17T02:49:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcfc00 con 0x5627130f4d80 2024-09-17T02:49:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec800 con 0x5627130f5200 2024-09-17T02:49:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.0 v2:172.21.15.131:6808/1154840320 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 96 out=49b] v94'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f5200 2024-09-17T02:49:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcf000 con 0x562712d0b180 2024-09-17T02:49:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 8 ==== osd_op_reply(40 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ed000 con 0x5627130f4900 2024-09-17T02:49:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 2 ==== osd_op_reply(41 [pgnls-filter start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce800 con 0x5627130f4900 2024-09-17T02:49:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 3 ==== osd_op_reply(42 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.748+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562713bb3000 con 0x5627130f4000 2024-09-17T02:49:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce400 con 0x5627130f4d80 2024-09-17T02:49:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311e000 con 0x5627130f4900 2024-09-17T02:49:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 4 ==== osd_op_reply(45 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcec00 con 0x5627130f4900 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311e400 con 0x5627130f4900 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcf000 con 0x5627130f4480 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 6 ==== osd_op_reply(48 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311e800 con 0x562712d0b600 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 9 ==== osd_op_reply(49 [pgnls-filter start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130edc00 con 0x5627130f5200 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.0 v2:172.21.15.131:6808/1154840320 4 ==== osd_op_reply(50 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f5200 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.752+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311ec00 con 0x562712d0b600 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 10 ==== osd_op_reply(51 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec000 con 0x562712d0b180 2024-09-17T02:49:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 9 ==== osd_op_reply(52 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311f000 con 0x5627130f4000 2024-09-17T02:49:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ecc00 con 0x5627130f4000 2024-09-17T02:49:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311f400 con 0x562712d0ad00 2024-09-17T02:49:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 7 ==== osd_op_reply(55 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ed400 con 0x562712d0b180 2024-09-17T02:49:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 96 out=49b] v94'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311f800 con 0x562712d0ad00 2024-09-17T02:49:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ed800 con 0x562712d0b180 2024-09-17T02:49:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 96 out=49b] v94'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.756+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 96 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311f000 con 0x562712d0b180 2024-09-17T02:49:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 96 out=81b] v96'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcf400 con 0x562712d0b600 2024-09-17T02:49:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 11 ==== osd_op_reply(60 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311e000 con 0x562712d0b600 2024-09-17T02:49:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 12 ==== osd_op_reply(61 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec400 con 0x562712d0ad00 2024-09-17T02:49:07.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311e400 con 0x562712d0b600 2024-09-17T02:49:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 13 ==== osd_op_reply(63 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec800 con 0x562712d0b600 2024-09-17T02:49:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.760+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311fc00 con 0x562712d0ad00 2024-09-17T02:49:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.764+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 10 ==== osd_op_reply(65 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.764+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce800 con 0x5627130f4000 2024-09-17T02:49:07.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.764+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 8 ==== osd_op_reply(66 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.764+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130edc00 con 0x562712d0b600 2024-09-17T02:49:07.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.764+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 15 ==== osd_op_reply(67 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.764+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcec00 con 0x5627130f4480 2024-09-17T02:49:07.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.764+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.764+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ed000 con 0x5627130f4000 2024-09-17T02:49:07.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.764+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 9 ==== osd_op_reply(69 [pgnls-filter start_epoch 96 out=49b] v94'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.764+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcf000 con 0x562712d0b180 2024-09-17T02:49:07.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 13 ==== osd_op_reply(70 [pgnls-filter start_epoch 96 out=49b] v94'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec000 con 0x5627130f4d80 2024-09-17T02:49:07.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562713bb3000 con 0x5627130f4000 2024-09-17T02:49:07.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 10 ==== osd_op_reply(72 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce800 con 0x562712d0b600 2024-09-17T02:49:07.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 16 ==== osd_op_reply(73 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311f400 con 0x562712d0ad00 2024-09-17T02:49:07.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 11 ==== osd_op_reply(74 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcfc00 con 0x5627130f4d80 2024-09-17T02:49:07.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 8 ==== osd_op_reply(75 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.768+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311f800 con 0x5627130f4480 2024-09-17T02:49:07.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce400 con 0x562712d0b180 2024-09-17T02:49:07.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 14 ==== osd_op_reply(77 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311fc00 con 0x562712d0ad00 2024-09-17T02:49:07.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 12 ==== osd_op_reply(78 [pgnls-filter start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcf400 con 0x5627130f5200 2024-09-17T02:49:07.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.0 v2:172.21.15.131:6808/1154840320 5 ==== osd_op_reply(79 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f5200 2024-09-17T02:49:07.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311e800 con 0x5627130f4480 2024-09-17T02:49:07.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 9 ==== osd_op_reply(80 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562713bb3000 con 0x5627130f4480 2024-09-17T02:49:07.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311ec00 con 0x562712d0b600 2024-09-17T02:49:07.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 17 ==== osd_op_reply(82 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562713bb3400 con 0x562712d0b600 2024-09-17T02:49:07.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 18 ==== osd_op_reply(83 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.772+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311e000 con 0x5627130f4000 2024-09-17T02:49:07.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 11 ==== osd_op_reply(84 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ecc00 con 0x562712d0b180 2024-09-17T02:49:07.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 15 ==== osd_op_reply(85 [pgnls-filter start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce800 con 0x562712d0ad00 2024-09-17T02:49:07.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 13 ==== osd_op_reply(86 [pgnls-filter start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec400 con 0x5627130f4d80 2024-09-17T02:49:07.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcfc00 con 0x562712d0b180 2024-09-17T02:49:07.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 16 ==== osd_op_reply(88 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ed000 con 0x5627130f4d80 2024-09-17T02:49:07.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 10 ==== osd_op_reply(89 [pgnls-filter start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcec00 con 0x5627130f4900 2024-09-17T02:49:07.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 7 ==== osd_op_reply(90 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311e400 con 0x562712d0b180 2024-09-17T02:49:07.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 17 ==== osd_op_reply(91 [pgnls-filter start_epoch 96 out=49b] v94'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.776+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcf000 con 0x5627130f5200 2024-09-17T02:49:07.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.0 v2:172.21.15.131:6808/1154840320 6 ==== osd_op_reply(92 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f5200 2024-09-17T02:49:07.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311ec00 con 0x5627130f4480 2024-09-17T02:49:07.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 11 ==== osd_op_reply(93 [pgnls-filter start_epoch 96 out=49b] v94'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec800 con 0x562712d0b180 2024-09-17T02:49:07.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311f000 con 0x5627130f4d80 2024-09-17T02:49:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 11 ==== osd_op_reply(95 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130edc00 con 0x5627130f5200 2024-09-17T02:49:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.0 v2:172.21.15.131:6808/1154840320 7 ==== osd_op_reply(96 [pgnls-filter start_epoch 96 out=49b] v94'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f5200 2024-09-17T02:49:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311f800 con 0x562712d0b180 2024-09-17T02:49:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 19 ==== osd_op_reply(97 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec000 con 0x5627130f4900 2024-09-17T02:49:07.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 8 ==== osd_op_reply(98 [pgnls-filter start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311e800 con 0x5627130f4900 2024-09-17T02:49:07.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 9 ==== osd_op_reply(99 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.780+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ed400 con 0x5627130f4000 2024-09-17T02:49:07.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 12 ==== osd_op_reply(100 [pgnls-filter start_epoch 96 out=49b] v94'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce400 con 0x5627130f4d80 2024-09-17T02:49:07.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 12 ==== osd_op_reply(101 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ed800 con 0x5627130f4900 2024-09-17T02:49:07.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 10 ==== osd_op_reply(102 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ecc00 con 0x5627130f4900 2024-09-17T02:49:07.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 11 ==== osd_op_reply(103 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562713bb3400 con 0x5627130f4900 2024-09-17T02:49:07.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 12 ==== osd_op_reply(104 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec800 con 0x5627130f4480 2024-09-17T02:49:07.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.2 v2:172.21.15.131:6800/163801004 12 ==== osd_op_reply(105 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4480 2024-09-17T02:49:07.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271312c000 con 0x562712d0b600 2024-09-17T02:49:07.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 19 ==== osd_op_reply(106 [pgnls-filter start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec000 con 0x5627130f5200 2024-09-17T02:49:07.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.0 v2:172.21.15.131:6808/1154840320 8 ==== osd_op_reply(107 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f5200 2024-09-17T02:49:07.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.784+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271312c400 con 0x562712d0b600 2024-09-17T02:49:07.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 20 ==== osd_op_reply(108 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311e400 con 0x562712d0b180 2024-09-17T02:49:07.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 20 ==== osd_op_reply(109 [pgnls-filter start_epoch 96 out=49b] v94'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271312c800 con 0x5627130f4000 2024-09-17T02:49:07.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 13 ==== osd_op_reply(110 [pgnls-filter start_epoch 96 out=49b] v94'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311ec00 con 0x5627130f4000 2024-09-17T02:49:07.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.5 v2:172.21.15.189:6826/3175183672 14 ==== osd_op_reply(111 [pgnls-filter start_epoch 96 out=49b] v94'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f4000 2024-09-17T02:49:07.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271312cc00 con 0x562712d0ad00 2024-09-17T02:49:07.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 14 ==== osd_op_reply(112 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311f800 con 0x562712d0b180 2024-09-17T02:49:07.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 96 out=49b] v94'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.788+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271312d000 con 0x562712d0ad00 2024-09-17T02:49:07.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 15 ==== osd_op_reply(114 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcfc00 con 0x562712d0b180 2024-09-17T02:49:07.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 96 out=49b] v94'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271312d400 con 0x562712d0b180 2024-09-17T02:49:07.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 23 ==== osd_op_reply(116 [pgnls-filter start_epoch 96 out=81b] v96'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-09-17T02:49:07.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 10 datascan.operator(): handling object 10000000001.0 2024-09-17T02:49:07.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562712fcf400 con 0x562712d0b180 2024-09-17T02:49:07.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 24 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-09-17T02:49:07.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-09-17T02:49:07.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-09-17T02:49:07.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x56271311f400 con 0x5627130f4d80 2024-09-17T02:49:07.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 13 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-09-17T02:49:07.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x56271311fc00 con 0x562712d0b180 2024-09-17T02:49:07.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 25 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) ==== 154+0+520 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-09-17T02:49:07.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-09-17T02:49:07.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 20 datascan.get_frag_of: frag is * 2024-09-17T02:49:07.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.792+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x56271311e000 con 0x5627130f4d80 2024-09-17T02:49:07.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.796+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 14 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.796+0000 7fee200968c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-09-17T02:49:07.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.796+0000 7fee200968c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-09-17T02:49:07.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.796+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x56271312d400 con 0x5627130f4d80 2024-09-17T02:49:07.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.796+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 15 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.796+0000 7fee200968c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-09-17T02:49:07.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.796+0000 7fee200968c0 20 datascan.read_dentry: file_head not found in result 2024-09-17T02:49:07.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.796+0000 7fee200968c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-09-17T02:49:07.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.796+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x56271311e800 con 0x5627130f4d80 2024-09-17T02:49:07.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.800+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 16 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v96'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.800+0000 7fee200968c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-09-17T02:49:07.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.800+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271311e400 con 0x562712d0b180 2024-09-17T02:49:07.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.800+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.1 v2:172.21.15.131:6824/1616383728 26 ==== osd_op_reply(123 [pgnls-filter start_epoch 96 out=49b] v94'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x562712d0b180 2024-09-17T02:49:07.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.800+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271312d800 con 0x562712d0b600 2024-09-17T02:49:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.800+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 21 ==== osd_op_reply(124 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.800+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fcec00 con 0x5627130f4900 2024-09-17T02:49:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.800+0000 7fee1f056640 1 -- 172.21.15.131:0/1412293244 <== osd.3 v2:172.21.15.131:6816/4047536388 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 96 out=49b] v94'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92500 con 0x5627130f4900 2024-09-17T02:49:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.800+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x56271312dc00 con 0x5627130f5200 2024-09-17T02:49:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.800+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.0 v2:172.21.15.131:6808/1154840320 9 ==== osd_op_reply(126 [pgnls-filter start_epoch 96 out=49b] v94'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x5627130f5200 2024-09-17T02:49:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.800+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562713bb3000 con 0x562712d0b600 2024-09-17T02:49:07.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.4 v2:172.21.15.189:6818/1965940492 22 ==== osd_op_reply(127 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x562712d0b600 2024-09-17T02:49:07.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee1984b640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562712fce400 con 0x5627130f4d80 2024-09-17T02:49:07.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 <== osd.7 v2:172.21.15.189:6810/3586470400 17 ==== osd_op_reply(128 [pgnls-filter start_epoch 96 out=49b] v94'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92780 con 0x5627130f4d80 2024-09-17T02:49:07.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee1a04c640 1 -- 172.21.15.131:0/1412293244 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 96 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x5627130ec000 con 0x562712d0ad00 2024-09-17T02:49:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee1e054640 1 -- 172.21.15.131:0/1412293244 <== osd.6 v2:172.21.15.189:6802/952142225 16 ==== osd_op_reply(129 [pgnls-filter start_epoch 96 out=49b] v94'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562712d92000 con 0x562712d0ad00 2024-09-17T02:49:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5627130f5200 msgr2=0x5627130fd600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x5627130f5200 0x5627130fd600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x562712d0b180 msgr2=0x562713a52c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x562712d0b180 0x562713a52c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x5627130f4480 msgr2=0x5627130fc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x5627130f4480 0x5627130fc580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x5627130f4900 msgr2=0x5627130fcb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x5627130f4900 0x5627130fcb00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x562712d0b600 msgr2=0x562713a53180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x562712d0b600 0x562713a53180 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 reap_dead start 2024-09-17T02:49:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5627130f4000 msgr2=0x5627130fc000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5627130f4000 0x5627130fc000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 reap_dead start 2024-09-17T02:49:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x562712d0ad00 msgr2=0x562713a52680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x562712d0ad00 0x562713a52680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x5627130f4d80 msgr2=0x5627130fd080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x5627130f4d80 0x5627130fd080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x562712d0ba80 msgr2=0x562713a52100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x562712d0ba80 0x562713a52100 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x562713ba3590 tx=0x5627130a2420 comp rx=0 tx=0).stop 2024-09-17T02:49:07.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 msgr2=0x562713a51080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x562713ad9740 tx=0x562712da7800 comp rx=0 tx=0).stop 2024-09-17T02:49:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 shutdown_connections 2024-09-17T02:49:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x5627130f4000 0x5627130fc000 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x5627130f4d80 0x5627130fd080 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x562712d0ad00 0x562713a52680 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x562712d0ba80 0x562713a52100 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 --2- 172.21.15.131:0/1412293244 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d08d80 0x562713a51080 secure :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x562713ad9740 tx=0x562712da7800 comp rx=0 tx=0).stop 2024-09-17T02:49:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 >> 172.21.15.131:0/1412293244 conn(0x562712d09200 msgr2=0x562712fb6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:07.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee1e855640 1 -- 172.21.15.131:0/1412293244 reap_dead start 2024-09-17T02:49:07.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 shutdown_connections 2024-09-17T02:49:07.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.804+0000 7fee200968c0 1 -- 172.21.15.131:0/1412293244 wait complete. 2024-09-17T02:49:07.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.808+0000 7fee200968c0 1 -- 172.21.15.131:0/1608015145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d09b00 msgr2=0x562713a50000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.808+0000 7fee200968c0 1 --2- 172.21.15.131:0/1608015145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d09b00 0x562713a50000 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x562713ba2120 tx=0x562713ba41e0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.808+0000 7fee200968c0 1 -- 172.21.15.131:0/1608015145 shutdown_connections 2024-09-17T02:49:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.808+0000 7fee200968c0 1 --2- 172.21.15.131:0/1608015145 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562712d09f80 0x562713a50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.808+0000 7fee200968c0 1 --2- 172.21.15.131:0/1608015145 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562712d09b00 0x562713a50000 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.808+0000 7fee200968c0 1 --2- 172.21.15.131:0/1608015145 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562712d09680 0x562713a50580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.808+0000 7fee200968c0 1 -- 172.21.15.131:0/1608015145 >> 172.21.15.131:0/1608015145 conn(0x562712d0a400 msgr2=0x56270ec85800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.808+0000 7fee200968c0 1 -- 172.21.15.131:0/1608015145 shutdown_connections 2024-09-17T02:49:07.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.808+0000 7fee200968c0 1 -- 172.21.15.131:0/1608015145 wait complete. 2024-09-17T02:49:07.814 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.163459, result: 2024-09-17T02:49:07.814 DEBUG:teuthology.orchestra.run.smithi131:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-09-17T02:49:07.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.876+0000 7f1511ade8c0 1 -- start start 2024-09-17T02:49:07.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.876+0000 7f1511ade8c0 1 Processor -- start 2024-09-17T02:49:07.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c7680 0x562b6b1be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c7b00 0x562b6b1be000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c7f80 0x562b6b1beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562b6be26180 con 0x562b6b0c7b00 2024-09-17T02:49:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562b6be26300 con 0x562b6b0c7680 2024-09-17T02:49:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562b6be26480 con 0x562b6b0c7f80 2024-09-17T02:49:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150fa9c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c7b00 0x562b6b1be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150fa9c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c7b00 0x562b6b1be000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44816/0 (socket says 172.21.15.131:44816) 2024-09-17T02:49:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150fa9c640 1 -- 172.21.15.131:0/2558516270 learned_addr learned my addr 172.21.15.131:0/2558516270 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f151029d640 1 --2- 172.21.15.131:0/2558516270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c7680 0x562b6b1be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1510a9e640 1 --2- 172.21.15.131:0/2558516270 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c7f80 0x562b6b1beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150fa9c640 1 -- 172.21.15.131:0/2558516270 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c7f80 msgr2=0x562b6b1beb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150fa9c640 1 --2- 172.21.15.131:0/2558516270 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c7f80 0x562b6b1beb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150fa9c640 1 -- 172.21.15.131:0/2558516270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c7680 msgr2=0x562b6b1be580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150fa9c640 1 --2- 172.21.15.131:0/2558516270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c7680 0x562b6b1be580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150fa9c640 1 -- 172.21.15.131:0/2558516270 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b6703a1e0 con 0x562b6b0c7b00 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150fa9c640 1 --2- 172.21.15.131:0/2558516270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c7b00 0x562b6b1be000 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x562b6bf6c0f0 tx=0x562b6bf6a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5d091eda923be7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150da98640 1 -- 172.21.15.131:0/2558516270 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562b6b07d520 con 0x562b6b0c7b00 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150da98640 1 -- 172.21.15.131:0/2558516270 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562b6b09a540 con 0x562b6b0c7b00 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150da98640 1 -- 172.21.15.131:0/2558516270 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562b6b07d1e0 con 0x562b6b0c7b00 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2558516270 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b6703b4a0 con 0x562b6b0c7b00 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2558516270 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x562b6703ba40 con 0x562b6b0c7b00 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 4 waiting for MDS map... 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150da98640 1 -- 172.21.15.131:0/2558516270 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x562b6bf78a00 con 0x562b6b0c7b00 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f150da98640 1 -- 172.21.15.131:0/2558516270 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 72) ==== 2893+0+0 (secure 0 0 0) 0x562b6b09d180 con 0x562b6b0c7b00 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 4 Got MDS map 72 2024-09-17T02:49:07.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 4 datascan.main: Using metadata pool output 2024-09-17T02:49:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 4 datascan.main: connecting to RADOS... 2024-09-17T02:49:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 1 Processor -- start 2024-09-17T02:49:07.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.880+0000 7f1511ade8c0 1 -- start start 2024-09-17T02:49:07.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c6d80 0x562b6b1bf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c6900 0x562b6b1bf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c8880 0x562b6b1bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562b6be26600 con 0x562b6b0c8880 2024-09-17T02:49:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562b6be26780 con 0x562b6b0c6900 2024-09-17T02:49:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562b6be26900 con 0x562b6b0c6d80 2024-09-17T02:49:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150fa9c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c8880 0x562b6b1bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150fa9c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c8880 0x562b6b1bfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44822/0 (socket says 172.21.15.131:44822) 2024-09-17T02:49:07.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150fa9c640 1 -- 172.21.15.131:0/2101380188 learned_addr learned my addr 172.21.15.131:0/2101380188 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150fa9c640 1 -- 172.21.15.131:0/2101380188 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c6d80 msgr2=0x562b6b1bf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150fa9c640 1 --2- 172.21.15.131:0/2101380188 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c6d80 0x562b6b1bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150fa9c640 1 -- 172.21.15.131:0/2101380188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c6900 msgr2=0x562b6b1bf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150fa9c640 1 --2- 172.21.15.131:0/2101380188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c6900 0x562b6b1bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150fa9c640 1 -- 172.21.15.131:0/2101380188 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b6703bc20 con 0x562b6b0c8880 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150fa9c640 1 --2- 172.21.15.131:0/2101380188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c8880 0x562b6b1bfb80 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x562b6bf6d560 tx=0x562b6bf6a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2966adeb82e17e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150ba94640 1 -- 172.21.15.131:0/2101380188 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562b6b07d040 con 0x562b6b0c8880 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150ba94640 1 -- 172.21.15.131:0/2101380188 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562b6b09afc0 con 0x562b6b0c8880 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f150ba94640 1 -- 172.21.15.131:0/2101380188 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562b6b07d520 con 0x562b6b0c8880 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2101380188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c8880 msgr2=0x562b6b1bfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/2101380188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c8880 0x562b6b1bfb80 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x562b6bf6d560 tx=0x562b6bf6a4e0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2101380188 shutdown_connections 2024-09-17T02:49:07.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/2101380188 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c8880 0x562b6b1bfb80 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/2101380188 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c6900 0x562b6b1bf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/2101380188 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c6d80 0x562b6b1bf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2101380188 >> 172.21.15.131:0/2101380188 conn(0x562b6b0c7200 msgr2=0x562b6b374000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2101380188 shutdown_connections 2024-09-17T02:49:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2101380188 wait complete. 2024-09-17T02:49:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 Processor -- start 2024-09-17T02:49:07.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.884+0000 7f1511ade8c0 1 -- start start 2024-09-17T02:49:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c6900 0x562b6b1bf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c6d80 0x562b6b1bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c8880 0x562b6b1bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x562b6be27c80 con 0x562b6b0c6d80 2024-09-17T02:49:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x562b6be27b00 con 0x562b6b0c6900 2024-09-17T02:49:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x562b6be27980 con 0x562b6b0c8880 2024-09-17T02:49:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f151029d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c6d80 0x562b6b1bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f151029d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c6d80 0x562b6b1bf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44828/0 (socket says 172.21.15.131:44828) 2024-09-17T02:49:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 learned_addr learned my addr 172.21.15.131:0/1066845947 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:07.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c6900 0x562b6b1bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f150fa9c640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c8880 0x562b6b1bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c8880 msgr2=0x562b6b1bfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f151029d640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c8880 0x562b6b1bfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c6900 msgr2=0x562b6b1bf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f151029d640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c6900 0x562b6b1bf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b6703a1e0 con 0x562b6b0c6d80 2024-09-17T02:49:07.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c6900 0x562b6b1bf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f151029d640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c6d80 0x562b6b1bf080 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x562b6be99770 tx=0x562b6b165800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f150a291640 1 -- 172.21.15.131:0/1066845947 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562b6b07d380 con 0x562b6b0c6d80 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f150a291640 1 -- 172.21.15.131:0/1066845947 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562b6b09ae00 con 0x562b6b0c6d80 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562b6703b4a0 con 0x562b6b0c6d80 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f150a291640 1 -- 172.21.15.131:0/1066845947 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562b6b07d040 con 0x562b6b0c6d80 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b6703ba40 con 0x562b6b0c6d80 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 4 datascan.init: resolving metadata pool 13 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f150a291640 1 -- 172.21.15.131:0/1066845947 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x562b6b388c00 con 0x562b6b0c6d80 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f150a291640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x562b6b0c9a80 0x562b6b1c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f150a291640 1 -- 172.21.15.131:0/1066845947 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x562b6b390600 con 0x562b6b0c6d80 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-17T02:49:07.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 4 datascan.main: resolving metadata pool 13 2024-09-17T02:49:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x562b6b0c8d00 0x562b6b1c0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b0c8d00 2024-09-17T02:49:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x562b6b0c9a80 0x562b6b1c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f150fa9c640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x562b6b0c8d00 0x562b6b1c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.888+0000 7f150fa9c640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x562b6b0c8d00 0x562b6b1c0680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 1 ==== osd_op_reply(1 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f150fa9c640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x562b6b498000 0x562b6b49c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498000 2024-09-17T02:49:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x562b6b0c9a80 0x562b6b1c0100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x562b6b1554a0 tx=0x562b6b46e4e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x562b6b498000 0x562b6b49c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x562b6b498000 0x562b6b49c000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== osd_op_reply(2 [pgnls start_epoch 96 out=177b] v96'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x562b6b498480 0x562b6b49c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498480 2024-09-17T02:49:07.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f151029d640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x562b6b498480 0x562b6b49c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f151029d640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x562b6b498480 0x562b6b49c580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(3 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150780 con 0x562b6b498480 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b0c8d00 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 2 ==== osd_op_reply(4 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f150fa9c640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x562b6b498900 0x562b6b49cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.892+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498900 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f150fa9c640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x562b6b498900 0x562b6b49cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f150fa9c640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x562b6b498900 0x562b6b49cb00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(5 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f150fa9c640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x562b6b498d80 0x562b6b49d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498d80 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x562b6b498d80 0x562b6b49d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x562b6b498d80 0x562b6b49d080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 1 ==== osd_op_reply(6 [pgnls start_epoch 96 out=149b] v95'26 uv26 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498d80 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 2 ==== osd_op_reply(7 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x562b6b499200 0x562b6b49d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b499200 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f151029d640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x562b6b499200 0x562b6b49d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.896+0000 7f151029d640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x562b6b499200 0x562b6b49d600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== osd_op_reply(8 [pgnls start_epoch 96 out=73b] v95'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b0c8d00 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 3 ==== osd_op_reply(9 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498d80 2024-09-17T02:49:07.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 3 ==== osd_op_reply(10 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x562b6b499680 0x562b6b49db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b499680 2024-09-17T02:49:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f150fa9c640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x562b6b499680 0x562b6b49db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f150fa9c640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x562b6b499680 0x562b6b49db80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== osd_op_reply(11 [pgnls start_epoch 96 out=81b] v95'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x562b6b150000 con 0x562b6b499680 2024-09-17T02:49:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498480 2024-09-17T02:49:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.5 v2:172.21.15.189:6826/3175183672 2 ==== osd_op_reply(12 [pgnls start_epoch 96 out=49b] v95'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150780 con 0x562b6b498480 2024-09-17T02:49:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.900+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498900 2024-09-17T02:49:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 2 ==== osd_op_reply(13 [pgnls start_epoch 96 out=77b] v95'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498000 2024-09-17T02:49:07.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 2 ==== osd_op_reply(14 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b499200 2024-09-17T02:49:07.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 2 ==== osd_op_reply(15 [pgnls start_epoch 96 out=73b] v95'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b499200 2024-09-17T02:49:07.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 3 ==== osd_op_reply(16 [pgnls start_epoch 96 out=49b] v95'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b0c8d00 2024-09-17T02:49:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 4 ==== osd_op_reply(17 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498d80 2024-09-17T02:49:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 4 ==== osd_op_reply(18 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b499200 2024-09-17T02:49:07.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 4 ==== osd_op_reply(19 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498000 2024-09-17T02:49:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 3 ==== osd_op_reply(20 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x562b6b499b00 0x562b6b49e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b499b00 2024-09-17T02:49:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.904+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x562b6b499b00 0x562b6b49e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f1510a9e640 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x562b6b499b00 0x562b6b49e100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:07.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== osd_op_reply(21 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b499b00 2024-09-17T02:49:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498900 2024-09-17T02:49:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 3 ==== osd_op_reply(22 [pgnls start_epoch 96 out=97b] v95'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b499200 2024-09-17T02:49:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 5 ==== osd_op_reply(23 [pgnls start_epoch 96 out=74b] v91'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b0c8d00 2024-09-17T02:49:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 5 ==== osd_op_reply(24 [pgnls start_epoch 96 out=73b] v95'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498d80 2024-09-17T02:49:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 5 ==== osd_op_reply(25 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b0c8d00 2024-09-17T02:49:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 6 ==== osd_op_reply(26 [pgnls start_epoch 96 out=98b] v95'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498900 2024-09-17T02:49:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 4 ==== osd_op_reply(27 [pgnls start_epoch 96 out=76b] v95'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b499b00 2024-09-17T02:49:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.2 v2:172.21.15.131:6800/163801004 2 ==== osd_op_reply(28 [pgnls start_epoch 96 out=73b] v95'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150500 con 0x562b6b499b00 2024-09-17T02:49:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b498900 2024-09-17T02:49:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 5 ==== osd_op_reply(29 [pgnls start_epoch 96 out=73b] v95'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b0c8d00 2024-09-17T02:49:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 7 ==== osd_op_reply(30 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b499b00 2024-09-17T02:49:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.2 v2:172.21.15.131:6800/163801004 3 ==== osd_op_reply(31 [pgnls start_epoch 96 out=73b] v91'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150500 con 0x562b6b499b00 2024-09-17T02:49:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.908+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b0c8d00 2024-09-17T02:49:07.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 8 ==== osd_op_reply(32 [pgnls start_epoch 96 out=101b] v95'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-09-17T02:49:07.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b494c00 con 0x562b6b498000 2024-09-17T02:49:07.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 4 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-09-17T02:49:07.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b495000 con 0x562b6b498000 2024-09-17T02:49:07.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 5 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-09-17T02:49:07.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b495400 con 0x562b6b498000 2024-09-17T02:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 6 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 20 datascan.scan_links: injected ino detected 2024-09-17T02:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-09-17T02:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b494800 con 0x562b6b498000 2024-09-17T02:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 7 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-09-17T02:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b494c00 con 0x562b6b498000 2024-09-17T02:49:07.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 8 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-09-17T02:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-17T02:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-09-17T02:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-17T02:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-09-17T02:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-17T02:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-09-17T02:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.912+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b495000 con 0x562b6b498d80 2024-09-17T02:49:07.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 6 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b495400 con 0x562b6b499200 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b495800 con 0x562b6b499680 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.4 v2:172.21.15.189:6818/1965940492 2 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x562b6b150000 con 0x562b6b499680 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b495c00 con 0x562b6b499200 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c4000 con 0x562b6b498900 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 6 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c4400 con 0x562b6b0c8d00 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 9 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-17T02:49:07.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-09-17T02:49:07.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c4800 con 0x562b6b0c8d00 2024-09-17T02:49:07.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 10 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-09-17T02:49:07.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-17T02:49:07.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-09-17T02:49:07.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-09-17T02:49:07.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-09-17T02:49:07.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c4c00 con 0x562b6b498900 2024-09-17T02:49:07.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-09-17T02:49:07.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-17T02:49:07.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-09-17T02:49:07.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-17T02:49:07.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-09-17T02:49:07.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5000 con 0x562b6b0c8d00 2024-09-17T02:49:07.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 11 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.916+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b0c8d00 2024-09-17T02:49:07.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 12 ==== osd_op_reply(47 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498000 2024-09-17T02:49:07.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 9 ==== osd_op_reply(48 [pgnls start_epoch 96 out=177b] v96'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498480 2024-09-17T02:49:07.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.5 v2:172.21.15.189:6826/3175183672 3 ==== osd_op_reply(49 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150780 con 0x562b6b498480 2024-09-17T02:49:07.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b0c8d00 2024-09-17T02:49:07.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 13 ==== osd_op_reply(50 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498900 2024-09-17T02:49:07.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 8 ==== osd_op_reply(51 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498d80 2024-09-17T02:49:07.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 7 ==== osd_op_reply(52 [pgnls start_epoch 96 out=149b] v95'26 uv26 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.920+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498d80 2024-09-17T02:49:07.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 8 ==== osd_op_reply(53 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b499200 2024-09-17T02:49:07.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 8 ==== osd_op_reply(54 [pgnls start_epoch 96 out=73b] v95'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b0c8d00 2024-09-17T02:49:07.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 14 ==== osd_op_reply(55 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498d80 2024-09-17T02:49:07.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 9 ==== osd_op_reply(56 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b499680 2024-09-17T02:49:07.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.4 v2:172.21.15.189:6818/1965940492 3 ==== osd_op_reply(57 [pgnls start_epoch 96 out=81b] v95'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x562b6b150000 con 0x562b6b499680 2024-09-17T02:49:07.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498480 2024-09-17T02:49:07.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.5 v2:172.21.15.189:6826/3175183672 4 ==== osd_op_reply(58 [pgnls start_epoch 96 out=49b] v95'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150780 con 0x562b6b498480 2024-09-17T02:49:07.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498900 2024-09-17T02:49:07.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 9 ==== osd_op_reply(59 [pgnls start_epoch 96 out=77b] v95'8 uv8 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498000 2024-09-17T02:49:07.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 10 ==== osd_op_reply(60 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.924+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b499200 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 9 ==== osd_op_reply(61 [pgnls start_epoch 96 out=73b] v95'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b499200 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 10 ==== osd_op_reply(62 [pgnls start_epoch 96 out=49b] v95'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b0c8d00 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 15 ==== osd_op_reply(63 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498d80 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 10 ==== osd_op_reply(64 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b499200 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 11 ==== osd_op_reply(65 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498000 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 11 ==== osd_op_reply(66 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b499b00 2024-09-17T02:49:07.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.2 v2:172.21.15.131:6800/163801004 4 ==== osd_op_reply(67 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b499b00 2024-09-17T02:49:07.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498900 2024-09-17T02:49:07.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 10 ==== osd_op_reply(68 [pgnls start_epoch 96 out=97b] v95'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b499200 2024-09-17T02:49:07.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 12 ==== osd_op_reply(69 [pgnls start_epoch 96 out=74b] v91'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b0c8d00 2024-09-17T02:49:07.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 16 ==== osd_op_reply(70 [pgnls start_epoch 96 out=73b] v95'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.928+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498d80 2024-09-17T02:49:07.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 11 ==== osd_op_reply(71 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b0c8d00 2024-09-17T02:49:07.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 17 ==== osd_op_reply(72 [pgnls start_epoch 96 out=98b] v95'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498900 2024-09-17T02:49:07.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 11 ==== osd_op_reply(73 [pgnls start_epoch 96 out=76b] v95'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b499b00 2024-09-17T02:49:07.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.2 v2:172.21.15.131:6800/163801004 5 ==== osd_op_reply(74 [pgnls start_epoch 96 out=73b] v95'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150500 con 0x562b6b499b00 2024-09-17T02:49:07.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498900 2024-09-17T02:49:07.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 12 ==== osd_op_reply(75 [pgnls start_epoch 96 out=73b] v95'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b0c8d00 2024-09-17T02:49:07.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 18 ==== osd_op_reply(76 [pgnls start_epoch 96 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b499b00 2024-09-17T02:49:07.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.2 v2:172.21.15.131:6800/163801004 6 ==== osd_op_reply(77 [pgnls start_epoch 96 out=73b] v91'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x562b6b150500 con 0x562b6b499b00 2024-09-17T02:49:07.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 96 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b0c8d00 2024-09-17T02:49:07.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 19 ==== osd_op_reply(78 [pgnls start_epoch 96 out=101b] v95'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-09-17T02:49:07.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c4000 con 0x562b6b498000 2024-09-17T02:49:07.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 12 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-09-17T02:49:07.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5800 con 0x562b6b498000 2024-09-17T02:49:07.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.932+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 13 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-09-17T02:49:07.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5c00 con 0x562b6b498000 2024-09-17T02:49:07.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 14 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 20 datascan.scan_links: injected ino detected 2024-09-17T02:49:07.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-09-17T02:49:07.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-09-17T02:49:07.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-09-17T02:49:07.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b495800 con 0x562b6b498000 2024-09-17T02:49:07.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 15 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-09-17T02:49:07.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77000 con 0x562b6b498000 2024-09-17T02:49:07.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 16 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-09-17T02:49:07.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-17T02:49:07.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-09-17T02:49:07.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-17T02:49:07.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-09-17T02:49:07.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-17T02:49:07.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-09-17T02:49:07.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77400 con 0x562b6b498d80 2024-09-17T02:49:07.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.6 v2:172.21.15.189:6802/952142225 12 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498d80 2024-09-17T02:49:07.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-09-17T02:49:07.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6bf77c00 con 0x562b6b499200 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.936+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c4000 con 0x562b6b499680 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.4 v2:172.21.15.189:6818/1965940492 4 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x562b6b150000 con 0x562b6b499680 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5800 con 0x562b6b499200 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 14 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b494800 con 0x562b6b498900 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 13 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-09-17T02:49:07.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b494c00 con 0x562b6b0c8d00 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 20 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b495000 con 0x562b6b0c8d00 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 21 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-09-17T02:49:07.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c4400 con 0x562b6b498900 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.3 v2:172.21.15.131:6816/4047536388 14 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x562b6b150000 con 0x562b6b498900 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c4800 con 0x562b6b0c8d00 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 22 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) ==== 154+0+1041 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-09-17T02:49:07.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c4c00 con 0x562b6b498000 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 17 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.940+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5000 con 0x562b6b498000 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.944+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 18 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v96'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.944+0000 7f1511ade8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.944+0000 7f1511ade8c0 10 datascan.scan_links: processing 1 injected_inos 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.944+0000 7f1511ade8c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.944+0000 7f1511ade8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.944+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5400 con 0x562b6b498000 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.944+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 19 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.944+0000 7f1511ade8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-09-17T02:49:07.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.944+0000 7f1511ade8c0 20 datascan.scan_links: first is now 2 2024-09-17T02:49:07.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.944+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c4000 con 0x562b6b498000 2024-09-17T02:49:07.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.948+0000 7f1510a9e640 1 -- 172.21.15.131:0/1066845947 <== osd.7 v2:172.21.15.189:6810/3586470400 20 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v96'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x562b6b150500 con 0x562b6b498000 2024-09-17T02:49:07.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.948+0000 7f1511ade8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-09-17T02:49:07.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.948+0000 7f1511ade8c0 10 datascan.scan_links: updating inotable 2024-09-17T02:49:07.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.948+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5c00 con 0x562b6b0c8d00 2024-09-17T02:49:07.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.948+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 23 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.948+0000 7f1511ade8c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-09-17T02:49:07.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.948+0000 7f1511ade8c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-09-17T02:49:07.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.948+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c4c00 con 0x562b6b0c8d00 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f150fa9c640 1 -- 172.21.15.131:0/1066845947 <== osd.1 v2:172.21.15.131:6824/1616383728 24 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v96'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x562b6b150000 con 0x562b6b0c8d00 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 10 datascan.scan_links: updating snaptable 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e96) -- 0x562b6b4c5000 con 0x562b6b499200 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 <== osd.0 v2:172.21.15.131:6808/1154840320 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x562b6b150780 con 0x562b6b499200 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x562b6b499200 msgr2=0x562b6b49d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x562b6b499200 0x562b6b49d600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x562b6b0c8d00 msgr2=0x562b6b1c0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x562b6b0c8d00 0x562b6b1c0680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x562b6b499b00 msgr2=0x562b6b49e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x562b6b499b00 0x562b6b49e100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x562b6b498900 msgr2=0x562b6b49cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x562b6b498900 0x562b6b49cb00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x562b6b499680 msgr2=0x562b6b49db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 reap_dead start 2024-09-17T02:49:07.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x562b6b499680 0x562b6b49db80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x562b6b498480 msgr2=0x562b6b49c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x562b6b498480 0x562b6b49c580 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x562b6b498d80 msgr2=0x562b6b49d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x562b6b498d80 0x562b6b49d080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x562b6b498000 msgr2=0x562b6b49c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x562b6b498000 0x562b6b49c000 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x562b6b0c9a80 msgr2=0x562b6b1c0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x562b6b0c9a80 0x562b6b1c0100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x562b6b1554a0 tx=0x562b6b46e4e0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f151029d640 1 -- 172.21.15.131:0/1066845947 reap_dead start 2024-09-17T02:49:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c6d80 msgr2=0x562b6b1bf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c6d80 0x562b6b1bf080 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x562b6be99770 tx=0x562b6b165800 comp rx=0 tx=0).stop 2024-09-17T02:49:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 shutdown_connections 2024-09-17T02:49:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/1066845947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c6d80 0x562b6b1bf080 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 >> 172.21.15.131:0/1066845947 conn(0x562b6b0c7200 msgr2=0x562b6b374000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 shutdown_connections 2024-09-17T02:49:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/1066845947 wait complete. 2024-09-17T02:49:07.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2558516270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c7b00 msgr2=0x562b6b1be000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:07.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/2558516270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c7b00 0x562b6b1be000 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x562b6bf6c0f0 tx=0x562b6bf6a1e0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2558516270 shutdown_connections 2024-09-17T02:49:07.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/2558516270 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x562b6b0c7f80 0x562b6b1beb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/2558516270 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x562b6b0c7b00 0x562b6b1be000 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 --2- 172.21.15.131:0/2558516270 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x562b6b0c7680 0x562b6b1be580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:07.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2558516270 >> 172.21.15.131:0/2558516270 conn(0x562b6b0c8400 msgr2=0x562b67043800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:07.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2558516270 shutdown_connections 2024-09-17T02:49:07.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:07.952+0000 7f1511ade8c0 1 -- 172.21.15.131:0/2558516270 wait complete. 2024-09-17T02:49:07.963 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.148927, result: 2024-09-17T02:49:07.963 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T02:49:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.124+0000 7f93d0e24640 1 -- 172.21.15.131:0/373685261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93cc0cf710 msgr2=0x7f93cc15f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:08.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.124+0000 7f93d0e24640 1 --2- 172.21.15.131:0/373685261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93cc0cf710 0x7f93cc15f290 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f93c4009840 tx=0x7f93c401c820 comp rx=0 tx=0).stop 2024-09-17T02:49:08.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.124+0000 7f93d0e24640 1 -- 172.21.15.131:0/373685261 shutdown_connections 2024-09-17T02:49:08.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.124+0000 7f93d0e24640 1 --2- 172.21.15.131:0/373685261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93cc0cf710 0x7f93cc15f290 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.124+0000 7f93d0e24640 1 -- 172.21.15.131:0/373685261 >> 172.21.15.131:0/373685261 conn(0x7f93cc0d1b10 msgr2=0x7f93cc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:08.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.124+0000 7f93d0e24640 1 -- 172.21.15.131:0/373685261 shutdown_connections 2024-09-17T02:49:08.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.124+0000 7f93d0e24640 1 -- 172.21.15.131:0/373685261 wait complete. 2024-09-17T02:49:08.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93d0e24640 1 Processor -- start 2024-09-17T02:49:08.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93d0e24640 1 -- start start 2024-09-17T02:49:08.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93d0e24640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93cc0cf710 0x7f93cc0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:08.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93d0e24640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93cc0d3470 0x7f93cc148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:08.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93d0e24640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93cc15f7d0 0x7f93cc149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:08.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93d0e24640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc16cf40 con 0x7f93cc15f7d0 2024-09-17T02:49:08.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93d0e24640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc16cdc0 con 0x7f93cc0cf710 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93d0e24640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f93cc16d0c0 con 0x7f93cc0d3470 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93cad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93cc15f7d0 0x7f93cc149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93cad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93cc15f7d0 0x7f93cc149440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44852/0 (socket says 172.21.15.131:44852) 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93cad9d640 1 -- 172.21.15.131:0/793946789 learned_addr learned my addr 172.21.15.131:0/793946789 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93ca59c640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93cc0cf710 0x7f93cc0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93c9d9b640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93cc0d3470 0x7f93cc148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93cad9d640 1 -- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93cc0d3470 msgr2=0x7f93cc148f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93cad9d640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93cc0d3470 0x7f93cc148f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93cad9d640 1 -- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93cc0cf710 msgr2=0x7f93cc0ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93cad9d640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93cc0cf710 0x7f93cc0ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93cad9d640 1 -- 172.21.15.131:0/793946789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93cc148720 con 0x7f93cc15f7d0 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93cad9d640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93cc15f7d0 0x7f93cc149440 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00c3c0 tx=0x7f93bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:08.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93baffd640 1 -- 172.21.15.131:0/793946789 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc011790 con 0x7f93cc15f7d0 2024-09-17T02:49:08.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93baffd640 1 -- 172.21.15.131:0/793946789 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93bc017070 con 0x7f93cc15f7d0 2024-09-17T02:49:08.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93d0e24640 1 -- 172.21.15.131:0/793946789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93cc157070 con 0x7f93cc15f7d0 2024-09-17T02:49:08.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93baffd640 1 -- 172.21.15.131:0/793946789 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93bc011c60 con 0x7f93cc15f7d0 2024-09-17T02:49:08.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93d0e24640 1 -- 172.21.15.131:0/793946789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93cc149f80 con 0x7f93cc15f7d0 2024-09-17T02:49:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.128+0000 7f93d0e24640 1 -- 172.21.15.131:0/793946789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93cc156de0 con 0x7f93cc15f7d0 2024-09-17T02:49:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.132+0000 7f93baffd640 1 -- 172.21.15.131:0/793946789 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f93bc01c460 con 0x7f93cc15f7d0 2024-09-17T02:49:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.132+0000 7f93baffd640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93ac073200 0x7f93ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.132+0000 7f93baffd640 1 -- 172.21.15.131:0/793946789 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f93bc094ba0 con 0x7f93cc15f7d0 2024-09-17T02:49:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.132+0000 7f93ca59c640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93ac073200 0x7f93ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:08.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.132+0000 7f93ca59c640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93ac073200 0x7f93ac075710 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f93c40af000 tx=0x7f93c401b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:08.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.136+0000 7f93baffd640 1 -- 172.21.15.131:0/793946789 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f93bc05fa40 con 0x7f93cc15f7d0 2024-09-17T02:49:08.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.324+0000 7f93d0e24640 1 -- 172.21.15.131:0/793946789 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f93cc14a1e0 con 0x7f93cc15f7d0 2024-09-17T02:49:08.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.496+0000 7f93baffd640 1 -- 172.21.15.131:0/793946789 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v73) ==== 162+0+0 (secure 0 0 0) 0x7f93bc05f340 con 0x7f93cc15f7d0 2024-09-17T02:49:08.493 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T02:49:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 -- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93ac073200 msgr2=0x7f93ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93ac073200 0x7f93ac075710 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f93c40af000 tx=0x7f93c401b060 comp rx=0 tx=0).stop 2024-09-17T02:49:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 -- 172.21.15.131:0/793946789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93cc15f7d0 msgr2=0x7f93cc149440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93cc15f7d0 0x7f93cc149440 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00c3c0 tx=0x7f93bc00c880 comp rx=0 tx=0).stop 2024-09-17T02:49:08.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 -- 172.21.15.131:0/793946789 shutdown_connections 2024-09-17T02:49:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f93ac073200 0x7f93ac075710 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f93cc15f7d0 0x7f93cc149440 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f93cc0d3470 0x7f93cc148f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 --2- 172.21.15.131:0/793946789 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f93cc0cf710 0x7f93cc0ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 -- 172.21.15.131:0/793946789 >> 172.21.15.131:0/793946789 conn(0x7f93cc0d1b10 msgr2=0x7f93cc155430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 -- 172.21.15.131:0/793946789 shutdown_connections 2024-09-17T02:49:08.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.500+0000 7f93d0e24640 1 -- 172.21.15.131:0/793946789 wait complete. 2024-09-17T02:49:08.525 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:49:08.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.676+0000 7ff776999640 1 -- 172.21.15.131:0/2301358153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff75c003a40 msgr2=0x7ff75c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:08.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.676+0000 7ff776999640 1 --2- 172.21.15.131:0/2301358153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff75c003a40 0x7ff75c003eb0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff7680016f0 tx=0x7ff76801c7b0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.676+0000 7ff774f36640 1 -- 172.21.15.131:0/2301358153 reap_dead start 2024-09-17T02:49:08.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.676+0000 7ff776999640 1 -- 172.21.15.131:0/2301358153 shutdown_connections 2024-09-17T02:49:08.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.676+0000 7ff776999640 1 -- 172.21.15.131:0/2301358153 >> 172.21.15.131:0/2301358153 conn(0x7ff77006eea0 msgr2=0x7ff77006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:08.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.676+0000 7ff776999640 1 -- 172.21.15.131:0/2301358153 shutdown_connections 2024-09-17T02:49:08.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.676+0000 7ff776999640 1 -- 172.21.15.131:0/2301358153 wait complete. 2024-09-17T02:49:08.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.676+0000 7ff776999640 1 Processor -- start 2024-09-17T02:49:08.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.676+0000 7ff776999640 1 -- start start 2024-09-17T02:49:08.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff776999640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff75c003a40 0x7ff77014b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:08.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff776999640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff7700c6320 0x7ff7701f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:08.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff776999640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7701540b0 0x7ff7701544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:08.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff776999640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff7701666e0 con 0x7ff75c003a40 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff776999640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff770166560 con 0x7ff7701540b0 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff776999640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff770166860 con 0x7ff7700c6320 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff75c003a40 0x7ff77014b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff75c003a40 0x7ff77014b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44874/0 (socket says 172.21.15.131:44874) 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76ffff640 1 -- 172.21.15.131:0/2171193873 learned_addr learned my addr 172.21.15.131:0/2171193873 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76ffff640 1 -- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff7700c6320 msgr2=0x7ff7701f5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76ffff640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff7700c6320 0x7ff7701f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff774f36640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7701540b0 0x7ff7701544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76ffff640 1 -- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7701540b0 msgr2=0x7ff7701544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76ffff640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7701540b0 0x7ff7701544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76ffff640 1 -- 172.21.15.131:0/2171193873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7701f6490 con 0x7ff75c003a40 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76ffff640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff75c003a40 0x7ff77014b040 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff768006040 tx=0x7ff768003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:08.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76cff9640 1 -- 172.21.15.131:0/2171193873 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7680ae070 con 0x7ff75c003a40 2024-09-17T02:49:08.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76cff9640 1 -- 172.21.15.131:0/2171193873 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff768003e50 con 0x7ff75c003a40 2024-09-17T02:49:08.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff776999640 1 -- 172.21.15.131:0/2171193873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff770154a00 con 0x7ff75c003a40 2024-09-17T02:49:08.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76cff9640 1 -- 172.21.15.131:0/2171193873 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768001310 con 0x7ff75c003a40 2024-09-17T02:49:08.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff776999640 1 -- 172.21.15.131:0/2171193873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77014dd30 con 0x7ff75c003a40 2024-09-17T02:49:08.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff776999640 1 -- 172.21.15.131:0/2171193873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff770073430 con 0x7ff75c003a40 2024-09-17T02:49:08.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76cff9640 1 -- 172.21.15.131:0/2171193873 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff768006350 con 0x7ff75c003a40 2024-09-17T02:49:08.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.680+0000 7ff76cff9640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 0x7ff750075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:08.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.684+0000 7ff76cff9640 1 -- 172.21.15.131:0/2171193873 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7ff76812e770 con 0x7ff75c003a40 2024-09-17T02:49:08.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.684+0000 7ff76f7fe640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 0x7ff750075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:08.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.684+0000 7ff76f7fe640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 0x7ff750075710 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff758002240 tx=0x7ff7580014a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:08.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.688+0000 7ff76cff9640 1 -- 172.21.15.131:0/2171193873 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff770154a00 con 0x7ff75c003a40 2024-09-17T02:49:08.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.876+0000 7ff776999640 1 -- 172.21.15.131:0/2171193873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff77006e9d0 con 0x7ff75c003a40 2024-09-17T02:49:08.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.876+0000 7ff76cff9640 1 -- 172.21.15.131:0/2171193873 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5231 (secure 0 0 0) 0x7ff768016510 con 0x7ff75c003a40 2024-09-17T02:49:08.875 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:08.876 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":74,"btime":"2024-09-17T02:49:08:505822+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:49:08.505793+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"c","rank":0,"incarnation":74,"state":"up:replay","state_seq":1,"addr":"172.21.15.131:6835/1957540940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1957540940},{"type":"v1","addr":"172.21.15.131:6835","nonce":1957540940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T02:49:08.876 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 74 2024-09-17T02:49:08.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 -- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 msgr2=0x7ff750075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:08.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 0x7ff750075710 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff758002240 tx=0x7ff7580014a0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 -- 172.21.15.131:0/2171193873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff75c003a40 msgr2=0x7ff77014b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:08.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff75c003a40 0x7ff77014b040 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff768006040 tx=0x7ff768003c60 comp rx=0 tx=0).stop 2024-09-17T02:49:08.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 -- 172.21.15.131:0/2171193873 shutdown_connections 2024-09-17T02:49:08.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff750073200 0x7ff750075710 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff7701540b0 0x7ff7701544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff7700c6320 0x7ff7701f5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 --2- 172.21.15.131:0/2171193873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff75c003a40 0x7ff77014b040 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:08.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 -- 172.21.15.131:0/2171193873 >> 172.21.15.131:0/2171193873 conn(0x7ff77006eea0 msgr2=0x7ff7700b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:08.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 -- 172.21.15.131:0/2171193873 shutdown_connections 2024-09-17T02:49:08.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:08.880+0000 7ff776999640 1 -- 172.21.15.131:0/2171193873 wait complete. 2024-09-17T02:49:09.300 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:47:25.797713+0000', 'modified': '2024-09-17T02:49:08.505793+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7521': {'gid': 7521, 'name': 'c', 'rank': 0, 'incarnation': 74, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/1957540940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1957540940}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1957540940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:49:09.300 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7521:up:replay 2024-09-17T02:49:10.301 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:49:10.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.472+0000 7f4930089640 1 -- 172.21.15.131:0/3784546987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4914004d50 msgr2=0x7f4914005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:10.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.472+0000 7f4930089640 1 --2- 172.21.15.131:0/3784546987 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4914004d50 0x7f4914005160 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f4920009eb0 tx=0x7f492001d5d0 comp rx=0 tx=0).stop 2024-09-17T02:49:10.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.472+0000 7f492e626640 1 -- 172.21.15.131:0/3784546987 reap_dead start 2024-09-17T02:49:10.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.472+0000 7f4930089640 1 -- 172.21.15.131:0/3784546987 shutdown_connections 2024-09-17T02:49:10.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.472+0000 7f4930089640 1 -- 172.21.15.131:0/3784546987 >> 172.21.15.131:0/3784546987 conn(0x7f49280d1ba0 msgr2=0x7f49280d4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:10.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.472+0000 7f4930089640 1 -- 172.21.15.131:0/3784546987 shutdown_connections 2024-09-17T02:49:10.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.472+0000 7f4930089640 1 -- 172.21.15.131:0/3784546987 wait complete. 2024-09-17T02:49:10.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f4930089640 1 Processor -- start 2024-09-17T02:49:10.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f4930089640 1 -- start start 2024-09-17T02:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f4930089640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914004d50 0x7f492814a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f4930089640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f492814a6d0 0x7f49281f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f4930089640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49281f92b0 0x7f492814bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f4930089640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f49281555e0 con 0x7f49281f92b0 2024-09-17T02:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f4930089640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4928155460 con 0x7f4914004d50 2024-09-17T02:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f4930089640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4928155760 con 0x7f492814a6d0 2024-09-17T02:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f492e626640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49281f92b0 0x7f492814bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f492e626640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49281f92b0 0x7f492814bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44894/0 (socket says 172.21.15.131:44894) 2024-09-17T02:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f492e626640 1 -- 172.21.15.131:0/1561933268 learned_addr learned my addr 172.21.15.131:0/1561933268 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f492de25640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914004d50 0x7f492814a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:10.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f492d624640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f492814a6d0 0x7f49281f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f492e626640 1 -- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f492814a6d0 msgr2=0x7f49281f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f492e626640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f492814a6d0 0x7f49281f8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f492e626640 1 -- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914004d50 msgr2=0x7f492814a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f492e626640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914004d50 0x7f492814a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f492e626640 1 -- 172.21.15.131:0/1561933268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49280b8ed0 con 0x7f49281f92b0 2024-09-17T02:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f492e626640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49281f92b0 0x7f492814bef0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f4920009eb0 tx=0x7f4920004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f491e7fc640 1 -- 172.21.15.131:0/1561933268 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4920009200 con 0x7f49281f92b0 2024-09-17T02:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f491e7fc640 1 -- 172.21.15.131:0/1561933268 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49200a9510 con 0x7f49281f92b0 2024-09-17T02:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f4930089640 1 -- 172.21.15.131:0/1561933268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49281f9a20 con 0x7f49281f92b0 2024-09-17T02:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f491e7fc640 1 -- 172.21.15.131:0/1561933268 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49200a9d10 con 0x7f49281f92b0 2024-09-17T02:49:10.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.476+0000 7f4930089640 1 -- 172.21.15.131:0/1561933268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49281f6c70 con 0x7f49281f92b0 2024-09-17T02:49:10.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.480+0000 7f4930089640 1 -- 172.21.15.131:0/1561933268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49281f9bf0 con 0x7f49281f92b0 2024-09-17T02:49:10.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.480+0000 7f491e7fc640 1 -- 172.21.15.131:0/1561933268 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f49200b9030 con 0x7f49281f92b0 2024-09-17T02:49:10.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.480+0000 7f491e7fc640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f490c073230 0x7f490c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:10.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.480+0000 7f491e7fc640 1 -- 172.21.15.131:0/1561933268 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f492012f1e0 con 0x7f49281f92b0 2024-09-17T02:49:10.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.480+0000 7f492de25640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f490c073230 0x7f490c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:10.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.480+0000 7f492de25640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f490c073230 0x7f490c075740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4924009b20 tx=0x7f49240091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:10.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.484+0000 7f491e7fc640 1 -- 172.21.15.131:0/1561933268 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f49200ad030 con 0x7f49281f92b0 2024-09-17T02:49:10.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.688+0000 7f4930089640 1 -- 172.21.15.131:0/1561933268 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4928161980 con 0x7f49281f92b0 2024-09-17T02:49:10.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f491e7fc640 1 -- 172.21.15.131:0/1561933268 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5231 (secure 0 0 0) 0x7f49200172e0 con 0x7f49281f92b0 2024-09-17T02:49:10.688 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:10.688 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":76,"btime":"2024-09-17T02:49:10:521134+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:49:09.526087+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"c","rank":0,"incarnation":74,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.131:6835/1957540940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1957540940},{"type":"v1","addr":"172.21.15.131:6835","nonce":1957540940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-17T02:49:10.688 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 76 2024-09-17T02:49:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 -- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f490c073230 msgr2=0x7f490c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f490c073230 0x7f490c075740 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4924009b20 tx=0x7f49240091e0 comp rx=0 tx=0).stop 2024-09-17T02:49:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 -- 172.21.15.131:0/1561933268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49281f92b0 msgr2=0x7f492814bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49281f92b0 0x7f492814bef0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f4920009eb0 tx=0x7f4920004f30 comp rx=0 tx=0).stop 2024-09-17T02:49:10.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 -- 172.21.15.131:0/1561933268 shutdown_connections 2024-09-17T02:49:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f490c073230 0x7f490c075740 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f49281f92b0 0x7f492814bef0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f492814a6d0 0x7f49281f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 --2- 172.21.15.131:0/1561933268 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4914004d50 0x7f492814a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 -- 172.21.15.131:0/1561933268 >> 172.21.15.131:0/1561933268 conn(0x7f49280d1ba0 msgr2=0x7f49280b9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 -- 172.21.15.131:0/1561933268 shutdown_connections 2024-09-17T02:49:10.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:10.692+0000 7f4930089640 1 -- 172.21.15.131:0/1561933268 wait complete. 2024-09-17T02:49:11.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:47:25.797713+0000', 'modified': '2024-09-17T02:49:09.526087+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7521': {'gid': 7521, 'name': 'c', 'rank': 0, 'incarnation': 74, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.131:6835/1957540940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1957540940}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1957540940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:49:11.112 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7521:up:rejoin 2024-09-17T02:49:12.113 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:49:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.280+0000 7f8b983ef640 1 -- 172.21.15.131:0/3928415246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b90155050 msgr2=0x7f8b90155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.280+0000 7f8b983ef640 1 --2- 172.21.15.131:0/3928415246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b90155050 0x7f8b90155460 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009c80 tx=0x7f8b8801cd70 comp rx=0 tx=0).stop 2024-09-17T02:49:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.280+0000 7f8b983ef640 1 -- 172.21.15.131:0/3928415246 shutdown_connections 2024-09-17T02:49:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.280+0000 7f8b983ef640 1 --2- 172.21.15.131:0/3928415246 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b90155050 0x7f8b90155460 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.280+0000 7f8b983ef640 1 --2- 172.21.15.131:0/3928415246 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8b90162330 0x7f8b90162740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.280+0000 7f8b983ef640 1 -- 172.21.15.131:0/3928415246 >> 172.21.15.131:0/3928415246 conn(0x7f8b900d1ba0 msgr2=0x7f8b900d4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.280+0000 7f8b983ef640 1 -- 172.21.15.131:0/3928415246 shutdown_connections 2024-09-17T02:49:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.280+0000 7f8b983ef640 1 -- 172.21.15.131:0/3928415246 wait complete. 2024-09-17T02:49:12.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b983ef640 1 Processor -- start 2024-09-17T02:49:12.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b983ef640 1 -- start start 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b983ef640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8b900cfb70 0x7f8b900ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b983ef640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b90155050 0x7f8b900c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b983ef640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b90162330 0x7f8b900cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b983ef640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90160dc0 con 0x7f8b90162330 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b983ef640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90160c40 con 0x7f8b90155050 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b983ef640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8b90160f40 con 0x7f8b900cfb70 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b9698c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b90162330 0x7f8b900cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b9698c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b90162330 0x7f8b900cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45468/0 (socket says 172.21.15.131:45468) 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b9698c640 1 -- 172.21.15.131:0/1341894117 learned_addr learned my addr 172.21.15.131:0/1341894117 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b9698c640 1 -- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8b900cfb70 msgr2=0x7f8b900ca920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b9698c640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8b900cfb70 0x7f8b900ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b9698c640 1 -- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b90155050 msgr2=0x7f8b900c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:12.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b9598a640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b90155050 0x7f8b900c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:12.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b9698c640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b90155050 0x7f8b900c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:12.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b9698c640 1 -- 172.21.15.131:0/1341894117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b900cae60 con 0x7f8b90162330 2024-09-17T02:49:12.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b9598a640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b90155050 0x7f8b900c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:12.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b9698c640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b90162330 0x7f8b900cb1a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009bd0 tx=0x7f8b880093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:12.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b86ffd640 1 -- 172.21.15.131:0/1341894117 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b880bc070 con 0x7f8b90162330 2024-09-17T02:49:12.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b86ffd640 1 -- 172.21.15.131:0/1341894117 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b880ad8e0 con 0x7f8b90162330 2024-09-17T02:49:12.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b983ef640 1 -- 172.21.15.131:0/1341894117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b900c8310 con 0x7f8b90162330 2024-09-17T02:49:12.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b86ffd640 1 -- 172.21.15.131:0/1341894117 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b880b2040 con 0x7f8b90162330 2024-09-17T02:49:12.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.284+0000 7f8b983ef640 1 -- 172.21.15.131:0/1341894117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b900c88d0 con 0x7f8b90162330 2024-09-17T02:49:12.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.288+0000 7f8b983ef640 1 -- 172.21.15.131:0/1341894117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b900c8080 con 0x7f8b90162330 2024-09-17T02:49:12.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.288+0000 7f8b86ffd640 1 -- 172.21.15.131:0/1341894117 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8b880bf070 con 0x7f8b90162330 2024-09-17T02:49:12.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.288+0000 7f8b86ffd640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b74073230 0x7f8b74075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:12.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.288+0000 7f8b86ffd640 1 -- 172.21.15.131:0/1341894117 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f8b8812e4f0 con 0x7f8b90162330 2024-09-17T02:49:12.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.288+0000 7f8b9618b640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b74073230 0x7f8b74075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:12.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.288+0000 7f8b9618b640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b74073230 0x7f8b74075740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8b80000f70 tx=0x7f8b800079c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:12.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.292+0000 7f8b86ffd640 1 -- 172.21.15.131:0/1341894117 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8b88016990 con 0x7f8b90162330 2024-09-17T02:49:12.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.488+0000 7f8b983ef640 1 -- 172.21.15.131:0/1341894117 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b90155460 con 0x7f8b90162330 2024-09-17T02:49:12.486 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.488+0000 7f8b86ffd640 1 -- 172.21.15.131:0/1341894117 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5238 (secure 0 0 0) 0x7f8b880f9180 con 0x7f8b90162330 2024-09-17T02:49:12.486 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:12.486 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":77,"btime":"2024-09-17T02:49:11:530527+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:49:11.530508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/1957540940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1957540940},{"type":"v1","addr":"172.21.15.131:6835","nonce":1957540940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7521,"qdb_cluster":[7521]},"id":6}]} 2024-09-17T02:49:12.486 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 77 2024-09-17T02:49:12.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 -- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b74073230 msgr2=0x7f8b74075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:12.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b74073230 0x7f8b74075740 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8b80000f70 tx=0x7f8b800079c0 comp rx=0 tx=0).stop 2024-09-17T02:49:12.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 -- 172.21.15.131:0/1341894117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b90162330 msgr2=0x7f8b900cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:12.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b90162330 0x7f8b900cb1a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f8b88009bd0 tx=0x7f8b880093d0 comp rx=0 tx=0).stop 2024-09-17T02:49:12.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 -- 172.21.15.131:0/1341894117 shutdown_connections 2024-09-17T02:49:12.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b74073230 0x7f8b74075740 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:12.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b90162330 0x7f8b900cb1a0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:12.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b90155050 0x7f8b900c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:12.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 --2- 172.21.15.131:0/1341894117 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8b900cfb70 0x7f8b900ca920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:12.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 -- 172.21.15.131:0/1341894117 >> 172.21.15.131:0/1341894117 conn(0x7f8b900d1ba0 msgr2=0x7f8b900b8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:12.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 -- 172.21.15.131:0/1341894117 shutdown_connections 2024-09-17T02:49:12.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:12.492+0000 7f8b983ef640 1 -- 172.21.15.131:0/1341894117 wait complete. 2024-09-17T02:49:12.911 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:47:25.797713+0000', 'modified': '2024-09-17T02:49:11.530508+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 96, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7521}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7521': {'gid': 7521, 'name': 'c', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.131:6835/1957540940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6834', 'nonce': 1957540940}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1957540940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7521, 'qdb_cluster': [7521]} 2024-09-17T02:49:12.911 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:49:12.912 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T02:49:13.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.080+0000 7f30b3c64640 1 -- 172.21.15.131:0/3578670933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30ac155240 msgr2=0x7f30ac073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.080+0000 7f30b3c64640 1 --2- 172.21.15.131:0/3578670933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30ac155240 0x7f30ac073430 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f309c009900 tx=0x7f309c01c8b0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.080+0000 7f30b3c64640 1 -- 172.21.15.131:0/3578670933 shutdown_connections 2024-09-17T02:49:13.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.080+0000 7f30b3c64640 1 --2- 172.21.15.131:0/3578670933 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30ac073970 0x7f30ac06dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.080+0000 7f30b3c64640 1 --2- 172.21.15.131:0/3578670933 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30ac155240 0x7f30ac073430 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.080+0000 7f30b3c64640 1 -- 172.21.15.131:0/3578670933 >> 172.21.15.131:0/3578670933 conn(0x7f30ac06eea0 msgr2=0x7f30ac06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:13.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.080+0000 7f30b3c64640 1 -- 172.21.15.131:0/3578670933 shutdown_connections 2024-09-17T02:49:13.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.080+0000 7f30b3c64640 1 -- 172.21.15.131:0/3578670933 wait complete. 2024-09-17T02:49:13.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b3c64640 1 Processor -- start 2024-09-17T02:49:13.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b3c64640 1 -- start start 2024-09-17T02:49:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b3c64640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30ac06e3e0 0x7f30ac157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b3c64640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac073970 0x7f30ac159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b3c64640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30ac155240 0x7f30ac151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b3c64640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac169010 con 0x7f30ac155240 2024-09-17T02:49:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b3c64640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac168e90 con 0x7f30ac073970 2024-09-17T02:49:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b3c64640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f30ac169190 con 0x7f30ac06e3e0 2024-09-17T02:49:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b2201640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30ac155240 0x7f30ac151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b2201640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30ac155240 0x7f30ac151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45490/0 (socket says 172.21.15.131:45490) 2024-09-17T02:49:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b2201640 1 -- 172.21.15.131:0/2871279870 learned_addr learned my addr 172.21.15.131:0/2871279870 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b1a00640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30ac06e3e0 0x7f30ac157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b11ff640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac073970 0x7f30ac159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b2201640 1 -- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30ac06e3e0 msgr2=0x7f30ac157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b2201640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30ac06e3e0 0x7f30ac157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b2201640 1 -- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac073970 msgr2=0x7f30ac159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b2201640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac073970 0x7f30ac159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b2201640 1 -- 172.21.15.131:0/2871279870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ac0ceb70 con 0x7f30ac155240 2024-09-17T02:49:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b2201640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30ac155240 0x7f30ac151fd0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f30a400c760 tx=0x7f30a400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30a27fc640 1 -- 172.21.15.131:0/2871279870 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4003b80 con 0x7f30ac155240 2024-09-17T02:49:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30a27fc640 1 -- 172.21.15.131:0/2871279870 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30a4003d20 con 0x7f30ac155240 2024-09-17T02:49:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b3c64640 1 -- 172.21.15.131:0/2871279870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ac0ce2d0 con 0x7f30ac155240 2024-09-17T02:49:13.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30a27fc640 1 -- 172.21.15.131:0/2871279870 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a400aab0 con 0x7f30ac155240 2024-09-17T02:49:13.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.084+0000 7f30b3c64640 1 -- 172.21.15.131:0/2871279870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ac0ce890 con 0x7f30ac155240 2024-09-17T02:49:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30a27fc640 1 -- 172.21.15.131:0/2871279870 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30a400ac50 con 0x7f30ac155240 2024-09-17T02:49:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30a27fc640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3090073230 0x7f3090075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30a27fc640 1 -- 172.21.15.131:0/2871279870 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f30a4094270 con 0x7f30ac155240 2024-09-17T02:49:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b1a00640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3090073230 0x7f3090075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30baf2f000 1 Processor -- start 2024-09-17T02:49:13.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30baf2f000 1 -- start start 2024-09-17T02:49:13.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b1a00640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3090073230 0x7f3090075740 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f309c0af000 tx=0x7f309c01b0f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:13.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1ba716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b920950 0x557a1ba71be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b909ca0 0x557a1ba72120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30baf2f000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557a1b946410 con 0x557a1b920950 2024-09-17T02:49:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30baf2f000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557a1b883b40 con 0x557a1b909ca0 2024-09-17T02:49:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30baf2f000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557a1b872f70 con 0x557a1b530460 2024-09-17T02:49:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b11ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b920950 0x557a1ba71be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b11ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b920950 0x557a1ba71be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45502/0 (socket says 172.21.15.131:45502) 2024-09-17T02:49:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b11ff640 1 -- 172.21.15.131:0/2455401571 learned_addr learned my addr 172.21.15.131:0/2455401571 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b1a00640 1 --2- 172.21.15.131:0/2455401571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1ba716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b2201640 1 --2- 172.21.15.131:0/2455401571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b909ca0 0x557a1ba72120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b11ff640 1 -- 172.21.15.131:0/2455401571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 msgr2=0x557a1ba716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b11ff640 1 --2- 172.21.15.131:0/2455401571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1ba716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b11ff640 1 -- 172.21.15.131:0/2455401571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b909ca0 msgr2=0x557a1ba72120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b11ff640 1 --2- 172.21.15.131:0/2455401571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b909ca0 0x557a1ba72120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b11ff640 1 -- 172.21.15.131:0/2455401571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1b83a5b0 con 0x557a1b920950 2024-09-17T02:49:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.088+0000 7f30b1a00640 1 --2- 172.21.15.131:0/2455401571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1ba716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30b11ff640 1 --2- 172.21.15.131:0/2455401571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b920950 0x557a1ba71be0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f30a800a820 tx=0x7f30a800ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc2fad800b9bcad server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f308f7fe640 1 -- 172.21.15.131:0/2455401571 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8015070 con 0x557a1b920950 2024-09-17T02:49:13.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f308f7fe640 1 -- 172.21.15.131:0/2455401571 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30a800cc30 con 0x557a1b920950 2024-09-17T02:49:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f308f7fe640 1 -- 172.21.15.131:0/2455401571 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8011650 con 0x557a1b920950 2024-09-17T02:49:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 -- 172.21.15.131:0/2455401571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b920950 msgr2=0x557a1ba71be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 --2- 172.21.15.131:0/2455401571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b920950 0x557a1ba71be0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f30a800a820 tx=0x7f30a800ace0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 -- 172.21.15.131:0/2455401571 shutdown_connections 2024-09-17T02:49:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 --2- 172.21.15.131:0/2455401571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b909ca0 0x557a1ba72120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 --2- 172.21.15.131:0/2455401571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b920950 0x557a1ba71be0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 --2- 172.21.15.131:0/2455401571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1ba716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 -- 172.21.15.131:0/2455401571 >> 172.21.15.131:0/2455401571 conn(0x557a1b9f6d90 msgr2=0x557a1ba26690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 -- 172.21.15.131:0/2455401571 shutdown_connections 2024-09-17T02:49:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 -- 172.21.15.131:0/2455401571 wait complete. 2024-09-17T02:49:13.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 -- start start 2024-09-17T02:49:13.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 Processor -- start 2024-09-17T02:49:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1ba78920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1ba78e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b920950 0x557a1ba793a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557a1b80d5f0 con 0x557a1b909ca0 2024-09-17T02:49:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557a1b876d80 con 0x557a1b920950 2024-09-17T02:49:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557a1b819520 con 0x557a1b530460 2024-09-17T02:49:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30baf2f000 10 client.0 _ll_register_callbacks cb 0x557a1b81d830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:49:13.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30b11ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1ba78e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30b11ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1ba78e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45508/0 (socket says 172.21.15.131:45508) 2024-09-17T02:49:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.092+0000 7f30b11ff640 1 -- 172.21.15.131:0/1802954093 learned_addr learned my addr 172.21.15.131:0/1802954093 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30b2201640 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b920950 0x557a1ba793a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30b1a00640 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1ba78920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30b11ff640 1 -- 172.21.15.131:0/1802954093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 msgr2=0x557a1ba78920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30b11ff640 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1ba78920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30b11ff640 1 -- 172.21.15.131:0/1802954093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b920950 msgr2=0x557a1ba793a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30b11ff640 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b920950 0x557a1ba793a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30b11ff640 1 -- 172.21.15.131:0/1802954093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1b811190 con 0x557a1b909ca0 2024-09-17T02:49:13.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30b11ff640 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1ba78e60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f30a8008010 tx=0x7f30a8012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10bddbd3fe8d413 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f308cff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:49:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f308cff9640 1 -- 172.21.15.131:0/1802954093 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8020070 con 0x557a1b909ca0 2024-09-17T02:49:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f308cff9640 1 -- 172.21.15.131:0/1802954093 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f30a8012c30 con 0x557a1b909ca0 2024-09-17T02:49:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 -- 172.21.15.131:0/1802954093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557a1b80b110 con 0x557a1b909ca0 2024-09-17T02:49:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f308cff9640 1 -- 172.21.15.131:0/1802954093 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f30a802e050 con 0x557a1b909ca0 2024-09-17T02:49:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 10 client.7605 fetch_fsmap learned FSMap version 77 2024-09-17T02:49:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 -- 172.21.15.131:0/1802954093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x557a1b8f7f10 con 0x557a1b909ca0 2024-09-17T02:49:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f308cff9640 1 -- 172.21.15.131:0/1802954093 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x557a1b80b110 con 0x557a1b909ca0 2024-09-17T02:49:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f308cff9640 1 -- 172.21.15.131:0/1802954093 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7f30a8031020 con 0x557a1b909ca0 2024-09-17T02:49:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 10 client.7605 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T02:49:13.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 10 client.7605 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:49:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 20 client.7605 populate_metadata read hostname 'smithi131' 2024-09-17T02:49:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x557a1b5157d0 0x557a1ba82d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 4 client.7605 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:49:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 -- 172.21.15.131:0/1802954093 --> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a1b885a90 con 0x557a1b5157d0 2024-09-17T02:49:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30b1a00640 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x557a1b5157d0 0x557a1ba82d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f308cff9640 10 client.7605 ms_handle_connect on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30b1a00640 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x557a1b5157d0 0x557a1ba82d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=670222e1de792b96 server_cookie=8541af9421d7d103 in_seq=0 out_seq=0 2024-09-17T02:49:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f308cff9640 1 -- 172.21.15.131:0/1802954093 <== mds.0 v2:172.21.15.131:6834/1957540940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557a1b885a90 con 0x557a1b5157d0 2024-09-17T02:49:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f308cff9640 10 client.7605 handle_command_reply: tid=0 2024-09-17T02:49:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 client.7605 shutdown 2024-09-17T02:49:13.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 -- 172.21.15.131:0/1802954093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 msgr2=0x557a1ba78e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1ba78e60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f30a8008010 tx=0x7f30a8012630 comp rx=0 tx=0).stop 2024-09-17T02:49:13.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 -- 172.21.15.131:0/1802954093 shutdown_connections 2024-09-17T02:49:13.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x557a1b5157d0 0x557a1ba82d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b920950 0x557a1ba793a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1ba78e60 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f308cff9640 0 client.7605 ms_handle_reset on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:13.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 --2- 172.21.15.131:0/1802954093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1ba78920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 -- 172.21.15.131:0/1802954093 >> 172.21.15.131:0/1802954093 conn(0x557a1b9f6d90 msgr2=0x557a1ba2d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:13.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 -- 172.21.15.131:0/1802954093 shutdown_connections 2024-09-17T02:49:13.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 1 -- 172.21.15.131:0/1802954093 wait complete. 2024-09-17T02:49:13.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.096+0000 7f30baf2f000 20 client.7605 trim_cache size 0 max 16384 2024-09-17T02:49:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30baf2f000 1 Processor -- start 2024-09-17T02:49:13.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30baf2f000 1 -- start start 2024-09-17T02:49:13.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1bae7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1bae7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b920950 0x557a1bae84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30baf2f000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac203e80 con 0x557a1b909ca0 2024-09-17T02:49:13.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30baf2f000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30ac202e60 con 0x557a1b920950 2024-09-17T02:49:13.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30baf2f000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f30ac1fade0 con 0x557a1b530460 2024-09-17T02:49:13.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30b11ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1bae7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30b11ff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1bae7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45516/0 (socket says 172.21.15.131:45516) 2024-09-17T02:49:13.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30b11ff640 1 -- 172.21.15.131:0/3887991169 learned_addr learned my addr 172.21.15.131:0/3887991169 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:13.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30b2201640 1 --2- 172.21.15.131:0/3887991169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b920950 0x557a1bae84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30b11ff640 1 -- 172.21.15.131:0/3887991169 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 msgr2=0x557a1bae7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:13.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30b11ff640 1 --2- 172.21.15.131:0/3887991169 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1bae7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30b11ff640 1 -- 172.21.15.131:0/3887991169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b920950 msgr2=0x557a1bae84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:13.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30b11ff640 1 --2- 172.21.15.131:0/3887991169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b920950 0x557a1bae84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30b11ff640 1 -- 172.21.15.131:0/3887991169 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a1b8cedb0 con 0x557a1b909ca0 2024-09-17T02:49:13.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.116+0000 7f30b11ff640 1 --2- 172.21.15.131:0/3887991169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1bae7fb0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f30a8000c10 tx=0x7f30a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6919d47c9bf60b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:13.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f308effd640 1 -- 172.21.15.131:0/3887991169 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8020040 con 0x557a1b909ca0 2024-09-17T02:49:13.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f308effd640 1 -- 172.21.15.131:0/3887991169 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30a8012c30 con 0x557a1b909ca0 2024-09-17T02:49:13.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f308effd640 1 -- 172.21.15.131:0/3887991169 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8015040 con 0x557a1b909ca0 2024-09-17T02:49:13.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 -- 172.21.15.131:0/3887991169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 msgr2=0x557a1bae7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 --2- 172.21.15.131:0/3887991169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1bae7fb0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f30a8000c10 tx=0x7f30a800cde0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 -- 172.21.15.131:0/3887991169 shutdown_connections 2024-09-17T02:49:13.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 --2- 172.21.15.131:0/3887991169 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b920950 0x557a1bae84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 --2- 172.21.15.131:0/3887991169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b909ca0 0x557a1bae7fb0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 --2- 172.21.15.131:0/3887991169 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b530460 0x557a1bae7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 -- 172.21.15.131:0/3887991169 >> 172.21.15.131:0/3887991169 conn(0x557a1b9f6d90 msgr2=0x557a1ba280d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:13.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 -- 172.21.15.131:0/3887991169 shutdown_connections 2024-09-17T02:49:13.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 -- 172.21.15.131:0/3887991169 wait complete. 2024-09-17T02:49:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 -- start start 2024-09-17T02:49:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 Processor -- start 2024-09-17T02:49:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b530460 0x557a1ba70af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b909ca0 0x557a1ba71030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b920950 0x557a1baa7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x557a1b87b3b0 con 0x557a1b530460 2024-09-17T02:49:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x557a1b876090 con 0x557a1b909ca0 2024-09-17T02:49:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x557a1b882910 con 0x557a1b920950 2024-09-17T02:49:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30baf2f000 10 client.0 _ll_register_callbacks cb 0x7f30a800a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:49:13.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30b1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b530460 0x557a1ba70af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30b1a00640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b530460 0x557a1ba70af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45528/0 (socket says 172.21.15.131:45528) 2024-09-17T02:49:13.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30b1a00640 1 -- 172.21.15.131:0/4029592049 learned_addr learned my addr 172.21.15.131:0/4029592049 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:13.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30b11ff640 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b909ca0 0x557a1ba71030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30b1a00640 1 -- 172.21.15.131:0/4029592049 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b920950 msgr2=0x557a1baa7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:13.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30b1a00640 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b920950 0x557a1baa7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30b1a00640 1 -- 172.21.15.131:0/4029592049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b909ca0 msgr2=0x557a1ba71030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:13.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30b1a00640 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b909ca0 0x557a1ba71030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30b1a00640 1 -- 172.21.15.131:0/4029592049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a1b839ab0 con 0x557a1b530460 2024-09-17T02:49:13.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f30b1a00640 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b530460 0x557a1ba70af0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f309c01a4e0 tx=0x7f309c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234765eb3bcb5e6a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:13.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.120+0000 7f308cff9640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:49:13.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f308cff9640 1 -- 172.21.15.131:0/4029592049 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c0b2040 con 0x557a1b530460 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 -- 172.21.15.131:0/4029592049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30a8012c30 con 0x557a1b530460 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f308cff9640 1 -- 172.21.15.131:0/4029592049 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f309c0a58f0 con 0x557a1b530460 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f308cff9640 1 -- 172.21.15.131:0/4029592049 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f309c0a5af0 con 0x557a1b530460 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 10 client.7611 fetch_fsmap learned FSMap version 77 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 -- 172.21.15.131:0/4029592049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x557a1b818070 con 0x557a1b530460 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f308cff9640 1 -- 172.21.15.131:0/4029592049 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30a8012c30 con 0x557a1b530460 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f308cff9640 1 -- 172.21.15.131:0/4029592049 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7f309c0b5030 con 0x557a1b530460 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 10 client.7611 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 10 client.7611 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 20 client.7611 populate_metadata read hostname 'smithi131' 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x557a1b503830 0x557a1baeb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 4 client.7611 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:49:13.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 -- 172.21.15.131:0/4029592049 --> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] -- command(tid 0: {"prefix": "status"}) -- 0x557a1b881540 con 0x557a1b503830 2024-09-17T02:49:13.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b11ff640 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x557a1b503830 0x557a1baeb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f308cff9640 10 client.7611 ms_handle_connect on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:13.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b11ff640 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x557a1b503830 0x557a1baeb6e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbc26701c0eba0f5 server_cookie=d0a49cf28be8dc1a in_seq=0 out_seq=0 2024-09-17T02:49:13.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f308cff9640 1 -- 172.21.15.131:0/4029592049 <== mds.0 v2:172.21.15.131:6834/1957540940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x557a1b881540 con 0x557a1b503830 2024-09-17T02:49:13.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f308cff9640 10 client.7611 handle_command_reply: tid=0 2024-09-17T02:49:13.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 client.7611 shutdown 2024-09-17T02:49:13.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 -- 172.21.15.131:0/4029592049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b530460 msgr2=0x557a1ba70af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b530460 0x557a1ba70af0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f309c01a4e0 tx=0x7f309c019650 comp rx=0 tx=0).stop 2024-09-17T02:49:13.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 -- 172.21.15.131:0/4029592049 shutdown_connections 2024-09-17T02:49:13.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x557a1b503830 0x557a1baeb6e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x557a1b920950 0x557a1baa7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f308cff9640 0 client.7611 ms_handle_reset on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:13.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x557a1b909ca0 0x557a1ba71030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 --2- 172.21.15.131:0/4029592049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x557a1b530460 0x557a1ba70af0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 -- 172.21.15.131:0/4029592049 >> 172.21.15.131:0/4029592049 conn(0x557a1b9f6d90 msgr2=0x557a1ba28a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:13.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 -- 172.21.15.131:0/4029592049 shutdown_connections 2024-09-17T02:49:13.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 1 -- 172.21.15.131:0/4029592049 wait complete. 2024-09-17T02:49:13.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30baf2f000 20 client.7611 trim_cache size 0 max 16384 2024-09-17T02:49:13.129 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:49:13.129 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:49:13.129 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:49:13.129 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7521, 2024-09-17T02:49:13.129 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:49:13.129 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:49:13.129 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:49:13.129 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 4.6159286179999999, 2024-09-17T02:49:13.129 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 77, 2024-09-17T02:49:13.129 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 96, 2024-09-17T02:49:13.130 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 96, 2024-09-17T02:49:13.130 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 5.555602844 2024-09-17T02:49:13.130 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:49:13.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 -- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3090073230 msgr2=0x7f3090075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3090073230 0x7f3090075740 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f309c0af000 tx=0x7f309c01b0f0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 -- 172.21.15.131:0/2871279870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30ac155240 msgr2=0x7f30ac151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30ac155240 0x7f30ac151fd0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f30a400c760 tx=0x7f30a400cc20 comp rx=0 tx=0).stop 2024-09-17T02:49:13.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 -- 172.21.15.131:0/2871279870 shutdown_connections 2024-09-17T02:49:13.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3090073230 0x7f3090075740 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30ac155240 0x7f30ac151fd0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30ac073970 0x7f30ac159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 --2- 172.21.15.131:0/2871279870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30ac06e3e0 0x7f30ac157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 -- 172.21.15.131:0/2871279870 >> 172.21.15.131:0/2871279870 conn(0x7f30ac06eea0 msgr2=0x7f30ac0b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:13.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 -- 172.21.15.131:0/2871279870 shutdown_connections 2024-09-17T02:49:13.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.124+0000 7f30b3c64640 1 -- 172.21.15.131:0/2871279870 wait complete. 2024-09-17T02:49:13.141 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:49:13.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.292+0000 7fda56b76640 1 -- 172.21.15.131:0/777834632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda34004240 msgr2=0x7fda34004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.292+0000 7fda56b76640 1 --2- 172.21.15.131:0/777834632 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda34004240 0x7fda34004810 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fda3c001860 tx=0x7fda3c01f050 comp rx=0 tx=0).stop 2024-09-17T02:49:13.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.292+0000 7fda55113640 1 -- 172.21.15.131:0/777834632 reap_dead start 2024-09-17T02:49:13.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.292+0000 7fda56b76640 1 -- 172.21.15.131:0/777834632 shutdown_connections 2024-09-17T02:49:13.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.292+0000 7fda56b76640 1 -- 172.21.15.131:0/777834632 >> 172.21.15.131:0/777834632 conn(0x7fda5006eea0 msgr2=0x7fda5006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:13.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.292+0000 7fda56b76640 1 -- 172.21.15.131:0/777834632 shutdown_connections 2024-09-17T02:49:13.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.292+0000 7fda56b76640 1 -- 172.21.15.131:0/777834632 wait complete. 2024-09-17T02:49:13.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda56b76640 1 Processor -- start 2024-09-17T02:49:13.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda56b76640 1 -- start start 2024-09-17T02:49:13.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda56b76640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda50152ae0 0x7fda500c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda56b76640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50152280 0x7fda5015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda56b76640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda50152690 0x7fda5015a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda56b76640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fda500cbe80 con 0x7fda50152280 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda56b76640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fda500cbd00 con 0x7fda50152ae0 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda56b76640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fda500cc000 con 0x7fda50152690 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50152280 0x7fda5015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50152280 0x7fda5015a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45542/0 (socket says 172.21.15.131:45542) 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4ffff640 1 -- 172.21.15.131:0/463007049 learned_addr learned my addr 172.21.15.131:0/463007049 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda54912640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda50152ae0 0x7fda500c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda55113640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda50152690 0x7fda5015a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4ffff640 1 -- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda50152690 msgr2=0x7fda5015a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4ffff640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda50152690 0x7fda5015a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4ffff640 1 -- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda50152ae0 msgr2=0x7fda500c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4ffff640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda50152ae0 0x7fda500c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4ffff640 1 -- 172.21.15.131:0/463007049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda501f6cc0 con 0x7fda50152280 2024-09-17T02:49:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4ffff640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50152280 0x7fda5015a2c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fda3c01d750 tx=0x7fda3c001590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4d7fa640 1 -- 172.21.15.131:0/463007049 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c003600 con 0x7fda50152280 2024-09-17T02:49:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4d7fa640 1 -- 172.21.15.131:0/463007049 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda3c0b7030 con 0x7fda50152280 2024-09-17T02:49:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda4d7fa640 1 -- 172.21.15.131:0/463007049 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c003ad0 con 0x7fda50152280 2024-09-17T02:49:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda56b76640 1 -- 172.21.15.131:0/463007049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda501f6fa0 con 0x7fda50152280 2024-09-17T02:49:13.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda56b76640 1 -- 172.21.15.131:0/463007049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda501f7560 con 0x7fda50152280 2024-09-17T02:49:13.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.296+0000 7fda56b76640 1 -- 172.21.15.131:0/463007049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda50075b80 con 0x7fda50152280 2024-09-17T02:49:13.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.300+0000 7fda4d7fa640 1 -- 172.21.15.131:0/463007049 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fda3c003c70 con 0x7fda50152280 2024-09-17T02:49:13.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.300+0000 7fda4d7fa640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda38073230 0x7fda38075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:13.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.300+0000 7fda4d7fa640 1 -- 172.21.15.131:0/463007049 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7fda3c132070 con 0x7fda50152280 2024-09-17T02:49:13.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.300+0000 7fda54912640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda38073230 0x7fda38075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:13.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.300+0000 7fda54912640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda38073230 0x7fda38075740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fda40000c10 tx=0x7fda40001c00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:13.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.304+0000 7fda4d7fa640 1 -- 172.21.15.131:0/463007049 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fda3c018e00 con 0x7fda50152280 2024-09-17T02:49:13.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.496+0000 7fda56b76640 1 -- 172.21.15.131:0/463007049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fda500cc560 con 0x7fda50152280 2024-09-17T02:49:13.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.496+0000 7fda4d7fa640 1 -- 172.21.15.131:0/463007049 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) ==== 53+0+73 (secure 0 0 0) 0x7fda3c0fd020 con 0x7fda50152280 2024-09-17T02:49:13.494 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:49:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 -- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda38073230 msgr2=0x7fda38075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda38073230 0x7fda38075740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fda40000c10 tx=0x7fda40001c00 comp rx=0 tx=0).stop 2024-09-17T02:49:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 -- 172.21.15.131:0/463007049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50152280 msgr2=0x7fda5015a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50152280 0x7fda5015a2c0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fda3c01d750 tx=0x7fda3c001590 comp rx=0 tx=0).stop 2024-09-17T02:49:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 -- 172.21.15.131:0/463007049 shutdown_connections 2024-09-17T02:49:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fda38073230 0x7fda38075740 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fda50152690 0x7fda5015a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fda50152280 0x7fda5015a2c0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 --2- 172.21.15.131:0/463007049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fda50152ae0 0x7fda500c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 -- 172.21.15.131:0/463007049 >> 172.21.15.131:0/463007049 conn(0x7fda5006eea0 msgr2=0x7fda50155490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:13.497 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 -- 172.21.15.131:0/463007049 shutdown_connections 2024-09-17T02:49:13.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13.500+0000 7fda56b76640 1 -- 172.21.15.131:0/463007049 wait complete. 2024-09-17T02:49:13.515 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:49:13.515 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:49:13.515 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:49:13.515 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:49:13.515 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:49:13.515 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:49:13.515 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:49:13.515 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:49:13.516 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:49:13.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:13.516 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:49:13.521 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:49:13.521 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:49:13.521 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:49:13.521 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:49:13.521 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:49:13.521 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:49:13.521 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:49:13.521 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41953sec preferred_lft 41953sec 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:49:13.522 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:49:13.523 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:49:13.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:13.524 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:49:13.573 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:49:13.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:13.574 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:49:13.624 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:49:13.624 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:49:13.624 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:49:13.624 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:49:13.624 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:49:13.624 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:49:13.624 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:49:13.624 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:49:13.624 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:49:13.624 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:49:13.624 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:49:13.625 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:49:13.625 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:49:13.625 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:49:13.625 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:49:13.625 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:49:13.625 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:49:13.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:49:13.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:49:13.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:13.743 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:49:13.791 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:49:13.792 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:49:13.792 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:49:13.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:13.792 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:49:13.846 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:49:13.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:49:13.847 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:49:13.892 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:49:13.892 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:49:13.892 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:49:13.939 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:49:13.998 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:49:14.045 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:49:14.045 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:49:14.045 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:49:14.045 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:49:14.045 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:49:14.045 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:49:14.045 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:49:14.046 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:49:14.047 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:49:14.048 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:49:14.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:14.049 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:49:14.100 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:49:14.100 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:49:14.145 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:49:14.159 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:49:14.209 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:49:14.209 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:49:14.209 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:49:14.209 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:49:14.209 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:49:14.209 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:49:14.209 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:49:14.209 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:49:14.209 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:49:14.210 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:49:14.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:14.212 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:49:14.252 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:49:14.256+0000 7fb672a05580 -1 init, newargv = 0x7fb6600ad020 newargc=15 2024-09-17T02:49:14.252 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[47705]: starting ceph client 2024-09-17T02:49:14.258 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[47705]: starting fuse 2024-09-17T02:49:14.506 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-09-17T02:49:14.512+0000 7f058d3a5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:49:15.216 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:49:15.219 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:49:15.356 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:49:15.373 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:49:15.422 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:49:15.422 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:49:15.422 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:49:15.422 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:49:15.422 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:49:15.422 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:49:15.422 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:49:15.422 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:49:15.422 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:49:15.423 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:49:15.424 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:49:15.424 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:49:15.424 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:49:15.424 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:49:15.424 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:49:15.424 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:49:15.424 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:49:15.424 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:49:15.424 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:49:15.424 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:49:15.424 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:49:15.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:15.426 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:49:15.472 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:49:15.473 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:49:15.473 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:49:15.473 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:49:15.473 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:49:15.473 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:49:15.473 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:49:15.473 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:49:15.473 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:49:15.473 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:49:15.473 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:49:15.473 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:49:15.474 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:49:15.475 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:49:15.475 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:49:15.601 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.47705.asok 2024-09-17T02:49:15.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:49:15.612 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.47705.asok 2024-09-17T02:49:15.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:15.613 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.47705.asok status 2024-09-17T02:49:15.792 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:49:15.792 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:49:15.792 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "47705", 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7626, 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:49:15.793 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 7626 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1845850159 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1845850159 2024-09-17T02:49:15.794 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:15.795 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.7626 192.168.0.1:0/1845850159", 2024-09-17T02:49:15.795 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/1845850159", 2024-09-17T02:49:15.795 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:49:15.795 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 77, 2024-09-17T02:49:15.795 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 96, 2024-09-17T02:49:15.795 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:49:15.795 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:49:15.795 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:49:15.795 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:49:15.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:15.810 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:49:15.810 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:49:15.816 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:49:15.816 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:49:15.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:15.816 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:49:15.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:15.872 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-09-17T02:49:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.088+0000 7efcb0eea640 1 -- 172.21.15.131:0/395287520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcac0b2ef0 msgr2=0x7efcac0cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.088+0000 7efcb0eea640 1 --2- 172.21.15.131:0/395287520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcac0b2ef0 0x7efcac0cf6d0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009900 tx=0x7efc9c01c850 comp rx=0 tx=0).stop 2024-09-17T02:49:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.088+0000 7efcb0eea640 1 -- 172.21.15.131:0/395287520 shutdown_connections 2024-09-17T02:49:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.088+0000 7efcb0eea640 1 --2- 172.21.15.131:0/395287520 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7efcac0cbed0 0x7efcac0cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.088+0000 7efcb0eea640 1 --2- 172.21.15.131:0/395287520 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcac0b2ef0 0x7efcac0cf6d0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.088+0000 7efcb0eea640 1 -- 172.21.15.131:0/395287520 >> 172.21.15.131:0/395287520 conn(0x7efcac0d05a0 msgr2=0x7efcac0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.088+0000 7efcb0eea640 1 -- 172.21.15.131:0/395287520 shutdown_connections 2024-09-17T02:49:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.088+0000 7efcb0eea640 1 -- 172.21.15.131:0/395287520 wait complete. 2024-09-17T02:49:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.088+0000 7efcb0eea640 1 Processor -- start 2024-09-17T02:49:16.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.088+0000 7efcb0eea640 1 -- start start 2024-09-17T02:49:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcb0eea640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcac06f700 0x7efcac13fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcb0eea640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcac0b2ef0 0x7efcac159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcb0eea640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7efcac0cbed0 0x7efcac0cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcb0eea640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efcac074330 con 0x7efcac06f700 2024-09-17T02:49:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcb0eea640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efcac0741b0 con 0x7efcac0b2ef0 2024-09-17T02:49:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcb0eea640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7efcac0744b0 con 0x7efcac0cbed0 2024-09-17T02:49:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcaa59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcac06f700 0x7efcac13fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcaa59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcac06f700 0x7efcac13fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45560/0 (socket says 172.21.15.131:45560) 2024-09-17T02:49:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcaa59c640 1 -- 172.21.15.131:0/281855154 learned_addr learned my addr 172.21.15.131:0/281855154 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efca9d9b640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcac0b2ef0 0x7efcac159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcaa59c640 1 -- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7efcac0cbed0 msgr2=0x7efcac0cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcaad9d640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7efcac0cbed0 0x7efcac0cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcaa59c640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7efcac0cbed0 0x7efcac0cd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcaa59c640 1 -- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcac0b2ef0 msgr2=0x7efcac159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcaa59c640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcac0b2ef0 0x7efcac159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcaa59c640 1 -- 172.21.15.131:0/281855154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcac1faba0 con 0x7efcac06f700 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efca9d9b640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcac0b2ef0 0x7efcac159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcaa59c640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcac06f700 0x7efcac13fc80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7efc9c01d0f0 tx=0x7efc9c0a65d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efc9affd640 1 -- 172.21.15.131:0/281855154 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0ac070 con 0x7efcac06f700 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efc9affd640 1 -- 172.21.15.131:0/281855154 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc9c0054b0 con 0x7efcac06f700 2024-09-17T02:49:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcb0eea640 1 -- 172.21.15.131:0/281855154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcac1fae80 con 0x7efcac06f700 2024-09-17T02:49:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efc9affd640 1 -- 172.21.15.131:0/281855154 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c004290 con 0x7efcac06f700 2024-09-17T02:49:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efcb0eea640 1 -- 172.21.15.131:0/281855154 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcac1fb440 con 0x7efcac06f700 2024-09-17T02:49:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efc9affd640 1 -- 172.21.15.131:0/281855154 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7efc9c0af070 con 0x7efcac06f700 2024-09-17T02:49:16.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efc9affd640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efc8c073200 0x7efc8c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.092+0000 7efc9affd640 1 -- 172.21.15.131:0/281855154 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7efc9c12e5f0 con 0x7efcac06f700 2024-09-17T02:49:16.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efca9d9b640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efc8c073200 0x7efc8c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 Processor -- start 2024-09-17T02:49:16.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 -- start start 2024-09-17T02:49:16.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efca9d9b640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efc8c073200 0x7efc8c075710 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efcac0aa160 tx=0x7efca0008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e776640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e776b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e7770c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b17e53bbd0 con 0x55b17e62d1a0 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b17e52c430 con 0x55b17e625ae0 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b17e589ce0 con 0x55b17e60f970 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcaa59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e776640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcaa59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e776640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45562/0 (socket says 172.21.15.131:45562) 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcaa59c640 1 -- 172.21.15.131:0/3467298482 learned_addr learned my addr 172.21.15.131:0/3467298482 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efca9d9b640 1 --2- 172.21.15.131:0/3467298482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e776b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcaad9d640 1 --2- 172.21.15.131:0/3467298482 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e7770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcaa59c640 1 -- 172.21.15.131:0/3467298482 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 msgr2=0x55b17e7770c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcaa59c640 1 --2- 172.21.15.131:0/3467298482 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e7770c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcaa59c640 1 -- 172.21.15.131:0/3467298482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 msgr2=0x55b17e776b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcaa59c640 1 --2- 172.21.15.131:0/3467298482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e776b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcaa59c640 1 -- 172.21.15.131:0/3467298482 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17e52dd80 con 0x55b17e62d1a0 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcaa59c640 1 --2- 172.21.15.131:0/3467298482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e776640 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0f8c40 tx=0x7efc9c0f8b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaf69b777d034e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efc8bfff640 1 -- 172.21.15.131:0/3467298482 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0056a0 con 0x55b17e62d1a0 2024-09-17T02:49:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efc8bfff640 1 -- 172.21.15.131:0/3467298482 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc9c0f87c0 con 0x55b17e62d1a0 2024-09-17T02:49:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efc8bfff640 1 -- 172.21.15.131:0/3467298482 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0f93b0 con 0x55b17e62d1a0 2024-09-17T02:49:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 -- 172.21.15.131:0/3467298482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 msgr2=0x55b17e776640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 --2- 172.21.15.131:0/3467298482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e776640 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0f8c40 tx=0x7efc9c0f8b80 comp rx=0 tx=0).stop 2024-09-17T02:49:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 -- 172.21.15.131:0/3467298482 shutdown_connections 2024-09-17T02:49:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 --2- 172.21.15.131:0/3467298482 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e7770c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 --2- 172.21.15.131:0/3467298482 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e776b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 --2- 172.21.15.131:0/3467298482 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e776640 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 -- 172.21.15.131:0/3467298482 >> 172.21.15.131:0/3467298482 conn(0x55b17e6338c0 msgr2=0x55b17e72b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.096+0000 7efcb81b5000 1 -- 172.21.15.131:0/3467298482 shutdown_connections 2024-09-17T02:49:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 -- 172.21.15.131:0/3467298482 wait complete. 2024-09-17T02:49:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 -- start start 2024-09-17T02:49:16.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 Processor -- start 2024-09-17T02:49:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e764100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e625ae0 0x55b17e764a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e62d1a0 0x55b17e764f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b17e57cf20 con 0x55b17e625ae0 2024-09-17T02:49:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b17e579110 con 0x55b17e62d1a0 2024-09-17T02:49:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b17e51e6d0 con 0x55b17e60f970 2024-09-17T02:49:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 10 client.0 _ll_register_callbacks cb 0x55b17e51d330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:49:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efca9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e625ae0 0x55b17e764a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efca9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e625ae0 0x55b17e764a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45566/0 (socket says 172.21.15.131:45566) 2024-09-17T02:49:16.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efca9d9b640 1 -- 172.21.15.131:0/700726219 learned_addr learned my addr 172.21.15.131:0/700726219 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcaa59c640 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e764100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcaad9d640 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e62d1a0 0x55b17e764f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efca9d9b640 1 -- 172.21.15.131:0/700726219 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 msgr2=0x55b17e764100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efca9d9b640 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e764100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efca9d9b640 1 -- 172.21.15.131:0/700726219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e62d1a0 msgr2=0x55b17e764f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efca9d9b640 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e62d1a0 0x55b17e764f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efca9d9b640 1 -- 172.21.15.131:0/700726219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b17e517310 con 0x55b17e625ae0 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efca9d9b640 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e625ae0 0x55b17e764a20 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efca000cdf0 tx=0x7efca0004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14036faf8b5cd18 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efc897fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efc897fa640 1 -- 172.21.15.131:0/700726219 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca001f420 con 0x55b17e625ae0 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efc897fa640 1 -- 172.21.15.131:0/700726219 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7efca001f5c0 con 0x55b17e625ae0 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 -- 172.21.15.131:0/700726219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b17e511450 con 0x55b17e625ae0 2024-09-17T02:49:16.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efc897fa640 1 -- 172.21.15.131:0/700726219 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7efca002c050 con 0x55b17e625ae0 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 10 client.7638 fetch_fsmap learned FSMap version 77 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 -- 172.21.15.131:0/700726219 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55b17e534f60 con 0x55b17e625ae0 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efc897fa640 1 -- 172.21.15.131:0/700726219 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b17e511450 con 0x55b17e625ae0 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efc897fa640 1 -- 172.21.15.131:0/700726219 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7efca002f020 con 0x55b17e625ae0 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 10 client.7638 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 10 client.7638 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 20 client.7638 populate_metadata read hostname 'smithi131' 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b17e21bb80 0x55b17e765ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 4 client.7638 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.100+0000 7efcb81b5000 1 -- 172.21.15.131:0/700726219 --> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b17e58bc30 con 0x55b17e21bb80 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcaa59c640 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b17e21bb80 0x55b17e765ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efc897fa640 10 client.7638 ms_handle_connect on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcaa59c640 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b17e21bb80 0x55b17e765ca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcffe48893ee0041 server_cookie=57828f009aebf801 in_seq=0 out_seq=0 2024-09-17T02:49:16.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efc897fa640 1 -- 172.21.15.131:0/700726219 <== mds.0 v2:172.21.15.131:6834/1957540940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b17e58bc30 con 0x55b17e21bb80 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efc897fa640 10 client.7638 handle_command_reply: tid=0 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 1 client.7638 shutdown 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 1 -- 172.21.15.131:0/700726219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e625ae0 msgr2=0x55b17e764a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e625ae0 0x55b17e764a20 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efca000cdf0 tx=0x7efca0004690 comp rx=0 tx=0).stop 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 1 -- 172.21.15.131:0/700726219 shutdown_connections 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b17e21bb80 0x55b17e765ca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e62d1a0 0x55b17e764f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e625ae0 0x55b17e764a20 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efc897fa640 0 client.7638 ms_handle_reset on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 1 --2- 172.21.15.131:0/700726219 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e764100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 1 -- 172.21.15.131:0/700726219 >> 172.21.15.131:0/700726219 conn(0x55b17e6338c0 msgr2=0x55b17e72b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.104 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 1 -- 172.21.15.131:0/700726219 shutdown_connections 2024-09-17T02:49:16.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 1 -- 172.21.15.131:0/700726219 wait complete. 2024-09-17T02:49:16.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.104+0000 7efcb81b5000 20 client.7638 trim_cache size 0 max 16384 2024-09-17T02:49:16.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 Processor -- start 2024-09-17T02:49:16.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 -- start start 2024-09-17T02:49:16.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e764ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e763b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e7640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7efcac203e20 con 0x55b17e62d1a0 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7efcac202e30 con 0x55b17e625ae0 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7efcac142d80 con 0x55b17e60f970 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcaad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e7640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcaad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e7640c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45580/0 (socket says 172.21.15.131:45580) 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcaad9d640 1 -- 172.21.15.131:0/3894182882 learned_addr learned my addr 172.21.15.131:0/3894182882 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efca9d9b640 1 --2- 172.21.15.131:0/3894182882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e763b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcaad9d640 1 -- 172.21.15.131:0/3894182882 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 msgr2=0x55b17e764ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcaa59c640 1 --2- 172.21.15.131:0/3894182882 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e764ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcaad9d640 1 --2- 172.21.15.131:0/3894182882 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e764ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcaad9d640 1 -- 172.21.15.131:0/3894182882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 msgr2=0x55b17e763b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcaad9d640 1 --2- 172.21.15.131:0/3894182882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e763b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcaad9d640 1 -- 172.21.15.131:0/3894182882 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b17e5fdf30 con 0x55b17e62d1a0 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efca9d9b640 1 --2- 172.21.15.131:0/3894182882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e763b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcaad9d640 1 --2- 172.21.15.131:0/3894182882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e7640c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7efca400bda0 tx=0x7efca4012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6dd5fd0e409124d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efc8b7fe640 1 -- 172.21.15.131:0/3894182882 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4010870 con 0x55b17e62d1a0 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efc8b7fe640 1 -- 172.21.15.131:0/3894182882 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efca4010a10 con 0x55b17e62d1a0 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efc8b7fe640 1 -- 172.21.15.131:0/3894182882 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4010e60 con 0x55b17e62d1a0 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 -- 172.21.15.131:0/3894182882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 msgr2=0x55b17e7640c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 --2- 172.21.15.131:0/3894182882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e7640c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7efca400bda0 tx=0x7efca4012820 comp rx=0 tx=0).stop 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 -- 172.21.15.131:0/3894182882 shutdown_connections 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 --2- 172.21.15.131:0/3894182882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e62d1a0 0x55b17e7640c0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 --2- 172.21.15.131:0/3894182882 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e763b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 --2- 172.21.15.131:0/3894182882 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e60f970 0x55b17e764ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 -- 172.21.15.131:0/3894182882 >> 172.21.15.131:0/3894182882 conn(0x55b17e6338c0 msgr2=0x55b17e731a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 -- 172.21.15.131:0/3894182882 shutdown_connections 2024-09-17T02:49:16.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.124+0000 7efcb81b5000 1 -- 172.21.15.131:0/3894182882 wait complete. 2024-09-17T02:49:16.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 -- start start 2024-09-17T02:49:16.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 Processor -- start 2024-09-17T02:49:16.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e60f970 0x55b17e7ebbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e7ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e62d1a0 0x55b17e7ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b17e53bbd0 con 0x55b17e60f970 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b17e52c430 con 0x55b17e625ae0 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b17e5fdaa0 con 0x55b17e62d1a0 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 10 client.0 _ll_register_callbacks cb 0x7efca000a440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcaa59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e60f970 0x55b17e7ebbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcaa59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e60f970 0x55b17e7ebbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45588/0 (socket says 172.21.15.131:45588) 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcaa59c640 1 -- 172.21.15.131:0/4092917500 learned_addr learned my addr 172.21.15.131:0/4092917500 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efca9d9b640 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e7ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcaa59c640 1 -- 172.21.15.131:0/4092917500 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e62d1a0 msgr2=0x55b17e7ec630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcaad9d640 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e62d1a0 0x55b17e7ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcaa59c640 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e62d1a0 0x55b17e7ec630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcaa59c640 1 -- 172.21.15.131:0/4092917500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 msgr2=0x55b17e7ec0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcaa59c640 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e7ec0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcaa59c640 1 -- 172.21.15.131:0/4092917500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b17e5d4dd0 con 0x55b17e60f970 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efca9d9b640 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e7ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcaa59c640 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e60f970 0x55b17e7ebbb0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0981b0 tx=0x7efc9c0f8420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326eb02d3827b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efc897fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efc897fa640 1 -- 172.21.15.131:0/4092917500 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c119f90 con 0x55b17e60f970 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 -- 172.21.15.131:0/4092917500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efca4010a10 con 0x55b17e60f970 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efc897fa640 1 -- 172.21.15.131:0/4092917500 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7efc9c11e1d0 con 0x55b17e60f970 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efc897fa640 1 -- 172.21.15.131:0/4092917500 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7efc9c0056a0 con 0x55b17e60f970 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 10 client.7644 fetch_fsmap learned FSMap version 77 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 -- 172.21.15.131:0/4092917500 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55b17e50f840 con 0x55b17e60f970 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efc897fa640 1 -- 172.21.15.131:0/4092917500 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efca4010a10 con 0x55b17e60f970 2024-09-17T02:49:16.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efc897fa640 1 -- 172.21.15.131:0/4092917500 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7efc9c0b9070 con 0x55b17e60f970 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 10 client.7644 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 10 client.7644 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 20 client.7644 populate_metadata read hostname 'smithi131' 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b17e209be0 0x55b17e72c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 4 client.7644 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 -- 172.21.15.131:0/4092917500 --> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55b17e5876e0 con 0x55b17e209be0 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efca9d9b640 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b17e209be0 0x55b17e72c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efc897fa640 10 client.7644 ms_handle_connect on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efca9d9b640 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b17e209be0 0x55b17e72c020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf96023544816963 server_cookie=7ca1422b8b7b2b28 in_seq=0 out_seq=0 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efc897fa640 1 -- 172.21.15.131:0/4092917500 <== mds.0 v2:172.21.15.131:6834/1957540940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b17e5876e0 con 0x55b17e209be0 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efc897fa640 10 client.7644 handle_command_reply: tid=0 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.128+0000 7efcb81b5000 1 client.7644 shutdown 2024-09-17T02:49:16.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb81b5000 1 -- 172.21.15.131:0/4092917500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e60f970 msgr2=0x55b17e7ebbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb81b5000 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e60f970 0x55b17e7ebbb0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0981b0 tx=0x7efc9c0f8420 comp rx=0 tx=0).stop 2024-09-17T02:49:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb81b5000 1 -- 172.21.15.131:0/4092917500 shutdown_connections 2024-09-17T02:49:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb81b5000 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b17e209be0 0x55b17e72c020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb81b5000 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b17e62d1a0 0x55b17e7ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb81b5000 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b17e625ae0 0x55b17e7ec0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efc897fa640 0 client.7644 ms_handle_reset on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb81b5000 1 --2- 172.21.15.131:0/4092917500 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b17e60f970 0x55b17e7ebbb0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb81b5000 1 -- 172.21.15.131:0/4092917500 >> 172.21.15.131:0/4092917500 conn(0x55b17e6338c0 msgr2=0x55b17e732430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb81b5000 1 -- 172.21.15.131:0/4092917500 shutdown_connections 2024-09-17T02:49:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb81b5000 1 -- 172.21.15.131:0/4092917500 wait complete. 2024-09-17T02:49:16.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb81b5000 20 client.7644 trim_cache size 0 max 16384 2024-09-17T02:49:16.133 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:49:16.133 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-09-17T02:49:16.133 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "8992844c-fbc4-4c7b-b92b-66db1f2a5ed0", 2024-09-17T02:49:16.133 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-09-17T02:49:16.133 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:49:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 -- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efc8c073200 msgr2=0x7efc8c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efc8c073200 0x7efc8c075710 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efcac0aa160 tx=0x7efca0008040 comp rx=0 tx=0).stop 2024-09-17T02:49:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 -- 172.21.15.131:0/281855154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcac06f700 msgr2=0x7efcac13fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcac06f700 0x7efcac13fc80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7efc9c01d0f0 tx=0x7efc9c0a65d0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 -- 172.21.15.131:0/281855154 shutdown_connections 2024-09-17T02:49:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7efc8c073200 0x7efc8c075710 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7efcac0cbed0 0x7efcac0cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7efcac0b2ef0 0x7efcac159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 --2- 172.21.15.131:0/281855154 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7efcac06f700 0x7efcac13fc80 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 -- 172.21.15.131:0/281855154 >> 172.21.15.131:0/281855154 conn(0x7efcac0d05a0 msgr2=0x7efcac154180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 -- 172.21.15.131:0/281855154 shutdown_connections 2024-09-17T02:49:16.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.132+0000 7efcb0eea640 1 -- 172.21.15.131:0/281855154 wait complete. 2024-09-17T02:49:16.135 INFO:tasks.ceph.mds.c.smithi131.stderr:2024-09-17T02:49:16.132+0000 7f058939d640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | request=0 lock=0 dirtyparent=1 authpin=1 scrubqueue=0 0x558fe91b4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-17T02:49:16.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:16.148 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-17T02:49:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.304+0000 7fd425958640 1 -- 172.21.15.131:0/2212213059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd408003a90 msgr2=0x7fd408003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.304+0000 7fd425958640 1 --2- 172.21.15.131:0/2212213059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd408003a90 0x7fd408003f00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd410006fd0 tx=0x7fd41001cf20 comp rx=0 tx=0).stop 2024-09-17T02:49:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.304+0000 7fd425958640 1 -- 172.21.15.131:0/2212213059 shutdown_connections 2024-09-17T02:49:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.304+0000 7fd425958640 1 --2- 172.21.15.131:0/2212213059 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd408003a90 0x7fd408003f00 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.304+0000 7fd425958640 1 -- 172.21.15.131:0/2212213059 >> 172.21.15.131:0/2212213059 conn(0x7fd4200d2100 msgr2=0x7fd4200d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.304+0000 7fd425958640 1 -- 172.21.15.131:0/2212213059 shutdown_connections 2024-09-17T02:49:16.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.304+0000 7fd425958640 1 -- 172.21.15.131:0/2212213059 wait complete. 2024-09-17T02:49:16.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd425958640 1 Processor -- start 2024-09-17T02:49:16.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd425958640 1 -- start start 2024-09-17T02:49:16.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd425958640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd408003a90 0x7fd420151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd425958640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd4200b7460 0x7fd42015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd425958640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd420153e80 0x7fd42015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd425958640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd4201421f0 con 0x7fd420153e80 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd425958640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd420142070 con 0x7fd408003a90 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd425958640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd420142370 con 0x7fd4200b7460 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd420153e80 0x7fd42015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd420153e80 0x7fd42015b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45614/0 (socket says 172.21.15.131:45614) 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41f7fe640 1 -- 172.21.15.131:0/3599446991 learned_addr learned my addr 172.21.15.131:0/3599446991 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41e7fc640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd4200b7460 0x7fd42015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41effd640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd408003a90 0x7fd420151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41f7fe640 1 -- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd4200b7460 msgr2=0x7fd42015b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41f7fe640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd4200b7460 0x7fd42015b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41f7fe640 1 -- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd408003a90 msgr2=0x7fd420151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41f7fe640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd408003a90 0x7fd420151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41f7fe640 1 -- 172.21.15.131:0/3599446991 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4201f9520 con 0x7fd420153e80 2024-09-17T02:49:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41effd640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd408003a90 0x7fd420151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd41f7fe640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd420153e80 0x7fd42015b9a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd42015ca20 tx=0x7fd418001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd407fff640 1 -- 172.21.15.131:0/3599446991 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418011870 con 0x7fd420153e80 2024-09-17T02:49:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd407fff640 1 -- 172.21.15.131:0/3599446991 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd418017070 con 0x7fd420153e80 2024-09-17T02:49:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd407fff640 1 -- 172.21.15.131:0/3599446991 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd418011da0 con 0x7fd420153e80 2024-09-17T02:49:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd425958640 1 -- 172.21.15.131:0/3599446991 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4201f9800 con 0x7fd420153e80 2024-09-17T02:49:16.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.308+0000 7fd425958640 1 -- 172.21.15.131:0/3599446991 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4201f9dc0 con 0x7fd420153e80 2024-09-17T02:49:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.312+0000 7fd407fff640 1 -- 172.21.15.131:0/3599446991 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd41801c460 con 0x7fd420153e80 2024-09-17T02:49:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.312+0000 7fd407fff640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd40c073200 0x7fd40c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.312+0000 7fd407fff640 1 -- 172.21.15.131:0/3599446991 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7fd418094d60 con 0x7fd420153e80 2024-09-17T02:49:16.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.312+0000 7fd41effd640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd40c073200 0x7fd40c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.312+0000 7fd42cc23000 1 Processor -- start 2024-09-17T02:49:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.312+0000 7fd42cc23000 1 -- start start 2024-09-17T02:49:16.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.312+0000 7fd41effd640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd40c073200 0x7fd40c075710 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd4100ab000 tx=0x7fd410019d60 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d9f26d0 0x56370db5e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db5ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d70daa0 0x56370db5f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56370d9fdcd0 con 0x56370d649890 2024-09-17T02:49:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56370d913330 con 0x56370d9f26d0 2024-09-17T02:49:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56370d970bb0 con 0x56370d70daa0 2024-09-17T02:49:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd41e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db5ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd41e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db5ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45624/0 (socket says 172.21.15.131:45624) 2024-09-17T02:49:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd41e7fc640 1 -- 172.21.15.131:0/2522846819 learned_addr learned my addr 172.21.15.131:0/2522846819 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd41effd640 1 --2- 172.21.15.131:0/2522846819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d9f26d0 0x56370db5e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd41f7fe640 1 --2- 172.21.15.131:0/2522846819 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d70daa0 0x56370db5f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd41e7fc640 1 -- 172.21.15.131:0/2522846819 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d70daa0 msgr2=0x56370db5f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd41e7fc640 1 --2- 172.21.15.131:0/2522846819 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d70daa0 0x56370db5f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd41e7fc640 1 -- 172.21.15.131:0/2522846819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d9f26d0 msgr2=0x56370db5e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd41e7fc640 1 --2- 172.21.15.131:0/2522846819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d9f26d0 0x56370db5e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd41e7fc640 1 -- 172.21.15.131:0/2522846819 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56370d914c80 con 0x56370d649890 2024-09-17T02:49:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd41e7fc640 1 --2- 172.21.15.131:0/2522846819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db5ef30 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd4140099b0 tx=0x7fd414009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c57a0fde8b8deec server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd4057fa640 1 -- 172.21.15.131:0/2522846819 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414013d00 con 0x56370d649890 2024-09-17T02:49:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd4057fa640 1 -- 172.21.15.131:0/2522846819 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd414004020 con 0x56370d649890 2024-09-17T02:49:16.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd4057fa640 1 -- 172.21.15.131:0/2522846819 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414004450 con 0x56370d649890 2024-09-17T02:49:16.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 -- 172.21.15.131:0/2522846819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 msgr2=0x56370db5ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 --2- 172.21.15.131:0/2522846819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db5ef30 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd4140099b0 tx=0x7fd414009e70 comp rx=0 tx=0).stop 2024-09-17T02:49:16.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 -- 172.21.15.131:0/2522846819 shutdown_connections 2024-09-17T02:49:16.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 --2- 172.21.15.131:0/2522846819 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d70daa0 0x56370db5f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 --2- 172.21.15.131:0/2522846819 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db5ef30 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 --2- 172.21.15.131:0/2522846819 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d9f26d0 0x56370db5e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 -- 172.21.15.131:0/2522846819 >> 172.21.15.131:0/2522846819 conn(0x56370d61d5c0 msgr2=0x56370db1bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 -- 172.21.15.131:0/2522846819 shutdown_connections 2024-09-17T02:49:16.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 -- 172.21.15.131:0/2522846819 wait complete. 2024-09-17T02:49:16.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 -- start start 2024-09-17T02:49:16.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.316+0000 7fd42cc23000 1 Processor -- start 2024-09-17T02:49:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db65c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d70daa0 0x56370db661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 0x56370db666f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56370d963df0 con 0x56370d649890 2024-09-17T02:49:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56370d95ffe0 con 0x56370d70daa0 2024-09-17T02:49:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56370d9054d0 con 0x56370d9f26d0 2024-09-17T02:49:16.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 10 client.0 _ll_register_callbacks cb 0x56370d8e37f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:49:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db65c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db65c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45632/0 (socket says 172.21.15.131:45632) 2024-09-17T02:49:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41effd640 1 -- 172.21.15.131:0/397563313 learned_addr learned my addr 172.21.15.131:0/397563313 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41e7fc640 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d70daa0 0x56370db661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41f7fe640 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 0x56370db666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41effd640 1 -- 172.21.15.131:0/397563313 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 msgr2=0x56370db666f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41effd640 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 0x56370db666f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41effd640 1 -- 172.21.15.131:0/397563313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d70daa0 msgr2=0x56370db661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41effd640 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d70daa0 0x56370db661b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41effd640 1 -- 172.21.15.131:0/397563313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56370d8fe210 con 0x56370d649890 2024-09-17T02:49:16.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41f7fe640 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 0x56370db666f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:16.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41effd640 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db65c70 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd410001270 tx=0x7fd410006730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21400906e12194c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd3fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:49:16.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/397563313 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410003a70 con 0x56370d649890 2024-09-17T02:49:16.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/397563313 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7fd4100c9db0 con 0x56370d649890 2024-09-17T02:49:16.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 -- 172.21.15.131:0/397563313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56370d8f8350 con 0x56370d649890 2024-09-17T02:49:16.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/397563313 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd410003c10 con 0x56370d649890 2024-09-17T02:49:16.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 10 client.7656 fetch_fsmap learned FSMap version 77 2024-09-17T02:49:16.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 -- 172.21.15.131:0/397563313 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x56370d8e4fc0 con 0x56370d649890 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/397563313 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56370d8f8350 con 0x56370d649890 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/397563313 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7fd4100ba440 con 0x56370d649890 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 10 client.7656 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 10 client.7656 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 20 client.7656 populate_metadata read hostname 'smithi131' 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x56370d602930 0x56370db70050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 4 client.7656 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 -- 172.21.15.131:0/397563313 --> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56370d972b00 con 0x56370d602930 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41e7fc640 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x56370d602930 0x56370db70050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd3fe7fc640 10 client.7656 ms_handle_connect on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd41e7fc640 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x56370d602930 0x56370db70050 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aa665887a0fe899 server_cookie=aa35c974b9a594f5 in_seq=0 out_seq=0 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/397563313 <== mds.0 v2:172.21.15.131:6834/1957540940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56370d972b00 con 0x56370d602930 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd3fe7fc640 10 client.7656 handle_command_reply: tid=0 2024-09-17T02:49:16.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 client.7656 shutdown 2024-09-17T02:49:16.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 -- 172.21.15.131:0/397563313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 msgr2=0x56370db65c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db65c70 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd410001270 tx=0x7fd410006730 comp rx=0 tx=0).stop 2024-09-17T02:49:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 -- 172.21.15.131:0/397563313 shutdown_connections 2024-09-17T02:49:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x56370d602930 0x56370db70050 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 0x56370db666f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd3fe7fc640 0 client.7656 ms_handle_reset on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d70daa0 0x56370db661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 --2- 172.21.15.131:0/397563313 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d649890 0x56370db65c70 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 -- 172.21.15.131:0/397563313 >> 172.21.15.131:0/397563313 conn(0x56370d61d5c0 msgr2=0x56370db13d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 -- 172.21.15.131:0/397563313 shutdown_connections 2024-09-17T02:49:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 1 -- 172.21.15.131:0/397563313 wait complete. 2024-09-17T02:49:16.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.320+0000 7fd42cc23000 20 client.7656 trim_cache size 0 max 16384 2024-09-17T02:49:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd42cc23000 1 Processor -- start 2024-09-17T02:49:16.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd42cc23000 1 -- start start 2024-09-17T02:49:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d649890 0x56370db6c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d70daa0 0x56370db65c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d9f26d0 0x56370db66180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd42cc23000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56370db197f0 con 0x56370d9f26d0 2024-09-17T02:49:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd42cc23000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd420206650 con 0x56370d649890 2024-09-17T02:49:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd42cc23000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd420205630 con 0x56370d70daa0 2024-09-17T02:49:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd41f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d9f26d0 0x56370db66180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd41f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d9f26d0 0x56370db66180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45644/0 (socket says 172.21.15.131:45644) 2024-09-17T02:49:16.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd41f7fe640 1 -- 172.21.15.131:0/643393318 learned_addr learned my addr 172.21.15.131:0/643393318 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:16.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd41f7fe640 1 -- 172.21.15.131:0/643393318 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d70daa0 msgr2=0x56370db65c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd41f7fe640 1 --2- 172.21.15.131:0/643393318 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d70daa0 0x56370db65c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd41effd640 1 --2- 172.21.15.131:0/643393318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d649890 0x56370db6c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd41f7fe640 1 -- 172.21.15.131:0/643393318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d649890 msgr2=0x56370db6c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd41f7fe640 1 --2- 172.21.15.131:0/643393318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d649890 0x56370db6c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.340+0000 7fd41f7fe640 1 -- 172.21.15.131:0/643393318 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56370d9bd680 con 0x56370d9f26d0 2024-09-17T02:49:16.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41f7fe640 1 --2- 172.21.15.131:0/643393318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d9f26d0 0x56370db66180 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd418011fd0 tx=0x7fd41805f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9d84de1488c279 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd404ff9640 1 -- 172.21.15.131:0/643393318 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41805ef30 con 0x56370d9f26d0 2024-09-17T02:49:16.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd404ff9640 1 -- 172.21.15.131:0/643393318 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd41805fb20 con 0x56370d9f26d0 2024-09-17T02:49:16.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd404ff9640 1 -- 172.21.15.131:0/643393318 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41805e8e0 con 0x56370d9f26d0 2024-09-17T02:49:16.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 -- 172.21.15.131:0/643393318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d9f26d0 msgr2=0x56370db66180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 --2- 172.21.15.131:0/643393318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d9f26d0 0x56370db66180 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd418011fd0 tx=0x7fd41805f310 comp rx=0 tx=0).stop 2024-09-17T02:49:16.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 -- 172.21.15.131:0/643393318 shutdown_connections 2024-09-17T02:49:16.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 --2- 172.21.15.131:0/643393318 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d9f26d0 0x56370db66180 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 --2- 172.21.15.131:0/643393318 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d70daa0 0x56370db65c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 --2- 172.21.15.131:0/643393318 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d649890 0x56370db6c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 -- 172.21.15.131:0/643393318 >> 172.21.15.131:0/643393318 conn(0x56370d61d5c0 msgr2=0x56370db5dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 -- 172.21.15.131:0/643393318 shutdown_connections 2024-09-17T02:49:16.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 -- 172.21.15.131:0/643393318 wait complete. 2024-09-17T02:49:16.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 -- start start 2024-09-17T02:49:16.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 Processor -- start 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d649890 0x56370dbd41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d70daa0 0x56370dbd4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 0x56370dbd4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd420204fc0 con 0x56370d70daa0 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4200c7220 con 0x56370d649890 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56370d625680 con 0x56370d9f26d0 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd42cc23000 10 client.0 _ll_register_callbacks cb 0x7fd408003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d70daa0 0x56370dbd4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d70daa0 0x56370dbd4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45654/0 (socket says 172.21.15.131:45654) 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41e7fc640 1 -- 172.21.15.131:0/3269870893 learned_addr learned my addr 172.21.15.131:0/3269870893 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41f7fe640 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 0x56370dbd4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41effd640 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d649890 0x56370dbd41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41e7fc640 1 -- 172.21.15.131:0/3269870893 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 msgr2=0x56370dbd4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41e7fc640 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 0x56370dbd4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41e7fc640 1 -- 172.21.15.131:0/3269870893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d649890 msgr2=0x56370dbd41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41e7fc640 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d649890 0x56370dbd41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41e7fc640 1 -- 172.21.15.131:0/3269870893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56370d92a0b0 con 0x56370d70daa0 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.344+0000 7fd41f7fe640 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 0x56370dbd4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd41e7fc640 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d70daa0 0x56370dbd4710 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd414001ec0 tx=0x7fd414013e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6845273d87f321 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd3fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/3269870893 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4140042f0 con 0x56370d70daa0 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/3269870893 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7fd414004490 con 0x56370d70daa0 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 -- 172.21.15.131:0/3269870893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd41805fb20 con 0x56370d70daa0 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/3269870893 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd414036050 con 0x56370d70daa0 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 10 client.7662 fetch_fsmap learned FSMap version 77 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 -- 172.21.15.131:0/3269870893 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x56370d8f6610 con 0x56370d70daa0 2024-09-17T02:49:16.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/3269870893 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd41805fb20 con 0x56370d70daa0 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/3269870893 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7fd414039020 con 0x56370d70daa0 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 10 client.7662 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 10 client.7662 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 20 client.7662 populate_metadata read hostname 'smithi131' 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x56370d5f0990 0x56370db6a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 4 client.7662 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 -- 172.21.15.131:0/3269870893 --> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56370d96e5b0 con 0x56370d5f0990 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd41effd640 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x56370d5f0990 0x56370db6a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd3fe7fc640 10 client.7662 ms_handle_connect on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd41effd640 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x56370d5f0990 0x56370db6a270 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52d4ee91e7a307c4 server_cookie=7d7f3e26e397b53f in_seq=0 out_seq=0 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd3fe7fc640 1 -- 172.21.15.131:0/3269870893 <== mds.0 v2:172.21.15.131:6834/1957540940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+577 (crc 0 0 0) 0x56370d96e5b0 con 0x56370d5f0990 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd3fe7fc640 10 client.7662 handle_command_reply: tid=0 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 client.7662 shutdown 2024-09-17T02:49:16.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 -- 172.21.15.131:0/3269870893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d70daa0 msgr2=0x56370dbd4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d70daa0 0x56370dbd4710 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd414001ec0 tx=0x7fd414013e60 comp rx=0 tx=0).stop 2024-09-17T02:49:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 -- 172.21.15.131:0/3269870893 shutdown_connections 2024-09-17T02:49:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x56370d5f0990 0x56370db6a270 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56370d9f26d0 0x56370dbd4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd3fe7fc640 0 client.7662 ms_handle_reset on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56370d70daa0 0x56370dbd4710 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 --2- 172.21.15.131:0/3269870893 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56370d649890 0x56370dbd41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 -- 172.21.15.131:0/3269870893 >> 172.21.15.131:0/3269870893 conn(0x56370d61d5c0 msgr2=0x56370db64c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 -- 172.21.15.131:0/3269870893 shutdown_connections 2024-09-17T02:49:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 1 -- 172.21.15.131:0/3269870893 wait complete. 2024-09-17T02:49:16.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd42cc23000 20 client.7662 trim_cache size 0 max 16384 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout: "8992844c-fbc4-4c7b-b92b-66db1f2a5ed0": { 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir", 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "8992844c-fbc4-4c7b-b92b-66db1f2a5ed0", 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,repair" 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout: "8992844c-fbc4-4c7b-b92b-66db1f2a5ed0": { 2024-09-17T02:49:16.351 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:49:16.128590+0000", 2024-09-17T02:49:16.352 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir", 2024-09-17T02:49:16.352 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:49:16.352 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:49:16.352 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:49:16.352 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:49:16.352 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:16.352 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:16.352 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:49:16.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 -- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd40c073200 msgr2=0x7fd40c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd40c073200 0x7fd40c075710 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd4100ab000 tx=0x7fd410019d60 comp rx=0 tx=0).stop 2024-09-17T02:49:16.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 -- 172.21.15.131:0/3599446991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd420153e80 msgr2=0x7fd42015b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:16.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd420153e80 0x7fd42015b9a0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd42015ca20 tx=0x7fd418001a80 comp rx=0 tx=0).stop 2024-09-17T02:49:16.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 -- 172.21.15.131:0/3599446991 shutdown_connections 2024-09-17T02:49:16.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd40c073200 0x7fd40c075710 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd420153e80 0x7fd42015b9a0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd4200b7460 0x7fd42015b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 --2- 172.21.15.131:0/3599446991 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd408003a90 0x7fd420151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 -- 172.21.15.131:0/3599446991 >> 172.21.15.131:0/3599446991 conn(0x7fd4200d2100 msgr2=0x7fd420071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 -- 172.21.15.131:0/3599446991 shutdown_connections 2024-09-17T02:49:16.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:16.348+0000 7fd425958640 1 -- 172.21.15.131:0/3599446991 wait complete. 2024-09-17T02:49:16.365 INFO:tasks.cephfs.filesystem:scrub status for tag:8992844c-fbc4-4c7b-b92b-66db1f2a5ed0 - {'path': '/dir', 'tag': '8992844c-fbc4-4c7b-b92b-66db1f2a5ed0', 'options': 'recursive,repair'} 2024-09-17T02:49:45.324 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:49:45.328 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:49:46.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:46.367 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-17T02:49:46.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.540+0000 7f98b2080640 1 -- 172.21.15.131:0/44321250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9894003fe0 msgr2=0x7f9894004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.540+0000 7f98b2080640 1 --2- 172.21.15.131:0/44321250 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9894003fe0 0x7f9894004810 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f98a0009b60 tx=0x7f98a001d270 comp rx=0 tx=0).stop 2024-09-17T02:49:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.540+0000 7f98abfff640 1 -- 172.21.15.131:0/44321250 reap_dead start 2024-09-17T02:49:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.540+0000 7f98b2080640 1 -- 172.21.15.131:0/44321250 shutdown_connections 2024-09-17T02:49:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.540+0000 7f98b2080640 1 -- 172.21.15.131:0/44321250 >> 172.21.15.131:0/44321250 conn(0x7f98ac06eea0 msgr2=0x7f98ac06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.540+0000 7f98b2080640 1 -- 172.21.15.131:0/44321250 shutdown_connections 2024-09-17T02:49:46.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.540+0000 7f98b2080640 1 -- 172.21.15.131:0/44321250 wait complete. 2024-09-17T02:49:46.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.544+0000 7f98b2080640 1 Processor -- start 2024-09-17T02:49:46.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.544+0000 7f98b2080640 1 -- start start 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.544+0000 7f98b2080640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9894003fe0 0x7f98ac14a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.544+0000 7f98b2080640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98ac1f1800 0x7f98ac1f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.544+0000 7f98b2080640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac1f2150 0x7f98ac0b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.544+0000 7f98b2080640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac0cf290 con 0x7f98ac1f2150 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.544+0000 7f98b2080640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac0cf110 con 0x7f9894003fe0 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.544+0000 7f98b2080640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f98ac06dea0 con 0x7f98ac1f1800 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.544+0000 7f98abfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac1f2150 0x7f98ac0b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.544+0000 7f98abfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac1f2150 0x7f98ac0b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33428/0 (socket says 172.21.15.131:33428) 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.544+0000 7f98abfff640 1 -- 172.21.15.131:0/3984912822 learned_addr learned my addr 172.21.15.131:0/3984912822 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f98ab7fe640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9894003fe0 0x7f98ac14a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f98aaffd640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98ac1f1800 0x7f98ac1f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f98abfff640 1 -- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98ac1f1800 msgr2=0x7f98ac1f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f98abfff640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98ac1f1800 0x7f98ac1f1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f98abfff640 1 -- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9894003fe0 msgr2=0x7f98ac14a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f98abfff640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9894003fe0 0x7f98ac14a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f98abfff640 1 -- 172.21.15.131:0/3984912822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ac071480 con 0x7f98ac1f2150 2024-09-17T02:49:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f98abfff640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac1f2150 0x7f98ac0b3f40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f98ac160030 tx=0x7f98a4010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f9893fff640 1 -- 172.21.15.131:0/3984912822 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a400ec40 con 0x7f98ac1f2150 2024-09-17T02:49:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f9893fff640 1 -- 172.21.15.131:0/3984912822 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98a4013480 con 0x7f98ac1f2150 2024-09-17T02:49:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f98b2080640 1 -- 172.21.15.131:0/3984912822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ac0c4150 con 0x7f98ac1f2150 2024-09-17T02:49:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f9893fff640 1 -- 172.21.15.131:0/3984912822 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a40138b0 con 0x7f98ac1f2150 2024-09-17T02:49:46.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f98b2080640 1 -- 172.21.15.131:0/3984912822 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ac144940 con 0x7f98ac1f2150 2024-09-17T02:49:46.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.548+0000 7f98b934b000 1 Processor -- start 2024-09-17T02:49:46.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f9893fff640 1 -- 172.21.15.131:0/3984912822 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f98a4013a50 con 0x7f98ac1f2150 2024-09-17T02:49:46.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 -- start start 2024-09-17T02:49:46.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f9893fff640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9898073200 0x7f9898075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f9893fff640 1 -- 172.21.15.131:0/3984912822 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f98a4098150 con 0x7f98ac1f2150 2024-09-17T02:49:46.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98aaffd640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9898073200 0x7f9898075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3e1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3e2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3e26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fc6d27ecd0 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fc6d194330 con 0x55fc6ceca890 2024-09-17T02:49:46.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fc6d1f1bb0 con 0x55fc6d2736d0 2024-09-17T02:49:46.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98abfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3e26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98abfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3e26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33438/0 (socket says 172.21.15.131:33438) 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98abfff640 1 -- 172.21.15.131:0/588516423 learned_addr learned my addr 172.21.15.131:0/588516423 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98aaffd640 1 --2- 172.21.15.131:0/588516423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98ab7fe640 1 --2- 172.21.15.131:0/588516423 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3e1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98abfff640 1 -- 172.21.15.131:0/588516423 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 msgr2=0x55fc6d3e1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98abfff640 1 --2- 172.21.15.131:0/588516423 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3e1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98abfff640 1 -- 172.21.15.131:0/588516423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 msgr2=0x55fc6d3e2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98abfff640 1 --2- 172.21.15.131:0/588516423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3e2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98abfff640 1 -- 172.21.15.131:0/588516423 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc6d195c80 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98ab7fe640 1 --2- 172.21.15.131:0/588516423 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3e1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98aaffd640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9898073200 0x7f9898075710 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f98a00ae000 tx=0x7f98a001a010 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:46.552 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98abfff640 1 --2- 172.21.15.131:0/588516423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3e26a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f98a405ef40 tx=0x7f98a4062660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5797679ccbcaaad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98917fa640 1 -- 172.21.15.131:0/588516423 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4062e80 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98917fa640 1 -- 172.21.15.131:0/588516423 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98a4001c70 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98917fa640 1 -- 172.21.15.131:0/588516423 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a4062780 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 -- 172.21.15.131:0/588516423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 msgr2=0x55fc6d3e26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 --2- 172.21.15.131:0/588516423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3e26a0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f98a405ef40 tx=0x7f98a4062660 comp rx=0 tx=0).stop 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 -- 172.21.15.131:0/588516423 shutdown_connections 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 --2- 172.21.15.131:0/588516423 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3e26a0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 --2- 172.21.15.131:0/588516423 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3e2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 --2- 172.21.15.131:0/588516423 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3e1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 -- 172.21.15.131:0/588516423 >> 172.21.15.131:0/588516423 conn(0x55fc6ce9e5c0 msgr2=0x55fc6d39cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 -- 172.21.15.131:0/588516423 shutdown_connections 2024-09-17T02:49:46.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.552+0000 7f98b934b000 1 -- 172.21.15.131:0/588516423 wait complete. 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 -- start start 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 Processor -- start 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fc6d1e4df0 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fc6d1e0fe0 con 0x55fc6ceca890 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fc6d1864d0 con 0x55fc6d2736d0 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 10 client.0 _ll_register_callbacks cb 0x55fc6d1647f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3cca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33442/0 (socket says 172.21.15.131:33442) 2024-09-17T02:49:46.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98aaffd640 1 -- 172.21.15.131:0/1850981100 learned_addr learned my addr 172.21.15.131:0/1850981100 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98abfff640 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98ab7fe640 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98aaffd640 1 -- 172.21.15.131:0/1850981100 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 msgr2=0x55fc6d3ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98aaffd640 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98aaffd640 1 -- 172.21.15.131:0/1850981100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 msgr2=0x55fc6d3cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98aaffd640 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98aaffd640 1 -- 172.21.15.131:0/1850981100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc6d17f210 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98aaffd640 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3cca80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f98a00a6010 tx=0x7f98a00091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196d289b3555ab2f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f988a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f988a7fc640 1 -- 172.21.15.131:0/1850981100 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00c8020 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f988a7fc640 1 -- 172.21.15.131:0/1850981100 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f98a0005440 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 -- 172.21.15.131:0/1850981100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fc6d179350 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f988a7fc640 1 -- 172.21.15.131:0/1850981100 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f98a00d2050 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 10 client.7674 fetch_fsmap learned FSMap version 77 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 -- 172.21.15.131:0/1850981100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55fc6d165fc0 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f988a7fc640 1 -- 172.21.15.131:0/1850981100 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55fc6d179350 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f988a7fc640 1 -- 172.21.15.131:0/1850981100 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7f98a00d5020 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 10 client.7674 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 10 client.7674 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 20 client.7674 populate_metadata read hostname 'smithi131' 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55fc6ce83930 0x55fc6d3cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 4 client.7674 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98b934b000 1 -- 172.21.15.131:0/1850981100 --> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc6d1f3b00 con 0x55fc6ce83930 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.556+0000 7f98ab7fe640 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55fc6ce83930 0x55fc6d3cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f988a7fc640 10 client.7674 ms_handle_connect on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98ab7fe640 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55fc6ce83930 0x55fc6d3cd9d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=336c6f5a5a5437b5 server_cookie=7d1d972c4eeb40f2 in_seq=0 out_seq=0 2024-09-17T02:49:46.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f988a7fc640 1 -- 172.21.15.131:0/1850981100 <== mds.0 v2:172.21.15.131:6834/1957540940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fc6d1f3b00 con 0x55fc6ce83930 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f988a7fc640 10 client.7674 handle_command_reply: tid=0 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 1 client.7674 shutdown 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 1 -- 172.21.15.131:0/1850981100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 msgr2=0x55fc6d3cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3cca80 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f98a00a6010 tx=0x7f98a00091e0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 1 -- 172.21.15.131:0/1850981100 shutdown_connections 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55fc6ce83930 0x55fc6d3cd9d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3cca80 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f988a7fc640 0 client.7674 ms_handle_reset on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 1 --2- 172.21.15.131:0/1850981100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 1 -- 172.21.15.131:0/1850981100 >> 172.21.15.131:0/1850981100 conn(0x55fc6ce9e5c0 msgr2=0x55fc6d394d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 1 -- 172.21.15.131:0/1850981100 shutdown_connections 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 1 -- 172.21.15.131:0/1850981100 wait complete. 2024-09-17T02:49:46.561 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.560+0000 7f98b934b000 20 client.7674 trim_cache size 0 max 16384 2024-09-17T02:49:46.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 Processor -- start 2024-09-17T02:49:46.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 -- start start 2024-09-17T02:49:46.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6ceca890 0x55fc6d399460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3999a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6d2736d0 0x55fc6d3f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac1fee90 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac1fdea0 con 0x55fc6d2736d0 2024-09-17T02:49:46.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f98ac1474b0 con 0x55fc6ceca890 2024-09-17T02:49:46.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33458/0 (socket says 172.21.15.131:33458) 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98aaffd640 1 -- 172.21.15.131:0/1999750461 learned_addr learned my addr 172.21.15.131:0/1999750461 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98ab7fe640 1 --2- 172.21.15.131:0/1999750461 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6ceca890 0x55fc6d399460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98abfff640 1 --2- 172.21.15.131:0/1999750461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6d2736d0 0x55fc6d3f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98aaffd640 1 -- 172.21.15.131:0/1999750461 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6ceca890 msgr2=0x55fc6d399460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98aaffd640 1 --2- 172.21.15.131:0/1999750461 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6ceca890 0x55fc6d399460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98aaffd640 1 -- 172.21.15.131:0/1999750461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6d2736d0 msgr2=0x55fc6d3f11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98aaffd640 1 --2- 172.21.15.131:0/1999750461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6d2736d0 0x55fc6d3f11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98aaffd640 1 -- 172.21.15.131:0/1999750461 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc6d23e680 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98aaffd640 1 --2- 172.21.15.131:0/1999750461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3999a0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f98a00d0040 tx=0x7f9894005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3788027626941604 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f9890ff9640 1 -- 172.21.15.131:0/1999750461 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00db070 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f9890ff9640 1 -- 172.21.15.131:0/1999750461 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98a00c0070 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f9890ff9640 1 -- 172.21.15.131:0/1999750461 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00c8040 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 -- 172.21.15.131:0/1999750461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 msgr2=0x55fc6d3999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 --2- 172.21.15.131:0/1999750461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3999a0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f98a00d0040 tx=0x7f9894005840 comp rx=0 tx=0).stop 2024-09-17T02:49:46.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 -- 172.21.15.131:0/1999750461 shutdown_connections 2024-09-17T02:49:46.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 --2- 172.21.15.131:0/1999750461 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6d2736d0 0x55fc6d3f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 --2- 172.21.15.131:0/1999750461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3999a0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 --2- 172.21.15.131:0/1999750461 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6ceca890 0x55fc6d399460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 -- 172.21.15.131:0/1999750461 >> 172.21.15.131:0/1999750461 conn(0x55fc6ce9e5c0 msgr2=0x55fc6d397d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:46.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 -- 172.21.15.131:0/1999750461 shutdown_connections 2024-09-17T02:49:46.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.580+0000 7f98b934b000 1 -- 172.21.15.131:0/1999750461 wait complete. 2024-09-17T02:49:46.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 -- start start 2024-09-17T02:49:46.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 Processor -- start 2024-09-17T02:49:46.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3f4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fc6d1e0fe0 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fc6d1e4df0 con 0x55fc6ceca890 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fc6d39fd60 con 0x55fc6d2736d0 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 10 client.0 _ll_register_callbacks cb 0x7f98a00be030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98aaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33466/0 (socket says 172.21.15.131:33466) 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98aaffd640 1 -- 172.21.15.131:0/2195544628 learned_addr learned my addr 172.21.15.131:0/2195544628 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98abfff640 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98ab7fe640 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98aaffd640 1 -- 172.21.15.131:0/2195544628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 msgr2=0x55fc6d3f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98aaffd640 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98aaffd640 1 -- 172.21.15.131:0/2195544628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 msgr2=0x55fc6d3f4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98aaffd640 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3f4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98aaffd640 1 -- 172.21.15.131:0/2195544628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc6d1ab0b0 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98aaffd640 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3f4cb0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f98a00a6010 tx=0x7f98a00092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21516a2fcadfa5b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f988a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f988a7fc640 1 -- 172.21.15.131:0/2195544628 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00e4060 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f988a7fc640 1 -- 172.21.15.131:0/2195544628 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f98a00cd3f0 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 -- 172.21.15.131:0/2195544628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98a00c0070 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f988a7fc640 1 -- 172.21.15.131:0/2195544628 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f98a00ea050 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 10 client.7680 fetch_fsmap learned FSMap version 77 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 -- 172.21.15.131:0/2195544628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55fc6d177610 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f988a7fc640 1 -- 172.21.15.131:0/2195544628 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98a00c0070 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f988a7fc640 1 -- 172.21.15.131:0/2195544628 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7f98a00ed020 con 0x55fc6cf8eaa0 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 10 client.7680 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 10 client.7680 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 20 client.7680 populate_metadata read hostname 'smithi131' 2024-09-17T02:49:46.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55fc6ce71990 0x55fc6d3f5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 4 client.7680 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98b934b000 1 -- 172.21.15.131:0/2195544628 --> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fc6d1ef5b0 con 0x55fc6ce71990 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98ab7fe640 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55fc6ce71990 0x55fc6d3f5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f988a7fc640 10 client.7680 ms_handle_connect on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.584+0000 7f98ab7fe640 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55fc6ce71990 0x55fc6d3f5b20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ff7c8a1a95e936 server_cookie=80cb7578d7eeedfc in_seq=0 out_seq=0 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f988a7fc640 1 -- 172.21.15.131:0/2195544628 <== mds.0 v2:172.21.15.131:6834/1957540940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+377 (crc 0 0 0) 0x55fc6d1ef5b0 con 0x55fc6ce71990 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f988a7fc640 10 client.7680 handle_command_reply: tid=0 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 1 client.7680 shutdown 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 1 -- 172.21.15.131:0/2195544628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 msgr2=0x55fc6d3f4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3f4cb0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f98a00a6010 tx=0x7f98a00092d0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 1 -- 172.21.15.131:0/2195544628 shutdown_connections 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55fc6ce71990 0x55fc6d3f5b20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fc6d2736d0 0x55fc6d3f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fc6cf8eaa0 0x55fc6d3f4cb0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 1 --2- 172.21.15.131:0/2195544628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fc6ceca890 0x55fc6d3f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 1 -- 172.21.15.131:0/2195544628 >> 172.21.15.131:0/2195544628 conn(0x55fc6ce9e5c0 msgr2=0x55fc6d398700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:46.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f988a7fc640 0 client.7680 ms_handle_reset on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:46.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 1 -- 172.21.15.131:0/2195544628 shutdown_connections 2024-09-17T02:49:46.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 1 -- 172.21.15.131:0/2195544628 wait complete. 2024-09-17T02:49:46.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b934b000 20 client.7680 trim_cache size 0 max 16384 2024-09-17T02:49:46.588 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:49:46.588 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-09-17T02:49:46.588 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {}, 2024-09-17T02:49:46.588 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:49:46.589 INFO:teuthology.orchestra.run.smithi131.stdout: "8992844c-fbc4-4c7b-b92b-66db1f2a5ed0": { 2024-09-17T02:49:46.589 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:49:16.128590+0000", 2024-09-17T02:49:46.589 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/dir", 2024-09-17T02:49:46.589 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:49:46.589 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:49:46.589 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:49:46.589 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:49:46.589 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:46.589 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:46.589 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:49:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 -- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9898073200 msgr2=0x7f9898075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9898073200 0x7f9898075710 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f98a00ae000 tx=0x7f98a001a010 comp rx=0 tx=0).stop 2024-09-17T02:49:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 -- 172.21.15.131:0/3984912822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac1f2150 msgr2=0x7f98ac0b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac1f2150 0x7f98ac0b3f40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f98ac160030 tx=0x7f98a4010db0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 -- 172.21.15.131:0/3984912822 shutdown_connections 2024-09-17T02:49:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9898073200 0x7f9898075710 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98ac1f2150 0x7f98ac0b3f40 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98ac1f1800 0x7f98ac1f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 --2- 172.21.15.131:0/3984912822 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9894003fe0 0x7f98ac14a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 -- 172.21.15.131:0/3984912822 >> 172.21.15.131:0/3984912822 conn(0x7f98ac06eea0 msgr2=0x7f98ac15cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 -- 172.21.15.131:0/3984912822 shutdown_connections 2024-09-17T02:49:46.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.588+0000 7f98b2080640 1 -- 172.21.15.131:0/3984912822 wait complete. 2024-09-17T02:49:46.603 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-17T02:49:46.603 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:49:46.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.752+0000 7f867f7e8640 1 -- 172.21.15.131:0/2022314651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f867815c740 msgr2=0x7f86780d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.752+0000 7f867f7e8640 1 --2- 172.21.15.131:0/2022314651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f867815c740 0x7f86780d0130 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f86780cd850 tx=0x7f866801c890 comp rx=0 tx=0).stop 2024-09-17T02:49:46.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.752+0000 7f867f7e8640 1 -- 172.21.15.131:0/2022314651 shutdown_connections 2024-09-17T02:49:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.752+0000 7f867f7e8640 1 --2- 172.21.15.131:0/2022314651 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f86780706c0 0x7f8678070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.752+0000 7f867f7e8640 1 --2- 172.21.15.131:0/2022314651 >> v2:172.21.15.189:3300/0 conn(0x7f86780cca90 0x7f86780ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.752+0000 7f867f7e8640 1 --2- 172.21.15.131:0/2022314651 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f867815c740 0x7f86780d0130 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.752+0000 7f867f7e8640 1 -- 172.21.15.131:0/2022314651 >> 172.21.15.131:0/2022314651 conn(0x7f86780cf0b0 msgr2=0x7f86780cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.752+0000 7f867f7e8640 1 -- 172.21.15.131:0/2022314651 shutdown_connections 2024-09-17T02:49:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.752+0000 7f867f7e8640 1 -- 172.21.15.131:0/2022314651 wait complete. 2024-09-17T02:49:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867f7e8640 1 Processor -- start 2024-09-17T02:49:46.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867f7e8640 1 -- start start 2024-09-17T02:49:46.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867f7e8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86780706c0 0x7f86780cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867f7e8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86780cca90 0x7f8678157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867f7e8640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f867815c740 0x7f8678150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867f7e8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f86780b65d0 con 0x7f86780706c0 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867f7e8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f86780b6450 con 0x7f86780cca90 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867f7e8640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f86780b6750 con 0x7f867815c740 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867d584640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86780706c0 0x7f86780cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867d584640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86780706c0 0x7f86780cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33482/0 (socket says 172.21.15.131:33482) 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867d584640 1 -- 172.21.15.131:0/2489693593 learned_addr learned my addr 172.21.15.131:0/2489693593 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867cd83640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86780cca90 0x7f8678157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867dd85640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f867815c740 0x7f8678150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867d584640 1 -- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f867815c740 msgr2=0x7f8678150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867d584640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f867815c740 0x7f8678150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867d584640 1 -- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86780cca90 msgr2=0x7f8678157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867d584640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86780cca90 0x7f8678157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867d584640 1 -- 172.21.15.131:0/2489693593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86781fa270 con 0x7f86780706c0 2024-09-17T02:49:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867d584640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86780706c0 0x7f86780cb580 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f86680b0040 tx=0x7f86680a5870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f866dffb640 1 -- 172.21.15.131:0/2489693593 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680b2040 con 0x7f86780706c0 2024-09-17T02:49:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f866dffb640 1 -- 172.21.15.131:0/2489693593 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86680044b0 con 0x7f86780706c0 2024-09-17T02:49:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867f7e8640 1 -- 172.21.15.131:0/2489693593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86781fa550 con 0x7f86780706c0 2024-09-17T02:49:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f866dffb640 1 -- 172.21.15.131:0/2489693593 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668005290 con 0x7f86780706c0 2024-09-17T02:49:46.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.756+0000 7f867f7e8640 1 -- 172.21.15.131:0/2489693593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86781580a0 con 0x7f86780706c0 2024-09-17T02:49:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.760+0000 7f867f7e8640 1 -- 172.21.15.131:0/2489693593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86781faa70 con 0x7f86780706c0 2024-09-17T02:49:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.760+0000 7f866dffb640 1 -- 172.21.15.131:0/2489693593 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8668005430 con 0x7f86780706c0 2024-09-17T02:49:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.760+0000 7f866dffb640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f865c073200 0x7f865c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.760+0000 7f866dffb640 1 -- 172.21.15.131:0/2489693593 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f866812e490 con 0x7f86780706c0 2024-09-17T02:49:46.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.760+0000 7f867cd83640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f865c073200 0x7f865c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:46.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.760+0000 7f867cd83640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f865c073200 0x7f865c075710 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f86740058d0 tx=0x7f8674005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:46.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.764+0000 7f866dffb640 1 -- 172.21.15.131:0/2489693593 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f86781faa70 con 0x7f86780706c0 2024-09-17T02:49:46.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.960+0000 7f867f7e8640 1 -- 172.21.15.131:0/2489693593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86781f6580 con 0x7f86780706c0 2024-09-17T02:49:46.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.960+0000 7f866dffb640 1 -- 172.21.15.131:0/2489693593 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5238 (secure 0 0 0) 0x7f86680be060 con 0x7f86780706c0 2024-09-17T02:49:46.958 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:46.958 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":77,"btime":"2024-09-17T02:49:11:530527+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:49:11.530508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/1957540940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1957540940},{"type":"v1","addr":"172.21.15.131:6835","nonce":1957540940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7521,"qdb_cluster":[7521]},"id":6}]} 2024-09-17T02:49:46.958 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 77 2024-09-17T02:49:46.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 -- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f865c073200 msgr2=0x7f865c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f865c073200 0x7f865c075710 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f86740058d0 tx=0x7f8674005860 comp rx=0 tx=0).stop 2024-09-17T02:49:46.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 -- 172.21.15.131:0/2489693593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86780706c0 msgr2=0x7f86780cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:46.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86780706c0 0x7f86780cb580 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f86680b0040 tx=0x7f86680a5870 comp rx=0 tx=0).stop 2024-09-17T02:49:46.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 -- 172.21.15.131:0/2489693593 shutdown_connections 2024-09-17T02:49:46.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f865c073200 0x7f865c075710 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f867815c740 0x7f8678150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f86780cca90 0x7f8678157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 --2- 172.21.15.131:0/2489693593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f86780706c0 0x7f86780cb580 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:46.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 -- 172.21.15.131:0/2489693593 >> 172.21.15.131:0/2489693593 conn(0x7f86780cf0b0 msgr2=0x7f867815adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:46.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 -- 172.21.15.131:0/2489693593 shutdown_connections 2024-09-17T02:49:46.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:46.964+0000 7f867f7e8640 1 -- 172.21.15.131:0/2489693593 wait complete. 2024-09-17T02:49:46.978 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-09-17T02:49:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.124+0000 7fed62ba5640 1 -- 172.21.15.131:0/2727992090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5c06e5b0 msgr2=0x7fed5c15bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.124+0000 7fed62ba5640 1 --2- 172.21.15.131:0/2727992090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5c06e5b0 0x7fed5c15bea0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fed50009840 tx=0x7fed5001c820 comp rx=0 tx=0).stop 2024-09-17T02:49:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.124+0000 7fed62ba5640 1 -- 172.21.15.131:0/2727992090 shutdown_connections 2024-09-17T02:49:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.124+0000 7fed62ba5640 1 --2- 172.21.15.131:0/2727992090 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5c06e5b0 0x7fed5c15bea0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.124+0000 7fed62ba5640 1 -- 172.21.15.131:0/2727992090 >> 172.21.15.131:0/2727992090 conn(0x7fed5c06eea0 msgr2=0x7fed5c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.124+0000 7fed62ba5640 1 -- 172.21.15.131:0/2727992090 shutdown_connections 2024-09-17T02:49:47.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.124+0000 7fed62ba5640 1 -- 172.21.15.131:0/2727992090 wait complete. 2024-09-17T02:49:47.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.124+0000 7fed62ba5640 1 Processor -- start 2024-09-17T02:49:47.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.124+0000 7fed62ba5640 1 -- start start 2024-09-17T02:49:47.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed62ba5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fed5c0ced00 0x7fed5c0b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed62ba5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5c15c3e0 0x7fed5c14ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed62ba5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c14f820 0x7fed5c154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed62ba5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c169e90 con 0x7fed5c15c3e0 2024-09-17T02:49:47.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed62ba5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c169d10 con 0x7fed5c14f820 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed62ba5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fed5c16a010 con 0x7fed5c0ced00 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed5bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5c15c3e0 0x7fed5c14ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed5bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5c15c3e0 0x7fed5c14ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33500/0 (socket says 172.21.15.131:33500) 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed5bfff640 1 -- 172.21.15.131:0/3958104669 learned_addr learned my addr 172.21.15.131:0/3958104669 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed61142640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c14f820 0x7fed5c154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed60941640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fed5c0ced00 0x7fed5c0b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed5bfff640 1 -- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fed5c0ced00 msgr2=0x7fed5c0b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed5bfff640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fed5c0ced00 0x7fed5c0b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed5bfff640 1 -- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c14f820 msgr2=0x7fed5c154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed5bfff640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c14f820 0x7fed5c154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed5bfff640 1 -- 172.21.15.131:0/3958104669 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c0ae4f0 con 0x7fed5c15c3e0 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed5bfff640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5c15c3e0 0x7fed5c14ef30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fed5001aed0 tx=0x7fed50004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed597fa640 1 -- 172.21.15.131:0/3958104669 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed500be070 con 0x7fed5c15c3e0 2024-09-17T02:49:47.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed597fa640 1 -- 172.21.15.131:0/3958104669 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed50005070 con 0x7fed5c15c3e0 2024-09-17T02:49:47.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed62ba5640 1 -- 172.21.15.131:0/3958104669 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c0ae7d0 con 0x7fed5c15c3e0 2024-09-17T02:49:47.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed597fa640 1 -- 172.21.15.131:0/3958104669 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50005480 con 0x7fed5c15c3e0 2024-09-17T02:49:47.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed62ba5640 1 -- 172.21.15.131:0/3958104669 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c0aed90 con 0x7fed5c15c3e0 2024-09-17T02:49:47.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.128+0000 7fed597fa640 1 -- 172.21.15.131:0/3958104669 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fed500046d0 con 0x7fed5c15c3e0 2024-09-17T02:49:47.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed597fa640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fed48073230 0x7fed48075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed597fa640 1 -- 172.21.15.131:0/3958104669 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7fed5012ec90 con 0x7fed5c15c3e0 2024-09-17T02:49:47.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed60941640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fed48073230 0x7fed48075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 Processor -- start 2024-09-17T02:49:47.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 -- start start 2024-09-17T02:49:47.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed60941640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fed48073230 0x7fed48075740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fed4c000f70 tx=0x7fed4c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:47.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b635fd22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b635fd2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b635fd2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b635e70d00 con 0x55b635e8cb60 2024-09-17T02:49:47.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b635de4c90 con 0x55b635e821c0 2024-09-17T02:49:47.130 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b635dd40c0 con 0x55b635e6bd90 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed60941640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b635fd22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed60941640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b635fd22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33508/0 (socket says 172.21.15.131:33508) 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed60941640 1 -- 172.21.15.131:0/2435397296 learned_addr learned my addr 172.21.15.131:0/2435397296 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed5bfff640 1 --2- 172.21.15.131:0/2435397296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b635fd2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed61142640 1 --2- 172.21.15.131:0/2435397296 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b635fd2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed60941640 1 -- 172.21.15.131:0/2435397296 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 msgr2=0x55b635fd2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed60941640 1 --2- 172.21.15.131:0/2435397296 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b635fd2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed60941640 1 -- 172.21.15.131:0/2435397296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 msgr2=0x55b635fd2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed60941640 1 --2- 172.21.15.131:0/2435397296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b635fd2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed60941640 1 -- 172.21.15.131:0/2435397296 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b635daf290 con 0x55b635e8cb60 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed61142640 1 --2- 172.21.15.131:0/2435397296 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b635fd2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed60941640 1 --2- 172.21.15.131:0/2435397296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b635fd22e0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fed4c005460 tx=0x7fed4c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4415f970ed3b354 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed427fc640 1 -- 172.21.15.131:0/2435397296 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c00d8b0 con 0x55b635e8cb60 2024-09-17T02:49:47.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed427fc640 1 -- 172.21.15.131:0/2435397296 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed4c00da50 con 0x55b635e8cb60 2024-09-17T02:49:47.132 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed427fc640 1 -- 172.21.15.131:0/2435397296 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c00de40 con 0x55b635e8cb60 2024-09-17T02:49:47.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 -- 172.21.15.131:0/2435397296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 msgr2=0x55b635fd22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 --2- 172.21.15.131:0/2435397296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b635fd22e0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fed4c005460 tx=0x7fed4c010000 comp rx=0 tx=0).stop 2024-09-17T02:49:47.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 -- 172.21.15.131:0/2435397296 shutdown_connections 2024-09-17T02:49:47.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 --2- 172.21.15.131:0/2435397296 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b635fd2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 --2- 172.21.15.131:0/2435397296 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b635fd2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 --2- 172.21.15.131:0/2435397296 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b635fd22e0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.132+0000 7fed69e70000 1 -- 172.21.15.131:0/2435397296 >> 172.21.15.131:0/2435397296 conn(0x55b635a91760 msgr2=0x55b635fb5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:47.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 -- 172.21.15.131:0/2435397296 shutdown_connections 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 -- 172.21.15.131:0/2435397296 wait complete. 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 -- start start 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 Processor -- start 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e6bd90 0x55b635f8e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e821c0 0x55b635f8e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e8cb60 0x55b635fb48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b635d6e940 con 0x55b635e821c0 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b635dd7ed0 con 0x55b635e6bd90 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b635a99820 con 0x55b635e8cb60 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 10 client.0 _ll_register_callbacks cb 0x55b635db27f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed5bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e821c0 0x55b635f8e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed5bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e821c0 0x55b635f8e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33524/0 (socket says 172.21.15.131:33524) 2024-09-17T02:49:47.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed5bfff640 1 -- 172.21.15.131:0/2636584759 learned_addr learned my addr 172.21.15.131:0/2636584759 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed60941640 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e6bd90 0x55b635f8e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed61142640 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e8cb60 0x55b635fb48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed5bfff640 1 -- 172.21.15.131:0/2636584759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e8cb60 msgr2=0x55b635fb48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed5bfff640 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e8cb60 0x55b635fb48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed5bfff640 1 -- 172.21.15.131:0/2636584759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e6bd90 msgr2=0x55b635f8e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed5bfff640 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e6bd90 0x55b635f8e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed5bfff640 1 -- 172.21.15.131:0/2636584759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b635d87550 con 0x55b635e821c0 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed61142640 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e8cb60 0x55b635fb48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed5bfff640 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e821c0 0x55b635f8e5c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fed500b7000 tx=0x7fed500f9930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9642d526a266b64d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2636584759 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed500f8e70 con 0x55b635e821c0 2024-09-17T02:49:47.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2636584759 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7fed500f4cb0 con 0x55b635e821c0 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 -- 172.21.15.131:0/2636584759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b635d6c400 con 0x55b635e821c0 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2636584759 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fed500f9a60 con 0x55b635e821c0 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 10 client.7701 fetch_fsmap learned FSMap version 77 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 -- 172.21.15.131:0/2636584759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55b635d72480 con 0x55b635e821c0 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2636584759 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55b635d6c400 con 0x55b635e821c0 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2636584759 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7fed5012e030 con 0x55b635e821c0 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 10 client.7701 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 10 client.7701 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 20 client.7701 populate_metadata read hostname 'smithi131' 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b635a76ad0 0x55b635fe38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 4 client.7701 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed69e70000 1 -- 172.21.15.131:0/2636584759 --> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b635de6be0 con 0x55b635a76ad0 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.136+0000 7fed60941640 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b635a76ad0 0x55b635fe38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed3b7fe640 10 client.7701 ms_handle_connect on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:47.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed60941640 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b635a76ad0 0x55b635fe38a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=791638a85f6c5ea8 server_cookie=bc5b4f01abd5aafa in_seq=0 out_seq=0 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2636584759 <== mds.0 v2:172.21.15.131:6834/1957540940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b635de6be0 con 0x55b635a76ad0 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed3b7fe640 10 client.7701 handle_command_reply: tid=0 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 1 client.7701 shutdown 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 1 -- 172.21.15.131:0/2636584759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e821c0 msgr2=0x55b635f8e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e821c0 0x55b635f8e5c0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fed500b7000 tx=0x7fed500f9930 comp rx=0 tx=0).stop 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 1 -- 172.21.15.131:0/2636584759 shutdown_connections 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b635a76ad0 0x55b635fe38a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e8cb60 0x55b635fb48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e821c0 0x55b635f8e5c0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 1 --2- 172.21.15.131:0/2636584759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e6bd90 0x55b635f8e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed3b7fe640 0 client.7701 ms_handle_reset on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:47.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 1 -- 172.21.15.131:0/2636584759 >> 172.21.15.131:0/2636584759 conn(0x55b635a91760 msgr2=0x55b635f86680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:47.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 1 -- 172.21.15.131:0/2636584759 shutdown_connections 2024-09-17T02:49:47.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 1 -- 172.21.15.131:0/2636584759 wait complete. 2024-09-17T02:49:47.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.140+0000 7fed69e70000 20 client.7701 trim_cache size 0 max 16384 2024-09-17T02:49:47.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.152+0000 7fed69e70000 1 Processor -- start 2024-09-17T02:49:47.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.152+0000 7fed69e70000 1 -- start start 2024-09-17T02:49:47.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.152+0000 7fed69e70000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b6360435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.152+0000 7fed69e70000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b636043b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.152+0000 7fed69e70000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b636044040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.152+0000 7fed69e70000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b635dd71e0 con 0x55b635e8cb60 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.152+0000 7fed69e70000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b635de3a60 con 0x55b635e821c0 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.152+0000 7fed69e70000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b635da0300 con 0x55b635e6bd90 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed61142640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b636044040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed61142640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b636044040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33534/0 (socket says 172.21.15.131:33534) 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed61142640 1 -- 172.21.15.131:0/1685940238 learned_addr learned my addr 172.21.15.131:0/1685940238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed60941640 1 --2- 172.21.15.131:0/1685940238 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b6360435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed5bfff640 1 --2- 172.21.15.131:0/1685940238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b636043b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed61142640 1 -- 172.21.15.131:0/1685940238 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 msgr2=0x55b6360435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed61142640 1 --2- 172.21.15.131:0/1685940238 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b6360435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed61142640 1 -- 172.21.15.131:0/1685940238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 msgr2=0x55b636043b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:47.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed61142640 1 --2- 172.21.15.131:0/1685940238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b636043b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed61142640 1 -- 172.21.15.131:0/1685940238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b635e59d10 con 0x55b635e8cb60 2024-09-17T02:49:47.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed61142640 1 --2- 172.21.15.131:0/1685940238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b636044040 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fed5400af80 tx=0x7fed5400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca33cb5d740d016d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:47.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed41ffb640 1 -- 172.21.15.131:0/1685940238 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54003980 con 0x55b635e8cb60 2024-09-17T02:49:47.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed41ffb640 1 -- 172.21.15.131:0/1685940238 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed54003b40 con 0x55b635e8cb60 2024-09-17T02:49:47.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed41ffb640 1 -- 172.21.15.131:0/1685940238 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed540096a0 con 0x55b635e8cb60 2024-09-17T02:49:47.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 -- 172.21.15.131:0/1685940238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 msgr2=0x55b636044040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 --2- 172.21.15.131:0/1685940238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b636044040 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fed5400af80 tx=0x7fed5400b8e0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 -- 172.21.15.131:0/1685940238 shutdown_connections 2024-09-17T02:49:47.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 --2- 172.21.15.131:0/1685940238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b636044040 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 --2- 172.21.15.131:0/1685940238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b636043b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 --2- 172.21.15.131:0/1685940238 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b6360435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 -- 172.21.15.131:0/1685940238 >> 172.21.15.131:0/1685940238 conn(0x55b635a91760 msgr2=0x55b635f88690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:47.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 -- 172.21.15.131:0/1685940238 shutdown_connections 2024-09-17T02:49:47.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 -- 172.21.15.131:0/1685940238 wait complete. 2024-09-17T02:49:47.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 -- start start 2024-09-17T02:49:47.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 Processor -- start 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b636047a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b636047f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b6360484a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55b635ddc500 con 0x55b635e8cb60 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55b635d7f300 con 0x55b635e821c0 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55b635d91670 con 0x55b635e6bd90 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed69e70000 10 client.0 _ll_register_callbacks cb 0x7fed500045e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.156+0000 7fed61142640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b6360484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed61142640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b6360484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33548/0 (socket says 172.21.15.131:33548) 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed61142640 1 -- 172.21.15.131:0/2581123029 learned_addr learned my addr 172.21.15.131:0/2581123029 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed60941640 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b636047a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed5bfff640 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b636047f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed61142640 1 -- 172.21.15.131:0/2581123029 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 msgr2=0x55b636047a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed61142640 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b636047a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed61142640 1 -- 172.21.15.131:0/2581123029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 msgr2=0x55b636047f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed61142640 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b636047f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed61142640 1 -- 172.21.15.131:0/2581123029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b635d8ae50 con 0x55b635e8cb60 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed60941640 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b636047a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed61142640 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b6360484a0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fed54017040 tx=0x7fed54004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0eb5f4eb5d20e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed3b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2581123029 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54024020 con 0x55b635e8cb60 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2581123029 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7fed54009c40 con 0x55b635e8cb60 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 -- 172.21.15.131:0/2581123029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed54003b40 con 0x55b635e8cb60 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2581123029 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fed54009e80 con 0x55b635e8cb60 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 10 client.7707 fetch_fsmap learned FSMap version 77 2024-09-17T02:49:47.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 -- 172.21.15.131:0/2581123029 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) -- 0x55b635d792d0 con 0x55b635e8cb60 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2581123029 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed54003b40 con 0x55b635e8cb60 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2581123029 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 77) ==== 3502+0+0 (secure 0 0 0) 0x7fed540294f0 con 0x55b635e8cb60 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 10 client.7707 fetch_fsmap finished waiting for FSMap version 77 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 10 client.7707 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 20 client.7707 populate_metadata read hostname 'smithi131' 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b635a64b30 0x55b635fe5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 4 client.7707 mds_command: new command op to 7521 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 -- 172.21.15.131:0/2581123029 --> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55b635de2690 con 0x55b635a64b30 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed60941640 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b635a64b30 0x55b635fe5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed3b7fe640 10 client.7707 ms_handle_connect on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed60941640 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b635a64b30 0x55b635fe5ad0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fa458b79ee9ea6d server_cookie=e1ac83fc836f29f in_seq=0 out_seq=0 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed3b7fe640 1 -- 172.21.15.131:0/2581123029 <== mds.0 v2:172.21.15.131:6834/1957540940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b635de2690 con 0x55b635a64b30 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed3b7fe640 10 client.7707 handle_command_reply: tid=0 2024-09-17T02:49:47.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 client.7707 shutdown 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 -- 172.21.15.131:0/2581123029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 msgr2=0x55b6360484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b6360484a0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fed54017040 tx=0x7fed54004a80 comp rx=0 tx=0).stop 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 -- 172.21.15.131:0/2581123029 shutdown_connections 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.131:6834/1957540940,v1:172.21.15.131:6835/1957540940] conn(0x55b635a64b30 0x55b635fe5ad0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55b635e8cb60 0x55b6360484a0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55b635e821c0 0x55b636047f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed3b7fe640 0 client.7707 ms_handle_reset on v2:172.21.15.131:6834/1957540940 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 --2- 172.21.15.131:0/2581123029 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55b635e6bd90 0x55b636047a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 -- 172.21.15.131:0/2581123029 >> 172.21.15.131:0/2581123029 conn(0x55b635a91760 msgr2=0x55b635fdece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 -- 172.21.15.131:0/2581123029 shutdown_connections 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 1 -- 172.21.15.131:0/2581123029 wait complete. 2024-09-17T02:49:47.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed69e70000 20 client.7707 trim_cache size 0 max 16384 2024-09-17T02:49:47.163 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:47.163 INFO:teuthology.orchestra.run.smithi131.stdout:[] 2024-09-17T02:49:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 -- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fed48073230 msgr2=0x7fed48075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fed48073230 0x7fed48075740 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fed4c000f70 tx=0x7fed4c009040 comp rx=0 tx=0).stop 2024-09-17T02:49:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 -- 172.21.15.131:0/3958104669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5c15c3e0 msgr2=0x7fed5c14ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5c15c3e0 0x7fed5c14ef30 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fed5001aed0 tx=0x7fed50004040 comp rx=0 tx=0).stop 2024-09-17T02:49:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 -- 172.21.15.131:0/3958104669 shutdown_connections 2024-09-17T02:49:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fed48073230 0x7fed48075740 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fed5c14f820 0x7fed5c154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fed5c15c3e0 0x7fed5c14ef30 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 --2- 172.21.15.131:0/3958104669 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fed5c0ced00 0x7fed5c0b5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 -- 172.21.15.131:0/3958104669 >> 172.21.15.131:0/3958104669 conn(0x7fed5c06eea0 msgr2=0x7fed5c0743e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:47.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 -- 172.21.15.131:0/3958104669 shutdown_connections 2024-09-17T02:49:47.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.160+0000 7fed62ba5640 1 -- 172.21.15.131:0/3958104669 wait complete. 2024-09-17T02:49:47.175 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T02:49:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3993090137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c155240 msgr2=0x7fc69c073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 --2- 172.21.15.131:0/3993090137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c155240 0x7fc69c073430 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fc69c06a1c0 tx=0x7fc69001c670 comp rx=0 tx=0).stop 2024-09-17T02:49:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3993090137 shutdown_connections 2024-09-17T02:49:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 --2- 172.21.15.131:0/3993090137 >> v2:172.21.15.189:3300/0 conn(0x7fc69c073970 0x7fc69c06dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 --2- 172.21.15.131:0/3993090137 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c155240 0x7fc69c073430 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3993090137 >> 172.21.15.131:0/3993090137 conn(0x7fc69c06eea0 msgr2=0x7fc69c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3993090137 shutdown_connections 2024-09-17T02:49:47.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3993090137 wait complete. 2024-09-17T02:49:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 Processor -- start 2024-09-17T02:49:47.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 -- start start 2024-09-17T02:49:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c06e3e0 0x7fc69c0b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc69c073970 0x7fc69c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc69c155240 0x7fc69c0ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c1685b0 con 0x7fc69c06e3e0 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c168430 con 0x7fc69c073970 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc6a1a29640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc69c168730 con 0x7fc69c155240 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc69affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c06e3e0 0x7fc69c0b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc69affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c06e3e0 0x7fc69c0b57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33572/0 (socket says 172.21.15.131:33572) 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc69affd640 1 -- 172.21.15.131:0/3068686728 learned_addr learned my addr 172.21.15.131:0/3068686728 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc69a7fc640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc69c073970 0x7fc69c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc69b7fe640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc69c155240 0x7fc69c0ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc69affd640 1 -- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc69c155240 msgr2=0x7fc69c0ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc69affd640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc69c155240 0x7fc69c0ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc69affd640 1 -- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc69c073970 msgr2=0x7fc69c0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc69affd640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc69c073970 0x7fc69c0cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc69affd640 1 -- 172.21.15.131:0/3068686728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c0b2ef0 con 0x7fc69c06e3e0 2024-09-17T02:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.324+0000 7fc69b7fe640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc69c155240 0x7fc69c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.328+0000 7fc69affd640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c06e3e0 0x7fc69c0b57b0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fc690009850 tx=0x7fc6900a6410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.328+0000 7fc683fff640 1 -- 172.21.15.131:0/3068686728 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6900ac070 con 0x7fc69c06e3e0 2024-09-17T02:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.328+0000 7fc683fff640 1 -- 172.21.15.131:0/3068686728 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc690004320 con 0x7fc69c06e3e0 2024-09-17T02:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.328+0000 7fc683fff640 1 -- 172.21.15.131:0/3068686728 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6900a6a80 con 0x7fc69c06e3e0 2024-09-17T02:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.328+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3068686728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c155e10 con 0x7fc69c06e3e0 2024-09-17T02:49:47.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.328+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3068686728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c0b3470 con 0x7fc69c06e3e0 2024-09-17T02:49:47.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.328+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3068686728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69c06dea0 con 0x7fc69c06e3e0 2024-09-17T02:49:47.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.328+0000 7fc683fff640 1 -- 172.21.15.131:0/3068686728 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc6900af070 con 0x7fc69c06e3e0 2024-09-17T02:49:47.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.328+0000 7fc683fff640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc688073200 0x7fc688075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.328+0000 7fc683fff640 1 -- 172.21.15.131:0/3068686728 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7fc69012e3b0 con 0x7fc69c06e3e0 2024-09-17T02:49:47.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.328+0000 7fc69a7fc640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc688073200 0x7fc688075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.332+0000 7fc69a7fc640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc688073200 0x7fc688075710 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0058d0 tx=0x7fc68c005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:47.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.332+0000 7fc683fff640 1 -- 172.21.15.131:0/3068686728 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc69c06e7f0 con 0x7fc69c06e3e0 2024-09-17T02:49:47.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.528+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3068686728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc69c06e9d0 con 0x7fc69c06e3e0 2024-09-17T02:49:47.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.528+0000 7fc683fff640 1 -- 172.21.15.131:0/3068686728 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc6900f9200 con 0x7fc69c06e3e0 2024-09-17T02:49:47.525 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:47.526 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T02:49:47.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc688073200 msgr2=0x7fc688075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc688073200 0x7fc688075710 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0058d0 tx=0x7fc68c005860 comp rx=0 tx=0).stop 2024-09-17T02:49:47.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3068686728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c06e3e0 msgr2=0x7fc69c0b57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c06e3e0 0x7fc69c0b57b0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fc690009850 tx=0x7fc6900a6410 comp rx=0 tx=0).stop 2024-09-17T02:49:47.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3068686728 shutdown_connections 2024-09-17T02:49:47.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc688073200 0x7fc688075710 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc69c155240 0x7fc69c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc69c073970 0x7fc69c0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 --2- 172.21.15.131:0/3068686728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc69c06e3e0 0x7fc69c0b57b0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3068686728 >> 172.21.15.131:0/3068686728 conn(0x7fc69c06eea0 msgr2=0x7fc69c0ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:47.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3068686728 shutdown_connections 2024-09-17T02:49:47.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.532+0000 7fc6a1a29640 1 -- 172.21.15.131:0/3068686728 wait complete. 2024-09-17T02:49:47.546 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-17T02:49:47.546 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:49:47.546 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:49:47.621 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:49:47.691 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:49:47.691 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:49:47.691 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:49:47.691 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:49:47.697 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:49:47.697 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:49:47.745 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:49:47.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.948+0000 7f25fed15640 1 -- 172.21.15.131:0/3636068588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25f815c3e0 msgr2=0x7f25f80cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.946 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.948+0000 7f25fed15640 1 --2- 172.21.15.131:0/3636068588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25f815c3e0 0x7f25f80cbaf0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009c80 tx=0x7f25f001cda0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.948+0000 7f25fed15640 1 -- 172.21.15.131:0/3636068588 shutdown_connections 2024-09-17T02:49:47.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.948+0000 7f25fed15640 1 --2- 172.21.15.131:0/3636068588 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25f815c3e0 0x7f25f80cbaf0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.948+0000 7f25fed15640 1 --2- 172.21.15.131:0/3636068588 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25f806e5b0 0x7f25f815bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.948+0000 7f25fed15640 1 -- 172.21.15.131:0/3636068588 >> 172.21.15.131:0/3636068588 conn(0x7f25f806eea0 msgr2=0x7f25f806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:47.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.948+0000 7f25fed15640 1 -- 172.21.15.131:0/3636068588 shutdown_connections 2024-09-17T02:49:47.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.948+0000 7f25fed15640 1 -- 172.21.15.131:0/3636068588 wait complete. 2024-09-17T02:49:47.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fed15640 1 Processor -- start 2024-09-17T02:49:47.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fed15640 1 -- start start 2024-09-17T02:49:47.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fed15640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25f80ced00 0x7f25f814b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fed15640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25f815c3e0 0x7f25f8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fed15640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25f814d6f0 0x7f25f81507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fed15640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25f8168860 con 0x7f25f814d6f0 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fed15640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f25f81686e0 con 0x7f25f815c3e0 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fed15640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f25f81689e0 con 0x7f25f80ced00 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fd2b2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25f814d6f0 0x7f25f81507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fd2b2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25f814d6f0 0x7f25f81507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33592/0 (socket says 172.21.15.131:33592) 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fd2b2640 1 -- 172.21.15.131:0/1157188675 learned_addr learned my addr 172.21.15.131:0/1157188675 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fcab1640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25f80ced00 0x7f25f814b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25f7fff640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25f815c3e0 0x7f25f8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fd2b2640 1 -- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25f80ced00 msgr2=0x7f25f814b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fd2b2640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25f80ced00 0x7f25f814b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fd2b2640 1 -- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25f815c3e0 msgr2=0x7f25f8150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fd2b2640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25f815c3e0 0x7f25f8150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:47.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.952+0000 7f25fd2b2640 1 -- 172.21.15.131:0/1157188675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f814de60 con 0x7f25f814d6f0 2024-09-17T02:49:47.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.956+0000 7f25fd2b2640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25f814d6f0 0x7f25f81507c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009bd0 tx=0x7f25f00093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:47.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.956+0000 7f25f57fa640 1 -- 172.21.15.131:0/1157188675 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f00bc070 con 0x7f25f814d6f0 2024-09-17T02:49:47.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.956+0000 7f25f57fa640 1 -- 172.21.15.131:0/1157188675 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25f00042f0 con 0x7f25f814d6f0 2024-09-17T02:49:47.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.956+0000 7f25fed15640 1 -- 172.21.15.131:0/1157188675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f81f82c0 con 0x7f25f814d6f0 2024-09-17T02:49:47.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.956+0000 7f25f57fa640 1 -- 172.21.15.131:0/1157188675 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f00b2040 con 0x7f25f814d6f0 2024-09-17T02:49:47.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.956+0000 7f25fed15640 1 -- 172.21.15.131:0/1157188675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f81f8860 con 0x7f25f814d6f0 2024-09-17T02:49:47.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.956+0000 7f25fed15640 1 -- 172.21.15.131:0/1157188675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f814f6c0 con 0x7f25f814d6f0 2024-09-17T02:49:47.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.956+0000 7f25f57fa640 1 -- 172.21.15.131:0/1157188675 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f25f00bf070 con 0x7f25f814d6f0 2024-09-17T02:49:47.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.956+0000 7f25f57fa640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25e4073200 0x7f25e4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:47.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.956+0000 7f25f57fa640 1 -- 172.21.15.131:0/1157188675 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f25f012e3d0 con 0x7f25f814d6f0 2024-09-17T02:49:47.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.956+0000 7f25fcab1640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25e4073200 0x7f25e4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:47.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.960+0000 7f25fcab1640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25e4073200 0x7f25e4075710 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f25e800b810 tx=0x7f25e8008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:47.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:47.960+0000 7f25f57fa640 1 -- 172.21.15.131:0/1157188675 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f25f0016990 con 0x7f25f814d6f0 2024-09-17T02:49:48.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 -- 172.21.15.131:0/1157188675 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f25f815bea0 con 0x7f25f814d6f0 2024-09-17T02:49:48.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25f57fa640 1 -- 172.21.15.131:0/1157188675 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v96) ==== 80+0+248 (secure 0 0 0) 0x7f25f00f9300 con 0x7f25f814d6f0 2024-09-17T02:49:48.140 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6835/3013984008 2024-09-18T02:49:07.494740+0000 2024-09-17T02:49:48.140 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6834/3013984008 2024-09-18T02:49:07.494740+0000 2024-09-17T02:49:48.140 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6835/2766162522 2024-09-18T02:47:48.093626+0000 2024-09-17T02:49:48.140 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6834/2766162522 2024-09-18T02:47:48.093626+0000 2024-09-17T02:49:48.141 INFO:teuthology.orchestra.run.smithi131.stderr:listed 4 entries 2024-09-17T02:49:48.142 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 -- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25e4073200 msgr2=0x7f25e4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:48.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25e4073200 0x7f25e4075710 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f25e800b810 tx=0x7f25e8008040 comp rx=0 tx=0).stop 2024-09-17T02:49:48.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 -- 172.21.15.131:0/1157188675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25f814d6f0 msgr2=0x7f25f81507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:48.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25f814d6f0 0x7f25f81507c0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009bd0 tx=0x7f25f00093d0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 -- 172.21.15.131:0/1157188675 shutdown_connections 2024-09-17T02:49:48.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25e4073200 0x7f25e4075710 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25f814d6f0 0x7f25f81507c0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25f815c3e0 0x7f25f8150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 --2- 172.21.15.131:0/1157188675 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25f80ced00 0x7f25f814b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 -- 172.21.15.131:0/1157188675 >> 172.21.15.131:0/1157188675 conn(0x7f25f806eea0 msgr2=0x7f25f80b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:48.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 -- 172.21.15.131:0/1157188675 shutdown_connections 2024-09-17T02:49:48.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.144+0000 7f25fed15640 1 -- 172.21.15.131:0/1157188675 wait complete. 2024-09-17T02:49:48.161 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:49:48.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:48.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:49:48.171 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[47705]: fuse finished with error 0 and tester_r 0 2024-09-17T02:49:48.173 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:49:48.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:49:48.173 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:49:48.253 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:49:48.254 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:49:48.254 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:49:48.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:49:48.260 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:49:48.261 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:49:48.261 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:49:48.301 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:49:48.301 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:49:48.301 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:49:48.301 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:49:48.308 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:49:48.308 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:49:48.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.504+0000 7ff6670ed640 1 -- 172.21.15.131:0/2579273613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff644003a40 msgr2=0x7ff644003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:48.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.504+0000 7ff6670ed640 1 --2- 172.21.15.131:0/2579273613 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff644003a40 0x7ff644003eb0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff6500016f0 tx=0x7ff65001c7b0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.504+0000 7ff66568a640 1 -- 172.21.15.131:0/2579273613 reap_dead start 2024-09-17T02:49:48.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.504+0000 7ff6670ed640 1 -- 172.21.15.131:0/2579273613 shutdown_connections 2024-09-17T02:49:48.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.504+0000 7ff6670ed640 1 -- 172.21.15.131:0/2579273613 >> 172.21.15.131:0/2579273613 conn(0x7ff66006eea0 msgr2=0x7ff66006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:48.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.504+0000 7ff6670ed640 1 -- 172.21.15.131:0/2579273613 shutdown_connections 2024-09-17T02:49:48.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.504+0000 7ff6670ed640 1 -- 172.21.15.131:0/2579273613 wait complete. 2024-09-17T02:49:48.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff6670ed640 1 Processor -- start 2024-09-17T02:49:48.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff6670ed640 1 -- start start 2024-09-17T02:49:48.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff6670ed640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff644003a40 0x7ff66014adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:48.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff6670ed640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6600cb5e0 0x7ff6600cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:48.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff6670ed640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff660156b30 0x7ff660156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:48.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff6670ed640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff660168c60 con 0x7ff6600cb5e0 2024-09-17T02:49:48.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff6670ed640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff660168ae0 con 0x7ff644003a40 2024-09-17T02:49:48.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff6670ed640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff660168de0 con 0x7ff660156b30 2024-09-17T02:49:48.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6600cb5e0 0x7ff6600cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:48.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6600cb5e0 0x7ff6600cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33620/0 (socket says 172.21.15.131:33620) 2024-09-17T02:49:48.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65ffff640 1 -- 172.21.15.131:0/623473116 learned_addr learned my addr 172.21.15.131:0/623473116 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:48.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff664e89640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff644003a40 0x7ff66014adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:48.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65ffff640 1 -- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff660156b30 msgr2=0x7ff660156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:48.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65ffff640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff660156b30 0x7ff660156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65ffff640 1 -- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff644003a40 msgr2=0x7ff66014adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:48.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65ffff640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff644003a40 0x7ff66014adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65ffff640 1 -- 172.21.15.131:0/623473116 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff660157480 con 0x7ff6600cb5e0 2024-09-17T02:49:48.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65ffff640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6600cb5e0 0x7ff6600cb9f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff64c00d730 tx=0x7ff64c00dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:48.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65d7fa640 1 -- 172.21.15.131:0/623473116 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c018070 con 0x7ff6600cb5e0 2024-09-17T02:49:48.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65d7fa640 1 -- 172.21.15.131:0/623473116 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff64c0050d0 con 0x7ff6600cb5e0 2024-09-17T02:49:48.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff6670ed640 1 -- 172.21.15.131:0/623473116 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6601f8a50 con 0x7ff6600cb5e0 2024-09-17T02:49:48.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff65d7fa640 1 -- 172.21.15.131:0/623473116 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64c00b6c0 con 0x7ff6600cb5e0 2024-09-17T02:49:48.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.508+0000 7ff6670ed640 1 -- 172.21.15.131:0/623473116 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6601f8ff0 con 0x7ff6600cb5e0 2024-09-17T02:49:48.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.512+0000 7ff6670ed640 1 -- 172.21.15.131:0/623473116 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6600c4580 con 0x7ff6600cb5e0 2024-09-17T02:49:48.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.512+0000 7ff65d7fa640 1 -- 172.21.15.131:0/623473116 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff64c01b070 con 0x7ff6600cb5e0 2024-09-17T02:49:48.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.512+0000 7ff65d7fa640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff648073230 0x7ff648075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:48.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.512+0000 7ff65d7fa640 1 -- 172.21.15.131:0/623473116 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7ff64c093d90 con 0x7ff6600cb5e0 2024-09-17T02:49:48.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.512+0000 7ff664e89640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff648073230 0x7ff648075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:48.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.512+0000 7ff664e89640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff648073230 0x7ff648075740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff6500016c0 tx=0x7ff6500a8040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:48.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.516+0000 7ff65d7fa640 1 -- 172.21.15.131:0/623473116 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff660157480 con 0x7ff6600cb5e0 2024-09-17T02:49:48.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.720+0000 7ff6670ed640 1 -- 172.21.15.131:0/623473116 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6601660a0 con 0x7ff6600cb5e0 2024-09-17T02:49:48.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.720+0000 7ff65d7fa640 1 -- 172.21.15.131:0/623473116 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5238 (secure 0 0 0) 0x7ff64c05e500 con 0x7ff6600cb5e0 2024-09-17T02:49:48.717 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:48.717 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":77,"btime":"2024-09-17T02:49:11:530527+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:49:11.530508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/1957540940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1957540940},{"type":"v1","addr":"172.21.15.131:6835","nonce":1957540940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7521,"qdb_cluster":[7521]},"id":6}]} 2024-09-17T02:49:48.717 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 77 2024-09-17T02:49:48.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 -- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff648073230 msgr2=0x7ff648075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:48.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff648073230 0x7ff648075740 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff6500016c0 tx=0x7ff6500a8040 comp rx=0 tx=0).stop 2024-09-17T02:49:48.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 -- 172.21.15.131:0/623473116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6600cb5e0 msgr2=0x7ff6600cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:48.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6600cb5e0 0x7ff6600cb9f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7ff64c00d730 tx=0x7ff64c00dbf0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 -- 172.21.15.131:0/623473116 shutdown_connections 2024-09-17T02:49:48.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff648073230 0x7ff648075740 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff660156b30 0x7ff660156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff6600cb5e0 0x7ff6600cb9f0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 --2- 172.21.15.131:0/623473116 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff644003a40 0x7ff66014adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 -- 172.21.15.131:0/623473116 >> 172.21.15.131:0/623473116 conn(0x7ff66006eea0 msgr2=0x7ff6600c4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:48.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 -- 172.21.15.131:0/623473116 shutdown_connections 2024-09-17T02:49:48.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.724+0000 7ff6670ed640 1 -- 172.21.15.131:0/623473116 wait complete. 2024-09-17T02:49:48.739 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:49:48.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 -- 172.21.15.131:0/2115495429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826006e3e0 msgr2=0x7f826015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:48.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 --2- 172.21.15.131:0/2115495429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826006e3e0 0x7f826015bea0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f826006a1c0 tx=0x7f825401c6e0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 -- 172.21.15.131:0/2115495429 shutdown_connections 2024-09-17T02:49:48.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 --2- 172.21.15.131:0/2115495429 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826006e3e0 0x7f826015bea0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 --2- 172.21.15.131:0/2115495429 >> v2:172.21.15.189:3300/0 conn(0x7f82600ced00 0x7f826006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 -- 172.21.15.131:0/2115495429 >> 172.21.15.131:0/2115495429 conn(0x7f826006eea0 msgr2=0x7f826006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:48.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 -- 172.21.15.131:0/2115495429 shutdown_connections 2024-09-17T02:49:48.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 -- 172.21.15.131:0/2115495429 wait complete. 2024-09-17T02:49:48.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 Processor -- start 2024-09-17T02:49:48.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 -- start start 2024-09-17T02:49:48.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f826006e3e0 0x7f8260154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:48.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.884+0000 7f82648b1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f82600ced00 0x7f8260147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:48.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f82648b1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826015c3e0 0x7f82601511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:48.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f82648b1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8260168bf0 con 0x7f826015c3e0 2024-09-17T02:49:48.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f82648b1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f82600cf300 con 0x7f826006e3e0 2024-09-17T02:49:48.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f82648b1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8260168d70 con 0x7f82600ced00 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826015c3e0 0x7f82601511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826015c3e0 0x7f82601511d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33632/0 (socket says 172.21.15.131:33632) 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825ed9d640 1 -- 172.21.15.131:0/4219481254 learned_addr learned my addr 172.21.15.131:0/4219481254 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825e59c640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f826006e3e0 0x7f8260154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825dd9b640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f82600ced00 0x7f8260147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825ed9d640 1 -- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f82600ced00 msgr2=0x7f8260147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825ed9d640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f82600ced00 0x7f8260147fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825ed9d640 1 -- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f826006e3e0 msgr2=0x7f8260154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825ed9d640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f826006e3e0 0x7f8260154b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825ed9d640 1 -- 172.21.15.131:0/4219481254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8260148520 con 0x7f826015c3e0 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825ed9d640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826015c3e0 0x7f82601511d0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f825800b9e0 tx=0x7f825800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:48.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f824effd640 1 -- 172.21.15.131:0/4219481254 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258019070 con 0x7f826015c3e0 2024-09-17T02:49:48.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f824effd640 1 -- 172.21.15.131:0/4219481254 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825800cc10 con 0x7f826015c3e0 2024-09-17T02:49:48.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f82648b1640 1 -- 172.21.15.131:0/4219481254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82601faac0 con 0x7f826015c3e0 2024-09-17T02:49:48.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f824effd640 1 -- 172.21.15.131:0/4219481254 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82580146a0 con 0x7f826015c3e0 2024-09-17T02:49:48.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f82648b1640 1 -- 172.21.15.131:0/4219481254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82601f7530 con 0x7f826015c3e0 2024-09-17T02:49:48.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f82648b1640 1 -- 172.21.15.131:0/4219481254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826015bea0 con 0x7f826015c3e0 2024-09-17T02:49:48.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f824effd640 1 -- 172.21.15.131:0/4219481254 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f825801c070 con 0x7f826015c3e0 2024-09-17T02:49:48.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f824effd640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8240073230 0x7f8240075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:48.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f824effd640 1 -- 172.21.15.131:0/4219481254 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f82580942f0 con 0x7f826015c3e0 2024-09-17T02:49:48.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.888+0000 7f825e59c640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8240073230 0x7f8240075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:48.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.892+0000 7f825e59c640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8240073230 0x7f8240075740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f82500058d0 tx=0x7f8250005d90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:48.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:48.892+0000 7f824effd640 1 -- 172.21.15.131:0/4219481254 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f825805f020 con 0x7f826015c3e0 2024-09-17T02:49:49.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.084+0000 7f82648b1640 1 -- 172.21.15.131:0/4219481254 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f826014ca20 con 0x7f826015c3e0 2024-09-17T02:49:49.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.084+0000 7f824effd640 1 -- 172.21.15.131:0/4219481254 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5238 (secure 0 0 0) 0x7f825805e920 con 0x7f826015c3e0 2024-09-17T02:49:49.083 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:49.083 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":77,"btime":"2024-09-17T02:49:11:530527+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:49:11.530508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/1957540940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1957540940},{"type":"v1","addr":"172.21.15.131:6835","nonce":1957540940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7521,"qdb_cluster":[7521]},"id":6}]} 2024-09-17T02:49:49.083 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 77 2024-09-17T02:49:49.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 -- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8240073230 msgr2=0x7f8240075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8240073230 0x7f8240075740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f82500058d0 tx=0x7f8250005d90 comp rx=0 tx=0).stop 2024-09-17T02:49:49.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 -- 172.21.15.131:0/4219481254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826015c3e0 msgr2=0x7f82601511d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826015c3e0 0x7f82601511d0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f825800b9e0 tx=0x7f825800bea0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 -- 172.21.15.131:0/4219481254 shutdown_connections 2024-09-17T02:49:49.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8240073230 0x7f8240075740 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f826015c3e0 0x7f82601511d0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f82600ced00 0x7f8260147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 --2- 172.21.15.131:0/4219481254 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f826006e3e0 0x7f8260154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 -- 172.21.15.131:0/4219481254 >> 172.21.15.131:0/4219481254 conn(0x7f826006eea0 msgr2=0x7f82600cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:49.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 -- 172.21.15.131:0/4219481254 shutdown_connections 2024-09-17T02:49:49.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.088+0000 7f82648b1640 1 -- 172.21.15.131:0/4219481254 wait complete. 2024-09-17T02:49:49.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:49:49.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.244+0000 7fbf6be18640 1 -- 172.21.15.131:0/1616250087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf50003a60 msgr2=0x7fbf50003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.244+0000 7fbf6be18640 1 --2- 172.21.15.131:0/1616250087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf50003a60 0x7fbf50003ed0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009b30 tx=0x7fbf6001d230 comp rx=0 tx=0).stop 2024-09-17T02:49:49.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.244+0000 7fbf6be18640 1 -- 172.21.15.131:0/1616250087 shutdown_connections 2024-09-17T02:49:49.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.244+0000 7fbf6be18640 1 --2- 172.21.15.131:0/1616250087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf50003a60 0x7fbf50003ed0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.244+0000 7fbf6be18640 1 -- 172.21.15.131:0/1616250087 >> 172.21.15.131:0/1616250087 conn(0x7fbf640d1b10 msgr2=0x7fbf640d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:49.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.244+0000 7fbf6be18640 1 -- 172.21.15.131:0/1616250087 shutdown_connections 2024-09-17T02:49:49.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.244+0000 7fbf6be18640 1 -- 172.21.15.131:0/1616250087 wait complete. 2024-09-17T02:49:49.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6be18640 1 Processor -- start 2024-09-17T02:49:49.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6be18640 1 -- start start 2024-09-17T02:49:49.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6be18640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf50003a60 0x7fbf640cde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6be18640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf6415b200 0x7fbf6415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6be18640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6415ce60 0x7fbf6415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6be18640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbf64162c20 con 0x7fbf6415ce60 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6be18640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6415fdf0 con 0x7fbf6415b200 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6be18640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fbf64162da0 con 0x7fbf50003a60 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6a3b5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6415ce60 0x7fbf6415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6a3b5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6415ce60 0x7fbf6415d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33644/0 (socket says 172.21.15.131:33644) 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6a3b5640 1 -- 172.21.15.131:0/958096041 learned_addr learned my addr 172.21.15.131:0/958096041 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf69bb4640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf50003a60 0x7fbf640cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf693b3640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf6415b200 0x7fbf6415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6a3b5640 1 -- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf50003a60 msgr2=0x7fbf640cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6a3b5640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf50003a60 0x7fbf640cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6a3b5640 1 -- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf6415b200 msgr2=0x7fbf6415b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6a3b5640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf6415b200 0x7fbf6415b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6a3b5640 1 -- 172.21.15.131:0/958096041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6415bdc0 con 0x7fbf6415ce60 2024-09-17T02:49:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6a3b5640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6415ce60 0x7fbf6415d270 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fbf640ba520 tx=0x7fbf5c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf5a7fc640 1 -- 172.21.15.131:0/958096041 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c018070 con 0x7fbf6415ce60 2024-09-17T02:49:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf5a7fc640 1 -- 172.21.15.131:0/958096041 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf5c003980 con 0x7fbf6415ce60 2024-09-17T02:49:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6be18640 1 -- 172.21.15.131:0/958096041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6415c0a0 con 0x7fbf6415ce60 2024-09-17T02:49:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf5a7fc640 1 -- 172.21.15.131:0/958096041 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c003db0 con 0x7fbf6415ce60 2024-09-17T02:49:49.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.248+0000 7fbf6be18640 1 -- 172.21.15.131:0/958096041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6415c660 con 0x7fbf6415ce60 2024-09-17T02:49:49.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.252+0000 7fbf6be18640 1 -- 172.21.15.131:0/958096041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf641631d0 con 0x7fbf6415ce60 2024-09-17T02:49:49.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.252+0000 7fbf5a7fc640 1 -- 172.21.15.131:0/958096041 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbf5c01b070 con 0x7fbf6415ce60 2024-09-17T02:49:49.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.252+0000 7fbf5a7fc640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf48073200 0x7fbf48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.252+0000 7fbf5a7fc640 1 -- 172.21.15.131:0/958096041 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7fbf5c093b20 con 0x7fbf6415ce60 2024-09-17T02:49:49.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.252+0000 7fbf69bb4640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf48073200 0x7fbf48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.252+0000 7fbf69bb4640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf48073200 0x7fbf48075710 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbf600ae000 tx=0x7fbf60019fd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:49.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.256+0000 7fbf5a7fc640 1 -- 172.21.15.131:0/958096041 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbf5c05e7b0 con 0x7fbf6415ce60 2024-09-17T02:49:49.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.424+0000 7fbf6be18640 1 -- 172.21.15.131:0/958096041 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf6415bb50 con 0x7fbf6415ce60 2024-09-17T02:49:49.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf5a7fc640 1 -- 172.21.15.131:0/958096041 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+17549 (secure 0 0 0) 0x7fbf5c05e0b0 con 0x7fbf6415ce60 2024-09-17T02:49:49.424 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:49.424 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":96,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:49:07.494769+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:47:23.485820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-17T02:47:24.759592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1a","mappings":[{"from":6,"to":4}]},{"pgid":"14.1","mappings":[{"from":7,"to":0}]},{"pgid":"14.9","mappings":[{"from":7,"to":3}]},{"pgid":"14.f","mappings":[{"from":1,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0},{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6835/3013984008":"2024-09-18T02:49:07.494740+0000","172.21.15.131:6834/3013984008":"2024-09-18T02:49:07.494740+0000","172.21.15.131:6835/2766162522":"2024-09-18T02:47:48.093626+0000","172.21.15.131:6834/2766162522":"2024-09-18T02:47:48.093626+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:49:49.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 -- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf48073200 msgr2=0x7fbf48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf48073200 0x7fbf48075710 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbf600ae000 tx=0x7fbf60019fd0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 -- 172.21.15.131:0/958096041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6415ce60 msgr2=0x7fbf6415d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6415ce60 0x7fbf6415d270 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fbf640ba520 tx=0x7fbf5c00dc10 comp rx=0 tx=0).stop 2024-09-17T02:49:49.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 -- 172.21.15.131:0/958096041 shutdown_connections 2024-09-17T02:49:49.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf48073200 0x7fbf48075710 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf6415ce60 0x7fbf6415d270 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf6415b200 0x7fbf6415b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 --2- 172.21.15.131:0/958096041 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf50003a60 0x7fbf640cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 -- 172.21.15.131:0/958096041 >> 172.21.15.131:0/958096041 conn(0x7fbf640d1b10 msgr2=0x7fbf641618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:49.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 -- 172.21.15.131:0/958096041 shutdown_connections 2024-09-17T02:49:49.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.428+0000 7fbf6be18640 1 -- 172.21.15.131:0/958096041 wait complete. 2024-09-17T02:49:49.445 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T02:49:49.446 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:49:49.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.588+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/1007744510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf4815be80 msgr2=0x7fbf48071750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.588+0000 7fbf4cd9c640 1 --2- 172.21.15.131:0/1007744510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf4815be80 0x7fbf48071750 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbf48072bd0 tx=0x7fbf3801c800 comp rx=0 tx=0).stop 2024-09-17T02:49:49.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.588+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/1007744510 shutdown_connections 2024-09-17T02:49:49.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.588+0000 7fbf4cd9c640 1 --2- 172.21.15.131:0/1007744510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf4815be80 0x7fbf48071750 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.588+0000 7fbf4cd9c640 1 --2- 172.21.15.131:0/1007744510 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf4815ba70 0x7fbf48071210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.588+0000 7fbf4cd9c640 1 --2- 172.21.15.131:0/1007744510 >> v2:172.21.15.189:3300/0 conn(0x7fbf48152550 0x7fbf4806e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.588+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/1007744510 >> 172.21.15.131:0/1007744510 conn(0x7fbf480d1130 msgr2=0x7fbf480d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:49.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.588+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/1007744510 shutdown_connections 2024-09-17T02:49:49.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.588+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/1007744510 wait complete. 2024-09-17T02:49:49.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4cd9c640 1 Processor -- start 2024-09-17T02:49:49.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4cd9c640 1 -- start start 2024-09-17T02:49:49.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4cd9c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf48152550 0x7fbf48160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4cd9c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf4815ba70 0x7fbf4814d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4cd9c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf4815be80 0x7fbf4815a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4cd9c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4815f3f0 con 0x7fbf4815be80 2024-09-17T02:49:49.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4cd9c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4815f270 con 0x7fbf4815ba70 2024-09-17T02:49:49.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4cd9c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fbf4815f570 con 0x7fbf48152550 2024-09-17T02:49:49.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf46d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf4815be80 0x7fbf4815a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf46d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf4815be80 0x7fbf4815a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33672/0 (socket says 172.21.15.131:33672) 2024-09-17T02:49:49.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf46d9d640 1 -- 172.21.15.131:0/791085700 learned_addr learned my addr 172.21.15.131:0/791085700 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:49.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf45d9b640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf4815ba70 0x7fbf4814d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4659c640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf48152550 0x7fbf48160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf46d9d640 1 -- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf48152550 msgr2=0x7fbf48160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf46d9d640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf48152550 0x7fbf48160130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf46d9d640 1 -- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf4815ba70 msgr2=0x7fbf4814d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.591 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf46d9d640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf4815ba70 0x7fbf4814d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf46d9d640 1 -- 172.21.15.131:0/791085700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4814edb0 con 0x7fbf4815be80 2024-09-17T02:49:49.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf46d9d640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf4815be80 0x7fbf4815a4b0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fbf380b0040 tx=0x7fbf380a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:49.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf36ffd640 1 -- 172.21.15.131:0/791085700 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf380bc070 con 0x7fbf4815be80 2024-09-17T02:49:49.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf36ffd640 1 -- 172.21.15.131:0/791085700 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf38004450 con 0x7fbf4815be80 2024-09-17T02:49:49.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/791085700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4814f090 con 0x7fbf4815be80 2024-09-17T02:49:49.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf36ffd640 1 -- 172.21.15.131:0/791085700 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf380b2040 con 0x7fbf4815be80 2024-09-17T02:49:49.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/791085700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4814f650 con 0x7fbf4815be80 2024-09-17T02:49:49.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.592+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/791085700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf4806e840 con 0x7fbf4815be80 2024-09-17T02:49:49.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.596+0000 7fbf36ffd640 1 -- 172.21.15.131:0/791085700 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbf380bf070 con 0x7fbf4815be80 2024-09-17T02:49:49.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.596+0000 7fbf36ffd640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf28073230 0x7fbf28075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.596+0000 7fbf36ffd640 1 -- 172.21.15.131:0/791085700 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7fbf3812e080 con 0x7fbf4815be80 2024-09-17T02:49:49.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.596+0000 7fbf4659c640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf28073230 0x7fbf28075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.596+0000 7fbf4659c640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf28073230 0x7fbf28075740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000ac80 tx=0x7fbf4000a250 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:49.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.600+0000 7fbf36ffd640 1 -- 172.21.15.131:0/791085700 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbf380164b0 con 0x7fbf4815be80 2024-09-17T02:49:49.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.800+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/791085700 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf481607d0 con 0x7fbf4815be80 2024-09-17T02:49:49.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.800+0000 7fbf36ffd640 1 -- 172.21.15.131:0/791085700 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5238 (secure 0 0 0) 0x7fbf380f8e90 con 0x7fbf4815be80 2024-09-17T02:49:49.798 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:49.798 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":77,"btime":"2024-09-17T02:49:11:530527+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:49:11.530508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/1957540940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1957540940},{"type":"v1","addr":"172.21.15.131:6835","nonce":1957540940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7521,"qdb_cluster":[7521]},"id":6}]} 2024-09-17T02:49:49.798 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 77 2024-09-17T02:49:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf28073230 msgr2=0x7fbf28075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf28073230 0x7fbf28075740 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbf4000ac80 tx=0x7fbf4000a250 comp rx=0 tx=0).stop 2024-09-17T02:49:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/791085700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf4815be80 msgr2=0x7fbf4815a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf4815be80 0x7fbf4815a4b0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fbf380b0040 tx=0x7fbf380a5790 comp rx=0 tx=0).stop 2024-09-17T02:49:49.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/791085700 shutdown_connections 2024-09-17T02:49:49.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbf28073230 0x7fbf28075740 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbf4815be80 0x7fbf4815a4b0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbf4815ba70 0x7fbf4814d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 --2- 172.21.15.131:0/791085700 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbf48152550 0x7fbf48160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/791085700 >> 172.21.15.131:0/791085700 conn(0x7fbf480d1130 msgr2=0x7fbf480d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:49.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/791085700 shutdown_connections 2024-09-17T02:49:49.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.804+0000 7fbf4cd9c640 1 -- 172.21.15.131:0/791085700 wait complete. 2024-09-17T02:49:49.819 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:49:49.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.964+0000 7f5c258cd640 1 -- 172.21.15.131:0/3058901828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c0c003a60 msgr2=0x7f5c0c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.964+0000 7f5c258cd640 1 --2- 172.21.15.131:0/3058901828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c0c003a60 0x7f5c0c003ed0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009ee0 tx=0x7f5c1801d5f0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.964+0000 7f5c258cd640 1 -- 172.21.15.131:0/3058901828 shutdown_connections 2024-09-17T02:49:49.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.964+0000 7f5c258cd640 1 --2- 172.21.15.131:0/3058901828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c0c003a60 0x7f5c0c003ed0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.964+0000 7f5c258cd640 1 -- 172.21.15.131:0/3058901828 >> 172.21.15.131:0/3058901828 conn(0x7f5c2006eea0 msgr2=0x7f5c2006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:49.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.964+0000 7f5c258cd640 1 -- 172.21.15.131:0/3058901828 shutdown_connections 2024-09-17T02:49:49.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.964+0000 7f5c258cd640 1 -- 172.21.15.131:0/3058901828 wait complete. 2024-09-17T02:49:49.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c258cd640 1 Processor -- start 2024-09-17T02:49:49.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c258cd640 1 -- start start 2024-09-17T02:49:49.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c258cd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c0c003a60 0x7f5c201fa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c258cd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c201fabf0 0x7f5c201581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c258cd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5c201586f0 0x7f5c201e97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c258cd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5c200cc480 con 0x7f5c201fabf0 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c258cd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5c200cc300 con 0x7f5c0c003a60 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c258cd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5c200cc600 con 0x7f5c201586f0 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c1e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c201fabf0 0x7f5c201581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c1e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c201fabf0 0x7f5c201581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33700/0 (socket says 172.21.15.131:33700) 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c1e7fc640 1 -- 172.21.15.131:0/3627580408 learned_addr learned my addr 172.21.15.131:0/3627580408 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c1effd640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c0c003a60 0x7f5c201fa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c1f7fe640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5c201586f0 0x7f5c201e97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c1e7fc640 1 -- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5c201586f0 msgr2=0x7f5c201e97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c1e7fc640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5c201586f0 0x7f5c201e97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c1e7fc640 1 -- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c0c003a60 msgr2=0x7f5c201fa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c1e7fc640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c0c003a60 0x7f5c201fa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c1e7fc640 1 -- 172.21.15.131:0/3627580408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c20158f60 con 0x7f5c201fabf0 2024-09-17T02:49:49.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c1e7fc640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c201fabf0 0x7f5c201581b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800c700 tx=0x7f5c0800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:49.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c17fff640 1 -- 172.21.15.131:0/3627580408 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c08017070 con 0x7f5c201fabf0 2024-09-17T02:49:49.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c17fff640 1 -- 172.21.15.131:0/3627580408 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c08005380 con 0x7f5c201fabf0 2024-09-17T02:49:49.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c258cd640 1 -- 172.21.15.131:0/3627580408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c201f80c0 con 0x7f5c201fabf0 2024-09-17T02:49:49.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c17fff640 1 -- 172.21.15.131:0/3627580408 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800a8a0 con 0x7f5c201fabf0 2024-09-17T02:49:49.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.968+0000 7f5c258cd640 1 -- 172.21.15.131:0/3627580408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c201f8680 con 0x7f5c201fabf0 2024-09-17T02:49:49.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.972+0000 7f5c258cd640 1 -- 172.21.15.131:0/3627580408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c20075b80 con 0x7f5c201fabf0 2024-09-17T02:49:49.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.972+0000 7f5c17fff640 1 -- 172.21.15.131:0/3627580408 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5c0801a070 con 0x7f5c201fabf0 2024-09-17T02:49:49.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.972+0000 7f5c17fff640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5c00073200 0x7f5c00075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:49.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.972+0000 7f5c17fff640 1 -- 172.21.15.131:0/3627580408 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f5c080939c0 con 0x7f5c201fabf0 2024-09-17T02:49:49.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.972+0000 7f5c1effd640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5c00073200 0x7f5c00075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:49.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.972+0000 7f5c1effd640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5c00073200 0x7f5c00075710 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5c180af000 tx=0x7f5c1801a390 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:49.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:49.976+0000 7f5c17fff640 1 -- 172.21.15.131:0/3627580408 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5c08095e20 con 0x7f5c201fabf0 2024-09-17T02:49:50.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.164+0000 7f5c258cd640 1 -- 172.21.15.131:0/3627580408 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c20158cf0 con 0x7f5c201fabf0 2024-09-17T02:49:50.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c17fff640 1 -- 172.21.15.131:0/3627580408 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+5238 (secure 0 0 0) 0x7f5c2006e7f0 con 0x7f5c201fabf0 2024-09-17T02:49:50.164 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:50.164 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":77,"btime":"2024-09-17T02:49:11:530527+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:47:25.797713+0000","modified":"2024-09-17T02:49:11.530508+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":96,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7521},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7521":{"gid":7521,"name":"c","rank":0,"incarnation":74,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/1957540940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":1957540940},{"type":"v1","addr":"172.21.15.131:6835","nonce":1957540940}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7521,"qdb_cluster":[7521]},"id":6}]} 2024-09-17T02:49:50.164 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 77 2024-09-17T02:49:50.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 -- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5c00073200 msgr2=0x7f5c00075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5c00073200 0x7f5c00075710 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5c180af000 tx=0x7f5c1801a390 comp rx=0 tx=0).stop 2024-09-17T02:49:50.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 -- 172.21.15.131:0/3627580408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c201fabf0 msgr2=0x7f5c201581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c201fabf0 0x7f5c201581b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800c700 tx=0x7f5c0800cbc0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 -- 172.21.15.131:0/3627580408 shutdown_connections 2024-09-17T02:49:50.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5c00073200 0x7f5c00075710 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5c201586f0 0x7f5c201e97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5c201fabf0 0x7f5c201581b0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 --2- 172.21.15.131:0/3627580408 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5c0c003a60 0x7f5c201fa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 -- 172.21.15.131:0/3627580408 >> 172.21.15.131:0/3627580408 conn(0x7f5c2006eea0 msgr2=0x7f5c20155410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 -- 172.21.15.131:0/3627580408 shutdown_connections 2024-09-17T02:49:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.168+0000 7f5c258cd640 1 -- 172.21.15.131:0/3627580408 wait complete. 2024-09-17T02:49:50.184 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:49:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.336+0000 7f33731a3640 1 -- 172.21.15.131:0/4120557669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f336c06f6c0 msgr2=0x7f336c15b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.336+0000 7f33731a3640 1 --2- 172.21.15.131:0/4120557669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f336c06f6c0 0x7f336c15b870 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f3354009bc0 tx=0x7f335401cba0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.336+0000 7f33731a3640 1 -- 172.21.15.131:0/4120557669 shutdown_connections 2024-09-17T02:49:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.336+0000 7f33731a3640 1 --2- 172.21.15.131:0/4120557669 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f336c15bdb0 0x7f336c151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.336+0000 7f33731a3640 1 --2- 172.21.15.131:0/4120557669 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f336c06f6c0 0x7f336c15b870 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.336+0000 7f33731a3640 1 -- 172.21.15.131:0/4120557669 >> 172.21.15.131:0/4120557669 conn(0x7f336c073d10 msgr2=0x7f336c074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.336+0000 7f33731a3640 1 -- 172.21.15.131:0/4120557669 shutdown_connections 2024-09-17T02:49:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.336+0000 7f33731a3640 1 -- 172.21.15.131:0/4120557669 wait complete. 2024-09-17T02:49:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33731a3640 1 Processor -- start 2024-09-17T02:49:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33731a3640 1 -- start start 2024-09-17T02:49:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33731a3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f336c0cd470 0x7f336c0c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33731a3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f336c15bdb0 0x7f336c0af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33731a3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f336c0afa00 0x7f336c146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33731a3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f336c169000 con 0x7f336c15bdb0 2024-09-17T02:49:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33731a3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f336c0cc410 con 0x7f336c0afa00 2024-09-17T02:49:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33731a3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f336c169180 con 0x7f336c0cd470 2024-09-17T02:49:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f336bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f336c15bdb0 0x7f336c0af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f336bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f336c15bdb0 0x7f336c0af4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33710/0 (socket says 172.21.15.131:33710) 2024-09-17T02:49:50.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f336bfff640 1 -- 172.21.15.131:0/2283027453 learned_addr learned my addr 172.21.15.131:0/2283027453 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f3371740640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f336c0afa00 0x7f336c146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f336bfff640 1 -- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f336c0cd470 msgr2=0x7f336c0c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f3370f3f640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f336c0cd470 0x7f336c0c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f336bfff640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f336c0cd470 0x7f336c0c2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f336bfff640 1 -- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f336c0afa00 msgr2=0x7f336c146f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f336bfff640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f336c0afa00 0x7f336c146f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f336bfff640 1 -- 172.21.15.131:0/2283027453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c0ada20 con 0x7f336c15bdb0 2024-09-17T02:49:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f3371740640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f336c0afa00 0x7f336c146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f336bfff640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f336c15bdb0 0x7f336c0af4c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f335401b250 tx=0x7f33540a5a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:50.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33697fa640 1 -- 172.21.15.131:0/2283027453 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33540bb070 con 0x7f336c15bdb0 2024-09-17T02:49:50.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33697fa640 1 -- 172.21.15.131:0/2283027453 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3354004d40 con 0x7f336c15bdb0 2024-09-17T02:49:50.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33697fa640 1 -- 172.21.15.131:0/2283027453 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33540b2040 con 0x7f336c15bdb0 2024-09-17T02:49:50.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33731a3640 1 -- 172.21.15.131:0/2283027453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c0adcc0 con 0x7f336c15bdb0 2024-09-17T02:49:50.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.340+0000 7f33731a3640 1 -- 172.21.15.131:0/2283027453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c147ea0 con 0x7f336c15bdb0 2024-09-17T02:49:50.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.344+0000 7f33731a3640 1 -- 172.21.15.131:0/2283027453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c15b870 con 0x7f336c15bdb0 2024-09-17T02:49:50.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.344+0000 7f33697fa640 1 -- 172.21.15.131:0/2283027453 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f33540043a0 con 0x7f336c15bdb0 2024-09-17T02:49:50.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.344+0000 7f33697fa640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f334c073230 0x7f334c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:50.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.344+0000 7f33697fa640 1 -- 172.21.15.131:0/2283027453 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f335412e450 con 0x7f336c15bdb0 2024-09-17T02:49:50.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.344+0000 7f3370f3f640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f334c073230 0x7f334c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:50.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.344+0000 7f3370f3f640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f334c073230 0x7f334c075740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f335c001570 tx=0x7f335c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:50.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.348+0000 7f33697fa640 1 -- 172.21.15.131:0/2283027453 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f33540f1db0 con 0x7f336c15bdb0 2024-09-17T02:49:50.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.512+0000 7f33731a3640 1 -- 172.21.15.131:0/2283027453 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f336c0d0d80 con 0x7f336c15bdb0 2024-09-17T02:49:50.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33697fa640 1 -- 172.21.15.131:0/2283027453 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+17549 (secure 0 0 0) 0x7f33540a5e20 con 0x7f336c15bdb0 2024-09-17T02:49:50.511 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:50.512 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":96,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:49:07.494769+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:47:23.485820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-17T02:47:24.759592+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.1a","mappings":[{"from":6,"to":4}]},{"pgid":"14.1","mappings":[{"from":7,"to":0}]},{"pgid":"14.9","mappings":[{"from":7,"to":3}]},{"pgid":"14.f","mappings":[{"from":1,"to":0}]},{"pgid":"14.2d","mappings":[{"from":7,"to":0}]},{"pgid":"14.3d","mappings":[{"from":7,"to":0},{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6835/3013984008":"2024-09-18T02:49:07.494740+0000","172.21.15.131:6834/3013984008":"2024-09-18T02:49:07.494740+0000","172.21.15.131:6835/2766162522":"2024-09-18T02:47:48.093626+0000","172.21.15.131:6834/2766162522":"2024-09-18T02:47:48.093626+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:49:50.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 -- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f334c073230 msgr2=0x7f334c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f334c073230 0x7f334c075740 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f335c001570 tx=0x7f335c009040 comp rx=0 tx=0).stop 2024-09-17T02:49:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 -- 172.21.15.131:0/2283027453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f336c15bdb0 msgr2=0x7f336c0af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f336c15bdb0 0x7f336c0af4c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f335401b250 tx=0x7f33540a5a20 comp rx=0 tx=0).stop 2024-09-17T02:49:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 -- 172.21.15.131:0/2283027453 shutdown_connections 2024-09-17T02:49:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f334c073230 0x7f334c075740 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f336c0afa00 0x7f336c146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f336c15bdb0 0x7f336c0af4c0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 --2- 172.21.15.131:0/2283027453 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f336c0cd470 0x7f336c0c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 -- 172.21.15.131:0/2283027453 >> 172.21.15.131:0/2283027453 conn(0x7f336c073d10 msgr2=0x7f336c15ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 -- 172.21.15.131:0/2283027453 shutdown_connections 2024-09-17T02:49:50.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.516+0000 7f33731a3640 1 -- 172.21.15.131:0/2283027453 wait complete. 2024-09-17T02:49:50.532 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:49:50.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.680+0000 7f1258e69640 1 -- 172.21.15.131:0/4175730515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1240009fe0 msgr2=0x7f124000c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.680+0000 7f1258e69640 1 --2- 172.21.15.131:0/4175730515 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1240009fe0 0x7f124000c4c0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f124c002240 tx=0x7f124c01d260 comp rx=0 tx=0).stop 2024-09-17T02:49:50.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.680+0000 7f1252d9d640 1 -- 172.21.15.131:0/4175730515 reap_dead start 2024-09-17T02:49:50.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.680+0000 7f1258e69640 1 -- 172.21.15.131:0/4175730515 shutdown_connections 2024-09-17T02:49:50.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.680+0000 7f1258e69640 1 -- 172.21.15.131:0/4175730515 >> 172.21.15.131:0/4175730515 conn(0x7f125406eea0 msgr2=0x7f125406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:50.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.680+0000 7f1258e69640 1 -- 172.21.15.131:0/4175730515 shutdown_connections 2024-09-17T02:49:50.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.680+0000 7f1258e69640 1 -- 172.21.15.131:0/4175730515 wait complete. 2024-09-17T02:49:50.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1258e69640 1 Processor -- start 2024-09-17T02:49:50.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1258e69640 1 -- start start 2024-09-17T02:49:50.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1258e69640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1254150a20 0x7f12540c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:50.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1258e69640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125415e650 0x7f125415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1258e69640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f12541f3660 0x7f12541f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1258e69640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f125406e2b0 con 0x7f1254150a20 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1258e69640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1254070ea0 con 0x7f125415e650 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1258e69640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f125406e430 con 0x7f12541f3660 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f125259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1254150a20 0x7f12540c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f125259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1254150a20 0x7f12540c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33724/0 (socket says 172.21.15.131:33724) 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f125259c640 1 -- 172.21.15.131:0/255428502 learned_addr learned my addr 172.21.15.131:0/255428502 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1249d9b640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125415e650 0x7f125415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1252d9d640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f12541f3660 0x7f12541f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f125259c640 1 -- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f12541f3660 msgr2=0x7f12541f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f125259c640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f12541f3660 0x7f12541f3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f125259c640 1 -- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125415e650 msgr2=0x7f125415ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f125259c640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125415e650 0x7f125415ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f125259c640 1 -- 172.21.15.131:0/255428502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12540c8d20 con 0x7f1254150a20 2024-09-17T02:49:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1249d9b640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125415e650 0x7f125415ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f125259c640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1254150a20 0x7f12540c9e80 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1244004eb0 tx=0x7f1244001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f124b7fe640 1 -- 172.21.15.131:0/255428502 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244015020 con 0x7f1254150a20 2024-09-17T02:49:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f124b7fe640 1 -- 172.21.15.131:0/255428502 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1244011960 con 0x7f1254150a20 2024-09-17T02:49:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1258e69640 1 -- 172.21.15.131:0/255428502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12540c93a0 con 0x7f1254150a20 2024-09-17T02:49:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f124b7fe640 1 -- 172.21.15.131:0/255428502 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1244011d90 con 0x7f1254150a20 2024-09-17T02:49:50.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1258e69640 1 -- 172.21.15.131:0/255428502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12540c9960 con 0x7f1254150a20 2024-09-17T02:49:50.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f1258e69640 1 -- 172.21.15.131:0/255428502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1254073c10 con 0x7f1254150a20 2024-09-17T02:49:50.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.684+0000 7f124b7fe640 1 -- 172.21.15.131:0/255428502 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1244018020 con 0x7f1254150a20 2024-09-17T02:49:50.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.688+0000 7f124b7fe640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1234073200 0x7f1234075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:50.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.688+0000 7f124b7fe640 1 -- 172.21.15.131:0/255428502 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 6455+0+0 (secure 0 0 0) 0x7f1244094f20 con 0x7f1254150a20 2024-09-17T02:49:50.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.688+0000 7f1249d9b640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1234073200 0x7f1234075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:50.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.688+0000 7f1249d9b640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1234073200 0x7f1234075710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f123c009aa0 tx=0x7f123c0091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:50.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.692+0000 7f124b7fe640 1 -- 172.21.15.131:0/255428502 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f124401f020 con 0x7f1254150a20 2024-09-17T02:49:50.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.880+0000 7f1258e69640 1 -- 172.21.15.131:0/255428502 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f125406fa20 con 0x7f1254150a20 2024-09-17T02:49:50.876 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:49:50.880+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:49:50.878 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.880+0000 7f124b7fe640 1 -- 172.21.15.131:0/255428502 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) ==== 161+0+0 (secure 0 0 0) 0x7f124405fce0 con 0x7f1254150a20 2024-09-17T02:49:50.878 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:49:50.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 -- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1234073200 msgr2=0x7f1234075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1234073200 0x7f1234075710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f123c009aa0 tx=0x7f123c0091e0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 -- 172.21.15.131:0/255428502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1254150a20 msgr2=0x7f12540c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:50.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1254150a20 0x7f12540c9e80 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1244004eb0 tx=0x7f1244001640 comp rx=0 tx=0).stop 2024-09-17T02:49:50.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 -- 172.21.15.131:0/255428502 shutdown_connections 2024-09-17T02:49:50.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1234073200 0x7f1234075710 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f12541f3660 0x7f12541f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f125415e650 0x7f125415ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 --2- 172.21.15.131:0/255428502 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1254150a20 0x7f12540c9e80 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:50.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 -- 172.21.15.131:0/255428502 >> 172.21.15.131:0/255428502 conn(0x7f125406eea0 msgr2=0x7f12540b5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:50.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 -- 172.21.15.131:0/255428502 shutdown_connections 2024-09-17T02:49:50.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:50.884+0000 7f1258e69640 1 -- 172.21.15.131:0/255428502 wait complete. 2024-09-17T02:49:50.886 INFO:tasks.ceph.mds.c.smithi131.stderr: -2315> 2024-09-17T02:49:14.512+0000 7f058d3a5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:49:50.886 INFO:tasks.ceph.mds.c.smithi131.stderr: -1924> 2024-09-17T02:49:16.132+0000 7f058939d640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | request=0 lock=0 dirtyparent=1 authpin=1 scrubqueue=0 0x558fe91b4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-17T02:49:50.902 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T02:49:50.959 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:49:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.048+0000 7fb0c713b640 1 -- 172.21.15.131:0/2218624347 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb0c0071b90 msgr2=0x7fb0c00cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.048+0000 7fb0c713b640 1 --2- 172.21.15.131:0/2218624347 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb0c0071b90 0x7fb0c00cf760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009c80 tx=0x7fb0b801cd10 comp rx=0 tx=0).stop 2024-09-17T02:49:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.048+0000 7fb0c713b640 1 -- 172.21.15.131:0/2218624347 shutdown_connections 2024-09-17T02:49:51.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.048+0000 7fb0c713b640 1 --2- 172.21.15.131:0/2218624347 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb0c0071b90 0x7fb0c00cf760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.048+0000 7fb0c713b640 1 --2- 172.21.15.131:0/2218624347 >> v2:172.21.15.189:3300/0 conn(0x7fb0c0074500 0x7fb0c015f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.048+0000 7fb0c713b640 1 -- 172.21.15.131:0/2218624347 >> 172.21.15.131:0/2218624347 conn(0x7fb0c00d13d0 msgr2=0x7fb0c00734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.048+0000 7fb0c713b640 1 -- 172.21.15.131:0/2218624347 shutdown_connections 2024-09-17T02:49:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.048+0000 7fb0c713b640 1 -- 172.21.15.131:0/2218624347 wait complete. 2024-09-17T02:49:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c713b640 1 Processor -- start 2024-09-17T02:49:51.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c713b640 1 -- start start 2024-09-17T02:49:51.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c713b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb0c0074500 0x7fb0c014b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:51.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c713b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c00cdbb0 0x7fb0c013b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:51.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c713b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0c0153ae0 0x7fb0c0153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:51.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c713b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c0169a40 con 0x7fb0c0153ae0 2024-09-17T02:49:51.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c713b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c01698c0 con 0x7fb0c00cdbb0 2024-09-17T02:49:51.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c713b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c0169bc0 con 0x7fb0c0074500 2024-09-17T02:49:51.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c56d8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0c0153ae0 0x7fb0c0153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:51.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c56d8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0c0153ae0 0x7fb0c0153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:33754/0 (socket says 172.21.15.131:33754) 2024-09-17T02:49:51.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c56d8640 1 -- 172.21.15.131:0/71875333 learned_addr learned my addr 172.21.15.131:0/71875333 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c4ed7640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb0c0074500 0x7fb0c014b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0bffff640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c00cdbb0 0x7fb0c013b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c56d8640 1 -- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb0c0074500 msgr2=0x7fb0c014b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c56d8640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb0c0074500 0x7fb0c014b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c56d8640 1 -- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c00cdbb0 msgr2=0x7fb0c013b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c56d8640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c00cdbb0 0x7fb0c013b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c56d8640 1 -- 172.21.15.131:0/71875333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c014b6d0 con 0x7fb0c0153ae0 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c56d8640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0c0153ae0 0x7fb0c0153ef0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009bd0 tx=0x7fb0b8005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0bd7fa640 1 -- 172.21.15.131:0/71875333 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80bc070 con 0x7fb0c0153ae0 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0bd7fa640 1 -- 172.21.15.131:0/71875333 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0b8004130 con 0x7fb0c0153ae0 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c713b640 1 -- 172.21.15.131:0/71875333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c0154430 con 0x7fb0c0153ae0 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0bd7fa640 1 -- 172.21.15.131:0/71875333 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80b2040 con 0x7fb0c0153ae0 2024-09-17T02:49:51.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c713b640 1 -- 172.21.15.131:0/71875333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c01f6850 con 0x7fb0c0153ae0 2024-09-17T02:49:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.052+0000 7fb0c713b640 1 -- 172.21.15.131:0/71875333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c00cf760 con 0x7fb0c0153ae0 2024-09-17T02:49:51.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.056+0000 7fb0bd7fa640 1 -- 172.21.15.131:0/71875333 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb0b8004580 con 0x7fb0c0153ae0 2024-09-17T02:49:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.056+0000 7fb0bd7fa640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb0a4073230 0x7fb0a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.056+0000 7fb0bd7fa640 1 -- 172.21.15.131:0/71875333 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6541+0+0 (secure 0 0 0) 0x7fb0b812e3b0 con 0x7fb0c0153ae0 2024-09-17T02:49:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.056+0000 7fb0c4ed7640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb0a4073230 0x7fb0a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:51.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.056+0000 7fb0c4ed7640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb0a4073230 0x7fb0a4075740 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009ad0 tx=0x7fb0b40091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:51.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.060+0000 7fb0bd7fa640 1 -- 172.21.15.131:0/71875333 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb0c0154430 con 0x7fb0c0153ae0 2024-09-17T02:49:51.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.248+0000 7fb0c713b640 1 -- 172.21.15.131:0/71875333 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb0c0150f10 con 0x7fb0c0153ae0 2024-09-17T02:49:51.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.884+0000 7fb0bd7fa640 1 -- 172.21.15.131:0/71875333 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) ==== 104+0+0 (secure 0 0 0) 0x7fb0b8016990 con 0x7fb0c0153ae0 2024-09-17T02:49:51.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 -- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb0a4073230 msgr2=0x7fb0a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb0a4073230 0x7fb0a4075740 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4009ad0 tx=0x7fb0b40091e0 comp rx=0 tx=0).stop 2024-09-17T02:49:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 -- 172.21.15.131:0/71875333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0c0153ae0 msgr2=0x7fb0c0153ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0c0153ae0 0x7fb0c0153ef0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009bd0 tx=0x7fb0b8005a60 comp rx=0 tx=0).stop 2024-09-17T02:49:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 -- 172.21.15.131:0/71875333 shutdown_connections 2024-09-17T02:49:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb0a4073230 0x7fb0a4075740 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb0c0153ae0 0x7fb0c0153ef0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb0c00cdbb0 0x7fb0c013b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 --2- 172.21.15.131:0/71875333 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb0c0074500 0x7fb0c014b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 -- 172.21.15.131:0/71875333 >> 172.21.15.131:0/71875333 conn(0x7fb0c00d13d0 msgr2=0x7fb0c00b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 -- 172.21.15.131:0/71875333 shutdown_connections 2024-09-17T02:49:51.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:51.888+0000 7fb0c713b640 1 -- 172.21.15.131:0/71875333 wait complete. 2024-09-17T02:49:51.911 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T02:49:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.080+0000 7fad8bddd640 1 -- 172.21.15.131:0/1735654444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad70004010 msgr2=0x7fad70004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.080+0000 7fad8bddd640 1 --2- 172.21.15.131:0/1735654444 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad70004010 0x7fad70004810 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fad7400c2e0 tx=0x7fad7401cf60 comp rx=0 tx=0).stop 2024-09-17T02:49:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.080+0000 7fad8a37a640 1 -- 172.21.15.131:0/1735654444 reap_dead start 2024-09-17T02:49:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.080+0000 7fad8bddd640 1 -- 172.21.15.131:0/1735654444 shutdown_connections 2024-09-17T02:49:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.080+0000 7fad8bddd640 1 -- 172.21.15.131:0/1735654444 >> 172.21.15.131:0/1735654444 conn(0x7fad840d05a0 msgr2=0x7fad840d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.080+0000 7fad8bddd640 1 -- 172.21.15.131:0/1735654444 shutdown_connections 2024-09-17T02:49:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.080+0000 7fad8bddd640 1 -- 172.21.15.131:0/1735654444 wait complete. 2024-09-17T02:49:52.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.080+0000 7fad8bddd640 1 Processor -- start 2024-09-17T02:49:52.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.080+0000 7fad8bddd640 1 -- start start 2024-09-17T02:49:52.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad8bddd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad841425d0 0x7fad841429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:52.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad8bddd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad84142f20 0x7fad841f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:52.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad8bddd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fad841f6fc0 0x7fad841f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad8bddd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fad841689f0 con 0x7fad84142f20 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad8bddd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad84168870 con 0x7fad841425d0 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad8bddd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fad84168b70 con 0x7fad841f6fc0 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad89378640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad84142f20 0x7fad841f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad89378640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad84142f20 0x7fad841f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55436/0 (socket says 172.21.15.131:55436) 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad89378640 1 -- 172.21.15.131:0/2385951599 learned_addr learned my addr 172.21.15.131:0/2385951599 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad89b79640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad841425d0 0x7fad841429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad8a37a640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fad841f6fc0 0x7fad841f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad89378640 1 -- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fad841f6fc0 msgr2=0x7fad841f73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad89378640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fad841f6fc0 0x7fad841f73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad89378640 1 -- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad841425d0 msgr2=0x7fad841429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:52.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad89378640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad841425d0 0x7fad841429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:52.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad89378640 1 -- 172.21.15.131:0/2385951599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad84143400 con 0x7fad84142f20 2024-09-17T02:49:52.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad89378640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad84142f20 0x7fad841f6a80 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fad7401b660 tx=0x7fad74006d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:52.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad7a7fc640 1 -- 172.21.15.131:0/2385951599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74006280 con 0x7fad84142f20 2024-09-17T02:49:52.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad7a7fc640 1 -- 172.21.15.131:0/2385951599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad74006420 con 0x7fad84142f20 2024-09-17T02:49:52.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad8bddd640 1 -- 172.21.15.131:0/2385951599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad841436e0 con 0x7fad84142f20 2024-09-17T02:49:52.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad7a7fc640 1 -- 172.21.15.131:0/2385951599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74003940 con 0x7fad84142f20 2024-09-17T02:49:52.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad8bddd640 1 -- 172.21.15.131:0/2385951599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad8413cde0 con 0x7fad84142f20 2024-09-17T02:49:52.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad8bddd640 1 -- 172.21.15.131:0/2385951599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad8406fe00 con 0x7fad84142f20 2024-09-17T02:49:52.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.084+0000 7fad7a7fc640 1 -- 172.21.15.131:0/2385951599 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad74003ae0 con 0x7fad84142f20 2024-09-17T02:49:52.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.088+0000 7fad7a7fc640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad68073230 0x7fad68075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:52.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.088+0000 7fad7a7fc640 1 -- 172.21.15.131:0/2385951599 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 6541+0+0 (secure 0 0 0) 0x7fad740b4020 con 0x7fad84142f20 2024-09-17T02:49:52.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.088+0000 7fad89b79640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad68073230 0x7fad68075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:52.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.088+0000 7fad89b79640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad68073230 0x7fad68075740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fad8413efa0 tx=0x7fad800091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:52.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.092+0000 7fad7a7fc640 1 -- 172.21.15.131:0/2385951599 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad74016d10 con 0x7fad84142f20 2024-09-17T02:49:52.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:52.260+0000 7fad8bddd640 1 -- 172.21.15.131:0/2385951599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fad8413c9e0 con 0x7fad84142f20 2024-09-17T02:49:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.056+0000 7fad7a7fc640 1 -- 172.21.15.131:0/2385951599 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7fad740f8fd0 con 0x7fad84142f20 2024-09-17T02:49:53.054 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-09-17T02:49:53.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.060+0000 7fad8bddd640 1 -- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad68073230 msgr2=0x7fad68075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.060+0000 7fad8bddd640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad68073230 0x7fad68075740 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fad8413efa0 tx=0x7fad800091e0 comp rx=0 tx=0).stop 2024-09-17T02:49:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.060+0000 7fad8bddd640 1 -- 172.21.15.131:0/2385951599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad84142f20 msgr2=0x7fad841f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.060+0000 7fad8bddd640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad84142f20 0x7fad841f6a80 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fad7401b660 tx=0x7fad74006d70 comp rx=0 tx=0).stop 2024-09-17T02:49:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.064+0000 7fad8bddd640 1 -- 172.21.15.131:0/2385951599 shutdown_connections 2024-09-17T02:49:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.064+0000 7fad8bddd640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad68073230 0x7fad68075740 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.064+0000 7fad8bddd640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fad841f6fc0 0x7fad841f73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.064+0000 7fad8bddd640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad84142f20 0x7fad841f6a80 secure :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fad7401b660 tx=0x7fad74006d70 comp rx=0 tx=0).stop 2024-09-17T02:49:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.064+0000 7fad8bddd640 1 --2- 172.21.15.131:0/2385951599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad841425d0 0x7fad841429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:53.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.064+0000 7fad8bddd640 1 -- 172.21.15.131:0/2385951599 >> 172.21.15.131:0/2385951599 conn(0x7fad840d05a0 msgr2=0x7fad8415db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.064+0000 7fad8bddd640 1 -- 172.21.15.131:0/2385951599 shutdown_connections 2024-09-17T02:49:53.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.064+0000 7fad8bddd640 1 -- 172.21.15.131:0/2385951599 wait complete. 2024-09-17T02:49:53.084 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T02:49:53.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.252+0000 7f8853408640 1 -- 172.21.15.131:0/2438590680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f884c06f700 msgr2=0x7f884c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:53.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.252+0000 7f8853408640 1 --2- 172.21.15.131:0/2438590680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f884c06f700 0x7f884c06fb10 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f884c070f60 tx=0x7f883c01c830 comp rx=0 tx=0).stop 2024-09-17T02:49:53.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.252+0000 7f8853408640 1 -- 172.21.15.131:0/2438590680 shutdown_connections 2024-09-17T02:49:53.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.252+0000 7f8853408640 1 --2- 172.21.15.131:0/2438590680 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f884c06f700 0x7f884c06fb10 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:53.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.252+0000 7f8853408640 1 --2- 172.21.15.131:0/2438590680 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f884c0cbed0 0x7f884c0cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:53.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.252+0000 7f8853408640 1 --2- 172.21.15.131:0/2438590680 >> v2:172.21.15.189:3300/0 conn(0x7f884c0b2ef0 0x7f884c0cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:53.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.252+0000 7f8853408640 1 -- 172.21.15.131:0/2438590680 >> 172.21.15.131:0/2438590680 conn(0x7f884c0d05a0 msgr2=0x7f884c0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:53.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.252+0000 7f8853408640 1 -- 172.21.15.131:0/2438590680 shutdown_connections 2024-09-17T02:49:53.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.252+0000 7f8853408640 1 -- 172.21.15.131:0/2438590680 wait complete. 2024-09-17T02:49:53.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.252+0000 7f8853408640 1 Processor -- start 2024-09-17T02:49:53.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.252+0000 7f8853408640 1 -- start start 2024-09-17T02:49:53.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8853408640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f884c06f700 0x7f884c152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:53.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8853408640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f884c0b2ef0 0x7f884c1f9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:53.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8853408640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f884c0cbed0 0x7f884c0cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:53.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8853408640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f884c0731c0 con 0x7f884c0b2ef0 2024-09-17T02:49:53.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8853408640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f884c15f450 con 0x7f884c0cbed0 2024-09-17T02:49:53.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8853408640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f884c073340 con 0x7f884c06f700 2024-09-17T02:49:53.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88509a3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f884c0b2ef0 0x7f884c1f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:53.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88509a3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f884c0b2ef0 0x7f884c1f9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55460/0 (socket says 172.21.15.131:55460) 2024-09-17T02:49:53.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88509a3640 1 -- 172.21.15.131:0/421742604 learned_addr learned my addr 172.21.15.131:0/421742604 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88519a5640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f884c0cbed0 0x7f884c0cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88511a4640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f884c06f700 0x7f884c152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88509a3640 1 -- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f884c06f700 msgr2=0x7f884c152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88509a3640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f884c06f700 0x7f884c152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88509a3640 1 -- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f884c0cbed0 msgr2=0x7f884c0cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88509a3640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f884c0cbed0 0x7f884c0cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88509a3640 1 -- 172.21.15.131:0/421742604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c0cbcb0 con 0x7f884c0b2ef0 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88519a5640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f884c0cbed0 0x7f884c0cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88509a3640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f884c0b2ef0 0x7f884c1f9620 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f883c009850 tx=0x7f883c0a6750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8841ffb640 1 -- 172.21.15.131:0/421742604 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c0bd070 con 0x7f884c0b2ef0 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8841ffb640 1 -- 172.21.15.131:0/421742604 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f883c004430 con 0x7f884c0b2ef0 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8853408640 1 -- 172.21.15.131:0/421742604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884c153e60 con 0x7f884c0b2ef0 2024-09-17T02:49:53.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8841ffb640 1 -- 172.21.15.131:0/421742604 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883c005290 con 0x7f884c0b2ef0 2024-09-17T02:49:53.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8853408640 1 -- 172.21.15.131:0/421742604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884c152210 con 0x7f884c0b2ef0 2024-09-17T02:49:53.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8853408640 1 -- 172.21.15.131:0/421742604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884c0cf6d0 con 0x7f884c0b2ef0 2024-09-17T02:49:53.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8841ffb640 1 -- 172.21.15.131:0/421742604 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f883c0c0070 con 0x7f884c0b2ef0 2024-09-17T02:49:53.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8841ffb640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8830073230 0x7f8830075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:53.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f8841ffb640 1 -- 172.21.15.131:0/421742604 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 6074+0+0 (secure 0 0 0) 0x7f883c12e3e0 con 0x7f884c0b2ef0 2024-09-17T02:49:53.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.256+0000 7f88511a4640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8830073230 0x7f8830075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:53.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.260+0000 7f88511a4640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8830073230 0x7f8830075740 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8848009980 tx=0x7f8848005ca0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:53.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.260+0000 7f8841ffb640 1 -- 172.21.15.131:0/421742604 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f883c0164b0 con 0x7f884c0b2ef0 2024-09-17T02:49:53.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:53.440+0000 7f8853408640 1 -- 172.21.15.131:0/421742604 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f884c154e30 con 0x7f884c0b2ef0 2024-09-17T02:49:54.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.072+0000 7f8841ffb640 1 -- 172.21.15.131:0/421742604 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7f883c0f9290 con 0x7f884c0b2ef0 2024-09-17T02:49:54.068 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-09-17T02:49:54.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 -- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8830073230 msgr2=0x7f8830075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:54.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8830073230 0x7f8830075740 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8848009980 tx=0x7f8848005ca0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 -- 172.21.15.131:0/421742604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f884c0b2ef0 msgr2=0x7f884c1f9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:54.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f884c0b2ef0 0x7f884c1f9620 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f883c009850 tx=0x7f883c0a6750 comp rx=0 tx=0).stop 2024-09-17T02:49:54.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 -- 172.21.15.131:0/421742604 shutdown_connections 2024-09-17T02:49:54.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8830073230 0x7f8830075740 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f884c0cbed0 0x7f884c0cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f884c0b2ef0 0x7f884c1f9620 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 --2- 172.21.15.131:0/421742604 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f884c06f700 0x7f884c152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 -- 172.21.15.131:0/421742604 >> 172.21.15.131:0/421742604 conn(0x7f884c0d05a0 msgr2=0x7f884c0cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:54.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 -- 172.21.15.131:0/421742604 shutdown_connections 2024-09-17T02:49:54.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.076+0000 7f8853408640 1 -- 172.21.15.131:0/421742604 wait complete. 2024-09-17T02:49:54.106 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T02:49:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.268+0000 7f8df7e51640 1 -- 172.21.15.131:0/874540140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df0073970 msgr2=0x7f8df006dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.268+0000 7f8df7e51640 1 --2- 172.21.15.131:0/874540140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df0073970 0x7f8df006dea0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f8df006a1c0 tx=0x7f8de001db00 comp rx=0 tx=0).stop 2024-09-17T02:49:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.268+0000 7f8df7e51640 1 -- 172.21.15.131:0/874540140 shutdown_connections 2024-09-17T02:49:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.268+0000 7f8df7e51640 1 --2- 172.21.15.131:0/874540140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df0073970 0x7f8df006dea0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.268+0000 7f8df7e51640 1 --2- 172.21.15.131:0/874540140 >> v2:172.21.15.189:3300/0 conn(0x7f8df0155240 0x7f8df0073430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.268+0000 7f8df7e51640 1 -- 172.21.15.131:0/874540140 >> 172.21.15.131:0/874540140 conn(0x7f8df006eea0 msgr2=0x7f8df006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.268+0000 7f8df7e51640 1 -- 172.21.15.131:0/874540140 shutdown_connections 2024-09-17T02:49:54.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.268+0000 7f8df7e51640 1 -- 172.21.15.131:0/874540140 wait complete. 2024-09-17T02:49:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df7e51640 1 Processor -- start 2024-09-17T02:49:54.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df7e51640 1 -- start start 2024-09-17T02:49:54.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df7e51640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8df006e3e0 0x7f8df01fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:54.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df7e51640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df0073970 0x7f8df01f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:54.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df7e51640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8df0155240 0x7f8df00c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:54.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df7e51640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8df0167d80 con 0x7f8df0073970 2024-09-17T02:49:54.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df7e51640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8df015f640 con 0x7f8df0155240 2024-09-17T02:49:54.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df7e51640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8df0167f00 con 0x7f8df006e3e0 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df53ec640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df0073970 0x7f8df01f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df53ec640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df0073970 0x7f8df01f9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55480/0 (socket says 172.21.15.131:55480) 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df53ec640 1 -- 172.21.15.131:0/3292977417 learned_addr learned my addr 172.21.15.131:0/3292977417 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df63ee640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8df0155240 0x7f8df00c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df5bed640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8df006e3e0 0x7f8df01fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df53ec640 1 -- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8df006e3e0 msgr2=0x7f8df01fca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df53ec640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8df006e3e0 0x7f8df01fca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df53ec640 1 -- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8df0155240 msgr2=0x7f8df00c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df53ec640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8df0155240 0x7f8df00c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df53ec640 1 -- 172.21.15.131:0/3292977417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df01f9400 con 0x7f8df0073970 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df63ee640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8df0155240 0x7f8df00c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df5bed640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8df006e3e0 0x7f8df01fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df53ec640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df0073970 0x7f8df01f9e90 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f8de0000c10 tx=0x7f8de00a6870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8de67fc640 1 -- 172.21.15.131:0/3292977417 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de00bd070 con 0x7f8df0073970 2024-09-17T02:49:54.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8de67fc640 1 -- 172.21.15.131:0/3292977417 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8de00053e0 con 0x7f8df0073970 2024-09-17T02:49:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df7e51640 1 -- 172.21.15.131:0/3292977417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df01fd3c0 con 0x7f8df0073970 2024-09-17T02:49:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8de67fc640 1 -- 172.21.15.131:0/3292977417 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de00b3040 con 0x7f8df0073970 2024-09-17T02:49:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df7e51640 1 -- 172.21.15.131:0/3292977417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df01f9690 con 0x7f8df0073970 2024-09-17T02:49:54.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.272+0000 7f8df7e51640 1 -- 172.21.15.131:0/3292977417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df01fd8e0 con 0x7f8df0073970 2024-09-17T02:49:54.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.276+0000 7f8de67fc640 1 -- 172.21.15.131:0/3292977417 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8de00046d0 con 0x7f8df0073970 2024-09-17T02:49:54.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.276+0000 7f8de67fc640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dd4073200 0x7f8dd4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:54.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.276+0000 7f8de67fc640 1 -- 172.21.15.131:0/3292977417 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5507+0+0 (secure 0 0 0) 0x7f8de012e190 con 0x7f8df0073970 2024-09-17T02:49:54.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.276+0000 7f8df5bed640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dd4073200 0x7f8dd4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:54.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.276+0000 7f8df5bed640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dd4073200 0x7f8dd4075710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8dec005fd0 tx=0x7f8dec005dc0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:54.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.280+0000 7f8de67fc640 1 -- 172.21.15.131:0/3292977417 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8df01fd8e0 con 0x7f8df0073970 2024-09-17T02:49:54.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.448+0000 7f8df7e51640 1 -- 172.21.15.131:0/3292977417 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f8df01fa3d0 con 0x7f8df0073970 2024-09-17T02:49:54.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.456+0000 7f8de67fc640 1 -- 172.21.15.131:0/3292977417 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v41) ==== 71+0+0 (secure 0 0 0) 0x7f8df01fa3d0 con 0x7f8df0073970 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 -- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dd4073200 msgr2=0x7f8dd4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dd4073200 0x7f8dd4075710 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8dec005fd0 tx=0x7f8dec005dc0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 -- 172.21.15.131:0/3292977417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df0073970 msgr2=0x7f8df01f9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df0073970 0x7f8df01f9e90 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f8de0000c10 tx=0x7f8de00a6870 comp rx=0 tx=0).stop 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 -- 172.21.15.131:0/3292977417 shutdown_connections 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8dd4073200 0x7f8dd4075710 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8df0155240 0x7f8df00c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8df0073970 0x7f8df01f9e90 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 --2- 172.21.15.131:0/3292977417 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8df006e3e0 0x7f8df01fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 -- 172.21.15.131:0/3292977417 >> 172.21.15.131:0/3292977417 conn(0x7f8df006eea0 msgr2=0x7f8df006d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 -- 172.21.15.131:0/3292977417 shutdown_connections 2024-09-17T02:49:54.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.460+0000 7f8df7e51640 1 -- 172.21.15.131:0/3292977417 wait complete. 2024-09-17T02:49:54.478 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-09-17T02:49:54.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.640+0000 7f26ea276640 1 -- 172.21.15.131:0/2951799366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26e40cdbf0 msgr2=0x7f26e40ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:54.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.640+0000 7f26ea276640 1 --2- 172.21.15.131:0/2951799366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26e40cdbf0 0x7f26e40ce000 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f26e40757f0 tx=0x7f26dc01ccf0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.640+0000 7f26ea276640 1 -- 172.21.15.131:0/2951799366 shutdown_connections 2024-09-17T02:49:54.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.640+0000 7f26ea276640 1 --2- 172.21.15.131:0/2951799366 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26e40cdbf0 0x7f26e40ce000 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.640+0000 7f26ea276640 1 --2- 172.21.15.131:0/2951799366 >> v2:172.21.15.189:3300/0 conn(0x7f26e40c4ac0 0x7f26e40c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.640+0000 7f26ea276640 1 --2- 172.21.15.131:0/2951799366 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26e406e2c0 0x7f26e40c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.640+0000 7f26ea276640 1 -- 172.21.15.131:0/2951799366 >> 172.21.15.131:0/2951799366 conn(0x7f26e406eea0 msgr2=0x7f26e406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:54.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.640+0000 7f26ea276640 1 -- 172.21.15.131:0/2951799366 shutdown_connections 2024-09-17T02:49:54.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.640+0000 7f26ea276640 1 -- 172.21.15.131:0/2951799366 wait complete. 2024-09-17T02:49:54.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26ea276640 1 Processor -- start 2024-09-17T02:49:54.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26ea276640 1 -- start start 2024-09-17T02:49:54.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26ea276640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26e406e2c0 0x7f26e40c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26ea276640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26e40c4ac0 0x7f26e41fa900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26ea276640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26e40cdbf0 0x7f26e4156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26ea276640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f26e40ceed0 con 0x7f26e40cdbf0 2024-09-17T02:49:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26ea276640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f26e40ced50 con 0x7f26e40c4ac0 2024-09-17T02:49:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26ea276640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f26e40cf050 con 0x7f26e406e2c0 2024-09-17T02:49:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e8813640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26e40cdbf0 0x7f26e4156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e8813640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26e40cdbf0 0x7f26e4156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55494/0 (socket says 172.21.15.131:55494) 2024-09-17T02:49:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e8813640 1 -- 172.21.15.131:0/908493602 learned_addr learned my addr 172.21.15.131:0/908493602 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e3fff640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26e406e2c0 0x7f26e40c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e37fe640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26e40c4ac0 0x7f26e41fa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:54.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e8813640 1 -- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26e406e2c0 msgr2=0x7f26e40c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e8813640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26e406e2c0 0x7f26e40c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e8813640 1 -- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26e40c4ac0 msgr2=0x7f26e41fa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e8813640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26e40c4ac0 0x7f26e41fa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e8813640 1 -- 172.21.15.131:0/908493602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e415e740 con 0x7f26e40cdbf0 2024-09-17T02:49:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e3fff640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26e406e2c0 0x7f26e40c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:49:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e8813640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26e40cdbf0 0x7f26e4156860 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f26dc01b3a0 tx=0x7f26dc005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e0ff9640 1 -- 172.21.15.131:0/908493602 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0bb070 con 0x7f26e40cdbf0 2024-09-17T02:49:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e0ff9640 1 -- 172.21.15.131:0/908493602 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26dc004d40 con 0x7f26e40cdbf0 2024-09-17T02:49:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26ea276640 1 -- 172.21.15.131:0/908493602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e41fae40 con 0x7f26e40cdbf0 2024-09-17T02:49:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26e0ff9640 1 -- 172.21.15.131:0/908493602 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0b2040 con 0x7f26e40cdbf0 2024-09-17T02:49:54.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26ea276640 1 -- 172.21.15.131:0/908493602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e415ed00 con 0x7f26e40cdbf0 2024-09-17T02:49:54.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.644+0000 7f26ea276640 1 -- 172.21.15.131:0/908493602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e40c4ed0 con 0x7f26e40cdbf0 2024-09-17T02:49:54.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.648+0000 7f26e0ff9640 1 -- 172.21.15.131:0/908493602 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f26dc0be070 con 0x7f26e40cdbf0 2024-09-17T02:49:54.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.648+0000 7f26e0ff9640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26d0073200 0x7f26d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:54.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.648+0000 7f26e0ff9640 1 -- 172.21.15.131:0/908493602 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5507+0+0 (secure 0 0 0) 0x7f26dc12e1c0 con 0x7f26e40cdbf0 2024-09-17T02:49:54.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.648+0000 7f26e3fff640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26d0073200 0x7f26d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:54.645 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.648+0000 7f26e3fff640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26d0073200 0x7f26d0075710 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f26d4005fd0 tx=0x7f26d4005e40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:54.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.652+0000 7f26e0ff9640 1 -- 172.21.15.131:0/908493602 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f26dc016970 con 0x7f26e40cdbf0 2024-09-17T02:49:54.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:54.884+0000 7f26ea276640 1 -- 172.21.15.131:0/908493602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f26e415e4b0 con 0x7f26e40cdbf0 2024-09-17T02:49:55.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.072+0000 7f26e0ff9640 1 -- 172.21.15.131:0/908493602 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v408) ==== 167+0+0 (secure 0 0 0) 0x7f26dc0f9410 con 0x7f26e40cdbf0 2024-09-17T02:49:55.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 -- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26d0073200 msgr2=0x7f26d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26d0073200 0x7f26d0075710 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f26d4005fd0 tx=0x7f26d4005e40 comp rx=0 tx=0).stop 2024-09-17T02:49:55.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 -- 172.21.15.131:0/908493602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26e40cdbf0 msgr2=0x7f26e4156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26e40cdbf0 0x7f26e4156860 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f26dc01b3a0 tx=0x7f26dc005b00 comp rx=0 tx=0).stop 2024-09-17T02:49:55.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 -- 172.21.15.131:0/908493602 shutdown_connections 2024-09-17T02:49:55.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26d0073200 0x7f26d0075710 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26e40cdbf0 0x7f26e4156860 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f26e40c4ac0 0x7f26e41fa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 --2- 172.21.15.131:0/908493602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26e406e2c0 0x7f26e40c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 -- 172.21.15.131:0/908493602 >> 172.21.15.131:0/908493602 conn(0x7f26e406eea0 msgr2=0x7f26e4158c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:55.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 -- 172.21.15.131:0/908493602 shutdown_connections 2024-09-17T02:49:55.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.076+0000 7f26ea276640 1 -- 172.21.15.131:0/908493602 wait complete. 2024-09-17T02:49:55.099 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-09-17T02:49:55.099 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-17T02:49:55.100 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:49:55.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.256+0000 7fdb07a41640 1 -- 172.21.15.131:0/800163466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb000cfb70 msgr2=0x7fdb00161980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.256+0000 7fdb07a41640 1 --2- 172.21.15.131:0/800163466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb000cfb70 0x7fdb00161980 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fdb0015fe70 tx=0x7fdafc01c6b0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.256+0000 7fdb07a41640 1 -- 172.21.15.131:0/800163466 shutdown_connections 2024-09-17T02:49:55.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.256+0000 7fdb07a41640 1 --2- 172.21.15.131:0/800163466 >> v2:172.21.15.189:3300/0 conn(0x7fdb00162330 0x7fdb00162740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.256+0000 7fdb07a41640 1 --2- 172.21.15.131:0/800163466 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb000cfb70 0x7fdb00161980 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.256+0000 7fdb07a41640 1 -- 172.21.15.131:0/800163466 >> 172.21.15.131:0/800163466 conn(0x7fdb000d1ba0 msgr2=0x7fdb000d4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:55.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.256+0000 7fdb07a41640 1 -- 172.21.15.131:0/800163466 shutdown_connections 2024-09-17T02:49:55.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.256+0000 7fdb07a41640 1 -- 172.21.15.131:0/800163466 wait complete. 2024-09-17T02:49:55.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.256+0000 7fdb07a41640 1 Processor -- start 2024-09-17T02:49:55.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.256+0000 7fdb07a41640 1 -- start start 2024-09-17T02:49:55.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb07a41640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb000cfb70 0x7fdb0014afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb07a41640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb00155050 0x7fdb001fc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb07a41640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb00162330 0x7fdb001fc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb07a41640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00160a40 con 0x7fdb000cfb70 2024-09-17T02:49:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb07a41640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001608c0 con 0x7fdb00155050 2024-09-17T02:49:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb07a41640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdb00160bc0 con 0x7fdb00162330 2024-09-17T02:49:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb057dd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb000cfb70 0x7fdb0014afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb057dd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb000cfb70 0x7fdb0014afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55512/0 (socket says 172.21.15.131:55512) 2024-09-17T02:49:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb057dd640 1 -- 172.21.15.131:0/1675078468 learned_addr learned my addr 172.21.15.131:0/1675078468 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb04fdc640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb00155050 0x7fdb001fc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb05fde640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb00162330 0x7fdb001fc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb057dd640 1 -- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb00162330 msgr2=0x7fdb001fc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb057dd640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb00162330 0x7fdb001fc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb057dd640 1 -- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb00155050 msgr2=0x7fdb001fc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb057dd640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb00155050 0x7fdb001fc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb057dd640 1 -- 172.21.15.131:0/1675078468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0006e900 con 0x7fdb000cfb70 2024-09-17T02:49:55.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb05fde640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb00162330 0x7fdb001fc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:55.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb057dd640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb000cfb70 0x7fdb0014afa0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009850 tx=0x7fdafc005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:55.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdaf5ffb640 1 -- 172.21.15.131:0/1675078468 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0b2040 con 0x7fdb000cfb70 2024-09-17T02:49:55.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdaf5ffb640 1 -- 172.21.15.131:0/1675078468 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdafc004330 con 0x7fdb000cfb70 2024-09-17T02:49:55.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb07a41640 1 -- 172.21.15.131:0/1675078468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb001fcce0 con 0x7fdb000cfb70 2024-09-17T02:49:55.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdaf5ffb640 1 -- 172.21.15.131:0/1675078468 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc005290 con 0x7fdb000cfb70 2024-09-17T02:49:55.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb07a41640 1 -- 172.21.15.131:0/1675078468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0014b4e0 con 0x7fdb000cfb70 2024-09-17T02:49:55.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdb07a41640 1 -- 172.21.15.131:0/1675078468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb001fcb00 con 0x7fdb000cfb70 2024-09-17T02:49:55.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.260+0000 7fdaf5ffb640 1 -- 172.21.15.131:0/1675078468 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdafc0bd060 con 0x7fdb000cfb70 2024-09-17T02:49:55.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.264+0000 7fdaf5ffb640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 0x7fdae4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.264+0000 7fdaf5ffb640 1 -- 172.21.15.131:0/1675078468 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5507+0+0 (secure 0 0 0) 0x7fdafc12d5c0 con 0x7fdb000cfb70 2024-09-17T02:49:55.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.264+0000 7fdb04fdc640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 0x7fdae4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.264+0000 7fdb04fdc640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 0x7fdae4075710 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00058d0 tx=0x7fdaf0005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:55.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.268+0000 7fdaf5ffb640 1 -- 172.21.15.131:0/1675078468 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdb001fcb00 con 0x7fdb000cfb70 2024-09-17T02:49:55.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.424+0000 7fdb07a41640 1 -- 172.21.15.131:0/1675078468 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb001610f0 con 0x7fdb000cfb70 2024-09-17T02:49:55.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdaf5ffb640 1 -- 172.21.15.131:0/1675078468 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+13620 (secure 0 0 0) 0x7fdb00162c80 con 0x7fdb000cfb70 2024-09-17T02:49:55.423 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:55.423 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":99,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:49:54.062693+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6835/1957540940":"2024-09-18T02:49:50.883859+0000","172.21.15.131:6834/1957540940":"2024-09-18T02:49:50.883859+0000","172.21.15.131:6835/3013984008":"2024-09-18T02:49:07.494740+0000","172.21.15.131:6834/3013984008":"2024-09-18T02:49:07.494740+0000","172.21.15.131:6835/2766162522":"2024-09-18T02:47:48.093626+0000","172.21.15.131:6834/2766162522":"2024-09-18T02:47:48.093626+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:49:55.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 -- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 msgr2=0x7fdae4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 0x7fdae4075710 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00058d0 tx=0x7fdaf0005860 comp rx=0 tx=0).stop 2024-09-17T02:49:55.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 -- 172.21.15.131:0/1675078468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb000cfb70 msgr2=0x7fdb0014afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb000cfb70 0x7fdb0014afa0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009850 tx=0x7fdafc005e70 comp rx=0 tx=0).stop 2024-09-17T02:49:55.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 -- 172.21.15.131:0/1675078468 shutdown_connections 2024-09-17T02:49:55.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdae4073200 0x7fdae4075710 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdb00162330 0x7fdb001fc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdb00155050 0x7fdb001fc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 --2- 172.21.15.131:0/1675078468 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdb000cfb70 0x7fdb0014afa0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 -- 172.21.15.131:0/1675078468 >> 172.21.15.131:0/1675078468 conn(0x7fdb000d1ba0 msgr2=0x7fdb00151550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:55.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 -- 172.21.15.131:0/1675078468 shutdown_connections 2024-09-17T02:49:55.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.428+0000 7fdb07a41640 1 -- 172.21.15.131:0/1675078468 wait complete. 2024-09-17T02:49:55.445 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T02:40:41.163614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T02:49:55.445 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T02:49:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.604+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2832914660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89e4004d50 msgr2=0x7f89e4005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.604+0000 7f89ff8e0640 1 --2- 172.21.15.131:0/2832914660 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89e4004d50 0x7f89e4005220 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f89f80cfe30 tx=0x7f89f001d5b0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.604+0000 7f89fde7d640 1 -- 172.21.15.131:0/2832914660 reap_dead start 2024-09-17T02:49:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.604+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2832914660 shutdown_connections 2024-09-17T02:49:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.604+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2832914660 >> 172.21.15.131:0/2832914660 conn(0x7f89f806eea0 msgr2=0x7f89f806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:55.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.604+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2832914660 shutdown_connections 2024-09-17T02:49:55.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.604+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2832914660 wait complete. 2024-09-17T02:49:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89ff8e0640 1 Processor -- start 2024-09-17T02:49:55.604 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89ff8e0640 1 -- start start 2024-09-17T02:49:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89ff8e0640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f89e4004d50 0x7f89f815a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89ff8e0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89f81546c0 0x7f89f8154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89ff8e0640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89f81561a0 0x7f89f81565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89ff8e0640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f89f80cbe80 con 0x7f89f81546c0 2024-09-17T02:49:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89ff8e0640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f89f80cbd00 con 0x7f89f81561a0 2024-09-17T02:49:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89ff8e0640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f89f80cc000 con 0x7f89e4004d50 2024-09-17T02:49:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89fce7b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89f81546c0 0x7f89f8154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89fce7b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89f81546c0 0x7f89f8154ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55538/0 (socket says 172.21.15.131:55538) 2024-09-17T02:49:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89fce7b640 1 -- 172.21.15.131:0/2380860288 learned_addr learned my addr 172.21.15.131:0/2380860288 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89fde7d640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89f81561a0 0x7f89f81565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89fd67c640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f89e4004d50 0x7f89f815a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89fce7b640 1 -- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f89e4004d50 msgr2=0x7f89f815a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89fce7b640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f89e4004d50 0x7f89f815a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89fce7b640 1 -- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89f81561a0 msgr2=0x7f89f81565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89fce7b640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89f81561a0 0x7f89f81565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89fce7b640 1 -- 172.21.15.131:0/2380860288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f81f3210 con 0x7f89f81546c0 2024-09-17T02:49:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89fce7b640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89f81546c0 0x7f89f8154ad0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f89e80066c0 tx=0x7f89e8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89edffb640 1 -- 172.21.15.131:0/2380860288 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8015020 con 0x7f89f81546c0 2024-09-17T02:49:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89edffb640 1 -- 172.21.15.131:0/2380860288 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89e8011670 con 0x7f89f81546c0 2024-09-17T02:49:55.608 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.608+0000 7f89edffb640 1 -- 172.21.15.131:0/2380860288 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8011ac0 con 0x7f89f81546c0 2024-09-17T02:49:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.612+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2380860288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f81f34f0 con 0x7f89f81546c0 2024-09-17T02:49:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.612+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2380860288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f81f3ab0 con 0x7f89f81546c0 2024-09-17T02:49:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.612+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2380860288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f8075b80 con 0x7f89f81546c0 2024-09-17T02:49:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.612+0000 7f89edffb640 1 -- 172.21.15.131:0/2380860288 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f89e8018020 con 0x7f89f81546c0 2024-09-17T02:49:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.612+0000 7f89edffb640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89dc073200 0x7f89dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.612+0000 7f89edffb640 1 -- 172.21.15.131:0/2380860288 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5507+0+0 (secure 0 0 0) 0x7f89e8094b90 con 0x7f89f81546c0 2024-09-17T02:49:55.609 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.612+0000 7f89fd67c640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89dc073200 0x7f89dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.612+0000 7f89fd67c640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89dc073200 0x7f89dc075710 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f89f4001060 tx=0x7f89f4000ee0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:55.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.616+0000 7f89edffb640 1 -- 172.21.15.131:0/2380860288 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f89e801f020 con 0x7f89f81546c0 2024-09-17T02:49:55.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2380860288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f89f80cc590 con 0x7f89f81546c0 2024-09-17T02:49:55.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89edffb640 1 -- 172.21.15.131:0/2380860288 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f89e805fc70 con 0x7f89f81546c0 2024-09-17T02:49:55.796 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-09-17T02:49:55.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89dc073200 msgr2=0x7f89dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89dc073200 0x7f89dc075710 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f89f4001060 tx=0x7f89f4000ee0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2380860288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89f81546c0 msgr2=0x7f89f8154ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89f81546c0 0x7f89f8154ad0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f89e80066c0 tx=0x7f89e8001640 comp rx=0 tx=0).stop 2024-09-17T02:49:55.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2380860288 shutdown_connections 2024-09-17T02:49:55.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f89dc073200 0x7f89dc075710 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f89f81561a0 0x7f89f81565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f89f81546c0 0x7f89f8154ad0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 --2- 172.21.15.131:0/2380860288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f89e4004d50 0x7f89f815a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2380860288 >> 172.21.15.131:0/2380860288 conn(0x7f89f806eea0 msgr2=0x7f89f80cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:55.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2380860288 shutdown_connections 2024-09-17T02:49:55.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.800+0000 7f89ff8e0640 1 -- 172.21.15.131:0/2380860288 wait complete. 2024-09-17T02:49:55.819 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-09-17T02:49:55.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.972+0000 7f3a0bf05640 1 -- 172.21.15.131:0/1233045898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a040ced00 msgr2=0x7f3a0406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.972+0000 7f3a0bf05640 1 --2- 172.21.15.131:0/1233045898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a040ced00 0x7f3a0406dea0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009840 tx=0x7f3a0001c820 comp rx=0 tx=0).stop 2024-09-17T02:49:55.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.972+0000 7f3a0bf05640 1 -- 172.21.15.131:0/1233045898 shutdown_connections 2024-09-17T02:49:55.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.972+0000 7f3a0bf05640 1 --2- 172.21.15.131:0/1233045898 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a0415c5b0 0x7f3a040cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.972+0000 7f3a0bf05640 1 --2- 172.21.15.131:0/1233045898 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a040ced00 0x7f3a0406dea0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.972+0000 7f3a0bf05640 1 -- 172.21.15.131:0/1233045898 >> 172.21.15.131:0/1233045898 conn(0x7f3a0406eea0 msgr2=0x7f3a0406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:55.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.972+0000 7f3a0bf05640 1 -- 172.21.15.131:0/1233045898 shutdown_connections 2024-09-17T02:49:55.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.972+0000 7f3a0bf05640 1 -- 172.21.15.131:0/1233045898 wait complete. 2024-09-17T02:49:55.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0bf05640 1 Processor -- start 2024-09-17T02:49:55.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0bf05640 1 -- start start 2024-09-17T02:49:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0bf05640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a0406e3e0 0x7f3a041fa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0bf05640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a040ced00 0x7f3a0414c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0bf05640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a0415c5b0 0x7f3a04155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0bf05640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041686c0 con 0x7f3a040ced00 2024-09-17T02:49:55.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0bf05640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04168540 con 0x7f3a0415c5b0 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0bf05640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3a04168840 con 0x7f3a0406e3e0 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a094a0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a040ced00 0x7f3a0414c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a094a0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a040ced00 0x7f3a0414c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55556/0 (socket says 172.21.15.131:55556) 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a094a0640 1 -- 172.21.15.131:0/3758776759 learned_addr learned my addr 172.21.15.131:0/3758776759 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a09ca1640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a0406e3e0 0x7f3a041fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0a4a2640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a0415c5b0 0x7f3a04155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a094a0640 1 -- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a0406e3e0 msgr2=0x7f3a041fa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a094a0640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a0406e3e0 0x7f3a041fa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a094a0640 1 -- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a0415c5b0 msgr2=0x7f3a04155200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a094a0640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a0415c5b0 0x7f3a04155200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a094a0640 1 -- 172.21.15.131:0/3758776759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a041f6550 con 0x7f3a040ced00 2024-09-17T02:49:55.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0a4a2640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a0415c5b0 0x7f3a04155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a094a0640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a040ced00 0x7f3a0414c120 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f39f400c450 tx=0x7f39f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f39fa7fc640 1 -- 172.21.15.131:0/3758776759 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4016020 con 0x7f3a040ced00 2024-09-17T02:49:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f39fa7fc640 1 -- 172.21.15.131:0/3758776759 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39f4012920 con 0x7f3a040ced00 2024-09-17T02:49:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0bf05640 1 -- 172.21.15.131:0/3758776759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0415d890 con 0x7f3a040ced00 2024-09-17T02:49:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f39fa7fc640 1 -- 172.21.15.131:0/3758776759 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4012d50 con 0x7f3a040ced00 2024-09-17T02:49:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.976+0000 7f3a0bf05640 1 -- 172.21.15.131:0/3758776759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0415de10 con 0x7f3a040ced00 2024-09-17T02:49:55.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.980+0000 7f3a0bf05640 1 -- 172.21.15.131:0/3758776759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a0406dea0 con 0x7f3a040ced00 2024-09-17T02:49:55.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.980+0000 7f39fa7fc640 1 -- 172.21.15.131:0/3758776759 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f39f4019020 con 0x7f3a040ced00 2024-09-17T02:49:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.980+0000 7f39fa7fc640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f39e8073200 0x7f39e8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.980+0000 7f39fa7fc640 1 -- 172.21.15.131:0/3758776759 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5507+0+0 (secure 0 0 0) 0x7f39f4094540 con 0x7f3a040ced00 2024-09-17T02:49:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.980+0000 7f3a09ca1640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f39e8073200 0x7f39e8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:55.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.980+0000 7f3a09ca1640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f39e8073200 0x7f39e8075710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3a000af000 tx=0x7f3a0001b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:55.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:55.984+0000 7f39fa7fc640 1 -- 172.21.15.131:0/3758776759 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f39f4020030 con 0x7f3a040ced00 2024-09-17T02:49:56.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:56.212+0000 7f3a0bf05640 1 -- 172.21.15.131:0/3758776759 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f3a0415b370 con 0x7f3a040ced00 2024-09-17T02:49:57.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.084+0000 7f39fa7fc640 1 -- 172.21.15.131:0/3758776759 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v410) ==== 149+0+0 (secure 0 0 0) 0x7f3a041f6550 con 0x7f3a040ced00 2024-09-17T02:49:57.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 -- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f39e8073200 msgr2=0x7f39e8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f39e8073200 0x7f39e8075710 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3a000af000 tx=0x7f3a0001b060 comp rx=0 tx=0).stop 2024-09-17T02:49:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 -- 172.21.15.131:0/3758776759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a040ced00 msgr2=0x7f3a0414c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a040ced00 0x7f3a0414c120 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f39f400c450 tx=0x7f39f400c910 comp rx=0 tx=0).stop 2024-09-17T02:49:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 -- 172.21.15.131:0/3758776759 shutdown_connections 2024-09-17T02:49:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f39e8073200 0x7f39e8075710 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a0415c5b0 0x7f3a04155200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a040ced00 0x7f3a0414c120 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 --2- 172.21.15.131:0/3758776759 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a0406e3e0 0x7f3a041fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 -- 172.21.15.131:0/3758776759 >> 172.21.15.131:0/3758776759 conn(0x7f3a0406eea0 msgr2=0x7f3a0415a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 -- 172.21.15.131:0/3758776759 shutdown_connections 2024-09-17T02:49:57.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.088+0000 7f3a0bf05640 1 -- 172.21.15.131:0/3758776759 wait complete. 2024-09-17T02:49:57.109 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T02:49:57.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 -- 172.21.15.131:0/2438509842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7100d2b70 msgr2=0x7fa7100d2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 --2- 172.21.15.131:0/2438509842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7100d2b70 0x7fa7100d2f80 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa708009bc0 tx=0x7fa70801cba0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 -- 172.21.15.131:0/2438509842 shutdown_connections 2024-09-17T02:49:57.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 --2- 172.21.15.131:0/2438509842 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa7100d2b70 0x7fa7100d2f80 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 -- 172.21.15.131:0/2438509842 >> 172.21.15.131:0/2438509842 conn(0x7fa7100d1b10 msgr2=0x7fa7100d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:57.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 -- 172.21.15.131:0/2438509842 shutdown_connections 2024-09-17T02:49:57.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 -- 172.21.15.131:0/2438509842 wait complete. 2024-09-17T02:49:57.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 Processor -- start 2024-09-17T02:49:57.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 -- start start 2024-09-17T02:49:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa7100cf710 0x7fa7100cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7100d2b70 0x7fa71014f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa71015f9a0 0x7fa71014f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa71016d260 con 0x7fa71015f9a0 2024-09-17T02:49:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa71016d0e0 con 0x7fa7100d2b70 2024-09-17T02:49:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa7149bb640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa71016d3e0 con 0x7fa7100cf710 2024-09-17T02:49:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.272+0000 7fa70ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa71015f9a0 0x7fa71014f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70ed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa71015f9a0 0x7fa71014f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55572/0 (socket says 172.21.15.131:55572) 2024-09-17T02:49:57.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70ed9d640 1 -- 172.21.15.131:0/1000692877 learned_addr learned my addr 172.21.15.131:0/1000692877 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70dd9b640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7100d2b70 0x7fa71014f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70ed9d640 1 -- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa7100cf710 msgr2=0x7fa7100cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70e59c640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa7100cf710 0x7fa7100cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70ed9d640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa7100cf710 0x7fa7100cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70ed9d640 1 -- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7100d2b70 msgr2=0x7fa71014f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70ed9d640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7100d2b70 0x7fa71014f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70ed9d640 1 -- 172.21.15.131:0/1000692877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7100b4810 con 0x7fa71015f9a0 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70dd9b640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7100d2b70 0x7fa71014f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70ed9d640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa71015f9a0 0x7fa71014f9f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa7080197b0 tx=0x7fa708005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa706ffd640 1 -- 172.21.15.131:0/1000692877 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7080bc070 con 0x7fa71015f9a0 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa706ffd640 1 -- 172.21.15.131:0/1000692877 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa708004330 con 0x7fa71015f9a0 2024-09-17T02:49:57.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa7149bb640 1 -- 172.21.15.131:0/1000692877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7100b4af0 con 0x7fa71015f9a0 2024-09-17T02:49:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa706ffd640 1 -- 172.21.15.131:0/1000692877 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7080b2040 con 0x7fa71015f9a0 2024-09-17T02:49:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa7149bb640 1 -- 172.21.15.131:0/1000692877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7100b50b0 con 0x7fa71015f9a0 2024-09-17T02:49:57.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa7149bb640 1 -- 172.21.15.131:0/1000692877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71014e5a0 con 0x7fa71015f9a0 2024-09-17T02:49:57.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa706ffd640 1 -- 172.21.15.131:0/1000692877 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa7080bf070 con 0x7fa71015f9a0 2024-09-17T02:49:57.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa706ffd640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f0073230 0x7fa6f0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:57.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa706ffd640 1 -- 172.21.15.131:0/1000692877 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5507+0+0 (secure 0 0 0) 0x7fa70812df30 con 0x7fa71015f9a0 2024-09-17T02:49:57.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.276+0000 7fa70e59c640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f0073230 0x7fa6f0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:57.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.280+0000 7fa70e59c640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f0073230 0x7fa6f0075740 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa7100b7610 tx=0x7fa6f8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:57.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.280+0000 7fa706ffd640 1 -- 172.21.15.131:0/1000692877 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa7080a5e20 con 0x7fa71015f9a0 2024-09-17T02:49:57.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.448+0000 7fa7149bb640 1 -- 172.21.15.131:0/1000692877 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa7100d2f80 con 0x7fa71015f9a0 2024-09-17T02:49:57.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.448+0000 7fa706ffd640 1 -- 172.21.15.131:0/1000692877 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v41) ==== 59+0+2294 (secure 0 0 0) 0x7fa708016790 con 0x7fa71015f9a0 2024-09-17T02:49:57.447 INFO:teuthology.orchestra.run.smithi131.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T02:49:57.447 INFO:teuthology.orchestra.run.smithi131.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T02:49:57.447 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced client_mount_timeout 30 2024-09-17T02:49:57.447 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_client 20/20 2024-09-17T02:49:57.447 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_mgr 20/20 2024-09-17T02:49:57.447 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_ms 1/1 2024-09-17T02:49:57.447 INFO:teuthology.orchestra.run.smithi131.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T02:49:57.447 INFO:teuthology.orchestra.run.smithi131.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T02:49:57.447 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3855.598857 2024-09-17T02:49:57.447 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3598.912856 2024-09-17T02:49:57.447 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3977.024117 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3823.557728 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4643.710284 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4452.598518 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6659.946233 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4539.778076 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds 20/20 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_ms 2/2 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_frag true 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_scatterstat true 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_verify_scatter true 2024-09-17T02:49:57.448 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T02:49:57.449 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T02:49:57.449 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T02:49:57.449 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced client_mount_timeout 600 2024-09-17T02:49:57.449 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_client 20/20 2024-09-17T02:49:57.449 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_ms 1/1 2024-09-17T02:49:57.449 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T02:49:57.449 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T02:49:57.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 -- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f0073230 msgr2=0x7fa6f0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f0073230 0x7fa6f0075740 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa7100b7610 tx=0x7fa6f8009040 comp rx=0 tx=0).stop 2024-09-17T02:49:57.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 -- 172.21.15.131:0/1000692877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa71015f9a0 msgr2=0x7fa71014f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa71015f9a0 0x7fa71014f9f0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa7080197b0 tx=0x7fa708005b10 comp rx=0 tx=0).stop 2024-09-17T02:49:57.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 -- 172.21.15.131:0/1000692877 shutdown_connections 2024-09-17T02:49:57.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa6f0073230 0x7fa6f0075740 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa71015f9a0 0x7fa71014f9f0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7100d2b70 0x7fa71014f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 --2- 172.21.15.131:0/1000692877 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa7100cf710 0x7fa7100cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 -- 172.21.15.131:0/1000692877 >> 172.21.15.131:0/1000692877 conn(0x7fa7100d1b10 msgr2=0x7fa710069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:57.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 -- 172.21.15.131:0/1000692877 shutdown_connections 2024-09-17T02:49:57.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.452+0000 7fa7149bb640 1 -- 172.21.15.131:0/1000692877 wait complete. 2024-09-17T02:49:57.469 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T02:49:57.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.624+0000 7f5f2df25640 1 -- 172.21.15.131:0/3997928598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f2815cfa0 msgr2=0x7f5f2815d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.624+0000 7f5f2df25640 1 --2- 172.21.15.131:0/3997928598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f2815cfa0 0x7f5f2815d3b0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009c80 tx=0x7f5f2001cd10 comp rx=0 tx=0).stop 2024-09-17T02:49:57.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.624+0000 7f5f2df25640 1 -- 172.21.15.131:0/3997928598 shutdown_connections 2024-09-17T02:49:57.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.624+0000 7f5f2df25640 1 --2- 172.21.15.131:0/3997928598 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f2815cfa0 0x7f5f2815d3b0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.624+0000 7f5f2df25640 1 --2- 172.21.15.131:0/3997928598 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5f281566f0 0x7f5f28156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.624+0000 7f5f2df25640 1 -- 172.21.15.131:0/3997928598 >> 172.21.15.131:0/3997928598 conn(0x7f5f280d2100 msgr2=0x7f5f280d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:57.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.624+0000 7f5f2df25640 1 -- 172.21.15.131:0/3997928598 shutdown_connections 2024-09-17T02:49:57.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.624+0000 7f5f2df25640 1 -- 172.21.15.131:0/3997928598 wait complete. 2024-09-17T02:49:57.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f2df25640 1 Processor -- start 2024-09-17T02:49:57.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f2df25640 1 -- start start 2024-09-17T02:49:57.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f2df25640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f280cc750 0x7f5f280c3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f2df25640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f281566f0 0x7f5f280c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f2df25640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5f2815cfa0 0x7f5f280c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f2df25640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5f280716e0 con 0x7f5f281566f0 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f2df25640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5f28071560 con 0x7f5f280cc750 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f2df25640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5f28071860 con 0x7f5f2815cfa0 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f26ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f281566f0 0x7f5f280c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f26ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f281566f0 0x7f5f280c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55590/0 (socket says 172.21.15.131:55590) 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f26ffd640 1 -- 172.21.15.131:0/199434514 learned_addr learned my addr 172.21.15.131:0/199434514 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f27fff640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5f2815cfa0 0x7f5f280c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f277fe640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f280cc750 0x7f5f280c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f26ffd640 1 -- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5f2815cfa0 msgr2=0x7f5f280c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f26ffd640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5f2815cfa0 0x7f5f280c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f26ffd640 1 -- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f280cc750 msgr2=0x7f5f280c3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f26ffd640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f280cc750 0x7f5f280c3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f26ffd640 1 -- 172.21.15.131:0/199434514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f28156260 con 0x7f5f281566f0 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f27fff640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5f2815cfa0 0x7f5f280c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f277fe640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f280cc750 0x7f5f280c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f26ffd640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f281566f0 0x7f5f280c7fd0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00c3c0 tx=0x7f5f1c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f0ffff640 1 -- 172.21.15.131:0/199434514 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c017070 con 0x7f5f281566f0 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f0ffff640 1 -- 172.21.15.131:0/199434514 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f1c0053c0 con 0x7f5f281566f0 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f2df25640 1 -- 172.21.15.131:0/199434514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f280b4fd0 con 0x7f5f281566f0 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f0ffff640 1 -- 172.21.15.131:0/199434514 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c003860 con 0x7f5f281566f0 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.628+0000 7f5f2df25640 1 -- 172.21.15.131:0/199434514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f280b5590 con 0x7f5f281566f0 2024-09-17T02:49:57.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.632+0000 7f5f2df25640 1 -- 172.21.15.131:0/199434514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2815d3b0 con 0x7f5f281566f0 2024-09-17T02:49:57.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.632+0000 7f5f0ffff640 1 -- 172.21.15.131:0/199434514 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5f1c01a070 con 0x7f5f281566f0 2024-09-17T02:49:57.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.632+0000 7f5f0ffff640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5f14073230 0x7f5f14075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:57.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.632+0000 7f5f0ffff640 1 -- 172.21.15.131:0/199434514 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5507+0+0 (secure 0 0 0) 0x7f5f1c093470 con 0x7f5f281566f0 2024-09-17T02:49:57.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.632+0000 7f5f277fe640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5f14073230 0x7f5f14075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:57.630 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.632+0000 7f5f277fe640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5f14073230 0x7f5f14075740 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005fd0 tx=0x7f5f18005dc0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:57.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.636+0000 7f5f0ffff640 1 -- 172.21.15.131:0/199434514 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5f1c05e550 con 0x7f5f281566f0 2024-09-17T02:49:57.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.796+0000 7f5f2df25640 1 -- 172.21.15.131:0/199434514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5f280d2f10 con 0x7f5f281566f0 2024-09-17T02:49:57.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.804+0000 7f5f0ffff640 1 -- 172.21.15.131:0/199434514 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f5f1c05de50 con 0x7f5f281566f0 2024-09-17T02:49:57.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 -- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5f14073230 msgr2=0x7f5f14075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5f14073230 0x7f5f14075740 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005fd0 tx=0x7f5f18005dc0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 -- 172.21.15.131:0/199434514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f281566f0 msgr2=0x7f5f280c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:57.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f281566f0 0x7f5f280c7fd0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00c3c0 tx=0x7f5f1c00c880 comp rx=0 tx=0).stop 2024-09-17T02:49:57.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 -- 172.21.15.131:0/199434514 shutdown_connections 2024-09-17T02:49:57.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5f14073230 0x7f5f14075740 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5f2815cfa0 0x7f5f280c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5f281566f0 0x7f5f280c7fd0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 --2- 172.21.15.131:0/199434514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5f280cc750 0x7f5f280c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:57.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 -- 172.21.15.131:0/199434514 >> 172.21.15.131:0/199434514 conn(0x7f5f280d2100 msgr2=0x7f5f28150cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:57.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 -- 172.21.15.131:0/199434514 shutdown_connections 2024-09-17T02:49:57.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:57.808+0000 7f5f2df25640 1 -- 172.21.15.131:0/199434514 wait complete. 2024-09-17T02:49:57.823 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:49:57.823 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:49:57.904 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:49:57.975 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:49:57.975 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:49:57.975 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:49:57.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:49:57.981 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:49:57.981 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:49:57.987 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:49:57.988 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:49:57.988 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:49:58.027 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:49:58.028 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:49:58.028 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:49:58.033 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:49:58.033 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T02:49:58.033 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:49:58.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.232+0000 7f47630c4640 1 -- 172.21.15.131:0/2488425450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4744005920 msgr2=0x7f4744005d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:58.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.232+0000 7f47630c4640 1 --2- 172.21.15.131:0/2488425450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4744005920 0x7f4744005d30 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f474c009b40 tx=0x7f474c01d250 comp rx=0 tx=0).stop 2024-09-17T02:49:58.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.232+0000 7f47630c4640 1 -- 172.21.15.131:0/2488425450 shutdown_connections 2024-09-17T02:49:58.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.232+0000 7f47630c4640 1 --2- 172.21.15.131:0/2488425450 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4744005920 0x7f4744005d30 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:58.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.232+0000 7f47630c4640 1 -- 172.21.15.131:0/2488425450 >> 172.21.15.131:0/2488425450 conn(0x7f475c06eea0 msgr2=0x7f475c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:58.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.232+0000 7f47630c4640 1 -- 172.21.15.131:0/2488425450 shutdown_connections 2024-09-17T02:49:58.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.232+0000 7f47630c4640 1 -- 172.21.15.131:0/2488425450 wait complete. 2024-09-17T02:49:58.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47630c4640 1 Processor -- start 2024-09-17T02:49:58.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47630c4640 1 -- start start 2024-09-17T02:49:58.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47630c4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4744005920 0x7f475c0c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47630c4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f475c1f4a30 0x7f475c1f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47630c4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475c1f5380 0x7f475c1f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47630c4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f475c06fb40 con 0x7f475c1f5380 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47630c4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f475c06f9c0 con 0x7f475c1f4a30 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47630c4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f475c06fcc0 con 0x7f4744005920 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4761661640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475c1f5380 0x7f475c1f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4761661640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475c1f5380 0x7f475c1f6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55596/0 (socket says 172.21.15.131:55596) 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4761661640 1 -- 172.21.15.131:0/3519198676 learned_addr learned my addr 172.21.15.131:0/3519198676 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4753fff640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f475c1f4a30 0x7f475c1f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4760e60640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4744005920 0x7f475c0c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4761661640 1 -- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4744005920 msgr2=0x7f475c0c9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4761661640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4744005920 0x7f475c0c9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:58.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4761661640 1 -- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f475c1f4a30 msgr2=0x7f475c1f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:58.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4761661640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f475c1f4a30 0x7f475c1f4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:58.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4761661640 1 -- 172.21.15.131:0/3519198676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475c1564e0 con 0x7f475c1f5380 2024-09-17T02:49:58.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4753fff640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f475c1f4a30 0x7f475c1f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:49:58.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f4761661640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475c1f5380 0x7f475c1f6e60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f475c072250 tx=0x7f475400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:58.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47517fa640 1 -- 172.21.15.131:0/3519198676 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4754003b80 con 0x7f475c1f5380 2024-09-17T02:49:58.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47517fa640 1 -- 172.21.15.131:0/3519198676 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4754003e10 con 0x7f475c1f5380 2024-09-17T02:49:58.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47630c4640 1 -- 172.21.15.131:0/3519198676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475c1567c0 con 0x7f475c1f5380 2024-09-17T02:49:58.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47517fa640 1 -- 172.21.15.131:0/3519198676 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475400ba60 con 0x7f475c1f5380 2024-09-17T02:49:58.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47630c4640 1 -- 172.21.15.131:0/3519198676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f475c156d80 con 0x7f475c1f5380 2024-09-17T02:49:58.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.236+0000 7f47630c4640 1 -- 172.21.15.131:0/3519198676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f475c073430 con 0x7f475c1f5380 2024-09-17T02:49:58.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.240+0000 7f47517fa640 1 -- 172.21.15.131:0/3519198676 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f475400bc00 con 0x7f475c1f5380 2024-09-17T02:49:58.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.240+0000 7f47517fa640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4748073200 0x7f4748075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:58.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.240+0000 7f47517fa640 1 -- 172.21.15.131:0/3519198676 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5507+0+0 (secure 0 0 0) 0x7f47540943c0 con 0x7f475c1f5380 2024-09-17T02:49:58.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.240+0000 7f4760e60640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4748073200 0x7f4748075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:58.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.240+0000 7f4760e60640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4748073200 0x7f4748075710 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4758001a60 tx=0x7f4758004610 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:58.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.244+0000 7f47517fa640 1 -- 172.21.15.131:0/3519198676 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f475405f680 con 0x7f475c1f5380 2024-09-17T02:49:58.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.448+0000 7f47630c4640 1 -- 172.21.15.131:0/3519198676 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f475c0a9890 con 0x7f475c1f5380 2024-09-17T02:49:58.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.448+0000 7f47517fa640 1 -- 172.21.15.131:0/3519198676 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) ==== 94+0+3923 (secure 0 0 0) 0x7f475405ef80 con 0x7f475c1f5380 2024-09-17T02:49:58.446 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:58.446 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":79,"btime":"2024-09-17T02:49:51:886445+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5471,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2838003859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2838003859},{"type":"v1","addr":"172.21.15.189:6835","nonce":2838003859}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":5477,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1554771753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1554771753},{"type":"v1","addr":"172.21.15.189:6837","nonce":1554771753}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":7032,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6833/2821817591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2821817591},{"type":"v1","addr":"172.21.15.131:6833","nonce":2821817591}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/3123840452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6834","nonce":3123840452},{"type":"v1","addr":"172.21.15.131:6835","nonce":3123840452}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[]} 2024-09-17T02:49:58.447 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 79 2024-09-17T02:49:58.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 -- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4748073200 msgr2=0x7f4748075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:58.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4748073200 0x7f4748075710 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4758001a60 tx=0x7f4758004610 comp rx=0 tx=0).stop 2024-09-17T02:49:58.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 -- 172.21.15.131:0/3519198676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475c1f5380 msgr2=0x7f475c1f6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:58.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475c1f5380 0x7f475c1f6e60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f475c072250 tx=0x7f475400db70 comp rx=0 tx=0).stop 2024-09-17T02:49:58.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 -- 172.21.15.131:0/3519198676 shutdown_connections 2024-09-17T02:49:58.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4748073200 0x7f4748075710 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:58.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f475c1f5380 0x7f475c1f6e60 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:58.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f475c1f4a30 0x7f475c1f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:58.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 --2- 172.21.15.131:0/3519198676 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4744005920 0x7f475c0c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:58.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 -- 172.21.15.131:0/3519198676 >> 172.21.15.131:0/3519198676 conn(0x7f475c06eea0 msgr2=0x7f475c0b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:58.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 -- 172.21.15.131:0/3519198676 shutdown_connections 2024-09-17T02:49:58.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.452+0000 7f47630c4640 1 -- 172.21.15.131:0/3519198676 wait complete. 2024-09-17T02:49:58.469 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T02:49:58.470 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T02:49:58.470 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T02:49:58.470 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:49:58.470 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T02:49:58.470 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T02:49:58.470 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T02:49:58.470 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:49:58.471 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T02:49:58.471 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T02:49:58.471 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T02:49:58.471 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:49:58.471 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T02:49:58.471 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T02:49:58.471 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T02:49:58.471 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:49:58.485 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T02:49:58.486 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T02:49:58.486 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T02:49:58.487 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T02:49:58.490 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T02:49:58.490 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T02:49:58.494 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T02:49:58.495 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T02:49:58.528 INFO:tasks.ceph.mds.c:Started 2024-09-17T02:49:58.528 DEBUG:teuthology.parallel:result is None 2024-09-17T02:49:58.529 INFO:tasks.ceph.mds.a:Started 2024-09-17T02:49:58.529 DEBUG:teuthology.parallel:result is None 2024-09-17T02:49:58.538 INFO:tasks.ceph.mds.b:Started 2024-09-17T02:49:58.538 DEBUG:teuthology.parallel:result is None 2024-09-17T02:49:58.539 INFO:tasks.ceph.mds.d:Started 2024-09-17T02:49:58.539 DEBUG:teuthology.parallel:result is None 2024-09-17T02:49:58.540 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T02:49:58.615 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:49:58.616 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:49:58.634 INFO:tasks.ceph.mds.d.smithi189.stdout:starting mds.d at 2024-09-17T02:49:58.643 INFO:tasks.ceph.mds.b.smithi189.stdout:starting mds.b at 2024-09-17T02:49:58.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.684+0000 7f8b5e197640 1 -- 172.21.15.131:0/748593772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b5815fb60 msgr2=0x7f8b580d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:58.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.684+0000 7f8b5e197640 1 --2- 172.21.15.131:0/748593772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b5815fb60 0x7f8b580d0460 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8b58073ce0 tx=0x7f8b4801c800 comp rx=0 tx=0).stop 2024-09-17T02:49:58.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.684+0000 7f8b5e197640 1 -- 172.21.15.131:0/748593772 shutdown_connections 2024-09-17T02:49:58.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.684+0000 7f8b5e197640 1 --2- 172.21.15.131:0/748593772 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b5815fb60 0x7f8b580d0460 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:58.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.684+0000 7f8b5e197640 1 --2- 172.21.15.131:0/748593772 >> v2:172.21.15.189:3300/0 conn(0x7f8b5815f750 0x7f8b580cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:58.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.684+0000 7f8b5e197640 1 -- 172.21.15.131:0/748593772 >> 172.21.15.131:0/748593772 conn(0x7f8b5806fbe0 msgr2=0x7f8b5806fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:58.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b5e197640 1 -- 172.21.15.131:0/748593772 shutdown_connections 2024-09-17T02:49:58.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b5e197640 1 -- 172.21.15.131:0/748593772 wait complete. 2024-09-17T02:49:58.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b5e197640 1 Processor -- start 2024-09-17T02:49:58.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b5e197640 1 -- start start 2024-09-17T02:49:58.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b5e197640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b5815b310 0x7f8b581597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b5e197640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b5815f750 0x7f8b58159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b5e197640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8b5815fb60 0x7f8b5815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b5e197640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8b58168700 con 0x7f8b5815b310 2024-09-17T02:49:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b5e197640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8b58168580 con 0x7f8b5815f750 2024-09-17T02:49:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b5e197640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8b58168880 con 0x7f8b5815fb60 2024-09-17T02:49:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b577fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b5815b310 0x7f8b581597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b577fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b5815b310 0x7f8b581597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55670/0 (socket says 172.21.15.131:55670) 2024-09-17T02:49:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b577fe640 1 -- 172.21.15.131:0/3284898203 learned_addr learned my addr 172.21.15.131:0/3284898203 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b57fff640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8b5815fb60 0x7f8b5815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b56ffd640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b5815f750 0x7f8b58159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:58.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b577fe640 1 -- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8b5815fb60 msgr2=0x7f8b5815b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b577fe640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8b5815fb60 0x7f8b5815b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b577fe640 1 -- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b5815f750 msgr2=0x7f8b58159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b577fe640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b5815f750 0x7f8b58159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b577fe640 1 -- 172.21.15.131:0/3284898203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b580cae40 con 0x7f8b5815b310 2024-09-17T02:49:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.688+0000 7f8b577fe640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b5815b310 0x7f8b581597c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000cb70 tx=0x7f8b5000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.692+0000 7f8b3ffff640 1 -- 172.21.15.131:0/3284898203 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50016020 con 0x7f8b5815b310 2024-09-17T02:49:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.692+0000 7f8b3ffff640 1 -- 172.21.15.131:0/3284898203 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b5000ab60 con 0x7f8b5815b310 2024-09-17T02:49:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.692+0000 7f8b5e197640 1 -- 172.21.15.131:0/3284898203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b580cb2b0 con 0x7f8b5815b310 2024-09-17T02:49:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.692+0000 7f8b3ffff640 1 -- 172.21.15.131:0/3284898203 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50012420 con 0x7f8b5815b310 2024-09-17T02:49:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.692+0000 7f8b5e197640 1 -- 172.21.15.131:0/3284898203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b5815d120 con 0x7f8b5815b310 2024-09-17T02:49:58.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.692+0000 7f8b5e197640 1 -- 172.21.15.131:0/3284898203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b58154a50 con 0x7f8b5815b310 2024-09-17T02:49:58.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.692+0000 7f8b3ffff640 1 -- 172.21.15.131:0/3284898203 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8b50019020 con 0x7f8b5815b310 2024-09-17T02:49:58.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.692+0000 7f8b3ffff640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b44073200 0x7f8b44075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:58.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.692+0000 7f8b3ffff640 1 -- 172.21.15.131:0/3284898203 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5507+0+0 (secure 0 0 0) 0x7f8b500942f0 con 0x7f8b5815b310 2024-09-17T02:49:58.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.692+0000 7f8b56ffd640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b44073200 0x7f8b44075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:58.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.692+0000 7f8b56ffd640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b44073200 0x7f8b44075710 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8b581544b0 tx=0x7f8b4c005f90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:58.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.696+0000 7f8b3ffff640 1 -- 172.21.15.131:0/3284898203 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8b50020020 con 0x7f8b5815b310 2024-09-17T02:49:58.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:58.868+0000 7f8b5e197640 1 -- 172.21.15.131:0/3284898203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8b580b6fa0 con 0x7f8b5815b310 2024-09-17T02:49:59.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.124+0000 7f8b3ffff640 1 -- 172.21.15.131:0/3284898203 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v101) ==== 97+0+0 (secure 0 0 0) 0x7f8b5005ed70 con 0x7f8b5815b310 2024-09-17T02:49:59.119 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-09-17T02:49:59.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 -- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b44073200 msgr2=0x7f8b44075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b44073200 0x7f8b44075710 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f8b581544b0 tx=0x7f8b4c005f90 comp rx=0 tx=0).stop 2024-09-17T02:49:59.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 -- 172.21.15.131:0/3284898203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b5815b310 msgr2=0x7f8b581597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b5815b310 0x7f8b581597c0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000cb70 tx=0x7f8b5000cc60 comp rx=0 tx=0).stop 2024-09-17T02:49:59.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 -- 172.21.15.131:0/3284898203 shutdown_connections 2024-09-17T02:49:59.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8b44073200 0x7f8b44075710 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8b5815fb60 0x7f8b5815b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8b5815f750 0x7f8b58159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 --2- 172.21.15.131:0/3284898203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8b5815b310 0x7f8b581597c0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 -- 172.21.15.131:0/3284898203 >> 172.21.15.131:0/3284898203 conn(0x7f8b5806fbe0 msgr2=0x7f8b58071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:59.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 -- 172.21.15.131:0/3284898203 shutdown_connections 2024-09-17T02:49:59.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.128+0000 7f8b5e197640 1 -- 172.21.15.131:0/3284898203 wait complete. 2024-09-17T02:49:59.151 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T02:49:59.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.308+0000 7fa6a5a53640 1 -- 172.21.15.131:0/1263762736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa688003a40 msgr2=0x7fa688003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.308+0000 7fa6a5a53640 1 --2- 172.21.15.131:0/1263762736 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa688003a40 0x7fa688003eb0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fa694006fd0 tx=0x7fa69401cf30 comp rx=0 tx=0).stop 2024-09-17T02:49:59.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.308+0000 7fa69f7fe640 1 -- 172.21.15.131:0/1263762736 reap_dead start 2024-09-17T02:49:59.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.308+0000 7fa6a5a53640 1 -- 172.21.15.131:0/1263762736 shutdown_connections 2024-09-17T02:49:59.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.308+0000 7fa6a5a53640 1 -- 172.21.15.131:0/1263762736 >> 172.21.15.131:0/1263762736 conn(0x7fa6a006eea0 msgr2=0x7fa6a006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:59.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.308+0000 7fa6a5a53640 1 -- 172.21.15.131:0/1263762736 shutdown_connections 2024-09-17T02:49:59.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.308+0000 7fa6a5a53640 1 -- 172.21.15.131:0/1263762736 wait complete. 2024-09-17T02:49:59.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.308+0000 7fa6a5a53640 1 Processor -- start 2024-09-17T02:49:59.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.308+0000 7fa6a5a53640 1 -- start start 2024-09-17T02:49:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa6a5a53640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa688003a40 0x7fa6a00cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa6a5a53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6a0153990 0x7fa6a01f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa6a5a53640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6a01f89e0 0x7fa6a0166110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa6a5a53640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0168c60 con 0x7fa6a0153990 2024-09-17T02:49:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa6a5a53640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a0168ae0 con 0x7fa6a01f89e0 2024-09-17T02:49:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa6a5a53640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a0168de0 con 0x7fa688003a40 2024-09-17T02:49:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa69e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6a0153990 0x7fa6a01f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa69e7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6a0153990 0x7fa6a01f84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55684/0 (socket says 172.21.15.131:55684) 2024-09-17T02:49:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa69e7fc640 1 -- 172.21.15.131:0/3742473286 learned_addr learned my addr 172.21.15.131:0/3742473286 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa69e7fc640 1 -- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa688003a40 msgr2=0x7fa6a00cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa69effd640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa688003a40 0x7fa6a00cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa69f7fe640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6a01f89e0 0x7fa6a0166110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa69e7fc640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa688003a40 0x7fa6a00cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa69e7fc640 1 -- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6a01f89e0 msgr2=0x7fa6a0166110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa69e7fc640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6a01f89e0 0x7fa6a0166110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa69e7fc640 1 -- 172.21.15.131:0/3742473286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a00caa50 con 0x7fa6a0153990 2024-09-17T02:49:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa69e7fc640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6a0153990 0x7fa6a01f84a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fa690011880 tx=0x7fa690011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa687fff640 1 -- 172.21.15.131:0/3742473286 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69000f9a0 con 0x7fa6a0153990 2024-09-17T02:49:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa687fff640 1 -- 172.21.15.131:0/3742473286 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa69000fb40 con 0x7fa6a0153990 2024-09-17T02:49:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa6a5a53640 1 -- 172.21.15.131:0/3742473286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a00cad30 con 0x7fa6a0153990 2024-09-17T02:49:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa687fff640 1 -- 172.21.15.131:0/3742473286 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690003600 con 0x7fa6a0153990 2024-09-17T02:49:59.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa6a5a53640 1 -- 172.21.15.131:0/3742473286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a00cb2f0 con 0x7fa6a0153990 2024-09-17T02:49:59.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa6a5a53640 1 -- 172.21.15.131:0/3742473286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a00ca7c0 con 0x7fa6a0153990 2024-09-17T02:49:59.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa687fff640 1 -- 172.21.15.131:0/3742473286 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa6900037a0 con 0x7fa6a0153990 2024-09-17T02:49:59.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.312+0000 7fa687fff640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa68c073230 0x7fa68c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:59.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.316+0000 7fa687fff640 1 -- 172.21.15.131:0/3742473286 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5249+0+0 (secure 0 0 0) 0x7fa690095000 con 0x7fa6a0153990 2024-09-17T02:49:59.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.316+0000 7fa69effd640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa68c073230 0x7fa68c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:59.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.316+0000 7fa69effd640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa68c073230 0x7fa68c075740 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa6940ab000 tx=0x7fa6940a7040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:59.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.320+0000 7fa687fff640 1 -- 172.21.15.131:0/3742473286 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa6900601e0 con 0x7fa6a0153990 2024-09-17T02:49:59.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.544+0000 7fa6a5a53640 1 -- 172.21.15.131:0/3742473286 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa6a00c4580 con 0x7fa6a0153990 2024-09-17T02:49:59.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.544+0000 7fa687fff640 1 -- 172.21.15.131:0/3742473286 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) ==== 80+0+5304 (secure 0 0 0) 0x7fa69005fae0 con 0x7fa6a0153990 2024-09-17T02:49:59.541 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:59.541 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:49:59.541 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-09-17T02:49:59.541 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm4U1cHhAATu86VrwT2vMjL/cW2SICMw==", 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmVpUSKxAA4l5zGHXg0VRasiyBI/j4ig==", 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.542 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmwHuCJRAAlOj+za8ZmqMI55PsShjKDw==", 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.543 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm9f+9MhAAS8z9jZFYon3nrmu9/4JMPg==", 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-09-17T02:49:59.544 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD16+hmV9xkBxAAT+IDf2J4awJfqYcx2xDoNA==", 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD46+hmOhJsGBAAdlA4BSdOPk8BySUTfo1ghg==", 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.545 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD76+hm3mSoKBAAQPoGFy816yWZQW6vPIK6bA==", 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD+6+hmpZaPNxAAp0EV6bn5SugDMwA4RnnJwg==", 2024-09-17T02:49:59.546 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAF7OhmgDfVAxAALACK3bk+PK3UhG20b498HQ==", 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.547 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAI7Ohm6w44FBAA90GkelUAbiNjAwn+6IJWcg==", 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-09-17T02:49:59.548 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAL7Ohm6S6TIxAArISJW//rmEONG2bC/sfkXA==", 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAO7OhmICwTMxAArAr7DhcO8A0FDSu7VEBehw==", 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.549 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.550 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.550 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.550 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-09-17T02:49:59.550 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA==", 2024-09-17T02:49:59.550 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.550 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmduVoBRAAwr50jSXw6sCyo+5t+3iT7A==", 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.551 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmBBCDABAAMw9581QyaOxJk5fs35o+ng==", 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-09-17T02:49:59.552 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmgznRCRAA1Z8d32ADoSG+7RHtdPGmTQ==", 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmpUfvABAAzclxwjR4E+UxvtkbwVqDXw==", 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:49:59.553 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmnWmJGBAA6fzewrm8ELqUAz/MczTOGA==", 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.554 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-09-17T02:49:59.555 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmrTq3ExAAuR8+2bItlpHqqFzlvPxASA==", 2024-09-17T02:49:59.555 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:49:59.555 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:49:59.555 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:49:59.555 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:49:59.555 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.555 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.555 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-09-17T02:49:59.555 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:49:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 -- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa68c073230 msgr2=0x7fa68c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa68c073230 0x7fa68c075740 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa6940ab000 tx=0x7fa6940a7040 comp rx=0 tx=0).stop 2024-09-17T02:49:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 -- 172.21.15.131:0/3742473286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6a0153990 msgr2=0x7fa6a01f84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6a0153990 0x7fa6a01f84a0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fa690011880 tx=0x7fa690011d40 comp rx=0 tx=0).stop 2024-09-17T02:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 -- 172.21.15.131:0/3742473286 shutdown_connections 2024-09-17T02:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa68c073230 0x7fa68c075740 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa6a01f89e0 0x7fa6a0166110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa6a0153990 0x7fa6a01f84a0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 --2- 172.21.15.131:0/3742473286 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa688003a40 0x7fa6a00cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 -- 172.21.15.131:0/3742473286 >> 172.21.15.131:0/3742473286 conn(0x7fa6a006eea0 msgr2=0x7fa6a015c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 -- 172.21.15.131:0/3742473286 shutdown_connections 2024-09-17T02:49:59.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.548+0000 7fa6a5a53640 1 -- 172.21.15.131:0/3742473286 wait complete. 2024-09-17T02:49:59.562 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T02:49:59.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.708+0000 7fef7917a640 1 -- 172.21.15.131:0/444774932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef7406e2c0 msgr2=0x7fef740c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.708+0000 7fef7917a640 1 --2- 172.21.15.131:0/444774932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef7406e2c0 0x7fef740c4580 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fef740727f0 tx=0x7fef6401c850 comp rx=0 tx=0).stop 2024-09-17T02:49:59.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.708+0000 7fef7917a640 1 -- 172.21.15.131:0/444774932 shutdown_connections 2024-09-17T02:49:59.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.708+0000 7fef7917a640 1 --2- 172.21.15.131:0/444774932 >> v2:172.21.15.189:3300/0 conn(0x7fef740cdbf0 0x7fef740ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.708+0000 7fef7917a640 1 --2- 172.21.15.131:0/444774932 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef740c4ac0 0x7fef740c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.708+0000 7fef7917a640 1 --2- 172.21.15.131:0/444774932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef7406e2c0 0x7fef740c4580 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.708+0000 7fef7917a640 1 -- 172.21.15.131:0/444774932 >> 172.21.15.131:0/444774932 conn(0x7fef7406eea0 msgr2=0x7fef7406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:59.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.708+0000 7fef7917a640 1 -- 172.21.15.131:0/444774932 shutdown_connections 2024-09-17T02:49:59.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.708+0000 7fef7917a640 1 -- 172.21.15.131:0/444774932 wait complete. 2024-09-17T02:49:59.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7917a640 1 Processor -- start 2024-09-17T02:49:59.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7917a640 1 -- start start 2024-09-17T02:49:59.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7917a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef7406e2c0 0x7fef741567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:59.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7917a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef740c4ac0 0x7fef741552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:59.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7917a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef740cdbf0 0x7fef740cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:59.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7917a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fef740ceed0 con 0x7fef7406e2c0 2024-09-17T02:49:59.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7917a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fef740ced50 con 0x7fef740cdbf0 2024-09-17T02:49:59.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7917a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fef740cf050 con 0x7fef740c4ac0 2024-09-17T02:49:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef72d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef7406e2c0 0x7fef741567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef72d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef7406e2c0 0x7fef741567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55708/0 (socket says 172.21.15.131:55708) 2024-09-17T02:49:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef72d9d640 1 -- 172.21.15.131:0/519864514 learned_addr learned my addr 172.21.15.131:0/519864514 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:49:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7359e640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef740cdbf0 0x7fef740cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7259c640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef740c4ac0 0x7fef741552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef72d9d640 1 -- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef740c4ac0 msgr2=0x7fef741552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef72d9d640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef740c4ac0 0x7fef741552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef72d9d640 1 -- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef740cdbf0 msgr2=0x7fef740cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef72d9d640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef740cdbf0 0x7fef740cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef72d9d640 1 -- 172.21.15.131:0/519864514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef741557f0 con 0x7fef7406e2c0 2024-09-17T02:49:59.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7259c640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef740c4ac0 0x7fef741552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:49:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef72d9d640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef7406e2c0 0x7fef741567d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fef640b0040 tx=0x7fef640a6770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef5b7fe640 1 -- 172.21.15.131:0/519864514 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef640bc070 con 0x7fef7406e2c0 2024-09-17T02:49:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef5b7fe640 1 -- 172.21.15.131:0/519864514 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef64005030 con 0x7fef7406e2c0 2024-09-17T02:49:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7917a640 1 -- 172.21.15.131:0/519864514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef740c9320 con 0x7fef7406e2c0 2024-09-17T02:49:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef5b7fe640 1 -- 172.21.15.131:0/519864514 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef640044f0 con 0x7fef7406e2c0 2024-09-17T02:49:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7917a640 1 -- 172.21.15.131:0/519864514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef740cb040 con 0x7fef7406e2c0 2024-09-17T02:49:59.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.712+0000 7fef7917a640 1 -- 172.21.15.131:0/519864514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef740c4ed0 con 0x7fef7406e2c0 2024-09-17T02:49:59.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.716+0000 7fef5b7fe640 1 -- 172.21.15.131:0/519864514 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fef640bf070 con 0x7fef7406e2c0 2024-09-17T02:49:59.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.716+0000 7fef5b7fe640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef60073200 0x7fef60075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:49:59.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.716+0000 7fef5b7fe640 1 -- 172.21.15.131:0/519864514 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5249+0+0 (secure 0 0 0) 0x7fef6412e0e0 con 0x7fef7406e2c0 2024-09-17T02:49:59.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.716+0000 7fef7259c640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef60073200 0x7fef60075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:49:59.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.716+0000 7fef7259c640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef60073200 0x7fef60075710 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fef740c4890 tx=0x7fef6800a310 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:49:59.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.720+0000 7fef5b7fe640 1 -- 172.21.15.131:0/519864514 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fef640164d0 con 0x7fef7406e2c0 2024-09-17T02:49:59.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.876+0000 7fef7917a640 1 -- 172.21.15.131:0/519864514 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fef74165350 con 0x7fef7406e2c0 2024-09-17T02:49:59.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.876+0000 7fef5b7fe640 1 -- 172.21.15.131:0/519864514 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) ==== 84+0+65 (secure 0 0 0) 0x7fef640f9350 con 0x7fef7406e2c0 2024-09-17T02:49:59.874 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:49:59.874 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:49:59.874 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:49:59.874 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-09-17T02:49:59.874 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-09-17T02:49:59.874 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:49:59.874 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:49:59.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 -- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef60073200 msgr2=0x7fef60075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef60073200 0x7fef60075710 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fef740c4890 tx=0x7fef6800a310 comp rx=0 tx=0).stop 2024-09-17T02:49:59.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 -- 172.21.15.131:0/519864514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef7406e2c0 msgr2=0x7fef741567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:49:59.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef7406e2c0 0x7fef741567d0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fef640b0040 tx=0x7fef640a6770 comp rx=0 tx=0).stop 2024-09-17T02:49:59.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 -- 172.21.15.131:0/519864514 shutdown_connections 2024-09-17T02:49:59.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef60073200 0x7fef60075710 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef740cdbf0 0x7fef740cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef740c4ac0 0x7fef741552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 --2- 172.21.15.131:0/519864514 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef7406e2c0 0x7fef741567d0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:49:59.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 -- 172.21.15.131:0/519864514 >> 172.21.15.131:0/519864514 conn(0x7fef7406eea0 msgr2=0x7fef74164620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:49:59.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 -- 172.21.15.131:0/519864514 shutdown_connections 2024-09-17T02:49:59.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:49:59.880+0000 7fef7917a640 1 -- 172.21.15.131:0/519864514 wait complete. 2024-09-17T02:49:59.896 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T02:49:59.896 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T02:50:00.057 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.060+0000 7f4e51eb8640 1 -- 172.21.15.131:0/4058620169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e4c074f60 msgr2=0x7f4e4c0b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:00.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.060+0000 7f4e51eb8640 1 --2- 172.21.15.131:0/4058620169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e4c074f60 0x7f4e4c0b48a0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009840 tx=0x7f4e3401c790 comp rx=0 tx=0).stop 2024-09-17T02:50:00.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.060+0000 7f4e51eb8640 1 -- 172.21.15.131:0/4058620169 shutdown_connections 2024-09-17T02:50:00.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.060+0000 7f4e51eb8640 1 --2- 172.21.15.131:0/4058620169 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e4c074f60 0x7f4e4c0b48a0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:00.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.060+0000 7f4e51eb8640 1 -- 172.21.15.131:0/4058620169 >> 172.21.15.131:0/4058620169 conn(0x7f4e4c073d30 msgr2=0x7f4e4c074140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:00.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.060+0000 7f4e51eb8640 1 -- 172.21.15.131:0/4058620169 shutdown_connections 2024-09-17T02:50:00.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.060+0000 7f4e51eb8640 1 -- 172.21.15.131:0/4058620169 wait complete. 2024-09-17T02:50:00.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.060+0000 7f4e51eb8640 1 Processor -- start 2024-09-17T02:50:00.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.060+0000 7f4e51eb8640 1 -- start start 2024-09-17T02:50:00.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e51eb8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e4c15b810 0x7f4e4c1fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:00.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e51eb8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c15bc20 0x7f4e4c14b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:00.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e51eb8640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4e4c1f9350 0x7f4e4c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:00.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e51eb8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c169970 con 0x7f4e4c15b810 2024-09-17T02:50:00.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e51eb8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c1697f0 con 0x7f4e4c15bc20 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e51eb8640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4e4c169af0 con 0x7f4e4c1f9350 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e4b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e4c15b810 0x7f4e4c1fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e4b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e4c15b810 0x7f4e4c1fc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55730/0 (socket says 172.21.15.131:55730) 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e4b7fe640 1 -- 172.21.15.131:0/2755225288 learned_addr learned my addr 172.21.15.131:0/2755225288 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e42ffd640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c15bc20 0x7f4e4c14b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e4bfff640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4e4c1f9350 0x7f4e4c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e4b7fe640 1 -- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4e4c1f9350 msgr2=0x7f4e4c15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e4b7fe640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4e4c1f9350 0x7f4e4c15a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e4b7fe640 1 -- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c15bc20 msgr2=0x7f4e4c14b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e4b7fe640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c15bc20 0x7f4e4c14b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e4b7fe640 1 -- 172.21.15.131:0/2755225288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c15e790 con 0x7f4e4c15b810 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e42ffd640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c15bc20 0x7f4e4c14b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:00.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e4b7fe640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e4c15b810 0x7f4e4c1fc680 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00c3c0 tx=0x7f4e3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e48ff9640 1 -- 172.21.15.131:0/2755225288 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c017070 con 0x7f4e4c15b810 2024-09-17T02:50:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e48ff9640 1 -- 172.21.15.131:0/2755225288 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e3c003600 con 0x7f4e4c15b810 2024-09-17T02:50:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e51eb8640 1 -- 172.21.15.131:0/2755225288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4c1f8870 con 0x7f4e4c15b810 2024-09-17T02:50:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e48ff9640 1 -- 172.21.15.131:0/2755225288 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c003a50 con 0x7f4e4c15b810 2024-09-17T02:50:00.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e51eb8640 1 -- 172.21.15.131:0/2755225288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4c1f8e30 con 0x7f4e4c15b810 2024-09-17T02:50:00.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e51eb8640 1 -- 172.21.15.131:0/2755225288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4c074850 con 0x7f4e4c15b810 2024-09-17T02:50:00.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.064+0000 7f4e48ff9640 1 -- 172.21.15.131:0/2755225288 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4e3c01a070 con 0x7f4e4c15b810 2024-09-17T02:50:00.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.068+0000 7f4e48ff9640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4e2c073200 0x7f4e2c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:00.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.068+0000 7f4e48ff9640 1 -- 172.21.15.131:0/2755225288 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5249+0+0 (secure 0 0 0) 0x7f4e3c0932f0 con 0x7f4e4c15b810 2024-09-17T02:50:00.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.068+0000 7f4e42ffd640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4e2c073200 0x7f4e2c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:00.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.068+0000 7f4e42ffd640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4e2c073200 0x7f4e2c075710 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4e340af000 tx=0x7f4e34019530 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:00.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.072+0000 7f4e48ff9640 1 -- 172.21.15.131:0/2755225288 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4e3c05e4d0 con 0x7f4e4c15b810 2024-09-17T02:50:00.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:00.236+0000 7f4e51eb8640 1 -- 172.21.15.131:0/2755225288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4e4c1f8630 con 0x7f4e4c15b810 2024-09-17T02:50:01.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.148+0000 7f4e48ff9640 1 -- 172.21.15.131:0/2755225288 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) ==== 138+0+0 (secure 0 0 0) 0x7f4e3c05ddd0 con 0x7f4e4c15b810 2024-09-17T02:50:01.147 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-09-17T02:50:01.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.152+0000 7f4e51eb8640 1 -- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4e2c073200 msgr2=0x7f4e2c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:01.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.152+0000 7f4e51eb8640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4e2c073200 0x7f4e2c075710 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4e340af000 tx=0x7f4e34019530 comp rx=0 tx=0).stop 2024-09-17T02:50:01.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.152+0000 7f4e51eb8640 1 -- 172.21.15.131:0/2755225288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e4c15b810 msgr2=0x7f4e4c1fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:01.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.152+0000 7f4e51eb8640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e4c15b810 0x7f4e4c1fc680 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c00c3c0 tx=0x7f4e3c00c880 comp rx=0 tx=0).stop 2024-09-17T02:50:01.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.152+0000 7f4e51eb8640 1 -- 172.21.15.131:0/2755225288 shutdown_connections 2024-09-17T02:50:01.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.152+0000 7f4e51eb8640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4e2c073200 0x7f4e2c075710 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:01.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.152+0000 7f4e51eb8640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4e4c1f9350 0x7f4e4c15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:01.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.152+0000 7f4e51eb8640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4e4c15bc20 0x7f4e4c14b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:01.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.152+0000 7f4e51eb8640 1 --2- 172.21.15.131:0/2755225288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4e4c15b810 0x7f4e4c1fc680 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:01.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.152+0000 7f4e51eb8640 1 -- 172.21.15.131:0/2755225288 >> 172.21.15.131:0/2755225288 conn(0x7f4e4c073d30 msgr2=0x7f4e4c0b4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:01.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.152+0000 7f4e51eb8640 1 -- 172.21.15.131:0/2755225288 shutdown_connections 2024-09-17T02:50:01.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.156+0000 7f4e51eb8640 1 -- 172.21.15.131:0/2755225288 wait complete. 2024-09-17T02:50:01.185 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T02:50:01.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.356+0000 7ff828981640 1 -- 172.21.15.131:0/3859205814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8241622d0 msgr2=0x7ff8241626e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.356+0000 7ff828981640 1 --2- 172.21.15.131:0/3859205814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8241622d0 0x7ff8241626e0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7ff8240d5180 tx=0x7ff81401c890 comp rx=0 tx=0).stop 2024-09-17T02:50:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.356+0000 7ff828981640 1 -- 172.21.15.131:0/3859205814 shutdown_connections 2024-09-17T02:50:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.356+0000 7ff828981640 1 --2- 172.21.15.131:0/3859205814 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff8241622d0 0x7ff8241626e0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.356+0000 7ff828981640 1 --2- 172.21.15.131:0/3859205814 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff82415ece0 0x7ff82415f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.356+0000 7ff828981640 1 --2- 172.21.15.131:0/3859205814 >> v2:172.21.15.189:3300/0 conn(0x7ff824155640 0x7ff8240cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.356+0000 7ff828981640 1 -- 172.21.15.131:0/3859205814 >> 172.21.15.131:0/3859205814 conn(0x7ff8240d1b90 msgr2=0x7ff8240d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.356+0000 7ff828981640 1 -- 172.21.15.131:0/3859205814 shutdown_connections 2024-09-17T02:50:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.356+0000 7ff828981640 1 -- 172.21.15.131:0/3859205814 wait complete. 2024-09-17T02:50:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff828981640 1 Processor -- start 2024-09-17T02:50:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff828981640 1 -- start start 2024-09-17T02:50:01.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff828981640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff824155640 0x7ff82414f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:01.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff828981640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82415ece0 0x7ff8240cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:01.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff828981640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8241622d0 0x7ff8240c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:01.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff828981640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff82416ac70 con 0x7ff82415ece0 2024-09-17T02:50:01.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff828981640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff824155c60 con 0x7ff8241622d0 2024-09-17T02:50:01.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff828981640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff82416adf0 con 0x7ff824155640 2024-09-17T02:50:01.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff821d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82415ece0 0x7ff8240cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:01.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff821d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82415ece0 0x7ff8240cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50572/0 (socket says 172.21.15.131:50572) 2024-09-17T02:50:01.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff821d9b640 1 -- 172.21.15.131:0/413942353 learned_addr learned my addr 172.21.15.131:0/413942353 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:01.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff82259c640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff824155640 0x7ff82414f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:01.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff822d9d640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8241622d0 0x7ff8240c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff821d9b640 1 -- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff824155640 msgr2=0x7ff82414f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff821d9b640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff824155640 0x7ff82414f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff821d9b640 1 -- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8241622d0 msgr2=0x7ff8240c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff821d9b640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8241622d0 0x7ff8240c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff821d9b640 1 -- 172.21.15.131:0/413942353 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8241f9bd0 con 0x7ff82415ece0 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff822d9d640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8241622d0 0x7ff8240c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff821d9b640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82415ece0 0x7ff8240cd2f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff8140b0040 tx=0x7ff814004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff81affd640 1 -- 172.21.15.131:0/413942353 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8140b2040 con 0x7ff82415ece0 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff81affd640 1 -- 172.21.15.131:0/413942353 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff814005030 con 0x7ff82415ece0 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff828981640 1 -- 172.21.15.131:0/413942353 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8241f8e10 con 0x7ff82415ece0 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff81affd640 1 -- 172.21.15.131:0/413942353 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814005460 con 0x7ff82415ece0 2024-09-17T02:50:01.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff828981640 1 -- 172.21.15.131:0/413942353 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8241f93d0 con 0x7ff82415ece0 2024-09-17T02:50:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.360+0000 7ff828981640 1 -- 172.21.15.131:0/413942353 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82415f0f0 con 0x7ff82415ece0 2024-09-17T02:50:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.364+0000 7ff81affd640 1 -- 172.21.15.131:0/413942353 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff8140bd060 con 0x7ff82415ece0 2024-09-17T02:50:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.364+0000 7ff81affd640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff804073200 0x7ff804075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.364+0000 7ff81affd640 1 -- 172.21.15.131:0/413942353 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5619+0+0 (secure 0 0 0) 0x7ff81412d7b0 con 0x7ff82415ece0 2024-09-17T02:50:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.364+0000 7ff82259c640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff804073200 0x7ff804075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.364+0000 7ff82259c640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff804073200 0x7ff804075710 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff80c005fd0 tx=0x7ff80c005de0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:01.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.368+0000 7ff81affd640 1 -- 172.21.15.131:0/413942353 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff8140164b0 con 0x7ff82415ece0 2024-09-17T02:50:01.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:01.540+0000 7ff828981640 1 -- 172.21.15.131:0/413942353 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff8240cdb40 con 0x7ff82415ece0 2024-09-17T02:50:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.156+0000 7ff81affd640 1 -- 172.21.15.131:0/413942353 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) ==== 170+0+0 (secure 0 0 0) 0x7ff8140fde20 con 0x7ff82415ece0 2024-09-17T02:50:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 -- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff804073200 msgr2=0x7ff804075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff804073200 0x7ff804075710 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff80c005fd0 tx=0x7ff80c005de0 comp rx=0 tx=0).stop 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 -- 172.21.15.131:0/413942353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82415ece0 msgr2=0x7ff8240cd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82415ece0 0x7ff8240cd2f0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7ff8140b0040 tx=0x7ff814004040 comp rx=0 tx=0).stop 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 -- 172.21.15.131:0/413942353 shutdown_connections 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff804073200 0x7ff804075710 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff8241622d0 0x7ff8240c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff82415ece0 0x7ff8240cd2f0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 --2- 172.21.15.131:0/413942353 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff824155640 0x7ff82414f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 -- 172.21.15.131:0/413942353 >> 172.21.15.131:0/413942353 conn(0x7ff8240d1b90 msgr2=0x7ff8240b9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 -- 172.21.15.131:0/413942353 shutdown_connections 2024-09-17T02:50:02.157 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.160+0000 7ff828981640 1 -- 172.21.15.131:0/413942353 wait complete. 2024-09-17T02:50:02.193 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T02:50:02.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.360+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/4277950493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd480cce40 msgr2=0x7fcd4806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:02.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.360+0000 7fcd4eb1b640 1 --2- 172.21.15.131:0/4277950493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd480cce40 0x7fcd4806db20 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fcd38009840 tx=0x7fcd3801c760 comp rx=0 tx=0).stop 2024-09-17T02:50:02.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.360+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/4277950493 shutdown_connections 2024-09-17T02:50:02.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.360+0000 7fcd4eb1b640 1 --2- 172.21.15.131:0/4277950493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd481606a0 0x7fcd48152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:02.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.360+0000 7fcd4eb1b640 1 --2- 172.21.15.131:0/4277950493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd480cce40 0x7fcd4806db20 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:02.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.360+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/4277950493 >> 172.21.15.131:0/4277950493 conn(0x7fcd4806fbe0 msgr2=0x7fcd4806fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:02.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.360+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/4277950493 shutdown_connections 2024-09-17T02:50:02.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.360+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/4277950493 wait complete. 2024-09-17T02:50:02.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4eb1b640 1 Processor -- start 2024-09-17T02:50:02.359 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4eb1b640 1 -- start start 2024-09-17T02:50:02.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4eb1b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd480cce40 0x7fcd480c8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:02.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4eb1b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd480cd250 0x7fcd4815c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:02.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4eb1b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd4814af40 0x7fcd480c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:02.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4eb1b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcd480cc040 con 0x7fcd480cce40 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4eb1b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcd480cbec0 con 0x7fcd480cd250 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4eb1b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcd480cc1c0 con 0x7fcd4814af40 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4c8b7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd480cce40 0x7fcd480c8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4c8b7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd480cce40 0x7fcd480c8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50588/0 (socket says 172.21.15.131:50588) 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4c8b7640 1 -- 172.21.15.131:0/614196844 learned_addr learned my addr 172.21.15.131:0/614196844 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4d0b8640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd4814af40 0x7fcd480c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4c8b7640 1 -- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd4814af40 msgr2=0x7fcd480c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd47fff640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd480cd250 0x7fcd4815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4c8b7640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd4814af40 0x7fcd480c5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4c8b7640 1 -- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd480cd250 msgr2=0x7fcd4815c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4c8b7640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd480cd250 0x7fcd4815c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4c8b7640 1 -- 172.21.15.131:0/614196844 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd480cf9e0 con 0x7fcd480cce40 2024-09-17T02:50:02.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4c8b7640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd480cce40 0x7fcd480c8d40 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fcd3801cc40 tx=0x7fcd38005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd457fa640 1 -- 172.21.15.131:0/614196844 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd380b2040 con 0x7fcd480cce40 2024-09-17T02:50:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd457fa640 1 -- 172.21.15.131:0/614196844 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd380bb070 con 0x7fcd480cce40 2024-09-17T02:50:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/614196844 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd480cfcc0 con 0x7fcd480cce40 2024-09-17T02:50:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd457fa640 1 -- 172.21.15.131:0/614196844 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd38003600 con 0x7fcd480cce40 2024-09-17T02:50:02.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/614196844 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd4815c130 con 0x7fcd480cce40 2024-09-17T02:50:02.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.364+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/614196844 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd480cc6b0 con 0x7fcd480cce40 2024-09-17T02:50:02.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.368+0000 7fcd457fa640 1 -- 172.21.15.131:0/614196844 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcd380037a0 con 0x7fcd480cce40 2024-09-17T02:50:02.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.368+0000 7fcd457fa640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd34073200 0x7fcd34075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:02.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.368+0000 7fcd457fa640 1 -- 172.21.15.131:0/614196844 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 6001+0+0 (secure 0 0 0) 0x7fcd3812f1c0 con 0x7fcd480cce40 2024-09-17T02:50:02.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.368+0000 7fcd47fff640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd34073200 0x7fcd34075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:02.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.368+0000 7fcd47fff640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd34073200 0x7fcd34075710 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcd480cb130 tx=0x7fcd3c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:02.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.372+0000 7fcd457fa640 1 -- 172.21.15.131:0/614196844 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcd38016410 con 0x7fcd480cce40 2024-09-17T02:50:02.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:02.576+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/614196844 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcd481518b0 con 0x7fcd480cce40 2024-09-17T02:50:03.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.220+0000 7fcd457fa640 1 -- 172.21.15.131:0/614196844 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v82) ==== 174+0+0 (secure 0 0 0) 0x7fcd380fa220 con 0x7fcd480cce40 2024-09-17T02:50:03.227 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 15 and data pool 16 2024-09-17T02:50:03.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd34073200 msgr2=0x7fcd34075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd34073200 0x7fcd34075710 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcd480cb130 tx=0x7fcd3c009040 comp rx=0 tx=0).stop 2024-09-17T02:50:03.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/614196844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd480cce40 msgr2=0x7fcd480c8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd480cce40 0x7fcd480c8d40 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fcd3801cc40 tx=0x7fcd38005db0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/614196844 shutdown_connections 2024-09-17T02:50:03.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcd34073200 0x7fcd34075710 secure :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcd480cb130 tx=0x7fcd3c009040 comp rx=0 tx=0).stop 2024-09-17T02:50:03.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcd4814af40 0x7fcd480c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcd480cd250 0x7fcd4815c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 --2- 172.21.15.131:0/614196844 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcd480cce40 0x7fcd480c8d40 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/614196844 >> 172.21.15.131:0/614196844 conn(0x7fcd4806fbe0 msgr2=0x7fcd4814fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:03.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/614196844 shutdown_connections 2024-09-17T02:50:03.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.240+0000 7fcd4eb1b640 1 -- 172.21.15.131:0/614196844 wait complete. 2024-09-17T02:50:03.272 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:50:03.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.424+0000 7f9cc9b04640 1 -- 172.21.15.131:0/386411661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc4074310 msgr2=0x7f9cc40af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.424+0000 7f9cc9b04640 1 --2- 172.21.15.131:0/386411661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc4074310 0x7f9cc40af4c0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009840 tx=0x7f9cb401c700 comp rx=0 tx=0).stop 2024-09-17T02:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.424+0000 7f9cc9b04640 1 -- 172.21.15.131:0/386411661 shutdown_connections 2024-09-17T02:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.424+0000 7f9cc9b04640 1 --2- 172.21.15.131:0/386411661 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc4074310 0x7f9cc40af4c0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.424+0000 7f9cc9b04640 1 -- 172.21.15.131:0/386411661 >> 172.21.15.131:0/386411661 conn(0x7f9cc40d13d0 msgr2=0x7f9cc40734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.424+0000 7f9cc9b04640 1 -- 172.21.15.131:0/386411661 shutdown_connections 2024-09-17T02:50:03.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.424+0000 7f9cc9b04640 1 -- 172.21.15.131:0/386411661 wait complete. 2024-09-17T02:50:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc9b04640 1 Processor -- start 2024-09-17T02:50:03.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc9b04640 1 -- start start 2024-09-17T02:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc9b04640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc40cfac0 0x7f9cc4144300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc9b04640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc40cfed0 0x7f9cc414b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc9b04640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9cc414ba20 0x7f9cc414be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc9b04640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc406fc30 con 0x7f9cc40cfed0 2024-09-17T02:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc9b04640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc406fab0 con 0x7f9cc40cfac0 2024-09-17T02:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc9b04640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc406fdb0 con 0x7f9cc414ba20 2024-09-17T02:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc40cfed0 0x7f9cc414b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc40cfed0 0x7f9cc414b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50608/0 (socket says 172.21.15.131:50608) 2024-09-17T02:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc2ffd640 1 -- 172.21.15.131:0/1689970355 learned_addr learned my addr 172.21.15.131:0/1689970355 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:03.426 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc3fff640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9cc414ba20 0x7f9cc414be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc2ffd640 1 -- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9cc414ba20 msgr2=0x7f9cc414be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc2ffd640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9cc414ba20 0x7f9cc414be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc2ffd640 1 -- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc40cfac0 msgr2=0x7f9cc4144300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc37fe640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc40cfac0 0x7f9cc4144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc2ffd640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc40cfac0 0x7f9cc4144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc2ffd640 1 -- 172.21.15.131:0/1689970355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc4144840 con 0x7f9cc40cfed0 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc3fff640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9cc414ba20 0x7f9cc414be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc2ffd640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc40cfed0 0x7f9cc414b4e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00c3a0 tx=0x7f9cac00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cbbfff640 1 -- 172.21.15.131:0/1689970355 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac017070 con 0x7f9cc40cfed0 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cbbfff640 1 -- 172.21.15.131:0/1689970355 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cac003600 con 0x7f9cc40cfed0 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc9b04640 1 -- 172.21.15.131:0/1689970355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc41521c0 con 0x7f9cc40cfed0 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cbbfff640 1 -- 172.21.15.131:0/1689970355 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cac003a50 con 0x7f9cc40cfed0 2024-09-17T02:50:03.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc9b04640 1 -- 172.21.15.131:0/1689970355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc4152780 con 0x7f9cc40cfed0 2024-09-17T02:50:03.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.428+0000 7f9cc9b04640 1 -- 172.21.15.131:0/1689970355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc40af4c0 con 0x7f9cc40cfed0 2024-09-17T02:50:03.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.432+0000 7f9cbbfff640 1 -- 172.21.15.131:0/1689970355 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9cac01a070 con 0x7f9cc40cfed0 2024-09-17T02:50:03.428 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.432+0000 7f9cbbfff640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ca4073200 0x7f9ca4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:03.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.432+0000 7f9cbbfff640 1 -- 172.21.15.131:0/1689970355 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f9cac093a00 con 0x7f9cc40cfed0 2024-09-17T02:50:03.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.432+0000 7f9cc37fe640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ca4073200 0x7f9ca4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:03.429 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.432+0000 7f9cc37fe640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ca4073200 0x7f9ca4075710 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc007dc0 tx=0x7f9cbc0072f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:03.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.436+0000 7f9cbbfff640 1 -- 172.21.15.131:0/1689970355 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9cac021020 con 0x7f9cc40cfed0 2024-09-17T02:50:03.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.604+0000 7f9cc9b04640 1 -- 172.21.15.131:0/1689970355 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cc415f380 con 0x7f9cc40cfed0 2024-09-17T02:50:03.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cbbfff640 1 -- 172.21.15.131:0/1689970355 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+17021 (secure 0 0 0) 0x7f9cc415b980 con 0x7f9cc40cfed0 2024-09-17T02:50:03.604 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:03.604 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":106,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:50:02.583790+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:50:00.250118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-17T02:50:01.555264+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:50:03.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 -- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ca4073200 msgr2=0x7f9ca4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ca4073200 0x7f9ca4075710 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc007dc0 tx=0x7f9cbc0072f0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 -- 172.21.15.131:0/1689970355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc40cfed0 msgr2=0x7f9cc414b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc40cfed0 0x7f9cc414b4e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00c3a0 tx=0x7f9cac00c860 comp rx=0 tx=0).stop 2024-09-17T02:50:03.606 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 -- 172.21.15.131:0/1689970355 shutdown_connections 2024-09-17T02:50:03.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ca4073200 0x7f9ca4075710 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9cc414ba20 0x7f9cc414be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cc40cfed0 0x7f9cc414b4e0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 --2- 172.21.15.131:0/1689970355 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cc40cfac0 0x7f9cc4144300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 -- 172.21.15.131:0/1689970355 >> 172.21.15.131:0/1689970355 conn(0x7f9cc40d13d0 msgr2=0x7f9cc40ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:03.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 -- 172.21.15.131:0/1689970355 shutdown_connections 2024-09-17T02:50:03.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.608+0000 7f9cc9b04640 1 -- 172.21.15.131:0/1689970355 wait complete. 2024-09-17T02:50:03.628 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:50:03.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.788+0000 7fcba97db640 1 -- 172.21.15.131:0/3360593199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba406e2c0 msgr2=0x7fcba40c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.788+0000 7fcba97db640 1 --2- 172.21.15.131:0/3360593199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba406e2c0 0x7fcba40c4580 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fcba4074bb0 tx=0x7fcb9401c830 comp rx=0 tx=0).stop 2024-09-17T02:50:03.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.788+0000 7fcba97db640 1 -- 172.21.15.131:0/3360593199 shutdown_connections 2024-09-17T02:50:03.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.788+0000 7fcba97db640 1 --2- 172.21.15.131:0/3360593199 >> v2:172.21.15.189:3300/0 conn(0x7fcba40c4ac0 0x7fcba40c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.788+0000 7fcba97db640 1 --2- 172.21.15.131:0/3360593199 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba406e2c0 0x7fcba40c4580 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.788+0000 7fcba97db640 1 -- 172.21.15.131:0/3360593199 >> 172.21.15.131:0/3360593199 conn(0x7fcba406eea0 msgr2=0x7fcba406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:03.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.788+0000 7fcba97db640 1 -- 172.21.15.131:0/3360593199 shutdown_connections 2024-09-17T02:50:03.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.788+0000 7fcba97db640 1 -- 172.21.15.131:0/3360593199 wait complete. 2024-09-17T02:50:03.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba97db640 1 Processor -- start 2024-09-17T02:50:03.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba97db640 1 -- start start 2024-09-17T02:50:03.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba97db640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba406e2c0 0x7fcba41f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:03.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba97db640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcba40c4ac0 0x7fcba41497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:03.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba97db640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba40cdbf0 0x7fcba415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:03.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba97db640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcba40cef20 con 0x7fcba406e2c0 2024-09-17T02:50:03.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba97db640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcba40cf0a0 con 0x7fcba40cdbf0 2024-09-17T02:50:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba97db640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcba40cf220 con 0x7fcba40c4ac0 2024-09-17T02:50:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba406e2c0 0x7fcba41f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba2ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba406e2c0 0x7fcba41f7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50628/0 (socket says 172.21.15.131:50628) 2024-09-17T02:50:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba2ffd640 1 -- 172.21.15.131:0/3500433682 learned_addr learned my addr 172.21.15.131:0/3500433682 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcb9bfff640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcba40c4ac0 0x7fcba41497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:03.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba37fe640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba40cdbf0 0x7fcba415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:03.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba2ffd640 1 -- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcba40c4ac0 msgr2=0x7fcba41497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba2ffd640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcba40c4ac0 0x7fcba41497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba2ffd640 1 -- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba40cdbf0 msgr2=0x7fcba415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba2ffd640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba40cdbf0 0x7fcba415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.796 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba2ffd640 1 -- 172.21.15.131:0/3500433682 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba4156d40 con 0x7fcba406e2c0 2024-09-17T02:50:03.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.792+0000 7fcba37fe640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba40cdbf0 0x7fcba415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:03.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.796+0000 7fcba2ffd640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba406e2c0 0x7fcba41f7070 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fcb940b0040 tx=0x7fcb940a66c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:03.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.796+0000 7fcb9b7fe640 1 -- 172.21.15.131:0/3500433682 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb940bc070 con 0x7fcba406e2c0 2024-09-17T02:50:03.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.796+0000 7fcb9b7fe640 1 -- 172.21.15.131:0/3500433682 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb94004470 con 0x7fcba406e2c0 2024-09-17T02:50:03.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.796+0000 7fcba97db640 1 -- 172.21.15.131:0/3500433682 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba4156fe0 con 0x7fcba406e2c0 2024-09-17T02:50:03.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.796+0000 7fcb9b7fe640 1 -- 172.21.15.131:0/3500433682 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94005290 con 0x7fcba406e2c0 2024-09-17T02:50:03.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.796+0000 7fcba97db640 1 -- 172.21.15.131:0/3500433682 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba4149d10 con 0x7fcba406e2c0 2024-09-17T02:50:03.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.796+0000 7fcba97db640 1 -- 172.21.15.131:0/3500433682 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba40c4580 con 0x7fcba406e2c0 2024-09-17T02:50:03.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.796+0000 7fcb9b7fe640 1 -- 172.21.15.131:0/3500433682 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcb940bf070 con 0x7fcba406e2c0 2024-09-17T02:50:03.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.796+0000 7fcb9b7fe640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb84073200 0x7fcb84075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:03.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.796+0000 7fcb9b7fe640 1 -- 172.21.15.131:0/3500433682 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7fcb9412e630 con 0x7fcba406e2c0 2024-09-17T02:50:03.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.796+0000 7fcb9bfff640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb84073200 0x7fcb84075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:03.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.800+0000 7fcb9bfff640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb84073200 0x7fcb84075710 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0058d0 tx=0x7fcb8c005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:03.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.800+0000 7fcb9b7fe640 1 -- 172.21.15.131:0/3500433682 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcb940164b0 con 0x7fcba406e2c0 2024-09-17T02:50:03.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcba97db640 1 -- 172.21.15.131:0/3500433682 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcba4073310 con 0x7fcba406e2c0 2024-09-17T02:50:03.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcb9b7fe640 1 -- 172.21.15.131:0/3500433682 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+17021 (secure 0 0 0) 0x7fcb940f96c0 con 0x7fcba406e2c0 2024-09-17T02:50:03.972 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:03.972 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":106,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:50:02.583790+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:50:00.250118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-17T02:50:01.555264+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:50:03.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcba97db640 1 -- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb84073200 msgr2=0x7fcb84075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcba97db640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb84073200 0x7fcb84075710 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0058d0 tx=0x7fcb8c005860 comp rx=0 tx=0).stop 2024-09-17T02:50:03.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcba97db640 1 -- 172.21.15.131:0/3500433682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba406e2c0 msgr2=0x7fcba41f7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:03.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcba97db640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba406e2c0 0x7fcba41f7070 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fcb940b0040 tx=0x7fcb940a66c0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcba97db640 1 -- 172.21.15.131:0/3500433682 shutdown_connections 2024-09-17T02:50:03.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcba97db640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb84073200 0x7fcb84075710 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcba97db640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcba40cdbf0 0x7fcba415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcba97db640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcba40c4ac0 0x7fcba41497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcba97db640 1 --2- 172.21.15.131:0/3500433682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcba406e2c0 0x7fcba41f7070 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:03.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.976+0000 7fcba97db640 1 -- 172.21.15.131:0/3500433682 >> 172.21.15.131:0/3500433682 conn(0x7fcba406eea0 msgr2=0x7fcba40ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:03.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.980+0000 7fcba97db640 1 -- 172.21.15.131:0/3500433682 shutdown_connections 2024-09-17T02:50:03.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:03.980+0000 7fcba97db640 1 -- 172.21.15.131:0/3500433682 wait complete. 2024-09-17T02:50:03.996 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T02:50:04.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.164+0000 7f3a3bb97640 1 -- 172.21.15.131:0/3254440694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a34155240 msgr2=0x7f3a34073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:04.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.164+0000 7f3a3bb97640 1 --2- 172.21.15.131:0/3254440694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a34155240 0x7f3a34073430 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3a3406a1c0 tx=0x7f3a3001d9e0 comp rx=0 tx=0).stop 2024-09-17T02:50:04.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.164+0000 7f3a3bb97640 1 -- 172.21.15.131:0/3254440694 shutdown_connections 2024-09-17T02:50:04.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.164+0000 7f3a3bb97640 1 --2- 172.21.15.131:0/3254440694 >> v2:172.21.15.189:3300/0 conn(0x7f3a34073970 0x7f3a3406dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:04.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.164+0000 7f3a3bb97640 1 --2- 172.21.15.131:0/3254440694 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a34155240 0x7f3a34073430 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:04.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.164+0000 7f3a3bb97640 1 -- 172.21.15.131:0/3254440694 >> 172.21.15.131:0/3254440694 conn(0x7f3a3406eea0 msgr2=0x7f3a3406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:04.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.164+0000 7f3a3bb97640 1 -- 172.21.15.131:0/3254440694 shutdown_connections 2024-09-17T02:50:04.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.164+0000 7f3a3bb97640 1 -- 172.21.15.131:0/3254440694 wait complete. 2024-09-17T02:50:04.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.164+0000 7f3a3bb97640 1 Processor -- start 2024-09-17T02:50:04.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.164+0000 7f3a3bb97640 1 -- start start 2024-09-17T02:50:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a3bb97640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a3406e3e0 0x7f3a3415a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a3bb97640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a34073970 0x7f3a341faa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a3bb97640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a34155240 0x7f3a34150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a3bb97640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a34167d80 con 0x7f3a3406e3e0 2024-09-17T02:50:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a3bb97640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3415f640 con 0x7f3a34155240 2024-09-17T02:50:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a3bb97640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3a34167f00 con 0x7f3a34073970 2024-09-17T02:50:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a39933640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a3406e3e0 0x7f3a3415a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a39933640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a3406e3e0 0x7f3a3415a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50656/0 (socket says 172.21.15.131:50656) 2024-09-17T02:50:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a39933640 1 -- 172.21.15.131:0/1809982961 learned_addr learned my addr 172.21.15.131:0/1809982961 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a39132640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a34073970 0x7f3a341faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:04.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a3a134640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a34155240 0x7f3a34150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a39933640 1 -- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a34073970 msgr2=0x7f3a341faa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a39933640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a34073970 0x7f3a341faa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a39933640 1 -- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a34155240 msgr2=0x7f3a34150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a39933640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a34155240 0x7f3a34150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a39933640 1 -- 172.21.15.131:0/1809982961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a34159920 con 0x7f3a3406e3e0 2024-09-17T02:50:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a3a134640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a34155240 0x7f3a34150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a39933640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a3406e3e0 0x7f3a3415a690 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3a300b1040 tx=0x7f3a30004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a2a7fc640 1 -- 172.21.15.131:0/1809982961 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a300b3070 con 0x7f3a3406e3e0 2024-09-17T02:50:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a2a7fc640 1 -- 172.21.15.131:0/1809982961 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a300038f0 con 0x7f3a3406e3e0 2024-09-17T02:50:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a3bb97640 1 -- 172.21.15.131:0/1809982961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a34159c00 con 0x7f3a3406e3e0 2024-09-17T02:50:04.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a2a7fc640 1 -- 172.21.15.131:0/1809982961 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a30003d40 con 0x7f3a3406e3e0 2024-09-17T02:50:04.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a3bb97640 1 -- 172.21.15.131:0/1809982961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a34151120 con 0x7f3a3406e3e0 2024-09-17T02:50:04.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a3bb97640 1 -- 172.21.15.131:0/1809982961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a3406dea0 con 0x7f3a3406e3e0 2024-09-17T02:50:04.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a2a7fc640 1 -- 172.21.15.131:0/1809982961 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3a300bd060 con 0x7f3a3406e3e0 2024-09-17T02:50:04.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a2a7fc640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a18073230 0x7f3a18075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:04.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.168+0000 7f3a2a7fc640 1 -- 172.21.15.131:0/1809982961 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6101+0+0 (secure 0 0 0) 0x7f3a3012dac0 con 0x7f3a3406e3e0 2024-09-17T02:50:04.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.172+0000 7f3a39132640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a18073230 0x7f3a18075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:04.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.172+0000 7f3a39132640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a18073230 0x7f3a18075740 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3a34073840 tx=0x7f3a24005bf0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:04.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.176+0000 7f3a2a7fc640 1 -- 172.21.15.131:0/1809982961 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3a3406e7f0 con 0x7f3a3406e3e0 2024-09-17T02:50:04.378 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:04.380+0000 7f3a3bb97640 1 -- 172.21.15.131:0/1809982961 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3a340aad10 con 0x7f3a3406e3e0 2024-09-17T02:50:05.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.244+0000 7f3a2a7fc640 1 -- 172.21.15.131:0/1809982961 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) ==== 118+0+0 (secure 0 0 0) 0x7f3a300f8a70 con 0x7f3a3406e3e0 2024-09-17T02:50:05.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 -- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a18073230 msgr2=0x7f3a18075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:05.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a18073230 0x7f3a18075740 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3a34073840 tx=0x7f3a24005bf0 comp rx=0 tx=0).stop 2024-09-17T02:50:05.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 -- 172.21.15.131:0/1809982961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a3406e3e0 msgr2=0x7f3a3415a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:05.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a3406e3e0 0x7f3a3415a690 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3a300b1040 tx=0x7f3a30004150 comp rx=0 tx=0).stop 2024-09-17T02:50:05.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 -- 172.21.15.131:0/1809982961 shutdown_connections 2024-09-17T02:50:05.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a18073230 0x7f3a18075740 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:05.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a34155240 0x7f3a34150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:05.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a34073970 0x7f3a341faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:05.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 --2- 172.21.15.131:0/1809982961 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a3406e3e0 0x7f3a3415a690 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:05.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 -- 172.21.15.131:0/1809982961 >> 172.21.15.131:0/1809982961 conn(0x7f3a3406eea0 msgr2=0x7f3a3406d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:05.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 -- 172.21.15.131:0/1809982961 shutdown_connections 2024-09-17T02:50:05.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.248+0000 7f3a3bb97640 1 -- 172.21.15.131:0/1809982961 wait complete. 2024-09-17T02:50:05.266 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T02:50:05.266 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T02:50:05.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.432+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/1479429698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015c5b0 msgr2=0x7f9ab00cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:05.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.432+0000 7f9ab5c8e640 1 --2- 172.21.15.131:0/1479429698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015c5b0 0x7f9ab00cbaf0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f9aa8009bc0 tx=0x7f9aa801cba0 comp rx=0 tx=0).stop 2024-09-17T02:50:05.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.432+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/1479429698 shutdown_connections 2024-09-17T02:50:05.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.432+0000 7f9ab5c8e640 1 --2- 172.21.15.131:0/1479429698 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015c5b0 0x7f9ab00cbaf0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:05.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.432+0000 7f9ab5c8e640 1 --2- 172.21.15.131:0/1479429698 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9ab00ced00 0x7f9ab006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:05.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.432+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/1479429698 >> 172.21.15.131:0/1479429698 conn(0x7f9ab006eea0 msgr2=0x7f9ab006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:05.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/1479429698 shutdown_connections 2024-09-17T02:50:05.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/1479429698 wait complete. 2024-09-17T02:50:05.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9ab5c8e640 1 Processor -- start 2024-09-17T02:50:05.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9ab5c8e640 1 -- start start 2024-09-17T02:50:05.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9ab5c8e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab006e3e0 0x7f9ab0146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:05.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9ab5c8e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9ab00ced00 0x7f9ab01f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:05.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9ab5c8e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015c5b0 0x7f9ab015acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:05.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9ab5c8e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0168720 con 0x7f9ab015c5b0 2024-09-17T02:50:05.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9ab5c8e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01685a0 con 0x7f9ab006e3e0 2024-09-17T02:50:05.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9ab5c8e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab01688a0 con 0x7f9ab00ced00 2024-09-17T02:50:05.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9aaffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015c5b0 0x7f9ab015acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:05.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9aaffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015c5b0 0x7f9ab015acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50672/0 (socket says 172.21.15.131:50672) 2024-09-17T02:50:05.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9aaffff640 1 -- 172.21.15.131:0/2326681150 learned_addr learned my addr 172.21.15.131:0/2326681150 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:05.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9aaf7fe640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab006e3e0 0x7f9ab0146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:05.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9aaeffd640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9ab00ced00 0x7f9ab01f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:05.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9aaffff640 1 -- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9ab00ced00 msgr2=0x7f9ab01f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:05.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9aaffff640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9ab00ced00 0x7f9ab01f5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:05.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9aaffff640 1 -- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab006e3e0 msgr2=0x7f9ab0146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:05.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9aaffff640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab006e3e0 0x7f9ab0146410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:05.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.436+0000 7f9aaffff640 1 -- 172.21.15.131:0/2326681150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab0158270 con 0x7f9ab015c5b0 2024-09-17T02:50:05.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.440+0000 7f9aaffff640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015c5b0 0x7f9ab015acc0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80b0040 tx=0x7f9aa8005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:05.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.440+0000 7f9aa7fff640 1 -- 172.21.15.131:0/2326681150 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa80b2040 con 0x7f9ab015c5b0 2024-09-17T02:50:05.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.440+0000 7f9aa7fff640 1 -- 172.21.15.131:0/2326681150 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9aa8004040 con 0x7f9ab015c5b0 2024-09-17T02:50:05.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.440+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/2326681150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab0146950 con 0x7f9ab015c5b0 2024-09-17T02:50:05.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.440+0000 7f9aa7fff640 1 -- 172.21.15.131:0/2326681150 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8004210 con 0x7f9ab015c5b0 2024-09-17T02:50:05.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.440+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/2326681150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab01f6ce0 con 0x7f9ab015c5b0 2024-09-17T02:50:05.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.440+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/2326681150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab01f6770 con 0x7f9ab015c5b0 2024-09-17T02:50:05.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.440+0000 7f9aa7fff640 1 -- 172.21.15.131:0/2326681150 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9aa80bd030 con 0x7f9ab015c5b0 2024-09-17T02:50:05.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.440+0000 7f9aa7fff640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9a90073200 0x7f9a90075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:05.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.440+0000 7f9aa7fff640 1 -- 172.21.15.131:0/2326681150 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f9aa812e1e0 con 0x7f9ab015c5b0 2024-09-17T02:50:05.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.440+0000 7f9aaf7fe640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9a90073200 0x7f9a90075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:05.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.444+0000 7f9aaf7fe640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9a90073200 0x7f9a90075710 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9a980058d0 tx=0x7f9a98005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:05.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.444+0000 7f9aa7fff640 1 -- 172.21.15.131:0/2326681150 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9aa80c4060 con 0x7f9ab015c5b0 2024-09-17T02:50:05.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:05.656+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/2326681150 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9ab006dea0 con 0x7f9ab015c5b0 2024-09-17T02:50:06.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.252+0000 7f9aa7fff640 1 -- 172.21.15.131:0/2326681150 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) ==== 122+0+0 (secure 0 0 0) 0x7f9aa8016790 con 0x7f9ab015c5b0 2024-09-17T02:50:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9a90073200 msgr2=0x7f9a90075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9a90073200 0x7f9a90075710 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9a980058d0 tx=0x7f9a98005860 comp rx=0 tx=0).stop 2024-09-17T02:50:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/2326681150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015c5b0 msgr2=0x7f9ab015acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015c5b0 0x7f9ab015acc0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f9aa80b0040 tx=0x7f9aa8005a60 comp rx=0 tx=0).stop 2024-09-17T02:50:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/2326681150 shutdown_connections 2024-09-17T02:50:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9a90073200 0x7f9a90075710 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9ab015c5b0 0x7f9ab015acc0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9ab00ced00 0x7f9ab01f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 --2- 172.21.15.131:0/2326681150 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9ab006e3e0 0x7f9ab0146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/2326681150 >> 172.21.15.131:0/2326681150 conn(0x7f9ab006eea0 msgr2=0x7f9ab00b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:06.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/2326681150 shutdown_connections 2024-09-17T02:50:06.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.256+0000 7f9ab5c8e640 1 -- 172.21.15.131:0/2326681150 wait complete. 2024-09-17T02:50:06.279 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T02:50:06.279 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:50:06.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.440+0000 7f4a1613c640 1 -- 172.21.15.131:0/3763297480 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a100cec00 msgr2=0x7f4a100cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.440+0000 7f4a1613c640 1 --2- 172.21.15.131:0/3763297480 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a100cec00 0x7f4a100cf010 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4a08009c80 tx=0x7f4a0801cd10 comp rx=0 tx=0).stop 2024-09-17T02:50:06.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.440+0000 7f4a1613c640 1 -- 172.21.15.131:0/3763297480 shutdown_connections 2024-09-17T02:50:06.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.440+0000 7f4a1613c640 1 --2- 172.21.15.131:0/3763297480 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a100cec00 0x7f4a100cf010 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.440+0000 7f4a1613c640 1 --2- 172.21.15.131:0/3763297480 >> v2:172.21.15.189:3300/0 conn(0x7f4a1006e2c0 0x7f4a100c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.440+0000 7f4a1613c640 1 -- 172.21.15.131:0/3763297480 >> 172.21.15.131:0/3763297480 conn(0x7f4a1006eea0 msgr2=0x7f4a1006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:06.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.440+0000 7f4a1613c640 1 -- 172.21.15.131:0/3763297480 shutdown_connections 2024-09-17T02:50:06.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.440+0000 7f4a1613c640 1 -- 172.21.15.131:0/3763297480 wait complete. 2024-09-17T02:50:06.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a1613c640 1 Processor -- start 2024-09-17T02:50:06.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a1613c640 1 -- start start 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a1613c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a1006e2c0 0x7f4a1014cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a1613c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a100c4ac0 0x7f4a10151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a1613c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a100cec00 0x7f4a10151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a1613c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4a10169c60 con 0x7f4a100c4ac0 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a1613c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4a10169ae0 con 0x7f4a1006e2c0 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a1613c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4a10169de0 con 0x7f4a100cec00 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a0effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a100c4ac0 0x7f4a10151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a0effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a100c4ac0 0x7f4a10151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50686/0 (socket says 172.21.15.131:50686) 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a0effd640 1 -- 172.21.15.131:0/3440095627 learned_addr learned my addr 172.21.15.131:0/3440095627 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a0ffff640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a100cec00 0x7f4a10151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a0f7fe640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a1006e2c0 0x7f4a1014cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:06.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a0effd640 1 -- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a100cec00 msgr2=0x7f4a10151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a0effd640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a100cec00 0x7f4a10151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a0effd640 1 -- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a1006e2c0 msgr2=0x7f4a1014cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a0effd640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a1006e2c0 0x7f4a1014cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a0effd640 1 -- 172.21.15.131:0/3440095627 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a100ce9d0 con 0x7f4a100c4ac0 2024-09-17T02:50:06.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f4a0effd640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a100c4ac0 0x7f4a10151940 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c460 tx=0x7f4a0400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:06.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.444+0000 7f49f7fff640 1 -- 172.21.15.131:0/3440095627 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a040052d0 con 0x7f4a100c4ac0 2024-09-17T02:50:06.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.448+0000 7f49f7fff640 1 -- 172.21.15.131:0/3440095627 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a04005470 con 0x7f4a100c4ac0 2024-09-17T02:50:06.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.448+0000 7f4a1613c640 1 -- 172.21.15.131:0/3440095627 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a10150ea0 con 0x7f4a100c4ac0 2024-09-17T02:50:06.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.448+0000 7f49f7fff640 1 -- 172.21.15.131:0/3440095627 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04012870 con 0x7f4a100c4ac0 2024-09-17T02:50:06.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.448+0000 7f4a1613c640 1 -- 172.21.15.131:0/3440095627 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a10151140 con 0x7f4a100c4ac0 2024-09-17T02:50:06.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.448+0000 7f4a1613c640 1 -- 172.21.15.131:0/3440095627 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a10157b00 con 0x7f4a100c4ac0 2024-09-17T02:50:06.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.448+0000 7f49f7fff640 1 -- 172.21.15.131:0/3440095627 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4a04012a10 con 0x7f4a100c4ac0 2024-09-17T02:50:06.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.448+0000 7f49f7fff640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f49fc073230 0x7f49fc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:06.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.448+0000 7f49f7fff640 1 -- 172.21.15.131:0/3440095627 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f4a04094980 con 0x7f4a100c4ac0 2024-09-17T02:50:06.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.448+0000 7f4a0f7fe640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f49fc073230 0x7f49fc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:06.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.448+0000 7f4a0f7fe640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f49fc073230 0x7f49fc075740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009ad0 tx=0x7f4a000091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:06.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.452+0000 7f49f7fff640 1 -- 172.21.15.131:0/3440095627 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4a0405f9b0 con 0x7f4a100c4ac0 2024-09-17T02:50:06.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.640+0000 7f4a1613c640 1 -- 172.21.15.131:0/3440095627 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a100c4580 con 0x7f4a100c4ac0 2024-09-17T02:50:06.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f49f7fff640 1 -- 172.21.15.131:0/3440095627 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5237 (secure 0 0 0) 0x7f4a0405f2b0 con 0x7f4a100c4ac0 2024-09-17T02:50:06.639 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:06.639 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":86,"btime":"2024-09-17T02:50:06:247461+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:50:05.662097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7860":{"gid":7860,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2930400858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2930400858},{"type":"v1","addr":"172.21.15.131:6835","nonce":2930400858}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7860,"qdb_cluster":[7860]},"id":7}]} 2024-09-17T02:50:06.639 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 86 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 -- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f49fc073230 msgr2=0x7f49fc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f49fc073230 0x7f49fc075740 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009ad0 tx=0x7f4a000091e0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 -- 172.21.15.131:0/3440095627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a100c4ac0 msgr2=0x7f4a10151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a100c4ac0 0x7f4a10151940 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c460 tx=0x7f4a0400c920 comp rx=0 tx=0).stop 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 -- 172.21.15.131:0/3440095627 shutdown_connections 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f49fc073230 0x7f49fc075740 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4a100cec00 0x7f4a10151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4a100c4ac0 0x7f4a10151940 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 --2- 172.21.15.131:0/3440095627 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4a1006e2c0 0x7f4a1014cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 -- 172.21.15.131:0/3440095627 >> 172.21.15.131:0/3440095627 conn(0x7f4a1006eea0 msgr2=0x7f4a1014fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 -- 172.21.15.131:0/3440095627 shutdown_connections 2024-09-17T02:50:06.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.644+0000 7f4a1613c640 1 -- 172.21.15.131:0/3440095627 wait complete. 2024-09-17T02:50:06.660 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:50:06.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.804+0000 7f1e95742640 1 -- 172.21.15.131:0/816588390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e9006e2c0 msgr2=0x7f1e900c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.804+0000 7f1e95742640 1 --2- 172.21.15.131:0/816588390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e9006e2c0 0x7f1e900c4580 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f1e90071d30 tx=0x7f1e8001c830 comp rx=0 tx=0).stop 2024-09-17T02:50:06.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.804+0000 7f1e95742640 1 -- 172.21.15.131:0/816588390 shutdown_connections 2024-09-17T02:50:06.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.804+0000 7f1e95742640 1 --2- 172.21.15.131:0/816588390 >> v2:172.21.15.189:3300/0 conn(0x7f1e900cec00 0x7f1e900cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.804+0000 7f1e95742640 1 --2- 172.21.15.131:0/816588390 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e9006e2c0 0x7f1e900c4580 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.804+0000 7f1e95742640 1 -- 172.21.15.131:0/816588390 >> 172.21.15.131:0/816588390 conn(0x7f1e9006eea0 msgr2=0x7f1e9006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:06.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.804+0000 7f1e95742640 1 -- 172.21.15.131:0/816588390 shutdown_connections 2024-09-17T02:50:06.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.804+0000 7f1e95742640 1 -- 172.21.15.131:0/816588390 wait complete. 2024-09-17T02:50:06.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e95742640 1 Processor -- start 2024-09-17T02:50:06.803 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e95742640 1 -- start start 2024-09-17T02:50:06.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e95742640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e9006e2c0 0x7f1e90148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:06.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e95742640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1e900c4ac0 0x7f1e900c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:06.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e95742640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e900cec00 0x7f1e900c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:06.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e95742640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1e90169c60 con 0x7f1e9006e2c0 2024-09-17T02:50:06.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e95742640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1e90169ae0 con 0x7f1e900cec00 2024-09-17T02:50:06.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e95742640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f1e90169de0 con 0x7f1e900c4ac0 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e8effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e9006e2c0 0x7f1e90148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e8effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e9006e2c0 0x7f1e90148f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50708/0 (socket says 172.21.15.131:50708) 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e8effd640 1 -- 172.21.15.131:0/3766966553 learned_addr learned my addr 172.21.15.131:0/3766966553 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e8effd640 1 -- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1e900c4ac0 msgr2=0x7f1e900c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e8e7fc640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1e900c4ac0 0x7f1e900c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e8f7fe640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e900cec00 0x7f1e900c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e8effd640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1e900c4ac0 0x7f1e900c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e8effd640 1 -- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e900cec00 msgr2=0x7f1e900c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e8effd640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e900cec00 0x7f1e900c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e8effd640 1 -- 172.21.15.131:0/3766966553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e901494a0 con 0x7f1e9006e2c0 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e8effd640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e9006e2c0 0x7f1e90148f60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f1e8001aee0 tx=0x7f1e80004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e877fe640 1 -- 172.21.15.131:0/3766966553 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80004230 con 0x7f1e9006e2c0 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e877fe640 1 -- 172.21.15.131:0/3766966553 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e800044c0 con 0x7f1e9006e2c0 2024-09-17T02:50:06.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e95742640 1 -- 172.21.15.131:0/3766966553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e90158a00 con 0x7f1e9006e2c0 2024-09-17T02:50:06.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e877fe640 1 -- 172.21.15.131:0/3766966553 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80005270 con 0x7f1e9006e2c0 2024-09-17T02:50:06.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e95742640 1 -- 172.21.15.131:0/3766966553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e90158fc0 con 0x7f1e9006e2c0 2024-09-17T02:50:06.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.808+0000 7f1e95742640 1 -- 172.21.15.131:0/3766966553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e90159680 con 0x7f1e9006e2c0 2024-09-17T02:50:06.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.812+0000 7f1e877fe640 1 -- 172.21.15.131:0/3766966553 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1e800be020 con 0x7f1e9006e2c0 2024-09-17T02:50:06.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.812+0000 7f1e877fe640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1e70073200 0x7f1e70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:06.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.812+0000 7f1e877fe640 1 -- 172.21.15.131:0/3766966553 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f1e8012f380 con 0x7f1e9006e2c0 2024-09-17T02:50:06.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.812+0000 7f1e8e7fc640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1e70073200 0x7f1e70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:06.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.812+0000 7f1e8e7fc640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1e70073200 0x7f1e70075710 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1e9014a4c0 tx=0x7f1e78008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:06.812 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.816+0000 7f1e877fe640 1 -- 172.21.15.131:0/3766966553 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1e90159680 con 0x7f1e9006e2c0 2024-09-17T02:50:06.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.980+0000 7f1e95742640 1 -- 172.21.15.131:0/3766966553 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e900c4580 con 0x7f1e9006e2c0 2024-09-17T02:50:06.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e877fe640 1 -- 172.21.15.131:0/3766966553 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+17008 (secure 0 0 0) 0x7f1e800c5020 con 0x7f1e9006e2c0 2024-09-17T02:50:06.980 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:06.980 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":107,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:50:04.224176+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:50:00.250118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-17T02:50:01.555264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:50:06.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 -- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1e70073200 msgr2=0x7f1e70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1e70073200 0x7f1e70075710 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1e9014a4c0 tx=0x7f1e78008040 comp rx=0 tx=0).stop 2024-09-17T02:50:06.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 -- 172.21.15.131:0/3766966553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e9006e2c0 msgr2=0x7f1e90148f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:06.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e9006e2c0 0x7f1e90148f60 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f1e8001aee0 tx=0x7f1e80004040 comp rx=0 tx=0).stop 2024-09-17T02:50:06.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 -- 172.21.15.131:0/3766966553 shutdown_connections 2024-09-17T02:50:06.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1e70073200 0x7f1e70075710 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1e900cec00 0x7f1e900c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1e900c4ac0 0x7f1e900c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 --2- 172.21.15.131:0/3766966553 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1e9006e2c0 0x7f1e90148f60 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:06.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 -- 172.21.15.131:0/3766966553 >> 172.21.15.131:0/3766966553 conn(0x7f1e9006eea0 msgr2=0x7f1e900c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:06.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 -- 172.21.15.131:0/3766966553 shutdown_connections 2024-09-17T02:50:06.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:06.984+0000 7f1e95742640 1 -- 172.21.15.131:0/3766966553 wait complete. 2024-09-17T02:50:07.001 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T02:50:07.001 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:50:07.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.152+0000 7f0768535640 1 -- 172.21.15.131:0/2871087462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f074c005160 msgr2=0x7f074c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.152+0000 7f0768535640 1 --2- 172.21.15.131:0/2871087462 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f074c005160 0x7f074c00b9d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f07580016f0 tx=0x7f075801c7b0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.152+0000 7f0766ad2640 1 -- 172.21.15.131:0/2871087462 reap_dead start 2024-09-17T02:50:07.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.152+0000 7f0768535640 1 -- 172.21.15.131:0/2871087462 shutdown_connections 2024-09-17T02:50:07.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.152+0000 7f0768535640 1 -- 172.21.15.131:0/2871087462 >> 172.21.15.131:0/2871087462 conn(0x7f076006eea0 msgr2=0x7f076006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:07.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.152+0000 7f0768535640 1 -- 172.21.15.131:0/2871087462 shutdown_connections 2024-09-17T02:50:07.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.152+0000 7f0768535640 1 -- 172.21.15.131:0/2871087462 wait complete. 2024-09-17T02:50:07.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.152+0000 7f0768535640 1 Processor -- start 2024-09-17T02:50:07.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0768535640 1 -- start start 2024-09-17T02:50:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0768535640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f074c005160 0x7f0760148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0768535640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076015b060 0x7f07600b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0768535640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f07600b5760 0x7f07600b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0768535640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f076006e5b0 con 0x7f074c005160 2024-09-17T02:50:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0768535640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f07600cf320 con 0x7f076015b060 2024-09-17T02:50:07.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0768535640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f076006e730 con 0x7f07600b5760 2024-09-17T02:50:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f07662d1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f074c005160 0x7f0760148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f07662d1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f074c005160 0x7f0760148330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50726/0 (socket says 172.21.15.131:50726) 2024-09-17T02:50:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f07662d1640 1 -- 172.21.15.131:0/994224110 learned_addr learned my addr 172.21.15.131:0/994224110 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0766ad2640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f07600b5760 0x7f07600b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0765ad0640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076015b060 0x7f07600b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f07662d1640 1 -- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f07600b5760 msgr2=0x7f07600b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f07662d1640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f07600b5760 0x7f07600b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f07662d1640 1 -- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076015b060 msgr2=0x7f07600b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f07662d1640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076015b060 0x7f07600b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f07662d1640 1 -- 172.21.15.131:0/994224110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07600ceb30 con 0x7f074c005160 2024-09-17T02:50:07.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f07662d1640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f074c005160 0x7f0760148330 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f075000d7e0 tx=0x7f075000dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:07.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0756ffd640 1 -- 172.21.15.131:0/994224110 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750018070 con 0x7f074c005160 2024-09-17T02:50:07.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0756ffd640 1 -- 172.21.15.131:0/994224110 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07500052e0 con 0x7f074c005160 2024-09-17T02:50:07.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0768535640 1 -- 172.21.15.131:0/994224110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07601f10f0 con 0x7f074c005160 2024-09-17T02:50:07.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0756ffd640 1 -- 172.21.15.131:0/994224110 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075000b8c0 con 0x7f074c005160 2024-09-17T02:50:07.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0768535640 1 -- 172.21.15.131:0/994224110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07601f16b0 con 0x7f074c005160 2024-09-17T02:50:07.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0768535640 1 -- 172.21.15.131:0/994224110 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0760075b80 con 0x7f074c005160 2024-09-17T02:50:07.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.156+0000 7f0756ffd640 1 -- 172.21.15.131:0/994224110 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f075001b070 con 0x7f074c005160 2024-09-17T02:50:07.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.160+0000 7f0756ffd640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0744073200 0x7f0744075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.160+0000 7f0756ffd640 1 -- 172.21.15.131:0/994224110 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f0750093ae0 con 0x7f074c005160 2024-09-17T02:50:07.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.160+0000 7f0765ad0640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0744073200 0x7f0744075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.160+0000 7f0765ad0640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0744073200 0x7f0744075710 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f07600b56f0 tx=0x7f075c009500 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:07.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.164+0000 7f0756ffd640 1 -- 172.21.15.131:0/994224110 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f075005e3b0 con 0x7f074c005160 2024-09-17T02:50:07.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.332+0000 7f0768535640 1 -- 172.21.15.131:0/994224110 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f076015c140 con 0x7f0744073200 2024-09-17T02:50:07.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.344+0000 7f0756ffd640 1 -- 172.21.15.131:0/994224110 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241908 (secure 0 0 0) 0x7f076015c140 con 0x7f0744073200 2024-09-17T02:50:07.343 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:07.345 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:50:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 -- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0744073200 msgr2=0x7f0744075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0744073200 0x7f0744075710 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f07600b56f0 tx=0x7f075c009500 comp rx=0 tx=0).stop 2024-09-17T02:50:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 -- 172.21.15.131:0/994224110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f074c005160 msgr2=0x7f0760148330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f074c005160 0x7f0760148330 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f075000d7e0 tx=0x7f075000dca0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 -- 172.21.15.131:0/994224110 shutdown_connections 2024-09-17T02:50:07.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0744073200 0x7f0744075710 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f07600b5760 0x7f07600b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f076015b060 0x7f07600b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 --2- 172.21.15.131:0/994224110 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f074c005160 0x7f0760148330 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 -- 172.21.15.131:0/994224110 >> 172.21.15.131:0/994224110 conn(0x7f076006eea0 msgr2=0x7f07600b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:07.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 -- 172.21.15.131:0/994224110 shutdown_connections 2024-09-17T02:50:07.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.348+0000 7f0768535640 1 -- 172.21.15.131:0/994224110 wait complete. 2024-09-17T02:50:07.365 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-17T02:50:05.340721+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":84,"acting":84,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260672,"kb_used_data":11128,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714048,"statfs":{"total":773094113280,"available":772827185152,"internally_reserved":0,"allocated":11395072,"data_stored":5893695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.185500"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.237466+0000","last_change":"2024-09-17T02:50:03.241677+0000","last_active":"2024-09-17T02:50:04.237466+0000","last_peered":"2024-09-17T02:50:04.237466+0000","last_clean":"2024-09-17T02:50:04.237466+0000","last_became_active":"2024-09-17T02:50:03.239752+0000","last_became_peered":"2024-09-17T02:50:03.239752+0000","last_unstale":"2024-09-17T02:50:04.237466+0000","last_undegraded":"2024-09-17T02:50:04.237466+0000","last_fullsized":"2024-09-17T02:50:04.237466+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:44.064439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.240282+0000","last_change":"2024-09-17T02:50:03.228396+0000","last_active":"2024-09-17T02:50:04.240282+0000","last_peered":"2024-09-17T02:50:04.240282+0000","last_clean":"2024-09-17T02:50:04.240282+0000","last_became_active":"2024-09-17T02:50:03.227470+0000","last_became_peered":"2024-09-17T02:50:03.227470+0000","last_unstale":"2024-09-17T02:50:04.240282+0000","last_undegraded":"2024-09-17T02:50:04.240282+0000","last_fullsized":"2024-09-17T02:50:04.240282+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:42.470286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.239195+0000","last_change":"2024-09-17T02:50:03.238717+0000","last_active":"2024-09-17T02:50:04.239195+0000","last_peered":"2024-09-17T02:50:04.239195+0000","last_clean":"2024-09-17T02:50:04.239195+0000","last_became_active":"2024-09-17T02:50:03.233598+0000","last_became_peered":"2024-09-17T02:50:03.233598+0000","last_unstale":"2024-09-17T02:50:04.239195+0000","last_undegraded":"2024-09-17T02:50:04.239195+0000","last_fullsized":"2024-09-17T02:50:04.239195+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:35.342045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.237935+0000","last_change":"2024-09-17T02:50:03.223094+0000","last_active":"2024-09-17T02:50:04.237935+0000","last_peered":"2024-09-17T02:50:04.237935+0000","last_clean":"2024-09-17T02:50:04.237935+0000","last_became_active":"2024-09-17T02:50:03.222080+0000","last_became_peered":"2024-09-17T02:50:03.222080+0000","last_unstale":"2024-09-17T02:50:04.237935+0000","last_undegraded":"2024-09-17T02:50:04.237935+0000","last_fullsized":"2024-09-17T02:50:04.237935+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:43.479467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.239528+0000","last_change":"2024-09-17T02:50:03.238740+0000","last_active":"2024-09-17T02:50:04.239528+0000","last_peered":"2024-09-17T02:50:04.239528+0000","last_clean":"2024-09-17T02:50:04.239528+0000","last_became_active":"2024-09-17T02:50:03.237799+0000","last_became_peered":"2024-09-17T02:50:03.237799+0000","last_unstale":"2024-09-17T02:50:04.239528+0000","last_undegraded":"2024-09-17T02:50:04.239528+0000","last_fullsized":"2024-09-17T02:50:04.239528+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:51.653171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.167185+0000","last_change":"2024-09-17T02:50:01.167185+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.167185+0000","last_undegraded":"2024-09-17T02:50:01.167185+0000","last_fullsized":"2024-09-17T02:50:01.167185+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.241154+0000","last_change":"2024-09-17T02:50:03.234438+0000","last_active":"2024-09-17T02:50:04.241154+0000","last_peered":"2024-09-17T02:50:04.241154+0000","last_clean":"2024-09-17T02:50:04.241154+0000","last_became_active":"2024-09-17T02:50:03.232223+0000","last_became_peered":"2024-09-17T02:50:03.232223+0000","last_unstale":"2024-09-17T02:50:04.241154+0000","last_undegraded":"2024-09-17T02:50:04.241154+0000","last_fullsized":"2024-09-17T02:50:04.241154+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:51.809518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.170277+0000","last_change":"2024-09-17T02:50:01.170277+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.170277+0000","last_undegraded":"2024-09-17T02:50:01.170277+0000","last_fullsized":"2024-09-17T02:50:01.170277+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.161640+0000","last_change":"2024-09-17T02:50:01.161640+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.161640+0000","last_undegraded":"2024-09-17T02:50:01.161640+0000","last_fullsized":"2024-09-17T02:50:01.161640+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.241510+0000","last_change":"2024-09-17T02:50:02.171911+0000","last_active":"2024-09-17T02:50:04.241510+0000","last_peered":"2024-09-17T02:50:04.241510+0000","last_clean":"2024-09-17T02:50:04.241510+0000","last_became_active":"2024-09-17T02:50:02.170970+0000","last_became_peered":"2024-09-17T02:50:02.170970+0000","last_unstale":"2024-09-17T02:50:04.241510+0000","last_undegraded":"2024-09-17T02:50:04.241510+0000","last_fullsized":"2024-09-17T02:50:04.241510+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:58.409485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.164779+0000","last_change":"2024-09-17T02:50:01.164779+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.164779+0000","last_undegraded":"2024-09-17T02:50:01.164779+0000","last_fullsized":"2024-09-17T02:50:01.164779+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.166622+0000","last_change":"2024-09-17T02:50:01.166622+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.166622+0000","last_undegraded":"2024-09-17T02:50:01.166622+0000","last_fullsized":"2024-09-17T02:50:01.166622+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.167363+0000","last_change":"2024-09-17T02:50:01.167363+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.167363+0000","last_undegraded":"2024-09-17T02:50:01.167363+0000","last_fullsized":"2024-09-17T02:50:01.167363+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.165565+0000","last_change":"2024-09-17T02:50:01.165565+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.165565+0000","last_undegraded":"2024-09-17T02:50:01.165565+0000","last_fullsized":"2024-09-17T02:50:01.165565+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.169952+0000","last_change":"2024-09-17T02:50:01.169952+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.169952+0000","last_undegraded":"2024-09-17T02:50:01.169952+0000","last_fullsized":"2024-09-17T02:50:01.169952+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.240582+0000","last_change":"2024-09-17T02:50:02.202455+0000","last_active":"2024-09-17T02:50:04.240582+0000","last_peered":"2024-09-17T02:50:04.240582+0000","last_clean":"2024-09-17T02:50:04.240582+0000","last_became_active":"2024-09-17T02:50:02.201736+0000","last_became_peered":"2024-09-17T02:50:02.201736+0000","last_unstale":"2024-09-17T02:50:04.240582+0000","last_undegraded":"2024-09-17T02:50:04.240582+0000","last_fullsized":"2024-09-17T02:50:04.240582+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:49.449183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.239865+0000","last_change":"2024-09-17T02:50:03.259763+0000","last_active":"2024-09-17T02:50:04.239865+0000","last_peered":"2024-09-17T02:50:04.239865+0000","last_clean":"2024-09-17T02:50:04.239865+0000","last_became_active":"2024-09-17T02:50:03.259035+0000","last_became_peered":"2024-09-17T02:50:03.259035+0000","last_unstale":"2024-09-17T02:50:04.239865+0000","last_undegraded":"2024-09-17T02:50:04.239865+0000","last_fullsized":"2024-09-17T02:50:04.239865+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:05.559336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.168968+0000","last_change":"2024-09-17T02:50:01.168968+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.168968+0000","last_undegraded":"2024-09-17T02:50:01.168968+0000","last_fullsized":"2024-09-17T02:50:01.168968+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.163934+0000","last_change":"2024-09-17T02:50:01.163934+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.163934+0000","last_undegraded":"2024-09-17T02:50:01.163934+0000","last_fullsized":"2024-09-17T02:50:01.163934+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.161494+0000","last_change":"2024-09-17T02:50:01.161494+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.161494+0000","last_undegraded":"2024-09-17T02:50:01.161494+0000","last_fullsized":"2024-09-17T02:50:01.161494+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.241704+0000","last_change":"2024-09-17T02:50:02.195474+0000","last_active":"2024-09-17T02:50:04.241704+0000","last_peered":"2024-09-17T02:50:04.241704+0000","last_clean":"2024-09-17T02:50:04.241704+0000","last_became_active":"2024-09-17T02:50:02.194616+0000","last_became_peered":"2024-09-17T02:50:02.194616+0000","last_unstale":"2024-09-17T02:50:04.241704+0000","last_undegraded":"2024-09-17T02:50:04.241704+0000","last_fullsized":"2024-09-17T02:50:04.241704+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:30:52.770430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.161479+0000","last_change":"2024-09-17T02:50:01.161479+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.161479+0000","last_undegraded":"2024-09-17T02:50:01.161479+0000","last_fullsized":"2024-09-17T02:50:01.161479+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.163911+0000","last_change":"2024-09-17T02:50:01.163911+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.163911+0000","last_undegraded":"2024-09-17T02:50:01.163911+0000","last_fullsized":"2024-09-17T02:50:01.163911+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.240228+0000","last_change":"2024-09-17T02:50:03.236807+0000","last_active":"2024-09-17T02:50:04.240228+0000","last_peered":"2024-09-17T02:50:04.240228+0000","last_clean":"2024-09-17T02:50:04.240228+0000","last_became_active":"2024-09-17T02:50:03.235361+0000","last_became_peered":"2024-09-17T02:50:03.235361+0000","last_unstale":"2024-09-17T02:50:04.240228+0000","last_undegraded":"2024-09-17T02:50:04.240228+0000","last_fullsized":"2024-09-17T02:50:04.240228+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:41.560146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.168787+0000","last_change":"2024-09-17T02:50:01.168787+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.168787+0000","last_undegraded":"2024-09-17T02:50:01.168787+0000","last_fullsized":"2024-09-17T02:50:01.168787+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.166806+0000","last_change":"2024-09-17T02:50:01.166806+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.166806+0000","last_undegraded":"2024-09-17T02:50:01.166806+0000","last_fullsized":"2024-09-17T02:50:01.166806+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.169029+0000","last_change":"2024-09-17T02:50:01.169029+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.169029+0000","last_undegraded":"2024-09-17T02:50:01.169029+0000","last_fullsized":"2024-09-17T02:50:01.169029+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.168904+0000","last_change":"2024-09-17T02:50:01.168904+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.168904+0000","last_undegraded":"2024-09-17T02:50:01.168904+0000","last_fullsized":"2024-09-17T02:50:01.168904+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.170404+0000","last_change":"2024-09-17T02:50:01.170404+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.170404+0000","last_undegraded":"2024-09-17T02:50:01.170404+0000","last_fullsized":"2024-09-17T02:50:01.170404+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.168800+0000","last_change":"2024-09-17T02:50:01.168800+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.168800+0000","last_undegraded":"2024-09-17T02:50:01.168800+0000","last_fullsized":"2024-09-17T02:50:01.168800+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.238436+0000","last_change":"2024-09-17T02:50:02.205086+0000","last_active":"2024-09-17T02:50:04.238436+0000","last_peered":"2024-09-17T02:50:04.238436+0000","last_clean":"2024-09-17T02:50:04.238436+0000","last_became_active":"2024-09-17T02:50:02.202045+0000","last_became_peered":"2024-09-17T02:50:02.202045+0000","last_unstale":"2024-09-17T02:50:04.238436+0000","last_undegraded":"2024-09-17T02:50:04.238436+0000","last_fullsized":"2024-09-17T02:50:04.238436+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:28.182659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.170029+0000","last_change":"2024-09-17T02:50:01.170029+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.170029+0000","last_undegraded":"2024-09-17T02:50:01.170029+0000","last_fullsized":"2024-09-17T02:50:01.170029+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.169027+0000","last_change":"2024-09-17T02:50:01.169027+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.169027+0000","last_undegraded":"2024-09-17T02:50:01.169027+0000","last_fullsized":"2024-09-17T02:50:01.169027+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.239213+0000","last_change":"2024-09-17T02:50:03.248915+0000","last_active":"2024-09-17T02:50:04.239213+0000","last_peered":"2024-09-17T02:50:04.239213+0000","last_clean":"2024-09-17T02:50:04.239213+0000","last_became_active":"2024-09-17T02:50:03.248029+0000","last_became_peered":"2024-09-17T02:50:03.248029+0000","last_unstale":"2024-09-17T02:50:04.239213+0000","last_undegraded":"2024-09-17T02:50:04.239213+0000","last_fullsized":"2024-09-17T02:50:04.239213+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:03.354582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.163948+0000","last_change":"2024-09-17T02:50:01.163948+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.163948+0000","last_undegraded":"2024-09-17T02:50:01.163948+0000","last_fullsized":"2024-09-17T02:50:01.163948+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.164113+0000","last_change":"2024-09-17T02:50:01.164113+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.164113+0000","last_undegraded":"2024-09-17T02:50:01.164113+0000","last_fullsized":"2024-09-17T02:50:01.164113+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.163949+0000","last_change":"2024-09-17T02:50:01.163949+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.163949+0000","last_undegraded":"2024-09-17T02:50:01.163949+0000","last_fullsized":"2024-09-17T02:50:01.163949+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.169115+0000","last_change":"2024-09-17T02:50:01.169115+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.169115+0000","last_undegraded":"2024-09-17T02:50:01.169115+0000","last_fullsized":"2024-09-17T02:50:01.169115+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.166543+0000","last_change":"2024-09-17T02:50:01.166543+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.166543+0000","last_undegraded":"2024-09-17T02:50:01.166543+0000","last_fullsized":"2024-09-17T02:50:01.166543+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.170418+0000","last_change":"2024-09-17T02:50:01.170418+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.170418+0000","last_undegraded":"2024-09-17T02:50:01.170418+0000","last_fullsized":"2024-09-17T02:50:01.170418+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T02:50:02.154424+0000","last_change":"2024-09-17T02:50:02.154424+0000","last_active":"2024-09-17T02:50:02.154424+0000","last_peered":"2024-09-17T02:50:02.154424+0000","last_clean":"2024-09-17T02:50:02.154424+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:02.154424+0000","last_undegraded":"2024-09-17T02:50:02.154424+0000","last_fullsized":"2024-09-17T02:50:02.154424+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":104,"state":"creating+peering","last_fresh":"2024-09-17T02:50:01.162954+0000","last_change":"2024-09-17T02:50:01.162954+0000","last_active":"2024-09-17T02:50:01.150594+0000","last_peered":"2024-09-17T02:50:01.150594+0000","last_clean":"2024-09-17T02:50:01.150594+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T02:50:01.162954+0000","last_undegraded":"2024-09-17T02:50:01.162954+0000","last_fullsized":"2024-09-17T02:50:01.162954+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":269,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-17T02:50:01.168250+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:50:01.168250+0000","last_peered":"2024-09-17T02:50:01.168250+0000","last_clean":"2024-09-17T02:50:01.168250+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:50:01.168250+0000","last_undegraded":"2024-09-17T02:50:01.168250+0000","last_fullsized":"2024-09-17T02:50:01.168250+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":1},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1269760,"data_stored":586413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509556,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1863680,"data_stored":1176781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1269760,"data_stored":586413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":586413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1269760,"data_stored":586413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509556,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33012,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338828,"statfs":{"total":96636764160,"available":96602959872,"internally_reserved":0,"allocated":1863680,"data_stored":1176781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1269760,"data_stored":586413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1318912,"data_stored":608068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:50:07.370 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:50:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.564+0000 7f65dadc5640 1 -- 172.21.15.131:0/78673901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d4070df0 msgr2=0x7f65d406ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.564+0000 7f65dadc5640 1 --2- 172.21.15.131:0/78673901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d4070df0 0x7f65d406ffb0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f65d4070d50 tx=0x7f65c401ccc0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.564+0000 7f65dadc5640 1 -- 172.21.15.131:0/78673901 shutdown_connections 2024-09-17T02:50:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.564+0000 7f65dadc5640 1 --2- 172.21.15.131:0/78673901 >> v2:172.21.15.189:3300/0 conn(0x7f65d40cc940 0x7f65d4160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.564+0000 7f65dadc5640 1 --2- 172.21.15.131:0/78673901 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d4070df0 0x7f65d406ffb0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.564+0000 7f65dadc5640 1 -- 172.21.15.131:0/78673901 >> 172.21.15.131:0/78673901 conn(0x7f65d40cdbb0 msgr2=0x7f65d40cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65dadc5640 1 -- 172.21.15.131:0/78673901 shutdown_connections 2024-09-17T02:50:07.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65dadc5640 1 -- 172.21.15.131:0/78673901 wait complete. 2024-09-17T02:50:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65dadc5640 1 Processor -- start 2024-09-17T02:50:07.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65dadc5640 1 -- start start 2024-09-17T02:50:07.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65dadc5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d40704f0 0x7f65d41507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65dadc5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f65d4070df0 0x7f65d40cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65dadc5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65d40cc940 0x7f65d40cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65dadc5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4160f10 con 0x7f65d40704f0 2024-09-17T02:50:07.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65dadc5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f65d4160d90 con 0x7f65d40cc940 2024-09-17T02:50:07.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65dadc5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f65d4169ae0 con 0x7f65d4070df0 2024-09-17T02:50:07.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65d8b61640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d40704f0 0x7f65d41507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65d8b61640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d40704f0 0x7f65d41507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50736/0 (socket says 172.21.15.131:50736) 2024-09-17T02:50:07.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65d8b61640 1 -- 172.21.15.131:0/1101653202 learned_addr learned my addr 172.21.15.131:0/1101653202 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65d9362640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65d40cc940 0x7f65d40cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65d3fff640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f65d4070df0 0x7f65d40cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65d8b61640 1 -- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f65d4070df0 msgr2=0x7f65d40cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65d8b61640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f65d4070df0 0x7f65d40cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65d8b61640 1 -- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65d40cc940 msgr2=0x7f65d40cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65d8b61640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65d40cc940 0x7f65d40cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65d8b61640 1 -- 172.21.15.131:0/1101653202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d40d08b0 con 0x7f65d40704f0 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.568+0000 7f65d8b61640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d40704f0 0x7f65d41507b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f65c401a8f0 tx=0x7f65c4005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.572+0000 7f65d17fa640 1 -- 172.21.15.131:0/1101653202 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c40b2040 con 0x7f65d40704f0 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.572+0000 7f65d17fa640 1 -- 172.21.15.131:0/1101653202 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65c4004040 con 0x7f65d40704f0 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.572+0000 7f65dadc5640 1 -- 172.21.15.131:0/1101653202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d40d0b90 con 0x7f65d40704f0 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.572+0000 7f65d17fa640 1 -- 172.21.15.131:0/1101653202 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4004430 con 0x7f65d40704f0 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.572+0000 7f65dadc5640 1 -- 172.21.15.131:0/1101653202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d4157a80 con 0x7f65d40704f0 2024-09-17T02:50:07.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.572+0000 7f65dadc5640 1 -- 172.21.15.131:0/1101653202 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65d4157d40 con 0x7f65d40704f0 2024-09-17T02:50:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.572+0000 7f65d17fa640 1 -- 172.21.15.131:0/1101653202 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f65c40bd030 con 0x7f65d40704f0 2024-09-17T02:50:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.572+0000 7f65d17fa640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f65b4073200 0x7f65b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.572+0000 7f65d17fa640 1 -- 172.21.15.131:0/1101653202 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f65c412f2e0 con 0x7f65d40704f0 2024-09-17T02:50:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.572+0000 7f65d3fff640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f65b4073200 0x7f65b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.572+0000 7f65d3fff640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f65b4073200 0x7f65b4075710 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f65d40d04f0 tx=0x7f65bc009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:07.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.576+0000 7f65d17fa640 1 -- 172.21.15.131:0/1101653202 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f65c40c4020 con 0x7f65d40704f0 2024-09-17T02:50:07.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.736+0000 7f65dadc5640 1 -- 172.21.15.131:0/1101653202 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65d4157270 con 0x7f65b4073200 2024-09-17T02:50:07.746 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.748+0000 7f65d17fa640 1 -- 172.21.15.131:0/1101653202 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252485 (secure 0 0 0) 0x7f65d4157270 con 0x7f65b4073200 2024-09-17T02:50:07.749 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:07.750 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:50:07.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.752+0000 7f65dadc5640 1 -- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f65b4073200 msgr2=0x7f65b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.752+0000 7f65dadc5640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f65b4073200 0x7f65b4075710 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f65d40d04f0 tx=0x7f65bc009040 comp rx=0 tx=0).stop 2024-09-17T02:50:07.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.756+0000 7f65dadc5640 1 -- 172.21.15.131:0/1101653202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d40704f0 msgr2=0x7f65d41507b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.756+0000 7f65dadc5640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d40704f0 0x7f65d41507b0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f65c401a8f0 tx=0x7f65c4005ae0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.756+0000 7f65dadc5640 1 -- 172.21.15.131:0/1101653202 shutdown_connections 2024-09-17T02:50:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.756+0000 7f65dadc5640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f65b4073200 0x7f65b4075710 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.756+0000 7f65dadc5640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f65d40cc940 0x7f65d40cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.756+0000 7f65dadc5640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f65d4070df0 0x7f65d40cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.756+0000 7f65dadc5640 1 --2- 172.21.15.131:0/1101653202 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65d40704f0 0x7f65d41507b0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.756+0000 7f65dadc5640 1 -- 172.21.15.131:0/1101653202 >> 172.21.15.131:0/1101653202 conn(0x7f65d40cdbb0 msgr2=0x7f65d40b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.756+0000 7f65dadc5640 1 -- 172.21.15.131:0/1101653202 shutdown_connections 2024-09-17T02:50:07.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.756+0000 7f65dadc5640 1 -- 172.21.15.131:0/1101653202 wait complete. 2024-09-17T02:50:07.769 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-17T02:50:07.346333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261072,"kb_used_data":11516,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713648,"statfs":{"total":773094113280,"available":772826775552,"internally_reserved":0,"allocated":11792384,"data_stored":6084048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.007078"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.242624+0000","last_change":"2024-09-17T02:50:03.248975+0000","last_active":"2024-09-17T02:50:04.242624+0000","last_peered":"2024-09-17T02:50:04.242624+0000","last_clean":"2024-09-17T02:50:04.242624+0000","last_became_active":"2024-09-17T02:50:03.239040+0000","last_became_peered":"2024-09-17T02:50:03.239040+0000","last_unstale":"2024-09-17T02:50:04.242624+0000","last_undegraded":"2024-09-17T02:50:04.242624+0000","last_fullsized":"2024-09-17T02:50:04.242624+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:13:00.433665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.236316+0000","last_change":"2024-09-17T02:50:03.243444+0000","last_active":"2024-09-17T02:50:04.236316+0000","last_peered":"2024-09-17T02:50:04.236316+0000","last_clean":"2024-09-17T02:50:04.236316+0000","last_became_active":"2024-09-17T02:50:03.243091+0000","last_became_peered":"2024-09-17T02:50:03.243091+0000","last_unstale":"2024-09-17T02:50:04.236316+0000","last_undegraded":"2024-09-17T02:50:04.236316+0000","last_fullsized":"2024-09-17T02:50:04.236316+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:14.347391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.247337+0000","last_change":"2024-09-17T02:50:03.251345+0000","last_active":"2024-09-17T02:50:04.247337+0000","last_peered":"2024-09-17T02:50:04.247337+0000","last_clean":"2024-09-17T02:50:04.247337+0000","last_became_active":"2024-09-17T02:50:03.250568+0000","last_became_peered":"2024-09-17T02:50:03.250568+0000","last_unstale":"2024-09-17T02:50:04.247337+0000","last_undegraded":"2024-09-17T02:50:04.247337+0000","last_fullsized":"2024-09-17T02:50:04.247337+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:01.334916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.240943+0000","last_change":"2024-09-17T02:50:03.227316+0000","last_active":"2024-09-17T02:50:04.240943+0000","last_peered":"2024-09-17T02:50:04.240943+0000","last_clean":"2024-09-17T02:50:04.240943+0000","last_became_active":"2024-09-17T02:50:03.226381+0000","last_became_peered":"2024-09-17T02:50:03.226381+0000","last_unstale":"2024-09-17T02:50:04.240943+0000","last_undegraded":"2024-09-17T02:50:04.240943+0000","last_fullsized":"2024-09-17T02:50:04.240943+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:21:17.937378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.239730+0000","last_change":"2024-09-17T02:50:03.235868+0000","last_active":"2024-09-17T02:50:04.239730+0000","last_peered":"2024-09-17T02:50:04.239730+0000","last_clean":"2024-09-17T02:50:04.239730+0000","last_became_active":"2024-09-17T02:50:03.234959+0000","last_became_peered":"2024-09-17T02:50:03.234959+0000","last_unstale":"2024-09-17T02:50:04.239730+0000","last_undegraded":"2024-09-17T02:50:04.239730+0000","last_fullsized":"2024-09-17T02:50:04.239730+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:22.947229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.252531+0000","last_change":"2024-09-17T02:50:03.257179+0000","last_active":"2024-09-17T02:50:04.252531+0000","last_peered":"2024-09-17T02:50:04.252531+0000","last_clean":"2024-09-17T02:50:04.252531+0000","last_became_active":"2024-09-17T02:50:03.256848+0000","last_became_peered":"2024-09-17T02:50:03.256848+0000","last_unstale":"2024-09-17T02:50:04.252531+0000","last_undegraded":"2024-09-17T02:50:04.252531+0000","last_fullsized":"2024-09-17T02:50:04.252531+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:23:01.334916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.252783+0000","last_change":"2024-09-17T02:50:03.275790+0000","last_active":"2024-09-17T02:50:04.252783+0000","last_peered":"2024-09-17T02:50:04.252783+0000","last_clean":"2024-09-17T02:50:04.252783+0000","last_became_active":"2024-09-17T02:50:03.275470+0000","last_became_peered":"2024-09-17T02:50:03.275470+0000","last_unstale":"2024-09-17T02:50:04.252783+0000","last_undegraded":"2024-09-17T02:50:04.252783+0000","last_fullsized":"2024-09-17T02:50:04.252783+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:20.785187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.237466+0000","last_change":"2024-09-17T02:50:03.241677+0000","last_active":"2024-09-17T02:50:04.237466+0000","last_peered":"2024-09-17T02:50:04.237466+0000","last_clean":"2024-09-17T02:50:04.237466+0000","last_became_active":"2024-09-17T02:50:03.239752+0000","last_became_peered":"2024-09-17T02:50:03.239752+0000","last_unstale":"2024-09-17T02:50:04.237466+0000","last_undegraded":"2024-09-17T02:50:04.237466+0000","last_fullsized":"2024-09-17T02:50:04.237466+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:37:44.064439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.235354+0000","last_change":"2024-09-17T02:50:03.237512+0000","last_active":"2024-09-17T02:50:04.235354+0000","last_peered":"2024-09-17T02:50:04.235354+0000","last_clean":"2024-09-17T02:50:04.235354+0000","last_became_active":"2024-09-17T02:50:03.237055+0000","last_became_peered":"2024-09-17T02:50:03.237055+0000","last_unstale":"2024-09-17T02:50:04.235354+0000","last_undegraded":"2024-09-17T02:50:04.235354+0000","last_fullsized":"2024-09-17T02:50:04.235354+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:15.839044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.240282+0000","last_change":"2024-09-17T02:50:03.228396+0000","last_active":"2024-09-17T02:50:04.240282+0000","last_peered":"2024-09-17T02:50:04.240282+0000","last_clean":"2024-09-17T02:50:04.240282+0000","last_became_active":"2024-09-17T02:50:03.227470+0000","last_became_peered":"2024-09-17T02:50:03.227470+0000","last_unstale":"2024-09-17T02:50:04.240282+0000","last_undegraded":"2024-09-17T02:50:04.240282+0000","last_fullsized":"2024-09-17T02:50:04.240282+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:06:42.470286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.976679+0000","last_change":"2024-09-17T02:50:03.228331+0000","last_active":"2024-09-17T02:50:04.976679+0000","last_peered":"2024-09-17T02:50:04.976679+0000","last_clean":"2024-09-17T02:50:04.976679+0000","last_became_active":"2024-09-17T02:50:03.227799+0000","last_became_peered":"2024-09-17T02:50:03.227799+0000","last_unstale":"2024-09-17T02:50:04.976679+0000","last_undegraded":"2024-09-17T02:50:04.976679+0000","last_fullsized":"2024-09-17T02:50:04.976679+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:50.666141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.253653+0000","last_change":"2024-09-17T02:50:03.272446+0000","last_active":"2024-09-17T02:50:04.253653+0000","last_peered":"2024-09-17T02:50:04.253653+0000","last_clean":"2024-09-17T02:50:04.253653+0000","last_became_active":"2024-09-17T02:50:03.272017+0000","last_became_peered":"2024-09-17T02:50:03.272017+0000","last_unstale":"2024-09-17T02:50:04.253653+0000","last_undegraded":"2024-09-17T02:50:04.253653+0000","last_fullsized":"2024-09-17T02:50:04.253653+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:19:55.014546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.238927+0000","last_change":"2024-09-17T02:50:03.249980+0000","last_active":"2024-09-17T02:50:04.238927+0000","last_peered":"2024-09-17T02:50:04.238927+0000","last_clean":"2024-09-17T02:50:04.238927+0000","last_became_active":"2024-09-17T02:50:03.249101+0000","last_became_peered":"2024-09-17T02:50:03.249101+0000","last_unstale":"2024-09-17T02:50:04.238927+0000","last_undegraded":"2024-09-17T02:50:04.238927+0000","last_fullsized":"2024-09-17T02:50:04.238927+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:36.361645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.253424+0000","last_change":"2024-09-17T02:50:03.256844+0000","last_active":"2024-09-17T02:50:04.253424+0000","last_peered":"2024-09-17T02:50:04.253424+0000","last_clean":"2024-09-17T02:50:04.253424+0000","last_became_active":"2024-09-17T02:50:03.256510+0000","last_became_peered":"2024-09-17T02:50:03.256510+0000","last_unstale":"2024-09-17T02:50:04.253424+0000","last_undegraded":"2024-09-17T02:50:04.253424+0000","last_fullsized":"2024-09-17T02:50:04.253424+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:45:15.979297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.235605+0000","last_change":"2024-09-17T02:50:03.233237+0000","last_active":"2024-09-17T02:50:04.235605+0000","last_peered":"2024-09-17T02:50:04.235605+0000","last_clean":"2024-09-17T02:50:04.235605+0000","last_became_active":"2024-09-17T02:50:03.232887+0000","last_became_peered":"2024-09-17T02:50:03.232887+0000","last_unstale":"2024-09-17T02:50:04.235605+0000","last_undegraded":"2024-09-17T02:50:04.235605+0000","last_fullsized":"2024-09-17T02:50:04.235605+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:19.260875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.978631+0000","last_change":"2024-09-17T02:50:03.258504+0000","last_active":"2024-09-17T02:50:04.978631+0000","last_peered":"2024-09-17T02:50:04.978631+0000","last_clean":"2024-09-17T02:50:04.978631+0000","last_became_active":"2024-09-17T02:50:03.258152+0000","last_became_peered":"2024-09-17T02:50:03.258152+0000","last_unstale":"2024-09-17T02:50:04.978631+0000","last_undegraded":"2024-09-17T02:50:04.978631+0000","last_fullsized":"2024-09-17T02:50:04.978631+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:15.981629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.247394+0000","last_change":"2024-09-17T02:50:03.251623+0000","last_active":"2024-09-17T02:50:04.247394+0000","last_peered":"2024-09-17T02:50:04.247394+0000","last_clean":"2024-09-17T02:50:04.247394+0000","last_became_active":"2024-09-17T02:50:03.246215+0000","last_became_peered":"2024-09-17T02:50:03.246215+0000","last_unstale":"2024-09-17T02:50:04.247394+0000","last_undegraded":"2024-09-17T02:50:04.247394+0000","last_fullsized":"2024-09-17T02:50:04.247394+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:20.785187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.235932+0000","last_change":"2024-09-17T02:50:03.236566+0000","last_active":"2024-09-17T02:50:04.235932+0000","last_peered":"2024-09-17T02:50:04.235932+0000","last_clean":"2024-09-17T02:50:04.235932+0000","last_became_active":"2024-09-17T02:50:03.236160+0000","last_became_peered":"2024-09-17T02:50:03.236160+0000","last_unstale":"2024-09-17T02:50:04.235932+0000","last_undegraded":"2024-09-17T02:50:04.235932+0000","last_fullsized":"2024-09-17T02:50:04.235932+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:47.565564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.241214+0000","last_change":"2024-09-17T02:50:03.251882+0000","last_active":"2024-09-17T02:50:04.241214+0000","last_peered":"2024-09-17T02:50:04.241214+0000","last_clean":"2024-09-17T02:50:04.241214+0000","last_became_active":"2024-09-17T02:50:03.251102+0000","last_became_peered":"2024-09-17T02:50:03.251102+0000","last_unstale":"2024-09-17T02:50:04.241214+0000","last_undegraded":"2024-09-17T02:50:04.241214+0000","last_fullsized":"2024-09-17T02:50:04.241214+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:29:19.450132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.978467+0000","last_change":"2024-09-17T02:50:03.233685+0000","last_active":"2024-09-17T02:50:04.978467+0000","last_peered":"2024-09-17T02:50:04.978467+0000","last_clean":"2024-09-17T02:50:04.978467+0000","last_became_active":"2024-09-17T02:50:03.233014+0000","last_became_peered":"2024-09-17T02:50:03.233014+0000","last_unstale":"2024-09-17T02:50:04.978467+0000","last_undegraded":"2024-09-17T02:50:04.978467+0000","last_fullsized":"2024-09-17T02:50:04.978467+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:51.748152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.248646+0000","last_change":"2024-09-17T02:50:03.244442+0000","last_active":"2024-09-17T02:50:04.248646+0000","last_peered":"2024-09-17T02:50:04.248646+0000","last_clean":"2024-09-17T02:50:04.248646+0000","last_became_active":"2024-09-17T02:50:03.243677+0000","last_became_peered":"2024-09-17T02:50:03.243677+0000","last_unstale":"2024-09-17T02:50:04.248646+0000","last_undegraded":"2024-09-17T02:50:04.248646+0000","last_fullsized":"2024-09-17T02:50:04.248646+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:43:42.297393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.975405+0000","last_change":"2024-09-17T02:50:03.271543+0000","last_active":"2024-09-17T02:50:04.975405+0000","last_peered":"2024-09-17T02:50:04.975405+0000","last_clean":"2024-09-17T02:50:04.975405+0000","last_became_active":"2024-09-17T02:50:03.270860+0000","last_became_peered":"2024-09-17T02:50:03.270860+0000","last_unstale":"2024-09-17T02:50:04.975405+0000","last_undegraded":"2024-09-17T02:50:04.975405+0000","last_fullsized":"2024-09-17T02:50:04.975405+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:22:15.796505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.239195+0000","last_change":"2024-09-17T02:50:03.238717+0000","last_active":"2024-09-17T02:50:04.239195+0000","last_peered":"2024-09-17T02:50:04.239195+0000","last_clean":"2024-09-17T02:50:04.239195+0000","last_became_active":"2024-09-17T02:50:03.233598+0000","last_became_peered":"2024-09-17T02:50:03.233598+0000","last_unstale":"2024-09-17T02:50:04.239195+0000","last_undegraded":"2024-09-17T02:50:04.239195+0000","last_fullsized":"2024-09-17T02:50:04.239195+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:35.342045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.237935+0000","last_change":"2024-09-17T02:50:03.223094+0000","last_active":"2024-09-17T02:50:04.237935+0000","last_peered":"2024-09-17T02:50:04.237935+0000","last_clean":"2024-09-17T02:50:04.237935+0000","last_became_active":"2024-09-17T02:50:03.222080+0000","last_became_peered":"2024-09-17T02:50:03.222080+0000","last_unstale":"2024-09-17T02:50:04.237935+0000","last_undegraded":"2024-09-17T02:50:04.237935+0000","last_fullsized":"2024-09-17T02:50:04.237935+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:43.479467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.978742+0000","last_change":"2024-09-17T02:50:03.231512+0000","last_active":"2024-09-17T02:50:04.978742+0000","last_peered":"2024-09-17T02:50:04.978742+0000","last_clean":"2024-09-17T02:50:04.978742+0000","last_became_active":"2024-09-17T02:50:03.231108+0000","last_became_peered":"2024-09-17T02:50:03.231108+0000","last_unstale":"2024-09-17T02:50:04.978742+0000","last_undegraded":"2024-09-17T02:50:04.978742+0000","last_fullsized":"2024-09-17T02:50:04.978742+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:13.573037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.978723+0000","last_change":"2024-09-17T02:50:03.230779+0000","last_active":"2024-09-17T02:50:04.978723+0000","last_peered":"2024-09-17T02:50:04.978723+0000","last_clean":"2024-09-17T02:50:04.978723+0000","last_became_active":"2024-09-17T02:50:03.230360+0000","last_became_peered":"2024-09-17T02:50:03.230360+0000","last_unstale":"2024-09-17T02:50:04.978723+0000","last_undegraded":"2024-09-17T02:50:04.978723+0000","last_fullsized":"2024-09-17T02:50:04.978723+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:20:00.043136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.978290+0000","last_change":"2024-09-17T02:50:03.230551+0000","last_active":"2024-09-17T02:50:04.978290+0000","last_peered":"2024-09-17T02:50:04.978290+0000","last_clean":"2024-09-17T02:50:04.978290+0000","last_became_active":"2024-09-17T02:50:03.230115+0000","last_became_peered":"2024-09-17T02:50:03.230115+0000","last_unstale":"2024-09-17T02:50:04.978290+0000","last_undegraded":"2024-09-17T02:50:04.978290+0000","last_fullsized":"2024-09-17T02:50:04.978290+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:30:57.953011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.253931+0000","last_change":"2024-09-17T02:50:03.248434+0000","last_active":"2024-09-17T02:50:04.253931+0000","last_peered":"2024-09-17T02:50:04.253931+0000","last_clean":"2024-09-17T02:50:04.253931+0000","last_became_active":"2024-09-17T02:50:03.247793+0000","last_became_peered":"2024-09-17T02:50:03.247793+0000","last_unstale":"2024-09-17T02:50:04.253931+0000","last_undegraded":"2024-09-17T02:50:04.253931+0000","last_fullsized":"2024-09-17T02:50:04.253931+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:24.641898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.239528+0000","last_change":"2024-09-17T02:50:03.238740+0000","last_active":"2024-09-17T02:50:04.239528+0000","last_peered":"2024-09-17T02:50:04.239528+0000","last_clean":"2024-09-17T02:50:04.239528+0000","last_became_active":"2024-09-17T02:50:03.237799+0000","last_became_peered":"2024-09-17T02:50:03.237799+0000","last_unstale":"2024-09-17T02:50:04.239528+0000","last_undegraded":"2024-09-17T02:50:04.239528+0000","last_fullsized":"2024-09-17T02:50:04.239528+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:51.653171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.976533+0000","last_change":"2024-09-17T02:50:03.229159+0000","last_active":"2024-09-17T02:50:04.976533+0000","last_peered":"2024-09-17T02:50:04.976533+0000","last_clean":"2024-09-17T02:50:04.976533+0000","last_became_active":"2024-09-17T02:50:03.228274+0000","last_became_peered":"2024-09-17T02:50:03.228274+0000","last_unstale":"2024-09-17T02:50:04.976533+0000","last_undegraded":"2024-09-17T02:50:04.976533+0000","last_fullsized":"2024-09-17T02:50:04.976533+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:34.175627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.252946+0000","last_change":"2024-09-17T02:50:03.256403+0000","last_active":"2024-09-17T02:50:04.252946+0000","last_peered":"2024-09-17T02:50:04.252946+0000","last_clean":"2024-09-17T02:50:04.252946+0000","last_became_active":"2024-09-17T02:50:03.256017+0000","last_became_peered":"2024-09-17T02:50:03.256017+0000","last_unstale":"2024-09-17T02:50:04.252946+0000","last_undegraded":"2024-09-17T02:50:04.252946+0000","last_fullsized":"2024-09-17T02:50:04.252946+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:10.545949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.974352+0000","last_change":"2024-09-17T02:50:03.244860+0000","last_active":"2024-09-17T02:50:04.974352+0000","last_peered":"2024-09-17T02:50:04.974352+0000","last_clean":"2024-09-17T02:50:04.974352+0000","last_became_active":"2024-09-17T02:50:03.243967+0000","last_became_peered":"2024-09-17T02:50:03.243967+0000","last_unstale":"2024-09-17T02:50:04.974352+0000","last_undegraded":"2024-09-17T02:50:04.974352+0000","last_fullsized":"2024-09-17T02:50:04.974352+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:57.478746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.246394+0000","last_change":"2024-09-17T02:50:03.245414+0000","last_active":"2024-09-17T02:50:04.246394+0000","last_peered":"2024-09-17T02:50:04.246394+0000","last_clean":"2024-09-17T02:50:04.246394+0000","last_became_active":"2024-09-17T02:50:03.244627+0000","last_became_peered":"2024-09-17T02:50:03.244627+0000","last_unstale":"2024-09-17T02:50:04.246394+0000","last_undegraded":"2024-09-17T02:50:04.246394+0000","last_fullsized":"2024-09-17T02:50:04.246394+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:59:18.783696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1f","version":"106'4","reported_seq":19,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.975230+0000","last_change":"2024-09-17T02:50:02.202901+0000","last_active":"2024-09-17T02:50:04.975230+0000","last_peered":"2024-09-17T02:50:04.975230+0000","last_clean":"2024-09-17T02:50:04.975230+0000","last_became_active":"2024-09-17T02:50:02.202088+0000","last_became_peered":"2024-09-17T02:50:02.202088+0000","last_unstale":"2024-09-17T02:50:04.975230+0000","last_undegraded":"2024-09-17T02:50:04.975230+0000","last_fullsized":"2024-09-17T02:50:04.975230+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:11:11.409939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.241154+0000","last_change":"2024-09-17T02:50:03.234438+0000","last_active":"2024-09-17T02:50:04.241154+0000","last_peered":"2024-09-17T02:50:04.241154+0000","last_clean":"2024-09-17T02:50:04.241154+0000","last_became_active":"2024-09-17T02:50:03.232223+0000","last_became_peered":"2024-09-17T02:50:03.232223+0000","last_unstale":"2024-09-17T02:50:04.241154+0000","last_undegraded":"2024-09-17T02:50:04.241154+0000","last_fullsized":"2024-09-17T02:50:04.241154+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:51.809518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.246868+0000","last_change":"2024-09-17T02:50:02.196819+0000","last_active":"2024-09-17T02:50:04.246868+0000","last_peered":"2024-09-17T02:50:04.246868+0000","last_clean":"2024-09-17T02:50:04.246868+0000","last_became_active":"2024-09-17T02:50:02.195731+0000","last_became_peered":"2024-09-17T02:50:02.195731+0000","last_unstale":"2024-09-17T02:50:04.246868+0000","last_undegraded":"2024-09-17T02:50:04.246868+0000","last_fullsized":"2024-09-17T02:50:04.246868+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:16.484535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.235551+0000","last_change":"2024-09-17T02:50:03.226026+0000","last_active":"2024-09-17T02:50:04.235551+0000","last_peered":"2024-09-17T02:50:04.235551+0000","last_clean":"2024-09-17T02:50:04.235551+0000","last_became_active":"2024-09-17T02:50:03.225115+0000","last_became_peered":"2024-09-17T02:50:03.225115+0000","last_unstale":"2024-09-17T02:50:04.235551+0000","last_undegraded":"2024-09-17T02:50:04.235551+0000","last_fullsized":"2024-09-17T02:50:04.235551+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:01:52.404508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.235647+0000","last_change":"2024-09-17T02:50:02.179004+0000","last_active":"2024-09-17T02:50:04.235647+0000","last_peered":"2024-09-17T02:50:04.235647+0000","last_clean":"2024-09-17T02:50:04.235647+0000","last_became_active":"2024-09-17T02:50:02.178630+0000","last_became_peered":"2024-09-17T02:50:02.178630+0000","last_unstale":"2024-09-17T02:50:04.235647+0000","last_undegraded":"2024-09-17T02:50:04.235647+0000","last_fullsized":"2024-09-17T02:50:04.235647+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:32.439369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.975866+0000","last_change":"2024-09-17T02:50:03.239135+0000","last_active":"2024-09-17T02:50:04.975866+0000","last_peered":"2024-09-17T02:50:04.975866+0000","last_clean":"2024-09-17T02:50:04.975866+0000","last_became_active":"2024-09-17T02:50:03.238685+0000","last_became_peered":"2024-09-17T02:50:03.238685+0000","last_unstale":"2024-09-17T02:50:04.975866+0000","last_undegraded":"2024-09-17T02:50:04.975866+0000","last_fullsized":"2024-09-17T02:50:04.975866+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:57:38.993746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.241510+0000","last_change":"2024-09-17T02:50:02.171911+0000","last_active":"2024-09-17T02:50:04.241510+0000","last_peered":"2024-09-17T02:50:04.241510+0000","last_clean":"2024-09-17T02:50:04.241510+0000","last_became_active":"2024-09-17T02:50:02.170970+0000","last_became_peered":"2024-09-17T02:50:02.170970+0000","last_unstale":"2024-09-17T02:50:04.241510+0000","last_undegraded":"2024-09-17T02:50:04.241510+0000","last_fullsized":"2024-09-17T02:50:04.241510+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:11:58.409485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.241560+0000","last_change":"2024-09-17T02:50:03.258751+0000","last_active":"2024-09-17T02:50:04.241560+0000","last_peered":"2024-09-17T02:50:04.241560+0000","last_clean":"2024-09-17T02:50:04.241560+0000","last_became_active":"2024-09-17T02:50:03.258044+0000","last_became_peered":"2024-09-17T02:50:03.258044+0000","last_unstale":"2024-09-17T02:50:04.241560+0000","last_undegraded":"2024-09-17T02:50:04.241560+0000","last_fullsized":"2024-09-17T02:50:04.241560+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:58:52.005742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1b","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.975636+0000","last_change":"2024-09-17T02:50:02.188174+0000","last_active":"2024-09-17T02:50:04.975636+0000","last_peered":"2024-09-17T02:50:04.975636+0000","last_clean":"2024-09-17T02:50:04.975636+0000","last_became_active":"2024-09-17T02:50:02.187713+0000","last_became_peered":"2024-09-17T02:50:02.187713+0000","last_unstale":"2024-09-17T02:50:04.975636+0000","last_undegraded":"2024-09-17T02:50:04.975636+0000","last_fullsized":"2024-09-17T02:50:04.975636+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:41:16.727032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.242421+0000","last_change":"2024-09-17T02:50:03.237719+0000","last_active":"2024-09-17T02:50:04.242421+0000","last_peered":"2024-09-17T02:50:04.242421+0000","last_clean":"2024-09-17T02:50:04.242421+0000","last_became_active":"2024-09-17T02:50:03.236811+0000","last_became_peered":"2024-09-17T02:50:03.236811+0000","last_unstale":"2024-09-17T02:50:04.242421+0000","last_undegraded":"2024-09-17T02:50:04.242421+0000","last_fullsized":"2024-09-17T02:50:04.242421+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:55:06.545885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1a","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.978278+0000","last_change":"2024-09-17T02:50:02.193710+0000","last_active":"2024-09-17T02:50:04.978278+0000","last_peered":"2024-09-17T02:50:04.978278+0000","last_clean":"2024-09-17T02:50:04.978278+0000","last_became_active":"2024-09-17T02:50:02.193264+0000","last_became_peered":"2024-09-17T02:50:02.193264+0000","last_unstale":"2024-09-17T02:50:04.978278+0000","last_undegraded":"2024-09-17T02:50:04.978278+0000","last_fullsized":"2024-09-17T02:50:04.978278+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:56.076336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.976631+0000","last_change":"2024-09-17T02:50:03.248781+0000","last_active":"2024-09-17T02:50:04.976631+0000","last_peered":"2024-09-17T02:50:04.976631+0000","last_clean":"2024-09-17T02:50:04.976631+0000","last_became_active":"2024-09-17T02:50:03.248485+0000","last_became_peered":"2024-09-17T02:50:03.248485+0000","last_unstale":"2024-09-17T02:50:04.976631+0000","last_undegraded":"2024-09-17T02:50:04.976631+0000","last_fullsized":"2024-09-17T02:50:04.976631+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:10:33.131906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.977429+0000","last_change":"2024-09-17T02:50:02.194688+0000","last_active":"2024-09-17T02:50:04.977429+0000","last_peered":"2024-09-17T02:50:04.977429+0000","last_clean":"2024-09-17T02:50:04.977429+0000","last_became_active":"2024-09-17T02:50:02.194037+0000","last_became_peered":"2024-09-17T02:50:02.194037+0000","last_unstale":"2024-09-17T02:50:04.977429+0000","last_undegraded":"2024-09-17T02:50:04.977429+0000","last_fullsized":"2024-09-17T02:50:04.977429+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:41:45.976035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.243081+0000","last_change":"2024-09-17T02:50:03.257565+0000","last_active":"2024-09-17T02:50:04.243081+0000","last_peered":"2024-09-17T02:50:04.243081+0000","last_clean":"2024-09-17T02:50:04.243081+0000","last_became_active":"2024-09-17T02:50:03.253367+0000","last_became_peered":"2024-09-17T02:50:03.253367+0000","last_unstale":"2024-09-17T02:50:04.243081+0000","last_undegraded":"2024-09-17T02:50:04.243081+0000","last_fullsized":"2024-09-17T02:50:04.243081+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:50:02.680897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.974745+0000","last_change":"2024-09-17T02:50:02.213950+0000","last_active":"2024-09-17T02:50:04.974745+0000","last_peered":"2024-09-17T02:50:04.974745+0000","last_clean":"2024-09-17T02:50:04.974745+0000","last_became_active":"2024-09-17T02:50:02.213305+0000","last_became_peered":"2024-09-17T02:50:02.213305+0000","last_unstale":"2024-09-17T02:50:04.974745+0000","last_undegraded":"2024-09-17T02:50:04.974745+0000","last_fullsized":"2024-09-17T02:50:04.974745+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:26.754424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.237110+0000","last_change":"2024-09-17T02:50:03.239071+0000","last_active":"2024-09-17T02:50:04.237110+0000","last_peered":"2024-09-17T02:50:04.237110+0000","last_clean":"2024-09-17T02:50:04.237110+0000","last_became_active":"2024-09-17T02:50:03.238160+0000","last_became_peered":"2024-09-17T02:50:03.238160+0000","last_unstale":"2024-09-17T02:50:04.237110+0000","last_undegraded":"2024-09-17T02:50:04.237110+0000","last_fullsized":"2024-09-17T02:50:04.237110+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:40.820002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.253287+0000","last_change":"2024-09-17T02:50:02.198464+0000","last_active":"2024-09-17T02:50:04.253287+0000","last_peered":"2024-09-17T02:50:04.253287+0000","last_clean":"2024-09-17T02:50:04.253287+0000","last_became_active":"2024-09-17T02:50:02.197985+0000","last_became_peered":"2024-09-17T02:50:02.197985+0000","last_unstale":"2024-09-17T02:50:04.253287+0000","last_undegraded":"2024-09-17T02:50:04.253287+0000","last_fullsized":"2024-09-17T02:50:04.253287+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:35:13.343561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.237759+0000","last_change":"2024-09-17T02:50:03.235490+0000","last_active":"2024-09-17T02:50:04.237759+0000","last_peered":"2024-09-17T02:50:04.237759+0000","last_clean":"2024-09-17T02:50:04.237759+0000","last_became_active":"2024-09-17T02:50:03.221959+0000","last_became_peered":"2024-09-17T02:50:03.221959+0000","last_unstale":"2024-09-17T02:50:04.237759+0000","last_undegraded":"2024-09-17T02:50:04.237759+0000","last_fullsized":"2024-09-17T02:50:04.237759+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:33.777066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.240582+0000","last_change":"2024-09-17T02:50:02.202455+0000","last_active":"2024-09-17T02:50:04.240582+0000","last_peered":"2024-09-17T02:50:04.240582+0000","last_clean":"2024-09-17T02:50:04.240582+0000","last_became_active":"2024-09-17T02:50:02.201736+0000","last_became_peered":"2024-09-17T02:50:02.201736+0000","last_unstale":"2024-09-17T02:50:04.240582+0000","last_undegraded":"2024-09-17T02:50:04.240582+0000","last_fullsized":"2024-09-17T02:50:04.240582+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:49.449183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.239865+0000","last_change":"2024-09-17T02:50:03.259763+0000","last_active":"2024-09-17T02:50:04.239865+0000","last_peered":"2024-09-17T02:50:04.239865+0000","last_clean":"2024-09-17T02:50:04.239865+0000","last_became_active":"2024-09-17T02:50:03.259035+0000","last_became_peered":"2024-09-17T02:50:03.259035+0000","last_unstale":"2024-09-17T02:50:04.239865+0000","last_undegraded":"2024-09-17T02:50:04.239865+0000","last_fullsized":"2024-09-17T02:50:04.239865+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:05.559336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.15","version":"106'3","reported_seq":18,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.252749+0000","last_change":"2024-09-17T02:50:02.199393+0000","last_active":"2024-09-17T02:50:04.252749+0000","last_peered":"2024-09-17T02:50:04.252749+0000","last_clean":"2024-09-17T02:50:04.252749+0000","last_became_active":"2024-09-17T02:50:02.198949+0000","last_became_peered":"2024-09-17T02:50:02.198949+0000","last_unstale":"2024-09-17T02:50:04.252749+0000","last_undegraded":"2024-09-17T02:50:04.252749+0000","last_fullsized":"2024-09-17T02:50:04.252749+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:37:43.482653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.975357+0000","last_change":"2024-09-17T02:50:03.263959+0000","last_active":"2024-09-17T02:50:04.975357+0000","last_peered":"2024-09-17T02:50:04.975357+0000","last_clean":"2024-09-17T02:50:04.975357+0000","last_became_active":"2024-09-17T02:50:03.263264+0000","last_became_peered":"2024-09-17T02:50:03.263264+0000","last_unstale":"2024-09-17T02:50:04.975357+0000","last_undegraded":"2024-09-17T02:50:04.975357+0000","last_fullsized":"2024-09-17T02:50:04.975357+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:13.836490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.14","version":"106'6","reported_seq":21,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.975654+0000","last_change":"2024-09-17T02:50:02.202160+0000","last_active":"2024-09-17T02:50:04.975654+0000","last_peered":"2024-09-17T02:50:04.975654+0000","last_clean":"2024-09-17T02:50:04.975654+0000","last_became_active":"2024-09-17T02:50:02.201861+0000","last_became_peered":"2024-09-17T02:50:02.201861+0000","last_unstale":"2024-09-17T02:50:04.975654+0000","last_undegraded":"2024-09-17T02:50:04.975654+0000","last_fullsized":"2024-09-17T02:50:04.975654+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:31:33.592724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.235324+0000","last_change":"2024-09-17T02:50:03.261411+0000","last_active":"2024-09-17T02:50:04.235324+0000","last_peered":"2024-09-17T02:50:04.235324+0000","last_clean":"2024-09-17T02:50:04.235324+0000","last_became_active":"2024-09-17T02:50:03.261027+0000","last_became_peered":"2024-09-17T02:50:03.261027+0000","last_unstale":"2024-09-17T02:50:04.235324+0000","last_undegraded":"2024-09-17T02:50:04.235324+0000","last_fullsized":"2024-09-17T02:50:04.235324+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:56:06.083472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.239930+0000","last_change":"2024-09-17T02:50:02.216577+0000","last_active":"2024-09-17T02:50:04.239930+0000","last_peered":"2024-09-17T02:50:04.239930+0000","last_clean":"2024-09-17T02:50:04.239930+0000","last_became_active":"2024-09-17T02:50:02.212635+0000","last_became_peered":"2024-09-17T02:50:02.212635+0000","last_unstale":"2024-09-17T02:50:04.239930+0000","last_undegraded":"2024-09-17T02:50:04.239930+0000","last_fullsized":"2024-09-17T02:50:04.239930+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:07:12.569207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.235597+0000","last_change":"2024-09-17T02:50:03.255790+0000","last_active":"2024-09-17T02:50:04.235597+0000","last_peered":"2024-09-17T02:50:04.235597+0000","last_clean":"2024-09-17T02:50:04.235597+0000","last_became_active":"2024-09-17T02:50:03.255423+0000","last_became_peered":"2024-09-17T02:50:03.255423+0000","last_unstale":"2024-09-17T02:50:04.235597+0000","last_undegraded":"2024-09-17T02:50:04.235597+0000","last_fullsized":"2024-09-17T02:50:04.235597+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:32.646432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.241704+0000","last_change":"2024-09-17T02:50:02.195474+0000","last_active":"2024-09-17T02:50:04.241704+0000","last_peered":"2024-09-17T02:50:04.241704+0000","last_clean":"2024-09-17T02:50:04.241704+0000","last_became_active":"2024-09-17T02:50:02.194616+0000","last_became_peered":"2024-09-17T02:50:02.194616+0000","last_unstale":"2024-09-17T02:50:04.241704+0000","last_undegraded":"2024-09-17T02:50:04.241704+0000","last_fullsized":"2024-09-17T02:50:04.241704+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:30:52.770430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.975259+0000","last_change":"2024-09-17T02:50:03.275940+0000","last_active":"2024-09-17T02:50:04.975259+0000","last_peered":"2024-09-17T02:50:04.975259+0000","last_clean":"2024-09-17T02:50:04.975259+0000","last_became_active":"2024-09-17T02:50:03.275288+0000","last_became_peered":"2024-09-17T02:50:03.275288+0000","last_unstale":"2024-09-17T02:50:04.975259+0000","last_undegraded":"2024-09-17T02:50:04.975259+0000","last_fullsized":"2024-09-17T02:50:04.975259+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:38.889956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.237967+0000","last_change":"2024-09-17T02:50:02.211967+0000","last_active":"2024-09-17T02:50:04.237967+0000","last_peered":"2024-09-17T02:50:04.237967+0000","last_clean":"2024-09-17T02:50:04.237967+0000","last_became_active":"2024-09-17T02:50:02.204543+0000","last_became_peered":"2024-09-17T02:50:02.204543+0000","last_unstale":"2024-09-17T02:50:04.237967+0000","last_undegraded":"2024-09-17T02:50:04.237967+0000","last_fullsized":"2024-09-17T02:50:04.237967+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:07:32.954425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.247655+0000","last_change":"2024-09-17T02:50:03.252543+0000","last_active":"2024-09-17T02:50:04.247655+0000","last_peered":"2024-09-17T02:50:04.247655+0000","last_clean":"2024-09-17T02:50:04.247655+0000","last_became_active":"2024-09-17T02:50:03.245258+0000","last_became_peered":"2024-09-17T02:50:03.245258+0000","last_unstale":"2024-09-17T02:50:04.247655+0000","last_undegraded":"2024-09-17T02:50:04.247655+0000","last_fullsized":"2024-09-17T02:50:04.247655+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:56:10.545949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.10","version":"106'8","reported_seq":23,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.975366+0000","last_change":"2024-09-17T02:50:02.186109+0000","last_active":"2024-09-17T02:50:04.975366+0000","last_peered":"2024-09-17T02:50:04.975366+0000","last_clean":"2024-09-17T02:50:04.975366+0000","last_became_active":"2024-09-17T02:50:02.185628+0000","last_became_peered":"2024-09-17T02:50:02.185628+0000","last_unstale":"2024-09-17T02:50:04.975366+0000","last_undegraded":"2024-09-17T02:50:04.975366+0000","last_fullsized":"2024-09-17T02:50:04.975366+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:23.186901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.240228+0000","last_change":"2024-09-17T02:50:03.236807+0000","last_active":"2024-09-17T02:50:04.240228+0000","last_peered":"2024-09-17T02:50:04.240228+0000","last_clean":"2024-09-17T02:50:04.240228+0000","last_became_active":"2024-09-17T02:50:03.235361+0000","last_became_peered":"2024-09-17T02:50:03.235361+0000","last_unstale":"2024-09-17T02:50:04.240228+0000","last_undegraded":"2024-09-17T02:50:04.240228+0000","last_fullsized":"2024-09-17T02:50:04.240228+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:41.560146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.253604+0000","last_change":"2024-09-17T02:50:02.204886+0000","last_active":"2024-09-17T02:50:04.253604+0000","last_peered":"2024-09-17T02:50:04.253604+0000","last_clean":"2024-09-17T02:50:04.253604+0000","last_became_active":"2024-09-17T02:50:02.204277+0000","last_became_peered":"2024-09-17T02:50:02.204277+0000","last_unstale":"2024-09-17T02:50:04.253604+0000","last_undegraded":"2024-09-17T02:50:04.253604+0000","last_fullsized":"2024-09-17T02:50:04.253604+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:36:22.615630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.977843+0000","last_change":"2024-09-17T02:50:03.255952+0000","last_active":"2024-09-17T02:50:04.977843+0000","last_peered":"2024-09-17T02:50:04.977843+0000","last_clean":"2024-09-17T02:50:04.977843+0000","last_became_active":"2024-09-17T02:50:03.255590+0000","last_became_peered":"2024-09-17T02:50:03.255590+0000","last_unstale":"2024-09-17T02:50:04.977843+0000","last_undegraded":"2024-09-17T02:50:04.977843+0000","last_fullsized":"2024-09-17T02:50:04.977843+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:29:54.514404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.d","version":"106'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.978484+0000","last_change":"2024-09-17T02:50:02.191573+0000","last_active":"2024-09-17T02:50:04.978484+0000","last_peered":"2024-09-17T02:50:04.978484+0000","last_clean":"2024-09-17T02:50:04.978484+0000","last_became_active":"2024-09-17T02:50:02.191137+0000","last_became_peered":"2024-09-17T02:50:02.191137+0000","last_unstale":"2024-09-17T02:50:04.978484+0000","last_undegraded":"2024-09-17T02:50:04.978484+0000","last_fullsized":"2024-09-17T02:50:04.978484+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:10:44.704124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.974789+0000","last_change":"2024-09-17T02:50:03.275234+0000","last_active":"2024-09-17T02:50:04.974789+0000","last_peered":"2024-09-17T02:50:04.974789+0000","last_clean":"2024-09-17T02:50:04.974789+0000","last_became_active":"2024-09-17T02:50:03.271004+0000","last_became_peered":"2024-09-17T02:50:03.271004+0000","last_unstale":"2024-09-17T02:50:04.974789+0000","last_undegraded":"2024-09-17T02:50:04.974789+0000","last_fullsized":"2024-09-17T02:50:04.974789+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:24:19.229397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.253827+0000","last_change":"2024-09-17T02:50:02.200484+0000","last_active":"2024-09-17T02:50:04.253827+0000","last_peered":"2024-09-17T02:50:04.253827+0000","last_clean":"2024-09-17T02:50:04.253827+0000","last_became_active":"2024-09-17T02:50:02.199258+0000","last_became_peered":"2024-09-17T02:50:02.199258+0000","last_unstale":"2024-09-17T02:50:04.253827+0000","last_undegraded":"2024-09-17T02:50:04.253827+0000","last_fullsized":"2024-09-17T02:50:04.253827+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:45:14.011752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.241483+0000","last_change":"2024-09-17T02:50:03.256578+0000","last_active":"2024-09-17T02:50:04.241483+0000","last_peered":"2024-09-17T02:50:04.241483+0000","last_clean":"2024-09-17T02:50:04.241483+0000","last_became_active":"2024-09-17T02:50:03.249803+0000","last_became_peered":"2024-09-17T02:50:03.249803+0000","last_unstale":"2024-09-17T02:50:04.241483+0000","last_undegraded":"2024-09-17T02:50:04.241483+0000","last_fullsized":"2024-09-17T02:50:04.241483+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:40:03.801586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.b","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.246805+0000","last_change":"2024-09-17T02:50:02.215559+0000","last_active":"2024-09-17T02:50:04.246805+0000","last_peered":"2024-09-17T02:50:04.246805+0000","last_clean":"2024-09-17T02:50:04.246805+0000","last_became_active":"2024-09-17T02:50:02.214996+0000","last_became_peered":"2024-09-17T02:50:02.214996+0000","last_unstale":"2024-09-17T02:50:04.246805+0000","last_undegraded":"2024-09-17T02:50:04.246805+0000","last_fullsized":"2024-09-17T02:50:04.246805+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:48.348160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.975062+0000","last_change":"2024-09-17T02:50:03.276653+0000","last_active":"2024-09-17T02:50:04.975062+0000","last_peered":"2024-09-17T02:50:04.975062+0000","last_clean":"2024-09-17T02:50:04.975062+0000","last_became_active":"2024-09-17T02:50:03.276218+0000","last_became_peered":"2024-09-17T02:50:03.276218+0000","last_unstale":"2024-09-17T02:50:04.975062+0000","last_undegraded":"2024-09-17T02:50:04.975062+0000","last_fullsized":"2024-09-17T02:50:04.975062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:17:39.691569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.246059+0000","last_change":"2024-09-17T02:50:02.215360+0000","last_active":"2024-09-17T02:50:04.246059+0000","last_peered":"2024-09-17T02:50:04.246059+0000","last_clean":"2024-09-17T02:50:04.246059+0000","last_became_active":"2024-09-17T02:50:02.214768+0000","last_became_peered":"2024-09-17T02:50:02.214768+0000","last_unstale":"2024-09-17T02:50:04.246059+0000","last_undegraded":"2024-09-17T02:50:04.246059+0000","last_fullsized":"2024-09-17T02:50:04.246059+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:35.806416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.974880+0000","last_change":"2024-09-17T02:50:03.268508+0000","last_active":"2024-09-17T02:50:04.974880+0000","last_peered":"2024-09-17T02:50:04.974880+0000","last_clean":"2024-09-17T02:50:04.974880+0000","last_became_active":"2024-09-17T02:50:03.263907+0000","last_became_peered":"2024-09-17T02:50:03.263907+0000","last_unstale":"2024-09-17T02:50:04.974880+0000","last_undegraded":"2024-09-17T02:50:04.974880+0000","last_fullsized":"2024-09-17T02:50:04.974880+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:17.073536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.253169+0000","last_change":"2024-09-17T02:50:02.215827+0000","last_active":"2024-09-17T02:50:04.253169+0000","last_peered":"2024-09-17T02:50:04.253169+0000","last_clean":"2024-09-17T02:50:04.253169+0000","last_became_active":"2024-09-17T02:50:02.215449+0000","last_became_peered":"2024-09-17T02:50:02.215449+0000","last_unstale":"2024-09-17T02:50:04.253169+0000","last_undegraded":"2024-09-17T02:50:04.253169+0000","last_fullsized":"2024-09-17T02:50:04.253169+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:18:46.561734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.242773+0000","last_change":"2024-09-17T02:50:03.256876+0000","last_active":"2024-09-17T02:50:04.242773+0000","last_peered":"2024-09-17T02:50:04.242773+0000","last_clean":"2024-09-17T02:50:04.242773+0000","last_became_active":"2024-09-17T02:50:03.255999+0000","last_became_peered":"2024-09-17T02:50:03.255999+0000","last_unstale":"2024-09-17T02:50:04.242773+0000","last_undegraded":"2024-09-17T02:50:04.242773+0000","last_fullsized":"2024-09-17T02:50:04.242773+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:37:22.051886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.238436+0000","last_change":"2024-09-17T02:50:02.205086+0000","last_active":"2024-09-17T02:50:04.238436+0000","last_peered":"2024-09-17T02:50:04.238436+0000","last_clean":"2024-09-17T02:50:04.238436+0000","last_became_active":"2024-09-17T02:50:02.202045+0000","last_became_peered":"2024-09-17T02:50:02.202045+0000","last_unstale":"2024-09-17T02:50:04.238436+0000","last_undegraded":"2024-09-17T02:50:04.238436+0000","last_fullsized":"2024-09-17T02:50:04.238436+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:28.182659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.974724+0000","last_change":"2024-09-17T02:50:03.261187+0000","last_active":"2024-09-17T02:50:04.974724+0000","last_peered":"2024-09-17T02:50:04.974724+0000","last_clean":"2024-09-17T02:50:04.974724+0000","last_became_active":"2024-09-17T02:50:03.260588+0000","last_became_peered":"2024-09-17T02:50:03.260588+0000","last_unstale":"2024-09-17T02:50:04.974724+0000","last_undegraded":"2024-09-17T02:50:04.974724+0000","last_fullsized":"2024-09-17T02:50:04.974724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:37:34.435434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.7","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.247843+0000","last_change":"2024-09-17T02:50:02.208031+0000","last_active":"2024-09-17T02:50:04.247843+0000","last_peered":"2024-09-17T02:50:04.247843+0000","last_clean":"2024-09-17T02:50:04.247843+0000","last_became_active":"2024-09-17T02:50:02.207439+0000","last_became_peered":"2024-09-17T02:50:02.207439+0000","last_unstale":"2024-09-17T02:50:04.247843+0000","last_undegraded":"2024-09-17T02:50:04.247843+0000","last_fullsized":"2024-09-17T02:50:04.247843+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:22:07.804751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.976006+0000","last_change":"2024-09-17T02:50:03.231862+0000","last_active":"2024-09-17T02:50:04.976006+0000","last_peered":"2024-09-17T02:50:04.976006+0000","last_clean":"2024-09-17T02:50:04.976006+0000","last_became_active":"2024-09-17T02:50:03.231480+0000","last_became_peered":"2024-09-17T02:50:03.231480+0000","last_unstale":"2024-09-17T02:50:04.976006+0000","last_undegraded":"2024-09-17T02:50:04.976006+0000","last_fullsized":"2024-09-17T02:50:04.976006+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:45:28.320400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.6","version":"107'1","reported_seq":16,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.247523+0000","last_change":"2024-09-17T02:50:02.207675+0000","last_active":"2024-09-17T02:50:04.247523+0000","last_peered":"2024-09-17T02:50:04.247523+0000","last_clean":"2024-09-17T02:50:04.247523+0000","last_became_active":"2024-09-17T02:50:02.199993+0000","last_became_peered":"2024-09-17T02:50:02.199993+0000","last_unstale":"2024-09-17T02:50:04.247523+0000","last_undegraded":"2024-09-17T02:50:04.247523+0000","last_fullsized":"2024-09-17T02:50:04.247523+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:31:01.006609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.239213+0000","last_change":"2024-09-17T02:50:03.248915+0000","last_active":"2024-09-17T02:50:04.239213+0000","last_peered":"2024-09-17T02:50:04.239213+0000","last_clean":"2024-09-17T02:50:04.239213+0000","last_became_active":"2024-09-17T02:50:03.248029+0000","last_became_peered":"2024-09-17T02:50:03.248029+0000","last_unstale":"2024-09-17T02:50:04.239213+0000","last_undegraded":"2024-09-17T02:50:04.239213+0000","last_fullsized":"2024-09-17T02:50:04.239213+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:08:03.354582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.976240+0000","last_change":"2024-09-17T02:50:02.195945+0000","last_active":"2024-09-17T02:50:04.976240+0000","last_peered":"2024-09-17T02:50:04.976240+0000","last_clean":"2024-09-17T02:50:04.976240+0000","last_became_active":"2024-09-17T02:50:02.195499+0000","last_became_peered":"2024-09-17T02:50:02.195499+0000","last_unstale":"2024-09-17T02:50:04.976240+0000","last_undegraded":"2024-09-17T02:50:04.976240+0000","last_fullsized":"2024-09-17T02:50:04.976240+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:27:36.790578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.978599+0000","last_change":"2024-09-17T02:50:03.247632+0000","last_active":"2024-09-17T02:50:04.978599+0000","last_peered":"2024-09-17T02:50:04.978599+0000","last_clean":"2024-09-17T02:50:04.978599+0000","last_became_active":"2024-09-17T02:50:03.247260+0000","last_became_peered":"2024-09-17T02:50:03.247260+0000","last_unstale":"2024-09-17T02:50:04.978599+0000","last_undegraded":"2024-09-17T02:50:04.978599+0000","last_fullsized":"2024-09-17T02:50:04.978599+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:05.117374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.241595+0000","last_change":"2024-09-17T02:50:02.218607+0000","last_active":"2024-09-17T02:50:04.241595+0000","last_peered":"2024-09-17T02:50:04.241595+0000","last_clean":"2024-09-17T02:50:04.241595+0000","last_became_active":"2024-09-17T02:50:02.217911+0000","last_became_peered":"2024-09-17T02:50:02.217911+0000","last_unstale":"2024-09-17T02:50:04.241595+0000","last_undegraded":"2024-09-17T02:50:04.241595+0000","last_fullsized":"2024-09-17T02:50:04.241595+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:31:49.289370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.241088+0000","last_change":"2024-09-17T02:50:03.270759+0000","last_active":"2024-09-17T02:50:04.241088+0000","last_peered":"2024-09-17T02:50:04.241088+0000","last_clean":"2024-09-17T02:50:04.241088+0000","last_became_active":"2024-09-17T02:50:03.266869+0000","last_became_peered":"2024-09-17T02:50:03.266869+0000","last_unstale":"2024-09-17T02:50:04.241088+0000","last_undegraded":"2024-09-17T02:50:04.241088+0000","last_fullsized":"2024-09-17T02:50:04.241088+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:15:54.228399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.975969+0000","last_change":"2024-09-17T02:50:02.191068+0000","last_active":"2024-09-17T02:50:04.975969+0000","last_peered":"2024-09-17T02:50:04.975969+0000","last_clean":"2024-09-17T02:50:04.975969+0000","last_became_active":"2024-09-17T02:50:02.190620+0000","last_became_peered":"2024-09-17T02:50:02.190620+0000","last_unstale":"2024-09-17T02:50:04.975969+0000","last_undegraded":"2024-09-17T02:50:04.975969+0000","last_fullsized":"2024-09-17T02:50:04.975969+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:15:13.551875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.242297+0000","last_change":"2024-09-17T02:50:03.227872+0000","last_active":"2024-09-17T02:50:04.242297+0000","last_peered":"2024-09-17T02:50:04.242297+0000","last_clean":"2024-09-17T02:50:04.242297+0000","last_became_active":"2024-09-17T02:50:03.226982+0000","last_became_peered":"2024-09-17T02:50:03.226982+0000","last_unstale":"2024-09-17T02:50:04.242297+0000","last_undegraded":"2024-09-17T02:50:04.242297+0000","last_fullsized":"2024-09-17T02:50:04.242297+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:29:35.893055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.246423+0000","last_change":"2024-09-17T02:50:02.199720+0000","last_active":"2024-09-17T02:50:04.246423+0000","last_peered":"2024-09-17T02:50:04.246423+0000","last_clean":"2024-09-17T02:50:04.246423+0000","last_became_active":"2024-09-17T02:50:02.198778+0000","last_became_peered":"2024-09-17T02:50:02.198778+0000","last_unstale":"2024-09-17T02:50:04.246423+0000","last_undegraded":"2024-09-17T02:50:04.246423+0000","last_fullsized":"2024-09-17T02:50:04.246423+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:37.276221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.252886+0000","last_change":"2024-09-17T02:50:03.252666+0000","last_active":"2024-09-17T02:50:04.252886+0000","last_peered":"2024-09-17T02:50:04.252886+0000","last_clean":"2024-09-17T02:50:04.252886+0000","last_became_active":"2024-09-17T02:50:03.251491+0000","last_became_peered":"2024-09-17T02:50:03.251491+0000","last_unstale":"2024-09-17T02:50:04.252886+0000","last_undegraded":"2024-09-17T02:50:04.252886+0000","last_fullsized":"2024-09-17T02:50:04.252886+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:54:06.196326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.977624+0000","last_change":"2024-09-17T02:50:02.202459+0000","last_active":"2024-09-17T02:50:04.977624+0000","last_peered":"2024-09-17T02:50:04.977624+0000","last_clean":"2024-09-17T02:50:04.977624+0000","last_became_active":"2024-09-17T02:50:02.202233+0000","last_became_peered":"2024-09-17T02:50:02.202233+0000","last_unstale":"2024-09-17T02:50:04.977624+0000","last_undegraded":"2024-09-17T02:50:04.977624+0000","last_fullsized":"2024-09-17T02:50:04.977624+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:48.912493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.235409+0000","last_change":"2024-09-17T02:50:03.241112+0000","last_active":"2024-09-17T02:50:04.235409+0000","last_peered":"2024-09-17T02:50:04.235409+0000","last_clean":"2024-09-17T02:50:04.235409+0000","last_became_active":"2024-09-17T02:50:03.240702+0000","last_became_peered":"2024-09-17T02:50:03.240702+0000","last_unstale":"2024-09-17T02:50:04.235409+0000","last_undegraded":"2024-09-17T02:50:04.235409+0000","last_fullsized":"2024-09-17T02:50:04.235409+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:26:40.425800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.246139+0000","last_change":"2024-09-17T02:50:02.195758+0000","last_active":"2024-09-17T02:50:04.246139+0000","last_peered":"2024-09-17T02:50:04.246139+0000","last_clean":"2024-09-17T02:50:04.246139+0000","last_became_active":"2024-09-17T02:50:02.194921+0000","last_became_peered":"2024-09-17T02:50:02.194921+0000","last_unstale":"2024-09-17T02:50:04.246139+0000","last_undegraded":"2024-09-17T02:50:04.246139+0000","last_fullsized":"2024-09-17T02:50:04.246139+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:41:10.933564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.253442+0000","last_change":"2024-09-17T02:50:03.275647+0000","last_active":"2024-09-17T02:50:04.253442+0000","last_peered":"2024-09-17T02:50:04.253442+0000","last_clean":"2024-09-17T02:50:04.253442+0000","last_became_active":"2024-09-17T02:50:03.275298+0000","last_became_peered":"2024-09-17T02:50:03.275298+0000","last_unstale":"2024-09-17T02:50:04.253442+0000","last_undegraded":"2024-09-17T02:50:04.253442+0000","last_fullsized":"2024-09-17T02:50:04.253442+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:02.154424+0000","last_clean_scrub_stamp":"2024-09-17T02:50:02.154424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:38:55.158531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"106'2","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.242175+0000","last_change":"2024-09-17T02:50:02.216926+0000","last_active":"2024-09-17T02:50:04.242175+0000","last_peered":"2024-09-17T02:50:04.242175+0000","last_clean":"2024-09-17T02:50:04.242175+0000","last_became_active":"2024-09-17T02:50:02.212076+0000","last_became_peered":"2024-09-17T02:50:02.212076+0000","last_unstale":"2024-09-17T02:50:04.242175+0000","last_undegraded":"2024-09-17T02:50:04.242175+0000","last_fullsized":"2024-09-17T02:50:04.242175+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:50:01.150594+0000","last_clean_scrub_stamp":"2024-09-17T02:50:01.150594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:04.113544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":275,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-09-17T02:50:04.977476+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:50:04.977476+0000","last_peered":"2024-09-17T02:50:04.977476+0000","last_clean":"2024-09-17T02:50:04.977476+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:50:04.977476+0000","last_undegraded":"2024-09-17T02:50:04.977476+0000","last_fullsized":"2024-09-17T02:50:04.977476+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509557,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1327104,"data_stored":613800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509557,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1921024,"data_stored":1203640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509558,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1323008,"data_stored":613182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509556,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1331200,"data_stored":614760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509557,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1296,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1327104,"data_stored":613260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509557,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1921024,"data_stored":1203608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509558,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1323008,"data_stored":613730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574965,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1318912,"data_stored":608068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:50:07.773 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T02:50:07.776 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T02:50:07.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 -- 172.21.15.131:0/678430999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ca8003a40 msgr2=0x7f3ca8003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 --2- 172.21.15.131:0/678430999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ca8003a40 0x7f3ca8003eb0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000bb10 tx=0x7f3cb001cd90 comp rx=0 tx=0).stop 2024-09-17T02:50:07.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbed9d640 1 -- 172.21.15.131:0/678430999 reap_dead start 2024-09-17T02:50:07.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 -- 172.21.15.131:0/678430999 shutdown_connections 2024-09-17T02:50:07.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 -- 172.21.15.131:0/678430999 >> 172.21.15.131:0/678430999 conn(0x7f3cc00d0490 msgr2=0x7f3cc006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:07.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 -- 172.21.15.131:0/678430999 shutdown_connections 2024-09-17T02:50:07.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 -- 172.21.15.131:0/678430999 wait complete. 2024-09-17T02:50:07.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 Processor -- start 2024-09-17T02:50:07.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 -- start start 2024-09-17T02:50:07.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ca8003a40 0x7f3cc015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3cc0155500 0x7f3cc0158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc0155910 0x7f3cc01f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc01687a0 con 0x7f3cc0155910 2024-09-17T02:50:07.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0168620 con 0x7f3ca8003a40 2024-09-17T02:50:07.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cc48e2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc0168920 con 0x7f3cc0155500 2024-09-17T02:50:07.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc0155910 0x7f3cc01f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc0155910 0x7f3cc01f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50740/0 (socket says 172.21.15.131:50740) 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbed9d640 1 -- 172.21.15.131:0/3883652577 learned_addr learned my addr 172.21.15.131:0/3883652577 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbdd9b640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3cc0155500 0x7f3cc0158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbed9d640 1 -- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3cc0155500 msgr2=0x7f3cc0158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbed9d640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3cc0155500 0x7f3cc0158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbed9d640 1 -- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ca8003a40 msgr2=0x7f3cc015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbe59c640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ca8003a40 0x7f3cc015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbed9d640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ca8003a40 0x7f3cc015b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbed9d640 1 -- 172.21.15.131:0/3883652577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc01f73f0 con 0x7f3cc0155910 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.972+0000 7f3cbed9d640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc0155910 0x7f3cc01f6630 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00b4c80 tx=0x7f3cb800dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.976+0000 7f3caeffd640 1 -- 172.21.15.131:0/3883652577 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8003d90 con 0x7f3cc0155910 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.976+0000 7f3caeffd640 1 -- 172.21.15.131:0/3883652577 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cb800b890 con 0x7f3cc0155910 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.976+0000 7f3cc48e2640 1 -- 172.21.15.131:0/3883652577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc015a560 con 0x7f3cc0155910 2024-09-17T02:50:07.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.976+0000 7f3caeffd640 1 -- 172.21.15.131:0/3883652577 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb800bcc0 con 0x7f3cc0155910 2024-09-17T02:50:07.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.976+0000 7f3cc48e2640 1 -- 172.21.15.131:0/3883652577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc015ab20 con 0x7f3cc0155910 2024-09-17T02:50:07.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.976+0000 7f3cc48e2640 1 -- 172.21.15.131:0/3883652577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc01f7160 con 0x7f3cc0155910 2024-09-17T02:50:07.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.976+0000 7f3caeffd640 1 -- 172.21.15.131:0/3883652577 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3cb8006070 con 0x7f3cc0155910 2024-09-17T02:50:07.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.976+0000 7f3caeffd640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ca0073200 0x7f3ca0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:07.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.976+0000 7f3caeffd640 1 -- 172.21.15.131:0/3883652577 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f3cb8094940 con 0x7f3cc0155910 2024-09-17T02:50:07.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.976+0000 7f3cbe59c640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ca0073200 0x7f3ca0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:07.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.976+0000 7f3cbe59c640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ca0073200 0x7f3ca0075710 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00b0000 tx=0x7f3cb00a7040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:07.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:07.980+0000 7f3caeffd640 1 -- 172.21.15.131:0/3883652577 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3cb800be60 con 0x7f3cc0155910 2024-09-17T02:50:08.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.204+0000 7f3cc48e2640 1 -- 172.21.15.131:0/3883652577 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3cc00b3b90 con 0x7f3cc0155910 2024-09-17T02:50:08.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.208+0000 7f3caeffd640 1 -- 172.21.15.131:0/3883652577 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v35) ==== 190+0+147 (secure 0 0 0) 0x7f3cb805f7d0 con 0x7f3cc0155910 2024-09-17T02:50:08.205 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-09-17T02:50:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA== 2024-09-17T02:50:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-09-17T02:50:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-09-17T02:50:08.205 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T02:50:08.205 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-09-17T02:50:08.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.208+0000 7f3cc48e2640 1 -- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ca0073200 msgr2=0x7f3ca0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:08.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.208+0000 7f3cc48e2640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ca0073200 0x7f3ca0075710 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00b0000 tx=0x7f3cb00a7040 comp rx=0 tx=0).stop 2024-09-17T02:50:08.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.208+0000 7f3cc48e2640 1 -- 172.21.15.131:0/3883652577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc0155910 msgr2=0x7f3cc01f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:08.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.208+0000 7f3cc48e2640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc0155910 0x7f3cc01f6630 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00b4c80 tx=0x7f3cb800dd80 comp rx=0 tx=0).stop 2024-09-17T02:50:08.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.208+0000 7f3cc48e2640 1 -- 172.21.15.131:0/3883652577 shutdown_connections 2024-09-17T02:50:08.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.208+0000 7f3cc48e2640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3ca0073200 0x7f3ca0075710 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.208+0000 7f3cc48e2640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3cc0155910 0x7f3cc01f6630 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.208+0000 7f3cc48e2640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3cc0155500 0x7f3cc0158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.208+0000 7f3cc48e2640 1 --2- 172.21.15.131:0/3883652577 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ca8003a40 0x7f3cc015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.212+0000 7f3cc48e2640 1 -- 172.21.15.131:0/3883652577 >> 172.21.15.131:0/3883652577 conn(0x7f3cc00d0490 msgr2=0x7f3cc015eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:08.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.212+0000 7f3cc48e2640 1 -- 172.21.15.131:0/3883652577 shutdown_connections 2024-09-17T02:50:08.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.212+0000 7f3cc48e2640 1 -- 172.21.15.131:0/3883652577 wait complete. 2024-09-17T02:50:08.226 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:50:08.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.388+0000 7f82eaa54640 1 -- 172.21.15.131:0/1110803700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d0004b20 msgr2=0x7f82d0004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:08.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.388+0000 7f82eaa54640 1 --2- 172.21.15.131:0/1110803700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d0004b20 0x7f82d0004f90 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f82d400c610 tx=0x7f82d401d0b0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.388+0000 7f82eaa54640 1 -- 172.21.15.131:0/1110803700 shutdown_connections 2024-09-17T02:50:08.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.388+0000 7f82eaa54640 1 --2- 172.21.15.131:0/1110803700 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82d0004b20 0x7f82d0004f90 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.388+0000 7f82eaa54640 1 -- 172.21.15.131:0/1110803700 >> 172.21.15.131:0/1110803700 conn(0x7f82e406eea0 msgr2=0x7f82e406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:08.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.388+0000 7f82eaa54640 1 -- 172.21.15.131:0/1110803700 shutdown_connections 2024-09-17T02:50:08.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.388+0000 7f82eaa54640 1 -- 172.21.15.131:0/1110803700 wait complete. 2024-09-17T02:50:08.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82eaa54640 1 Processor -- start 2024-09-17T02:50:08.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82eaa54640 1 -- start start 2024-09-17T02:50:08.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82eaa54640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82d0004b20 0x7f82e41fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:08.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82eaa54640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f82e41e7b70 0x7f82e4200520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:08.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82eaa54640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82e4200a60 0x7f82e4149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:08.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82eaa54640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f82e415e7d0 con 0x7f82e4200a60 2024-09-17T02:50:08.389 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82eaa54640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f82e4073c20 con 0x7f82d0004b20 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82eaa54640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f82e415e950 con 0x7f82e41e7b70 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e8ff1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82e4200a60 0x7f82e4149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e8ff1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82e4200a60 0x7f82e4149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50772/0 (socket says 172.21.15.131:50772) 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e8ff1640 1 -- 172.21.15.131:0/2935865848 learned_addr learned my addr 172.21.15.131:0/2935865848 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e37fe640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f82e41e7b70 0x7f82e4200520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e3fff640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82d0004b20 0x7f82e41fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e8ff1640 1 -- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f82e41e7b70 msgr2=0x7f82e4200520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e8ff1640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f82e41e7b70 0x7f82e4200520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e8ff1640 1 -- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82d0004b20 msgr2=0x7f82e41fd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e8ff1640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82d0004b20 0x7f82e41fd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e8ff1640 1 -- 172.21.15.131:0/2935865848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e40cc0f0 con 0x7f82e4200a60 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e37fe640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f82e41e7b70 0x7f82e4200520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e8ff1640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82e4200a60 0x7f82e4149040 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f82e415b1e0 tx=0x7f82dc006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:08.390 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e0ff9640 1 -- 172.21.15.131:0/2935865848 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc011900 con 0x7f82e4200a60 2024-09-17T02:50:08.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82eaa54640 1 -- 172.21.15.131:0/2935865848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e40cf1f0 con 0x7f82e4200a60 2024-09-17T02:50:08.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e0ff9640 1 -- 172.21.15.131:0/2935865848 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82dc018070 con 0x7f82e4200a60 2024-09-17T02:50:08.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82eaa54640 1 -- 172.21.15.131:0/2935865848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e40ceca0 con 0x7f82e4200a60 2024-09-17T02:50:08.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82e0ff9640 1 -- 172.21.15.131:0/2935865848 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc011d40 con 0x7f82e4200a60 2024-09-17T02:50:08.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.392+0000 7f82eaa54640 1 -- 172.21.15.131:0/2935865848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e41f7220 con 0x7f82e4200a60 2024-09-17T02:50:08.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.396+0000 7f82e0ff9640 1 -- 172.21.15.131:0/2935865848 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f82dc01d460 con 0x7f82e4200a60 2024-09-17T02:50:08.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.396+0000 7f82e0ff9640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f82c4073200 0x7f82c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:08.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.396+0000 7f82e0ff9640 1 -- 172.21.15.131:0/2935865848 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f82dc095980 con 0x7f82e4200a60 2024-09-17T02:50:08.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.396+0000 7f82e3fff640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f82c4073200 0x7f82c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:08.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.396+0000 7f82e3fff640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f82c4073200 0x7f82c4075710 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f82d40ac000 tx=0x7f82d4019e50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:08.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.400+0000 7f82e0ff9640 1 -- 172.21.15.131:0/2935865848 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f82dc060770 con 0x7f82e4200a60 2024-09-17T02:50:08.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.604+0000 7f82eaa54640 1 -- 172.21.15.131:0/2935865848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82e415eee0 con 0x7f82e4200a60 2024-09-17T02:50:08.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.604+0000 7f82e0ff9640 1 -- 172.21.15.131:0/2935865848 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5237 (secure 0 0 0) 0x7f82dc060070 con 0x7f82e4200a60 2024-09-17T02:50:08.602 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:08.602 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":86,"btime":"2024-09-17T02:50:06:247461+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:50:05.662097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7860":{"gid":7860,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2930400858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2930400858},{"type":"v1","addr":"172.21.15.131:6835","nonce":2930400858}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7860,"qdb_cluster":[7860]},"id":7}]} 2024-09-17T02:50:08.603 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 86 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 -- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f82c4073200 msgr2=0x7f82c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f82c4073200 0x7f82c4075710 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f82d40ac000 tx=0x7f82d4019e50 comp rx=0 tx=0).stop 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 -- 172.21.15.131:0/2935865848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82e4200a60 msgr2=0x7f82e4149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82e4200a60 0x7f82e4149040 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f82e415b1e0 tx=0x7f82dc006430 comp rx=0 tx=0).stop 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 -- 172.21.15.131:0/2935865848 shutdown_connections 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f82c4073200 0x7f82c4075710 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f82e4200a60 0x7f82e4149040 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f82e41e7b70 0x7f82e4200520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 --2- 172.21.15.131:0/2935865848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f82d0004b20 0x7f82e41fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 -- 172.21.15.131:0/2935865848 >> 172.21.15.131:0/2935865848 conn(0x7f82e406eea0 msgr2=0x7f82e40afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 -- 172.21.15.131:0/2935865848 shutdown_connections 2024-09-17T02:50:08.605 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:08.608+0000 7f82eaa54640 1 -- 172.21.15.131:0/2935865848 wait complete. 2024-09-17T02:50:09.026 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:50:02.583706+0000', 'modified': '2024-09-17T02:50:05.662097+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7860}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7860': {'gid': 7860, 'name': 'a', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6835/2930400858', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6833', 'nonce': 2930400858}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 2930400858}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7860, 'qdb_cluster': [7860]} 2024-09-17T02:50:09.026 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:50:09.027 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T02:50:09.108 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-09-17T02:50:09.112+0000 7ff265b46640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:50:09.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.192+0000 7fb6cb968640 1 -- 172.21.15.131:0/3496480297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b0003a40 msgr2=0x7fb6b0003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.192+0000 7fb6cb968640 1 --2- 172.21.15.131:0/3496480297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6b0003a40 0x7fb6b0003eb0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0009b40 tx=0x7fb6c001d290 comp rx=0 tx=0).stop 2024-09-17T02:50:09.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.192+0000 7fb6c9f05640 1 -- 172.21.15.131:0/3496480297 reap_dead start 2024-09-17T02:50:09.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.192+0000 7fb6cb968640 1 -- 172.21.15.131:0/3496480297 shutdown_connections 2024-09-17T02:50:09.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.192+0000 7fb6cb968640 1 -- 172.21.15.131:0/3496480297 >> 172.21.15.131:0/3496480297 conn(0x7fb6c406eea0 msgr2=0x7fb6c406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:09.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.192+0000 7fb6cb968640 1 -- 172.21.15.131:0/3496480297 shutdown_connections 2024-09-17T02:50:09.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.192+0000 7fb6cb968640 1 -- 172.21.15.131:0/3496480297 wait complete. 2024-09-17T02:50:09.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6cb968640 1 Processor -- start 2024-09-17T02:50:09.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6cb968640 1 -- start start 2024-09-17T02:50:09.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6cb968640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb6b0003a40 0x7fb6c4158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6cb968640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41f72e0 0x7fb6c41f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6cb968640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c40cdbf0 0x7fb6c40ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6cb968640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4167d70 con 0x7fb6c41f72e0 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6cb968640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4167bf0 con 0x7fb6c40cdbf0 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6cb968640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c4167ef0 con 0x7fb6b0003a40 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c8f03640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41f72e0 0x7fb6c41f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c8f03640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41f72e0 0x7fb6c41f76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50794/0 (socket says 172.21.15.131:50794) 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c8f03640 1 -- 172.21.15.131:0/1652274534 learned_addr learned my addr 172.21.15.131:0/1652274534 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c9704640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb6b0003a40 0x7fb6c4158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c8f03640 1 -- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb6b0003a40 msgr2=0x7fb6c4158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c8f03640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb6b0003a40 0x7fb6c4158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c8f03640 1 -- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c40cdbf0 msgr2=0x7fb6c40ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c9f05640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c40cdbf0 0x7fb6c40ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c8f03640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c40cdbf0 0x7fb6c40ce000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c8f03640 1 -- 172.21.15.131:0/1652274534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c40ce540 con 0x7fb6c41f72e0 2024-09-17T02:50:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c9704640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb6b0003a40 0x7fb6c4158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6c8f03640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41f72e0 0x7fb6c41f76f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c500 tx=0x7fb6b400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6b9ffb640 1 -- 172.21.15.131:0/1652274534 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4016020 con 0x7fb6c41f72e0 2024-09-17T02:50:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6b9ffb640 1 -- 172.21.15.131:0/1652274534 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6b40053c0 con 0x7fb6c41f72e0 2024-09-17T02:50:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6cb968640 1 -- 172.21.15.131:0/1652274534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c4154120 con 0x7fb6c41f72e0 2024-09-17T02:50:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6b9ffb640 1 -- 172.21.15.131:0/1652274534 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4007280 con 0x7fb6c41f72e0 2024-09-17T02:50:09.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.196+0000 7fb6cb968640 1 -- 172.21.15.131:0/1652274534 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c41546e0 con 0x7fb6c41f72e0 2024-09-17T02:50:09.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6b9ffb640 1 -- 172.21.15.131:0/1652274534 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb6b4007420 con 0x7fb6c41f72e0 2024-09-17T02:50:09.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6b9ffb640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb6a8073230 0x7fb6a8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6b9ffb640 1 -- 172.21.15.131:0/1652274534 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fb6b4093ed0 con 0x7fb6c41f72e0 2024-09-17T02:50:09.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6c9704640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb6a8073230 0x7fb6a8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6d2c33000 1 Processor -- start 2024-09-17T02:50:09.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6d2c33000 1 -- start start 2024-09-17T02:50:09.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6c9704640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb6a8073230 0x7fb6a8075740 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00ad000 tx=0x7fb6c001a030 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc81a1460 0x55fdc86e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc8591950 0x55fdc86e7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc857aca0 0x55fdc86e7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fdc85b7410 con 0x55fdc81a1460 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fdc84f4b40 con 0x55fdc8591950 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fdc84e3f70 con 0x55fdc857aca0 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6c9704640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc81a1460 0x55fdc86e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6c9704640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc81a1460 0x55fdc86e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50810/0 (socket says 172.21.15.131:50810) 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.200+0000 7fb6c9704640 1 -- 172.21.15.131:0/2703053581 learned_addr learned my addr 172.21.15.131:0/2703053581 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6c9f05640 1 --2- 172.21.15.131:0/2703053581 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc857aca0 0x55fdc86e7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6c8f03640 1 --2- 172.21.15.131:0/2703053581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc8591950 0x55fdc86e7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6c9704640 1 -- 172.21.15.131:0/2703053581 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc857aca0 msgr2=0x55fdc86e7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6c9704640 1 --2- 172.21.15.131:0/2703053581 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc857aca0 0x55fdc86e7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6c9704640 1 -- 172.21.15.131:0/2703053581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc8591950 msgr2=0x55fdc86e7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6c9704640 1 --2- 172.21.15.131:0/2703053581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc8591950 0x55fdc86e7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6c9704640 1 -- 172.21.15.131:0/2703053581 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc84ab5b0 con 0x55fdc81a1460 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6c9704640 1 --2- 172.21.15.131:0/2703053581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc81a1460 0x55fdc86e6b00 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00054c0 tx=0x7fb6c00b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89ce55986bad91c server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:09.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6aeffd640 1 -- 172.21.15.131:0/2703053581 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c00c8070 con 0x55fdc81a1460 2024-09-17T02:50:09.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6aeffd640 1 -- 172.21.15.131:0/2703053581 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6c0004b40 con 0x55fdc81a1460 2024-09-17T02:50:09.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6aeffd640 1 -- 172.21.15.131:0/2703053581 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0004ef0 con 0x55fdc81a1460 2024-09-17T02:50:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 -- 172.21.15.131:0/2703053581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc81a1460 msgr2=0x55fdc86e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/2703053581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc81a1460 0x55fdc86e6b00 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00054c0 tx=0x7fb6c00b8000 comp rx=0 tx=0).stop 2024-09-17T02:50:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 -- 172.21.15.131:0/2703053581 shutdown_connections 2024-09-17T02:50:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/2703053581 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc857aca0 0x55fdc86e7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/2703053581 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc8591950 0x55fdc86e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/2703053581 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc81a1460 0x55fdc86e6b00 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 -- 172.21.15.131:0/2703053581 >> 172.21.15.131:0/2703053581 conn(0x55fdc8667d90 msgr2=0x55fdc8697690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 -- 172.21.15.131:0/2703053581 shutdown_connections 2024-09-17T02:50:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 -- 172.21.15.131:0/2703053581 wait complete. 2024-09-17T02:50:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 -- start start 2024-09-17T02:50:09.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 Processor -- start 2024-09-17T02:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc81a1460 0x55fdc86d5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc857aca0 0x55fdc86d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc86d5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fdc847e5f0 con 0x55fdc8591950 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fdc84e7d80 con 0x55fdc81a1460 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fdc848a520 con 0x55fdc857aca0 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6d2c33000 10 client.0 _ll_register_callbacks cb 0x55fdc848e830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6c9f05640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc86d5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6c9f05640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc86d5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50826/0 (socket says 172.21.15.131:50826) 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.204+0000 7fb6c9f05640 1 -- 172.21.15.131:0/3062075729 learned_addr learned my addr 172.21.15.131:0/3062075729 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6c9f05640 1 -- 172.21.15.131:0/3062075729 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc857aca0 msgr2=0x55fdc86d5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6c9f05640 1 --2- 172.21.15.131:0/3062075729 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc857aca0 0x55fdc86d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6c9f05640 1 -- 172.21.15.131:0/3062075729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc81a1460 msgr2=0x55fdc86d5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6c9f05640 1 --2- 172.21.15.131:0/3062075729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc81a1460 0x55fdc86d5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6c9f05640 1 -- 172.21.15.131:0/3062075729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc8482190 con 0x55fdc8591950 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6c9f05640 1 --2- 172.21.15.131:0/3062075729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc86d5ba0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40cca70 tx=0x7fb6bc006410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b76dd44c397a95d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:09.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3062075729 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc012470 con 0x55fdc8591950 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3062075729 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fb6bc0051c0 con 0x55fdc8591950 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3062075729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fdc847c110 con 0x55fdc8591950 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3062075729 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fb6bc0053c0 con 0x55fdc8591950 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 10 client.7998 fetch_fsmap learned FSMap version 86 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3062075729 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x55fdc8568f10 con 0x55fdc8591950 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3062075729 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55fdc847c110 con 0x55fdc8591950 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3062075729 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7fb6bc010040 con 0x55fdc8591950 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 10 client.7998 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 10 client.7998 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 20 client.7998 populate_metadata read hostname 'smithi131' 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3062075729 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55fdc81867d0 0x55fdc86d65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 4 client.7998 mds_command: new command op to 7860 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:50:09.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3062075729 --> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdc84f6a90 con 0x55fdc81867d0 2024-09-17T02:50:09.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6c9704640 1 --2- 172.21.15.131:0/3062075729 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55fdc81867d0 0x55fdc86d65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6a3fff640 10 client.7998 ms_handle_connect on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:09.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6c9704640 1 --2- 172.21.15.131:0/3062075729 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55fdc81867d0 0x55fdc86d65b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc6b7f9cb97830e server_cookie=4a3cdb9a99f8dce8 in_seq=0 out_seq=0 2024-09-17T02:50:09.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3062075729 <== mds.0 v2:172.21.15.131:6833/2930400858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fdc84f6a90 con 0x55fdc81867d0 2024-09-17T02:50:09.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6a3fff640 10 client.7998 handle_command_reply: tid=0 2024-09-17T02:50:09.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 client.7998 shutdown 2024-09-17T02:50:09.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3062075729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 msgr2=0x55fdc86d5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3062075729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc86d5ba0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40cca70 tx=0x7fb6bc006410 comp rx=0 tx=0).stop 2024-09-17T02:50:09.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3062075729 shutdown_connections 2024-09-17T02:50:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3062075729 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55fdc81867d0 0x55fdc86d65b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3062075729 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc86d5ba0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3062075729 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc857aca0 0x55fdc86d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6a3fff640 0 client.7998 ms_handle_reset on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3062075729 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc81a1460 0x55fdc86d5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3062075729 >> 172.21.15.131:0/3062075729 conn(0x55fdc8667d90 msgr2=0x55fdc869f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3062075729 shutdown_connections 2024-09-17T02:50:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3062075729 wait complete. 2024-09-17T02:50:09.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.208+0000 7fb6d2c33000 20 client.7998 trim_cache size 0 max 16384 2024-09-17T02:50:09.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6d2c33000 1 Processor -- start 2024-09-17T02:50:09.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6d2c33000 1 -- start start 2024-09-17T02:50:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc81a1460 0x55fdc86a4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc857aca0 0x55fdc86a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc8591950 0x55fdc86fd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4202440 con 0x55fdc857aca0 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4201660 con 0x55fdc81a1460 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb6c4150eb0 con 0x55fdc8591950 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c8f03640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc857aca0 0x55fdc86a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c8f03640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc857aca0 0x55fdc86a46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50834/0 (socket says 172.21.15.131:50834) 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c8f03640 1 -- 172.21.15.131:0/1089631052 learned_addr learned my addr 172.21.15.131:0/1089631052 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c9f05640 1 --2- 172.21.15.131:0/1089631052 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc8591950 0x55fdc86fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c9704640 1 --2- 172.21.15.131:0/1089631052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc81a1460 0x55fdc86a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c8f03640 1 -- 172.21.15.131:0/1089631052 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc8591950 msgr2=0x55fdc86fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c8f03640 1 --2- 172.21.15.131:0/1089631052 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc8591950 0x55fdc86fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c8f03640 1 -- 172.21.15.131:0/1089631052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc81a1460 msgr2=0x55fdc86a4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:09.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c8f03640 1 --2- 172.21.15.131:0/1089631052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc81a1460 0x55fdc86a4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c8f03640 1 -- 172.21.15.131:0/1089631052 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc853fdb0 con 0x55fdc857aca0 2024-09-17T02:50:09.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c9f05640 1 --2- 172.21.15.131:0/1089631052 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc8591950 0x55fdc86fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:09.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6c8f03640 1 --2- 172.21.15.131:0/1089631052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc857aca0 0x55fdc86a46d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fb6b405baa0 tx=0x7fb6b405e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd22f81e91176bc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:09.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6ae7fc640 1 -- 172.21.15.131:0/1089631052 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b405e830 con 0x55fdc857aca0 2024-09-17T02:50:09.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6ae7fc640 1 -- 172.21.15.131:0/1089631052 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6b40631e0 con 0x55fdc857aca0 2024-09-17T02:50:09.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.228+0000 7fb6ae7fc640 1 -- 172.21.15.131:0/1089631052 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b405dcf0 con 0x55fdc857aca0 2024-09-17T02:50:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 -- 172.21.15.131:0/1089631052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc857aca0 msgr2=0x55fdc86a46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/1089631052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc857aca0 0x55fdc86a46d0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fb6b405baa0 tx=0x7fb6b405e180 comp rx=0 tx=0).stop 2024-09-17T02:50:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 -- 172.21.15.131:0/1089631052 shutdown_connections 2024-09-17T02:50:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/1089631052 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc8591950 0x55fdc86fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/1089631052 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc857aca0 0x55fdc86a46d0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/1089631052 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc81a1460 0x55fdc86a4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 -- 172.21.15.131:0/1089631052 >> 172.21.15.131:0/1089631052 conn(0x55fdc8667d90 msgr2=0x55fdc86a3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 -- 172.21.15.131:0/1089631052 shutdown_connections 2024-09-17T02:50:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 -- 172.21.15.131:0/1089631052 wait complete. 2024-09-17T02:50:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 -- start start 2024-09-17T02:50:09.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 Processor -- start 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc81a1460 0x55fdc875c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc857aca0 0x55fdc875cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc875d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fdc84afee0 con 0x55fdc8591950 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4151500 con 0x55fdc857aca0 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fdc849d1e0 con 0x55fdc81a1460 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 10 client.0 _ll_register_callbacks cb 0x7fb6bc001540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c9f05640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc875d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c9f05640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc875d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50844/0 (socket says 172.21.15.131:50844) 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c9f05640 1 -- 172.21.15.131:0/3358921658 learned_addr learned my addr 172.21.15.131:0/3358921658 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c9704640 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc81a1460 0x55fdc875c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c8f03640 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc857aca0 0x55fdc875cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c9f05640 1 -- 172.21.15.131:0/3358921658 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc81a1460 msgr2=0x55fdc875c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c9f05640 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc81a1460 0x55fdc875c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c9f05640 1 -- 172.21.15.131:0/3358921658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc857aca0 msgr2=0x55fdc875cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c9f05640 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc857aca0 0x55fdc875cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c9f05640 1 -- 172.21.15.131:0/3358921658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc84aaab0 con 0x55fdc8591950 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c8f03640 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc857aca0 0x55fdc875cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6c9f05640 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc875d150 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc004bc0 tx=0x7fb6bc004bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65191e4713ee1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3358921658 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc02e060 con 0x55fdc8591950 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3358921658 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fb6bc029c60 con 0x55fdc8591950 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3358921658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6b40631e0 con 0x55fdc8591950 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3358921658 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fb6bc005220 con 0x55fdc8591950 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 10 client.8004 fetch_fsmap learned FSMap version 86 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.232+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3358921658 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x55fdc8489070 con 0x55fdc8591950 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3358921658 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6b40631e0 con 0x55fdc8591950 2024-09-17T02:50:09.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3358921658 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7fb6bc01f470 con 0x55fdc8591950 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 10 client.8004 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 10 client.8004 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 20 client.8004 populate_metadata read hostname 'smithi131' 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55fdc8174830 0x55fdc86fa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 4 client.8004 mds_command: new command op to 7860 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3358921658 --> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] -- command(tid 0: {"prefix": "status"}) -- 0x55fdc84f2540 con 0x55fdc8174830 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6c9704640 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55fdc8174830 0x55fdc86fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6a3fff640 10 client.8004 ms_handle_connect on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6c9704640 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55fdc8174830 0x55fdc86fa880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd0edf4fe295dff4 server_cookie=761af99c7188fe21 in_seq=0 out_seq=0 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6a3fff640 1 -- 172.21.15.131:0/3358921658 <== mds.0 v2:172.21.15.131:6833/2930400858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55fdc84f2540 con 0x55fdc8174830 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6a3fff640 10 client.8004 handle_command_reply: tid=0 2024-09-17T02:50:09.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 client.8004 shutdown 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3358921658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 msgr2=0x55fdc875d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc875d150 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc004bc0 tx=0x7fb6bc004bf0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3358921658 shutdown_connections 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55fdc8174830 0x55fdc86fa880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc8591950 0x55fdc875d150 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc857aca0 0x55fdc875cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6a3fff640 0 client.8004 ms_handle_reset on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 --2- 172.21.15.131:0/3358921658 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc81a1460 0x55fdc875c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3358921658 >> 172.21.15.131:0/3358921658 conn(0x55fdc8667d90 msgr2=0x55fdc86a4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3358921658 shutdown_connections 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 1 -- 172.21.15.131:0/3358921658 wait complete. 2024-09-17T02:50:09.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6d2c33000 20 client.8004 trim_cache size 0 max 16384 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 7860, 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.0121931660000003, 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 86, 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 107, 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 107, 2024-09-17T02:50:09.238 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 10.614942982000001 2024-09-17T02:50:09.239 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:50:09.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 -- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb6a8073230 msgr2=0x7fb6a8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb6a8073230 0x7fb6a8075740 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00ad000 tx=0x7fb6c001a030 comp rx=0 tx=0).stop 2024-09-17T02:50:09.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 -- 172.21.15.131:0/1652274534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41f72e0 msgr2=0x7fb6c41f76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41f72e0 0x7fb6c41f76f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400c500 tx=0x7fb6b400c9c0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 -- 172.21.15.131:0/1652274534 shutdown_connections 2024-09-17T02:50:09.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb6a8073230 0x7fb6a8075740 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb6c40cdbf0 0x7fb6c40ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6c41f72e0 0x7fb6c41f76f0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 --2- 172.21.15.131:0/1652274534 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb6b0003a40 0x7fb6c4158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 -- 172.21.15.131:0/1652274534 >> 172.21.15.131:0/1652274534 conn(0x7fb6c406eea0 msgr2=0x7fb6c40b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:09.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 -- 172.21.15.131:0/1652274534 shutdown_connections 2024-09-17T02:50:09.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.236+0000 7fb6cb968640 1 -- 172.21.15.131:0/1652274534 wait complete. 2024-09-17T02:50:09.253 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:50:09.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.404+0000 7fe00d676640 1 -- 172.21.15.131:0/3961073780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff0005260 msgr2=0x7fdff000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.404+0000 7fe00d676640 1 --2- 172.21.15.131:0/3961073780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff0005260 0x7fdff000b9d0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fdff800c2e0 tx=0x7fdff801cf60 comp rx=0 tx=0).stop 2024-09-17T02:50:09.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.404+0000 7fe00d676640 1 -- 172.21.15.131:0/3961073780 shutdown_connections 2024-09-17T02:50:09.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.404+0000 7fe00d676640 1 --2- 172.21.15.131:0/3961073780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdff0005260 0x7fdff000b9d0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.404+0000 7fe00d676640 1 -- 172.21.15.131:0/3961073780 >> 172.21.15.131:0/3961073780 conn(0x7fe00806eea0 msgr2=0x7fe00806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:09.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.404+0000 7fe00d676640 1 -- 172.21.15.131:0/3961073780 shutdown_connections 2024-09-17T02:50:09.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.404+0000 7fe00d676640 1 -- 172.21.15.131:0/3961073780 wait complete. 2024-09-17T02:50:09.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe00d676640 1 Processor -- start 2024-09-17T02:50:09.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe00d676640 1 -- start start 2024-09-17T02:50:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe00d676640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdff0005260 0x7fe0081e80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe00d676640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe008141990 0x7fe008141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe00d676640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0081422e0 0x7fe008143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe00d676640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe00806fb40 con 0x7fe0081422e0 2024-09-17T02:50:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe00d676640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe00806f9c0 con 0x7fe008141990 2024-09-17T02:50:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe00d676640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe00806fcc0 con 0x7fdff0005260 2024-09-17T02:50:09.406 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0081422e0 0x7fe008143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe0077fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0081422e0 0x7fe008143470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:50874/0 (socket says 172.21.15.131:50874) 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe0077fe640 1 -- 172.21.15.131:0/3239226114 learned_addr learned my addr 172.21.15.131:0/3239226114 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe0067fc640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe008141990 0x7fe008141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe006ffd640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdff0005260 0x7fe0081e80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe0077fe640 1 -- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdff0005260 msgr2=0x7fe0081e80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe0077fe640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdff0005260 0x7fe0081e80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe0077fe640 1 -- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe008141990 msgr2=0x7fe008141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe0077fe640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe008141990 0x7fe008141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe0077fe640 1 -- 172.21.15.131:0/3239226114 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0081414e0 con 0x7fe0081422e0 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe006ffd640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdff0005260 0x7fe0081e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe0077fe640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0081422e0 0x7fe008143470 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fdff801b660 tx=0x7fdff8006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fdfef7fe640 1 -- 172.21.15.131:0/3239226114 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8001520 con 0x7fe0081422e0 2024-09-17T02:50:09.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fdfef7fe640 1 -- 172.21.15.131:0/3239226114 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdff8003620 con 0x7fe0081422e0 2024-09-17T02:50:09.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe00d676640 1 -- 172.21.15.131:0/3239226114 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0081547d0 con 0x7fe0081422e0 2024-09-17T02:50:09.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fdfef7fe640 1 -- 172.21.15.131:0/3239226114 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8003a50 con 0x7fe0081422e0 2024-09-17T02:50:09.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.408+0000 7fe00d676640 1 -- 172.21.15.131:0/3239226114 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0081e8ee0 con 0x7fe0081422e0 2024-09-17T02:50:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.412+0000 7fe00d676640 1 -- 172.21.15.131:0/3239226114 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00806dea0 con 0x7fe0081422e0 2024-09-17T02:50:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.412+0000 7fdfef7fe640 1 -- 172.21.15.131:0/3239226114 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdff80b7070 con 0x7fe0081422e0 2024-09-17T02:50:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.412+0000 7fdfef7fe640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdff4073230 0x7fdff4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.412+0000 7fdfef7fe640 1 -- 172.21.15.131:0/3239226114 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fdff812dfe0 con 0x7fe0081422e0 2024-09-17T02:50:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.412+0000 7fe006ffd640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdff4073230 0x7fdff4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:09.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.412+0000 7fe006ffd640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdff4073230 0x7fdff4075740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00ab40 tx=0x7fdffc00a2f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:09.413 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.416+0000 7fdfef7fe640 1 -- 172.21.15.131:0/3239226114 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdff80be030 con 0x7fe0081422e0 2024-09-17T02:50:09.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.620+0000 7fe00d676640 1 -- 172.21.15.131:0/3239226114 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe00806e9d0 con 0x7fe0081422e0 2024-09-17T02:50:09.619 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fdfef7fe640 1 -- 172.21.15.131:0/3239226114 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) ==== 53+0+73 (secure 0 0 0) 0x7fdff8016d10 con 0x7fe0081422e0 2024-09-17T02:50:09.619 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:50:09.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 -- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdff4073230 msgr2=0x7fdff4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdff4073230 0x7fdff4075740 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00ab40 tx=0x7fdffc00a2f0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 -- 172.21.15.131:0/3239226114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0081422e0 msgr2=0x7fe008143470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:09.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0081422e0 0x7fe008143470 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fdff801b660 tx=0x7fdff8006ce0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 -- 172.21.15.131:0/3239226114 shutdown_connections 2024-09-17T02:50:09.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdff4073230 0x7fdff4075740 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe0081422e0 0x7fe008143470 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe008141990 0x7fe008141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 --2- 172.21.15.131:0/3239226114 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdff0005260 0x7fe0081e80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:09.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 -- 172.21.15.131:0/3239226114 >> 172.21.15.131:0/3239226114 conn(0x7fe00806eea0 msgr2=0x7fe0080cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:09.622 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 -- 172.21.15.131:0/3239226114 shutdown_connections 2024-09-17T02:50:09.623 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09.624+0000 7fe00d676640 1 -- 172.21.15.131:0/3239226114 wait complete. 2024-09-17T02:50:09.642 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:50:09.642 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:50:09.642 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:50:09.642 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:50:09.642 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:09.643 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:50:09.643 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:50:09.643 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:50:09.643 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:50:09.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:09.643 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:50:09.649 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:50:09.649 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:50:09.649 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:50:09.649 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:09.649 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:50:09.649 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:09.649 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:50:09.649 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41896sec preferred_lft 41896sec 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:50:09.650 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:09.651 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:50:09.651 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:09.651 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:50:09.651 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:09.651 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:50:09.651 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:09.651 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:50:09.651 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:09.651 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:50:09.652 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:09.652 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:50:09.652 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:09.652 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:50:09.652 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:09.652 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:50:09.652 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:09.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:09.652 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:50:09.701 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:50:09.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:09.702 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:50:09.754 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:50:09.754 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:50:09.754 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:50:09.754 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:50:09.754 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:50:09.754 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:50:09.755 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:50:09.755 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:50:09.755 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:50:09.755 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:50:09.755 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:50:09.755 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:50:09.755 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:50:09.755 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:50:09.755 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:50:09.755 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:50:09.756 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:50:09.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:50:09.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:50:09.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:09.879 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:50:09.884 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:50:09.884 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:50:09.884 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:50:09.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:09.884 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:50:09.938 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:50:09.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:50:09.938 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:09.983 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:50:09.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:50:09.983 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:10.031 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:50:10.089 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:50:10.136 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:50:10.137 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:50:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:50:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:50:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:50:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:50:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:50:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:50:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:50:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:50:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:50:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:50:10.138 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:50:10.139 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:50:10.139 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:50:10.139 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:50:10.139 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:50:10.139 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:10.139 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:10.139 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:10.139 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:10.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:10.140 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:50:10.187 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:50:10.187 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:50:10.232 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:50:10.249 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:50:10.295 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:50:10.296 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:50:10.296 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:50:10.296 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:50:10.296 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:50:10.296 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:50:10.296 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:50:10.296 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:50:10.296 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:50:10.297 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:50:10.297 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:50:10.297 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:50:10.297 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:50:10.297 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:50:10.297 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:50:10.298 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:50:10.298 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:50:10.298 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:50:10.298 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:50:10.298 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:50:10.298 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:50:10.298 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:50:10.299 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:50:10.299 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:50:10.299 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:50:10.299 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:50:10.299 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:50:10.299 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:50:10.299 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:50:10.299 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:10.299 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:10.300 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:10.300 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:10.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:10.301 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:50:10.339 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:50:10.344+0000 7f631d0d3580 -1 init, newargv = 0x7f6310002290 newargc=15 2024-09-17T02:50:10.339 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49135]: starting ceph client 2024-09-17T02:50:10.345 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49135]: starting fuse 2024-09-17T02:50:11.445 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:50:11.461 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:50:11.509 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:50:11.509 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:50:11.509 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:50:11.509 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:50:11.509 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:50:11.509 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:50:11.510 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:11.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:11.512 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:11.512 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:11.512 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:11.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:11.512 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:50:11.555 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:50:11.556 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:50:11.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:50:11.556 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:50:11.556 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:50:11.556 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:50:11.556 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:50:11.556 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:11.556 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:50:11.556 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:50:11.556 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:50:11.556 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:50:11.557 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:50:11.558 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:11.558 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:50:11.558 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:50:11.680 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.49135.asok 2024-09-17T02:50:11.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:50:11.691 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49135.asok 2024-09-17T02:50:11.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:11.692 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49135.asok status 2024-09-17T02:50:11.873 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:50:11.873 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:50:11.873 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:50:11.873 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:50:11.873 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:50:11.873 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:50:11.873 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:50:11.873 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "49135", 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8016, 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 8016 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:50:11.874 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 27113961 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 27113961 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.8016 192.168.0.1:0/27113961", 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/27113961", 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 86, 2024-09-17T02:50:11.875 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 107, 2024-09-17T02:50:11.876 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:50:11.876 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:50:11.876 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:50:11.876 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:50:11.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:11.887 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:50:11.887 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:11.893 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:50:11.894 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:11.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:11.894 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:11.948 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:50:11.948 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:50:11.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:11.996 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-17T02:50:12.126 INFO:teuthology.orchestra.run.smithi131.stderr:6+0 records in 2024-09-17T02:50:12.126 INFO:teuthology.orchestra.run.smithi131.stderr:6+0 records out 2024-09-17T02:50:12.126 INFO:teuthology.orchestra.run.smithi131.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0718474 s, 87.6 MB/s 2024-09-17T02:50:12.127 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:50:12.127 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:50:12.134 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:50:12.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.340+0000 7f4187bab640 1 -- 172.21.15.131:0/3196178866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4180155240 msgr2=0x7f4180073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:12.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.340+0000 7f4187bab640 1 --2- 172.21.15.131:0/3196178866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4180155240 0x7f4180073430 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f417c00aa10 tx=0x7f417c01dc00 comp rx=0 tx=0).stop 2024-09-17T02:50:12.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.340+0000 7f4187bab640 1 -- 172.21.15.131:0/3196178866 shutdown_connections 2024-09-17T02:50:12.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.340+0000 7f4187bab640 1 --2- 172.21.15.131:0/3196178866 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4180073970 0x7f418006dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.340+0000 7f4187bab640 1 --2- 172.21.15.131:0/3196178866 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4180155240 0x7f4180073430 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.340+0000 7f4187bab640 1 -- 172.21.15.131:0/3196178866 >> 172.21.15.131:0/3196178866 conn(0x7f418006eea0 msgr2=0x7f418006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:12.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.340+0000 7f4187bab640 1 -- 172.21.15.131:0/3196178866 shutdown_connections 2024-09-17T02:50:12.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.340+0000 7f4187bab640 1 -- 172.21.15.131:0/3196178866 wait complete. 2024-09-17T02:50:12.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.340+0000 7f4187bab640 1 Processor -- start 2024-09-17T02:50:12.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.340+0000 7f4187bab640 1 -- start start 2024-09-17T02:50:12.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4187bab640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f418006e3e0 0x7f4180157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:12.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4187bab640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4180073970 0x7f41801531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:12.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4187bab640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180155240 0x7f4180154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:12.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4187bab640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4180167f00 con 0x7f4180073970 2024-09-17T02:50:12.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4187bab640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4180167d80 con 0x7f4180155240 2024-09-17T02:50:12.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4187bab640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4180168080 con 0x7f418006e3e0 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4185146640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4180073970 0x7f41801531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4185146640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4180073970 0x7f41801531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41812/0 (socket says 172.21.15.131:41812) 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4185146640 1 -- 172.21.15.131:0/688854655 learned_addr learned my addr 172.21.15.131:0/688854655 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4185947640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f418006e3e0 0x7f4180157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4186148640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180155240 0x7f4180154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4185146640 1 -- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f418006e3e0 msgr2=0x7f4180157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4185146640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f418006e3e0 0x7f4180157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4185146640 1 -- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180155240 msgr2=0x7f4180154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4185146640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180155240 0x7f4180154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4185146640 1 -- 172.21.15.131:0/688854655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41801f8f50 con 0x7f4180073970 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4185146640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4180073970 0x7f41801531b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f417000d720 tx=0x7f417000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f41767fc640 1 -- 172.21.15.131:0/688854655 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170017070 con 0x7f4180073970 2024-09-17T02:50:12.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f41767fc640 1 -- 172.21.15.131:0/688854655 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4170009d20 con 0x7f4180073970 2024-09-17T02:50:12.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f41767fc640 1 -- 172.21.15.131:0/688854655 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4170005280 con 0x7f4180073970 2024-09-17T02:50:12.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4187bab640 1 -- 172.21.15.131:0/688854655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41801f9230 con 0x7f4180073970 2024-09-17T02:50:12.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4187bab640 1 -- 172.21.15.131:0/688854655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41801f97f0 con 0x7f4180073970 2024-09-17T02:50:12.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.344+0000 7f4187bab640 1 -- 172.21.15.131:0/688854655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41801f8cc0 con 0x7f4180073970 2024-09-17T02:50:12.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.348+0000 7f41767fc640 1 -- 172.21.15.131:0/688854655 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4170005420 con 0x7f4180073970 2024-09-17T02:50:12.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.348+0000 7f41767fc640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4164073230 0x7f4164075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:12.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.348+0000 7f41767fc640 1 -- 172.21.15.131:0/688854655 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f4170094500 con 0x7f4180073970 2024-09-17T02:50:12.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.348+0000 7f4185947640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4164073230 0x7f4164075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:12.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.348+0000 7f4185947640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4164073230 0x7f4164075740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f417c0b0000 tx=0x7f417c01c440 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:12.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.352+0000 7f41767fc640 1 -- 172.21.15.131:0/688854655 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f417005f3a0 con 0x7f4180073970 2024-09-17T02:50:12.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.532+0000 7f4187bab640 1 -- 172.21.15.131:0/688854655 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f418006dea0 con 0x7f4180073970 2024-09-17T02:50:12.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.532+0000 7f41767fc640 1 -- 172.21.15.131:0/688854655 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) ==== 80+0+0 (secure 0 0 0) 0x7f417005eca0 con 0x7f4180073970 2024-09-17T02:50:12.530 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:50:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 -- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4164073230 msgr2=0x7f4164075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4164073230 0x7f4164075740 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f417c0b0000 tx=0x7f417c01c440 comp rx=0 tx=0).stop 2024-09-17T02:50:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 -- 172.21.15.131:0/688854655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4180073970 msgr2=0x7f41801531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4180073970 0x7f41801531b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f417000d720 tx=0x7f417000dbe0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 -- 172.21.15.131:0/688854655 shutdown_connections 2024-09-17T02:50:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4164073230 0x7f4164075740 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4180155240 0x7f4180154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4180073970 0x7f41801531b0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 --2- 172.21.15.131:0/688854655 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f418006e3e0 0x7f4180157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 -- 172.21.15.131:0/688854655 >> 172.21.15.131:0/688854655 conn(0x7f418006eea0 msgr2=0x7f41800b57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:12.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 -- 172.21.15.131:0/688854655 shutdown_connections 2024-09-17T02:50:12.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.536+0000 7f4187bab640 1 -- 172.21.15.131:0/688854655 wait complete. 2024-09-17T02:50:12.551 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:50:12.551 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:50:12.557 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:50:12.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/2379196762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0a40cec00 msgr2=0x7ff0a40cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:12.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 --2- 172.21.15.131:0/2379196762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0a40cec00 0x7ff0a40cf010 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7ff0a406a1c0 tx=0x7ff09c01c770 comp rx=0 tx=0).stop 2024-09-17T02:50:12.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/2379196762 shutdown_connections 2024-09-17T02:50:12.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 --2- 172.21.15.131:0/2379196762 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0a40cec00 0x7ff0a40cf010 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 --2- 172.21.15.131:0/2379196762 >> v2:172.21.15.189:3300/0 conn(0x7ff0a406e2c0 0x7ff0a40c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/2379196762 >> 172.21.15.131:0/2379196762 conn(0x7ff0a406eea0 msgr2=0x7ff0a406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:12.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/2379196762 shutdown_connections 2024-09-17T02:50:12.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/2379196762 wait complete. 2024-09-17T02:50:12.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 Processor -- start 2024-09-17T02:50:12.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 -- start start 2024-09-17T02:50:12.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0a406e2c0 0x7ff0a41fc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:12.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0a40c4ac0 0x7ff0a414e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:12.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0a40cec00 0x7ff0a40b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:12.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4169c60 con 0x7ff0a406e2c0 2024-09-17T02:50:12.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4169ae0 con 0x7ff0a40c4ac0 2024-09-17T02:50:12.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.764+0000 7ff0ac1e5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff0a4169de0 con 0x7ff0a40cec00 2024-09-17T02:50:12.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9f81640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0a406e2c0 0x7ff0a41fc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:12.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9f81640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0a406e2c0 0x7ff0a41fc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41828/0 (socket says 172.21.15.131:41828) 2024-09-17T02:50:12.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9f81640 1 -- 172.21.15.131:0/821966242 learned_addr learned my addr 172.21.15.131:0/821966242 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0aa782640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0a40cec00 0x7ff0a40b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9780640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0a40c4ac0 0x7ff0a414e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9f81640 1 -- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0a40cec00 msgr2=0x7ff0a40b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9f81640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0a40cec00 0x7ff0a40b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9f81640 1 -- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0a40c4ac0 msgr2=0x7ff0a414e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9f81640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0a40c4ac0 0x7ff0a414e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9f81640 1 -- 172.21.15.131:0/821966242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a41fccf0 con 0x7ff0a406e2c0 2024-09-17T02:50:12.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9780640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0a40c4ac0 0x7ff0a414e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:12.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9f81640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0a406e2c0 0x7ff0a41fc7b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7ff09400d720 tx=0x7ff09400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:12.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff09a7fc640 1 -- 172.21.15.131:0/821966242 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094017070 con 0x7ff0a406e2c0 2024-09-17T02:50:12.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff09a7fc640 1 -- 172.21.15.131:0/821966242 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff094009d20 con 0x7ff0a406e2c0 2024-09-17T02:50:12.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff09a7fc640 1 -- 172.21.15.131:0/821966242 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094005280 con 0x7ff0a406e2c0 2024-09-17T02:50:12.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/821966242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a40cba00 con 0x7ff0a406e2c0 2024-09-17T02:50:12.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/821966242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a40cbe70 con 0x7ff0a406e2c0 2024-09-17T02:50:12.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/821966242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a40cb070 con 0x7ff0a406e2c0 2024-09-17T02:50:12.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff09a7fc640 1 -- 172.21.15.131:0/821966242 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff094005420 con 0x7ff0a406e2c0 2024-09-17T02:50:12.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff09a7fc640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff088073200 0x7ff088075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:12.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff09a7fc640 1 -- 172.21.15.131:0/821966242 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7ff094094560 con 0x7ff0a406e2c0 2024-09-17T02:50:12.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.768+0000 7ff0a9780640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff088073200 0x7ff088075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:12.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.772+0000 7ff0a9780640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff088073200 0x7ff088075710 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00015b0 tx=0x7ff0a0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:12.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.772+0000 7ff09a7fc640 1 -- 172.21.15.131:0/821966242 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff0a41fccf0 con 0x7ff0a406e2c0 2024-09-17T02:50:12.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/821966242 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff0a4073310 con 0x7ff0a406e2c0 2024-09-17T02:50:12.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff09a7fc640 1 -- 172.21.15.131:0/821966242 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) ==== 80+0+0 (secure 0 0 0) 0x7ff09405f400 con 0x7ff0a406e2c0 2024-09-17T02:50:12.951 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:50:12.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff088073200 msgr2=0x7ff088075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:12.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff088073200 0x7ff088075710 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00015b0 tx=0x7ff0a0009040 comp rx=0 tx=0).stop 2024-09-17T02:50:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/821966242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0a406e2c0 msgr2=0x7ff0a41fc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0a406e2c0 0x7ff0a41fc7b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7ff09400d720 tx=0x7ff09400dbe0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/821966242 shutdown_connections 2024-09-17T02:50:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff088073200 0x7ff088075710 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff0a40cec00 0x7ff0a40b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff0a40c4ac0 0x7ff0a414e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 --2- 172.21.15.131:0/821966242 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff0a406e2c0 0x7ff0a41fc7b0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/821966242 >> 172.21.15.131:0/821966242 conn(0x7ff0a406eea0 msgr2=0x7ff0a414ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/821966242 shutdown_connections 2024-09-17T02:50:12.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:12.956+0000 7ff0ac1e5640 1 -- 172.21.15.131:0/821966242 wait complete. 2024-09-17T02:50:12.973 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:50:12.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:12.973 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:12.982 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49135]: fuse finished with error 0 and tester_r 0 2024-09-17T02:50:12.984 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:50:15.387 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:50:15.390 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:50:19.086 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:50:19.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:19.087 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:50:19.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:50:19.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.312+0000 7f21e22cf640 1 -- 172.21.15.131:0/3996653768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21dc1566f0 msgr2=0x7f21dc156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.312+0000 7f21e22cf640 1 --2- 172.21.15.131:0/3996653768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21dc1566f0 0x7f21dc156b00 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0720e0 tx=0x7f21d001c650 comp rx=0 tx=0).stop 2024-09-17T02:50:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.312+0000 7f21e22cf640 1 -- 172.21.15.131:0/3996653768 shutdown_connections 2024-09-17T02:50:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.312+0000 7f21e22cf640 1 --2- 172.21.15.131:0/3996653768 >> v2:172.21.15.189:3300/0 conn(0x7f21dc0cc750 0x7f21dc0ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.312+0000 7f21e22cf640 1 --2- 172.21.15.131:0/3996653768 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21dc1566f0 0x7f21dc156b00 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.312+0000 7f21e22cf640 1 -- 172.21.15.131:0/3996653768 >> 172.21.15.131:0/3996653768 conn(0x7f21dc0d2100 msgr2=0x7f21dc0d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.312+0000 7f21e22cf640 1 -- 172.21.15.131:0/3996653768 shutdown_connections 2024-09-17T02:50:19.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.312+0000 7f21e22cf640 1 -- 172.21.15.131:0/3996653768 wait complete. 2024-09-17T02:50:19.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e22cf640 1 Processor -- start 2024-09-17T02:50:19.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e22cf640 1 -- start start 2024-09-17T02:50:19.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e22cf640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21dc0cc750 0x7f21dc14eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:19.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e22cf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21dc1566f0 0x7f21dc0c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:19.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e22cf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21dc15cfa0 0x7f21dc0ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:19.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e22cf640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc0716b0 con 0x7f21dc1566f0 2024-09-17T02:50:19.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e22cf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc071530 con 0x7f21dc15cfa0 2024-09-17T02:50:19.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e22cf640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f21dc071830 con 0x7f21dc0cc750 2024-09-17T02:50:19.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21db7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21dc1566f0 0x7f21dc0c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:19.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21db7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21dc1566f0 0x7f21dc0c9b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41846/0 (socket says 172.21.15.131:41846) 2024-09-17T02:50:19.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21db7fe640 1 -- 172.21.15.131:0/4256201835 learned_addr learned my addr 172.21.15.131:0/4256201835 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:19.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21dbfff640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21dc0cc750 0x7f21dc14eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:19.315 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e086c640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21dc15cfa0 0x7f21dc0ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21db7fe640 1 -- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21dc0cc750 msgr2=0x7f21dc14eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21db7fe640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21dc0cc750 0x7f21dc14eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21db7fe640 1 -- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21dc15cfa0 msgr2=0x7f21dc0ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21db7fe640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21dc15cfa0 0x7f21dc0ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21db7fe640 1 -- 172.21.15.131:0/4256201835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc0b7b30 con 0x7f21dc1566f0 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e086c640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21dc15cfa0 0x7f21dc0ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21db7fe640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21dc1566f0 0x7f21dc0c9b90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c9a0 tx=0x7f21cc00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21d8ff9640 1 -- 172.21.15.131:0/4256201835 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc007bb0 con 0x7f21dc1566f0 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21d8ff9640 1 -- 172.21.15.131:0/4256201835 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21cc007e40 con 0x7f21dc1566f0 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e22cf640 1 -- 172.21.15.131:0/4256201835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc0ca610 con 0x7f21dc1566f0 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21d8ff9640 1 -- 172.21.15.131:0/4256201835 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc005280 con 0x7f21dc1566f0 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e22cf640 1 -- 172.21.15.131:0/4256201835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc0ca860 con 0x7f21dc1566f0 2024-09-17T02:50:19.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.316+0000 7f21e22cf640 1 -- 172.21.15.131:0/4256201835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21dc0ccb60 con 0x7f21dc1566f0 2024-09-17T02:50:19.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.320+0000 7f21d8ff9640 1 -- 172.21.15.131:0/4256201835 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f21cc005420 con 0x7f21dc1566f0 2024-09-17T02:50:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.320+0000 7f21d8ff9640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21c8073200 0x7f21c8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.320+0000 7f21d8ff9640 1 -- 172.21.15.131:0/4256201835 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f21cc094c20 con 0x7f21dc1566f0 2024-09-17T02:50:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.320+0000 7f21dbfff640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21c8073200 0x7f21c8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:19.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.320+0000 7f21dbfff640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21c8073200 0x7f21c8075710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f21d00ae000 tx=0x7f21d001ae90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:19.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.324+0000 7f21d8ff9640 1 -- 172.21.15.131:0/4256201835 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f21dc0b7b30 con 0x7f21dc1566f0 2024-09-17T02:50:19.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 -- 172.21.15.131:0/4256201835 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21dc0b1b70 con 0x7f21dc1566f0 2024-09-17T02:50:19.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21d8ff9640 1 -- 172.21.15.131:0/4256201835 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5237 (secure 0 0 0) 0x7f21cc094030 con 0x7f21dc1566f0 2024-09-17T02:50:19.528 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:19.528 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":86,"btime":"2024-09-17T02:50:06:247461+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:50:05.662097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7860":{"gid":7860,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2930400858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2930400858},{"type":"v1","addr":"172.21.15.131:6835","nonce":2930400858}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7860,"qdb_cluster":[7860]},"id":7}]} 2024-09-17T02:50:19.529 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 86 2024-09-17T02:50:19.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 -- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21c8073200 msgr2=0x7f21c8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:19.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21c8073200 0x7f21c8075710 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f21d00ae000 tx=0x7f21d001ae90 comp rx=0 tx=0).stop 2024-09-17T02:50:19.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 -- 172.21.15.131:0/4256201835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21dc1566f0 msgr2=0x7f21dc0c9b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:19.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21dc1566f0 0x7f21dc0c9b90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c9a0 tx=0x7f21cc00ce60 comp rx=0 tx=0).stop 2024-09-17T02:50:19.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 -- 172.21.15.131:0/4256201835 shutdown_connections 2024-09-17T02:50:19.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21c8073200 0x7f21c8075710 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:19.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21dc15cfa0 0x7f21dc0ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:19.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21dc1566f0 0x7f21dc0c9b90 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:19.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 --2- 172.21.15.131:0/4256201835 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21dc0cc750 0x7f21dc14eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:19.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 -- 172.21.15.131:0/4256201835 >> 172.21.15.131:0/4256201835 conn(0x7f21dc0d2100 msgr2=0x7f21dc15d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:19.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 -- 172.21.15.131:0/4256201835 shutdown_connections 2024-09-17T02:50:19.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.532+0000 7f21e22cf640 1 -- 172.21.15.131:0/4256201835 wait complete. 2024-09-17T02:50:19.552 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-17T02:50:19.729 INFO:teuthology.orchestra.run.smithi131.stdout:{"message":"","return_code":0} 2024-09-17T02:50:19.729 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T02:50:19.730 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:50:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.888+0000 7f8c181ad640 1 -- 172.21.15.131:0/3690750194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bfc003a60 msgr2=0x7f8bfc003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.888+0000 7f8c181ad640 1 --2- 172.21.15.131:0/3690750194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bfc003a60 0x7f8bfc003ed0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009b60 tx=0x7f8c0c01d270 comp rx=0 tx=0).stop 2024-09-17T02:50:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.888+0000 7f8c181ad640 1 -- 172.21.15.131:0/3690750194 shutdown_connections 2024-09-17T02:50:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.888+0000 7f8c181ad640 1 --2- 172.21.15.131:0/3690750194 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bfc003a60 0x7f8bfc003ed0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.888+0000 7f8c181ad640 1 -- 172.21.15.131:0/3690750194 >> 172.21.15.131:0/3690750194 conn(0x7f8c100d0830 msgr2=0x7f8c100d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.888+0000 7f8c181ad640 1 -- 172.21.15.131:0/3690750194 shutdown_connections 2024-09-17T02:50:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.888+0000 7f8c181ad640 1 -- 172.21.15.131:0/3690750194 wait complete. 2024-09-17T02:50:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c181ad640 1 Processor -- start 2024-09-17T02:50:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c181ad640 1 -- start start 2024-09-17T02:50:19.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c181ad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bfc003a60 0x7f8c10152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:19.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c181ad640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c10154fe0 0x7f8c10157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:19.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c181ad640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c10158270 0x7f8c10155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:19.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c181ad640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8c100b3f60 con 0x7f8bfc003a60 2024-09-17T02:50:19.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c181ad640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8c100b3de0 con 0x7f8c10154fe0 2024-09-17T02:50:19.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c181ad640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8c100b40e0 con 0x7f8c10158270 2024-09-17T02:50:19.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c15f49640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bfc003a60 0x7f8c10152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:19.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c15f49640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bfc003a60 0x7f8c10152440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41876/0 (socket says 172.21.15.131:41876) 2024-09-17T02:50:19.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c15f49640 1 -- 172.21.15.131:0/3520164184 learned_addr learned my addr 172.21.15.131:0/3520164184 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:19.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c15748640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c10154fe0 0x7f8c10157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:19.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c1674a640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c10158270 0x7f8c10155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:19.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c15f49640 1 -- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c10158270 msgr2=0x7f8c10155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:19.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c15f49640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c10158270 0x7f8c10155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:19.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c15f49640 1 -- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c10154fe0 msgr2=0x7f8c10157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:19.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c15f49640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c10154fe0 0x7f8c10157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:19.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c15f49640 1 -- 172.21.15.131:0/3520164184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c100d0ec0 con 0x7f8bfc003a60 2024-09-17T02:50:19.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c15f49640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bfc003a60 0x7f8c10152440 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0af040 tx=0x7f8c0c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:19.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c067fc640 1 -- 172.21.15.131:0/3520164184 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0b1040 con 0x7f8bfc003a60 2024-09-17T02:50:19.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c067fc640 1 -- 172.21.15.131:0/3520164184 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c0c0acaa0 con 0x7f8bfc003a60 2024-09-17T02:50:19.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c067fc640 1 -- 172.21.15.131:0/3520164184 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0046d0 con 0x7f8bfc003a60 2024-09-17T02:50:19.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c181ad640 1 -- 172.21.15.131:0/3520164184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c10158680 con 0x7f8bfc003a60 2024-09-17T02:50:19.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.892+0000 7f8c181ad640 1 -- 172.21.15.131:0/3520164184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c1015dc60 con 0x7f8bfc003a60 2024-09-17T02:50:19.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.896+0000 7f8c181ad640 1 -- 172.21.15.131:0/3520164184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c1006e610 con 0x7f8bfc003a60 2024-09-17T02:50:19.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.896+0000 7f8c067fc640 1 -- 172.21.15.131:0/3520164184 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8c0c0bf060 con 0x7f8bfc003a60 2024-09-17T02:50:19.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.896+0000 7f8c067fc640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bf4073200 0x7f8bf4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:19.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.896+0000 7f8c067fc640 1 -- 172.21.15.131:0/3520164184 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f8c0c12e510 con 0x7f8bfc003a60 2024-09-17T02:50:19.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.896+0000 7f8c15748640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bf4073200 0x7f8bf4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:19.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.896+0000 7f8c15748640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bf4073200 0x7f8bf4075710 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8c00001870 tx=0x7f8c00001bd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:19.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:19.900+0000 7f8c067fc640 1 -- 172.21.15.131:0/3520164184 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8c10158680 con 0x7f8bfc003a60 2024-09-17T02:50:20.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.100+0000 7f8c181ad640 1 -- 172.21.15.131:0/3520164184 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c10160900 con 0x7f8bfc003a60 2024-09-17T02:50:20.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.100+0000 7f8c067fc640 1 -- 172.21.15.131:0/3520164184 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5237 (secure 0 0 0) 0x7f8c0c016f80 con 0x7f8bfc003a60 2024-09-17T02:50:20.097 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:20.097 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":86,"btime":"2024-09-17T02:50:06:247461+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:50:05.662097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7860":{"gid":7860,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2930400858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2930400858},{"type":"v1","addr":"172.21.15.131:6835","nonce":2930400858}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7860,"qdb_cluster":[7860]},"id":7}]} 2024-09-17T02:50:20.097 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 86 2024-09-17T02:50:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.100+0000 7f8c181ad640 1 -- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bf4073200 msgr2=0x7f8bf4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.100+0000 7f8c181ad640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bf4073200 0x7f8bf4075710 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8c00001870 tx=0x7f8c00001bd0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.104+0000 7f8c181ad640 1 -- 172.21.15.131:0/3520164184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bfc003a60 msgr2=0x7f8c10152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.104+0000 7f8c181ad640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bfc003a60 0x7f8c10152440 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0af040 tx=0x7f8c0c0056a0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.104+0000 7f8c181ad640 1 -- 172.21.15.131:0/3520164184 shutdown_connections 2024-09-17T02:50:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.104+0000 7f8c181ad640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8bf4073200 0x7f8bf4075710 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.104+0000 7f8c181ad640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c10158270 0x7f8c10155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.104+0000 7f8c181ad640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c10154fe0 0x7f8c10157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.104+0000 7f8c181ad640 1 --2- 172.21.15.131:0/3520164184 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8bfc003a60 0x7f8c10152440 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.104+0000 7f8c181ad640 1 -- 172.21.15.131:0/3520164184 >> 172.21.15.131:0/3520164184 conn(0x7f8c100d0830 msgr2=0x7f8c1015a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.104+0000 7f8c181ad640 1 -- 172.21.15.131:0/3520164184 shutdown_connections 2024-09-17T02:50:20.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.104+0000 7f8c181ad640 1 -- 172.21.15.131:0/3520164184 wait complete. 2024-09-17T02:50:20.118 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-09-17T02:50:20.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.184+0000 7ff333b8f080 1 Processor -- start 2024-09-17T02:50:20.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.184+0000 7ff333b8f080 1 -- start start 2024-09-17T02:50:20.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585eec55360 0x5585eedaca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5585eec9c890 0x5585eedb3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 0x5585eec98a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5585eed27960 con 0x5585eed26ed0 2024-09-17T02:50:20.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5585eeba1a00 con 0x5585eec55360 2024-09-17T02:50:20.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5585eec9ae90 con 0x5585eec9c890 2024-09-17T02:50:20.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff32de67640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 0x5585eec98a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff32de67640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 0x5585eec98a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41900/0 (socket says 172.21.15.131:41900) 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff32de67640 1 -- 172.21.15.131:0/2610497152 learned_addr learned my addr 172.21.15.131:0/2610497152 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff32ce65640 1 --2- 172.21.15.131:0/2610497152 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5585eec9c890 0x5585eedb3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff32d666640 1 --2- 172.21.15.131:0/2610497152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585eec55360 0x5585eedaca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff32de67640 1 -- 172.21.15.131:0/2610497152 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5585eec9c890 msgr2=0x5585eedb3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff32de67640 1 --2- 172.21.15.131:0/2610497152 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5585eec9c890 0x5585eedb3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff32de67640 1 -- 172.21.15.131:0/2610497152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585eec55360 msgr2=0x5585eedaca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff32de67640 1 --2- 172.21.15.131:0/2610497152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585eec55360 0x5585eedaca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff32de67640 1 -- 172.21.15.131:0/2610497152 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585eeba0a20 con 0x5585eed26ed0 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff32de67640 1 --2- 172.21.15.131:0/2610497152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 0x5585eec98a70 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff32800b500 tx=0x7ff328006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be8c3828d559faf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff3277fe640 1 -- 172.21.15.131:0/2610497152 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328018070 con 0x5585eed26ed0 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff3277fe640 1 -- 172.21.15.131:0/2610497152 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff328009700 con 0x5585eed26ed0 2024-09-17T02:50:20.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff3277fe640 1 -- 172.21.15.131:0/2610497152 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328009b70 con 0x5585eed26ed0 2024-09-17T02:50:20.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 -- 172.21.15.131:0/2610497152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 msgr2=0x5585eec98a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 --2- 172.21.15.131:0/2610497152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 0x5585eec98a70 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7ff32800b500 tx=0x7ff328006f90 comp rx=0 tx=0).stop 2024-09-17T02:50:20.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 -- 172.21.15.131:0/2610497152 shutdown_connections 2024-09-17T02:50:20.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 --2- 172.21.15.131:0/2610497152 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 0x5585eec98a70 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 --2- 172.21.15.131:0/2610497152 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5585eec9c890 0x5585eedb3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 --2- 172.21.15.131:0/2610497152 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585eec55360 0x5585eedaca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 -- 172.21.15.131:0/2610497152 >> 172.21.15.131:0/2610497152 conn(0x5585eec3fd00 msgr2=0x5585eec9b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:20.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 -- 172.21.15.131:0/2610497152 shutdown_connections 2024-09-17T02:50:20.187 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 -- 172.21.15.131:0/2610497152 wait complete. 2024-09-17T02:50:20.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 Processor -- start 2024-09-17T02:50:20.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.188+0000 7ff333b8f080 1 -- start start 2024-09-17T02:50:20.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff333b8f080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5585eec55360 0x5585eed12b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff333b8f080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585eec9c890 0x5585eedbedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff333b8f080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 0x5585eedb3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff333b8f080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5585eec95220 con 0x5585eed26ed0 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff333b8f080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5585eec94f80 con 0x5585eec9c890 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff333b8f080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5585eec99e70 con 0x5585eec55360 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32de67640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 0x5585eedb3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32de67640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 0x5585eedb3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41916/0 (socket says 172.21.15.131:41916) 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32de67640 1 -- 172.21.15.131:0/246887584 learned_addr learned my addr 172.21.15.131:0/246887584 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32de67640 1 -- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5585eec55360 msgr2=0x5585eed12b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32d666640 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5585eec55360 0x5585eed12b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32ce65640 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585eec9c890 0x5585eedbedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32de67640 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5585eec55360 0x5585eed12b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32de67640 1 -- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585eec9c890 msgr2=0x5585eedbedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32de67640 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5585eec9c890 0x5585eedbedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32de67640 1 -- 172.21.15.131:0/246887584 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585eeb7ac10 con 0x5585eed26ed0 2024-09-17T02:50:20.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32de67640 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 0x5585eedb3d00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff328013000 tx=0x7ff328013640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff3267fc640 1 -- 172.21.15.131:0/246887584 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328018040 con 0x5585eed26ed0 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff3267fc640 1 -- 172.21.15.131:0/246887584 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff328021070 con 0x5585eed26ed0 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff333b8f080 1 -- 172.21.15.131:0/246887584 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5585eedbf6c0 con 0x5585eed26ed0 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff3267fc640 1 -- 172.21.15.131:0/246887584 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328013be0 con 0x5585eed26ed0 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff333b8f080 1 -- 172.21.15.131:0/246887584 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585eedbfc00 con 0x5585eed26ed0 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff3267fc640 1 -- 172.21.15.131:0/246887584 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff328026460 con 0x5585eed26ed0 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff3267fc640 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff310071c10 0x7ff310074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff3267fc640 1 -- 172.21.15.131:0/246887584 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7ff328094470 con 0x5585eed26ed0 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff333b8f080 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x5585eedc0af0 0x5585eedc0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff32d666640 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff310071c10 0x7ff310074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.192+0000 7ff333b8f080 1 -- 172.21.15.131:0/246887584 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107) -- 0x5585eed143b0 con 0x5585eedc0af0 2024-09-17T02:50:20.192 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff32ce65640 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x5585eedc0af0 0x5585eedc0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff32d666640 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff310071c10 0x7ff310074120 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff320004540 tx=0x7ff3200044d0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:20.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff32ce65640 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x5585eedc0af0 0x5585eedc0f00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:20.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff32ce65640 1 -- 172.21.15.131:0/246887584 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7ff3180a76e0 con 0x5585eedc0af0 2024-09-17T02:50:20.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff333b8f080 1 -- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x5585eedc0af0 msgr2=0x5585eedc0f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff333b8f080 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x5585eedc0af0 0x5585eedc0f00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff333b8f080 1 -- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff310071c10 msgr2=0x7ff310074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff333b8f080 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff310071c10 0x7ff310074120 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff320004540 tx=0x7ff3200044d0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff333b8f080 1 -- 172.21.15.131:0/246887584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 msgr2=0x5585eedb3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff333b8f080 1 --2- 172.21.15.131:0/246887584 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5585eed26ed0 0x5585eedb3d00 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7ff328013000 tx=0x7ff328013640 comp rx=0 tx=0).stop 2024-09-17T02:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff32de67640 1 -- 172.21.15.131:0/246887584 reap_dead start 2024-09-17T02:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff333b8f080 1 -- 172.21.15.131:0/246887584 shutdown_connections 2024-09-17T02:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff333b8f080 1 -- 172.21.15.131:0/246887584 >> 172.21.15.131:0/246887584 conn(0x5585eec3fd00 msgr2=0x5585eec96f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff333b8f080 1 -- 172.21.15.131:0/246887584 shutdown_connections 2024-09-17T02:50:20.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.196+0000 7ff333b8f080 1 -- 172.21.15.131:0/246887584 wait complete. 2024-09-17T02:50:20.198 INFO:teuthology.orchestra.run.smithi131.stdout:����� 2024-09-17T02:50:20.199 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:50:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.344+0000 7f0173d8d640 1 -- 172.21.15.131:0/2465858612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016c0cf710 msgr2=0x7f016c15f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.344+0000 7f0173d8d640 1 --2- 172.21.15.131:0/2465858612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016c0cf710 0x7f016c15f290 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f0160009900 tx=0x7f016001c8e0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 -- 172.21.15.131:0/2465858612 shutdown_connections 2024-09-17T02:50:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 --2- 172.21.15.131:0/2465858612 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f016c15f7d0 0x7f016c162410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 --2- 172.21.15.131:0/2465858612 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016c0cf710 0x7f016c15f290 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 -- 172.21.15.131:0/2465858612 >> 172.21.15.131:0/2465858612 conn(0x7f016c0d1b10 msgr2=0x7f016c0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 -- 172.21.15.131:0/2465858612 shutdown_connections 2024-09-17T02:50:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 -- 172.21.15.131:0/2465858612 wait complete. 2024-09-17T02:50:20.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 Processor -- start 2024-09-17T02:50:20.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 -- start start 2024-09-17T02:50:20.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f016c0cf710 0x7f016c0c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016c15f7d0 0x7f016c151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f016c162950 0x7f016c14f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f016c0d3500 con 0x7f016c15f7d0 2024-09-17T02:50:20.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f016c0d3380 con 0x7f016c162950 2024-09-17T02:50:20.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0173d8d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f016c0d3680 con 0x7f016c0cf710 2024-09-17T02:50:20.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0171328640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016c15f7d0 0x7f016c151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0171328640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016c15f7d0 0x7f016c151320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41948/0 (socket says 172.21.15.131:41948) 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0171328640 1 -- 172.21.15.131:0/1066111357 learned_addr learned my addr 172.21.15.131:0/1066111357 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0171b29640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f016c0cf710 0x7f016c0c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f017232a640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f016c162950 0x7f016c14f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0171328640 1 -- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f016c0cf710 msgr2=0x7f016c0c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0171328640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f016c0cf710 0x7f016c0c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0171328640 1 -- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f016c162950 msgr2=0x7f016c14f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0171328640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f016c162950 0x7f016c14f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0171328640 1 -- 172.21.15.131:0/1066111357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c06a820 con 0x7f016c15f7d0 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f017232a640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f016c162950 0x7f016c14f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.348+0000 7f0171328640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016c15f7d0 0x7f016c151320 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f015c00ea20 tx=0x7f015c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.352+0000 7f016a7fc640 1 -- 172.21.15.131:0/1066111357 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c00cde0 con 0x7f016c15f7d0 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.352+0000 7f016a7fc640 1 -- 172.21.15.131:0/1066111357 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f015c004c20 con 0x7f016c15f7d0 2024-09-17T02:50:20.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.352+0000 7f0173d8d640 1 -- 172.21.15.131:0/1066111357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c0c5ed0 con 0x7f016c15f7d0 2024-09-17T02:50:20.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.352+0000 7f016a7fc640 1 -- 172.21.15.131:0/1066111357 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c005020 con 0x7f016c15f7d0 2024-09-17T02:50:20.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.352+0000 7f0173d8d640 1 -- 172.21.15.131:0/1066111357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c0c6490 con 0x7f016c15f7d0 2024-09-17T02:50:20.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.352+0000 7f0173d8d640 1 -- 172.21.15.131:0/1066111357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016c0c5c40 con 0x7f016c15f7d0 2024-09-17T02:50:20.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.352+0000 7f016a7fc640 1 -- 172.21.15.131:0/1066111357 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f015c0051c0 con 0x7f016c15f7d0 2024-09-17T02:50:20.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.352+0000 7f016a7fc640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0158073230 0x7f0158075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:20.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.352+0000 7f016a7fc640 1 -- 172.21.15.131:0/1066111357 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f015c0950d0 con 0x7f016c15f7d0 2024-09-17T02:50:20.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.352+0000 7f0171b29640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0158073230 0x7f0158075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:20.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.356+0000 7f0171b29640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0158073230 0x7f0158075740 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f01600af000 tx=0x7f016001b120 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:20.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.356+0000 7f016a7fc640 1 -- 172.21.15.131:0/1066111357 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f015c016030 con 0x7f016c15f7d0 2024-09-17T02:50:20.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 -- 172.21.15.131:0/1066111357 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f016c15f290 con 0x7f016c15f7d0 2024-09-17T02:50:20.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f016a7fc640 1 -- 172.21.15.131:0/1066111357 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) ==== 53+0+73 (secure 0 0 0) 0x7f015c05fff0 con 0x7f016c15f7d0 2024-09-17T02:50:20.560 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:50:20.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 -- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0158073230 msgr2=0x7f0158075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0158073230 0x7f0158075740 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f01600af000 tx=0x7f016001b120 comp rx=0 tx=0).stop 2024-09-17T02:50:20.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 -- 172.21.15.131:0/1066111357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016c15f7d0 msgr2=0x7f016c151320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:20.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016c15f7d0 0x7f016c151320 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f015c00ea20 tx=0x7f015c00eee0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 -- 172.21.15.131:0/1066111357 shutdown_connections 2024-09-17T02:50:20.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0158073230 0x7f0158075740 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.564 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f016c162950 0x7f016c14f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.565 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f016c15f7d0 0x7f016c151320 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 --2- 172.21.15.131:0/1066111357 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f016c0cf710 0x7f016c0c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:20.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 -- 172.21.15.131:0/1066111357 >> 172.21.15.131:0/1066111357 conn(0x7f016c0d1b10 msgr2=0x7f016c15d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:20.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 -- 172.21.15.131:0/1066111357 shutdown_connections 2024-09-17T02:50:20.566 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20.564+0000 7f0173d8d640 1 -- 172.21.15.131:0/1066111357 wait complete. 2024-09-17T02:50:20.581 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:50:20.581 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:50:20.582 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:50:20.582 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:50:20.582 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:20.582 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:50:20.582 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:50:20.582 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:50:20.582 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:50:20.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:20.582 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:50:20.588 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:50:20.588 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:50:20.588 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:50:20.588 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:20.588 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:50:20.588 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:20.589 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:50:20.589 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:20.589 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:50:20.589 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:50:20.589 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:20.589 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:50:20.589 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:50:20.589 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:20.589 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41885sec preferred_lft 41885sec 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:20.590 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:50:20.591 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:20.591 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:50:20.591 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:20.591 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:50:20.591 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:20.591 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:50:20.591 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:20.591 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:50:20.591 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:50:20.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:20.591 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:50:20.641 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:50:20.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:20.643 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:50:20.695 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:50:20.695 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:50:20.695 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:50:20.695 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:50:20.695 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:50:20.695 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:50:20.695 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:50:20.695 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:50:20.695 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:50:20.695 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:50:20.696 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:50:20.696 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:50:20.696 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:50:20.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:50:20.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:50:20.696 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:50:20.696 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:50:20.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:50:20.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:50:20.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:20.816 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:50:20.868 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:50:20.868 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:50:20.869 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:50:20.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:20.869 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:50:20.923 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:50:20.923 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:50:20.923 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:20.971 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:50:20.972 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:50:20.972 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:21.019 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:50:21.076 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:50:21.130 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:50:21.131 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:50:21.132 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:50:21.132 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:50:21.132 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:50:21.132 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:21.132 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:21.132 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:21.132 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:21.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:21.133 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:50:21.179 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:50:21.179 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:50:21.228 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:50:21.244 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:50:21.292 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:50:21.293 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:50:21.294 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:50:21.295 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:50:21.295 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:21.295 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:21.295 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:21.295 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:21.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:21.295 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:50:21.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:50:21.344+0000 7f21f3c0c580 -1 init, newargv = 0x55f8c5e72670 newargc=15 2024-09-17T02:50:21.340 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49387]: starting ceph client 2024-09-17T02:50:21.346 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49387]: starting fuse 2024-09-17T02:50:22.444 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:50:22.460 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:50:22.508 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:50:22.508 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:50:22.508 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:50:22.509 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:50:22.510 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:50:22.511 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:50:22.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:50:22.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:22.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:50:22.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:22.511 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:50:22.511 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:22.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:22.512 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:50:22.563 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:50:22.564 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:50:22.564 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:50:22.564 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:50:22.564 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:50:22.564 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:50:22.564 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:50:22.564 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:22.564 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:50:22.564 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:50:22.564 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:50:22.564 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:50:22.565 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:50:22.566 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:22.566 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:50:22.566 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:50:22.688 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.49387.asok 2024-09-17T02:50:22.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:50:22.700 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49387.asok 2024-09-17T02:50:22.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:22.701 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49387.asok status 2024-09-17T02:50:22.878 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:50:22.879 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:50:22.879 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:50:22.879 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:50:22.879 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:50:22.879 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:50:22.879 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:50:22.879 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "49387", 2024-09-17T02:50:22.879 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:50:22.879 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:50:22.879 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:50:22.879 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8064, 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 8064 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3889849431 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:50:22.880 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3889849431 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.8064 192.168.0.1:0/3889849431", 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/3889849431", 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 86, 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 107, 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:50:22.881 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:50:22.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:22.895 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:50:22.895 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:22.902 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:50:22.903 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:22.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:22.903 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:22.959 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:50:22.959 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:50:23.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:23.008 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-17T02:50:23.140 INFO:teuthology.orchestra.run.smithi131.stderr:6+0 records in 2024-09-17T02:50:23.140 INFO:teuthology.orchestra.run.smithi131.stderr:6+0 records out 2024-09-17T02:50:23.140 INFO:teuthology.orchestra.run.smithi131.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.076916 s, 81.8 MB/s 2024-09-17T02:50:23.141 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:50:23.141 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:50:23.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:23.149 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-17T02:50:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:6+0 records in 2024-09-17T02:50:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:6+0 records out 2024-09-17T02:50:23.272 INFO:teuthology.orchestra.run.smithi131.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0695646 s, 90.4 MB/s 2024-09-17T02:50:23.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:23.273 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-17T02:50:23.283 INFO:teuthology.orchestra.run.smithi131.stdout:./ 2024-09-17T02:50:23.283 INFO:teuthology.orchestra.run.smithi131.stdout:./file2_sixmegs 2024-09-17T02:50:23.283 INFO:teuthology.orchestra.run.smithi131.stdout:./file3_sixmegs 2024-09-17T02:50:23.283 INFO:teuthology.orchestra.run.smithi131.stdout:./file1_sixmegs 2024-09-17T02:50:23.284 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:50:23.284 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:50:23.285 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:50:23.285 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:23.285 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-17T02:50:23.285 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:50:23.393 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:50:23.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:50:23.404 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:50:23.404 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:50:23.404 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:50:23.404 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:23.404 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-09-17T02:50:23.404 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:50:23.514 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627777 2024-09-17T02:50:23.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:50:23.523 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:50:23.523 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:50:23.524 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:50:23.524 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:23.524 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-09-17T02:50:23.524 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:50:23.630 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627778 2024-09-17T02:50:23.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:50:23.639 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:50:23.639 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:50:23.639 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:50:23.639 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:50:23.639 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-09-17T02:50:23.640 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:50:23.745 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627776 2024-09-17T02:50:23.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:50:23.754 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:50:23.754 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:50:23.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:50:24.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 -- 172.21.15.131:0/669965924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cfc0ced00 msgr2=0x7f9cfc06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:24.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 --2- 172.21.15.131:0/669965924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cfc0ced00 0x7f9cfc06dea0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0cfe00 tx=0x7f9cec01c800 comp rx=0 tx=0).stop 2024-09-17T02:50:24.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 -- 172.21.15.131:0/669965924 shutdown_connections 2024-09-17T02:50:24.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 --2- 172.21.15.131:0/669965924 >> v2:172.21.15.189:3300/0 conn(0x7f9cfc15c5b0 0x7f9cfc0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 --2- 172.21.15.131:0/669965924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cfc0ced00 0x7f9cfc06dea0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 -- 172.21.15.131:0/669965924 >> 172.21.15.131:0/669965924 conn(0x7f9cfc06eea0 msgr2=0x7f9cfc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:24.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 -- 172.21.15.131:0/669965924 shutdown_connections 2024-09-17T02:50:24.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 -- 172.21.15.131:0/669965924 wait complete. 2024-09-17T02:50:24.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 Processor -- start 2024-09-17T02:50:24.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 -- start start 2024-09-17T02:50:24.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9cfc06e3e0 0x7f9cfc151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:24.012 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cfc0ced00 0x7f9cfc0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cfc15c5b0 0x7f9cfc0b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc168720 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc1685a0 con 0x7f9cfc0ced00 2024-09-17T02:50:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.011+0000 7f9d01b56640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9cfc1688a0 con 0x7f9cfc06e3e0 2024-09-17T02:50:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cfc15c5b0 0x7f9cfc0b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cfc15c5b0 0x7f9cfc0b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58254/0 (socket says 172.21.15.131:58254) 2024-09-17T02:50:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfbfff640 1 -- 172.21.15.131:0/708844017 learned_addr learned my addr 172.21.15.131:0/708844017 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:24.013 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfb7fe640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9cfc06e3e0 0x7f9cfc151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:24.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfaffd640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cfc0ced00 0x7f9cfc0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:24.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfbfff640 1 -- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9cfc06e3e0 msgr2=0x7f9cfc151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:24.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfbfff640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9cfc06e3e0 0x7f9cfc151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfbfff640 1 -- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cfc0ced00 msgr2=0x7f9cfc0b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:24.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfbfff640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cfc0ced00 0x7f9cfc0b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfbfff640 1 -- 172.21.15.131:0/708844017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc158e90 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfaffd640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cfc0ced00 0x7f9cfc0b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:24.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfbfff640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cfc15c5b0 0x7f9cfc0b4a20 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e940 tx=0x7f9cf400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:24.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9ce3fff640 1 -- 172.21.15.131:0/708844017 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4004d30 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9ce3fff640 1 -- 172.21.15.131:0/708844017 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cf400cb20 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9d01b56640 1 -- 172.21.15.131:0/708844017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cfc159170 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9ce3fff640 1 -- 172.21.15.131:0/708844017 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4005020 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9d01b56640 1 -- 172.21.15.131:0/708844017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cfc0b5870 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9d01b56640 1 -- 172.21.15.131:0/708844017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cfc158c00 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9ce3fff640 1 -- 172.21.15.131:0/708844017 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9cf40051c0 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9ce3fff640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ce8073200 0x7f9ce8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9ce3fff640 1 -- 172.21.15.131:0/708844017 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f9cf4094e30 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.016 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.015+0000 7f9cfb7fe640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ce8073200 0x7f9ce8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:24.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.019+0000 7f9cfb7fe640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ce8073200 0x7f9ce8075710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0af000 tx=0x7f9cec01b040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:24.018 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.019+0000 7f9ce3fff640 1 -- 172.21.15.131:0/708844017 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9cf4016030 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.203+0000 7f9d01b56640 1 -- 172.21.15.131:0/708844017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9cfc06dea0 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.203+0000 7f9ce3fff640 1 -- 172.21.15.131:0/708844017 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) ==== 80+0+0 (secure 0 0 0) 0x7f9cf405fc40 con 0x7f9cfc15c5b0 2024-09-17T02:50:24.200 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.203+0000 7f9d01b56640 1 -- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ce8073200 msgr2=0x7f9ce8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.203+0000 7f9d01b56640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ce8073200 0x7f9ce8075710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0af000 tx=0x7f9cec01b040 comp rx=0 tx=0).stop 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.203+0000 7f9d01b56640 1 -- 172.21.15.131:0/708844017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cfc15c5b0 msgr2=0x7f9cfc0b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.203+0000 7f9d01b56640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cfc15c5b0 0x7f9cfc0b4a20 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e940 tx=0x7f9cf400ee00 comp rx=0 tx=0).stop 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.207+0000 7f9d01b56640 1 -- 172.21.15.131:0/708844017 shutdown_connections 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.207+0000 7f9d01b56640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ce8073200 0x7f9ce8075710 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.207+0000 7f9d01b56640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9cfc15c5b0 0x7f9cfc0b4a20 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.207+0000 7f9d01b56640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9cfc0ced00 0x7f9cfc0b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.207+0000 7f9d01b56640 1 --2- 172.21.15.131:0/708844017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9cfc06e3e0 0x7f9cfc151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.207+0000 7f9d01b56640 1 -- 172.21.15.131:0/708844017 >> 172.21.15.131:0/708844017 conn(0x7f9cfc06eea0 msgr2=0x7f9cfc152e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.207+0000 7f9d01b56640 1 -- 172.21.15.131:0/708844017 shutdown_connections 2024-09-17T02:50:24.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.207+0000 7f9d01b56640 1 -- 172.21.15.131:0/708844017 wait complete. 2024-09-17T02:50:24.221 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:50:24.222 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:50:24.227 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:50:24.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.435+0000 7f23fb871640 1 -- 172.21.15.131:0/415474840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0005290 msgr2=0x7f23e000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:24.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.435+0000 7f23fb871640 1 --2- 172.21.15.131:0/415474840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0005290 0x7f23e000b9d0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f23f0006fa0 tx=0x7f23f001f680 comp rx=0 tx=0).stop 2024-09-17T02:50:24.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.435+0000 7f23fb871640 1 -- 172.21.15.131:0/415474840 shutdown_connections 2024-09-17T02:50:24.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.435+0000 7f23fb871640 1 --2- 172.21.15.131:0/415474840 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0005290 0x7f23e000b9d0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.435+0000 7f23fb871640 1 -- 172.21.15.131:0/415474840 >> 172.21.15.131:0/415474840 conn(0x7f23f40cd6a0 msgr2=0x7f23f40cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:24.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.435+0000 7f23fb871640 1 -- 172.21.15.131:0/415474840 shutdown_connections 2024-09-17T02:50:24.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.435+0000 7f23fb871640 1 -- 172.21.15.131:0/415474840 wait complete. 2024-09-17T02:50:24.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23fb871640 1 Processor -- start 2024-09-17T02:50:24.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23fb871640 1 -- start start 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23fb871640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0005290 0x7f23f415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23fb871640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f415e690 0x7f23f41f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23fb871640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f23f41f7270 0x7f23f41f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23fb871640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f23f40cb200 con 0x7f23e0005290 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23fb871640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f23f40cb080 con 0x7f23f415e690 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23fb871640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f23f40cb380 con 0x7f23f41f7270 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f960d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0005290 0x7f23f415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f960d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0005290 0x7f23f415e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58278/0 (socket says 172.21.15.131:58278) 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f960d640 1 -- 172.21.15.131:0/544260799 learned_addr learned my addr 172.21.15.131:0/544260799 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f9e0e640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f23f41f7270 0x7f23f41f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f8e0c640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f415e690 0x7f23f41f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f960d640 1 -- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f23f41f7270 msgr2=0x7f23f41f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:24.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f960d640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f23f41f7270 0x7f23f41f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f960d640 1 -- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f415e690 msgr2=0x7f23f41f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:24.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f960d640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f415e690 0x7f23f41f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f960d640 1 -- 172.21.15.131:0/544260799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f4159bc0 con 0x7f23e0005290 2024-09-17T02:50:24.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f8e0c640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f415e690 0x7f23f41f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:24.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23f960d640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0005290 0x7f23f415e150 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00db20 tx=0x7f23ec0020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:24.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23e5ffb640 1 -- 172.21.15.131:0/544260799 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec016020 con 0x7f23e0005290 2024-09-17T02:50:24.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23e5ffb640 1 -- 172.21.15.131:0/544260799 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23ec00b970 con 0x7f23e0005290 2024-09-17T02:50:24.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23fb871640 1 -- 172.21.15.131:0/544260799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f4159ea0 con 0x7f23e0005290 2024-09-17T02:50:24.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23e5ffb640 1 -- 172.21.15.131:0/544260799 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ec00bdc0 con 0x7f23e0005290 2024-09-17T02:50:24.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23fb871640 1 -- 172.21.15.131:0/544260799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f415a460 con 0x7f23e0005290 2024-09-17T02:50:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.439+0000 7f23fb871640 1 -- 172.21.15.131:0/544260799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f40cfc70 con 0x7f23e0005290 2024-09-17T02:50:24.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.443+0000 7f23e5ffb640 1 -- 172.21.15.131:0/544260799 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f23ec019020 con 0x7f23e0005290 2024-09-17T02:50:24.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.443+0000 7f23e5ffb640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f23d8073230 0x7f23d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:24.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.443+0000 7f23e5ffb640 1 -- 172.21.15.131:0/544260799 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f23ec0941b0 con 0x7f23e0005290 2024-09-17T02:50:24.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.443+0000 7f23f8e0c640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f23d8073230 0x7f23d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:24.441 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.443+0000 7f23f8e0c640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f23d8073230 0x7f23d8075740 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f23f4157410 tx=0x7f23e80091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:24.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.447+0000 7f23e5ffb640 1 -- 172.21.15.131:0/544260799 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f23ec020030 con 0x7f23e0005290 2024-09-17T02:50:24.625 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.627+0000 7f23fb871640 1 -- 172.21.15.131:0/544260799 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f23f40b6770 con 0x7f23e0005290 2024-09-17T02:50:24.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.627+0000 7f23e5ffb640 1 -- 172.21.15.131:0/544260799 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) ==== 80+0+0 (secure 0 0 0) 0x7f23ec05f0d0 con 0x7f23e0005290 2024-09-17T02:50:24.626 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 -- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f23d8073230 msgr2=0x7f23d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f23d8073230 0x7f23d8075740 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f23f4157410 tx=0x7f23e80091e0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 -- 172.21.15.131:0/544260799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0005290 msgr2=0x7f23f415e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0005290 0x7f23f415e150 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00db20 tx=0x7f23ec0020c0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 -- 172.21.15.131:0/544260799 shutdown_connections 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f23d8073230 0x7f23d8075740 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f23f41f7270 0x7f23f41f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f23f415e690 0x7f23f41f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 --2- 172.21.15.131:0/544260799 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f23e0005290 0x7f23f415e150 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 -- 172.21.15.131:0/544260799 >> 172.21.15.131:0/544260799 conn(0x7f23f40cd6a0 msgr2=0x7f23f40c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 -- 172.21.15.131:0/544260799 shutdown_connections 2024-09-17T02:50:24.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:24.631+0000 7f23fb871640 1 -- 172.21.15.131:0/544260799 wait complete. 2024-09-17T02:50:24.648 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:50:24.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:24.648 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:24.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[49387]: fuse finished with error 0 and tester_r 0 2024-09-17T02:50:24.661 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:50:30.763 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:50:30.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:30.763 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:50:30.771 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:50:30.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.983+0000 7f6877ac6640 1 -- 172.21.15.131:0/2719184863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68700d0630 msgr2=0x7f68700cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:30.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.983+0000 7f6877ac6640 1 --2- 172.21.15.131:0/2719184863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68700d0630 0x7f68700cc480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6860009840 tx=0x7f686001c7f0 comp rx=0 tx=0).stop 2024-09-17T02:50:30.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.983+0000 7f6877ac6640 1 -- 172.21.15.131:0/2719184863 shutdown_connections 2024-09-17T02:50:30.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.983+0000 7f6877ac6640 1 --2- 172.21.15.131:0/2719184863 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68700d0630 0x7f68700cc480 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:30.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.983+0000 7f6877ac6640 1 --2- 172.21.15.131:0/2719184863 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f687015b310 0x7f68700cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:30.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.983+0000 7f6877ac6640 1 -- 172.21.15.131:0/2719184863 >> 172.21.15.131:0/2719184863 conn(0x7f687006fbe0 msgr2=0x7f687006fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:30.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.983+0000 7f6877ac6640 1 -- 172.21.15.131:0/2719184863 shutdown_connections 2024-09-17T02:50:30.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.983+0000 7f6877ac6640 1 -- 172.21.15.131:0/2719184863 wait complete. 2024-09-17T02:50:30.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.983+0000 7f6877ac6640 1 Processor -- start 2024-09-17T02:50:30.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.983+0000 7f6877ac6640 1 -- start start 2024-09-17T02:50:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6877ac6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f687015b310 0x7f6870155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6877ac6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f687015f750 0x7f6870156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6877ac6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6870155a10 0x7f6870156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6877ac6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6870168e30 con 0x7f687015f750 2024-09-17T02:50:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6877ac6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6870168cb0 con 0x7f6870155a10 2024-09-17T02:50:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6877ac6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6870168fb0 con 0x7f687015b310 2024-09-17T02:50:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6875061640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f687015f750 0x7f6870156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6875061640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f687015f750 0x7f6870156440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:58286/0 (socket says 172.21.15.131:58286) 2024-09-17T02:50:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6875061640 1 -- 172.21.15.131:0/2535947238 learned_addr learned my addr 172.21.15.131:0/2535947238 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6875862640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f687015b310 0x7f6870155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:30.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6876063640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6870155a10 0x7f6870156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6875061640 1 -- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f687015b310 msgr2=0x7f6870155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6875061640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f687015b310 0x7f6870155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6875061640 1 -- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6870155a10 msgr2=0x7f6870156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6875061640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6870155a10 0x7f6870156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6875061640 1 -- 172.21.15.131:0/2535947238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68701586e0 con 0x7f687015f750 2024-09-17T02:50:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6875061640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f687015f750 0x7f6870156440 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f686400c3c0 tx=0x7f686400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f686e7fc640 1 -- 172.21.15.131:0/2535947238 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864017070 con 0x7f687015f750 2024-09-17T02:50:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6877ac6640 1 -- 172.21.15.131:0/2535947238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6870158400 con 0x7f687015f750 2024-09-17T02:50:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f686e7fc640 1 -- 172.21.15.131:0/2535947238 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68640053c0 con 0x7f687015f750 2024-09-17T02:50:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f686e7fc640 1 -- 172.21.15.131:0/2535947238 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864003860 con 0x7f687015f750 2024-09-17T02:50:30.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6877ac6640 1 -- 172.21.15.131:0/2535947238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68701613d0 con 0x7f687015f750 2024-09-17T02:50:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f6877ac6640 1 -- 172.21.15.131:0/2535947238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68700cc480 con 0x7f687015f750 2024-09-17T02:50:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f686e7fc640 1 -- 172.21.15.131:0/2535947238 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f686401a070 con 0x7f687015f750 2024-09-17T02:50:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.987+0000 7f686e7fc640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f685c073200 0x7f685c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.991+0000 7f686e7fc640 1 -- 172.21.15.131:0/2535947238 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f6864093700 con 0x7f687015f750 2024-09-17T02:50:30.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.991+0000 7f6875862640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f685c073200 0x7f685c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:30.989 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.991+0000 7f6875862640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f685c073200 0x7f685c075710 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6868009a40 tx=0x7f6868010040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:30.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:30.995+0000 7f686e7fc640 1 -- 172.21.15.131:0/2535947238 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f686405dec0 con 0x7f687015f750 2024-09-17T02:50:31.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 -- 172.21.15.131:0/2535947238 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f687006da40 con 0x7f687015f750 2024-09-17T02:50:31.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f686e7fc640 1 -- 172.21.15.131:0/2535947238 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5237 (secure 0 0 0) 0x7f68640628f0 con 0x7f687015f750 2024-09-17T02:50:31.192 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:31.192 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":86,"btime":"2024-09-17T02:50:06:247461+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:50:05.662097+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7860},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7860":{"gid":7860,"name":"a","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2930400858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2930400858},{"type":"v1","addr":"172.21.15.131:6835","nonce":2930400858}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7860,"qdb_cluster":[7860]},"id":7}]} 2024-09-17T02:50:31.192 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 86 2024-09-17T02:50:31.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 -- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f685c073200 msgr2=0x7f685c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f685c073200 0x7f685c075710 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6868009a40 tx=0x7f6868010040 comp rx=0 tx=0).stop 2024-09-17T02:50:31.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 -- 172.21.15.131:0/2535947238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f687015f750 msgr2=0x7f6870156440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f687015f750 0x7f6870156440 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f686400c3c0 tx=0x7f686400c880 comp rx=0 tx=0).stop 2024-09-17T02:50:31.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 -- 172.21.15.131:0/2535947238 shutdown_connections 2024-09-17T02:50:31.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f685c073200 0x7f685c075710 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6870155a10 0x7f6870156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f687015f750 0x7f6870156440 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 --2- 172.21.15.131:0/2535947238 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f687015b310 0x7f6870155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 -- 172.21.15.131:0/2535947238 >> 172.21.15.131:0/2535947238 conn(0x7f687006fbe0 msgr2=0x7f68700b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 -- 172.21.15.131:0/2535947238 shutdown_connections 2024-09-17T02:50:31.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.195+0000 7f6877ac6640 1 -- 172.21.15.131:0/2535947238 wait complete. 2024-09-17T02:50:31.213 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-17T02:50:31.393 INFO:teuthology.orchestra.run.smithi131.stdout:{"message":"","return_code":0} 2024-09-17T02:50:31.393 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-17T02:50:31.393 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:50:31.393 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:50:31.399 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:50:31.399 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi131.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-09-17T02:50:31.399 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:50:31.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:31.399 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:50:31.446 INFO:teuthology.orchestra.run.smithi131.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T02:50:31.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:50:31.448 DEBUG:teuthology.orchestra.run.smithi131:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-17T02:50:31.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:31.492 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-09-17T02:50:31.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.655+0000 7fcc2606b640 1 -- 172.21.15.131:0/3751195236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc2015b2d0 msgr2=0x7fcc2015b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.655+0000 7fcc2606b640 1 --2- 172.21.15.131:0/3751195236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc2015b2d0 0x7fcc2015b6e0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fcc20073ca0 tx=0x7fcc1001c830 comp rx=0 tx=0).stop 2024-09-17T02:50:31.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.655+0000 7fcc2606b640 1 -- 172.21.15.131:0/3751195236 shutdown_connections 2024-09-17T02:50:31.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.655+0000 7fcc2606b640 1 --2- 172.21.15.131:0/3751195236 >> v2:172.21.15.189:3300/0 conn(0x7fcc20073d10 0x7fcc20074120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.655+0000 7fcc2606b640 1 --2- 172.21.15.131:0/3751195236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc2015b2d0 0x7fcc2015b6e0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.655+0000 7fcc2606b640 1 -- 172.21.15.131:0/3751195236 >> 172.21.15.131:0/3751195236 conn(0x7fcc2006fdb0 msgr2=0x7fcc200701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.655+0000 7fcc2606b640 1 -- 172.21.15.131:0/3751195236 shutdown_connections 2024-09-17T02:50:31.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.655+0000 7fcc2606b640 1 -- 172.21.15.131:0/3751195236 wait complete. 2024-09-17T02:50:31.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc2606b640 1 Processor -- start 2024-09-17T02:50:31.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc2606b640 1 -- start start 2024-09-17T02:50:31.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc2606b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc20073d10 0x7fcc2014dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc2606b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc2015b2d0 0x7fcc201f8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc2606b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc2015ed40 0x7fcc201594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc2606b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20169030 con 0x7fcc20073d10 2024-09-17T02:50:31.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc2606b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20168eb0 con 0x7fcc2015ed40 2024-09-17T02:50:31.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc2606b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcc201691b0 con 0x7fcc2015b2d0 2024-09-17T02:50:31.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc20073d10 0x7fcc2014dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc20073d10 0x7fcc2014dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40114/0 (socket says 172.21.15.131:40114) 2024-09-17T02:50:31.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1f7fe640 1 -- 172.21.15.131:0/1770747485 learned_addr learned my addr 172.21.15.131:0/1770747485 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:31.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1f7fe640 1 -- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc2015b2d0 msgr2=0x7fcc201f8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1ffff640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc2015ed40 0x7fcc201594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1effd640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc2015b2d0 0x7fcc201f8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc2015b2d0 0x7fcc201f8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1f7fe640 1 -- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc2015ed40 msgr2=0x7fcc201594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc2015ed40 0x7fcc201594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1f7fe640 1 -- 172.21.15.131:0/1770747485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc20157830 con 0x7fcc20073d10 2024-09-17T02:50:31.659 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1ffff640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc2015ed40 0x7fcc201594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:31.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc20073d10 0x7fcc2014dd00 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009230 tx=0x7fcc10004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc07fff640 1 -- 172.21.15.131:0/1770747485 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc100bd020 con 0x7fcc20073d10 2024-09-17T02:50:31.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc07fff640 1 -- 172.21.15.131:0/1770747485 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc10005040 con 0x7fcc20073d10 2024-09-17T02:50:31.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc2606b640 1 -- 172.21.15.131:0/1770747485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc20157d60 con 0x7fcc20073d10 2024-09-17T02:50:31.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc07fff640 1 -- 172.21.15.131:0/1770747485 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc10005410 con 0x7fcc20073d10 2024-09-17T02:50:31.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.659+0000 7fcc2606b640 1 -- 172.21.15.131:0/1770747485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2014d500 con 0x7fcc20073d10 2024-09-17T02:50:31.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7fcc07fff640 1 -- 172.21.15.131:0/1770747485 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcc100c0020 con 0x7fcc20073d10 2024-09-17T02:50:31.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7fcc07fff640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc0c073230 0x7fcc0c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7fcc07fff640 1 -- 172.21.15.131:0/1770747485 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fcc1012f1d0 con 0x7fcc20073d10 2024-09-17T02:50:31.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7fcc1effd640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc0c073230 0x7fcc0c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7fcc2d336000 1 Processor -- start 2024-09-17T02:50:31.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7fcc2d336000 1 -- start start 2024-09-17T02:50:31.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7fcc1effd640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc0c073230 0x7fcc0c075740 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcc20154810 tx=0x7fcc14005bf0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7f981b3cb640 1 -- 172.21.15.131:0/2005862965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98140b44b0 msgr2=0x7f98140b48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7f981b3cb640 1 --2- 172.21.15.131:0/2005862965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98140b44b0 0x7f98140b48c0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f9814073980 tx=0x7f980c01cca0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7f981b3cb640 1 -- 172.21.15.131:0/2005862965 shutdown_connections 2024-09-17T02:50:31.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7f981b3cb640 1 --2- 172.21.15.131:0/2005862965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98140b44b0 0x7f98140b48c0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7f981b3cb640 1 --2- 172.21.15.131:0/2005862965 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f981415f490 0x7f98140b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7f981b3cb640 1 --2- 172.21.15.131:0/2005862965 >> v2:172.21.15.189:3300/0 conn(0x7f9814074880 0x7f981415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7f981b3cb640 1 -- 172.21.15.131:0/2005862965 >> 172.21.15.131:0/2005862965 conn(0x7f9814070bc0 msgr2=0x7f9814073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7f981b3cb640 1 -- 172.21.15.131:0/2005862965 shutdown_connections 2024-09-17T02:50:31.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.663+0000 7f981b3cb640 1 -- 172.21.15.131:0/2005862965 wait complete. 2024-09-17T02:50:31.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c9a51a0 0x555b0caee640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c99dae0 0x555b0caeeb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c987970 0x555b0caef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555b0c8b3bd0 con 0x555b0c9a51a0 2024-09-17T02:50:31.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555b0c8a4430 con 0x555b0c987970 2024-09-17T02:50:31.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555b0c901ce0 con 0x555b0c99dae0 2024-09-17T02:50:31.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc1f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c9a51a0 0x555b0caee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc1f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c9a51a0 0x555b0caee640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40124/0 (socket says 172.21.15.131:40124) 2024-09-17T02:50:31.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc1f7fe640 1 -- 172.21.15.131:0/1191941346 learned_addr learned my addr 172.21.15.131:0/1191941346 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:31.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc1f7fe640 1 -- 172.21.15.131:0/1191941346 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c99dae0 msgr2=0x555b0caeeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1191941346 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c99dae0 0x555b0caeeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc1ffff640 1 --2- 172.21.15.131:0/1191941346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c987970 0x555b0caef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc1f7fe640 1 -- 172.21.15.131:0/1191941346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c987970 msgr2=0x555b0caef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1191941346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c987970 0x555b0caef0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc1f7fe640 1 -- 172.21.15.131:0/1191941346 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b0c8a5d80 con 0x555b0c9a51a0 2024-09-17T02:50:31.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1191941346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c9a51a0 0x555b0caee640 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fcc100f9810 tx=0x7fcc10009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3412f94ac348a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc057fa640 1 -- 172.21.15.131:0/1191941346 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc100fe030 con 0x555b0c9a51a0 2024-09-17T02:50:31.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc057fa640 1 -- 172.21.15.131:0/1191941346 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc100fa210 con 0x555b0c9a51a0 2024-09-17T02:50:31.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc057fa640 1 -- 172.21.15.131:0/1191941346 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc100f9b10 con 0x555b0c9a51a0 2024-09-17T02:50:31.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f981b3cb640 1 Processor -- start 2024-09-17T02:50:31.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 -- 172.21.15.131:0/1191941346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c9a51a0 msgr2=0x555b0caee640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1191941346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c9a51a0 0x555b0caee640 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fcc100f9810 tx=0x7fcc10009260 comp rx=0 tx=0).stop 2024-09-17T02:50:31.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 -- 172.21.15.131:0/1191941346 shutdown_connections 2024-09-17T02:50:31.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1191941346 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c987970 0x555b0caef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1191941346 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c99dae0 0x555b0caeeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1191941346 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c9a51a0 0x555b0caee640 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 -- 172.21.15.131:0/1191941346 >> 172.21.15.131:0/1191941346 conn(0x555b0c9ab8c0 msgr2=0x555b0caa34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f981b3cb640 1 -- start start 2024-09-17T02:50:31.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 -- 172.21.15.131:0/1191941346 shutdown_connections 2024-09-17T02:50:31.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 -- 172.21.15.131:0/1191941346 wait complete. 2024-09-17T02:50:31.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 -- start start 2024-09-17T02:50:31.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7fcc2d336000 1 Processor -- start 2024-09-17T02:50:31.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f981b3cb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9814074880 0x7f981414f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f981b3cb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98140b44b0 0x7f9814137a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f981b3cb640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f981415f490 0x7f9814150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f981b3cb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f981415fa20 con 0x7f98140b44b0 2024-09-17T02:50:31.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f981b3cb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f981415f8a0 con 0x7f9814074880 2024-09-17T02:50:31.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f981b3cb640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f981406e7b0 con 0x7f981415f490 2024-09-17T02:50:31.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9818966640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98140b44b0 0x7f9814137a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9818966640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98140b44b0 0x7f9814137a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40138/0 (socket says 172.21.15.131:40138) 2024-09-17T02:50:31.672 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9818966640 1 -- 172.21.15.131:0/1414488223 learned_addr learned my addr 172.21.15.131:0/1414488223 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9819968640 1 --2- 172.21.15.131:0/1414488223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f981415f490 0x7f9814150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9819167640 1 --2- 172.21.15.131:0/1414488223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9814074880 0x7f981414f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9818966640 1 -- 172.21.15.131:0/1414488223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f981415f490 msgr2=0x7f9814150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9818966640 1 --2- 172.21.15.131:0/1414488223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f981415f490 0x7f9814150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9818966640 1 -- 172.21.15.131:0/1414488223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9814074880 msgr2=0x7f981414f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9818966640 1 --2- 172.21.15.131:0/1414488223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9814074880 0x7f981414f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9818966640 1 -- 172.21.15.131:0/1414488223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9814151980 con 0x7f98140b44b0 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9819167640 1 --2- 172.21.15.131:0/1414488223 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9814074880 0x7f981414f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9819968640 1 --2- 172.21.15.131:0/1414488223 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f981415f490 0x7f9814150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.667+0000 7f9818966640 1 --2- 172.21.15.131:0/1414488223 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98140b44b0 0x7f9814137a10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f980400c940 tx=0x7f980400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f9809ffb640 1 -- 172.21.15.131:0/1414488223 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9804004220 con 0x7f98140b44b0 2024-09-17T02:50:31.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f9809ffb640 1 -- 172.21.15.131:0/1414488223 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98040079b0 con 0x7f98140b44b0 2024-09-17T02:50:31.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f981b3cb640 1 -- 172.21.15.131:0/1414488223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9814151c60 con 0x7f98140b44b0 2024-09-17T02:50:31.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f9809ffb640 1 -- 172.21.15.131:0/1414488223 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98040044b0 con 0x7f98140b44b0 2024-09-17T02:50:31.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f981b3cb640 1 -- 172.21.15.131:0/1414488223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981414a230 con 0x7f98140b44b0 2024-09-17T02:50:31.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c987970 0x555b0cadc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c99dae0 0x555b0cadca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c9a51a0 0x555b0cadcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555b0c8f4f20 con 0x555b0c987970 2024-09-17T02:50:31.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555b0c8f1110 con 0x555b0c9a51a0 2024-09-17T02:50:31.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555b0c8966d0 con 0x555b0c99dae0 2024-09-17T02:50:31.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 10 client.0 _ll_register_callbacks cb 0x555b0c895330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:50:31.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1effd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c99dae0 0x555b0cadca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c9a51a0 0x555b0cadcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1ffff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c9a51a0 0x555b0cadcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.189:3300/0 says I am v2:172.21.15.131:60208/0 (socket says 172.21.15.131:60208) 2024-09-17T02:50:31.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1ffff640 1 -- 172.21.15.131:0/1058010056 learned_addr learned my addr 172.21.15.131:0/1058010056 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:31.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f981b3cb640 1 -- 172.21.15.131:0/1414488223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f981406f4f0 con 0x7f98140b44b0 2024-09-17T02:50:31.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c987970 0x555b0cadc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1f7fe640 1 -- 172.21.15.131:0/1058010056 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c99dae0 msgr2=0x555b0cadca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c99dae0 0x555b0cadca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1f7fe640 1 -- 172.21.15.131:0/1058010056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c9a51a0 msgr2=0x555b0cadcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c9a51a0 0x555b0cadcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1f7fe640 1 -- 172.21.15.131:0/1058010056 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b0c88f310 con 0x555b0c987970 2024-09-17T02:50:31.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c987970 0x555b0cadc100 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fcc100f6d80 tx=0x7fcc10004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8559d8a185ff229 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcbfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:50:31.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1058010056 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc100fe040 con 0x555b0c987970 2024-09-17T02:50:31.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1058010056 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fcc10103210 con 0x555b0c987970 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 -- 172.21.15.131:0/1058010056 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555b0c889450 con 0x555b0c987970 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1058010056 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fcc100fe3f0 con 0x555b0c987970 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 10 client.8103 fetch_fsmap learned FSMap version 86 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 -- 172.21.15.131:0/1058010056 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x555b0c8acf60 con 0x555b0c987970 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1058010056 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555b0c889450 con 0x555b0c987970 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1058010056 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7fcc100f8d70 con 0x555b0c987970 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 10 client.8103 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 10 client.8103 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 20 client.8103 populate_metadata read hostname 'smithi131' 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x555b0c593b80 0x555b0caddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 4 client.8103 mds_command: new command op to 7860 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 -- 172.21.15.131:0/1058010056 --> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b0c903c30 con 0x555b0c593b80 2024-09-17T02:50:31.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1effd640 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x555b0c593b80 0x555b0caddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcbfe7fc640 10 client.8103 ms_handle_connect on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:31.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f9809ffb640 1 -- 172.21.15.131:0/1414488223 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9804003600 con 0x7f98140b44b0 2024-09-17T02:50:31.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc1effd640 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x555b0c593b80 0x555b0caddcd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=110c923a5c76878d server_cookie=e334e85bc035b824 in_seq=0 out_seq=0 2024-09-17T02:50:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f9809ffb640 1 --2- 172.21.15.131:0/1414488223 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f97f8073230 0x7f97f8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f9809ffb640 1 -- 172.21.15.131:0/1414488223 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f98040948e0 con 0x7f98140b44b0 2024-09-17T02:50:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f9809ffb640 1 -- 172.21.15.131:0/1414488223 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+514 (secure 0 0 0) 0x7f9804014030 con 0x7f98140b44b0 2024-09-17T02:50:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1058010056 <== mds.0 v2:172.21.15.131:6833/2930400858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555b0c903c30 con 0x555b0c593b80 2024-09-17T02:50:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcbfe7fc640 10 client.8103 handle_command_reply: tid=0 2024-09-17T02:50:31.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 client.8103 shutdown 2024-09-17T02:50:31.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f9819167640 1 --2- 172.21.15.131:0/1414488223 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f97f8073230 0x7f97f8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7f981b3cb640 1 -- 172.21.15.131:0/1414488223 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f981406ec60 con 0x7f98140b44b0 2024-09-17T02:50:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 -- 172.21.15.131:0/1058010056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c987970 msgr2=0x555b0cadc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.671+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c987970 0x555b0cadc100 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fcc100f6d80 tx=0x7fcc10004520 comp rx=0 tx=0).stop 2024-09-17T02:50:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.675+0000 7fcc2d336000 1 -- 172.21.15.131:0/1058010056 shutdown_connections 2024-09-17T02:50:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.675+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x555b0c593b80 0x555b0caddcd0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.675+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c9a51a0 0x555b0cadcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.675+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c99dae0 0x555b0cadca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.675+0000 7fcbfe7fc640 0 client.8103 ms_handle_reset on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.675+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1058010056 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c987970 0x555b0cadc100 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.675+0000 7fcc2d336000 1 -- 172.21.15.131:0/1058010056 >> 172.21.15.131:0/1058010056 conn(0x555b0c9ab8c0 msgr2=0x555b0caa34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.675+0000 7fcc2d336000 1 -- 172.21.15.131:0/1058010056 shutdown_connections 2024-09-17T02:50:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.675+0000 7fcc2d336000 1 -- 172.21.15.131:0/1058010056 wait complete. 2024-09-17T02:50:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.675+0000 7fcc2d336000 20 client.8103 trim_cache size 0 max 16384 2024-09-17T02:50:31.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.675+0000 7f9819167640 1 --2- 172.21.15.131:0/1414488223 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f97f8073230 0x7f97f8075740 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9810005fd0 tx=0x7f9810005d00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.691+0000 7fcc2d336000 1 Processor -- start 2024-09-17T02:50:31.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.691+0000 7fcc2d336000 1 -- start start 2024-09-17T02:50:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.691+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c987970 0x555b0cb047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.691+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb04d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.691+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c9a51a0 0x555b0cadbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.691+0000 7fcc2d336000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20205f20 con 0x555b0c99dae0 2024-09-17T02:50:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.691+0000 7fcc2d336000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcc20204f00 con 0x555b0c9a51a0 2024-09-17T02:50:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.691+0000 7fcc2d336000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcc20152380 con 0x555b0c987970 2024-09-17T02:50:31.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.691+0000 7fcc1effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb04d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc1effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb04d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40160/0 (socket says 172.21.15.131:40160) 2024-09-17T02:50:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc1effd640 1 -- 172.21.15.131:0/1585389475 learned_addr learned my addr 172.21.15.131:0/1585389475 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1585389475 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c987970 0x555b0cb047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc1effd640 1 -- 172.21.15.131:0/1585389475 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c987970 msgr2=0x555b0cb047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc1effd640 1 --2- 172.21.15.131:0/1585389475 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c987970 0x555b0cb047e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc1effd640 1 -- 172.21.15.131:0/1585389475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c9a51a0 msgr2=0x555b0cadbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc1ffff640 1 --2- 172.21.15.131:0/1585389475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c9a51a0 0x555b0cadbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc1effd640 1 --2- 172.21.15.131:0/1585389475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c9a51a0 0x555b0cadbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc1effd640 1 -- 172.21.15.131:0/1585389475 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b0c975f30 con 0x555b0c99dae0 2024-09-17T02:50:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc1effd640 1 --2- 172.21.15.131:0/1585389475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb04d20 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400df60 tx=0x7fcc14014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7625cc85804916 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc04ff9640 1 -- 172.21.15.131:0/1585389475 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1402d670 con 0x555b0c99dae0 2024-09-17T02:50:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc04ff9640 1 -- 172.21.15.131:0/1585389475 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc14028070 con 0x555b0c99dae0 2024-09-17T02:50:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc04ff9640 1 -- 172.21.15.131:0/1585389475 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1402dab0 con 0x555b0c99dae0 2024-09-17T02:50:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 -- 172.21.15.131:0/1585389475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 msgr2=0x555b0cb04d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1585389475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb04d20 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400df60 tx=0x7fcc14014000 comp rx=0 tx=0).stop 2024-09-17T02:50:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 -- 172.21.15.131:0/1585389475 shutdown_connections 2024-09-17T02:50:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1585389475 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c9a51a0 0x555b0cadbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1585389475 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb04d20 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1585389475 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c987970 0x555b0cb047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 -- 172.21.15.131:0/1585389475 >> 172.21.15.131:0/1585389475 conn(0x555b0c9ab8c0 msgr2=0x555b0cafa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 -- 172.21.15.131:0/1585389475 shutdown_connections 2024-09-17T02:50:31.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 -- 172.21.15.131:0/1585389475 wait complete. 2024-09-17T02:50:31.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 -- start start 2024-09-17T02:50:31.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 Processor -- start 2024-09-17T02:50:31.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c987970 0x555b0cb639a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb63ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c9a51a0 0x555b0cb64420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555b0c8f1110 con 0x555b0c99dae0 2024-09-17T02:50:31.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555b0c8b3bd0 con 0x555b0c987970 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.695+0000 7fcc2d336000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555b0c975aa0 con 0x555b0c9a51a0 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 10 client.0 _ll_register_callbacks cb 0x7fcc1011e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb63ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb63ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40166/0 (socket says 172.21.15.131:40166) 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1effd640 1 -- 172.21.15.131:0/1182788751 learned_addr learned my addr 172.21.15.131:0/1182788751 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c987970 0x555b0cb639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1ffff640 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c9a51a0 0x555b0cb64420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1effd640 1 -- 172.21.15.131:0/1182788751 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c9a51a0 msgr2=0x555b0cb64420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1effd640 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c9a51a0 0x555b0cb64420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1effd640 1 -- 172.21.15.131:0/1182788751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c987970 msgr2=0x555b0cb639a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1effd640 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c987970 0x555b0cb639a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1effd640 1 -- 172.21.15.131:0/1182788751 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b0c94cdd0 con 0x555b0c99dae0 2024-09-17T02:50:31.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1effd640 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb63ee0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400dbe0 tx=0x7fcc14004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bac758960010cf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcbfe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:50:31.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1182788751 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc14005240 con 0x555b0c99dae0 2024-09-17T02:50:31.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1182788751 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7fcc14036c40 con 0x555b0c99dae0 2024-09-17T02:50:31.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 -- 172.21.15.131:0/1182788751 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc14028070 con 0x555b0c99dae0 2024-09-17T02:50:31.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1182788751 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fcc14036e40 con 0x555b0c99dae0 2024-09-17T02:50:31.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 10 client.8109 fetch_fsmap learned FSMap version 86 2024-09-17T02:50:31.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 -- 172.21.15.131:0/1182788751 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x555b0c887840 con 0x555b0c99dae0 2024-09-17T02:50:31.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1182788751 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc14028070 con 0x555b0c99dae0 2024-09-17T02:50:31.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1182788751 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7fcc140053e0 con 0x555b0c99dae0 2024-09-17T02:50:31.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 10 client.8109 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T02:50:31.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 10 client.8109 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:50:31.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 20 client.8109 populate_metadata read hostname 'smithi131' 2024-09-17T02:50:31.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x555b0c581be0 0x555b0caa42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 4 client.8109 mds_command: new command op to 7860 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-09-17T02:50:31.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 -- 172.21.15.131:0/1182788751 --> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x555b0c8ff6e0 con 0x555b0c581be0 2024-09-17T02:50:31.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x555b0c581be0 0x555b0caa42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcbfe7fc640 10 client.8109 ms_handle_connect on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:31.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc1f7fe640 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x555b0c581be0 0x555b0caa42a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b238aed705dd5326 server_cookie=4cfedcf52cbe9365 in_seq=0 out_seq=0 2024-09-17T02:50:31.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcbfe7fc640 1 -- 172.21.15.131:0/1182788751 <== mds.0 v2:172.21.15.131:6833/2930400858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555b0c8ff6e0 con 0x555b0c581be0 2024-09-17T02:50:31.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcbfe7fc640 10 client.8109 handle_command_reply: tid=0 2024-09-17T02:50:31.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 client.8109 shutdown 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 -- 172.21.15.131:0/1182788751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 msgr2=0x555b0cb63ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb63ee0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400dbe0 tx=0x7fcc14004a80 comp rx=0 tx=0).stop 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 -- 172.21.15.131:0/1182788751 shutdown_connections 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x555b0c581be0 0x555b0caa42a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcbfe7fc640 0 client.8109 ms_handle_reset on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555b0c9a51a0 0x555b0cb64420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555b0c99dae0 0x555b0cb63ee0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 --2- 172.21.15.131:0/1182788751 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555b0c987970 0x555b0cb639a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 -- 172.21.15.131:0/1182788751 >> 172.21.15.131:0/1182788751 conn(0x555b0c9ab8c0 msgr2=0x555b0cafb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 -- 172.21.15.131:0/1182788751 shutdown_connections 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 1 -- 172.21.15.131:0/1182788751 wait complete. 2024-09-17T02:50:31.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.699+0000 7fcc2d336000 20 client.8109 trim_cache size 0 max 16384 2024-09-17T02:50:31.704 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:50:31.704 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-09-17T02:50:31.704 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "ea4d4c00-a7c4-41e2-be37-392148866f69", 2024-09-17T02:50:31.704 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-09-17T02:50:31.704 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:50:31.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 -- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc0c073230 msgr2=0x7fcc0c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc0c073230 0x7fcc0c075740 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fcc20154810 tx=0x7fcc14005bf0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 -- 172.21.15.131:0/1770747485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc20073d10 msgr2=0x7fcc2014dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc20073d10 0x7fcc2014dd00 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009230 tx=0x7fcc10004040 comp rx=0 tx=0).stop 2024-09-17T02:50:31.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 -- 172.21.15.131:0/1770747485 shutdown_connections 2024-09-17T02:50:31.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc0c073230 0x7fcc0c075740 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcc2015ed40 0x7fcc201594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcc2015b2d0 0x7fcc201f8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 --2- 172.21.15.131:0/1770747485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcc20073d10 0x7fcc2014dd00 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 -- 172.21.15.131:0/1770747485 >> 172.21.15.131:0/1770747485 conn(0x7fcc2006fdb0 msgr2=0x7fcc20164160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 -- 172.21.15.131:0/1770747485 shutdown_connections 2024-09-17T02:50:31.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.703+0000 7fcc2606b640 1 -- 172.21.15.131:0/1770747485 wait complete. 2024-09-17T02:50:31.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:50:31.717 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-17T02:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.871+0000 7f68b684b640 1 -- 172.21.15.131:0/1162342952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b00cfac0 msgr2=0x7f68b015ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.871+0000 7f68b684b640 1 --2- 172.21.15.131:0/1162342952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b00cfac0 0x7f68b015ef70 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f68b00b5d40 tx=0x7f68a001c6e0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.871+0000 7f68b684b640 1 -- 172.21.15.131:0/1162342952 shutdown_connections 2024-09-17T02:50:31.868 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.871+0000 7f68b684b640 1 --2- 172.21.15.131:0/1162342952 >> v2:172.21.15.189:3300/0 conn(0x7f68b00cfed0 0x7f68b0073c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.871+0000 7f68b684b640 1 --2- 172.21.15.131:0/1162342952 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b00cfac0 0x7f68b015ef70 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.871+0000 7f68b684b640 1 -- 172.21.15.131:0/1162342952 >> 172.21.15.131:0/1162342952 conn(0x7f68b00d13d0 msgr2=0x7f68b00734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.871+0000 7f68b684b640 1 -- 172.21.15.131:0/1162342952 shutdown_connections 2024-09-17T02:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.871+0000 7f68b684b640 1 -- 172.21.15.131:0/1162342952 wait complete. 2024-09-17T02:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.871+0000 7f68b684b640 1 Processor -- start 2024-09-17T02:50:31.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.871+0000 7f68b684b640 1 -- start start 2024-09-17T02:50:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68b684b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b00cfac0 0x7f68b014a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68b684b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f68b00cfed0 0x7f68b014dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68b684b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b015f4b0 0x7f68b0150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68b684b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0169110 con 0x7f68b00cfac0 2024-09-17T02:50:31.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68b684b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0070170 con 0x7f68b015f4b0 2024-09-17T02:50:31.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68b684b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f68b0169290 con 0x7f68b00cfed0 2024-09-17T02:50:31.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68affff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b00cfac0 0x7f68b014a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68affff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b00cfac0 0x7f68b014a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40176/0 (socket says 172.21.15.131:40176) 2024-09-17T02:50:31.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68affff640 1 -- 172.21.15.131:0/2155653301 learned_addr learned my addr 172.21.15.131:0/2155653301 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:31.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68a77fe640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f68b00cfed0 0x7f68b014dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68b4de8640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b015f4b0 0x7f68b0150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68affff640 1 -- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f68b00cfed0 msgr2=0x7f68b014dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68affff640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f68b00cfed0 0x7f68b014dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68affff640 1 -- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b015f4b0 msgr2=0x7f68b0150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68affff640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b015f4b0 0x7f68b0150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68affff640 1 -- 172.21.15.131:0/2155653301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b014e840 con 0x7f68b00cfac0 2024-09-17T02:50:31.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68a77fe640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f68b00cfed0 0x7f68b014dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:31.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68b4de8640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b015f4b0 0x7f68b0150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:50:31.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68affff640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b00cfac0 0x7f68b014a110 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f68a0000c10 tx=0x7f68a00a8600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68ad7fa640 1 -- 172.21.15.131:0/2155653301 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a00a8a50 con 0x7f68b00cfac0 2024-09-17T02:50:31.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68ad7fa640 1 -- 172.21.15.131:0/2155653301 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68a00a8bf0 con 0x7f68b00cfac0 2024-09-17T02:50:31.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68b684b640 1 -- 172.21.15.131:0/2155653301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b014eb20 con 0x7f68b00cfac0 2024-09-17T02:50:31.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68ad7fa640 1 -- 172.21.15.131:0/2155653301 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a0004030 con 0x7f68b00cfac0 2024-09-17T02:50:31.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68b684b640 1 -- 172.21.15.131:0/2155653301 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b014f0e0 con 0x7f68b00cfac0 2024-09-17T02:50:31.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.875+0000 7f68ad7fa640 1 -- 172.21.15.131:0/2155653301 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f68a00b8030 con 0x7f68b00cfac0 2024-09-17T02:50:31.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68ad7fa640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6890073200 0x7f6890075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68ad7fa640 1 -- 172.21.15.131:0/2155653301 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f68a012e300 con 0x7f68b00cfac0 2024-09-17T02:50:31.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68a77fe640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6890073200 0x7f6890075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68bdb16000 1 Processor -- start 2024-09-17T02:50:31.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68bdb16000 1 -- start start 2024-09-17T02:50:31.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68a77fe640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6890073200 0x7f6890075710 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6898005fd0 tx=0x7f6898005e00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355ebfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593559a6890 0x559355ec0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 0x559355ec0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68bdb16000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559355d5acd0 con 0x559355d4f6d0 2024-09-17T02:50:31.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68bdb16000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559355c70330 con 0x5593559a6890 2024-09-17T02:50:31.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68bdb16000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559355ccdbb0 con 0x559355a6aaa0 2024-09-17T02:50:31.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68affff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355ebfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68affff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355ebfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40186/0 (socket says 172.21.15.131:40186) 2024-09-17T02:50:31.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68affff640 1 -- 172.21.15.131:0/3891726314 learned_addr learned my addr 172.21.15.131:0/3891726314 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:31.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68a77fe640 1 --2- 172.21.15.131:0/3891726314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593559a6890 0x559355ec0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68b4de8640 1 --2- 172.21.15.131:0/3891726314 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 0x559355ec0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68affff640 1 -- 172.21.15.131:0/3891726314 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 msgr2=0x559355ec0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68affff640 1 --2- 172.21.15.131:0/3891726314 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 0x559355ec0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68affff640 1 -- 172.21.15.131:0/3891726314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593559a6890 msgr2=0x559355ec0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68affff640 1 --2- 172.21.15.131:0/3891726314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593559a6890 0x559355ec0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68affff640 1 -- 172.21.15.131:0/3891726314 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559355c71c80 con 0x559355d4f6d0 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68affff640 1 --2- 172.21.15.131:0/3891726314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355ebfff0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f68a00f5e20 tx=0x7f68a00f8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f499c333e359d611 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68a5ffb640 1 -- 172.21.15.131:0/3891726314 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a00bf070 con 0x559355d4f6d0 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68a5ffb640 1 -- 172.21.15.131:0/3891726314 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68a00fd690 con 0x559355d4f6d0 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.879+0000 7f68a5ffb640 1 -- 172.21.15.131:0/3891726314 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a01024b0 con 0x559355d4f6d0 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 -- 172.21.15.131:0/3891726314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 msgr2=0x559355ebfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3891726314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355ebfff0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f68a00f5e20 tx=0x7f68a00f8a80 comp rx=0 tx=0).stop 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 -- 172.21.15.131:0/3891726314 shutdown_connections 2024-09-17T02:50:31.880 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3891726314 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 0x559355ec0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3891726314 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593559a6890 0x559355ec0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3891726314 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355ebfff0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 -- 172.21.15.131:0/3891726314 >> 172.21.15.131:0/3891726314 conn(0x55935597a5c0 msgr2=0x559355e78ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 -- 172.21.15.131:0/3891726314 shutdown_connections 2024-09-17T02:50:31.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 -- 172.21.15.131:0/3891726314 wait complete. 2024-09-17T02:50:31.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 -- start start 2024-09-17T02:50:31.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 Processor -- start 2024-09-17T02:50:31.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593559a6890 0x559355e81cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 0x559355e821f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355e82730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559355cc0df0 con 0x559355d4f6d0 2024-09-17T02:50:31.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559355cbcfe0 con 0x5593559a6890 2024-09-17T02:50:31.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559355c624d0 con 0x559355a6aaa0 2024-09-17T02:50:31.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 10 client.0 _ll_register_callbacks cb 0x559355c407f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:50:31.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68b4de8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355e82730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68b4de8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355e82730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40190/0 (socket says 172.21.15.131:40190) 2024-09-17T02:50:31.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68b4de8640 1 -- 172.21.15.131:0/682030470 learned_addr learned my addr 172.21.15.131:0/682030470 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:31.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68affff640 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593559a6890 0x559355e81cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68a77fe640 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 0x559355e821f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68b4de8640 1 -- 172.21.15.131:0/682030470 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 msgr2=0x559355e821f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68b4de8640 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 0x559355e821f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68b4de8640 1 -- 172.21.15.131:0/682030470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593559a6890 msgr2=0x559355e81cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68b4de8640 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593559a6890 0x559355e81cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68b4de8640 1 -- 172.21.15.131:0/682030470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559355c5b210 con 0x559355d4f6d0 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68b4de8640 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355e82730 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f68a800e810 tx=0x7f68a800e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b032465aac0582 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f688effd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f688effd640 1 -- 172.21.15.131:0/682030470 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8017020 con 0x559355d4f6d0 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f688effd640 1 -- 172.21.15.131:0/682030470 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f68a8013870 con 0x559355d4f6d0 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 -- 172.21.15.131:0/682030470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559355c55350 con 0x559355d4f6d0 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f688effd640 1 -- 172.21.15.131:0/682030470 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f68a8021050 con 0x559355d4f6d0 2024-09-17T02:50:31.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 10 client.8124 fetch_fsmap learned FSMap version 86 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.883+0000 7f68bdb16000 1 -- 172.21.15.131:0/682030470 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x559355c41fc0 con 0x559355d4f6d0 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f688effd640 1 -- 172.21.15.131:0/682030470 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559355c55350 con 0x559355d4f6d0 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f688effd640 1 -- 172.21.15.131:0/682030470 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7f68a8024020 con 0x559355d4f6d0 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 10 client.8124 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 10 client.8124 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 20 client.8124 populate_metadata read hostname 'smithi131' 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55935595f930 0x559355ed15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 4 client.8124 mds_command: new command op to 7860 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 -- 172.21.15.131:0/682030470 --> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559355ccfb00 con 0x55935595f930 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68affff640 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55935595f930 0x559355ed15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f688effd640 10 client.8124 ms_handle_connect on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:31.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68affff640 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55935595f930 0x559355ed15b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c6dee4ab8b803f7 server_cookie=cf3de36b1346e2ff in_seq=0 out_seq=0 2024-09-17T02:50:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f688effd640 1 -- 172.21.15.131:0/682030470 <== mds.0 v2:172.21.15.131:6833/2930400858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559355ccfb00 con 0x55935595f930 2024-09-17T02:50:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f688effd640 10 client.8124 handle_command_reply: tid=0 2024-09-17T02:50:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 client.8124 shutdown 2024-09-17T02:50:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 -- 172.21.15.131:0/682030470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 msgr2=0x559355e82730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355e82730 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f68a800e810 tx=0x7f68a800e900 comp rx=0 tx=0).stop 2024-09-17T02:50:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 -- 172.21.15.131:0/682030470 shutdown_connections 2024-09-17T02:50:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55935595f930 0x559355ed15b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559355d4f6d0 0x559355e82730 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 0x559355e821f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 --2- 172.21.15.131:0/682030470 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5593559a6890 0x559355e81cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f688effd640 0 client.8124 ms_handle_reset on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:31.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 -- 172.21.15.131:0/682030470 >> 172.21.15.131:0/682030470 conn(0x55935597a5c0 msgr2=0x559355e70360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 -- 172.21.15.131:0/682030470 shutdown_connections 2024-09-17T02:50:31.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 1 -- 172.21.15.131:0/682030470 wait complete. 2024-09-17T02:50:31.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.887+0000 7f68bdb16000 20 client.8124 trim_cache size 0 max 16384 2024-09-17T02:50:31.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.903+0000 7f68bdb16000 1 Processor -- start 2024-09-17T02:50:31.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.903+0000 7f68bdb16000 1 -- start start 2024-09-17T02:50:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355f31700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559355a6aaa0 0x559355f31c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355d4f6d0 0x559355f32180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f68b0201b80 con 0x5593559a6890 2024-09-17T02:50:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f68b01535f0 con 0x559355a6aaa0 2024-09-17T02:50:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f68b0158950 con 0x559355d4f6d0 2024-09-17T02:50:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68affff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355f31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68affff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355f31700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40196/0 (socket says 172.21.15.131:40196) 2024-09-17T02:50:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68affff640 1 -- 172.21.15.131:0/3215959867 learned_addr learned my addr 172.21.15.131:0/3215959867 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:31.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68affff640 1 -- 172.21.15.131:0/3215959867 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355d4f6d0 msgr2=0x559355f32180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68b4de8640 1 --2- 172.21.15.131:0/3215959867 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355d4f6d0 0x559355f32180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68affff640 1 --2- 172.21.15.131:0/3215959867 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355d4f6d0 0x559355f32180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68affff640 1 -- 172.21.15.131:0/3215959867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559355a6aaa0 msgr2=0x559355f31c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68affff640 1 --2- 172.21.15.131:0/3215959867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559355a6aaa0 0x559355f31c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68affff640 1 -- 172.21.15.131:0/3215959867 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559355d1a680 con 0x5593559a6890 2024-09-17T02:50:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68affff640 1 --2- 172.21.15.131:0/3215959867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355f31700 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f68a00f88b0 tx=0x7f68a00fb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d22beb3b3d4e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68a57fa640 1 -- 172.21.15.131:0/3215959867 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a00bf040 con 0x5593559a6890 2024-09-17T02:50:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68a57fa640 1 -- 172.21.15.131:0/3215959867 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68a00f4e20 con 0x5593559a6890 2024-09-17T02:50:31.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68a57fa640 1 -- 172.21.15.131:0/3215959867 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a011a2d0 con 0x5593559a6890 2024-09-17T02:50:31.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 -- 172.21.15.131:0/3215959867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 msgr2=0x559355f31700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3215959867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355f31700 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f68a00f88b0 tx=0x7f68a00fb040 comp rx=0 tx=0).stop 2024-09-17T02:50:31.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 -- 172.21.15.131:0/3215959867 shutdown_connections 2024-09-17T02:50:31.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3215959867 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355d4f6d0 0x559355f32180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3215959867 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559355a6aaa0 0x559355f31c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3215959867 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355f31700 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 -- 172.21.15.131:0/3215959867 >> 172.21.15.131:0/3215959867 conn(0x55935597a5c0 msgr2=0x559355e70360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 -- 172.21.15.131:0/3215959867 shutdown_connections 2024-09-17T02:50:31.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 -- 172.21.15.131:0/3215959867 wait complete. 2024-09-17T02:50:31.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 -- start start 2024-09-17T02:50:31.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.907+0000 7f68bdb16000 1 Processor -- start 2024-09-17T02:50:31.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355ed5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 0x559355ed56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559355d4f6d0 0x559355ef1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559355cc0100 con 0x5593559a6890 2024-09-17T02:50:31.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559355e7bd60 con 0x559355d4f6d0 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559355c54300 con 0x559355a6aaa0 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 10 client.0 _ll_register_callbacks cb 0x7f68a8001850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68affff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355ed5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68affff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355ed5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40212/0 (socket says 172.21.15.131:40212) 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68affff640 1 -- 172.21.15.131:0/3110436618 learned_addr learned my addr 172.21.15.131:0/3110436618 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68affff640 1 -- 172.21.15.131:0/3110436618 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 msgr2=0x559355ed56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68affff640 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 0x559355ed56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68affff640 1 -- 172.21.15.131:0/3110436618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559355d4f6d0 msgr2=0x559355ef1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b4de8640 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559355d4f6d0 0x559355ef1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68affff640 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559355d4f6d0 0x559355ef1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68affff640 1 -- 172.21.15.131:0/3110436618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559355c870b0 con 0x5593559a6890 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f688effd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68affff640 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355ed5180 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f68a00f5e20 tx=0x7f68a00f8a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388d4514233032bc server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:50:31.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f688effd640 1 -- 172.21.15.131:0/3110436618 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a011e070 con 0x5593559a6890 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f688effd640 1 -- 172.21.15.131:0/3110436618 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 6101+0+0 (secure 0 0 0) 0x7f68a011a2b0 con 0x5593559a6890 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 -- 172.21.15.131:0/3110436618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a00f4e20 con 0x5593559a6890 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f688effd640 1 -- 172.21.15.131:0/3110436618 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f68a011e210 con 0x5593559a6890 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 10 client.8130 fetch_fsmap learned FSMap version 86 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 -- 172.21.15.131:0/3110436618 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x559355c53610 con 0x5593559a6890 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f688effd640 1 -- 172.21.15.131:0/3110436618 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68a00f4e20 con 0x5593559a6890 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f688effd640 1 -- 172.21.15.131:0/3110436618 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7f68a00a7070 con 0x5593559a6890 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 10 client.8130 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 10 client.8130 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 20 client.8130 populate_metadata read hostname 'smithi131' 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55935594d990 0x559355ef24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 4 client.8130 mds_command: new command op to 7860 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:50:31.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 -- 172.21.15.131:0/3110436618 --> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559355ccb5b0 con 0x55935594d990 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68a77fe640 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55935594d990 0x559355ef24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f688effd640 10 client.8130 ms_handle_connect on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68a77fe640 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55935594d990 0x559355ef24c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1bae359e25f3774 server_cookie=b1ecd6d5a0681de9 in_seq=0 out_seq=0 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f688effd640 1 -- 172.21.15.131:0/3110436618 <== mds.0 v2:172.21.15.131:6833/2930400858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+571 (crc 0 0 0) 0x559355ccb5b0 con 0x55935594d990 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f688effd640 10 client.8130 handle_command_reply: tid=0 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 client.8130 shutdown 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 -- 172.21.15.131:0/3110436618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 msgr2=0x559355ed5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355ed5180 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f68a00f5e20 tx=0x7f68a00f8a40 comp rx=0 tx=0).stop 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 -- 172.21.15.131:0/3110436618 shutdown_connections 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55935594d990 0x559355ef24c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559355d4f6d0 0x559355ef1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559355a6aaa0 0x559355ed56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 --2- 172.21.15.131:0/3110436618 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5593559a6890 0x559355ed5180 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f688effd640 0 client.8130 ms_handle_reset on v2:172.21.15.131:6833/2930400858 2024-09-17T02:50:31.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 -- 172.21.15.131:0/3110436618 >> 172.21.15.131:0/3110436618 conn(0x55935597a5c0 msgr2=0x559355e70360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 -- 172.21.15.131:0/3110436618 shutdown_connections 2024-09-17T02:50:31.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 1 -- 172.21.15.131:0/3110436618 wait complete. 2024-09-17T02:50:31.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68bdb16000 20 client.8130 trim_cache size 0 max 16384 2024-09-17T02:50:31.914 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:50:31.914 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-17T02:50:31.914 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-09-17T02:50:31.914 INFO:teuthology.orchestra.run.smithi131.stdout: "ea4d4c00-a7c4-41e2-be37-392148866f69": { 2024-09-17T02:50:31.914 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:50:31.914 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "ea4d4c00-a7c4-41e2-be37-392148866f69", 2024-09-17T02:50:31.914 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,repair" 2024-09-17T02:50:31.915 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:50:31.915 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:50:31.915 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:50:31.915 INFO:teuthology.orchestra.run.smithi131.stdout: "ea4d4c00-a7c4-41e2-be37-392148866f69": { 2024-09-17T02:50:31.915 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:50:31.699931+0000", 2024-09-17T02:50:31.915 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:50:31.915 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:50:31.915 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:50:31.915 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:50:31.915 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:50:31.915 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:50:31.916 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:50:31.916 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:50:31.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 -- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6890073200 msgr2=0x7f6890075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6890073200 0x7f6890075710 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6898005fd0 tx=0x7f6898005e00 comp rx=0 tx=0).stop 2024-09-17T02:50:31.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 -- 172.21.15.131:0/2155653301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b00cfac0 msgr2=0x7f68b014a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:50:31.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b00cfac0 0x7f68b014a110 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f68a0000c10 tx=0x7f68a00a8600 comp rx=0 tx=0).stop 2024-09-17T02:50:31.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 -- 172.21.15.131:0/2155653301 shutdown_connections 2024-09-17T02:50:31.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6890073200 0x7f6890075710 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f68b015f4b0 0x7f68b0150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f68b00cfed0 0x7f68b014dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 --2- 172.21.15.131:0/2155653301 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f68b00cfac0 0x7f68b014a110 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:50:31.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 -- 172.21.15.131:0/2155653301 >> 172.21.15.131:0/2155653301 conn(0x7f68b00d13d0 msgr2=0x7f68b01594f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:50:31.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 -- 172.21.15.131:0/2155653301 shutdown_connections 2024-09-17T02:50:31.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:31.911+0000 7f68b684b640 1 -- 172.21.15.131:0/2155653301 wait complete. 2024-09-17T02:50:31.929 INFO:tasks.cephfs.filesystem:scrub status for tag:ea4d4c00-a7c4-41e2-be37-392148866f69 - {'path': '/', 'tag': 'ea4d4c00-a7c4-41e2-be37-392148866f69', 'options': 'recursive,repair'} 2024-09-17T02:50:33.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:33.259+0000 7f9809ffb640 1 -- 172.21.15.131:0/1414488223 <== mon.0 v2:172.21.15.131:3300/0 7 ==== log(4 entries from seq 1 at 2024-09-17T02:50:31.705247+0000) ==== 695+0+0 (secure 0 0 0) 0x7f9804007e00 con 0x7f98140b44b0 2024-09-17T02:50:33.257 INFO:teuthology.orchestra.run.smithi131.stdout: cluster: 2024-09-17T02:50:33.257 INFO:teuthology.orchestra.run.smithi131.stdout: id: f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:50:33.257 INFO:teuthology.orchestra.run.smithi131.stdout: health: HEALTH_OK 2024-09-17T02:50:33.257 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:33.257 INFO:teuthology.orchestra.run.smithi131.stdout: services: 2024-09-17T02:50:33.257 INFO:teuthology.orchestra.run.smithi131.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-09-17T02:50:33.257 INFO:teuthology.orchestra.run.smithi131.stdout: mgr: x(active, since 10m), standbys: y 2024-09-17T02:50:33.257 INFO:teuthology.orchestra.run.smithi131.stdout: mds: 1/1 daemons up, 3 standby 2024-09-17T02:50:33.257 INFO:teuthology.orchestra.run.smithi131.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 10m) 2024-09-17T02:50:33.257 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout: data: 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout: volumes: 1/1 healthy 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout: pools: 3 pools, 97 pgs 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout: objects: 30 objects, 19 MiB 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout: pgs: 97 active+clean 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout: io: 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 2 op/s wr 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:50:33.258 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:50:31.705247+0000 mds.a [INF] scrub queued for path: / 2024-09-17T02:50:33.259 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:50:31.705252+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-09-17T02:50:33.259 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:50:31.705258+0000 mds.a [INF] scrub summary: active paths [/] 2024-09-17T02:50:33.259 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:50:31.707182+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-09-17T02:50:34.268 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:50:33.628621+0000 mds.a [INF] scrub summary: idle 2024-09-17T02:50:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:34.271+0000 7f9809ffb640 1 -- 172.21.15.131:0/1414488223 <== mon.0 v2:172.21.15.131:3300/0 8 ==== log(1 entries from seq 5 at 2024-09-17T02:50:33.628621+0000) ==== 189+0+0 (secure 0 0 0) 0x7f9804092d90 con 0x7f98140b44b0 2024-09-17T02:50:35.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:35.275+0000 7f9809ffb640 1 -- 172.21.15.131:0/1414488223 <== mon.0 v2:172.21.15.131:3300/0 9 ==== log(8 entries from seq 157 at 2024-09-17T02:50:34.534735+0000) ==== 2206+0+0 (secure 0 0 0) 0x7f9804095000 con 0x7f98140b44b0 2024-09-17T02:50:36.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:50:36.299+0000 7f9809ffb640 1 -- 172.21.15.131:0/1414488223 <== mon.0 v2:172.21.15.131:3300/0 10 ==== log(4 entries from seq 962 at 2024-09-17T02:50:35.288589+0000) ==== 1082+0+0 (secure 0 0 0) 0x7f980405b2e0 con 0x7f98140b44b0 2024-09-17T02:50:45.521 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:50:45.525 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:51:01.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:51:01.931 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-17T02:51:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.103+0000 7fc682622640 1 -- 172.21.15.131:0/1716710303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc67c0cec00 msgr2=0x7fc67c0cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.103+0000 7fc682622640 1 --2- 172.21.15.131:0/1716710303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc67c0cec00 0x7fc67c0cf010 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fc670009840 tx=0x7fc67001c790 comp rx=0 tx=0).stop 2024-09-17T02:51:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.103+0000 7fc682622640 1 -- 172.21.15.131:0/1716710303 shutdown_connections 2024-09-17T02:51:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.103+0000 7fc682622640 1 --2- 172.21.15.131:0/1716710303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc67c0cec00 0x7fc67c0cf010 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.103+0000 7fc682622640 1 -- 172.21.15.131:0/1716710303 >> 172.21.15.131:0/1716710303 conn(0x7fc67c06eea0 msgr2=0x7fc67c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.103+0000 7fc682622640 1 -- 172.21.15.131:0/1716710303 shutdown_connections 2024-09-17T02:51:02.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.103+0000 7fc682622640 1 -- 172.21.15.131:0/1716710303 wait complete. 2024-09-17T02:51:02.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc682622640 1 Processor -- start 2024-09-17T02:51:02.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc682622640 1 -- start start 2024-09-17T02:51:02.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc682622640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc67c06e2c0 0x7fc67c1f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc682622640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc67c0c4c90 0x7fc67c158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc682622640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc67c0cec00 0x7fc67c0c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc682622640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c16ae80 con 0x7fc67c0cec00 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc682622640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c16ad00 con 0x7fc67c06e2c0 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc682622640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc67c16b000 con 0x7fc67c0c4c90 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc680bbf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc67c0cec00 0x7fc67c0c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc67bfff640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc67c06e2c0 0x7fc67c1f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc680bbf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc67c0cec00 0x7fc67c0c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46762/0 (socket says 172.21.15.131:46762) 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc680bbf640 1 -- 172.21.15.131:0/2223447894 learned_addr learned my addr 172.21.15.131:0/2223447894 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc67b7fe640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc67c0c4c90 0x7fc67c158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc680bbf640 1 -- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc67c0c4c90 msgr2=0x7fc67c158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc680bbf640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc67c0c4c90 0x7fc67c158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc680bbf640 1 -- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc67c06e2c0 msgr2=0x7fc67c1f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc680bbf640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc67c06e2c0 0x7fc67c1f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc680bbf640 1 -- 172.21.15.131:0/2223447894 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67c0c9e60 con 0x7fc67c0cec00 2024-09-17T02:51:02.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc67b7fe640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc67c0c4c90 0x7fc67c158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:02.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc680bbf640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc67c0cec00 0x7fc67c0c7870 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc67400c760 tx=0x7fc67400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc678ff9640 1 -- 172.21.15.131:0/2223447894 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674003db0 con 0x7fc67c0cec00 2024-09-17T02:51:02.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc678ff9640 1 -- 172.21.15.131:0/2223447894 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc67400ce10 con 0x7fc67c0cec00 2024-09-17T02:51:02.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc678ff9640 1 -- 172.21.15.131:0/2223447894 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6740052d0 con 0x7fc67c0cec00 2024-09-17T02:51:02.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc682622640 1 -- 172.21.15.131:0/2223447894 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67c14fcf0 con 0x7fc67c0cec00 2024-09-17T02:51:02.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.107+0000 7fc682622640 1 -- 172.21.15.131:0/2223447894 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67c1502b0 con 0x7fc67c0cec00 2024-09-17T02:51:02.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.111+0000 7fc678ff9640 1 -- 172.21.15.131:0/2223447894 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc674005470 con 0x7fc67c0cec00 2024-09-17T02:51:02.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.111+0000 7fc678ff9640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc668073200 0x7fc668075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.111+0000 7fc678ff9640 1 -- 172.21.15.131:0/2223447894 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6217+0+0 (secure 0 0 0) 0x7fc674093d70 con 0x7fc67c0cec00 2024-09-17T02:51:02.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.111+0000 7fc67bfff640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc668073200 0x7fc668075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.111+0000 7fc6898ed000 1 Processor -- start 2024-09-17T02:51:02.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.111+0000 7fc6898ed000 1 -- start start 2024-09-17T02:51:02.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.111+0000 7fc67bfff640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc668073200 0x7fc668075710 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc66c007fd0 tx=0x7fc66c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1182a26d0 0x55e11840e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 0x55e11840ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117fbdaa0 0x55e11840f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e1182adcd0 con 0x55e117fbdaa0 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e1181c3330 con 0x55e117ef9890 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55e118220bb0 con 0x55e1182a26d0 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc680bbf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117fbdaa0 0x55e11840f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc680bbf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117fbdaa0 0x55e11840f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46766/0 (socket says 172.21.15.131:46766) 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc680bbf640 1 -- 172.21.15.131:0/1281294595 learned_addr learned my addr 172.21.15.131:0/1281294595 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc67b7fe640 1 --2- 172.21.15.131:0/1281294595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 0x55e11840ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc67bfff640 1 --2- 172.21.15.131:0/1281294595 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1182a26d0 0x55e11840e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc680bbf640 1 -- 172.21.15.131:0/1281294595 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1182a26d0 msgr2=0x55e11840e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc680bbf640 1 --2- 172.21.15.131:0/1281294595 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1182a26d0 0x55e11840e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc680bbf640 1 -- 172.21.15.131:0/1281294595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 msgr2=0x55e11840ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc680bbf640 1 --2- 172.21.15.131:0/1281294595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 0x55e11840ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc680bbf640 1 -- 172.21.15.131:0/1281294595 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1181c4c80 con 0x55e117fbdaa0 2024-09-17T02:51:02.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc67b7fe640 1 --2- 172.21.15.131:0/1281294595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 0x55e11840ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:02.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc67bfff640 1 --2- 172.21.15.131:0/1281294595 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1182a26d0 0x55e11840e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:02.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc680bbf640 1 --2- 172.21.15.131:0/1281294595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117fbdaa0 0x55e11840f470 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc674003fd0 tx=0x7fc67405dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f12a27e84a0a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc661ffb640 1 -- 172.21.15.131:0/1281294595 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67405e5b0 con 0x55e117fbdaa0 2024-09-17T02:51:02.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc661ffb640 1 -- 172.21.15.131:0/1281294595 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc674063090 con 0x55e117fbdaa0 2024-09-17T02:51:02.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc661ffb640 1 -- 172.21.15.131:0/1281294595 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674067f40 con 0x55e117fbdaa0 2024-09-17T02:51:02.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 -- 172.21.15.131:0/1281294595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117fbdaa0 msgr2=0x55e11840f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1281294595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117fbdaa0 0x55e11840f470 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc674003fd0 tx=0x7fc67405dd20 comp rx=0 tx=0).stop 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 -- 172.21.15.131:0/1281294595 shutdown_connections 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1281294595 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117fbdaa0 0x55e11840f470 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1281294595 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 0x55e11840ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1281294595 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1182a26d0 0x55e11840e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 -- 172.21.15.131:0/1281294595 >> 172.21.15.131:0/1281294595 conn(0x55e117ecd5c0 msgr2=0x55e1183cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 -- 172.21.15.131:0/1281294595 shutdown_connections 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 -- 172.21.15.131:0/1281294595 wait complete. 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 -- start start 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.115+0000 7fc6898ed000 1 Processor -- start 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117ef9890 0x55e1183caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117fbdaa0 0x55e1183caf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1182a26d0 0x55e1183d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e118213df0 con 0x55e117ef9890 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55e11820ffe0 con 0x55e117fbdaa0 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55e1181b54d0 con 0x55e1182a26d0 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 10 client.0 _ll_register_callbacks cb 0x55e1181937f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc67bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117ef9890 0x55e1183caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc67bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117ef9890 0x55e1183caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46776/0 (socket says 172.21.15.131:46776) 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc67bfff640 1 -- 172.21.15.131:0/1129843315 learned_addr learned my addr 172.21.15.131:0/1129843315 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc680bbf640 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1182a26d0 0x55e1183d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc67bfff640 1 -- 172.21.15.131:0/1129843315 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1182a26d0 msgr2=0x55e1183d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc67bfff640 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1182a26d0 0x55e1183d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc67bfff640 1 -- 172.21.15.131:0/1129843315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117fbdaa0 msgr2=0x55e1183caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc67bfff640 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117fbdaa0 0x55e1183caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc67bfff640 1 -- 172.21.15.131:0/1129843315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1181ae210 con 0x55e117ef9890 2024-09-17T02:51:02.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc67bfff640 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117ef9890 0x55e1183caa00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0146d0 tx=0x7fc66c001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978fb055d6c761de server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc65affd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:51:02.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc65affd640 1 -- 172.21.15.131:0/1129843315 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc66c013680 con 0x55e117ef9890 2024-09-17T02:51:02.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc65affd640 1 -- 172.21.15.131:0/1129843315 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6217+0+0 (secure 0 0 0) 0x7fc66c013820 con 0x55e117ef9890 2024-09-17T02:51:02.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 -- 172.21.15.131:0/1129843315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1181a8350 con 0x55e117ef9890 2024-09-17T02:51:02.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc65affd640 1 -- 172.21.15.131:0/1129843315 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc66c02d050 con 0x55e117ef9890 2024-09-17T02:51:02.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 10 client.8145 fetch_fsmap learned FSMap version 86 2024-09-17T02:51:02.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 -- 172.21.15.131:0/1129843315 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x55e118194fc0 con 0x55e117ef9890 2024-09-17T02:51:02.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc65affd640 1 -- 172.21.15.131:0/1129843315 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55e1181a8350 con 0x55e117ef9890 2024-09-17T02:51:02.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc65affd640 1 -- 172.21.15.131:0/1129843315 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7fc66c030020 con 0x55e117ef9890 2024-09-17T02:51:02.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 10 client.8145 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T02:51:02.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 10 client.8145 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 20 client.8145 populate_metadata read hostname 'smithi131' 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55e117eb2930 0x55e118415e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 4 client.8145 mds_command: new command op to 7860 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 -- 172.21.15.131:0/1129843315 --> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e118222b00 con 0x55e117eb2930 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc67b7fe640 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55e117eb2930 0x55e118415e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc65affd640 10 client.8145 ms_handle_connect on v2:172.21.15.131:6833/2930400858 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc67b7fe640 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55e117eb2930 0x55e118415e80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e3e6c03af53b305 server_cookie=de07b2599e811041 in_seq=0 out_seq=0 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc65affd640 1 -- 172.21.15.131:0/1129843315 <== mds.0 v2:172.21.15.131:6833/2930400858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e118222b00 con 0x55e117eb2930 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc65affd640 10 client.8145 handle_command_reply: tid=0 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 client.8145 shutdown 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 -- 172.21.15.131:0/1129843315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117ef9890 msgr2=0x55e1183caa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117ef9890 0x55e1183caa00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0146d0 tx=0x7fc66c001090 comp rx=0 tx=0).stop 2024-09-17T02:51:02.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 -- 172.21.15.131:0/1129843315 shutdown_connections 2024-09-17T02:51:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55e117eb2930 0x55e118415e80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e1182a26d0 0x55e1183d15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117fbdaa0 0x55e1183caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1129843315 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e117ef9890 0x55e1183caa00 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 -- 172.21.15.131:0/1129843315 >> 172.21.15.131:0/1129843315 conn(0x55e117ecd5c0 msgr2=0x55e1183c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc65affd640 0 client.8145 ms_handle_reset on v2:172.21.15.131:6833/2930400858 2024-09-17T02:51:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 -- 172.21.15.131:0/1129843315 shutdown_connections 2024-09-17T02:51:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 1 -- 172.21.15.131:0/1129843315 wait complete. 2024-09-17T02:51:02.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.119+0000 7fc6898ed000 20 client.8145 trim_cache size 0 max 16384 2024-09-17T02:51:02.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc6898ed000 1 Processor -- start 2024-09-17T02:51:02.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc6898ed000 1 -- start start 2024-09-17T02:51:02.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 0x55e118480400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e117fbdaa0 0x55e118480940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e118480e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc6898ed000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55e1183c9b20 con 0x55e1182a26d0 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc6898ed000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c205880 con 0x55e117ef9890 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc6898ed000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc67c1fc560 con 0x55e117fbdaa0 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc680bbf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e118480e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc680bbf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e118480e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46788/0 (socket says 172.21.15.131:46788) 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc680bbf640 1 -- 172.21.15.131:0/1676578457 learned_addr learned my addr 172.21.15.131:0/1676578457 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc67b7fe640 1 --2- 172.21.15.131:0/1676578457 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e117fbdaa0 0x55e118480940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.139+0000 7fc67bfff640 1 --2- 172.21.15.131:0/1676578457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 0x55e118480400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc680bbf640 1 -- 172.21.15.131:0/1676578457 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e117fbdaa0 msgr2=0x55e118480940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc680bbf640 1 --2- 172.21.15.131:0/1676578457 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e117fbdaa0 0x55e118480940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc680bbf640 1 -- 172.21.15.131:0/1676578457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 msgr2=0x55e118480400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc680bbf640 1 --2- 172.21.15.131:0/1676578457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 0x55e118480400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc680bbf640 1 -- 172.21.15.131:0/1676578457 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e11826d680 con 0x55e1182a26d0 2024-09-17T02:51:02.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc67b7fe640 1 --2- 172.21.15.131:0/1676578457 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e117fbdaa0 0x55e118480940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:02.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc67bfff640 1 --2- 172.21.15.131:0/1676578457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 0x55e118480400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:02.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc680bbf640 1 --2- 172.21.15.131:0/1676578457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e118480e80 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc67405a5c0 tx=0x7fc674092000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2bcbd20bf9282b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6617fa640 1 -- 172.21.15.131:0/1676578457 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67407c020 con 0x55e1182a26d0 2024-09-17T02:51:02.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6617fa640 1 -- 172.21.15.131:0/1676578457 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc674063090 con 0x55e1182a26d0 2024-09-17T02:51:02.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6617fa640 1 -- 172.21.15.131:0/1676578457 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674067ab0 con 0x55e1182a26d0 2024-09-17T02:51:02.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 -- 172.21.15.131:0/1676578457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 msgr2=0x55e118480e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1676578457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e118480e80 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc67405a5c0 tx=0x7fc674092000 comp rx=0 tx=0).stop 2024-09-17T02:51:02.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 -- 172.21.15.131:0/1676578457 shutdown_connections 2024-09-17T02:51:02.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1676578457 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e118480e80 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1676578457 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e117fbdaa0 0x55e118480940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1676578457 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117ef9890 0x55e118480400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 -- 172.21.15.131:0/1676578457 >> 172.21.15.131:0/1676578457 conn(0x55e117ecd5c0 msgr2=0x55e118420fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 -- 172.21.15.131:0/1676578457 shutdown_connections 2024-09-17T02:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 -- 172.21.15.131:0/1676578457 wait complete. 2024-09-17T02:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 -- start start 2024-09-17T02:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 Processor -- start 2024-09-17T02:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e117ef9890 0x55e118484650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117fbdaa0 0x55e118484b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e1184850d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c151560 con 0x55e1182a26d0 2024-09-17T02:51:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c150f10 con 0x55e117fbdaa0 2024-09-17T02:51:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55e117ed5680 con 0x55e117ef9890 2024-09-17T02:51:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc6898ed000 10 client.0 _ll_register_callbacks cb 0x7fc66c017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:51:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc680bbf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e1184850d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc680bbf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e1184850d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46804/0 (socket says 172.21.15.131:46804) 2024-09-17T02:51:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc680bbf640 1 -- 172.21.15.131:0/1664542220 learned_addr learned my addr 172.21.15.131:0/1664542220 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.143+0000 7fc67bfff640 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e117ef9890 0x55e118484650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc67b7fe640 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117fbdaa0 0x55e118484b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc680bbf640 1 -- 172.21.15.131:0/1664542220 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e117ef9890 msgr2=0x55e118484650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc680bbf640 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e117ef9890 0x55e118484650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc680bbf640 1 -- 172.21.15.131:0/1664542220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117fbdaa0 msgr2=0x55e118484b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc680bbf640 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117fbdaa0 0x55e118484b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc680bbf640 1 -- 172.21.15.131:0/1664542220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1181da0b0 con 0x55e1182a26d0 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc67b7fe640 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117fbdaa0 0x55e118484b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc680bbf640 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e1184850d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc67405a5c0 tx=0x7fc67405e140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333826cc0a1652ce server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc65affd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc65affd640 1 -- 172.21.15.131:0/1664542220 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67407c040 con 0x55e1182a26d0 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc65affd640 1 -- 172.21.15.131:0/1664542220 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 6217+0+0 (secure 0 0 0) 0x7fc67405d810 con 0x55e1182a26d0 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 -- 172.21.15.131:0/1664542220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc674063090 con 0x55e1182a26d0 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc65affd640 1 -- 172.21.15.131:0/1664542220 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fc674005820 con 0x55e1182a26d0 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 10 client.8151 fetch_fsmap learned FSMap version 86 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 -- 172.21.15.131:0/1664542220 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) -- 0x55e1181a6610 con 0x55e1182a26d0 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc65affd640 1 -- 172.21.15.131:0/1664542220 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc674063090 con 0x55e1182a26d0 2024-09-17T02:51:02.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc65affd640 1 -- 172.21.15.131:0/1664542220 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 86) ==== 3502+0+0 (secure 0 0 0) 0x7fc674074170 con 0x55e1182a26d0 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 10 client.8151 fetch_fsmap finished waiting for FSMap version 86 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 10 client.8151 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 20 client.8151 populate_metadata read hostname 'smithi131' 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55e117ea0990 0x55e1183c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 4 client.8151 mds_command: new command op to 7860 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 -- 172.21.15.131:0/1664542220 --> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e11821e5b0 con 0x55e117ea0990 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc67bfff640 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55e117ea0990 0x55e1183c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc65affd640 10 client.8151 ms_handle_connect on v2:172.21.15.131:6833/2930400858 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc67bfff640 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55e117ea0990 0x55e1183c58c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55c85e36d422bf7d server_cookie=ea0b671bfd12e44d in_seq=0 out_seq=0 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc65affd640 1 -- 172.21.15.131:0/1664542220 <== mds.0 v2:172.21.15.131:6833/2930400858 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55e11821e5b0 con 0x55e117ea0990 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc65affd640 10 client.8151 handle_command_reply: tid=0 2024-09-17T02:51:02.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 client.8151 shutdown 2024-09-17T02:51:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 -- 172.21.15.131:0/1664542220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 msgr2=0x55e1184850d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e1184850d0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc67405a5c0 tx=0x7fc67405e140 comp rx=0 tx=0).stop 2024-09-17T02:51:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 -- 172.21.15.131:0/1664542220 shutdown_connections 2024-09-17T02:51:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.131:6833/2930400858,v1:172.21.15.131:6835/2930400858] conn(0x55e117ea0990 0x55e1183c58c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55e1182a26d0 0x55e1184850d0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55e117fbdaa0 0x55e118484b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc65affd640 0 client.8151 ms_handle_reset on v2:172.21.15.131:6833/2930400858 2024-09-17T02:51:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 --2- 172.21.15.131:0/1664542220 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55e117ef9890 0x55e118484650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 -- 172.21.15.131:0/1664542220 >> 172.21.15.131:0/1664542220 conn(0x55e117ecd5c0 msgr2=0x55e1183d05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.155 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 -- 172.21.15.131:0/1664542220 shutdown_connections 2024-09-17T02:51:02.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 1 -- 172.21.15.131:0/1664542220 wait complete. 2024-09-17T02:51:02.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc6898ed000 20 client.8151 trim_cache size 0 max 16384 2024-09-17T02:51:02.156 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:51:02.156 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-09-17T02:51:02.156 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {}, 2024-09-17T02:51:02.157 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:51:02.157 INFO:teuthology.orchestra.run.smithi131.stdout: "ea4d4c00-a7c4-41e2-be37-392148866f69": { 2024-09-17T02:51:02.157 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:50:31.699931+0000", 2024-09-17T02:51:02.157 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:51:02.157 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:51:02.157 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:51:02.157 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:51:02.158 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:51:02.158 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:02.158 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:02.158 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:51:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 -- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc668073200 msgr2=0x7fc668075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc668073200 0x7fc668075710 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc66c007fd0 tx=0x7fc66c009040 comp rx=0 tx=0).stop 2024-09-17T02:51:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 -- 172.21.15.131:0/2223447894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc67c0cec00 msgr2=0x7fc67c0c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc67c0cec00 0x7fc67c0c7870 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc67400c760 tx=0x7fc67400cc20 comp rx=0 tx=0).stop 2024-09-17T02:51:02.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 -- 172.21.15.131:0/2223447894 shutdown_connections 2024-09-17T02:51:02.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc668073200 0x7fc668075710 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc67c0cec00 0x7fc67c0c7870 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc67c0c4c90 0x7fc67c158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 --2- 172.21.15.131:0/2223447894 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc67c06e2c0 0x7fc67c1f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 -- 172.21.15.131:0/2223447894 >> 172.21.15.131:0/2223447894 conn(0x7fc67c06eea0 msgr2=0x7fc67c15a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 -- 172.21.15.131:0/2223447894 shutdown_connections 2024-09-17T02:51:02.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.147+0000 7fc682622640 1 -- 172.21.15.131:0/2223447894 wait complete. 2024-09-17T02:51:02.165 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-17T02:51:02.266 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:51:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.431+0000 7fd914088640 1 -- 172.21.15.131:0/3696026262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f8004810 msgr2=0x7fd8f8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.431+0000 7fd914088640 1 --2- 172.21.15.131:0/3696026262 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f8004810 0x7fd8f8004c20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009b30 tx=0x7fd8fc01d270 comp rx=0 tx=0).stop 2024-09-17T02:51:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.431+0000 7fd912625640 1 -- 172.21.15.131:0/3696026262 reap_dead start 2024-09-17T02:51:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.431+0000 7fd914088640 1 -- 172.21.15.131:0/3696026262 shutdown_connections 2024-09-17T02:51:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.431+0000 7fd914088640 1 -- 172.21.15.131:0/3696026262 >> 172.21.15.131:0/3696026262 conn(0x7fd90c0d1130 msgr2=0x7fd90c0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.431+0000 7fd914088640 1 -- 172.21.15.131:0/3696026262 shutdown_connections 2024-09-17T02:51:02.430 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.431+0000 7fd914088640 1 -- 172.21.15.131:0/3696026262 wait complete. 2024-09-17T02:51:02.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd914088640 1 Processor -- start 2024-09-17T02:51:02.431 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd914088640 1 -- start start 2024-09-17T02:51:02.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd914088640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f8004810 0x7fd90c1f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd914088640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd90c1582c0 0x7fd90c0c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd914088640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd90c0c6ef0 0x7fd90c1f7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd914088640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c160500 con 0x7fd8f8004810 2024-09-17T02:51:02.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd914088640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd90c0d0f00 con 0x7fd90c0c6ef0 2024-09-17T02:51:02.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd914088640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd90c160680 con 0x7fd90c1582c0 2024-09-17T02:51:02.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd911e24640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f8004810 0x7fd90c1f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd911e24640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f8004810 0x7fd90c1f4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46816/0 (socket says 172.21.15.131:46816) 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd911e24640 1 -- 172.21.15.131:0/2112610816 learned_addr learned my addr 172.21.15.131:0/2112610816 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd911623640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd90c1582c0 0x7fd90c0c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd912625640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd90c0c6ef0 0x7fd90c1f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd911e24640 1 -- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd90c1582c0 msgr2=0x7fd90c0c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd911e24640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd90c1582c0 0x7fd90c0c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd911e24640 1 -- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd90c0c6ef0 msgr2=0x7fd90c1f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd911e24640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd90c0c6ef0 0x7fd90c1f7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd911e24640 1 -- 172.21.15.131:0/2112610816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90c0c7300 con 0x7fd8f8004810 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd911e24640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f8004810 0x7fd90c1f4580 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd904010a40 tx=0x7fd904010f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd9027fc640 1 -- 172.21.15.131:0/2112610816 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90400ed40 con 0x7fd8f8004810 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd9027fc640 1 -- 172.21.15.131:0/2112610816 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd904004030 con 0x7fd8f8004810 2024-09-17T02:51:02.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd914088640 1 -- 172.21.15.131:0/2112610816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90c158d80 con 0x7fd8f8004810 2024-09-17T02:51:02.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd9027fc640 1 -- 172.21.15.131:0/2112610816 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd904004480 con 0x7fd8f8004810 2024-09-17T02:51:02.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.435+0000 7fd914088640 1 -- 172.21.15.131:0/2112610816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90c159340 con 0x7fd8f8004810 2024-09-17T02:51:02.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.439+0000 7fd914088640 1 -- 172.21.15.131:0/2112610816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90c0cdd10 con 0x7fd8f8004810 2024-09-17T02:51:02.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.439+0000 7fd9027fc640 1 -- 172.21.15.131:0/2112610816 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd904003600 con 0x7fd8f8004810 2024-09-17T02:51:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.439+0000 7fd9027fc640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8f0073200 0x7fd8f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.439+0000 7fd9027fc640 1 -- 172.21.15.131:0/2112610816 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 6217+0+0 (secure 0 0 0) 0x7fd904095500 con 0x7fd8f8004810 2024-09-17T02:51:02.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.439+0000 7fd911623640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8f0073200 0x7fd8f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.439+0000 7fd911623640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8f0073200 0x7fd8f0075710 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009b30 tx=0x7fd8fc01a010 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.440 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.443+0000 7fd9027fc640 1 -- 172.21.15.131:0/2112610816 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd90c0c7300 con 0x7fd8f8004810 2024-09-17T02:51:02.647 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.651+0000 7fd914088640 1 -- 172.21.15.131:0/2112610816 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd90c156ea0 con 0x7fd8f8004810 2024-09-17T02:51:02.648 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:51:02.651+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:51:02.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd9027fc640 1 -- 172.21.15.131:0/2112610816 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v87) ==== 161+0+0 (secure 0 0 0) 0x7fd904060520 con 0x7fd8f8004810 2024-09-17T02:51:02.650 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:51:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 -- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8f0073200 msgr2=0x7fd8f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8f0073200 0x7fd8f0075710 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc009b30 tx=0x7fd8fc01a010 comp rx=0 tx=0).stop 2024-09-17T02:51:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 -- 172.21.15.131:0/2112610816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f8004810 msgr2=0x7fd90c1f4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f8004810 0x7fd90c1f4580 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fd904010a40 tx=0x7fd904010f00 comp rx=0 tx=0).stop 2024-09-17T02:51:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 -- 172.21.15.131:0/2112610816 shutdown_connections 2024-09-17T02:51:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd8f0073200 0x7fd8f0075710 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd90c0c6ef0 0x7fd90c1f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd90c1582c0 0x7fd90c0c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 --2- 172.21.15.131:0/2112610816 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd8f8004810 0x7fd90c1f4580 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 -- 172.21.15.131:0/2112610816 >> 172.21.15.131:0/2112610816 conn(0x7fd90c0d1130 msgr2=0x7fd90c152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 -- 172.21.15.131:0/2112610816 shutdown_connections 2024-09-17T02:51:02.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.655+0000 7fd914088640 1 -- 172.21.15.131:0/2112610816 wait complete. 2024-09-17T02:51:02.671 INFO:tasks.ceph.mds.a.smithi131.stderr: -7094> 2024-09-17T02:50:09.112+0000 7ff265b46640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:51:02.676 DEBUG:teuthology.orchestra.run.smithi131:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-09-17T02:51:02.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 1 -- start start 2024-09-17T02:51:02.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 1 Processor -- start 2024-09-17T02:51:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53edf600 0x555e54bdeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edd200 0x555e54bdf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edcd80 0x555e54bde580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555e54d26000 con 0x555e53edcd80 2024-09-17T02:51:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555e54d26180 con 0x555e53edd200 2024-09-17T02:51:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555e54d26300 con 0x555e53edf600 2024-09-17T02:51:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a8bab640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edcd80 0x555e54bde580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a8bab640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edcd80 0x555e54bde580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46832/0 (socket says 172.21.15.131:46832) 2024-09-17T02:51:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a8bab640 1 -- 172.21.15.131:0/496216489 learned_addr learned my addr 172.21.15.131:0/496216489 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a7ba9640 1 --2- 172.21.15.131:0/496216489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edd200 0x555e54bdf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a83aa640 1 --2- 172.21.15.131:0/496216489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53edf600 0x555e54bdeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a8bab640 1 -- 172.21.15.131:0/496216489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53edf600 msgr2=0x555e54bdeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a8bab640 1 --2- 172.21.15.131:0/496216489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53edf600 0x555e54bdeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a8bab640 1 -- 172.21.15.131:0/496216489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edd200 msgr2=0x555e54bdf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a8bab640 1 --2- 172.21.15.131:0/496216489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edd200 0x555e54bdf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a8bab640 1 -- 172.21.15.131:0/496216489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e54bd6b40 con 0x555e53edcd80 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a8bab640 1 --2- 172.21.15.131:0/496216489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edcd80 0x555e54bde580 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x555e54c75980 tx=0x555e54d2e120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3038e48d0d66ec85 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a5ba5640 1 -- 172.21.15.131:0/496216489 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555e53e31380 con 0x555e53edcd80 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a5ba5640 1 -- 172.21.15.131:0/496216489 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555e53e4e8c0 con 0x555e53edcd80 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a5ba5640 1 -- 172.21.15.131:0/496216489 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555e53e311e0 con 0x555e53edcd80 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 1 -- 172.21.15.131:0/496216489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e54bd70e0 con 0x555e53edcd80 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 1 -- 172.21.15.131:0/496216489 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x555e54bd72c0 con 0x555e53edcd80 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 4 waiting for MDS map... 2024-09-17T02:51:02.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a5ba5640 1 -- 172.21.15.131:0/496216489 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x555e54bd8e00 con 0x555e53edcd80 2024-09-17T02:51:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a5ba5640 1 -- 172.21.15.131:0/496216489 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 87) ==== 2893+0+0 (secure 0 0 0) 0x555e53e51180 con 0x555e53edcd80 2024-09-17T02:51:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 4 Got MDS map 87 2024-09-17T02:51:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 10 main: JournalTool::main 2024-09-17T02:51:02.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.751+0000 7f37a9beb400 4 main: JournalTool: connecting to RADOS... 2024-09-17T02:51:02.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 Processor -- start 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 -- start start 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edc480 0x555e54bdf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edc000 0x555e54bdfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53ede880 0x555e54be0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555e54d26480 con 0x555e53edc000 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555e54d26600 con 0x555e53edc480 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555e54d26780 con 0x555e53ede880 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a83aa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edc000 0x555e54bdfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a83aa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edc000 0x555e54bdfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46838/0 (socket says 172.21.15.131:46838) 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a83aa640 1 -- 172.21.15.131:0/1107459960 learned_addr learned my addr 172.21.15.131:0/1107459960 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a7ba9640 1 --2- 172.21.15.131:0/1107459960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edc480 0x555e54bdf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a7ba9640 1 --2- 172.21.15.131:0/1107459960 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53ede880 0x555e54be0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a83aa640 1 -- 172.21.15.131:0/1107459960 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53ede880 msgr2=0x555e54be0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a83aa640 1 --2- 172.21.15.131:0/1107459960 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53ede880 0x555e54be0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a83aa640 1 -- 172.21.15.131:0/1107459960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edc480 msgr2=0x555e54bdf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a83aa640 1 --2- 172.21.15.131:0/1107459960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edc480 0x555e54bdf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a83aa640 1 -- 172.21.15.131:0/1107459960 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e54bd70e0 con 0x555e53edc000 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a83aa640 1 --2- 172.21.15.131:0/1107459960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edc000 0x555e54bdfb80 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x555e54d3cd80 tx=0x555e54d56c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=437e674731647b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a3ba1640 1 -- 172.21.15.131:0/1107459960 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555e53e31380 con 0x555e53edc000 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a3ba1640 1 -- 172.21.15.131:0/1107459960 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555e53e4ee00 con 0x555e53edc000 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a3ba1640 1 -- 172.21.15.131:0/1107459960 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555e53e311e0 con 0x555e53edc000 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 -- 172.21.15.131:0/1107459960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edc000 msgr2=0x555e54bdfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 --2- 172.21.15.131:0/1107459960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edc000 0x555e54bdfb80 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x555e54d3cd80 tx=0x555e54d56c00 comp rx=0 tx=0).stop 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 -- 172.21.15.131:0/1107459960 shutdown_connections 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 --2- 172.21.15.131:0/1107459960 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53ede880 0x555e54be0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 --2- 172.21.15.131:0/1107459960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edc000 0x555e54bdfb80 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 --2- 172.21.15.131:0/1107459960 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edc480 0x555e54bdf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 -- 172.21.15.131:0/1107459960 >> 172.21.15.131:0/1107459960 conn(0x555e53edc900 msgr2=0x555e5413c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 -- 172.21.15.131:0/1107459960 shutdown_connections 2024-09-17T02:51:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.755+0000 7f37a9beb400 1 -- 172.21.15.131:0/1107459960 wait complete. 2024-09-17T02:51:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a9beb400 1 Processor -- start 2024-09-17T02:51:02.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a9beb400 1 -- start start 2024-09-17T02:51:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a9beb400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edc000 0x555e54bdfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a9beb400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53edc480 0x555e54bdf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a9beb400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53ede880 0x555e54be0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a9beb400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555e54d27e00 con 0x555e53ede880 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a9beb400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555e54d27c80 con 0x555e53edc000 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a9beb400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555e54d27b00 con 0x555e53edc480 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a7ba9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53ede880 0x555e54be0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a7ba9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53ede880 0x555e54be0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46842/0 (socket says 172.21.15.131:46842) 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a7ba9640 1 -- 172.21.15.131:0/2921150458 learned_addr learned my addr 172.21.15.131:0/2921150458 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a7ba9640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edc000 0x555e54bdfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a83aa640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53edc480 0x555e54bdf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a7ba9640 1 -- 172.21.15.131:0/2921150458 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53edc480 msgr2=0x555e54bdf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a7ba9640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53edc480 0x555e54bdf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a7ba9640 1 -- 172.21.15.131:0/2921150458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edc000 msgr2=0x555e54bdfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a7ba9640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edc000 0x555e54bdfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a7ba9640 1 -- 172.21.15.131:0/2921150458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e54160000 con 0x555e53ede880 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a7ba9640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53ede880 0x555e54be0100 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x555e54c750b0 tx=0x555e54d56d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a239e640 1 -- 172.21.15.131:0/2921150458 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555e53e31520 con 0x555e53ede880 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a239e640 1 -- 172.21.15.131:0/2921150458 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555e53e4e000 con 0x555e53ede880 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555e541601e0 con 0x555e53ede880 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a239e640 1 -- 172.21.15.131:0/2921150458 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555e53e31380 con 0x555e53ede880 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e541605a0 con 0x555e53ede880 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.759+0000 7f37a9beb400 4 main: JournalTool: resolving pool 15 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a239e640 1 -- 172.21.15.131:0/2921150458 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x555e5415ec00 con 0x555e53ede880 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a239e640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x555e53ede400 0x555e54be0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a239e640 1 -- 172.21.15.131:0/2921150458 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x555e5415ca00 con 0x555e53ede880 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a9beb400 4 main: JournalTool: creating IoCtx.. 2024-09-17T02:51:02.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a9beb400 4 main: Executing for rank 0 2024-09-17T02:51:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a9beb400 4 Filtering by inode '1099511627777' 2024-09-17T02:51:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a9beb400 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x555e53eded00 0x555e54be0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x555e54156800 con 0x555e53eded00 2024-09-17T02:51:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a7ba9640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x555e53eded00 0x555e54be0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a83aa640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x555e53ede400 0x555e54be0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a7ba9640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x555e53eded00 0x555e54be0c00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a83aa640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x555e53ede400 0x555e54be0680 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x555e54d3cd80 tx=0x555e5422c420 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a7ba9640 1 -- 172.21.15.131:0/2921150458 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x555e53f04000 con 0x555e53eded00 2024-09-17T02:51:02.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a9beb400 4 Pointer 400.00000000 is readable 2024-09-17T02:51:02.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a9beb400 4 JournalScanner::scan: reading header object '200.00000000' 2024-09-17T02:51:02.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a9beb400 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x555e53eddb00 0x555e54be1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x555e54156c00 con 0x555e53eddb00 2024-09-17T02:51:02.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a8bab640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x555e53eddb00 0x555e54be1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.763+0000 7f37a8bab640 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x555e53eddb00 0x555e54be1180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a8bab640 1 -- 172.21.15.131:0/2921150458 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv29 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x555e53f04500 con 0x555e53eddb00 2024-09-17T02:51:02.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 10 Header 0x400000 0x405f0e 0x40628a 2024-09-17T02:51:02.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 10 Starting journal scan from offset 0x405f0e 2024-09-17T02:51:02.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x555e54157000 con 0x555e53eddb00 2024-09-17T02:51:02.765 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a8bab640 1 -- 172.21.15.131:0/2921150458 <== osd.3 v2:172.21.15.131:6816/4047536388 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv27 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x555e53f04500 con 0x555e53eddb00 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 10 Parsing data, 0x37c bytes available 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 10 Attempting decode at 0x405f0e 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 10 Consumed 0x37c bytes 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 10 Valid entry at 0x405f0e 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x555e54157400 con 0x555e53eddb00 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a8bab640 1 -- 172.21.15.131:0/2921150458 <== osd.3 v2:172.21.15.131:6816/4047536388 3 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x555e53f04500 con 0x555e53eddb00 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 4 Reached end of journal objects 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 4 Scanned objects, 0 missing, 1 valid 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 4 Events scanned, 0 gaps 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 4 Found 1 valid events 2024-09-17T02:51:02.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 4 Selected 0 events events for processing 2024-09-17T02:51:02.767 INFO:teuthology.orchestra.run.smithi131.stdout:Events by type: 2024-09-17T02:51:02.767 INFO:teuthology.orchestra.run.smithi131.stdout:Errors: 0 2024-09-17T02:51:02.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 4 0 events 2024-09-17T02:51:02.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x555e53eded00 msgr2=0x555e54be0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x555e53eded00 0x555e54be0c00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x555e53eddb00 msgr2=0x555e54be1180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x555e53eddb00 0x555e54be1180 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x555e53ede400 msgr2=0x555e54be0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x555e53ede400 0x555e54be0680 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x555e54d3cd80 tx=0x555e5422c420 comp rx=0 tx=0).stop 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53ede880 msgr2=0x555e54be0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 --2- 172.21.15.131:0/2921150458 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53ede880 0x555e54be0100 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x555e54c750b0 tx=0x555e54d56d20 comp rx=0 tx=0).stop 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a83aa640 1 -- 172.21.15.131:0/2921150458 reap_dead start 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a83aa640 1 -- 172.21.15.131:0/2921150458 reap_dead start 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 shutdown_connections 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 >> 172.21.15.131:0/2921150458 conn(0x555e53edc900 msgr2=0x555e5413c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 shutdown_connections 2024-09-17T02:51:02.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/2921150458 wait complete. 2024-09-17T02:51:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/496216489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edcd80 msgr2=0x555e54bde580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 --2- 172.21.15.131:0/496216489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edcd80 0x555e54bde580 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x555e54c75980 tx=0x555e54d2e120 comp rx=0 tx=0).stop 2024-09-17T02:51:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/496216489 shutdown_connections 2024-09-17T02:51:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 --2- 172.21.15.131:0/496216489 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555e53edcd80 0x555e54bde580 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 --2- 172.21.15.131:0/496216489 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555e53edd200 0x555e54bdf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 --2- 172.21.15.131:0/496216489 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555e53edf600 0x555e54bdeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/496216489 >> 172.21.15.131:0/496216489 conn(0x555e53eddf80 msgr2=0x555e4fdf7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/496216489 shutdown_connections 2024-09-17T02:51:02.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.767+0000 7f37a9beb400 1 -- 172.21.15.131:0/496216489 wait complete. 2024-09-17T02:51:02.770 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:51:02.773 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.096851, result: Events by type: Errors: 0 2024-09-17T02:51:02.773 DEBUG:teuthology.orchestra.run.smithi131:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-09-17T02:51:02.832 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.835+0000 7fcc10816400 1 -- start start 2024-09-17T02:51:02.833 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.835+0000 7fcc10816400 1 Processor -- start 2024-09-17T02:51:02.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.835+0000 7fcc10816400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6d200 0x555dfc76f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.835+0000 7fcc10816400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6cd80 0x555dfc76e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.835+0000 7fcc10816400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6c900 0x555dfc76e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.835+0000 7fcc10816400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555dfba76000 con 0x555dfba6c900 2024-09-17T02:51:02.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.835+0000 7fcc10816400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555dfba76180 con 0x555dfba6d200 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.835+0000 7fcc10816400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555dfba76300 con 0x555dfba6cd80 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.835+0000 7fcc0f7d6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6c900 0x555dfc76e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.835+0000 7fcc0f7d6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6c900 0x555dfc76e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46880/0 (socket says 172.21.15.131:46880) 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.835+0000 7fcc0f7d6640 1 -- 172.21.15.131:0/480104136 learned_addr learned my addr 172.21.15.131:0/480104136 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0e7d4640 1 --2- 172.21.15.131:0/480104136 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6cd80 0x555dfc76e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0efd5640 1 --2- 172.21.15.131:0/480104136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6d200 0x555dfc76f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0f7d6640 1 -- 172.21.15.131:0/480104136 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6cd80 msgr2=0x555dfc76e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0f7d6640 1 --2- 172.21.15.131:0/480104136 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6cd80 0x555dfc76e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0f7d6640 1 -- 172.21.15.131:0/480104136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6d200 msgr2=0x555dfc76f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0f7d6640 1 --2- 172.21.15.131:0/480104136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6d200 0x555dfc76f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0f7d6640 1 -- 172.21.15.131:0/480104136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dfbac8b40 con 0x555dfba6c900 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0f7d6640 1 --2- 172.21.15.131:0/480104136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6c900 0x555dfc76e000 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x555dfb9c3590 tx=0x555dfbaa7aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7aa4c901c6b9164 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.836 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0c7d0640 1 -- 172.21.15.131:0/480104136 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555dfb9c1520 con 0x555dfba6c900 2024-09-17T02:51:02.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0c7d0640 1 -- 172.21.15.131:0/480104136 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555dfb9de540 con 0x555dfba6c900 2024-09-17T02:51:02.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0c7d0640 1 -- 172.21.15.131:0/480104136 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555dfb9c1380 con 0x555dfba6c900 2024-09-17T02:51:02.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 1 -- 172.21.15.131:0/480104136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dfbac90e0 con 0x555dfba6c900 2024-09-17T02:51:02.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 1 -- 172.21.15.131:0/480104136 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x555dfbac92c0 con 0x555dfba6c900 2024-09-17T02:51:02.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 4 waiting for MDS map... 2024-09-17T02:51:02.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0c7d0640 1 -- 172.21.15.131:0/480104136 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x555df7957800 con 0x555dfba6c900 2024-09-17T02:51:02.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0c7d0640 1 -- 172.21.15.131:0/480104136 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 87) ==== 2893+0+0 (secure 0 0 0) 0x555dfb9e1180 con 0x555dfba6c900 2024-09-17T02:51:02.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 4 Got MDS map 87 2024-09-17T02:51:02.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 10 main: JournalTool::main 2024-09-17T02:51:02.837 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 4 main: JournalTool: connecting to RADOS... 2024-09-17T02:51:02.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 1 Processor -- start 2024-09-17T02:51:02.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 1 -- start start 2024-09-17T02:51:02.841 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6c000 0x555dfc76f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6e880 0x555dfc76fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 0x555dfc770100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555dfba76480 con 0x555dfba6ed00 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555dfba76600 con 0x555dfba6e880 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc10816400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555dfba76780 con 0x555dfba6c000 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0e7d4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 0x555dfc770100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0e7d4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 0x555dfc770100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46894/0 (socket says 172.21.15.131:46894) 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0e7d4640 1 -- 172.21.15.131:0/3417551533 learned_addr learned my addr 172.21.15.131:0/3417551533 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0efd5640 1 --2- 172.21.15.131:0/3417551533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6e880 0x555dfc76fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0e7d4640 1 -- 172.21.15.131:0/3417551533 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6c000 msgr2=0x555dfc76f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0e7d4640 1 --2- 172.21.15.131:0/3417551533 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6c000 0x555dfc76f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0e7d4640 1 -- 172.21.15.131:0/3417551533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6e880 msgr2=0x555dfc76fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.842 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0e7d4640 1 --2- 172.21.15.131:0/3417551533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6e880 0x555dfc76fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0e7d4640 1 -- 172.21.15.131:0/3417551533 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dfbac8960 con 0x555dfba6ed00 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0efd5640 1 --2- 172.21.15.131:0/3417551533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6e880 0x555dfc76fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0e7d4640 1 --2- 172.21.15.131:0/3417551533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 0x555dfc770100 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x555dfc8dec90 tx=0x555dfc8eac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3ceba3aa32d324 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0a7cc640 1 -- 172.21.15.131:0/3417551533 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555dfb9c1520 con 0x555dfba6ed00 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0a7cc640 1 -- 172.21.15.131:0/3417551533 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555dfb9dee00 con 0x555dfba6ed00 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.839+0000 7fcc0a7cc640 1 -- 172.21.15.131:0/3417551533 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555dfb9c1380 con 0x555dfba6ed00 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 -- 172.21.15.131:0/3417551533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 msgr2=0x555dfc770100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 --2- 172.21.15.131:0/3417551533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 0x555dfc770100 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x555dfc8dec90 tx=0x555dfc8eac00 comp rx=0 tx=0).stop 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 -- 172.21.15.131:0/3417551533 shutdown_connections 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 --2- 172.21.15.131:0/3417551533 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 0x555dfc770100 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 --2- 172.21.15.131:0/3417551533 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6e880 0x555dfc76fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 --2- 172.21.15.131:0/3417551533 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6c000 0x555dfc76f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.843 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 -- 172.21.15.131:0/3417551533 >> 172.21.15.131:0/3417551533 conn(0x555dfba6c480 msgr2=0x555dfc788800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 -- 172.21.15.131:0/3417551533 shutdown_connections 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 -- 172.21.15.131:0/3417551533 wait complete. 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 Processor -- start 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 -- start start 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6c000 0x555dfc771180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6e880 0x555dfc770c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 0x555dfc770680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555dfba77e00 con 0x555dfba6ed00 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555dfba77c80 con 0x555dfba6c000 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555dfba77b00 con 0x555dfba6e880 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc0e7d4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 0x555dfc770680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc0e7d4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 0x555dfc770680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46906/0 (socket says 172.21.15.131:46906) 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc0e7d4640 1 -- 172.21.15.131:0/2058632191 learned_addr learned my addr 172.21.15.131:0/2058632191 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc0e7d4640 1 -- 172.21.15.131:0/2058632191 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6e880 msgr2=0x555dfc770c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc0efd5640 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6e880 0x555dfc770c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc0e7d4640 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6e880 0x555dfc770c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc0e7d4640 1 -- 172.21.15.131:0/2058632191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6c000 msgr2=0x555dfc771180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc0e7d4640 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6c000 0x555dfc771180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc0e7d4640 1 -- 172.21.15.131:0/2058632191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dfbac9e00 con 0x555dfba6ed00 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc0e7d4640 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 0x555dfc770680 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x555dfc8dff50 tx=0x555dfc8eade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc08fc9640 1 -- 172.21.15.131:0/2058632191 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555dfb9c0820 con 0x555dfba6ed00 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc08fc9640 1 -- 172.21.15.131:0/2058632191 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555dfb9dec40 con 0x555dfba6ed00 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555dfbac8960 con 0x555dfba6ed00 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc08fc9640 1 -- 172.21.15.131:0/2058632191 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x555dfb9c1520 con 0x555dfba6ed00 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dfbce4000 con 0x555dfba6ed00 2024-09-17T02:51:02.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.843+0000 7fcc10816400 4 main: JournalTool: resolving pool 15 2024-09-17T02:51:02.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc08fc9640 1 -- 172.21.15.131:0/2058632191 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x555dfbcd6c00 con 0x555dfba6ed00 2024-09-17T02:51:02.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc08fc9640 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x555dfba6fa80 0x555dfc771700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc08fc9640 1 -- 172.21.15.131:0/2058632191 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x555dfbcde400 con 0x555dfba6ed00 2024-09-17T02:51:02.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 main: JournalTool: creating IoCtx.. 2024-09-17T02:51:02.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 main: Executing for rank 0 2024-09-17T02:51:02.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 Filtering by inode '1099511627778' 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x555dfba6db00 0x555dfc770100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x555dfbde2000 con 0x555dfba6db00 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0efd5640 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x555dfba6fa80 0x555dfc771700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0e7d4640 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x555dfba6db00 0x555dfc770100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0e7d4640 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x555dfba6db00 0x555dfc770100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0e7d4640 1 -- 172.21.15.131:0/2058632191 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x555dfba94000 con 0x555dfba6db00 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 Pointer 400.00000000 is readable 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 JournalScanner::scan: reading header object '200.00000000' 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x555dfba6d680 0x555dfc76fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x555dfbde2400 con 0x555dfba6d680 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0efd5640 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x555dfba6fa80 0x555dfc771700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x555dfbaa91d0 tx=0x555dfbde0840 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0f7d6640 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x555dfba6d680 0x555dfc76fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0f7d6640 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x555dfba6d680 0x555dfc76fb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0f7d6640 1 -- 172.21.15.131:0/2058632191 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv29 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x555dfba94500 con 0x555dfba6d680 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 10 Header 0x400000 0x405f0e 0x40628a 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 10 Starting journal scan from offset 0x405f0e 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x555dfbde3400 con 0x555dfba6d680 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0f7d6640 1 -- 172.21.15.131:0/2058632191 <== osd.3 v2:172.21.15.131:6816/4047536388 2 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv27 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x555dfba94500 con 0x555dfba6d680 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 10 Parsing data, 0x37c bytes available 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 10 Attempting decode at 0x405f0e 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 10 Consumed 0x37c bytes 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 10 Valid entry at 0x405f0e 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) -- 0x555dfbde3800 con 0x555dfba6d680 2024-09-17T02:51:02.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0f7d6640 1 -- 172.21.15.131:0/2058632191 <== osd.3 v2:172.21.15.131:6816/4047536388 3 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x555dfba94500 con 0x555dfba6d680 2024-09-17T02:51:02.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 Reached end of journal objects 2024-09-17T02:51:02.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 Scanned objects, 0 missing, 1 valid 2024-09-17T02:51:02.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 Events scanned, 0 gaps 2024-09-17T02:51:02.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 Found 1 valid events 2024-09-17T02:51:02.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 Selected 0 events events for processing 2024-09-17T02:51:02.851 INFO:teuthology.orchestra.run.smithi131.stdout:Events by type: 2024-09-17T02:51:02.851 INFO:teuthology.orchestra.run.smithi131.stdout:Errors: 0 2024-09-17T02:51:02.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 4 0 events 2024-09-17T02:51:02.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x555dfba6db00 msgr2=0x555dfc770100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x555dfba6db00 0x555dfc770100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x555dfba6d680 msgr2=0x555dfc76fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x555dfba6d680 0x555dfc76fb80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x555dfba6fa80 msgr2=0x555dfc771700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x555dfba6fa80 0x555dfc771700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x555dfbaa91d0 tx=0x555dfbde0840 comp rx=0 tx=0).stop 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 msgr2=0x555dfc770680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 --2- 172.21.15.131:0/2058632191 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6ed00 0x555dfc770680 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x555dfc8dff50 tx=0x555dfc8eade0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0efd5640 1 -- 172.21.15.131:0/2058632191 reap_dead start 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc0efd5640 1 -- 172.21.15.131:0/2058632191 reap_dead start 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 shutdown_connections 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 >> 172.21.15.131:0/2058632191 conn(0x555dfba6c480 msgr2=0x555dfc788800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 shutdown_connections 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.847+0000 7fcc10816400 1 -- 172.21.15.131:0/2058632191 wait complete. 2024-09-17T02:51:02.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.851+0000 7fcc10816400 1 -- 172.21.15.131:0/480104136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6c900 msgr2=0x555dfc76e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.851+0000 7fcc10816400 1 --2- 172.21.15.131:0/480104136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6c900 0x555dfc76e000 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x555dfb9c3590 tx=0x555dfbaa7aa0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.851+0000 7fcc10816400 1 -- 172.21.15.131:0/480104136 shutdown_connections 2024-09-17T02:51:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.851+0000 7fcc10816400 1 --2- 172.21.15.131:0/480104136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555dfba6c900 0x555dfc76e000 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.851+0000 7fcc10816400 1 --2- 172.21.15.131:0/480104136 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555dfba6cd80 0x555dfc76e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.851+0000 7fcc10816400 1 --2- 172.21.15.131:0/480104136 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555dfba6d200 0x555dfc76f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.851+0000 7fcc10816400 1 -- 172.21.15.131:0/480104136 >> 172.21.15.131:0/480104136 conn(0x555dfba6df80 msgr2=0x555dfc788000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.851+0000 7fcc10816400 1 -- 172.21.15.131:0/480104136 shutdown_connections 2024-09-17T02:51:02.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.851+0000 7fcc10816400 1 -- 172.21.15.131:0/480104136 wait complete. 2024-09-17T02:51:02.855 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.082295, result: Events by type: Errors: 0 2024-09-17T02:51:02.855 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-09-17T02:51:02.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.915+0000 7ff6b7d90080 1 Processor -- start 2024-09-17T02:51:02.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.915+0000 7ff6b7d90080 1 -- start start 2024-09-17T02:51:02.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4fa84c9f0 0x55f4fa84d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f4fa84ce00 0x55f4fa8d1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa8695e0 0x55f4fa8d1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f4fa7ccf40 con 0x55f4fa8695e0 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f4fa7d9d00 con 0x55f4fa84c9f0 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f4fa76ea00 con 0x55f4fa84ce00 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b2068640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa8695e0 0x55f4fa8d1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b2068640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa8695e0 0x55f4fa8d1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46930/0 (socket says 172.21.15.131:46930) 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b2068640 1 -- 172.21.15.131:0/693294203 learned_addr learned my addr 172.21.15.131:0/693294203 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b1066640 1 --2- 172.21.15.131:0/693294203 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f4fa84ce00 0x55f4fa8d1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b2068640 1 -- 172.21.15.131:0/693294203 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f4fa84ce00 msgr2=0x55f4fa8d1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b2068640 1 --2- 172.21.15.131:0/693294203 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f4fa84ce00 0x55f4fa8d1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b2068640 1 -- 172.21.15.131:0/693294203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4fa84c9f0 msgr2=0x55f4fa84d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b1867640 1 --2- 172.21.15.131:0/693294203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4fa84c9f0 0x55f4fa84d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b2068640 1 --2- 172.21.15.131:0/693294203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4fa84c9f0 0x55f4fa84d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b2068640 1 -- 172.21.15.131:0/693294203 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4fa84c6c0 con 0x55f4fa8695e0 2024-09-17T02:51:02.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b2068640 1 --2- 172.21.15.131:0/693294203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa8695e0 0x55f4fa8d1fa0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4004f50 tx=0x7ff6a4017fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96edaaf93f674c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6a3fff640 1 -- 172.21.15.131:0/693294203 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a40b0040 con 0x55f4fa8695e0 2024-09-17T02:51:02.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6a3fff640 1 -- 172.21.15.131:0/693294203 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6a40ba070 con 0x55f4fa8695e0 2024-09-17T02:51:02.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6a3fff640 1 -- 172.21.15.131:0/693294203 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a40b5410 con 0x55f4fa8695e0 2024-09-17T02:51:02.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 -- 172.21.15.131:0/693294203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa8695e0 msgr2=0x55f4fa8d1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 --2- 172.21.15.131:0/693294203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa8695e0 0x55f4fa8d1fa0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff6a4004f50 tx=0x7ff6a4017fa0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 -- 172.21.15.131:0/693294203 shutdown_connections 2024-09-17T02:51:02.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 --2- 172.21.15.131:0/693294203 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa8695e0 0x55f4fa8d1fa0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 --2- 172.21.15.131:0/693294203 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f4fa84ce00 0x55f4fa8d1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 --2- 172.21.15.131:0/693294203 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4fa84c9f0 0x55f4fa84d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 -- 172.21.15.131:0/693294203 >> 172.21.15.131:0/693294203 conn(0x55f4fa80cd00 msgr2=0x55f4fa8f5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 -- 172.21.15.131:0/693294203 shutdown_connections 2024-09-17T02:51:02.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 -- 172.21.15.131:0/693294203 wait complete. 2024-09-17T02:51:02.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 Processor -- start 2024-09-17T02:51:02.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.919+0000 7ff6b7d90080 1 -- start start 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b7d90080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4fa84c9f0 0x55f4fa8ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b7d90080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa84ce00 0x55f4fa8ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b7d90080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f4fa8695e0 0x55f4fa865c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b7d90080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f4fa8f5b60 con 0x55f4fa84ce00 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b7d90080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f4fa8602e0 con 0x55f4fa84c9f0 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b7d90080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f4fa8f13c0 con 0x55f4fa8695e0 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b1066640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa84ce00 0x55f4fa8ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b1066640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa84ce00 0x55f4fa8ebf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46940/0 (socket says 172.21.15.131:46940) 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b1066640 1 -- 172.21.15.131:0/3888866512 learned_addr learned my addr 172.21.15.131:0/3888866512 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b1867640 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4fa84c9f0 0x55f4fa8ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b1066640 1 -- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f4fa8695e0 msgr2=0x55f4fa865c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b2068640 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f4fa8695e0 0x55f4fa865c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b1066640 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f4fa8695e0 0x55f4fa865c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b1066640 1 -- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4fa84c9f0 msgr2=0x55f4fa8ea0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b1066640 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4fa84c9f0 0x55f4fa8ea0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b1066640 1 -- 172.21.15.131:0/3888866512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4fa8f1890 con 0x55f4fa84ce00 2024-09-17T02:51:02.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b1867640 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f4fa84c9f0 0x55f4fa8ea0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:02.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b1066640 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa84ce00 0x55f4fa8ebf30 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800ce00 tx=0x7ff6a800a670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6a2ffd640 1 -- 172.21.15.131:0/3888866512 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800e810 con 0x55f4fa84ce00 2024-09-17T02:51:02.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6a2ffd640 1 -- 172.21.15.131:0/3888866512 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6a800e9b0 con 0x55f4fa84ce00 2024-09-17T02:51:02.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6a2ffd640 1 -- 172.21.15.131:0/3888866512 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6a800ede0 con 0x55f4fa84ce00 2024-09-17T02:51:02.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b7d90080 1 -- 172.21.15.131:0/3888866512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f4fa8f1b30 con 0x55f4fa84ce00 2024-09-17T02:51:02.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6b7d90080 1 -- 172.21.15.131:0/3888866512 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4fa8f2270 con 0x55f4fa84ce00 2024-09-17T02:51:02.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6a2ffd640 1 -- 172.21.15.131:0/3888866512 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff6a801f440 con 0x55f4fa84ce00 2024-09-17T02:51:02.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.923+0000 7ff6a2ffd640 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff694071be0 0x7ff6940740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.927+0000 7ff6a2ffd640 1 -- 172.21.15.131:0/3888866512 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7ff6a8093680 con 0x55f4fa84ce00 2024-09-17T02:51:02.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.927+0000 7ff6b7d90080 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55f4fa8ecdf0 0x55f4fa8ed200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:02.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.927+0000 7ff6b1867640 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff694071be0 0x7ff6940740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.927+0000 7ff6b7d90080 1 -- 172.21.15.131:0/3888866512 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) -- 0x55f4fa8ed740 con 0x55f4fa8ecdf0 2024-09-17T02:51:02.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.927+0000 7ff6b2068640 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55f4fa8ecdf0 0x55f4fa8ed200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:02.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.927+0000 7ff6b1867640 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff694071be0 0x7ff6940740f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac006fd0 tx=0x7ff6ac008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.927+0000 7ff6b2068640 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55f4fa8ecdf0 0x55f4fa8ed200 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:02.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b2068640 1 -- 172.21.15.131:0/3888866512 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v110'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7ff6a40b3040 con 0x55f4fa8ecdf0 2024-09-17T02:51:02.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b7d90080 1 -- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55f4fa8ecdf0 msgr2=0x55f4fa8ed200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b7d90080 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55f4fa8ecdf0 0x55f4fa8ed200 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:02.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b7d90080 1 -- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff694071be0 msgr2=0x7ff6940740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b7d90080 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff694071be0 0x7ff6940740f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac006fd0 tx=0x7ff6ac008040 comp rx=0 tx=0).stop 2024-09-17T02:51:02.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b7d90080 1 -- 172.21.15.131:0/3888866512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa84ce00 msgr2=0x55f4fa8ebf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:02.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b7d90080 1 --2- 172.21.15.131:0/3888866512 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f4fa84ce00 0x55f4fa8ebf30 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff6a800ce00 tx=0x7ff6a800a670 comp rx=0 tx=0).stop 2024-09-17T02:51:02.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b2068640 1 -- 172.21.15.131:0/3888866512 reap_dead start 2024-09-17T02:51:02.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b7d90080 1 -- 172.21.15.131:0/3888866512 shutdown_connections 2024-09-17T02:51:02.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b7d90080 1 -- 172.21.15.131:0/3888866512 >> 172.21.15.131:0/3888866512 conn(0x55f4fa80cd00 msgr2=0x55f4fa8f53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:02.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b7d90080 1 -- 172.21.15.131:0/3888866512 shutdown_connections 2024-09-17T02:51:02.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:02.931+0000 7ff6b7d90080 1 -- 172.21.15.131:0/3888866512 wait complete. 2024-09-17T02:51:02.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T02:51:03.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 -- 172.21.15.131:0/1352543189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5698155240 msgr2=0x7f5698073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:03.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 --2- 172.21.15.131:0/1352543189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5698155240 0x7f5698073430 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5694009bc0 tx=0x7f569401cb60 comp rx=0 tx=0).stop 2024-09-17T02:51:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 -- 172.21.15.131:0/1352543189 shutdown_connections 2024-09-17T02:51:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 --2- 172.21.15.131:0/1352543189 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f569806e5b0 0x7f56980cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 --2- 172.21.15.131:0/1352543189 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5698155240 0x7f5698073430 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 -- 172.21.15.131:0/1352543189 >> 172.21.15.131:0/1352543189 conn(0x7f569806eea0 msgr2=0x7f569806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 -- 172.21.15.131:0/1352543189 shutdown_connections 2024-09-17T02:51:03.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 -- 172.21.15.131:0/1352543189 wait complete. 2024-09-17T02:51:03.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 Processor -- start 2024-09-17T02:51:03.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 -- start start 2024-09-17T02:51:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5698073970 0x7f56980b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5698155240 0x7f56980a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5698143730 0x7f5698143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f56980cbf10 con 0x7f5698073970 2024-09-17T02:51:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f56980cbd90 con 0x7f5698143730 2024-09-17T02:51:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f56a03ed640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5698168c80 con 0x7f5698155240 2024-09-17T02:51:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f569e189640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5698073970 0x7f56980b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f569e189640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5698073970 0x7f56980b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46954/0 (socket says 172.21.15.131:46954) 2024-09-17T02:51:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f569e189640 1 -- 172.21.15.131:0/1231390303 learned_addr learned my addr 172.21.15.131:0/1231390303 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:03.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f569d988640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5698155240 0x7f56980a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.143+0000 7f569e98a640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5698143730 0x7f5698143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f569e189640 1 -- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5698155240 msgr2=0x7f56980a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f569e189640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5698155240 0x7f56980a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f569e189640 1 -- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5698143730 msgr2=0x7f5698143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f569e189640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5698143730 0x7f5698143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f569e189640 1 -- 172.21.15.131:0/1231390303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5698144080 con 0x7f5698073970 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f569d988640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5698155240 0x7f56980a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f569e189640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5698073970 0x7f56980b4ab0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f569401b210 tx=0x7f5694009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f568effd640 1 -- 172.21.15.131:0/1231390303 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56940bb070 con 0x7f5698073970 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f568effd640 1 -- 172.21.15.131:0/1231390303 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5694004d40 con 0x7f5698073970 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f56a03ed640 1 -- 172.21.15.131:0/1231390303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56980ae4f0 con 0x7f5698073970 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f568effd640 1 -- 172.21.15.131:0/1231390303 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56940b2040 con 0x7f5698073970 2024-09-17T02:51:03.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f56a03ed640 1 -- 172.21.15.131:0/1231390303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56980aeab0 con 0x7f5698073970 2024-09-17T02:51:03.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f56a03ed640 1 -- 172.21.15.131:0/1231390303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56980ae0a0 con 0x7f5698073970 2024-09-17T02:51:03.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f568effd640 1 -- 172.21.15.131:0/1231390303 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5694004240 con 0x7f5698073970 2024-09-17T02:51:03.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f568effd640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f567c073230 0x7f567c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:03.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f568effd640 1 -- 172.21.15.131:0/1231390303 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f569412e6a0 con 0x7f5698073970 2024-09-17T02:51:03.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f569d988640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f567c073230 0x7f567c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:03.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.147+0000 7f569d988640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f567c073230 0x7f567c075740 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5698144810 tx=0x7f5688009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:03.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.151+0000 7f568effd640 1 -- 172.21.15.131:0/1231390303 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f56980ae0a0 con 0x7f5698073970 2024-09-17T02:51:03.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.359+0000 7f56a03ed640 1 -- 172.21.15.131:0/1231390303 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f56980cab60 con 0x7f5698073970 2024-09-17T02:51:03.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.707+0000 7f568effd640 1 -- 172.21.15.131:0/1231390303 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v88) ==== 162+0+0 (secure 0 0 0) 0x7f56940c4020 con 0x7f5698073970 2024-09-17T02:51:03.705 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T02:51:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 -- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f567c073230 msgr2=0x7f567c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f567c073230 0x7f567c075740 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f5698144810 tx=0x7f5688009040 comp rx=0 tx=0).stop 2024-09-17T02:51:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 -- 172.21.15.131:0/1231390303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5698073970 msgr2=0x7f56980b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5698073970 0x7f56980b4ab0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f569401b210 tx=0x7f5694009200 comp rx=0 tx=0).stop 2024-09-17T02:51:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 -- 172.21.15.131:0/1231390303 shutdown_connections 2024-09-17T02:51:03.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f567c073230 0x7f567c075740 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5698143730 0x7f5698143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5698155240 0x7f56980a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 --2- 172.21.15.131:0/1231390303 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5698073970 0x7f56980b4ab0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 -- 172.21.15.131:0/1231390303 >> 172.21.15.131:0/1231390303 conn(0x7f569806eea0 msgr2=0x7f569815e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 -- 172.21.15.131:0/1231390303 shutdown_connections 2024-09-17T02:51:03.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.711+0000 7f56a03ed640 1 -- 172.21.15.131:0/1231390303 wait complete. 2024-09-17T02:51:03.741 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:51:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.895+0000 7fc616d97640 1 -- 172.21.15.131:0/58887572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cec00 msgr2=0x7fc6100cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.895+0000 7fc616d97640 1 --2- 172.21.15.131:0/58887572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cec00 0x7fc6100cf010 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc608009bc0 tx=0x7fc60801cb30 comp rx=0 tx=0).stop 2024-09-17T02:51:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.895+0000 7fc616d97640 1 -- 172.21.15.131:0/58887572 shutdown_connections 2024-09-17T02:51:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.895+0000 7fc616d97640 1 --2- 172.21.15.131:0/58887572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cec00 0x7fc6100cf010 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.895+0000 7fc616d97640 1 --2- 172.21.15.131:0/58887572 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc61006e2c0 0x7fc6100c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.895+0000 7fc616d97640 1 -- 172.21.15.131:0/58887572 >> 172.21.15.131:0/58887572 conn(0x7fc61006eea0 msgr2=0x7fc61006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:03.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.895+0000 7fc616d97640 1 -- 172.21.15.131:0/58887572 shutdown_connections 2024-09-17T02:51:03.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.895+0000 7fc616d97640 1 -- 172.21.15.131:0/58887572 wait complete. 2024-09-17T02:51:03.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc616d97640 1 Processor -- start 2024-09-17T02:51:03.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc616d97640 1 -- start start 2024-09-17T02:51:03.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc616d97640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc61006e2c0 0x7fc61014dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:03.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc616d97640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc6100c4ac0 0x7fc6100b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:03.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc616d97640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cec00 0x7fc6100cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:03.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc616d97640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc610169cf0 con 0x7fc6100cec00 2024-09-17T02:51:03.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc616d97640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc61015cb90 con 0x7fc61006e2c0 2024-09-17T02:51:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc616d97640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc610169e70 con 0x7fc6100c4ac0 2024-09-17T02:51:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc615334640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cec00 0x7fc6100cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc615334640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cec00 0x7fc6100cbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46972/0 (socket says 172.21.15.131:46972) 2024-09-17T02:51:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc615334640 1 -- 172.21.15.131:0/2225472968 learned_addr learned my addr 172.21.15.131:0/2225472968 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc60ffff640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc6100c4ac0 0x7fc6100b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc614b33640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc61006e2c0 0x7fc61014dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc615334640 1 -- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc6100c4ac0 msgr2=0x7fc6100b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc615334640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc6100c4ac0 0x7fc6100b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc615334640 1 -- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc61006e2c0 msgr2=0x7fc61014dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc615334640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc61006e2c0 0x7fc61014dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc615334640 1 -- 172.21.15.131:0/2225472968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6100b5240 con 0x7fc6100cec00 2024-09-17T02:51:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc615334640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cec00 0x7fc6100cbc80 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc60801c5a0 tx=0x7fc608009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc60d7fa640 1 -- 172.21.15.131:0/2225472968 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080b2040 con 0x7fc6100cec00 2024-09-17T02:51:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc60d7fa640 1 -- 172.21.15.131:0/2225472968 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6080bb070 con 0x7fc6100cec00 2024-09-17T02:51:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc60d7fa640 1 -- 172.21.15.131:0/2225472968 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608004590 con 0x7fc6100cec00 2024-09-17T02:51:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc616d97640 1 -- 172.21.15.131:0/2225472968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6101fbe20 con 0x7fc6100cec00 2024-09-17T02:51:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc616d97640 1 -- 172.21.15.131:0/2225472968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6101fc3e0 con 0x7fc6100cec00 2024-09-17T02:51:03.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.899+0000 7fc616d97640 1 -- 172.21.15.131:0/2225472968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6100c4580 con 0x7fc6100cec00 2024-09-17T02:51:03.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.903+0000 7fc60d7fa640 1 -- 172.21.15.131:0/2225472968 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc6080c0460 con 0x7fc6100cec00 2024-09-17T02:51:03.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.903+0000 7fc60d7fa640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5f0073200 0x7fc5f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:03.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.903+0000 7fc60d7fa640 1 -- 172.21.15.131:0/2225472968 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7fc60812f9b0 con 0x7fc6100cec00 2024-09-17T02:51:03.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.903+0000 7fc614b33640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5f0073200 0x7fc5f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:03.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.903+0000 7fc614b33640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5f0073200 0x7fc5f0075710 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc600005fd0 tx=0x7fc600005dc0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:03.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:03.907+0000 7fc60d7fa640 1 -- 172.21.15.131:0/2225472968 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc6080167b0 con 0x7fc6100cec00 2024-09-17T02:51:04.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.095+0000 7fc616d97640 1 -- 172.21.15.131:0/2225472968 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc610073310 con 0x7fc6100cec00 2024-09-17T02:51:04.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.095+0000 7fc60d7fa640 1 -- 172.21.15.131:0/2225472968 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5232 (secure 0 0 0) 0x7fc6080fa860 con 0x7fc6100cec00 2024-09-17T02:51:04.092 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:04.092 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":89,"btime":"2024-09-17T02:51:03:720806+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:51:03.720765+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":89,"state":"up:replay","state_seq":1,"addr":"172.21.15.131:6835/1272267157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1272267157},{"type":"v1","addr":"172.21.15.131:6835","nonce":1272267157}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-17T02:51:04.093 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 89 2024-09-17T02:51:04.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.095+0000 7fc616d97640 1 -- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5f0073200 msgr2=0x7fc5f0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:04.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.095+0000 7fc616d97640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5f0073200 0x7fc5f0075710 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc600005fd0 tx=0x7fc600005dc0 comp rx=0 tx=0).stop 2024-09-17T02:51:04.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.095+0000 7fc616d97640 1 -- 172.21.15.131:0/2225472968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cec00 msgr2=0x7fc6100cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:04.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.095+0000 7fc616d97640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cec00 0x7fc6100cbc80 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc60801c5a0 tx=0x7fc608009200 comp rx=0 tx=0).stop 2024-09-17T02:51:04.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.099+0000 7fc616d97640 1 -- 172.21.15.131:0/2225472968 shutdown_connections 2024-09-17T02:51:04.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.099+0000 7fc616d97640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc5f0073200 0x7fc5f0075710 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.099+0000 7fc616d97640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc6100cec00 0x7fc6100cbc80 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.099+0000 7fc616d97640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc6100c4ac0 0x7fc6100b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.099+0000 7fc616d97640 1 --2- 172.21.15.131:0/2225472968 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc61006e2c0 0x7fc61014dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.099+0000 7fc616d97640 1 -- 172.21.15.131:0/2225472968 >> 172.21.15.131:0/2225472968 conn(0x7fc61006eea0 msgr2=0x7fc6101532e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.099+0000 7fc616d97640 1 -- 172.21.15.131:0/2225472968 shutdown_connections 2024-09-17T02:51:04.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:04.099+0000 7fc616d97640 1 -- 172.21.15.131:0/2225472968 wait complete. 2024-09-17T02:51:04.516 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:50:02.583706+0000', 'modified': '2024-09-17T02:51:03.720765+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8175}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8175': {'gid': 8175, 'name': 'a', 'rank': 0, 'incarnation': 89, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/1272267157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6833', 'nonce': 1272267157}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1272267157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:51:04.517 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8175:up:replay 2024-09-17T02:51:05.518 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:51:05.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.687+0000 7fc3fcd14640 1 -- 172.21.15.131:0/673951531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3f81566f0 msgr2=0x7fc3f8156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:05.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.687+0000 7fc3fcd14640 1 --2- 172.21.15.131:0/673951531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3f81566f0 0x7fc3f8156b00 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80720e0 tx=0x7fc3f001cb10 comp rx=0 tx=0).stop 2024-09-17T02:51:05.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.687+0000 7fc3fcd14640 1 -- 172.21.15.131:0/673951531 shutdown_connections 2024-09-17T02:51:05.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.687+0000 7fc3fcd14640 1 --2- 172.21.15.131:0/673951531 >> v2:172.21.15.189:3300/0 conn(0x7fc3f80cc750 0x7fc3f80ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:05.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.687+0000 7fc3fcd14640 1 --2- 172.21.15.131:0/673951531 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3f81566f0 0x7fc3f8156b00 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:05.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.687+0000 7fc3fcd14640 1 -- 172.21.15.131:0/673951531 >> 172.21.15.131:0/673951531 conn(0x7fc3f80d2100 msgr2=0x7fc3f80d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:05.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.687+0000 7fc3fcd14640 1 -- 172.21.15.131:0/673951531 shutdown_connections 2024-09-17T02:51:05.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.687+0000 7fc3fcd14640 1 -- 172.21.15.131:0/673951531 wait complete. 2024-09-17T02:51:05.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3fcd14640 1 Processor -- start 2024-09-17T02:51:05.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3fcd14640 1 -- start start 2024-09-17T02:51:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3fcd14640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3f80cc750 0x7fc3f814f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3fcd14640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3f81566f0 0x7fc3f80cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3fcd14640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc3f815cfa0 0x7fc3f80cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3fcd14640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8071700 con 0x7fc3f81566f0 2024-09-17T02:51:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3fcd14640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8071580 con 0x7fc3f80cc750 2024-09-17T02:51:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3fcd14640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f8071880 con 0x7fc3f815cfa0 2024-09-17T02:51:05.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3f81566f0 0x7fc3f80cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:05.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3f81566f0 0x7fc3f80cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:46984/0 (socket says 172.21.15.131:46984) 2024-09-17T02:51:05.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f5d9b640 1 -- 172.21.15.131:0/3417078696 learned_addr learned my addr 172.21.15.131:0/3417078696 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:05.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f659c640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3f80cc750 0x7fc3f814f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:05.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f6d9d640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc3f815cfa0 0x7fc3f80cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:05.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f5d9b640 1 -- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc3f815cfa0 msgr2=0x7fc3f80cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:05.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f5d9b640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc3f815cfa0 0x7fc3f80cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:05.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f5d9b640 1 -- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3f80cc750 msgr2=0x7fc3f814f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:05.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f5d9b640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3f80cc750 0x7fc3f814f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:05.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f5d9b640 1 -- 172.21.15.131:0/3417078696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f814f750 con 0x7fc3f81566f0 2024-09-17T02:51:05.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f6d9d640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc3f815cfa0 0x7fc3f80cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:05.691 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3f5d9b640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3f81566f0 0x7fc3f80cb680 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000d750 tx=0x7fc3e000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3eeffd640 1 -- 172.21.15.131:0/3417078696 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e0017070 con 0x7fc3f81566f0 2024-09-17T02:51:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3eeffd640 1 -- 172.21.15.131:0/3417078696 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3e0009cc0 con 0x7fc3f81566f0 2024-09-17T02:51:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3eeffd640 1 -- 172.21.15.131:0/3417078696 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e00106c0 con 0x7fc3f81566f0 2024-09-17T02:51:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3fcd14640 1 -- 172.21.15.131:0/3417078696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f80cabe0 con 0x7fc3f81566f0 2024-09-17T02:51:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.691+0000 7fc3fcd14640 1 -- 172.21.15.131:0/3417078696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f80cae80 con 0x7fc3f81566f0 2024-09-17T02:51:05.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.695+0000 7fc3fcd14640 1 -- 172.21.15.131:0/3417078696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f80ccb60 con 0x7fc3f81566f0 2024-09-17T02:51:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.695+0000 7fc3eeffd640 1 -- 172.21.15.131:0/3417078696 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc3e001a070 con 0x7fc3f81566f0 2024-09-17T02:51:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.695+0000 7fc3eeffd640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc3d8073230 0x7fc3d8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.695+0000 7fc3eeffd640 1 -- 172.21.15.131:0/3417078696 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7fc3e0094aa0 con 0x7fc3f81566f0 2024-09-17T02:51:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.695+0000 7fc3f659c640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc3d8073230 0x7fc3d8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:05.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.695+0000 7fc3f659c640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc3d8073230 0x7fc3d8075740 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00af000 tx=0x7fc3f001b350 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:05.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.699+0000 7fc3eeffd640 1 -- 172.21.15.131:0/3417078696 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc3e000de60 con 0x7fc3f81566f0 2024-09-17T02:51:05.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.891+0000 7fc3fcd14640 1 -- 172.21.15.131:0/3417078696 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3f81fad70 con 0x7fc3f81566f0 2024-09-17T02:51:05.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.891+0000 7fc3eeffd640 1 -- 172.21.15.131:0/3417078696 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5232 (secure 0 0 0) 0x7fc3e005f850 con 0x7fc3f81566f0 2024-09-17T02:51:05.889 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:05.889 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":91,"btime":"2024-09-17T02:51:05:736006+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:51:04.740634+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":89,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.131:6835/1272267157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1272267157},{"type":"v1","addr":"172.21.15.131:6835","nonce":1272267157}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-17T02:51:05.889 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 91 2024-09-17T02:51:05.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 -- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc3d8073230 msgr2=0x7fc3d8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:05.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc3d8073230 0x7fc3d8075740 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc3f00af000 tx=0x7fc3f001b350 comp rx=0 tx=0).stop 2024-09-17T02:51:05.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 -- 172.21.15.131:0/3417078696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3f81566f0 msgr2=0x7fc3f80cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:05.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3f81566f0 0x7fc3f80cb680 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc3e000d750 tx=0x7fc3e000dc10 comp rx=0 tx=0).stop 2024-09-17T02:51:05.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 -- 172.21.15.131:0/3417078696 shutdown_connections 2024-09-17T02:51:05.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc3d8073230 0x7fc3d8075740 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:05.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc3f815cfa0 0x7fc3f80cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:05.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc3f81566f0 0x7fc3f80cb680 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:05.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 --2- 172.21.15.131:0/3417078696 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc3f80cc750 0x7fc3f814f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:05.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 -- 172.21.15.131:0/3417078696 >> 172.21.15.131:0/3417078696 conn(0x7fc3f80d2100 msgr2=0x7fc3f815d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:05.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 -- 172.21.15.131:0/3417078696 shutdown_connections 2024-09-17T02:51:05.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:05.895+0000 7fc3fcd14640 1 -- 172.21.15.131:0/3417078696 wait complete. 2024-09-17T02:51:06.313 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:50:02.583706+0000', 'modified': '2024-09-17T02:51:04.740634+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8175}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8175': {'gid': 8175, 'name': 'a', 'rank': 0, 'incarnation': 89, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.131:6835/1272267157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6833', 'nonce': 1272267157}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1272267157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:51:06.313 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8175:up:rejoin 2024-09-17T02:51:07.314 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:51:07.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.483+0000 7f1893936640 1 -- 172.21.15.131:0/1549276042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c15f750 msgr2=0x7f188c0cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:07.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.483+0000 7f1893936640 1 --2- 172.21.15.131:0/1549276042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c15f750 0x7f188c0cff20 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f1880009840 tx=0x7f188001c820 comp rx=0 tx=0).stop 2024-09-17T02:51:07.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.483+0000 7f1893936640 1 -- 172.21.15.131:0/1549276042 shutdown_connections 2024-09-17T02:51:07.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.483+0000 7f1893936640 1 --2- 172.21.15.131:0/1549276042 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c15f750 0x7f188c0cff20 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:07.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.483+0000 7f1893936640 1 -- 172.21.15.131:0/1549276042 >> 172.21.15.131:0/1549276042 conn(0x7f188c06fbe0 msgr2=0x7f188c06fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:07.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.483+0000 7f1893936640 1 -- 172.21.15.131:0/1549276042 shutdown_connections 2024-09-17T02:51:07.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.483+0000 7f1893936640 1 -- 172.21.15.131:0/1549276042 wait complete. 2024-09-17T02:51:07.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1893936640 1 Processor -- start 2024-09-17T02:51:07.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1893936640 1 -- start start 2024-09-17T02:51:07.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1893936640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f188c15b310 0x7f188c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:07.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1893936640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f188c15f750 0x7f188c0b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:07.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1893936640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c15fb60 0x7f188c14ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:07.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1893936640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f188c16a200 con 0x7f188c15fb60 2024-09-17T02:51:07.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1893936640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f188c16a080 con 0x7f188c15b310 2024-09-17T02:51:07.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1893936640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f188c16a380 con 0x7f188c15f750 2024-09-17T02:51:07.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1891ed3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c15fb60 0x7f188c14ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:07.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1891ed3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c15fb60 0x7f188c14ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47008/0 (socket says 172.21.15.131:47008) 2024-09-17T02:51:07.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1891ed3640 1 -- 172.21.15.131:0/705239236 learned_addr learned my addr 172.21.15.131:0/705239236 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:07.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f18916d2640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f188c15b310 0x7f188c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:07.491 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1890ed1640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f188c15f750 0x7f188c0b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1891ed3640 1 -- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f188c15f750 msgr2=0x7f188c0b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1891ed3640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f188c15f750 0x7f188c0b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:07.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1891ed3640 1 -- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f188c15b310 msgr2=0x7f188c153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1891ed3640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f188c15b310 0x7f188c153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1891ed3640 1 -- 172.21.15.131:0/705239236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c071060 con 0x7f188c15fb60 2024-09-17T02:51:07.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1891ed3640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c15fb60 0x7f188c14ed70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c3c0 tx=0x7f187c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1889ffb640 1 -- 172.21.15.131:0/705239236 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c011790 con 0x7f188c15fb60 2024-09-17T02:51:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1889ffb640 1 -- 172.21.15.131:0/705239236 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f187c017070 con 0x7f188c15fb60 2024-09-17T02:51:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1893936640 1 -- 172.21.15.131:0/705239236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c06dcd0 con 0x7f188c15fb60 2024-09-17T02:51:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1889ffb640 1 -- 172.21.15.131:0/705239236 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c011ce0 con 0x7f188c15fb60 2024-09-17T02:51:07.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.487+0000 7f1893936640 1 -- 172.21.15.131:0/705239236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c14f7f0 con 0x7f188c15fb60 2024-09-17T02:51:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.491+0000 7f1893936640 1 -- 172.21.15.131:0/705239236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188c0cff20 con 0x7f188c15fb60 2024-09-17T02:51:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.491+0000 7f1889ffb640 1 -- 172.21.15.131:0/705239236 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f187c01c460 con 0x7f188c15fb60 2024-09-17T02:51:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.491+0000 7f1889ffb640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1878073230 0x7f1878075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.491+0000 7f1889ffb640 1 -- 172.21.15.131:0/705239236 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f187c094110 con 0x7f188c15fb60 2024-09-17T02:51:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.491+0000 7f18916d2640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1878073230 0x7f1878075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:07.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.491+0000 7f18916d2640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1878073230 0x7f1878075740 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1884001e40 tx=0x7f1884009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:07.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.495+0000 7f1889ffb640 1 -- 172.21.15.131:0/705239236 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f187c05ef70 con 0x7f188c15fb60 2024-09-17T02:51:07.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.699+0000 7f1893936640 1 -- 172.21.15.131:0/705239236 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f188c165cc0 con 0x7f188c15fb60 2024-09-17T02:51:07.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.699+0000 7f1889ffb640 1 -- 172.21.15.131:0/705239236 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5239 (secure 0 0 0) 0x7f187c05e870 con 0x7f188c15fb60 2024-09-17T02:51:07.696 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:07.696 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":92,"btime":"2024-09-17T02:51:06:749632+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:51:06.749617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8175},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8175":{"gid":8175,"name":"a","rank":0,"incarnation":89,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/1272267157","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":1272267157},{"type":"v1","addr":"172.21.15.131:6835","nonce":1272267157}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8175,"qdb_cluster":[8175]},"id":7}]} 2024-09-17T02:51:07.696 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 92 2024-09-17T02:51:07.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 -- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1878073230 msgr2=0x7f1878075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:07.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1878073230 0x7f1878075740 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1884001e40 tx=0x7f1884009040 comp rx=0 tx=0).stop 2024-09-17T02:51:07.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 -- 172.21.15.131:0/705239236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c15fb60 msgr2=0x7f188c14ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c15fb60 0x7f188c14ed70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f187c00c3c0 tx=0x7f187c00c880 comp rx=0 tx=0).stop 2024-09-17T02:51:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 -- 172.21.15.131:0/705239236 shutdown_connections 2024-09-17T02:51:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1878073230 0x7f1878075740 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f188c15fb60 0x7f188c14ed70 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f188c15f750 0x7f188c0b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 --2- 172.21.15.131:0/705239236 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f188c15b310 0x7f188c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 -- 172.21.15.131:0/705239236 >> 172.21.15.131:0/705239236 conn(0x7f188c06fbe0 msgr2=0x7f188c074660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 -- 172.21.15.131:0/705239236 shutdown_connections 2024-09-17T02:51:07.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:07.703+0000 7f1893936640 1 -- 172.21.15.131:0/705239236 wait complete. 2024-09-17T02:51:08.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 92, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:50:02.583706+0000', 'modified': '2024-09-17T02:51:06.749617+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8175}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8175': {'gid': 8175, 'name': 'a', 'rank': 0, 'incarnation': 89, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.131:6835/1272267157', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6833', 'nonce': 1272267157}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 1272267157}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8175, 'qdb_cluster': [8175]} 2024-09-17T02:51:08.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:51:08.122 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T02:51:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.291+0000 7f659a0ea640 1 -- 172.21.15.131:0/3359821221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6594071920 msgr2=0x7f65940cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.291+0000 7f659a0ea640 1 --2- 172.21.15.131:0/3359821221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6594071920 0x7f65940cf610 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f6588009840 tx=0x7f658801c790 comp rx=0 tx=0).stop 2024-09-17T02:51:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.291+0000 7f659a0ea640 1 -- 172.21.15.131:0/3359821221 shutdown_connections 2024-09-17T02:51:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.291+0000 7f659a0ea640 1 --2- 172.21.15.131:0/3359821221 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6594071920 0x7f65940cf610 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.291+0000 7f659a0ea640 1 -- 172.21.15.131:0/3359821221 >> 172.21.15.131:0/3359821221 conn(0x7f659406fdb0 msgr2=0x7f65940701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.291+0000 7f659a0ea640 1 -- 172.21.15.131:0/3359821221 shutdown_connections 2024-09-17T02:51:08.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.291+0000 7f659a0ea640 1 -- 172.21.15.131:0/3359821221 wait complete. 2024-09-17T02:51:08.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f659a0ea640 1 Processor -- start 2024-09-17T02:51:08.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f659a0ea640 1 -- start start 2024-09-17T02:51:08.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f659a0ea640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f659415b7d0 0x7f65941f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f659a0ea640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f659415bbe0 0x7f6594148380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f659a0ea640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6594148ae0 0x7f659414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f659a0ea640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6594169b80 con 0x7f659415bbe0 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f659a0ea640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6594169a00 con 0x7f6594148ae0 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f659a0ea640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6594169d00 con 0x7f659415b7d0 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f659415bbe0 0x7f6594148380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f659415bbe0 0x7f6594148380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47036/0 (socket says 172.21.15.131:47036) 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f6592ffd640 1 -- 172.21.15.131:0/4079695795 learned_addr learned my addr 172.21.15.131:0/4079695795 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f65937fe640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f659415b7d0 0x7f65941f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f6593fff640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6594148ae0 0x7f659414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f6592ffd640 1 -- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f659415b7d0 msgr2=0x7f65941f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f6592ffd640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f659415b7d0 0x7f65941f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f6592ffd640 1 -- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6594148ae0 msgr2=0x7f659414fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f6592ffd640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6594148ae0 0x7f659414fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f6592ffd640 1 -- 172.21.15.131:0/4079695795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659406ed50 con 0x7f659415bbe0 2024-09-17T02:51:08.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f65937fe640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f659415b7d0 0x7f65941f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f6593fff640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6594148ae0 0x7f659414fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f6592ffd640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f659415bbe0 0x7f6594148380 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f658801ae40 tx=0x7f65880a5600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f657bfff640 1 -- 172.21.15.131:0/4079695795 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880bc070 con 0x7f659415bbe0 2024-09-17T02:51:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f657bfff640 1 -- 172.21.15.131:0/4079695795 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6588004430 con 0x7f659415bbe0 2024-09-17T02:51:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f659a0ea640 1 -- 172.21.15.131:0/4079695795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659406e850 con 0x7f659415bbe0 2024-09-17T02:51:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f657bfff640 1 -- 172.21.15.131:0/4079695795 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880b2040 con 0x7f659415bbe0 2024-09-17T02:51:08.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.295+0000 7f659a0ea640 1 -- 172.21.15.131:0/4079695795 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6594165bc0 con 0x7f659415bbe0 2024-09-17T02:51:08.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f657bfff640 1 -- 172.21.15.131:0/4079695795 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f65880bf070 con 0x7f659415bbe0 2024-09-17T02:51:08.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f657bfff640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6580073230 0x7f6580075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f657bfff640 1 -- 172.21.15.131:0/4079695795 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f658812dcf0 con 0x7f659415bbe0 2024-09-17T02:51:08.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65937fe640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6580073230 0x7f6580075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65a13b5000 1 Processor -- start 2024-09-17T02:51:08.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65a13b5000 1 -- start start 2024-09-17T02:51:08.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65937fe640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6580073230 0x7f6580075740 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f658c007e90 tx=0x7f658c0073a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e50866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e5086be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e5087120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65a13b5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5572e4f5b410 con 0x5572e4f35950 2024-09-17T02:51:08.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65a13b5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5572e4e98b40 con 0x5572e4b45460 2024-09-17T02:51:08.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65a13b5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5572e4e87f70 con 0x5572e4f1eca0 2024-09-17T02:51:08.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e5086be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e5086be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47038/0 (socket says 172.21.15.131:47038) 2024-09-17T02:51:08.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f6592ffd640 1 -- 172.21.15.131:0/3791752255 learned_addr learned my addr 172.21.15.131:0/3791752255 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f6593fff640 1 --2- 172.21.15.131:0/3791752255 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e5087120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65937fe640 1 --2- 172.21.15.131:0/3791752255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e50866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f6592ffd640 1 -- 172.21.15.131:0/3791752255 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 msgr2=0x5572e5087120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f6592ffd640 1 --2- 172.21.15.131:0/3791752255 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e5087120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f6592ffd640 1 -- 172.21.15.131:0/3791752255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 msgr2=0x5572e50866a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f6592ffd640 1 --2- 172.21.15.131:0/3791752255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e50866a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f6592ffd640 1 -- 172.21.15.131:0/3791752255 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e4e4f5b0 con 0x5572e4f35950 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.299+0000 7f65937fe640 1 --2- 172.21.15.131:0/3791752255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e50866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f6592ffd640 1 --2- 172.21.15.131:0/3791752255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e5086be0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f65880b0040 tx=0x7f65880a56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdccd7634f714f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65797fa640 1 -- 172.21.15.131:0/3791752255 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880f7ed0 con 0x5572e4f35950 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65797fa640 1 -- 172.21.15.131:0/3791752255 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65880f83c0 con 0x5572e4f35950 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65797fa640 1 -- 172.21.15.131:0/3791752255 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880fcea0 con 0x5572e4f35950 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 -- 172.21.15.131:0/3791752255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 msgr2=0x5572e5086be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 --2- 172.21.15.131:0/3791752255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e5086be0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f65880b0040 tx=0x7f65880a56f0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 -- 172.21.15.131:0/3791752255 shutdown_connections 2024-09-17T02:51:08.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 --2- 172.21.15.131:0/3791752255 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e5087120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 --2- 172.21.15.131:0/3791752255 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e5086be0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 --2- 172.21.15.131:0/3791752255 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e50866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 -- 172.21.15.131:0/3791752255 >> 172.21.15.131:0/3791752255 conn(0x5572e500bd90 msgr2=0x5572e503b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 -- 172.21.15.131:0/3791752255 shutdown_connections 2024-09-17T02:51:08.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 -- 172.21.15.131:0/3791752255 wait complete. 2024-09-17T02:51:08.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 -- start start 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 Processor -- start 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e508d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e508de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e508e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5572e4e225f0 con 0x5572e4f35950 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5572e4e8bd80 con 0x5572e4b45460 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5572e4e2e520 con 0x5572e4f1eca0 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65a13b5000 10 client.0 _ll_register_callbacks cb 0x5572e4e32830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f6593fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e508e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f6593fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e508e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47044/0 (socket says 172.21.15.131:47044) 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f6593fff640 1 -- 172.21.15.131:0/72545297 learned_addr learned my addr 172.21.15.131:0/72545297 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f6592ffd640 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e508de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65937fe640 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e508d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f6593fff640 1 -- 172.21.15.131:0/72545297 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 msgr2=0x5572e508de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f6593fff640 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e508de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f6593fff640 1 -- 172.21.15.131:0/72545297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 msgr2=0x5572e508d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f6593fff640 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e508d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f6593fff640 1 -- 172.21.15.131:0/72545297 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e4e26190 con 0x5572e4f35950 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.303+0000 7f65937fe640 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e508d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f6593fff640 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e508e3a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f658400edc0 tx=0x7f6584001440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1e606ad8216c08 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65727fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65727fc640 1 -- 172.21.15.131:0/72545297 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6584005230 con 0x5572e4f35950 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 -- 172.21.15.131:0/72545297 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572e4e20110 con 0x5572e4f35950 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65727fc640 1 -- 172.21.15.131:0/72545297 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f65840053d0 con 0x5572e4f35950 2024-09-17T02:51:08.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65727fc640 1 -- 172.21.15.131:0/72545297 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f658401f050 con 0x5572e4f35950 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 10 client.8241 fetch_fsmap learned FSMap version 92 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 -- 172.21.15.131:0/72545297 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5572e4f0cf10 con 0x5572e4f35950 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65727fc640 1 -- 172.21.15.131:0/72545297 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5572e4e20110 con 0x5572e4f35950 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65727fc640 1 -- 172.21.15.131:0/72545297 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f6584022020 con 0x5572e4f35950 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 10 client.8241 fetch_fsmap finished waiting for FSMap version 92 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 10 client.8241 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 20 client.8241 populate_metadata read hostname 'smithi131' 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5572e4b2a7d0 0x5572e5097d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 4 client.8241 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 -- 172.21.15.131:0/72545297 --> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572e4e9aa90 con 0x5572e4b2a7d0 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65937fe640 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5572e4b2a7d0 0x5572e5097d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65727fc640 10 client.8241 ms_handle_connect on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65937fe640 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5572e4b2a7d0 0x5572e5097d00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c96a7d0bba77d8 server_cookie=e4b9538f735852bb in_seq=0 out_seq=0 2024-09-17T02:51:08.307 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65727fc640 1 -- 172.21.15.131:0/72545297 <== mds.0 v2:172.21.15.131:6833/1272267157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572e4e9aa90 con 0x5572e4b2a7d0 2024-09-17T02:51:08.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65727fc640 10 client.8241 handle_command_reply: tid=0 2024-09-17T02:51:08.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 client.8241 shutdown 2024-09-17T02:51:08.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 -- 172.21.15.131:0/72545297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 msgr2=0x5572e508e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e508e3a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f658400edc0 tx=0x7f6584001440 comp rx=0 tx=0).stop 2024-09-17T02:51:08.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 -- 172.21.15.131:0/72545297 shutdown_connections 2024-09-17T02:51:08.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5572e4b2a7d0 0x5572e5097d00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e508e3a0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e508de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 --2- 172.21.15.131:0/72545297 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e508d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.308 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65727fc640 0 client.8241 ms_handle_reset on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 -- 172.21.15.131:0/72545297 >> 172.21.15.131:0/72545297 conn(0x5572e500bd90 msgr2=0x5572e5042440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 -- 172.21.15.131:0/72545297 shutdown_connections 2024-09-17T02:51:08.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 1 -- 172.21.15.131:0/72545297 wait complete. 2024-09-17T02:51:08.309 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.307+0000 7f65a13b5000 20 client.8241 trim_cache size 0 max 16384 2024-09-17T02:51:08.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f65a13b5000 1 Processor -- start 2024-09-17T02:51:08.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f65a13b5000 1 -- start start 2024-09-17T02:51:08.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4b45460 0x5572e50f8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f1eca0 0x5572e50f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4f35950 0x5572e50f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f65a13b5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6594205bd0 con 0x5572e4f1eca0 2024-09-17T02:51:08.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f65a13b5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f6594204bd0 con 0x5572e4f35950 2024-09-17T02:51:08.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f65a13b5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6594203f90 con 0x5572e4b45460 2024-09-17T02:51:08.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f1eca0 0x5572e50f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6592ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f1eca0 0x5572e50f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47056/0 (socket says 172.21.15.131:47056) 2024-09-17T02:51:08.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6592ffd640 1 -- 172.21.15.131:0/2734459442 learned_addr learned my addr 172.21.15.131:0/2734459442 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6592ffd640 1 -- 172.21.15.131:0/2734459442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4b45460 msgr2=0x5572e50f8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6593fff640 1 --2- 172.21.15.131:0/2734459442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4f35950 0x5572e50f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f65937fe640 1 --2- 172.21.15.131:0/2734459442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4b45460 0x5572e50f8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6592ffd640 1 --2- 172.21.15.131:0/2734459442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4b45460 0x5572e50f8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6592ffd640 1 -- 172.21.15.131:0/2734459442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4f35950 msgr2=0x5572e50f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6592ffd640 1 --2- 172.21.15.131:0/2734459442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4f35950 0x5572e50f96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6592ffd640 1 -- 172.21.15.131:0/2734459442 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e4ee3db0 con 0x5572e4f1eca0 2024-09-17T02:51:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6592ffd640 1 --2- 172.21.15.131:0/2734459442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f1eca0 0x5572e50f91a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f658812c020 tx=0x7f658801c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a908e421e9521268 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6578ff9640 1 -- 172.21.15.131:0/2734459442 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6588101de0 con 0x5572e4f1eca0 2024-09-17T02:51:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.327+0000 7f6578ff9640 1 -- 172.21.15.131:0/2734459442 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65880f83c0 con 0x5572e4f1eca0 2024-09-17T02:51:08.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f6578ff9640 1 -- 172.21.15.131:0/2734459442 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65880fcea0 con 0x5572e4f1eca0 2024-09-17T02:51:08.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 -- 172.21.15.131:0/2734459442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f1eca0 msgr2=0x5572e50f91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 --2- 172.21.15.131:0/2734459442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f1eca0 0x5572e50f91a0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f658812c020 tx=0x7f658801c790 comp rx=0 tx=0).stop 2024-09-17T02:51:08.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 -- 172.21.15.131:0/2734459442 shutdown_connections 2024-09-17T02:51:08.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 --2- 172.21.15.131:0/2734459442 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4f35950 0x5572e50f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 --2- 172.21.15.131:0/2734459442 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f1eca0 0x5572e50f91a0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 --2- 172.21.15.131:0/2734459442 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4b45460 0x5572e50f8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 -- 172.21.15.131:0/2734459442 >> 172.21.15.131:0/2734459442 conn(0x5572e500bd90 msgr2=0x5572e503acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 -- 172.21.15.131:0/2734459442 shutdown_connections 2024-09-17T02:51:08.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 -- 172.21.15.131:0/2734459442 wait complete. 2024-09-17T02:51:08.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 -- start start 2024-09-17T02:51:08.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 Processor -- start 2024-09-17T02:51:08.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e50fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e50fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e50fd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5572e4f0ca80 con 0x5572e4f35950 2024-09-17T02:51:08.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5572e4e8b090 con 0x5572e4b45460 2024-09-17T02:51:08.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5572e4e903b0 con 0x5572e4f1eca0 2024-09-17T02:51:08.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65a13b5000 10 client.0 _ll_register_callbacks cb 0x7f6584007cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f6593fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e50fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f6593fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e50fd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47060/0 (socket says 172.21.15.131:47060) 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f6593fff640 1 -- 172.21.15.131:0/3081305765 learned_addr learned my addr 172.21.15.131:0/3081305765 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65937fe640 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e50fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f6592ffd640 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e50fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f6593fff640 1 -- 172.21.15.131:0/3081305765 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 msgr2=0x5572e50fcb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f6593fff640 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e50fcb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f6593fff640 1 -- 172.21.15.131:0/3081305765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 msgr2=0x5572e50fc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f6593fff640 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e50fc5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f6593fff640 1 -- 172.21.15.131:0/3081305765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e4e4eab0 con 0x5572e4f35950 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f6593fff640 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e50fd060 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f658400e290 tx=0x7f6584014c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72c06c71a352750 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.331+0000 7f65727fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65727fc640 1 -- 172.21.15.131:0/3081305765 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f658402c640 con 0x5572e4f35950 2024-09-17T02:51:08.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65727fc640 1 -- 172.21.15.131:0/3081305765 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f65840049d0 con 0x5572e4f35950 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 -- 172.21.15.131:0/3081305765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65880f83c0 con 0x5572e4f35950 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65727fc640 1 -- 172.21.15.131:0/3081305765 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f6584001940 con 0x5572e4f35950 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 10 client.8247 fetch_fsmap learned FSMap version 92 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 -- 172.21.15.131:0/3081305765 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x5572e4e2d070 con 0x5572e4f35950 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65727fc640 1 -- 172.21.15.131:0/3081305765 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65880f83c0 con 0x5572e4f35950 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65727fc640 1 -- 172.21.15.131:0/3081305765 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f658402a070 con 0x5572e4f35950 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 10 client.8247 fetch_fsmap finished waiting for FSMap version 92 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 10 client.8247 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 20 client.8247 populate_metadata read hostname 'smithi131' 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5572e4b18830 0x5572e50b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 4 client.8247 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 -- 172.21.15.131:0/3081305765 --> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] -- command(tid 0: {"prefix": "status"}) -- 0x5572e4e96540 con 0x5572e4b18830 2024-09-17T02:51:08.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65937fe640 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5572e4b18830 0x5572e50b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65727fc640 10 client.8247 ms_handle_connect on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65937fe640 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5572e4b18830 0x5572e50b88e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b6e1600c31643ac server_cookie=b19629cb2ccdb907 in_seq=0 out_seq=0 2024-09-17T02:51:08.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65727fc640 1 -- 172.21.15.131:0/3081305765 <== mds.0 v2:172.21.15.131:6833/1272267157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x5572e4e96540 con 0x5572e4b18830 2024-09-17T02:51:08.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65727fc640 10 client.8247 handle_command_reply: tid=0 2024-09-17T02:51:08.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 client.8247 shutdown 2024-09-17T02:51:08.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 -- 172.21.15.131:0/3081305765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 msgr2=0x5572e50fd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e50fd060 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f658400e290 tx=0x7f6584014c90 comp rx=0 tx=0).stop 2024-09-17T02:51:08.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 -- 172.21.15.131:0/3081305765 shutdown_connections 2024-09-17T02:51:08.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5572e4b18830 0x5572e50b88e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5572e4f35950 0x5572e50fd060 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65727fc640 0 client.8247 ms_handle_reset on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5572e4f1eca0 0x5572e50fcb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 --2- 172.21.15.131:0/3081305765 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5572e4b45460 0x5572e50fc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 -- 172.21.15.131:0/3081305765 >> 172.21.15.131:0/3081305765 conn(0x5572e500bd90 msgr2=0x5572e503b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 -- 172.21.15.131:0/3081305765 shutdown_connections 2024-09-17T02:51:08.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 1 -- 172.21.15.131:0/3081305765 wait complete. 2024-09-17T02:51:08.337 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f65a13b5000 20 client.8247 trim_cache size 0 max 16384 2024-09-17T02:51:08.337 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:51:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:51:08.337 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:51:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8175, 2024-09-17T02:51:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:51:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:51:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:51:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 4.6132296000000004, 2024-09-17T02:51:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 92, 2024-09-17T02:51:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 110, 2024-09-17T02:51:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 110, 2024-09-17T02:51:08.338 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 5.5642327920000003 2024-09-17T02:51:08.338 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:51:08.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 -- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6580073230 msgr2=0x7f6580075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6580073230 0x7f6580075740 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f658c007e90 tx=0x7f658c0073a0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 -- 172.21.15.131:0/4079695795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f659415bbe0 msgr2=0x7f6594148380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f659415bbe0 0x7f6594148380 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f658801ae40 tx=0x7f65880a5600 comp rx=0 tx=0).stop 2024-09-17T02:51:08.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 -- 172.21.15.131:0/4079695795 shutdown_connections 2024-09-17T02:51:08.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6580073230 0x7f6580075740 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6594148ae0 0x7f659414fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f659415bbe0 0x7f6594148380 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 --2- 172.21.15.131:0/4079695795 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f659415b7d0 0x7f65941f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 -- 172.21.15.131:0/4079695795 >> 172.21.15.131:0/4079695795 conn(0x7f659406fdb0 msgr2=0x7f659414a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 -- 172.21.15.131:0/4079695795 shutdown_connections 2024-09-17T02:51:08.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.335+0000 7f659a0ea640 1 -- 172.21.15.131:0/4079695795 wait complete. 2024-09-17T02:51:08.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-17T02:51:08.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:51:08.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-09-17T02:51:08.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.511+0000 7f21f453b640 1 -- 172.21.15.131:0/4244805436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d8004d50 msgr2=0x7f21d8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.511+0000 7f21f453b640 1 --2- 172.21.15.131:0/4244805436 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21d8004d50 0x7f21d8005160 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009ee0 tx=0x7f21e401d5f0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.511+0000 7f21f2ad8640 1 -- 172.21.15.131:0/4244805436 reap_dead start 2024-09-17T02:51:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.511+0000 7f21f453b640 1 -- 172.21.15.131:0/4244805436 shutdown_connections 2024-09-17T02:51:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.511+0000 7f21f453b640 1 -- 172.21.15.131:0/4244805436 >> 172.21.15.131:0/4244805436 conn(0x7f21ec06fdb0 msgr2=0x7f21ec0701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.511+0000 7f21f453b640 1 -- 172.21.15.131:0/4244805436 shutdown_connections 2024-09-17T02:51:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.511+0000 7f21f453b640 1 -- 172.21.15.131:0/4244805436 wait complete. 2024-09-17T02:51:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f453b640 1 Processor -- start 2024-09-17T02:51:08.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f453b640 1 -- start start 2024-09-17T02:51:08.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f453b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21ec155da0 0x7f21ec1e8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f453b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21ec1e7000 0x7f21ec1e7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f453b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21ec130430 0x7f21ec130840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f453b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1456c0 con 0x7f21ec155da0 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f453b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec145540 con 0x7f21ec130430 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f453b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec145840 con 0x7f21ec1e7000 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f22d7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21ec155da0 0x7f21ec1e8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f22d7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21ec155da0 0x7f21ec1e8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47068/0 (socket says 172.21.15.131:47068) 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f22d7640 1 -- 172.21.15.131:0/1070241266 learned_addr learned my addr 172.21.15.131:0/1070241266 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f1ad6640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21ec1e7000 0x7f21ec1e7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21ec130430 0x7f21ec130840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f22d7640 1 -- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21ec1e7000 msgr2=0x7f21ec1e7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f22d7640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21ec1e7000 0x7f21ec1e7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f22d7640 1 -- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21ec130430 msgr2=0x7f21ec130840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f22d7640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21ec130430 0x7f21ec130840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f22d7640 1 -- 172.21.15.131:0/1070241266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21ec1e9350 con 0x7f21ec155da0 2024-09-17T02:51:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21ec130430 0x7f21ec130840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f22d7640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21ec155da0 0x7f21ec1e8e10 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f21e000c730 tx=0x7f21e000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21deffd640 1 -- 172.21.15.131:0/1070241266 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0017070 con 0x7f21ec155da0 2024-09-17T02:51:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21deffd640 1 -- 172.21.15.131:0/1070241266 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21e0005380 con 0x7f21ec155da0 2024-09-17T02:51:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f453b640 1 -- 172.21.15.131:0/1070241266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21ec1e7d70 con 0x7f21ec155da0 2024-09-17T02:51:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21deffd640 1 -- 172.21.15.131:0/1070241266 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0003940 con 0x7f21ec155da0 2024-09-17T02:51:08.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.515+0000 7f21f453b640 1 -- 172.21.15.131:0/1070241266 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21ec1e8330 con 0x7f21ec155da0 2024-09-17T02:51:08.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.519+0000 7f21deffd640 1 -- 172.21.15.131:0/1070241266 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f21e001a070 con 0x7f21ec155da0 2024-09-17T02:51:08.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.519+0000 7f21deffd640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21d0073200 0x7f21d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.519+0000 7f21deffd640 1 -- 172.21.15.131:0/1070241266 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f21e0093ad0 con 0x7f21ec155da0 2024-09-17T02:51:08.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.519+0000 7f21f1ad6640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21d0073200 0x7f21d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.519+0000 7f21fb806000 1 Processor -- start 2024-09-17T02:51:08.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.519+0000 7f21fb806000 1 -- start start 2024-09-17T02:51:08.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.519+0000 7f21f1ad6640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21d0073200 0x7f21d0075710 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0b40b0 tx=0x7f21e80091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780e021a0 0x559780f4b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f4c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559780d10bd0 con 0x559780de4970 2024-09-17T02:51:08.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559780d01430 con 0x559780dfaae0 2024-09-17T02:51:08.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559780d5ece0 con 0x559780e021a0 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f2ad8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f4c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f2ad8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f4c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47082/0 (socket says 172.21.15.131:47082) 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3284718633 learned_addr learned my addr 172.21.15.131:0/3284718633 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f1ad6640 1 --2- 172.21.15.131:0/3284718633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f22d7640 1 --2- 172.21.15.131:0/3284718633 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780e021a0 0x559780f4b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3284718633 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780e021a0 msgr2=0x559780f4b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/3284718633 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780e021a0 0x559780f4b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3284718633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 msgr2=0x559780f4bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/3284718633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f4bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3284718633 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559780d02d80 con 0x559780de4970 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f22d7640 1 --2- 172.21.15.131:0/3284718633 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780e021a0 0x559780f4b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:08.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/3284718633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f4c0c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f21e4002240 tx=0x7f21e401a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd0fbe6af55977 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21cffff640 1 -- 172.21.15.131:0/3284718633 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e40ad040 con 0x559780de4970 2024-09-17T02:51:08.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21cffff640 1 -- 172.21.15.131:0/3284718633 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21e4004d40 con 0x559780de4970 2024-09-17T02:51:08.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21cffff640 1 -- 172.21.15.131:0/3284718633 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e4003ba0 con 0x559780de4970 2024-09-17T02:51:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 -- 172.21.15.131:0/3284718633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 msgr2=0x559780f4c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 --2- 172.21.15.131:0/3284718633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f4c0c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f21e4002240 tx=0x7f21e401a390 comp rx=0 tx=0).stop 2024-09-17T02:51:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 -- 172.21.15.131:0/3284718633 shutdown_connections 2024-09-17T02:51:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 --2- 172.21.15.131:0/3284718633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f4c0c0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 --2- 172.21.15.131:0/3284718633 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f4bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 --2- 172.21.15.131:0/3284718633 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780e021a0 0x559780f4b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 -- 172.21.15.131:0/3284718633 >> 172.21.15.131:0/3284718633 conn(0x559780e088c0 msgr2=0x559780f004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 -- 172.21.15.131:0/3284718633 shutdown_connections 2024-09-17T02:51:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 -- 172.21.15.131:0/3284718633 wait complete. 2024-09-17T02:51:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 -- start start 2024-09-17T02:51:08.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.523+0000 7f21fb806000 1 Processor -- start 2024-09-17T02:51:08.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f38552cb640 1 -- 172.21.15.131:0/1409921929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3830004810 msgr2=0x7f3830004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f38552cb640 1 --2- 172.21.15.131:0/1409921929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3830004810 0x7f3830004c20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f384000c260 tx=0x7f384001ced0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f38552cb640 1 -- 172.21.15.131:0/1409921929 shutdown_connections 2024-09-17T02:51:08.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f38552cb640 1 --2- 172.21.15.131:0/1409921929 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3830004810 0x7f3830004c20 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f38552cb640 1 -- 172.21.15.131:0/1409921929 >> 172.21.15.131:0/1409921929 conn(0x7f3850070bc0 msgr2=0x7f3850073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f38552cb640 1 -- 172.21.15.131:0/1409921929 shutdown_connections 2024-09-17T02:51:08.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f38552cb640 1 -- 172.21.15.131:0/1409921929 wait complete. 2024-09-17T02:51:08.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f07640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f07b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780e021a0 0x559780f36f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559780d51f20 con 0x559780de4970 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x559780d4e110 con 0x559780dfaae0 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559780cf36d0 con 0x559780e021a0 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 10 client.0 _ll_register_callbacks cb 0x559780cf2330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f22d7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f07640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f22d7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f07640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47100/0 (socket says 172.21.15.131:47100) 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f22d7640 1 -- 172.21.15.131:0/3691002224 learned_addr learned my addr 172.21.15.131:0/3691002224 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f1ad6640 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780e021a0 0x559780f36f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f22d7640 1 -- 172.21.15.131:0/3691002224 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780e021a0 msgr2=0x559780f36f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f22d7640 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780e021a0 0x559780f36f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f22d7640 1 -- 172.21.15.131:0/3691002224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 msgr2=0x559780f07b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f22d7640 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f07b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f22d7640 1 -- 172.21.15.131:0/3691002224 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559780cec310 con 0x559780de4970 2024-09-17T02:51:08.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f1ad6640 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f22d7640 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f07640 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f21e0000fc0 tx=0x7f21e005daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f57b3101006ad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3691002224 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e0083020 con 0x559780de4970 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3691002224 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f21e00679d0 con 0x559780de4970 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 -- 172.21.15.131:0/3691002224 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559780ce6450 con 0x559780de4970 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3691002224 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f21e007d050 con 0x559780de4970 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 10 client.8262 fetch_fsmap learned FSMap version 92 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 -- 172.21.15.131:0/3691002224 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x559780d09f60 con 0x559780de4970 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3691002224 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x559780ce6450 con 0x559780de4970 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3691002224 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f21e005d520 con 0x559780de4970 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 10 client.8262 fetch_fsmap finished waiting for FSMap version 92 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 10 client.8262 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:51:08.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 20 client.8262 populate_metadata read hostname 'smithi131' 2024-09-17T02:51:08.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5597809f0b80 0x559780f528c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 4 client.8262 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:51:08.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 -- 172.21.15.131:0/3691002224 --> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559780d60c30 con 0x5597809f0b80 2024-09-17T02:51:08.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f1ad6640 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5597809f0b80 0x559780f528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f38552cb640 1 Processor -- start 2024-09-17T02:51:08.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f38552cb640 1 -- start start 2024-09-17T02:51:08.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21cd7fa640 10 client.8262 ms_handle_connect on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21f1ad6640 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5597809f0b80 0x559780f528c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de26765065acd921 server_cookie=f588f84a90a9b307 in_seq=0 out_seq=0 2024-09-17T02:51:08.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3691002224 <== mds.0 v2:172.21.15.131:6833/1272267157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559780d60c30 con 0x5597809f0b80 2024-09-17T02:51:08.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21cd7fa640 10 client.8262 handle_command_reply: tid=0 2024-09-17T02:51:08.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 client.8262 shutdown 2024-09-17T02:51:08.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 -- 172.21.15.131:0/3691002224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 msgr2=0x559780f07640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f07640 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f21e0000fc0 tx=0x7f21e005daf0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 -- 172.21.15.131:0/3691002224 shutdown_connections 2024-09-17T02:51:08.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5597809f0b80 0x559780f528c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780e021a0 0x559780f36f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f07b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21cd7fa640 0 client.8262 ms_handle_reset on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 --2- 172.21.15.131:0/3691002224 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780de4970 0x559780f07640 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 -- 172.21.15.131:0/3691002224 >> 172.21.15.131:0/3691002224 conn(0x559780e088c0 msgr2=0x559780f00c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 -- 172.21.15.131:0/3691002224 shutdown_connections 2024-09-17T02:51:08.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 1 -- 172.21.15.131:0/3691002224 wait complete. 2024-09-17T02:51:08.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.527+0000 7f21fb806000 20 client.8262 trim_cache size 0 max 16384 2024-09-17T02:51:08.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38552cb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3830004810 0x7f385013d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38552cb640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f385014e5b0 0x7f385013a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38552cb640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f385014c4a0 0x7f385014c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38552cb640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f385015f360 con 0x7f3830004810 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38552cb640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3850074df0 con 0x7f385014c4a0 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38552cb640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f385015f4e0 con 0x7f385014e5b0 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f384effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3830004810 0x7f385013d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f384effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3830004810 0x7f385013d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47104/0 (socket says 172.21.15.131:47104) 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f384effd640 1 -- 172.21.15.131:0/1977139069 learned_addr learned my addr 172.21.15.131:0/1977139069 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f384f7fe640 1 --2- 172.21.15.131:0/1977139069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f385014c4a0 0x7f385014c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f3847fff640 1 --2- 172.21.15.131:0/1977139069 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f385014e5b0 0x7f385013a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f384effd640 1 -- 172.21.15.131:0/1977139069 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f385014e5b0 msgr2=0x7f385013a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f384effd640 1 --2- 172.21.15.131:0/1977139069 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f385014e5b0 0x7f385013a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f384effd640 1 -- 172.21.15.131:0/1977139069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f385014c4a0 msgr2=0x7f385014c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f384effd640 1 --2- 172.21.15.131:0/1977139069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f385014c4a0 0x7f385014c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f384effd640 1 -- 172.21.15.131:0/1977139069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385014cdf0 con 0x7f3830004810 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f384effd640 1 --2- 172.21.15.131:0/1977139069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3830004810 0x7f385013d5e0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f3838003eb0 tx=0x7f3838001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38477fe640 1 -- 172.21.15.131:0/1977139069 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838015070 con 0x7f3830004810 2024-09-17T02:51:08.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38477fe640 1 -- 172.21.15.131:0/1977139069 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3838011840 con 0x7f3830004810 2024-09-17T02:51:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38552cb640 1 -- 172.21.15.131:0/1977139069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385014ecc0 con 0x7f3830004810 2024-09-17T02:51:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38477fe640 1 -- 172.21.15.131:0/1977139069 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3838011c50 con 0x7f3830004810 2024-09-17T02:51:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38552cb640 1 -- 172.21.15.131:0/1977139069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385014f280 con 0x7f3830004810 2024-09-17T02:51:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38552cb640 1 -- 172.21.15.131:0/1977139069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f385006e080 con 0x7f3830004810 2024-09-17T02:51:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38477fe640 1 -- 172.21.15.131:0/1977139069 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3838018070 con 0x7f3830004810 2024-09-17T02:51:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38477fe640 1 --2- 172.21.15.131:0/1977139069 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3834073200 0x7f3834075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38477fe640 1 -- 172.21.15.131:0/1977139069 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f3838095060 con 0x7f3830004810 2024-09-17T02:51:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38477fe640 1 -- 172.21.15.131:0/1977139069 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+516 (secure 0 0 0) 0x7f385014cdf0 con 0x7f3830004810 2024-09-17T02:51:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f3847fff640 1 --2- 172.21.15.131:0/1977139069 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3834073200 0x7f3834075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.531+0000 7f38552cb640 1 -- 172.21.15.131:0/1977139069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f38500b5970 con 0x7f3830004810 2024-09-17T02:51:08.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.535+0000 7f38477fe640 1 -- 172.21.15.131:0/1977139069 <== mon.0 v2:172.21.15.131:3300/0 7 ==== log(2 entries from seq 988 at 2024-09-17T02:51:06.749735+0000) ==== 389+0+0 (secure 0 0 0) 0x7f3838093450 con 0x7f3830004810 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: cluster: 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: id: f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: health: HEALTH_OK 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: services: 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: mgr: x(active, since 10m), standbys: y 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: mds: 1/1 daemons up, 3 standby 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: data: 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: volumes: 1/1 healthy 2024-09-17T02:51:08.537 INFO:teuthology.orchestra.run.smithi131.stdout: pools: 3 pools, 97 pgs 2024-09-17T02:51:08.538 INFO:teuthology.orchestra.run.smithi131.stdout: objects: 30 objects, 19 MiB 2024-09-17T02:51:08.538 INFO:teuthology.orchestra.run.smithi131.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-09-17T02:51:08.538 INFO:teuthology.orchestra.run.smithi131.stdout: pgs: 97 active+clean 2024-09-17T02:51:08.538 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:08.538 INFO:teuthology.orchestra.run.smithi131.stdout: io: 2024-09-17T02:51:08.538 INFO:teuthology.orchestra.run.smithi131.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-09-17T02:51:08.538 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:08.538 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:08.538 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:51:06.749735+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-17T02:51:08.538 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:51:06.749793+0000 mon.a [INF] Cluster is now healthy 2024-09-17T02:51:08.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.535+0000 7f3847fff640 1 --2- 172.21.15.131:0/1977139069 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3834073200 0x7f3834075710 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f38400ab000 tx=0x7f3840019d10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.547+0000 7f21fb806000 1 Processor -- start 2024-09-17T02:51:08.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.547+0000 7f21fb806000 1 -- start start 2024-09-17T02:51:08.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.547+0000 7f21fb806000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780de4970 0x559780f04370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.547+0000 7f21fb806000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f63280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.547+0000 7f21fb806000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780f637c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.547+0000 7f21fb806000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x559780f06760 con 0x559780e021a0 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.547+0000 7f21fb806000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec2012f0 con 0x559780dfaae0 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.547+0000 7f21fb806000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f21ec200330 con 0x559780de4970 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21f2ad8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780f637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21f2ad8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780f637c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47120/0 (socket says 172.21.15.131:47120) 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3577026828 learned_addr learned my addr 172.21.15.131:0/3577026828 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21f22d7640 1 --2- 172.21.15.131:0/3577026828 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780de4970 0x559780f04370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21f1ad6640 1 --2- 172.21.15.131:0/3577026828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f63280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3577026828 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780de4970 msgr2=0x559780f04370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/3577026828 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780de4970 0x559780f04370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3577026828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 msgr2=0x559780f63280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/3577026828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f63280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3577026828 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559780dd2f30 con 0x559780e021a0 2024-09-17T02:51:08.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/3577026828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780f637c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009a40 tx=0x7f21d8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d81a695dfe108d server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21cf7fe640 1 -- 172.21.15.131:0/3577026828 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e40bc070 con 0x559780e021a0 2024-09-17T02:51:08.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21cf7fe640 1 -- 172.21.15.131:0/3577026828 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21e4003980 con 0x559780e021a0 2024-09-17T02:51:08.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21cf7fe640 1 -- 172.21.15.131:0/3577026828 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e40ad070 con 0x559780e021a0 2024-09-17T02:51:08.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 -- 172.21.15.131:0/3577026828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 msgr2=0x559780f637c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 --2- 172.21.15.131:0/3577026828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780f637c0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009a40 tx=0x7f21d8005840 comp rx=0 tx=0).stop 2024-09-17T02:51:08.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 -- 172.21.15.131:0/3577026828 shutdown_connections 2024-09-17T02:51:08.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 --2- 172.21.15.131:0/3577026828 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780f637c0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 --2- 172.21.15.131:0/3577026828 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780f63280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 --2- 172.21.15.131:0/3577026828 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780de4970 0x559780f04370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 -- 172.21.15.131:0/3577026828 >> 172.21.15.131:0/3577026828 conn(0x559780e088c0 msgr2=0x559780f4a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 -- 172.21.15.131:0/3577026828 shutdown_connections 2024-09-17T02:51:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 -- 172.21.15.131:0/3577026828 wait complete. 2024-09-17T02:51:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 -- start start 2024-09-17T02:51:08.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 Processor -- start 2024-09-17T02:51:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.551+0000 7f21fb806000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780de4970 0x559780fc0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780fc0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780fc12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1ffce0 con 0x559780e021a0 2024-09-17T02:51:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f21ec1435d0 con 0x559780dfaae0 2024-09-17T02:51:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x559780a138d0 con 0x559780de4970 2024-09-17T02:51:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 10 client.0 _ll_register_callbacks cb 0x7f21e00055f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:51:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f2ad8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780fc12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f2ad8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780fc12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47122/0 (socket says 172.21.15.131:47122) 2024-09-17T02:51:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3922110728 learned_addr learned my addr 172.21.15.131:0/3922110728 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.553 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f1ad6640 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780fc0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f22d7640 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780de4970 0x559780fc0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3922110728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780de4970 msgr2=0x559780fc0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780de4970 0x559780fc0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3922110728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 msgr2=0x559780fc0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780fc0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f2ad8640 1 -- 172.21.15.131:0/3922110728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559780da9dd0 con 0x559780e021a0 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f22d7640 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780de4970 0x559780fc0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f2ad8640 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780fc12d0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009e00 tx=0x7f21e4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255367b0f82c35c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3922110728 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21e40bc070 con 0x559780e021a0 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3922110728 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f21e4005240 con 0x559780e021a0 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 -- 172.21.15.131:0/3922110728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21e4003980 con 0x559780e021a0 2024-09-17T02:51:08.554 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3922110728 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f21e40c7050 con 0x559780e021a0 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 10 client.8271 fetch_fsmap learned FSMap version 92 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 -- 172.21.15.131:0/3922110728 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x559780ce4840 con 0x559780e021a0 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3922110728 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21e4003980 con 0x559780e021a0 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3922110728 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f21e40ca020 con 0x559780e021a0 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 10 client.8271 fetch_fsmap finished waiting for FSMap version 92 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 10 client.8271 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 20 client.8271 populate_metadata read hostname 'smithi131' 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5597809debe0 0x559780f02ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 4 client.8271 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 -- 172.21.15.131:0/3922110728 --> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x559780d5c6e0 con 0x5597809debe0 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f22d7640 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5597809debe0 0x559780f02ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21cd7fa640 10 client.8271 ms_handle_connect on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.555 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21f22d7640 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5597809debe0 0x559780f02ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d930f97cd7eb085 server_cookie=a2a74b6b9ff6839b in_seq=0 out_seq=0 2024-09-17T02:51:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21cd7fa640 1 -- 172.21.15.131:0/3922110728 <== mds.0 v2:172.21.15.131:6833/1272267157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559780d5c6e0 con 0x5597809debe0 2024-09-17T02:51:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21cd7fa640 10 client.8271 handle_command_reply: tid=0 2024-09-17T02:51:08.556 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 client.8271 shutdown 2024-09-17T02:51:08.556 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f4552f0a640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-09-17T02:51:08.556 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f4552f0a640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-09-17T02:51:08.556 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f4552f0a640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-09-17T02:51:08.557 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f4552f0a640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-09-17T02:51:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 -- 172.21.15.131:0/3922110728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 msgr2=0x559780fc12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780fc12d0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009e00 tx=0x7f21e4009310 comp rx=0 tx=0).stop 2024-09-17T02:51:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 -- 172.21.15.131:0/3922110728 shutdown_connections 2024-09-17T02:51:08.557 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x5597809debe0 0x559780f02ae0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x559780e021a0 0x559780fc12d0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x559780dfaae0 0x559780fc0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 --2- 172.21.15.131:0/3922110728 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x559780de4970 0x559780fc0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21cd7fa640 0 client.8271 ms_handle_reset on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 -- 172.21.15.131:0/3922110728 >> 172.21.15.131:0/3922110728 conn(0x559780e088c0 msgr2=0x559780f4add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 -- 172.21.15.131:0/3922110728 shutdown_connections 2024-09-17T02:51:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 1 -- 172.21.15.131:0/3922110728 wait complete. 2024-09-17T02:51:08.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.555+0000 7f21fb806000 20 client.8271 trim_cache size 0 max 16384 2024-09-17T02:51:08.558 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:51:08.559 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0, 2024-09-17T02:51:08.559 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_tag": "058dac6f-2cd4-464c-816c-a4e52b1b7d46", 2024-09-17T02:51:08.559 INFO:teuthology.orchestra.run.smithi131.stdout: "mode": "asynchronous" 2024-09-17T02:51:08.559 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:51:08.559 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 -- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21d0073200 msgr2=0x7f21d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21d0073200 0x7f21d0075710 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f21ec0b40b0 tx=0x7f21e80091e0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 -- 172.21.15.131:0/1070241266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21ec155da0 msgr2=0x7f21ec1e8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21ec155da0 0x7f21ec1e8e10 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f21e000c730 tx=0x7f21e000cbf0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 -- 172.21.15.131:0/1070241266 shutdown_connections 2024-09-17T02:51:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f21d0073200 0x7f21d0075710 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f21ec130430 0x7f21ec130840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f21ec1e7000 0x7f21ec1e7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 --2- 172.21.15.131:0/1070241266 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f21ec155da0 0x7f21ec1e8e10 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 -- 172.21.15.131:0/1070241266 >> 172.21.15.131:0/1070241266 conn(0x7f21ec06fdb0 msgr2=0x7f21ec06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 -- 172.21.15.131:0/1070241266 shutdown_connections 2024-09-17T02:51:08.560 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.559+0000 7f21f453b640 1 -- 172.21.15.131:0/1070241266 wait complete. 2024-09-17T02:51:08.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:51:08.573 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-17T02:51:08.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.719+0000 7f48d0f98640 1 -- 172.21.15.131:0/3733298757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc0ced00 msgr2=0x7f48cc06dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.719+0000 7f48d0f98640 1 --2- 172.21.15.131:0/3733298757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc0ced00 0x7f48cc06dea0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0cfe00 tx=0x7f48c001c8c0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.719+0000 7f48d0f98640 1 -- 172.21.15.131:0/3733298757 shutdown_connections 2024-09-17T02:51:08.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.719+0000 7f48d0f98640 1 --2- 172.21.15.131:0/3733298757 >> v2:172.21.15.189:3300/0 conn(0x7f48cc15c5b0 0x7f48cc0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.719+0000 7f48d0f98640 1 --2- 172.21.15.131:0/3733298757 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc0ced00 0x7f48cc06dea0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.719+0000 7f48d0f98640 1 -- 172.21.15.131:0/3733298757 >> 172.21.15.131:0/3733298757 conn(0x7f48cc06eea0 msgr2=0x7f48cc06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.719+0000 7f48d0f98640 1 -- 172.21.15.131:0/3733298757 shutdown_connections 2024-09-17T02:51:08.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.719+0000 7f48d0f98640 1 -- 172.21.15.131:0/3733298757 wait complete. 2024-09-17T02:51:08.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48d0f98640 1 Processor -- start 2024-09-17T02:51:08.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48d0f98640 1 -- start start 2024-09-17T02:51:08.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48d0f98640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48cc06e3e0 0x7f48cc158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48d0f98640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc0ced00 0x7f48cc154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48d0f98640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f48cc15c5b0 0x7f48cc160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48d0f98640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc168750 con 0x7f48cc0ced00 2024-09-17T02:51:08.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48d0f98640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1685d0 con 0x7f48cc06e3e0 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48d0f98640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f48cc1688d0 con 0x7f48cc15c5b0 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48c9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc0ced00 0x7f48cc154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48c9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc0ced00 0x7f48cc154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47138/0 (socket says 172.21.15.131:47138) 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48c9d9b640 1 -- 172.21.15.131:0/3160676288 learned_addr learned my addr 172.21.15.131:0/3160676288 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48cad9d640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f48cc15c5b0 0x7f48cc160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48ca59c640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48cc06e3e0 0x7f48cc158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48c9d9b640 1 -- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f48cc15c5b0 msgr2=0x7f48cc160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f48cc15c5b0 0x7f48cc160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48c9d9b640 1 -- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48cc06e3e0 msgr2=0x7f48cc158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48cc06e3e0 0x7f48cc158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48c9d9b640 1 -- 172.21.15.131:0/3160676288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc1627b0 con 0x7f48cc0ced00 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48ca59c640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48cc06e3e0 0x7f48cc158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc0ced00 0x7f48cc154130 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00c3c0 tx=0x7f48bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48baffd640 1 -- 172.21.15.131:0/3160676288 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc017070 con 0x7f48cc0ced00 2024-09-17T02:51:08.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48baffd640 1 -- 172.21.15.131:0/3160676288 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48bc0053c0 con 0x7f48cc0ced00 2024-09-17T02:51:08.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48d0f98640 1 -- 172.21.15.131:0/3160676288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc162d70 con 0x7f48cc0ced00 2024-09-17T02:51:08.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48baffd640 1 -- 172.21.15.131:0/3160676288 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc003860 con 0x7f48cc0ced00 2024-09-17T02:51:08.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48d0f98640 1 -- 172.21.15.131:0/3160676288 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc160140 con 0x7f48cc0ced00 2024-09-17T02:51:08.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.723+0000 7f48baffd640 1 -- 172.21.15.131:0/3160676288 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f48bc01a070 con 0x7f48cc0ced00 2024-09-17T02:51:08.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48baffd640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48ac073200 0x7f48ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48baffd640 1 -- 172.21.15.131:0/3160676288 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f48bc093840 con 0x7f48cc0ced00 2024-09-17T02:51:08.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48ca59c640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48ac073200 0x7f48ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48d8263000 1 Processor -- start 2024-09-17T02:51:08.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48d8263000 1 -- start start 2024-09-17T02:51:08.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48ca59c640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48ac073200 0x7f48ac075710 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f48c00af000 tx=0x7f48c001b100 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48d8263000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf4776d0 0x55d9cf5e7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48d8263000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf0ce890 0x55d9cf5e8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48d8263000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf192aa0 0x55d9cf5e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48d8263000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d9cf482cd0 con 0x55d9cf4776d0 2024-09-17T02:51:08.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48d8263000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9cf398330 con 0x55d9cf192aa0 2024-09-17T02:51:08.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48d8263000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d9cf3f5bb0 con 0x55d9cf0ce890 2024-09-17T02:51:08.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48ca59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf4776d0 0x55d9cf5e7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48ca59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf4776d0 0x55d9cf5e7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47154/0 (socket says 172.21.15.131:47154) 2024-09-17T02:51:08.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48ca59c640 1 -- 172.21.15.131:0/2288296580 learned_addr learned my addr 172.21.15.131:0/2288296580 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48cad9d640 1 --2- 172.21.15.131:0/2288296580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf192aa0 0x55d9cf5e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48ca59c640 1 -- 172.21.15.131:0/2288296580 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf0ce890 msgr2=0x55d9cf5e8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48ca59c640 1 --2- 172.21.15.131:0/2288296580 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf0ce890 0x55d9cf5e8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48ca59c640 1 -- 172.21.15.131:0/2288296580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf192aa0 msgr2=0x55d9cf5e8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48ca59c640 1 --2- 172.21.15.131:0/2288296580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf192aa0 0x55d9cf5e8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48ca59c640 1 -- 172.21.15.131:0/2288296580 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9cf399c80 con 0x55d9cf4776d0 2024-09-17T02:51:08.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48cad9d640 1 --2- 172.21.15.131:0/2288296580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf192aa0 0x55d9cf5e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:08.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48ca59c640 1 --2- 172.21.15.131:0/2288296580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf4776d0 0x55d9cf5e7ff0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f48c001a4f0 tx=0x7f48c00b4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7282d9262361 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48abfff640 1 -- 172.21.15.131:0/2288296580 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c00c8070 con 0x55d9cf4776d0 2024-09-17T02:51:08.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48abfff640 1 -- 172.21.15.131:0/2288296580 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48c0004440 con 0x55d9cf4776d0 2024-09-17T02:51:08.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48abfff640 1 -- 172.21.15.131:0/2288296580 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c00b2040 con 0x55d9cf4776d0 2024-09-17T02:51:08.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48d8263000 1 -- 172.21.15.131:0/2288296580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf4776d0 msgr2=0x55d9cf5e7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48d8263000 1 --2- 172.21.15.131:0/2288296580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf4776d0 0x55d9cf5e7ff0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f48c001a4f0 tx=0x7f48c00b4000 comp rx=0 tx=0).stop 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.727+0000 7f48d8263000 1 -- 172.21.15.131:0/2288296580 shutdown_connections 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 --2- 172.21.15.131:0/2288296580 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf192aa0 0x55d9cf5e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 --2- 172.21.15.131:0/2288296580 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf0ce890 0x55d9cf5e8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 --2- 172.21.15.131:0/2288296580 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf4776d0 0x55d9cf5e7ff0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 -- 172.21.15.131:0/2288296580 >> 172.21.15.131:0/2288296580 conn(0x55d9cf0a25c0 msgr2=0x55d9cf5a0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 -- 172.21.15.131:0/2288296580 shutdown_connections 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 -- 172.21.15.131:0/2288296580 wait complete. 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 -- start start 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 Processor -- start 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf0ce890 0x55d9cf5a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf192aa0 0x55d9cf5aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf4776d0 0x55d9cf5aa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d9cf3e8df0 con 0x55d9cf0ce890 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d9cf3e4fe0 con 0x55d9cf4776d0 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d9cf38a4d0 con 0x55d9cf192aa0 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 10 client.0 _ll_register_callbacks cb 0x55d9cf3687f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48ca59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf0ce890 0x55d9cf5a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48ca59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf0ce890 0x55d9cf5a9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47166/0 (socket says 172.21.15.131:47166) 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48ca59c640 1 -- 172.21.15.131:0/2212376571 learned_addr learned my addr 172.21.15.131:0/2212376571 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf192aa0 0x55d9cf5aa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48cad9d640 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf4776d0 0x55d9cf5aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48ca59c640 1 -- 172.21.15.131:0/2212376571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf192aa0 msgr2=0x55d9cf5aa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48ca59c640 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf192aa0 0x55d9cf5aa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48ca59c640 1 -- 172.21.15.131:0/2212376571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf4776d0 msgr2=0x55d9cf5aa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48ca59c640 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf4776d0 0x55d9cf5aa730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48ca59c640 1 -- 172.21.15.131:0/2212376571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9cf383210 con 0x55d9cf0ce890 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48ca59c640 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf0ce890 0x55d9cf5a9cb0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f48c001a4f0 tx=0x7f48c00a5e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937ab4256ba9a629 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48a97fa640 1 -- 172.21.15.131:0/2212376571 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c00c8020 con 0x55d9cf0ce890 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 -- 172.21.15.131:0/2212376571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d9cf37d350 con 0x55d9cf0ce890 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48a97fa640 1 -- 172.21.15.131:0/2212376571 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f48c0005030 con 0x55d9cf0ce890 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48a97fa640 1 -- 172.21.15.131:0/2212376571 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f48c00d2050 con 0x55d9cf0ce890 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 10 client.8286 fetch_fsmap learned FSMap version 92 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 -- 172.21.15.131:0/2212376571 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55d9cf369fc0 con 0x55d9cf0ce890 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48a97fa640 1 -- 172.21.15.131:0/2212376571 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d9cf37d350 con 0x55d9cf0ce890 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48a97fa640 1 -- 172.21.15.131:0/2212376571 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f48c00d5020 con 0x55d9cf0ce890 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 10 client.8286 fetch_fsmap finished waiting for FSMap version 92 2024-09-17T02:51:08.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 10 client.8286 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 20 client.8286 populate_metadata read hostname 'smithi131' 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x55d9cf087930 0x55d9cf5f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 4 client.8286 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48d8263000 1 -- 172.21.15.131:0/2212376571 --> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9cf3f7b00 con 0x55d9cf087930 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.731+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x55d9cf087930 0x55d9cf5f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48a97fa640 10 client.8286 ms_handle_connect on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x55d9cf087930 0x55d9cf5f95b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=351932633e1400a server_cookie=be193ae4dc710561 in_seq=0 out_seq=0 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48a97fa640 1 -- 172.21.15.131:0/2212376571 <== mds.0 v2:172.21.15.131:6833/1272267157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9cf3f7b00 con 0x55d9cf087930 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48a97fa640 10 client.8286 handle_command_reply: tid=0 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 1 client.8286 shutdown 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 1 -- 172.21.15.131:0/2212376571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf0ce890 msgr2=0x55d9cf5a9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf0ce890 0x55d9cf5a9cb0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f48c001a4f0 tx=0x7f48c00a5e70 comp rx=0 tx=0).stop 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 1 -- 172.21.15.131:0/2212376571 shutdown_connections 2024-09-17T02:51:08.735 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x55d9cf087930 0x55d9cf5f95b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf4776d0 0x55d9cf5aa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf192aa0 0x55d9cf5aa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48a97fa640 0 client.8286 ms_handle_reset on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 1 --2- 172.21.15.131:0/2212376571 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf0ce890 0x55d9cf5a9cb0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 1 -- 172.21.15.131:0/2212376571 >> 172.21.15.131:0/2212376571 conn(0x55d9cf0a25c0 msgr2=0x55d9cf598360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 1 -- 172.21.15.131:0/2212376571 shutdown_connections 2024-09-17T02:51:08.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 1 -- 172.21.15.131:0/2212376571 wait complete. 2024-09-17T02:51:08.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.735+0000 7f48d8263000 20 client.8286 trim_cache size 0 max 16384 2024-09-17T02:51:08.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48d8263000 1 Processor -- start 2024-09-17T02:51:08.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48d8263000 1 -- start start 2024-09-17T02:51:08.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48d8263000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf0ce890 0x55d9cf5fc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48d8263000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf5fcb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48d8263000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf4776d0 0x55d9cf59e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48d8263000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d9cf5a7d80 con 0x55d9cf192aa0 2024-09-17T02:51:08.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48d8263000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc2014a0 con 0x55d9cf4776d0 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48d8263000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f48cc15b240 con 0x55d9cf0ce890 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48c9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf5fcb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48c9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf5fcb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47176/0 (socket says 172.21.15.131:47176) 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48c9d9b640 1 -- 172.21.15.131:0/2509814163 learned_addr learned my addr 172.21.15.131:0/2509814163 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48ca59c640 1 --2- 172.21.15.131:0/2509814163 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf0ce890 0x55d9cf5fc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48c9d9b640 1 -- 172.21.15.131:0/2509814163 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf0ce890 msgr2=0x55d9cf5fc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/2509814163 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf0ce890 0x55d9cf5fc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48cad9d640 1 --2- 172.21.15.131:0/2509814163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf4776d0 0x55d9cf59e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48c9d9b640 1 -- 172.21.15.131:0/2509814163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf4776d0 msgr2=0x55d9cf59e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.751+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/2509814163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf4776d0 0x55d9cf59e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48c9d9b640 1 -- 172.21.15.131:0/2509814163 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9cf442680 con 0x55d9cf192aa0 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/2509814163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf5fcb20 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f48bc05df30 tx=0x7f48bc01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8aac4bdc40ce389 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48ab7fe640 1 -- 172.21.15.131:0/2509814163 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc0679c0 con 0x55d9cf192aa0 2024-09-17T02:51:08.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48ab7fe640 1 -- 172.21.15.131:0/2509814163 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48bc091c30 con 0x55d9cf192aa0 2024-09-17T02:51:08.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48ab7fe640 1 -- 172.21.15.131:0/2509814163 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc05d200 con 0x55d9cf192aa0 2024-09-17T02:51:08.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 -- 172.21.15.131:0/2509814163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 msgr2=0x55d9cf5fcb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 --2- 172.21.15.131:0/2509814163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf5fcb20 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f48bc05df30 tx=0x7f48bc01e040 comp rx=0 tx=0).stop 2024-09-17T02:51:08.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 -- 172.21.15.131:0/2509814163 shutdown_connections 2024-09-17T02:51:08.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 --2- 172.21.15.131:0/2509814163 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf4776d0 0x55d9cf59e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 --2- 172.21.15.131:0/2509814163 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf5fcb20 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 --2- 172.21.15.131:0/2509814163 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf0ce890 0x55d9cf5fc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 -- 172.21.15.131:0/2509814163 >> 172.21.15.131:0/2509814163 conn(0x55d9cf0a25c0 msgr2=0x55d9cf59fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 -- 172.21.15.131:0/2509814163 shutdown_connections 2024-09-17T02:51:08.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 -- 172.21.15.131:0/2509814163 wait complete. 2024-09-17T02:51:08.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 -- start start 2024-09-17T02:51:08.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 Processor -- start 2024-09-17T02:51:08.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf0ce890 0x55d9cf65d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf65d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf4776d0 0x55d9cf65db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc15abf0 con 0x55d9cf192aa0 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc1f7150 con 0x55d9cf0ce890 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d9cf0aa680 con 0x55d9cf4776d0 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48d8263000 10 client.0 _ll_register_callbacks cb 0x7f48c00c6070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48c9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf65d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48c9d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf65d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47190/0 (socket says 172.21.15.131:47190) 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48c9d9b640 1 -- 172.21.15.131:0/3805474165 learned_addr learned my addr 172.21.15.131:0/3805474165 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48c9d9b640 1 -- 172.21.15.131:0/3805474165 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf4776d0 msgr2=0x55d9cf65db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48cad9d640 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf4776d0 0x55d9cf65db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48ca59c640 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf0ce890 0x55d9cf65d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf4776d0 0x55d9cf65db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48c9d9b640 1 -- 172.21.15.131:0/3805474165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf0ce890 msgr2=0x55d9cf65d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf0ce890 0x55d9cf65d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.755+0000 7f48c9d9b640 1 -- 172.21.15.131:0/3805474165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9cf3af0b0 con 0x55d9cf192aa0 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48c9d9b640 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf65d640 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f48bc05dca0 tx=0x7f48bc05a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d539a43421d8eef server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48a97fa640 1 -- 172.21.15.131:0/3805474165 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48bc05d410 con 0x55d9cf192aa0 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48a97fa640 1 -- 172.21.15.131:0/3805474165 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f48bc0598f0 con 0x55d9cf192aa0 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 -- 172.21.15.131:0/3805474165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48bc091c30 con 0x55d9cf192aa0 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48a97fa640 1 -- 172.21.15.131:0/3805474165 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f48bc06e050 con 0x55d9cf192aa0 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 10 client.8292 fetch_fsmap learned FSMap version 92 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 -- 172.21.15.131:0/3805474165 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x55d9cf37b610 con 0x55d9cf192aa0 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48a97fa640 1 -- 172.21.15.131:0/3805474165 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48bc091c30 con 0x55d9cf192aa0 2024-09-17T02:51:08.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48a97fa640 1 -- 172.21.15.131:0/3805474165 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f48bc059af0 con 0x55d9cf192aa0 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 10 client.8292 fetch_fsmap finished waiting for FSMap version 92 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 10 client.8292 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 20 client.8292 populate_metadata read hostname 'smithi131' 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x55d9cf075990 0x55d9cf5fa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 4 client.8292 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 -- 172.21.15.131:0/3805474165 --> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9cf3f35b0 con 0x55d9cf075990 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48ca59c640 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x55d9cf075990 0x55d9cf5fa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48a97fa640 10 client.8292 ms_handle_connect on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48ca59c640 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x55d9cf075990 0x55d9cf5fa510 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cccea708f35b1fd server_cookie=632a307440ff0b7e in_seq=0 out_seq=0 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48a97fa640 1 -- 172.21.15.131:0/3805474165 <== mds.0 v2:172.21.15.131:6833/1272267157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+571 (crc 0 0 0) 0x55d9cf3f35b0 con 0x55d9cf075990 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48a97fa640 10 client.8292 handle_command_reply: tid=0 2024-09-17T02:51:08.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 client.8292 shutdown 2024-09-17T02:51:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 -- 172.21.15.131:0/3805474165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 msgr2=0x55d9cf65d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf65d640 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f48bc05dca0 tx=0x7f48bc05a4a0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 -- 172.21.15.131:0/3805474165 shutdown_connections 2024-09-17T02:51:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x55d9cf075990 0x55d9cf5fa510 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d9cf4776d0 0x55d9cf65db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48a97fa640 0 client.8292 ms_handle_reset on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:08.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d9cf192aa0 0x55d9cf65d640 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 --2- 172.21.15.131:0/3805474165 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d9cf0ce890 0x55d9cf65d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 -- 172.21.15.131:0/3805474165 >> 172.21.15.131:0/3805474165 conn(0x55d9cf0a25c0 msgr2=0x55d9cf5a0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 -- 172.21.15.131:0/3805474165 shutdown_connections 2024-09-17T02:51:08.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 1 -- 172.21.15.131:0/3805474165 wait complete. 2024-09-17T02:51:08.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d8263000 20 client.8292 trim_cache size 0 max 16384 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": { 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: "058dac6f-2cd4-464c-816c-a4e52b1b7d46": { 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: "tag": "058dac6f-2cd4-464c-816c-a4e52b1b7d46", 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: "options": "recursive,repair" 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: "058dac6f-2cd4-464c-816c-a4e52b1b7d46": { 2024-09-17T02:51:08.764 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:51:08.555609+0000", 2024-09-17T02:51:08.765 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:51:08.765 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:51:08.765 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:51:08.765 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:51:08.765 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:51:08.765 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:08.765 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:08.765 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:51:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 -- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48ac073200 msgr2=0x7f48ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48ac073200 0x7f48ac075710 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f48c00af000 tx=0x7f48c001b100 comp rx=0 tx=0).stop 2024-09-17T02:51:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 -- 172.21.15.131:0/3160676288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc0ced00 msgr2=0x7f48cc154130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc0ced00 0x7f48cc154130 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f48bc00c3c0 tx=0x7f48bc00c880 comp rx=0 tx=0).stop 2024-09-17T02:51:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 -- 172.21.15.131:0/3160676288 shutdown_connections 2024-09-17T02:51:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48ac073200 0x7f48ac075710 secure :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f48c00af000 tx=0x7f48c001b100 comp rx=0 tx=0).stop 2024-09-17T02:51:08.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f48cc15c5b0 0x7f48cc160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48cc0ced00 0x7f48cc154130 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 --2- 172.21.15.131:0/3160676288 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f48cc06e3e0 0x7f48cc158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:08.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 -- 172.21.15.131:0/3160676288 >> 172.21.15.131:0/3160676288 conn(0x7f48cc06eea0 msgr2=0x7f48cc0b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:08.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 -- 172.21.15.131:0/3160676288 shutdown_connections 2024-09-17T02:51:08.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:08.759+0000 7f48d0f98640 1 -- 172.21.15.131:0/3160676288 wait complete. 2024-09-17T02:51:08.775 INFO:tasks.cephfs.filesystem:scrub status for tag:058dac6f-2cd4-464c-816c-a4e52b1b7d46 - {'path': '/', 'tag': '058dac6f-2cd4-464c-816c-a4e52b1b7d46', 'options': 'recursive,repair'} 2024-09-17T02:51:09.720 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-09-17T02:51:09.723+0000 7f4556f12640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:51:09.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:09.767+0000 7f38477fe640 1 -- 172.21.15.131:0/1977139069 <== mon.0 v2:172.21.15.131:3300/0 8 ==== log(8 entries from seq 1 at 2024-09-17T02:51:08.561159+0000) ==== 1407+0+0 (secure 0 0 0) 0x7f3838011df0 con 0x7f3830004810 2024-09-17T02:51:09.765 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:51:08.561159+0000 mds.a [INF] scrub queued for path: / 2024-09-17T02:51:09.765 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:51:08.561163+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-09-17T02:51:09.765 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:51:08.561169+0000 mds.a [INF] scrub summary: active paths [/] 2024-09-17T02:51:09.766 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:51:08.562496+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-09-17T02:51:09.766 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:51:08.562505+0000 mds.a [ERR] inode table repaired for inode: 0x10000000002 2024-09-17T02:51:09.766 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:51:08.562575+0000 mds.a [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-09-17T02:51:09.766 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:51:08.562581+0000 mds.a [ERR] inode table repaired for inode: 0x10000000001 2024-09-17T02:51:09.766 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:51:08.563536+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-09-17T02:51:14.443 INFO:teuthology.orchestra.run.smithi131.stdout:2024-09-17T02:51:12.778667+0000 mds.a [INF] scrub summary: idle 2024-09-17T02:51:14.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:14.447+0000 7f38477fe640 1 -- 172.21.15.131:0/1977139069 <== mon.0 v2:172.21.15.131:3300/0 9 ==== log(1 entries from seq 9 at 2024-09-17T02:51:12.778667+0000) ==== 189+0+0 (secure 0 0 0) 0x7f383805ba40 con 0x7f3830004810 2024-09-17T02:51:15.599 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:51:15.602 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:51:38.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:51:38.777 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-17T02:51:38.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.954+0000 7f44308e4640 1 -- 172.21.15.131:0/2864175625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f442c15c740 msgr2=0x7f442c0d0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:38.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.954+0000 7f44308e4640 1 --2- 172.21.15.131:0/2864175625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f442c15c740 0x7f442c0d0130 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f442c0cd850 tx=0x7f441c01c7d0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.954+0000 7f44308e4640 1 -- 172.21.15.131:0/2864175625 shutdown_connections 2024-09-17T02:51:38.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.954+0000 7f44308e4640 1 --2- 172.21.15.131:0/2864175625 >> v2:172.21.15.189:3300/0 conn(0x7f442c0cca90 0x7f442c0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.954+0000 7f44308e4640 1 --2- 172.21.15.131:0/2864175625 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f442c15c740 0x7f442c0d0130 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.954+0000 7f44308e4640 1 -- 172.21.15.131:0/2864175625 >> 172.21.15.131:0/2864175625 conn(0x7f442c0cf0b0 msgr2=0x7f442c0cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:38.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.954+0000 7f44308e4640 1 -- 172.21.15.131:0/2864175625 shutdown_connections 2024-09-17T02:51:38.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.954+0000 7f44308e4640 1 -- 172.21.15.131:0/2864175625 wait complete. 2024-09-17T02:51:38.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f44308e4640 1 Processor -- start 2024-09-17T02:51:38.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f44308e4640 1 -- start start 2024-09-17T02:51:38.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f44308e4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f442c0706c0 0x7f442c14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f44308e4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f442c0cca90 0x7f442c14fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f44308e4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f442c15c740 0x7f442c150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f44308e4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f442c16a060 con 0x7f442c15c740 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f44308e4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f442c169ee0 con 0x7f442c0cca90 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f44308e4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f442c16a1e0 con 0x7f442c0706c0 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f442ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f442c15c740 0x7f442c150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f442ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f442c15c740 0x7f442c150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43298/0 (socket says 172.21.15.131:43298) 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f442ad9d640 1 -- 172.21.15.131:0/1373588797 learned_addr learned my addr 172.21.15.131:0/1373588797 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f442a59c640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f442c0706c0 0x7f442c14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f4429d9b640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f442c0cca90 0x7f442c14fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f442ad9d640 1 -- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f442c0706c0 msgr2=0x7f442c14d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:38.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f442ad9d640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f442c0706c0 0x7f442c14d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f442ad9d640 1 -- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f442c0cca90 msgr2=0x7f442c14fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:38.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f442ad9d640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f442c0cca90 0x7f442c14fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f442ad9d640 1 -- 172.21.15.131:0/1373588797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442c15b870 con 0x7f442c15c740 2024-09-17T02:51:38.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f442ad9d640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f442c15c740 0x7f442c150300 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f442400c730 tx=0x7f442400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:38.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f441affd640 1 -- 172.21.15.131:0/1373588797 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424003b80 con 0x7f442c15c740 2024-09-17T02:51:38.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f441affd640 1 -- 172.21.15.131:0/1373588797 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4424003d40 con 0x7f442c15c740 2024-09-17T02:51:38.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f44308e4640 1 -- 172.21.15.131:0/1373588797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442c15bb50 con 0x7f442c15c740 2024-09-17T02:51:38.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f441affd640 1 -- 172.21.15.131:0/1373588797 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44240052d0 con 0x7f442c15c740 2024-09-17T02:51:38.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.958+0000 7f44308e4640 1 -- 172.21.15.131:0/1373588797 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442c14f5c0 con 0x7f442c15c740 2024-09-17T02:51:38.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f441affd640 1 -- 172.21.15.131:0/1373588797 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4424005680 con 0x7f442c15c740 2024-09-17T02:51:38.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f441affd640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f440c073230 0x7f440c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f441affd640 1 -- 172.21.15.131:0/1373588797 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f44240942a0 con 0x7f442c15c740 2024-09-17T02:51:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442a59c640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f440c073230 0x7f440c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f4437baf000 1 Processor -- start 2024-09-17T02:51:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f4437baf000 1 -- start start 2024-09-17T02:51:38.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442a59c640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f440c073230 0x7f440c075740 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f441c0af000 tx=0x7f441c01b010 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:38.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f4437baf000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda8179f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f4437baf000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda302890 0x555bda817f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f4437baf000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda3c6aa0 0x555bda818470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f4437baf000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555bda6b6cd0 con 0x555bda6ab6d0 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f4437baf000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555bda5cc330 con 0x555bda302890 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f4437baf000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555bda629bb0 con 0x555bda3c6aa0 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda8179f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda8179f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43304/0 (socket says 172.21.15.131:43304) 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442a59c640 1 -- 172.21.15.131:0/846461368 learned_addr learned my addr 172.21.15.131:0/846461368 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442ad9d640 1 --2- 172.21.15.131:0/846461368 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda3c6aa0 0x555bda818470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f4429d9b640 1 --2- 172.21.15.131:0/846461368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda302890 0x555bda817f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442a59c640 1 -- 172.21.15.131:0/846461368 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda3c6aa0 msgr2=0x555bda818470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442a59c640 1 --2- 172.21.15.131:0/846461368 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda3c6aa0 0x555bda818470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442a59c640 1 -- 172.21.15.131:0/846461368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda302890 msgr2=0x555bda817f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:38.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442a59c640 1 --2- 172.21.15.131:0/846461368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda302890 0x555bda817f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442a59c640 1 -- 172.21.15.131:0/846461368 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bda5cdc80 con 0x555bda6ab6d0 2024-09-17T02:51:38.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f442a59c640 1 --2- 172.21.15.131:0/846461368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda8179f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f441c01a400 tx=0x7f441c0a7050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f52d0324ea4243 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:38.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f440bfff640 1 -- 172.21.15.131:0/846461368 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c0c9070 con 0x555bda6ab6d0 2024-09-17T02:51:38.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f440bfff640 1 -- 172.21.15.131:0/846461368 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f441c005040 con 0x555bda6ab6d0 2024-09-17T02:51:38.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.962+0000 7f440bfff640 1 -- 172.21.15.131:0/846461368 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c0b2040 con 0x555bda6ab6d0 2024-09-17T02:51:38.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 -- 172.21.15.131:0/846461368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 msgr2=0x555bda8179f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:38.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 --2- 172.21.15.131:0/846461368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda8179f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f441c01a400 tx=0x7f441c0a7050 comp rx=0 tx=0).stop 2024-09-17T02:51:38.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 -- 172.21.15.131:0/846461368 shutdown_connections 2024-09-17T02:51:38.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 --2- 172.21.15.131:0/846461368 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda3c6aa0 0x555bda818470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 --2- 172.21.15.131:0/846461368 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda302890 0x555bda817f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 --2- 172.21.15.131:0/846461368 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda8179f0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 -- 172.21.15.131:0/846461368 >> 172.21.15.131:0/846461368 conn(0x555bda2d65c0 msgr2=0x555bda7d4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:38.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 -- 172.21.15.131:0/846461368 shutdown_connections 2024-09-17T02:51:38.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 -- 172.21.15.131:0/846461368 wait complete. 2024-09-17T02:51:38.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 -- start start 2024-09-17T02:51:38.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 Processor -- start 2024-09-17T02:51:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda7d39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda3c6aa0 0x555bda7d3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda6ab6d0 0x555bda7da5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555bda61cdf0 con 0x555bda302890 2024-09-17T02:51:38.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555bda618fe0 con 0x555bda3c6aa0 2024-09-17T02:51:38.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555bda5be4d0 con 0x555bda6ab6d0 2024-09-17T02:51:38.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4437baf000 10 client.0 _ll_register_callbacks cb 0x555bda59c7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:51:38.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f442a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda7d39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f442a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda7d39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43308/0 (socket says 172.21.15.131:43308) 2024-09-17T02:51:38.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f442a59c640 1 -- 172.21.15.131:0/391009413 learned_addr learned my addr 172.21.15.131:0/391009413 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:38.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f442ad9d640 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda6ab6d0 0x555bda7da5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4429d9b640 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda3c6aa0 0x555bda7d3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f442a59c640 1 -- 172.21.15.131:0/391009413 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda6ab6d0 msgr2=0x555bda7da5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:38.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f442a59c640 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda6ab6d0 0x555bda7da5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f442a59c640 1 -- 172.21.15.131:0/391009413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda3c6aa0 msgr2=0x555bda7d3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:38.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f442a59c640 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda3c6aa0 0x555bda7d3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f442a59c640 1 -- 172.21.15.131:0/391009413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bda5b7210 con 0x555bda302890 2024-09-17T02:51:38.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f4429d9b640 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda3c6aa0 0x555bda7d3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:38.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f442a59c640 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda7d39b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f441c01a400 tx=0x7f441c0198f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2b9abe08cca45b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:38.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.966+0000 7f44097fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:51:38.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f44097fa640 1 -- 172.21.15.131:0/391009413 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c0c9040 con 0x555bda302890 2024-09-17T02:51:38.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f44097fa640 1 -- 172.21.15.131:0/391009413 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f441c005380 con 0x555bda302890 2024-09-17T02:51:38.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 -- 172.21.15.131:0/391009413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555bda5b1350 con 0x555bda302890 2024-09-17T02:51:38.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f44097fa640 1 -- 172.21.15.131:0/391009413 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f441c0da050 con 0x555bda302890 2024-09-17T02:51:38.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 10 client.8307 fetch_fsmap learned FSMap version 92 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 -- 172.21.15.131:0/391009413 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x555bda59dfc0 con 0x555bda302890 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f44097fa640 1 -- 172.21.15.131:0/391009413 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x555bda5b1350 con 0x555bda302890 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f44097fa640 1 -- 172.21.15.131:0/391009413 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f441c0dd020 con 0x555bda302890 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 10 client.8307 fetch_fsmap finished waiting for FSMap version 92 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 10 client.8307 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 20 client.8307 populate_metadata read hostname 'smithi131' 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x555bda2bb930 0x555bda81ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 4 client.8307 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 -- 172.21.15.131:0/391009413 --> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bda62bb00 con 0x555bda2bb930 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4429d9b640 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x555bda2bb930 0x555bda81ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f44097fa640 10 client.8307 ms_handle_connect on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4429d9b640 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x555bda2bb930 0x555bda81ee80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=777ff1a483a4c9df server_cookie=86f62301771238ea in_seq=0 out_seq=0 2024-09-17T02:51:38.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f44097fa640 1 -- 172.21.15.131:0/391009413 <== mds.0 v2:172.21.15.131:6833/1272267157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555bda62bb00 con 0x555bda2bb930 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f44097fa640 10 client.8307 handle_command_reply: tid=0 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 client.8307 shutdown 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 -- 172.21.15.131:0/391009413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 msgr2=0x555bda7d39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda7d39b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f441c01a400 tx=0x7f441c0198f0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 -- 172.21.15.131:0/391009413 shutdown_connections 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x555bda2bb930 0x555bda81ee80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda6ab6d0 0x555bda7da5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda3c6aa0 0x555bda7d3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 --2- 172.21.15.131:0/391009413 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda7d39b0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f44097fa640 0 client.8307 ms_handle_reset on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 -- 172.21.15.131:0/391009413 >> 172.21.15.131:0/391009413 conn(0x555bda2d65c0 msgr2=0x555bda7ccd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 -- 172.21.15.131:0/391009413 shutdown_connections 2024-09-17T02:51:38.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 1 -- 172.21.15.131:0/391009413 wait complete. 2024-09-17T02:51:38.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.970+0000 7f4437baf000 20 client.8307 trim_cache size 0 max 16384 2024-09-17T02:51:38.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4437baf000 1 Processor -- start 2024-09-17T02:51:38.983 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4437baf000 1 -- start start 2024-09-17T02:51:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4437baf000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda302890 0x555bda889400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4437baf000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda3c6aa0 0x555bda889940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4437baf000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda889e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4437baf000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x555bda7d2b20 con 0x555bda6ab6d0 2024-09-17T02:51:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4437baf000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f442c2060f0 con 0x555bda302890 2024-09-17T02:51:38.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4437baf000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f442c2050d0 con 0x555bda3c6aa0 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f442ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda889e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f442ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda889e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43310/0 (socket says 172.21.15.131:43310) 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f442ad9d640 1 -- 172.21.15.131:0/4203750538 learned_addr learned my addr 172.21.15.131:0/4203750538 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f442a59c640 1 --2- 172.21.15.131:0/4203750538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda302890 0x555bda889400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4429d9b640 1 --2- 172.21.15.131:0/4203750538 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda3c6aa0 0x555bda889940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f442ad9d640 1 -- 172.21.15.131:0/4203750538 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda3c6aa0 msgr2=0x555bda889940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f442ad9d640 1 --2- 172.21.15.131:0/4203750538 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda3c6aa0 0x555bda889940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f442ad9d640 1 -- 172.21.15.131:0/4203750538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda302890 msgr2=0x555bda889400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f442ad9d640 1 --2- 172.21.15.131:0/4203750538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda302890 0x555bda889400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f442ad9d640 1 -- 172.21.15.131:0/4203750538 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bda676680 con 0x555bda6ab6d0 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4429d9b640 1 --2- 172.21.15.131:0/4203750538 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda3c6aa0 0x555bda889940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f442ad9d640 1 --2- 172.21.15.131:0/4203750538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda889e80 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f442405ea20 tx=0x7f4424094040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e094bbd107354bf server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f440b7fe640 1 -- 172.21.15.131:0/4203750538 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4424084070 con 0x555bda6ab6d0 2024-09-17T02:51:38.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f440b7fe640 1 -- 172.21.15.131:0/4203750538 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f442405e5a0 con 0x555bda6ab6d0 2024-09-17T02:51:38.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f440b7fe640 1 -- 172.21.15.131:0/4203750538 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442405f190 con 0x555bda6ab6d0 2024-09-17T02:51:38.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4437baf000 1 -- 172.21.15.131:0/4203750538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 msgr2=0x555bda889e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:38.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.990+0000 7f4437baf000 1 --2- 172.21.15.131:0/4203750538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda889e80 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f442405ea20 tx=0x7f4424094040 comp rx=0 tx=0).stop 2024-09-17T02:51:38.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 -- 172.21.15.131:0/4203750538 shutdown_connections 2024-09-17T02:51:38.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 --2- 172.21.15.131:0/4203750538 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda6ab6d0 0x555bda889e80 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 --2- 172.21.15.131:0/4203750538 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda3c6aa0 0x555bda889940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 --2- 172.21.15.131:0/4203750538 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda302890 0x555bda889400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 -- 172.21.15.131:0/4203750538 >> 172.21.15.131:0/4203750538 conn(0x555bda2d65c0 msgr2=0x555bda829fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:38.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 -- 172.21.15.131:0/4203750538 shutdown_connections 2024-09-17T02:51:38.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 -- 172.21.15.131:0/4203750538 wait complete. 2024-09-17T02:51:38.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 -- start start 2024-09-17T02:51:38.992 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 Processor -- start 2024-09-17T02:51:38.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda88d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda3c6aa0 0x555bda88db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda6ab6d0 0x555bda88e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f442c204ab0 con 0x555bda302890 2024-09-17T02:51:38.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f442c2044e0 con 0x555bda3c6aa0 2024-09-17T02:51:38.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555bda2de680 con 0x555bda6ab6d0 2024-09-17T02:51:38.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 10 client.0 _ll_register_callbacks cb 0x7f441c0c7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:51:38.993 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f442a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda88d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f442a59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda88d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43314/0 (socket says 172.21.15.131:43314) 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f442a59c640 1 -- 172.21.15.131:0/3443166010 learned_addr learned my addr 172.21.15.131:0/3443166010 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f442ad9d640 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda6ab6d0 0x555bda88e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f442a59c640 1 -- 172.21.15.131:0/3443166010 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda6ab6d0 msgr2=0x555bda88e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4429d9b640 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda3c6aa0 0x555bda88db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f442a59c640 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda6ab6d0 0x555bda88e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f442a59c640 1 -- 172.21.15.131:0/3443166010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda3c6aa0 msgr2=0x555bda88db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f442a59c640 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda3c6aa0 0x555bda88db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f442a59c640 1 -- 172.21.15.131:0/3443166010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bda5e30b0 con 0x555bda302890 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f442ad9d640 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda6ab6d0 0x555bda88e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f442a59c640 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda88d650 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f441c0d54c0 tx=0x7f441c0045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a7cb655398335b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f44097fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f44097fa640 1 -- 172.21.15.131:0/3443166010 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c0c9040 con 0x555bda302890 2024-09-17T02:51:38.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f44097fa640 1 -- 172.21.15.131:0/3443166010 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f441c0ced40 con 0x555bda302890 2024-09-17T02:51:38.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 -- 172.21.15.131:0/3443166010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f442405e5a0 con 0x555bda302890 2024-09-17T02:51:38.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f44097fa640 1 -- 172.21.15.131:0/3443166010 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f441c0d5740 con 0x555bda302890 2024-09-17T02:51:38.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 10 client.8313 fetch_fsmap learned FSMap version 92 2024-09-17T02:51:38.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 -- 172.21.15.131:0/3443166010 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=2+}) -- 0x555bda5af610 con 0x555bda302890 2024-09-17T02:51:38.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f44097fa640 1 -- 172.21.15.131:0/3443166010 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f442405e5a0 con 0x555bda302890 2024-09-17T02:51:38.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f44097fa640 1 -- 172.21.15.131:0/3443166010 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 92) ==== 3502+0+0 (secure 0 0 0) 0x7f441c0dd050 con 0x555bda302890 2024-09-17T02:51:38.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 10 client.8313 fetch_fsmap finished waiting for FSMap version 92 2024-09-17T02:51:38.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 10 client.8313 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-17T02:51:38.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 20 client.8313 populate_metadata read hostname 'smithi131' 2024-09-17T02:51:38.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x555bda2a9990 0x555bda7ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 4 client.8313 mds_command: new command op to 8175 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.994+0000 7f4437baf000 1 -- 172.21.15.131:0/3443166010 --> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555bda6275b0 con 0x555bda2a9990 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4429d9b640 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x555bda2a9990 0x555bda7ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44097fa640 10 client.8313 ms_handle_connect on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4429d9b640 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x555bda2a9990 0x555bda7ce8f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1587a7baa40dd3b server_cookie=4be289d8dc07382a in_seq=0 out_seq=0 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44097fa640 1 -- 172.21.15.131:0/3443166010 <== mds.0 v2:172.21.15.131:6833/1272267157 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x555bda6275b0 con 0x555bda2a9990 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44097fa640 10 client.8313 handle_command_reply: tid=0 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 1 client.8313 shutdown 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 1 -- 172.21.15.131:0/3443166010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 msgr2=0x555bda88d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda88d650 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f441c0d54c0 tx=0x7f441c0045d0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 1 -- 172.21.15.131:0/3443166010 shutdown_connections 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.131:6833/1272267157,v1:172.21.15.131:6835/1272267157] conn(0x555bda2a9990 0x555bda7ce8f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555bda6ab6d0 0x555bda88e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555bda3c6aa0 0x555bda88db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 1 --2- 172.21.15.131:0/3443166010 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555bda302890 0x555bda88d650 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44097fa640 0 client.8313 ms_handle_reset on v2:172.21.15.131:6833/1272267157 2024-09-17T02:51:38.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 1 -- 172.21.15.131:0/3443166010 >> 172.21.15.131:0/3443166010 conn(0x555bda2d65c0 msgr2=0x555bda7d95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:38.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 1 -- 172.21.15.131:0/3443166010 shutdown_connections 2024-09-17T02:51:38.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 1 -- 172.21.15.131:0/3443166010 wait complete. 2024-09-17T02:51:38.997 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f4437baf000 20 client.8313 trim_cache size 0 max 16384 2024-09-17T02:51:38.997 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:51:38.997 INFO:teuthology.orchestra.run.smithi131.stdout: "status": "no active scrubs running", 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: "scrubs": {}, 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: "scrub_stats": { 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: "058dac6f-2cd4-464c-816c-a4e52b1b7d46": { 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: "start_time": "2024-09-17T02:51:08.555609+0000", 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: "path": "/", 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_started": 0, 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_passed": 0, 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_failed": 0, 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: "uninline_skipped": 0 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:38.998 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:51:38.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 -- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f440c073230 msgr2=0x7f440c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:38.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f440c073230 0x7f440c075740 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f441c0af000 tx=0x7f441c01b010 comp rx=0 tx=0).stop 2024-09-17T02:51:38.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 -- 172.21.15.131:0/1373588797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f442c15c740 msgr2=0x7f442c150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:38.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f442c15c740 0x7f442c150300 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f442400c730 tx=0x7f442400cbf0 comp rx=0 tx=0).stop 2024-09-17T02:51:38.999 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 -- 172.21.15.131:0/1373588797 shutdown_connections 2024-09-17T02:51:39.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f440c073230 0x7f440c075740 secure :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f441c0af000 tx=0x7f441c01b010 comp rx=0 tx=0).stop 2024-09-17T02:51:39.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f442c15c740 0x7f442c150300 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f442c0cca90 0x7f442c14fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 --2- 172.21.15.131:0/1373588797 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f442c0706c0 0x7f442c14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 -- 172.21.15.131:0/1373588797 >> 172.21.15.131:0/1373588797 conn(0x7f442c0cf0b0 msgr2=0x7f442c155700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:39.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 -- 172.21.15.131:0/1373588797 shutdown_connections 2024-09-17T02:51:39.000 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:38.998+0000 7f44308e4640 1 -- 172.21.15.131:0/1373588797 wait complete. 2024-09-17T02:51:39.012 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-17T02:51:39.114 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:51:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.278+0000 7f708b2ad640 1 -- 172.21.15.131:0/1726653029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7070004b50 msgr2=0x7f7070004fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 --2- 172.21.15.131:0/1726653029 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7070004b50 0x7f7070004fc0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f707c0016f0 tx=0x7f707c01c7b0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708984a640 1 -- 172.21.15.131:0/1726653029 reap_dead start 2024-09-17T02:51:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 -- 172.21.15.131:0/1726653029 shutdown_connections 2024-09-17T02:51:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 -- 172.21.15.131:0/1726653029 >> 172.21.15.131:0/1726653029 conn(0x7f708406eea0 msgr2=0x7f708406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:39.274 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 -- 172.21.15.131:0/1726653029 shutdown_connections 2024-09-17T02:51:39.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 -- 172.21.15.131:0/1726653029 wait complete. 2024-09-17T02:51:39.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 Processor -- start 2024-09-17T02:51:39.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 -- start start 2024-09-17T02:51:39.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7070004b50 0x7f70841483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70841524f0 0x7f70841fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f70841fc310 0x7f70841fc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7084166690 con 0x7f7070004b50 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7084166510 con 0x7f70841524f0 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708b2ad640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7084166810 con 0x7f70841fc310 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f7089049640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7070004b50 0x7f70841483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f7089049640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7070004b50 0x7f70841483a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43328/0 (socket says 172.21.15.131:43328) 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f7089049640 1 -- 172.21.15.131:0/2163241445 learned_addr learned my addr 172.21.15.131:0/2163241445 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f7088848640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70841524f0 0x7f70841fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f708984a640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f70841fc310 0x7f70841fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f7089049640 1 -- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f70841fc310 msgr2=0x7f70841fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f7089049640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f70841fc310 0x7f70841fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f7089049640 1 -- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70841524f0 msgr2=0x7f70841fbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f7089049640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70841524f0 0x7f70841fbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.282+0000 7f7089049640 1 -- 172.21.15.131:0/2163241445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70840ca3f0 con 0x7f7070004b50 2024-09-17T02:51:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f7089049640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7070004b50 0x7f70841483a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c10 tx=0x7f707c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f7079ffb640 1 -- 172.21.15.131:0/2163241445 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c0bb070 con 0x7f7070004b50 2024-09-17T02:51:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f7079ffb640 1 -- 172.21.15.131:0/2163241445 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f707c006500 con 0x7f7070004b50 2024-09-17T02:51:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f708b2ad640 1 -- 172.21.15.131:0/2163241445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70840ca110 con 0x7f7070004b50 2024-09-17T02:51:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f7079ffb640 1 -- 172.21.15.131:0/2163241445 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c0ae070 con 0x7f7070004b50 2024-09-17T02:51:39.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f708b2ad640 1 -- 172.21.15.131:0/2163241445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70840c3ed0 con 0x7f7070004b50 2024-09-17T02:51:39.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f708b2ad640 1 -- 172.21.15.131:0/2163241445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7084073430 con 0x7f7070004b50 2024-09-17T02:51:39.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f7079ffb640 1 -- 172.21.15.131:0/2163241445 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f707c0a8480 con 0x7f7070004b50 2024-09-17T02:51:39.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f7079ffb640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7068073230 0x7f7068075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f7079ffb640 1 -- 172.21.15.131:0/2163241445 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 6303+0+0 (secure 0 0 0) 0x7f707c12ddc0 con 0x7f7070004b50 2024-09-17T02:51:39.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f7088848640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7068073230 0x7f7068075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.286+0000 7f7088848640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7068073230 0x7f7068075740 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f70740018d0 tx=0x7f7074000ee0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:39.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.290+0000 7f7079ffb640 1 -- 172.21.15.131:0/2163241445 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f707c016510 con 0x7f7070004b50 2024-09-17T02:51:39.481 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.486+0000 7f708b2ad640 1 -- 172.21.15.131:0/2163241445 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f70840cee30 con 0x7f7070004b50 2024-09-17T02:51:39.482 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:51:39.490+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:51:39.484 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.490+0000 7f7079ffb640 1 -- 172.21.15.131:0/2163241445 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7f707c0f84c0 con 0x7f7070004b50 2024-09-17T02:51:39.485 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:51:39.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.494+0000 7f708b2ad640 1 -- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7068073230 msgr2=0x7f7068075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.494+0000 7f708b2ad640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7068073230 0x7f7068075740 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f70740018d0 tx=0x7f7074000ee0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.494+0000 7f708b2ad640 1 -- 172.21.15.131:0/2163241445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7070004b50 msgr2=0x7f70841483a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.494+0000 7f708b2ad640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7070004b50 0x7f70841483a0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f707c000c10 tx=0x7f707c003c10 comp rx=0 tx=0).stop 2024-09-17T02:51:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.494+0000 7f708b2ad640 1 -- 172.21.15.131:0/2163241445 shutdown_connections 2024-09-17T02:51:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.494+0000 7f708b2ad640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7068073230 0x7f7068075740 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.494+0000 7f708b2ad640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f70841fc310 0x7f70841fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.494+0000 7f708b2ad640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f70841524f0 0x7f70841fbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.494+0000 7f708b2ad640 1 --2- 172.21.15.131:0/2163241445 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7070004b50 0x7f70841483a0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.494+0000 7f708b2ad640 1 -- 172.21.15.131:0/2163241445 >> 172.21.15.131:0/2163241445 conn(0x7f708406eea0 msgr2=0x7f7084154540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.498+0000 7f708b2ad640 1 -- 172.21.15.131:0/2163241445 shutdown_connections 2024-09-17T02:51:39.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.498+0000 7f708b2ad640 1 -- 172.21.15.131:0/2163241445 wait complete. 2024-09-17T02:51:39.491 INFO:tasks.ceph.mds.a.smithi131.stderr: -1216> 2024-09-17T02:51:08.555+0000 7f4552f0a640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-09-17T02:51:39.491 INFO:tasks.ceph.mds.a.smithi131.stderr: -1213> 2024-09-17T02:51:08.555+0000 7f4552f0a640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-09-17T02:51:39.491 INFO:tasks.ceph.mds.a.smithi131.stderr: -1195> 2024-09-17T02:51:08.555+0000 7f4552f0a640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-09-17T02:51:39.491 INFO:tasks.ceph.mds.a.smithi131.stderr: -1192> 2024-09-17T02:51:08.555+0000 7f4552f0a640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-09-17T02:51:39.491 INFO:tasks.ceph.mds.a.smithi131.stderr: -1009> 2024-09-17T02:51:09.723+0000 7f4556f12640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:51:39.511 DEBUG:teuthology.orchestra.run.smithi131:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-09-17T02:51:39.565 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:51:39.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.574+0000 7f8807893400 1 -- start start 2024-09-17T02:51:39.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.574+0000 7f8807893400 1 Processor -- start 2024-09-17T02:51:39.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563954df9a80 0x563955b4a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563954df9180 0x563955b4a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9c000 0x563955b4ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563955b66600 con 0x563954df9180 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563955b66480 con 0x563955c9c000 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x563955b66300 con 0x563954df9a80 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8805851640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563954df9180 0x563955b4a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8805851640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563954df9180 0x563955b4a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43366/0 (socket says 172.21.15.131:43366) 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8805851640 1 -- 172.21.15.131:0/942135084 learned_addr learned my addr 172.21.15.131:0/942135084 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8806052640 1 --2- 172.21.15.131:0/942135084 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563954df9a80 0x563955b4a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8806853640 1 --2- 172.21.15.131:0/942135084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9c000 0x563955b4ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8805851640 1 -- 172.21.15.131:0/942135084 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563954df9a80 msgr2=0x563955b4a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8805851640 1 --2- 172.21.15.131:0/942135084 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563954df9a80 0x563955b4a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8805851640 1 -- 172.21.15.131:0/942135084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9c000 msgr2=0x563955b4ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8805851640 1 --2- 172.21.15.131:0/942135084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9c000 0x563955b4ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8805851640 1 -- 172.21.15.131:0/942135084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563955b48960 con 0x563954df9180 2024-09-17T02:51:39.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8805851640 1 --2- 172.21.15.131:0/942135084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563954df9180 0x563955b4a580 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x563955be5920 tx=0x563954e9cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5720f79f0e3f50ee server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:39.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f880384d640 1 -- 172.21.15.131:0/942135084 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563954dbb520 con 0x563954df9180 2024-09-17T02:51:39.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f880384d640 1 -- 172.21.15.131:0/942135084 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563954dd8540 con 0x563954df9180 2024-09-17T02:51:39.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f880384d640 1 -- 172.21.15.131:0/942135084 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563954dbb1e0 con 0x563954df9180 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 1 -- 172.21.15.131:0/942135084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563955b48d20 con 0x563954df9180 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 1 -- 172.21.15.131:0/942135084 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x563955b48f00 con 0x563954df9180 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 4 waiting for MDS map... 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f880384d640 1 -- 172.21.15.131:0/942135084 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(111..111 src has 1..111) ==== 6389+0+0 (secure 0 0 0) 0x563955b53000 con 0x563954df9180 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f880384d640 1 -- 172.21.15.131:0/942135084 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 93) ==== 2893+0+0 (secure 0 0 0) 0x563954ddb180 con 0x563954df9180 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 4 Got MDS map 93 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 10 main: main 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 4 main: connecting to RADOS... 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 1 Processor -- start 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.578+0000 7f8807893400 1 -- start start 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563955c9c900 0x563955b4b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9cd80 0x563955b4b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9d200 0x563955b4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563955b66180 con 0x563955c9cd80 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563954de5380 con 0x563955c9d200 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x563954de5800 con 0x563955c9c900 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8806052640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9cd80 0x563955b4b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8806052640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9cd80 0x563955b4b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43368/0 (socket says 172.21.15.131:43368) 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8806052640 1 -- 172.21.15.131:0/827653409 learned_addr learned my addr 172.21.15.131:0/827653409 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8806052640 1 -- 172.21.15.131:0/827653409 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563955c9c900 msgr2=0x563955b4b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8806052640 1 --2- 172.21.15.131:0/827653409 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563955c9c900 0x563955b4b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8806052640 1 -- 172.21.15.131:0/827653409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9d200 msgr2=0x563955b4bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8805851640 1 --2- 172.21.15.131:0/827653409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9d200 0x563955b4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8806052640 1 --2- 172.21.15.131:0/827653409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9d200 0x563955b4bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8806052640 1 -- 172.21.15.131:0/827653409 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563955b48d20 con 0x563955c9cd80 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8806052640 1 --2- 172.21.15.131:0/827653409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9cd80 0x563955b4b600 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x563955be4f60 tx=0x563955b5c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59db27079e53c315 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8801849640 1 -- 172.21.15.131:0/827653409 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563954dbb380 con 0x563955c9cd80 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8801849640 1 -- 172.21.15.131:0/827653409 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563954dd8e00 con 0x563955c9cd80 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8801849640 1 -- 172.21.15.131:0/827653409 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563954dbb520 con 0x563955c9cd80 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- 172.21.15.131:0/827653409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9cd80 msgr2=0x563955b4b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 --2- 172.21.15.131:0/827653409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9cd80 0x563955b4b600 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x563955be4f60 tx=0x563955b5c360 comp rx=0 tx=0).stop 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- 172.21.15.131:0/827653409 shutdown_connections 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 --2- 172.21.15.131:0/827653409 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9d200 0x563955b4bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 --2- 172.21.15.131:0/827653409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9cd80 0x563955b4b600 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 --2- 172.21.15.131:0/827653409 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563955c9c900 0x563955b4b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- 172.21.15.131:0/827653409 >> 172.21.15.131:0/827653409 conn(0x563955c9c480 msgr2=0x563950d7e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- 172.21.15.131:0/827653409 shutdown_connections 2024-09-17T02:51:39.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- 172.21.15.131:0/827653409 wait complete. 2024-09-17T02:51:39.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 Processor -- start 2024-09-17T02:51:39.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- start start 2024-09-17T02:51:39.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563955c9c900 0x563955b4b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9cd80 0x563955b4b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9d200 0x563955b4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x563955b67200 con 0x563955c9d200 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x563955b66180 con 0x563955c9cd80 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.582+0000 7f8807893400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x563955b67380 con 0x563955c9c900 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8805851640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9d200 0x563955b4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8805851640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9d200 0x563955b4bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43378/0 (socket says 172.21.15.131:43378) 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8805851640 1 -- 172.21.15.131:0/3401309178 learned_addr learned my addr 172.21.15.131:0/3401309178 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8805851640 1 -- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563955c9c900 msgr2=0x563955b4b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8806052640 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9cd80 0x563955b4b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8805851640 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563955c9c900 0x563955b4b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8805851640 1 -- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9cd80 msgr2=0x563955b4b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8805851640 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9cd80 0x563955b4b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8805851640 1 -- 172.21.15.131:0/3401309178 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563955b49a40 con 0x563955c9d200 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8805851640 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9d200 0x563955b4bb80 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x563955cabb30 tx=0x563954e9cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8800046640 1 -- 172.21.15.131:0/3401309178 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563954dbb040 con 0x563955c9d200 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8800046640 1 -- 172.21.15.131:0/3401309178 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x563954dd8e00 con 0x563955c9d200 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8807893400 1 -- 172.21.15.131:0/3401309178 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563955b49c20 con 0x563955c9d200 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8800046640 1 -- 172.21.15.131:0/3401309178 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x563954dbb380 con 0x563955c9d200 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8807893400 1 -- 172.21.15.131:0/3401309178 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563955b481e0 con 0x563955c9d200 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8807893400 4 main: resolving pool 15 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8800046640 1 -- 172.21.15.131:0/3401309178 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x5639550c4c00 con 0x563955c9d200 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8800046640 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x563955c9f600 0x563955b4c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8800046640 1 -- 172.21.15.131:0/3401309178 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6389+0+0 (secure 0 0 0) 0x5639550c6200 con 0x563955c9d200 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8807893400 4 main: creating IoCtx.. 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8807893400 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x563955c9d680 0x563955b4c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8807893400 1 -- 172.21.15.131:0/3401309178 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) -- 0x5639551d0000 con 0x563955c9d680 2024-09-17T02:51:39.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8806853640 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x563955c9f600 0x563955b4c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8806052640 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x563955c9d680 0x563955b4c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8806052640 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x563955c9d680 0x563955b4c680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:39.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.586+0000 7f8806853640 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x563955c9f600 0x563955b4c100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x563955be57d0 tx=0x5639551c8480 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:39.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8806052640 1 -- 172.21.15.131:0/3401309178 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x563954f00000 con 0x563955c9d680 2024-09-17T02:51:39.584 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: "0": { 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: "data": { 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: "version": 5, 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: "inotable": { 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: "projected_free": [ 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: "start": 1099511627779, 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: "len": 1099511627773 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: ], 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: "free": [ 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:39.585 INFO:teuthology.orchestra.run.smithi131.stdout: "start": 1099511627779, 2024-09-17T02:51:39.586 INFO:teuthology.orchestra.run.smithi131.stdout: "len": 1099511627773 2024-09-17T02:51:39.586 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:39.586 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-09-17T02:51:39.586 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:39.586 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:39.586 INFO:teuthology.orchestra.run.smithi131.stdout: "result": 0 2024-09-17T02:51:39.586 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:39.586 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:51:39.586 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:39.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x563955c9d680 msgr2=0x563955b4c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x563955c9d680 0x563955b4c680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x563955c9f600 msgr2=0x563955b4c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x563955c9f600 0x563955b4c100 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x563955be57d0 tx=0x5639551c8480 comp rx=0 tx=0).stop 2024-09-17T02:51:39.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/3401309178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9d200 msgr2=0x563955b4bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 --2- 172.21.15.131:0/3401309178 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563955c9d200 0x563955b4bb80 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x563955cabb30 tx=0x563954e9cf00 comp rx=0 tx=0).stop 2024-09-17T02:51:39.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8806052640 1 -- 172.21.15.131:0/3401309178 reap_dead start 2024-09-17T02:51:39.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/3401309178 shutdown_connections 2024-09-17T02:51:39.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/3401309178 >> 172.21.15.131:0/3401309178 conn(0x563955c9c480 msgr2=0x563950d7e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:39.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/3401309178 shutdown_connections 2024-09-17T02:51:39.588 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/3401309178 wait complete. 2024-09-17T02:51:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/942135084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563954df9180 msgr2=0x563955b4a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 --2- 172.21.15.131:0/942135084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563954df9180 0x563955b4a580 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x563955be5920 tx=0x563954e9cf60 comp rx=0 tx=0).stop 2024-09-17T02:51:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/942135084 shutdown_connections 2024-09-17T02:51:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 --2- 172.21.15.131:0/942135084 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x563955c9c000 0x563955b4ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 --2- 172.21.15.131:0/942135084 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x563954df9180 0x563955b4a580 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 --2- 172.21.15.131:0/942135084 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x563954df9a80 0x563955b4a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/942135084 >> 172.21.15.131:0/942135084 conn(0x563954df9600 msgr2=0x563950d7f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/942135084 shutdown_connections 2024-09-17T02:51:39.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.590+0000 7f8807893400 1 -- 172.21.15.131:0/942135084 wait complete. 2024-09-17T02:51:39.590 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.079088, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-09-17T02:51:39.590 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:51:39.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:51:39.643 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:51:39.726 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:51:39.726 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:51:39.726 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:51:39.726 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:51:39.731 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:51:39.731 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:51:39.732 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:51:39.732 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:51:39.737 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:51:39.738 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:51:39.738 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:51:39.738 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:51:39.778 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:51:39.778 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:51:39.778 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:51:39.778 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:51:39.785 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:51:39.785 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:51:39.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.986+0000 7f80ae815640 1 -- 172.21.15.131:0/3164379382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80a806e3e0 msgr2=0x7f80a815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.986+0000 7f80ae815640 1 --2- 172.21.15.131:0/3164379382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80a806e3e0 0x7f80a815bea0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f809c009900 tx=0x7f809c01c700 comp rx=0 tx=0).stop 2024-09-17T02:51:39.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.986+0000 7f80ae815640 1 -- 172.21.15.131:0/3164379382 shutdown_connections 2024-09-17T02:51:39.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.986+0000 7f80ae815640 1 --2- 172.21.15.131:0/3164379382 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80a806e3e0 0x7f80a815bea0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.986+0000 7f80ae815640 1 --2- 172.21.15.131:0/3164379382 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80a80ced00 0x7f80a806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.986+0000 7f80ae815640 1 -- 172.21.15.131:0/3164379382 >> 172.21.15.131:0/3164379382 conn(0x7f80a806eea0 msgr2=0x7f80a806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:39.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.986+0000 7f80ae815640 1 -- 172.21.15.131:0/3164379382 shutdown_connections 2024-09-17T02:51:39.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.986+0000 7f80ae815640 1 -- 172.21.15.131:0/3164379382 wait complete. 2024-09-17T02:51:39.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80ae815640 1 Processor -- start 2024-09-17T02:51:39.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80ae815640 1 -- start start 2024-09-17T02:51:39.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80ae815640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80a806e3e0 0x7f80a80aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80ae815640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80a80ced00 0x7f80a80ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80ae815640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80a815c3e0 0x7f80a815d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80ae815640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f80a8168b60 con 0x7f80a815c3e0 2024-09-17T02:51:39.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80ae815640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80a81689e0 con 0x7f80a80ced00 2024-09-17T02:51:39.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80ae815640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f80a8168ce0 con 0x7f80a806e3e0 2024-09-17T02:51:39.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80acdb2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80a815c3e0 0x7f80a815d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80acdb2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80a815c3e0 0x7f80a815d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43400/0 (socket says 172.21.15.131:43400) 2024-09-17T02:51:39.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80acdb2640 1 -- 172.21.15.131:0/2993044539 learned_addr learned my addr 172.21.15.131:0/2993044539 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80a77fe640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80a80ced00 0x7f80a80ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80a7fff640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80a806e3e0 0x7f80a80aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80acdb2640 1 -- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80a806e3e0 msgr2=0x7f80a80aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80acdb2640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80a806e3e0 0x7f80a80aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80acdb2640 1 -- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80a80ced00 msgr2=0x7f80a80ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80acdb2640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80a80ced00 0x7f80a80ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80acdb2640 1 -- 172.21.15.131:0/2993044539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a80a9710 con 0x7f80a815c3e0 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80a77fe640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80a80ced00 0x7f80a80ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80acdb2640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80a815c3e0 0x7f80a815d160 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f80a000b970 tx=0x7f80a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80a4ff9640 1 -- 172.21.15.131:0/2993044539 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a0019070 con 0x7f80a815c3e0 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80a4ff9640 1 -- 172.21.15.131:0/2993044539 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80a000cc30 con 0x7f80a815c3e0 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80ae815640 1 -- 172.21.15.131:0/2993044539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a80a99f0 con 0x7f80a815c3e0 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80a4ff9640 1 -- 172.21.15.131:0/2993044539 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a00146a0 con 0x7f80a815c3e0 2024-09-17T02:51:39.986 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80ae815640 1 -- 172.21.15.131:0/2993044539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a80a9fb0 con 0x7f80a815c3e0 2024-09-17T02:51:39.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.990+0000 7f80ae815640 1 -- 172.21.15.131:0/2993044539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a815bea0 con 0x7f80a815c3e0 2024-09-17T02:51:39.987 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.994+0000 7f80a4ff9640 1 -- 172.21.15.131:0/2993044539 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f80a001c070 con 0x7f80a815c3e0 2024-09-17T02:51:39.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.994+0000 7f80a4ff9640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8094073200 0x7f8094075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:39.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.994+0000 7f80a4ff9640 1 -- 172.21.15.131:0/2993044539 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6389+0+0 (secure 0 0 0) 0x7f80a00941f0 con 0x7f80a815c3e0 2024-09-17T02:51:39.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.994+0000 7f80a7fff640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8094073200 0x7f8094075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:39.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.994+0000 7f80a7fff640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8094073200 0x7f8094075710 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f80980058d0 tx=0x7f8098005d90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:39.991 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:39.998+0000 7f80a4ff9640 1 -- 172.21.15.131:0/2993044539 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f80a005eff0 con 0x7f80a815c3e0 2024-09-17T02:51:40.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.194+0000 7f80ae815640 1 -- 172.21.15.131:0/2993044539 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80a815cc40 con 0x7f80a815c3e0 2024-09-17T02:51:40.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80a4ff9640 1 -- 172.21.15.131:0/2993044539 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+4396 (secure 0 0 0) 0x7f80a005e8f0 con 0x7f80a815c3e0 2024-09-17T02:51:40.190 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:40.191 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":93,"btime":"2024-09-17T02:51:39:491576+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":93,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:51:39.491567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-17T02:51:40.191 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 93 2024-09-17T02:51:40.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 -- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8094073200 msgr2=0x7f8094075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8094073200 0x7f8094075710 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f80980058d0 tx=0x7f8098005d90 comp rx=0 tx=0).stop 2024-09-17T02:51:40.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 -- 172.21.15.131:0/2993044539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80a815c3e0 msgr2=0x7f80a815d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80a815c3e0 0x7f80a815d160 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f80a000b970 tx=0x7f80a000be30 comp rx=0 tx=0).stop 2024-09-17T02:51:40.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 -- 172.21.15.131:0/2993044539 shutdown_connections 2024-09-17T02:51:40.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8094073200 0x7f8094075710 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f80a815c3e0 0x7f80a815d160 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f80a80ced00 0x7f80a80ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 --2- 172.21.15.131:0/2993044539 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80a806e3e0 0x7f80a80aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 -- 172.21.15.131:0/2993044539 >> 172.21.15.131:0/2993044539 conn(0x7f80a806eea0 msgr2=0x7f80a806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:40.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 -- 172.21.15.131:0/2993044539 shutdown_connections 2024-09-17T02:51:40.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.198+0000 7f80ae815640 1 -- 172.21.15.131:0/2993044539 wait complete. 2024-09-17T02:51:40.211 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:51:40.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.366+0000 7fbe70e08640 1 -- 172.21.15.131:0/1219014855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe6c0704f0 msgr2=0x7fbe6c160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.366+0000 7fbe70e08640 1 --2- 172.21.15.131:0/1219014855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe6c0704f0 0x7fbe6c160140 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c072b60 tx=0x7fbe5401c620 comp rx=0 tx=0).stop 2024-09-17T02:51:40.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.366+0000 7fbe70e08640 1 -- 172.21.15.131:0/1219014855 shutdown_connections 2024-09-17T02:51:40.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.366+0000 7fbe70e08640 1 --2- 172.21.15.131:0/1219014855 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe6c0704f0 0x7fbe6c160140 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.366+0000 7fbe70e08640 1 --2- 172.21.15.131:0/1219014855 >> v2:172.21.15.189:3300/0 conn(0x7fbe6c070df0 0x7fbe6c06ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.366+0000 7fbe70e08640 1 -- 172.21.15.131:0/1219014855 >> 172.21.15.131:0/1219014855 conn(0x7fbe6c0cdbb0 msgr2=0x7fbe6c0cdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:40.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.366+0000 7fbe70e08640 1 -- 172.21.15.131:0/1219014855 shutdown_connections 2024-09-17T02:51:40.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.366+0000 7fbe70e08640 1 -- 172.21.15.131:0/1219014855 wait complete. 2024-09-17T02:51:40.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe70e08640 1 Processor -- start 2024-09-17T02:51:40.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe70e08640 1 -- start start 2024-09-17T02:51:40.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe70e08640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe6c0704f0 0x7fbe6c0d0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:40.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe70e08640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe6c070df0 0x7fbe6c1fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:40.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe70e08640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbe6c160680 0x7fbe6c158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:40.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe70e08640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c0cc850 con 0x7fbe6c070df0 2024-09-17T02:51:40.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe70e08640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c06d5a0 con 0x7fbe6c0704f0 2024-09-17T02:51:40.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe70e08640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6c0cc9d0 con 0x7fbe6c160680 2024-09-17T02:51:40.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe69d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe6c070df0 0x7fbe6c1fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:40.365 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe69d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe6c070df0 0x7fbe6c1fdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43422/0 (socket says 172.21.15.131:43422) 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe69d9b640 1 -- 172.21.15.131:0/3782998760 learned_addr learned my addr 172.21.15.131:0/3782998760 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe6a59c640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe6c0704f0 0x7fbe6c0d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe6ad9d640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbe6c160680 0x7fbe6c158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe69d9b640 1 -- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbe6c160680 msgr2=0x7fbe6c158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe69d9b640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbe6c160680 0x7fbe6c158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe69d9b640 1 -- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe6c0704f0 msgr2=0x7fbe6c0d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe69d9b640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe6c0704f0 0x7fbe6c0d0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe69d9b640 1 -- 172.21.15.131:0/3782998760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c0d0d40 con 0x7fbe6c070df0 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe6ad9d640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbe6c160680 0x7fbe6c158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe69d9b640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe6c070df0 0x7fbe6c1fdb00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009850 tx=0x7fbe540a67d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe62ffd640 1 -- 172.21.15.131:0/3782998760 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe540bd070 con 0x7fbe6c070df0 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe62ffd640 1 -- 172.21.15.131:0/3782998760 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe540043f0 con 0x7fbe6c070df0 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe62ffd640 1 -- 172.21.15.131:0/3782998760 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54005290 con 0x7fbe6c070df0 2024-09-17T02:51:40.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe70e08640 1 -- 172.21.15.131:0/3782998760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6c0ceda0 con 0x7fbe6c070df0 2024-09-17T02:51:40.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe70e08640 1 -- 172.21.15.131:0/3782998760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6c0cf320 con 0x7fbe6c070df0 2024-09-17T02:51:40.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.370+0000 7fbe70e08640 1 -- 172.21.15.131:0/3782998760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6c160140 con 0x7fbe6c070df0 2024-09-17T02:51:40.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.374+0000 7fbe62ffd640 1 -- 172.21.15.131:0/3782998760 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbe540c0070 con 0x7fbe6c070df0 2024-09-17T02:51:40.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.374+0000 7fbe62ffd640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbe4c073200 0x7fbe4c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:40.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.374+0000 7fbe62ffd640 1 -- 172.21.15.131:0/3782998760 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6389+0+0 (secure 0 0 0) 0x7fbe5412e680 con 0x7fbe6c070df0 2024-09-17T02:51:40.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.374+0000 7fbe6a59c640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbe4c073200 0x7fbe4c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:40.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.374+0000 7fbe6a59c640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbe4c073200 0x7fbe4c075710 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005c50 tx=0x7fbe64005be0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:40.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.378+0000 7fbe62ffd640 1 -- 172.21.15.131:0/3782998760 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbe6c0ceda0 con 0x7fbe6c070df0 2024-09-17T02:51:40.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.582+0000 7fbe70e08640 1 -- 172.21.15.131:0/3782998760 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe6c1564e0 con 0x7fbe6c070df0 2024-09-17T02:51:40.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe62ffd640 1 -- 172.21.15.131:0/3782998760 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5224 (secure 0 0 0) 0x7fbe6c15a590 con 0x7fbe6c070df0 2024-09-17T02:51:40.578 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:40.578 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":94,"btime":"2024-09-17T02:51:40:493778+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2338820924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2338820924},{"type":"v1","addr":"172.21.15.131:6835","nonce":2338820924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":93,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:51:39.491567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-17T02:51:40.579 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 94 2024-09-17T02:51:40.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 -- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbe4c073200 msgr2=0x7fbe4c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbe4c073200 0x7fbe4c075710 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbe64005c50 tx=0x7fbe64005be0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 -- 172.21.15.131:0/3782998760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe6c070df0 msgr2=0x7fbe6c1fdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe6c070df0 0x7fbe6c1fdb00 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fbe54009850 tx=0x7fbe540a67d0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 -- 172.21.15.131:0/3782998760 shutdown_connections 2024-09-17T02:51:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbe4c073200 0x7fbe4c075710 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbe6c160680 0x7fbe6c158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbe6c070df0 0x7fbe6c1fdb00 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 --2- 172.21.15.131:0/3782998760 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbe6c0704f0 0x7fbe6c0d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 -- 172.21.15.131:0/3782998760 >> 172.21.15.131:0/3782998760 conn(0x7fbe6c0cdbb0 msgr2=0x7fbe6c0b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 -- 172.21.15.131:0/3782998760 shutdown_connections 2024-09-17T02:51:40.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.586+0000 7fbe70e08640 1 -- 172.21.15.131:0/3782998760 wait complete. 2024-09-17T02:51:40.599 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:51:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.750+0000 7fa72812d640 1 -- 172.21.15.131:0/2841824939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa72015c3e0 msgr2=0x7fa7200cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.750+0000 7fa72812d640 1 --2- 172.21.15.131:0/2841824939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa72015c3e0 0x7fa7200cbaf0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fa7200b61b0 tx=0x7fa71801cb80 comp rx=0 tx=0).stop 2024-09-17T02:51:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.750+0000 7fa72812d640 1 -- 172.21.15.131:0/2841824939 shutdown_connections 2024-09-17T02:51:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.750+0000 7fa72812d640 1 --2- 172.21.15.131:0/2841824939 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa72015c3e0 0x7fa7200cbaf0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.750+0000 7fa72812d640 1 --2- 172.21.15.131:0/2841824939 >> v2:172.21.15.189:3300/0 conn(0x7fa72006e5b0 0x7fa72015bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.750+0000 7fa72812d640 1 -- 172.21.15.131:0/2841824939 >> 172.21.15.131:0/2841824939 conn(0x7fa72006eea0 msgr2=0x7fa72006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.750+0000 7fa72812d640 1 -- 172.21.15.131:0/2841824939 shutdown_connections 2024-09-17T02:51:40.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.750+0000 7fa72812d640 1 -- 172.21.15.131:0/2841824939 wait complete. 2024-09-17T02:51:40.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.750+0000 7fa72812d640 1 Processor -- start 2024-09-17T02:51:40.745 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.750+0000 7fa72812d640 1 -- start start 2024-09-17T02:51:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa72812d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7200ced00 0x7fa7201fc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa72812d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa72015c3e0 0x7fa7201512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa72812d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa72014d130 0x7fa720153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa72812d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa720168750 con 0x7fa72014d130 2024-09-17T02:51:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa72812d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa7201685d0 con 0x7fa7200ced00 2024-09-17T02:51:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa72812d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa7201688d0 con 0x7fa72015c3e0 2024-09-17T02:51:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7266ca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa72014d130 0x7fa720153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:40.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7266ca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa72014d130 0x7fa720153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:43442/0 (socket says 172.21.15.131:43442) 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7266ca640 1 -- 172.21.15.131:0/3409585100 learned_addr learned my addr 172.21.15.131:0/3409585100 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7256c8640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa72015c3e0 0x7fa7201512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7266ca640 1 -- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa72015c3e0 msgr2=0x7fa7201512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa725ec9640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7200ced00 0x7fa7201fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7266ca640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa72015c3e0 0x7fa7201512c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7266ca640 1 -- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7200ced00 msgr2=0x7fa7201fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7266ca640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7200ced00 0x7fa7201fc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7266ca640 1 -- 172.21.15.131:0/3409585100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa720165a40 con 0x7fa72014d130 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7266ca640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa72014d130 0x7fa720153cb0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa71801c5f0 tx=0x7fa718005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7167fc640 1 -- 172.21.15.131:0/3409585100 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7180b2040 con 0x7fa72014d130 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7167fc640 1 -- 172.21.15.131:0/3409585100 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7180bb070 con 0x7fa72014d130 2024-09-17T02:51:40.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa72812d640 1 -- 172.21.15.131:0/3409585100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa720165d20 con 0x7fa72014d130 2024-09-17T02:51:40.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7167fc640 1 -- 172.21.15.131:0/3409585100 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7180045f0 con 0x7fa72014d130 2024-09-17T02:51:40.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa72812d640 1 -- 172.21.15.131:0/3409585100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7201fbc20 con 0x7fa72014d130 2024-09-17T02:51:40.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa72812d640 1 -- 172.21.15.131:0/3409585100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7201657b0 con 0x7fa72014d130 2024-09-17T02:51:40.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7167fc640 1 -- 172.21.15.131:0/3409585100 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa7180c0460 con 0x7fa72014d130 2024-09-17T02:51:40.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7167fc640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa704073230 0x7fa704075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:40.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa7167fc640 1 -- 172.21.15.131:0/3409585100 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6389+0+0 (secure 0 0 0) 0x7fa71812f060 con 0x7fa72014d130 2024-09-17T02:51:40.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.754+0000 7fa725ec9640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa704073230 0x7fa704075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:40.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.758+0000 7fa725ec9640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa704073230 0x7fa704075740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa71c000f70 tx=0x7fa71c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:40.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.762+0000 7fa7167fc640 1 -- 172.21.15.131:0/3409585100 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa718016770 con 0x7fa72014d130 2024-09-17T02:51:40.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.914+0000 7fa72812d640 1 -- 172.21.15.131:0/3409585100 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa72015bea0 con 0x7fa72014d130 2024-09-17T02:51:40.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.918+0000 7fa7167fc640 1 -- 172.21.15.131:0/3409585100 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+17461 (secure 0 0 0) 0x7fa7180f9e60 con 0x7fa72014d130 2024-09-17T02:51:40.912 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:40.913 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":111,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:51:39.491121+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:50:00.250118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-17T02:50:01.555264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.14","mappings":[{"from":4,"to":6}]},{"pgid":"16.15","mappings":[{"from":4,"to":3}]},{"pgid":"16.3d","mappings":[{"from":2,"to":3}]},{"pgid":"16.3f","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6835/1272267157":"2024-09-18T02:51:39.491060+0000","172.21.15.131:6833/1272267157":"2024-09-18T02:51:39.491060+0000","172.21.15.131:6835/2930400858":"2024-09-18T02:51:02.657204+0000","172.21.15.131:6833/2930400858":"2024-09-18T02:51:02.657204+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:51:40.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 -- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa704073230 msgr2=0x7fa704075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa704073230 0x7fa704075740 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa71c000f70 tx=0x7fa71c009040 comp rx=0 tx=0).stop 2024-09-17T02:51:40.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 -- 172.21.15.131:0/3409585100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa72014d130 msgr2=0x7fa720153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:40.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa72014d130 0x7fa720153cb0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa71801c5f0 tx=0x7fa718005ae0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 -- 172.21.15.131:0/3409585100 shutdown_connections 2024-09-17T02:51:40.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa704073230 0x7fa704075740 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa72014d130 0x7fa720153cb0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa72015c3e0 0x7fa7201512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 --2- 172.21.15.131:0/3409585100 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa7200ced00 0x7fa7201fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:40.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 -- 172.21.15.131:0/3409585100 >> 172.21.15.131:0/3409585100 conn(0x7fa72006eea0 msgr2=0x7fa72015a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:40.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 -- 172.21.15.131:0/3409585100 shutdown_connections 2024-09-17T02:51:40.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:40.922+0000 7fa72812d640 1 -- 172.21.15.131:0/3409585100 wait complete. 2024-09-17T02:51:40.934 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T02:51:40.934 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:51:41.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.094+0000 7f17bf4c1640 1 -- 172.21.15.131:0/1267434665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a4004810 msgr2=0x7f17a4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.094+0000 7f17bf4c1640 1 --2- 172.21.15.131:0/1267434665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a4004810 0x7f17a4004c20 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f17b4009b60 tx=0x7f17b401d290 comp rx=0 tx=0).stop 2024-09-17T02:51:41.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.094+0000 7f17bf4c1640 1 -- 172.21.15.131:0/1267434665 shutdown_connections 2024-09-17T02:51:41.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.094+0000 7f17bf4c1640 1 --2- 172.21.15.131:0/1267434665 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17a4004810 0x7f17a4004c20 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.094+0000 7f17bf4c1640 1 -- 172.21.15.131:0/1267434665 >> 172.21.15.131:0/1267434665 conn(0x7f17b806eea0 msgr2=0x7f17b806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:41.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.094+0000 7f17bf4c1640 1 -- 172.21.15.131:0/1267434665 shutdown_connections 2024-09-17T02:51:41.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.094+0000 7f17bf4c1640 1 -- 172.21.15.131:0/1267434665 wait complete. 2024-09-17T02:51:41.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bf4c1640 1 Processor -- start 2024-09-17T02:51:41.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bf4c1640 1 -- start start 2024-09-17T02:51:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bf4c1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a4004810 0x7f17b81fb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bf4c1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17b80c99c0 0x7f17b80c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bf4c1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b815ef50 0x7f17b815f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bf4c1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f17b815cbb0 con 0x7f17b815ef50 2024-09-17T02:51:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bf4c1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f17b815ca30 con 0x7f17a4004810 2024-09-17T02:51:41.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bf4c1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f17b80c50a0 con 0x7f17b80c99c0 2024-09-17T02:51:41.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bda5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b815ef50 0x7f17b815f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bda5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b815ef50 0x7f17b815f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37224/0 (socket says 172.21.15.131:37224) 2024-09-17T02:51:41.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bda5e640 1 -- 172.21.15.131:0/256069713 learned_addr learned my addr 172.21.15.131:0/256069713 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:41.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bca5c640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17b80c99c0 0x7f17b80c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bd25d640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a4004810 0x7f17b81fb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bda5e640 1 -- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17b80c99c0 msgr2=0x7f17b80c70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bda5e640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17b80c99c0 0x7f17b80c70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bda5e640 1 -- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a4004810 msgr2=0x7f17b81fb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bda5e640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a4004810 0x7f17b81fb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bda5e640 1 -- 172.21.15.131:0/256069713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b815f8a0 con 0x7f17b815ef50 2024-09-17T02:51:41.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bda5e640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b815ef50 0x7f17b815f360 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f17b80722f0 tx=0x7f17b000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:41.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17adffb640 1 -- 172.21.15.131:0/256069713 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0018070 con 0x7f17b815ef50 2024-09-17T02:51:41.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17adffb640 1 -- 172.21.15.131:0/256069713 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17b0003980 con 0x7f17b815ef50 2024-09-17T02:51:41.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bf4c1640 1 -- 172.21.15.131:0/256069713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b80ca380 con 0x7f17b815ef50 2024-09-17T02:51:41.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17adffb640 1 -- 172.21.15.131:0/256069713 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0003db0 con 0x7f17b815ef50 2024-09-17T02:51:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bf4c1640 1 -- 172.21.15.131:0/256069713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b81fa5f0 con 0x7f17b815ef50 2024-09-17T02:51:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.098+0000 7f17bf4c1640 1 -- 172.21.15.131:0/256069713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17b80ce400 con 0x7f17b815ef50 2024-09-17T02:51:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.102+0000 7f17adffb640 1 -- 172.21.15.131:0/256069713 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f17b001b070 con 0x7f17b815ef50 2024-09-17T02:51:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.102+0000 7f17adffb640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f179c073200 0x7f179c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.102+0000 7f17adffb640 1 -- 172.21.15.131:0/256069713 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6389+0+0 (secure 0 0 0) 0x7f17b0093e50 con 0x7f17b815ef50 2024-09-17T02:51:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.102+0000 7f17bd25d640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f179c073200 0x7f179c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.102+0000 7f17bd25d640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f179c073200 0x7f179c075710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f17a80018b0 tx=0x7f17a8001c20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:41.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.106+0000 7f17adffb640 1 -- 172.21.15.131:0/256069713 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f17b0022020 con 0x7f17b815ef50 2024-09-17T02:51:41.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.298+0000 7f17bf4c1640 1 -- 172.21.15.131:0/256069713 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17b80c9e30 con 0x7f17b815ef50 2024-09-17T02:51:41.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.298+0000 7f17adffb640 1 -- 172.21.15.131:0/256069713 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5224 (secure 0 0 0) 0x7f17b815f8a0 con 0x7f17b815ef50 2024-09-17T02:51:41.293 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:41.293 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":94,"btime":"2024-09-17T02:51:40:493778+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2338820924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2338820924},{"type":"v1","addr":"172.21.15.131:6835","nonce":2338820924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":93,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:51:39.491567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-17T02:51:41.293 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 94 2024-09-17T02:51:41.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 -- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f179c073200 msgr2=0x7f179c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f179c073200 0x7f179c075710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f17a80018b0 tx=0x7f17a8001c20 comp rx=0 tx=0).stop 2024-09-17T02:51:41.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 -- 172.21.15.131:0/256069713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b815ef50 msgr2=0x7f17b815f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b815ef50 0x7f17b815f360 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f17b80722f0 tx=0x7f17b000db60 comp rx=0 tx=0).stop 2024-09-17T02:51:41.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 -- 172.21.15.131:0/256069713 shutdown_connections 2024-09-17T02:51:41.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f179c073200 0x7f179c075710 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17b815ef50 0x7f17b815f360 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f17b80c99c0 0x7f17b80c70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 --2- 172.21.15.131:0/256069713 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17a4004810 0x7f17b81fb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 -- 172.21.15.131:0/256069713 >> 172.21.15.131:0/256069713 conn(0x7f17b806eea0 msgr2=0x7f17b80b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 -- 172.21.15.131:0/256069713 shutdown_connections 2024-09-17T02:51:41.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.302+0000 7f17bf4c1640 1 -- 172.21.15.131:0/256069713 wait complete. 2024-09-17T02:51:41.314 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:51:41.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.478+0000 7f992ab1f640 1 -- 172.21.15.131:0/3174042993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9924155240 msgr2=0x7f9924073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.478+0000 7f992ab1f640 1 --2- 172.21.15.131:0/3174042993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9924155240 0x7f9924073430 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f9914009840 tx=0x7f991401c7e0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.478+0000 7f992ab1f640 1 -- 172.21.15.131:0/3174042993 shutdown_connections 2024-09-17T02:51:41.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.478+0000 7f992ab1f640 1 --2- 172.21.15.131:0/3174042993 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f992406e5b0 0x7f99240cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.478+0000 7f992ab1f640 1 --2- 172.21.15.131:0/3174042993 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9924155240 0x7f9924073430 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.478+0000 7f992ab1f640 1 -- 172.21.15.131:0/3174042993 >> 172.21.15.131:0/3174042993 conn(0x7f992406eea0 msgr2=0x7f992406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:41.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.478+0000 7f992ab1f640 1 -- 172.21.15.131:0/3174042993 shutdown_connections 2024-09-17T02:51:41.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.478+0000 7f992ab1f640 1 -- 172.21.15.131:0/3174042993 wait complete. 2024-09-17T02:51:41.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f992ab1f640 1 Processor -- start 2024-09-17T02:51:41.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f992ab1f640 1 -- start start 2024-09-17T02:51:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f992ab1f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9924073970 0x7f992415a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f992ab1f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9924155240 0x7f99241f9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f992ab1f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f992414e0c0 0x7f99241f9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f992ab1f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f99240cbe70 con 0x7f9924073970 2024-09-17T02:51:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f992ab1f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f99240cbcf0 con 0x7f992414e0c0 2024-09-17T02:51:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f992ab1f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9924168c80 con 0x7f9924155240 2024-09-17T02:51:41.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99288bb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9924073970 0x7f992415a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99288bb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9924073970 0x7f992415a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37238/0 (socket says 172.21.15.131:37238) 2024-09-17T02:51:41.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99288bb640 1 -- 172.21.15.131:0/2260382803 learned_addr learned my addr 172.21.15.131:0/2260382803 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:41.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f9923fff640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9924155240 0x7f99241f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99288bb640 1 -- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9924155240 msgr2=0x7f99241f9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99288bb640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9924155240 0x7f99241f9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99288bb640 1 -- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f992414e0c0 msgr2=0x7f99241f9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99290bc640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f992414e0c0 0x7f99241f9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99288bb640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f992414e0c0 0x7f99241f9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99288bb640 1 -- 172.21.15.131:0/2260382803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992415acc0 con 0x7f9924073970 2024-09-17T02:51:41.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99290bc640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f992414e0c0 0x7f99241f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:41.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99288bb640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9924073970 0x7f992415a460 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f9914009790 tx=0x7f99140a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:41.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99217fa640 1 -- 172.21.15.131:0/2260382803 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99140bb070 con 0x7f9924073970 2024-09-17T02:51:41.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99217fa640 1 -- 172.21.15.131:0/2260382803 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9914004430 con 0x7f9924073970 2024-09-17T02:51:41.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f992ab1f640 1 -- 172.21.15.131:0/2260382803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9924158900 con 0x7f9924073970 2024-09-17T02:51:41.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f99217fa640 1 -- 172.21.15.131:0/2260382803 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99140b2040 con 0x7f9924073970 2024-09-17T02:51:41.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f992ab1f640 1 -- 172.21.15.131:0/2260382803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9924158ec0 con 0x7f9924073970 2024-09-17T02:51:41.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.482+0000 7f992ab1f640 1 -- 172.21.15.131:0/2260382803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99241586f0 con 0x7f9924073970 2024-09-17T02:51:41.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.486+0000 7f99217fa640 1 -- 172.21.15.131:0/2260382803 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f99140be070 con 0x7f9924073970 2024-09-17T02:51:41.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.486+0000 7f99217fa640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9910073200 0x7f9910075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.486+0000 7f99217fa640 1 -- 172.21.15.131:0/2260382803 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6389+0+0 (secure 0 0 0) 0x7f991412de70 con 0x7f9924073970 2024-09-17T02:51:41.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.486+0000 7f9923fff640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9910073200 0x7f9910075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.486+0000 7f9923fff640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9910073200 0x7f9910075710 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9918007fd0 tx=0x7f9918009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:41.483 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.490+0000 7f99217fa640 1 -- 172.21.15.131:0/2260382803 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f99140163d0 con 0x7f9924073970 2024-09-17T02:51:41.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.690+0000 7f992ab1f640 1 -- 172.21.15.131:0/2260382803 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99240cab60 con 0x7f9924073970 2024-09-17T02:51:41.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.690+0000 7f99217fa640 1 -- 172.21.15.131:0/2260382803 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5224 (secure 0 0 0) 0x7f99140f8be0 con 0x7f9924073970 2024-09-17T02:51:41.686 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:41.686 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":94,"btime":"2024-09-17T02:51:40:493778+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2338820924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2338820924},{"type":"v1","addr":"172.21.15.131:6835","nonce":2338820924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[{"mdsmap":{"epoch":93,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:50:02.583706+0000","modified":"2024-09-17T02:51:39.491567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-17T02:51:41.687 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 94 2024-09-17T02:51:41.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 -- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9910073200 msgr2=0x7f9910075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9910073200 0x7f9910075710 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9918007fd0 tx=0x7f9918009040 comp rx=0 tx=0).stop 2024-09-17T02:51:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 -- 172.21.15.131:0/2260382803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9924073970 msgr2=0x7f992415a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9924073970 0x7f992415a460 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f9914009790 tx=0x7f99140a5790 comp rx=0 tx=0).stop 2024-09-17T02:51:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 -- 172.21.15.131:0/2260382803 shutdown_connections 2024-09-17T02:51:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9910073200 0x7f9910075710 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f992414e0c0 0x7f99241f9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9924155240 0x7f99241f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 --2- 172.21.15.131:0/2260382803 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9924073970 0x7f992415a460 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 -- 172.21.15.131:0/2260382803 >> 172.21.15.131:0/2260382803 conn(0x7f992406eea0 msgr2=0x7f992415b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:41.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 -- 172.21.15.131:0/2260382803 shutdown_connections 2024-09-17T02:51:41.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.694+0000 7f992ab1f640 1 -- 172.21.15.131:0/2260382803 wait complete. 2024-09-17T02:51:41.707 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:51:41.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.854+0000 7f2b849f9640 1 -- 172.21.15.131:0/1517769461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b68005310 msgr2=0x7f2b6800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.854+0000 7f2b849f9640 1 --2- 172.21.15.131:0/1517769461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b68005310 0x7f2b6800b9d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f2b8015ee60 tx=0x7f2b7801d150 comp rx=0 tx=0).stop 2024-09-17T02:51:41.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.854+0000 7f2b849f9640 1 -- 172.21.15.131:0/1517769461 shutdown_connections 2024-09-17T02:51:41.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.854+0000 7f2b849f9640 1 --2- 172.21.15.131:0/1517769461 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b68005310 0x7f2b6800b9d0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.854+0000 7f2b849f9640 1 -- 172.21.15.131:0/1517769461 >> 172.21.15.131:0/1517769461 conn(0x7f2b800d1b10 msgr2=0x7f2b800d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:41.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.854+0000 7f2b849f9640 1 -- 172.21.15.131:0/1517769461 shutdown_connections 2024-09-17T02:51:41.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.854+0000 7f2b849f9640 1 -- 172.21.15.131:0/1517769461 wait complete. 2024-09-17T02:51:41.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b849f9640 1 Processor -- start 2024-09-17T02:51:41.852 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b849f9640 1 -- start start 2024-09-17T02:51:41.853 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b849f9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b68005310 0x7f2b80151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b849f9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b800cde20 0x7f2b801f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b849f9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b800ce230 0x7f2b801f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b849f9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8006d810 con 0x7f2b68005310 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b849f9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8006d690 con 0x7f2b800ce230 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b849f9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2b8015f290 con 0x7f2b800cde20 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b7e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b68005310 0x7f2b80151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b7e59c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b68005310 0x7f2b80151320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37252/0 (socket says 172.21.15.131:37252) 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b7e59c640 1 -- 172.21.15.131:0/39110848 learned_addr learned my addr 172.21.15.131:0/39110848 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b7ed9d640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b800ce230 0x7f2b801f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b7dd9b640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b800cde20 0x7f2b801f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b7e59c640 1 -- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b800cde20 msgr2=0x7f2b801f5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b7e59c640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b800cde20 0x7f2b801f5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b7e59c640 1 -- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b800ce230 msgr2=0x7f2b801f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:41.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b7e59c640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b800ce230 0x7f2b801f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:41.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b7e59c640 1 -- 172.21.15.131:0/39110848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b80151ce0 con 0x7f2b68005310 2024-09-17T02:51:41.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b7e59c640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b68005310 0x7f2b80151320 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000d500 tx=0x7f2b7000d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:41.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b6effd640 1 -- 172.21.15.131:0/39110848 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70016020 con 0x7f2b68005310 2024-09-17T02:51:41.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b6effd640 1 -- 172.21.15.131:0/39110848 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b70012630 con 0x7f2b68005310 2024-09-17T02:51:41.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b849f9640 1 -- 172.21.15.131:0/39110848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b80151fc0 con 0x7f2b68005310 2024-09-17T02:51:41.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b6effd640 1 -- 172.21.15.131:0/39110848 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70012a40 con 0x7f2b68005310 2024-09-17T02:51:41.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b849f9640 1 -- 172.21.15.131:0/39110848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b801f6aa0 con 0x7f2b68005310 2024-09-17T02:51:41.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.858+0000 7f2b849f9640 1 -- 172.21.15.131:0/39110848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b800d2460 con 0x7f2b68005310 2024-09-17T02:51:41.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.862+0000 7f2b6effd640 1 -- 172.21.15.131:0/39110848 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2b70019020 con 0x7f2b68005310 2024-09-17T02:51:41.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.862+0000 7f2b6effd640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b60073230 0x7f2b60075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:41.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.862+0000 7f2b6effd640 1 -- 172.21.15.131:0/39110848 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6389+0+0 (secure 0 0 0) 0x7f2b70094640 con 0x7f2b68005310 2024-09-17T02:51:41.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.862+0000 7f2b7dd9b640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b60073230 0x7f2b60075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:41.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.862+0000 7f2b7dd9b640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b60073230 0x7f2b60075740 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2b74001250 tx=0x7f2b74001160 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:41.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:41.866+0000 7f2b6effd640 1 -- 172.21.15.131:0/39110848 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2b70020020 con 0x7f2b68005310 2024-09-17T02:51:42.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.038+0000 7f2b849f9640 1 -- 172.21.15.131:0/39110848 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b80157710 con 0x7f2b68005310 2024-09-17T02:51:42.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.042+0000 7f2b6effd640 1 -- 172.21.15.131:0/39110848 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) ==== 74+0+17461 (secure 0 0 0) 0x7f2b7005f500 con 0x7f2b68005310 2024-09-17T02:51:42.037 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:42.037 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":111,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:51:39.491121+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:50:00.250118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-17T02:50:01.555264+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.14","mappings":[{"from":4,"to":6}]},{"pgid":"16.15","mappings":[{"from":4,"to":3}]},{"pgid":"16.3d","mappings":[{"from":2,"to":3}]},{"pgid":"16.3f","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6835/1272267157":"2024-09-18T02:51:39.491060+0000","172.21.15.131:6833/1272267157":"2024-09-18T02:51:39.491060+0000","172.21.15.131:6835/2930400858":"2024-09-18T02:51:02.657204+0000","172.21.15.131:6833/2930400858":"2024-09-18T02:51:02.657204+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:51:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 -- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b60073230 msgr2=0x7f2b60075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b60073230 0x7f2b60075740 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2b74001250 tx=0x7f2b74001160 comp rx=0 tx=0).stop 2024-09-17T02:51:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 -- 172.21.15.131:0/39110848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b68005310 msgr2=0x7f2b80151320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b68005310 0x7f2b80151320 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000d500 tx=0x7f2b7000d9c0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 -- 172.21.15.131:0/39110848 shutdown_connections 2024-09-17T02:51:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2b60073230 0x7f2b60075740 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2b800ce230 0x7f2b801f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2b800cde20 0x7f2b801f5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 --2- 172.21.15.131:0/39110848 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2b68005310 0x7f2b80151320 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 -- 172.21.15.131:0/39110848 >> 172.21.15.131:0/39110848 conn(0x7f2b800d1b10 msgr2=0x7f2b80158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 -- 172.21.15.131:0/39110848 shutdown_connections 2024-09-17T02:51:42.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.046+0000 7f2b849f9640 1 -- 172.21.15.131:0/39110848 wait complete. 2024-09-17T02:51:42.059 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:51:42.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/442001302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a640c4ac0 msgr2=0x7f5a640c4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:42.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 --2- 172.21.15.131:0/442001302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a640c4ac0 0x7f5a640c4ed0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009840 tx=0x7f5a6001c790 comp rx=0 tx=0).stop 2024-09-17T02:51:42.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/442001302 shutdown_connections 2024-09-17T02:51:42.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 --2- 172.21.15.131:0/442001302 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a640c4ac0 0x7f5a640c4ed0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 --2- 172.21.15.131:0/442001302 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a6406e2c0 0x7f5a640c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/442001302 >> 172.21.15.131:0/442001302 conn(0x7f5a6406eea0 msgr2=0x7f5a6406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:42.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/442001302 shutdown_connections 2024-09-17T02:51:42.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/442001302 wait complete. 2024-09-17T02:51:42.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 Processor -- start 2024-09-17T02:51:42.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 -- start start 2024-09-17T02:51:42.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a6406e2c0 0x7f5a64156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:42.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a640c4ac0 0x7f5a64152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:42.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a640cdbf0 0x7f5a64152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5a640cf2b0 con 0x7f5a6406e2c0 2024-09-17T02:51:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5a640cf130 con 0x7f5a640c4ac0 2024-09-17T02:51:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6c2c1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5a640cf430 con 0x7f5a640cdbf0 2024-09-17T02:51:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6a05d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a6406e2c0 0x7f5a64156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6a05d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a6406e2c0 0x7f5a64156c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37268/0 (socket says 172.21.15.131:37268) 2024-09-17T02:51:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.214+0000 7f5a6a05d640 1 -- 172.21.15.131:0/3596395712 learned_addr learned my addr 172.21.15.131:0/3596395712 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6a85e640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a640cdbf0 0x7f5a64152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6985c640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a640c4ac0 0x7f5a64152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6a05d640 1 -- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a640cdbf0 msgr2=0x7f5a64152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6a05d640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a640cdbf0 0x7f5a64152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6a05d640 1 -- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a640c4ac0 msgr2=0x7f5a64152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:42.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6a05d640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a640c4ac0 0x7f5a64152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6a05d640 1 -- 172.21.15.131:0/3596395712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a640c3c30 con 0x7f5a6406e2c0 2024-09-17T02:51:42.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6985c640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a640c4ac0 0x7f5a64152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:42.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6a85e640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a640cdbf0 0x7f5a64152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:42.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6a05d640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a6406e2c0 0x7f5a64156c00 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c970 tx=0x7f5a5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:42.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a5affd640 1 -- 172.21.15.131:0/3596395712 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a54004220 con 0x7f5a6406e2c0 2024-09-17T02:51:42.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a5affd640 1 -- 172.21.15.131:0/3596395712 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a54007950 con 0x7f5a6406e2c0 2024-09-17T02:51:42.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/3596395712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a640c3930 con 0x7f5a6406e2c0 2024-09-17T02:51:42.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a5affd640 1 -- 172.21.15.131:0/3596395712 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a540044b0 con 0x7f5a6406e2c0 2024-09-17T02:51:42.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/3596395712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6415b5d0 con 0x7f5a6406e2c0 2024-09-17T02:51:42.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/3596395712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a641463a0 con 0x7f5a6406e2c0 2024-09-17T02:51:42.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a5affd640 1 -- 172.21.15.131:0/3596395712 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5a54003600 con 0x7f5a6406e2c0 2024-09-17T02:51:42.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a5affd640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a48073200 0x7f5a48075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:42.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a5affd640 1 -- 172.21.15.131:0/3596395712 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6389+0+0 (secure 0 0 0) 0x7f5a54094960 con 0x7f5a6406e2c0 2024-09-17T02:51:42.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6985c640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a48073200 0x7f5a48075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:42.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.218+0000 7f5a6985c640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a48073200 0x7f5a48075710 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5a600ae000 tx=0x7f5a6001afd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:42.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.222+0000 7f5a5affd640 1 -- 172.21.15.131:0/3596395712 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5a54014030 con 0x7f5a6406e2c0 2024-09-17T02:51:42.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.414+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/3596395712 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5a640b19d0 con 0x7f5a6406e2c0 2024-09-17T02:51:42.498 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a5affd640 1 -- 172.21.15.131:0/3596395712 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) ==== 161+0+0 (secure 0 0 0) 0x7f5a5405f6b0 con 0x7f5a6406e2c0 2024-09-17T02:51:42.498 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:51:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a48073200 msgr2=0x7f5a48075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a48073200 0x7f5a48075710 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5a600ae000 tx=0x7f5a6001afd0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/3596395712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a6406e2c0 msgr2=0x7f5a64156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a6406e2c0 0x7f5a64156c00 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400c970 tx=0x7f5a5400ce30 comp rx=0 tx=0).stop 2024-09-17T02:51:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/3596395712 shutdown_connections 2024-09-17T02:51:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5a48073200 0x7f5a48075710 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5a640cdbf0 0x7f5a64152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5a640c4ac0 0x7f5a64152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 --2- 172.21.15.131:0/3596395712 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5a6406e2c0 0x7f5a64156c00 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/3596395712 >> 172.21.15.131:0/3596395712 conn(0x7f5a6406eea0 msgr2=0x7f5a64149920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/3596395712 shutdown_connections 2024-09-17T02:51:42.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.506+0000 7f5a6c2c1640 1 -- 172.21.15.131:0/3596395712 wait complete. 2024-09-17T02:51:42.522 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T02:51:42.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.686+0000 7f30e8723640 1 -- 172.21.15.131:0/828668251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30cc003a60 msgr2=0x7f30cc003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:42.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.686+0000 7f30e8723640 1 --2- 172.21.15.131:0/828668251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30cc003a60 0x7f30cc003ed0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f30dc001060 tx=0x7f30dc01cf60 comp rx=0 tx=0).stop 2024-09-17T02:51:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.686+0000 7f30e8723640 1 -- 172.21.15.131:0/828668251 shutdown_connections 2024-09-17T02:51:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.686+0000 7f30e8723640 1 --2- 172.21.15.131:0/828668251 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30cc003a60 0x7f30cc003ed0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.686+0000 7f30e8723640 1 -- 172.21.15.131:0/828668251 >> 172.21.15.131:0/828668251 conn(0x7f30e0073d10 msgr2=0x7f30e0074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.686+0000 7f30e8723640 1 -- 172.21.15.131:0/828668251 shutdown_connections 2024-09-17T02:51:42.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.686+0000 7f30e8723640 1 -- 172.21.15.131:0/828668251 wait complete. 2024-09-17T02:51:42.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.686+0000 7f30e8723640 1 Processor -- start 2024-09-17T02:51:42.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.686+0000 7f30e8723640 1 -- start start 2024-09-17T02:51:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e8723640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30cc003a60 0x7f30e01e8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e8723640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30e00c6130 0x7f30e00caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e8723640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30e014d0d0 0x7f30e01e7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e8723640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f30e01523e0 con 0x7f30cc003a60 2024-09-17T02:51:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e8723640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0152260 con 0x7f30e00c6130 2024-09-17T02:51:42.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e8723640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f30e0152560 con 0x7f30e014d0d0 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e64bf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30cc003a60 0x7f30e01e8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e64bf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30cc003a60 0x7f30e01e8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37280/0 (socket says 172.21.15.131:37280) 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e64bf640 1 -- 172.21.15.131:0/73109672 learned_addr learned my addr 172.21.15.131:0/73109672 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e5cbe640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30e00c6130 0x7f30e00caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e6cc0640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30e014d0d0 0x7f30e01e7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e64bf640 1 -- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30e014d0d0 msgr2=0x7f30e01e7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e64bf640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30e014d0d0 0x7f30e01e7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e64bf640 1 -- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30e00c6130 msgr2=0x7f30e00caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e64bf640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30e00c6130 0x7f30e00caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e64bf640 1 -- 172.21.15.131:0/73109672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e00c69f0 con 0x7f30cc003a60 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e6cc0640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30e014d0d0 0x7f30e01e7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e64bf640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30cc003a60 0x7f30e01e8f00 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f30dc01ca20 tx=0x7f30dc000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:42.685 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30d6ffd640 1 -- 172.21.15.131:0/73109672 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0017c0 con 0x7f30cc003a60 2024-09-17T02:51:42.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30d6ffd640 1 -- 172.21.15.131:0/73109672 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30dc006280 con 0x7f30cc003a60 2024-09-17T02:51:42.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e8723640 1 -- 172.21.15.131:0/73109672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e00c6cd0 con 0x7f30cc003a60 2024-09-17T02:51:42.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30d6ffd640 1 -- 172.21.15.131:0/73109672 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc003600 con 0x7f30cc003a60 2024-09-17T02:51:42.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e8723640 1 -- 172.21.15.131:0/73109672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e00c9090 con 0x7f30cc003a60 2024-09-17T02:51:42.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30e8723640 1 -- 172.21.15.131:0/73109672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e00cb820 con 0x7f30cc003a60 2024-09-17T02:51:42.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30d6ffd640 1 -- 172.21.15.131:0/73109672 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f30dc0037a0 con 0x7f30cc003a60 2024-09-17T02:51:42.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30d6ffd640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f30c4073200 0x7f30c4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:42.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.690+0000 7f30d6ffd640 1 -- 172.21.15.131:0/73109672 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6389+0+0 (secure 0 0 0) 0x7f30dc0b4020 con 0x7f30cc003a60 2024-09-17T02:51:42.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.694+0000 7f30e5cbe640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f30c4073200 0x7f30c4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:42.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.694+0000 7f30e5cbe640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f30c4073200 0x7f30c4075710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f30d00058d0 tx=0x7f30d0005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:42.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.698+0000 7f30d6ffd640 1 -- 172.21.15.131:0/73109672 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f30dc016d10 con 0x7f30cc003a60 2024-09-17T02:51:42.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:42.902+0000 7f30e8723640 1 -- 172.21.15.131:0/73109672 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f30e00d0d80 con 0x7f30cc003a60 2024-09-17T02:51:43.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.514+0000 7f30d6ffd640 1 -- 172.21.15.131:0/73109672 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) ==== 104+0+0 (secure 0 0 0) 0x7f30dc0f8de0 con 0x7f30cc003a60 2024-09-17T02:51:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 -- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f30c4073200 msgr2=0x7f30c4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f30c4073200 0x7f30c4075710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f30d00058d0 tx=0x7f30d0005860 comp rx=0 tx=0).stop 2024-09-17T02:51:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 -- 172.21.15.131:0/73109672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30cc003a60 msgr2=0x7f30e01e8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30cc003a60 0x7f30e01e8f00 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f30dc01ca20 tx=0x7f30dc000fb0 comp rx=0 tx=0).stop 2024-09-17T02:51:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 -- 172.21.15.131:0/73109672 shutdown_connections 2024-09-17T02:51:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f30c4073200 0x7f30c4075710 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f30e014d0d0 0x7f30e01e7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f30e00c6130 0x7f30e00caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 --2- 172.21.15.131:0/73109672 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f30cc003a60 0x7f30e01e8f00 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 -- 172.21.15.131:0/73109672 >> 172.21.15.131:0/73109672 conn(0x7f30e0073d10 msgr2=0x7f30e0150290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:43.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 -- 172.21.15.131:0/73109672 shutdown_connections 2024-09-17T02:51:43.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.518+0000 7f30e8723640 1 -- 172.21.15.131:0/73109672 wait complete. 2024-09-17T02:51:43.542 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T02:51:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.714+0000 7f558135d640 1 -- 172.21.15.131:0/2389479037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564004e70 msgr2=0x7f55640052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.714+0000 7f558135d640 1 --2- 172.21.15.131:0/2389479037 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564004e70 0x7f55640052a0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f5574009ee0 tx=0x7f557401d630 comp rx=0 tx=0).stop 2024-09-17T02:51:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.714+0000 7f557b7fe640 1 -- 172.21.15.131:0/2389479037 reap_dead start 2024-09-17T02:51:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.714+0000 7f558135d640 1 -- 172.21.15.131:0/2389479037 shutdown_connections 2024-09-17T02:51:43.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.714+0000 7f558135d640 1 -- 172.21.15.131:0/2389479037 >> 172.21.15.131:0/2389479037 conn(0x7f557c0d05a0 msgr2=0x7f557c0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:43.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.714+0000 7f558135d640 1 -- 172.21.15.131:0/2389479037 shutdown_connections 2024-09-17T02:51:43.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.714+0000 7f558135d640 1 -- 172.21.15.131:0/2389479037 wait complete. 2024-09-17T02:51:43.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.714+0000 7f558135d640 1 Processor -- start 2024-09-17T02:51:43.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.714+0000 7f558135d640 1 -- start start 2024-09-17T02:51:43.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f558135d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564004e70 0x7f557c1fc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:43.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f558135d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f557c149340 0x7f557c154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:43.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f558135d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f557c155530 0x7f557c15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:43.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f558135d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f557c167560 con 0x7f5564004e70 2024-09-17T02:51:43.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f558135d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f557c1673e0 con 0x7f557c155530 2024-09-17T02:51:43.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f558135d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f557c1676e0 con 0x7f557c149340 2024-09-17T02:51:43.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564004e70 0x7f557c1fc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:43.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564004e70 0x7f557c1fc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37298/0 (socket says 172.21.15.131:37298) 2024-09-17T02:51:43.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557affd640 1 -- 172.21.15.131:0/2832539904 learned_addr learned my addr 172.21.15.131:0/2832539904 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:43.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557b7fe640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f557c155530 0x7f557c15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:43.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557a7fc640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f557c149340 0x7f557c154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:43.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557affd640 1 -- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f557c149340 msgr2=0x7f557c154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:43.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557affd640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f557c149340 0x7f557c154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:43.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557affd640 1 -- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f557c155530 msgr2=0x7f557c15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:43.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557affd640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f557c155530 0x7f557c15caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:43.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557affd640 1 -- 172.21.15.131:0/2832539904 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557c155940 con 0x7f5564004e70 2024-09-17T02:51:43.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557b7fe640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f557c155530 0x7f557c15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:43.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f557affd640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564004e70 0x7f557c1fc400 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f556c00d920 tx=0x7f556c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:43.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f55637fe640 1 -- 172.21.15.131:0/2832539904 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c0016f0 con 0x7f5564004e70 2024-09-17T02:51:43.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f55637fe640 1 -- 172.21.15.131:0/2832539904 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f556c00b830 con 0x7f5564004e70 2024-09-17T02:51:43.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f55637fe640 1 -- 172.21.15.131:0/2832539904 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c00bca0 con 0x7f5564004e70 2024-09-17T02:51:43.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f558135d640 1 -- 172.21.15.131:0/2832539904 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557c15e740 con 0x7f5564004e70 2024-09-17T02:51:43.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f558135d640 1 -- 172.21.15.131:0/2832539904 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557c15ed00 con 0x7f5564004e70 2024-09-17T02:51:43.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f558135d640 1 -- 172.21.15.131:0/2832539904 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557c073370 con 0x7f5564004e70 2024-09-17T02:51:43.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.718+0000 7f55637fe640 1 -- 172.21.15.131:0/2832539904 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f556c003600 con 0x7f5564004e70 2024-09-17T02:51:43.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.722+0000 7f55637fe640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5568073200 0x7f5568075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:43.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.722+0000 7f55637fe640 1 -- 172.21.15.131:0/2832539904 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 6389+0+0 (secure 0 0 0) 0x7f556c094ac0 con 0x7f5564004e70 2024-09-17T02:51:43.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.722+0000 7f557a7fc640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5568073200 0x7f5568075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:43.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.722+0000 7f557a7fc640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5568073200 0x7f5568075710 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f557c1512d0 tx=0x7f5570009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:43.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.726+0000 7f55637fe640 1 -- 172.21.15.131:0/2832539904 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f557c155940 con 0x7f5564004e70 2024-09-17T02:51:43.890 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:43.898+0000 7f558135d640 1 -- 172.21.15.131:0/2832539904 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f557c150d40 con 0x7f5564004e70 2024-09-17T02:51:44.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.538+0000 7f55637fe640 1 -- 172.21.15.131:0/2832539904 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) ==== 181+0+0 (secure 0 0 0) 0x7f556c05f830 con 0x7f5564004e70 2024-09-17T02:51:44.531 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-09-17T02:51:44.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.542+0000 7f558135d640 1 -- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5568073200 msgr2=0x7f5568075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:44.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.542+0000 7f558135d640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5568073200 0x7f5568075710 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f557c1512d0 tx=0x7f5570009040 comp rx=0 tx=0).stop 2024-09-17T02:51:44.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.542+0000 7f558135d640 1 -- 172.21.15.131:0/2832539904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564004e70 msgr2=0x7f557c1fc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:44.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.542+0000 7f558135d640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564004e70 0x7f557c1fc400 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f556c00d920 tx=0x7f556c00dde0 comp rx=0 tx=0).stop 2024-09-17T02:51:44.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.546+0000 7f558135d640 1 -- 172.21.15.131:0/2832539904 shutdown_connections 2024-09-17T02:51:44.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.546+0000 7f558135d640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5568073200 0x7f5568075710 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:44.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.546+0000 7f558135d640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f557c155530 0x7f557c15caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:44.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.546+0000 7f558135d640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f557c149340 0x7f557c154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:44.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.546+0000 7f558135d640 1 --2- 172.21.15.131:0/2832539904 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5564004e70 0x7f557c1fc400 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:44.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.546+0000 7f558135d640 1 -- 172.21.15.131:0/2832539904 >> 172.21.15.131:0/2832539904 conn(0x7f557c0d05a0 msgr2=0x7f557c0b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:44.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.546+0000 7f558135d640 1 -- 172.21.15.131:0/2832539904 shutdown_connections 2024-09-17T02:51:44.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.546+0000 7f558135d640 1 -- 172.21.15.131:0/2832539904 wait complete. 2024-09-17T02:51:44.563 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T02:51:44.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.734+0000 7f2710c8c640 1 -- 172.21.15.131:0/3329681960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f270c06f700 msgr2=0x7f270c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:44.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.734+0000 7f2710c8c640 1 --2- 172.21.15.131:0/3329681960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f270c06f700 0x7f270c06fb10 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009840 tx=0x7f26fc01c6c0 comp rx=0 tx=0).stop 2024-09-17T02:51:44.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.734+0000 7f2710c8c640 1 -- 172.21.15.131:0/3329681960 shutdown_connections 2024-09-17T02:51:44.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.734+0000 7f2710c8c640 1 --2- 172.21.15.131:0/3329681960 >> v2:172.21.15.189:3300/0 conn(0x7f270c0cbed0 0x7f270c0cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:44.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.734+0000 7f2710c8c640 1 --2- 172.21.15.131:0/3329681960 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f270c06f700 0x7f270c06fb10 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.734+0000 7f2710c8c640 1 -- 172.21.15.131:0/3329681960 >> 172.21.15.131:0/3329681960 conn(0x7f270c0d05a0 msgr2=0x7f270c0d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.734+0000 7f2710c8c640 1 -- 172.21.15.131:0/3329681960 shutdown_connections 2024-09-17T02:51:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.734+0000 7f2710c8c640 1 -- 172.21.15.131:0/3329681960 wait complete. 2024-09-17T02:51:44.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.734+0000 7f2710c8c640 1 Processor -- start 2024-09-17T02:51:44.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.734+0000 7f2710c8c640 1 -- start start 2024-09-17T02:51:44.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2710c8c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f270c06f700 0x7f270c0cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:44.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2710c8c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f270c0b2ef0 0x7f270c1f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:44.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2710c8c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f270c0cbed0 0x7f270c166860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:44.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2710c8c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f270c073b40 con 0x7f270c0b2ef0 2024-09-17T02:51:44.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2710c8c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f270c0739c0 con 0x7f270c06f700 2024-09-17T02:51:44.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2710c8c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f270c073cc0 con 0x7f270c0cbed0 2024-09-17T02:51:44.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2709d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f270c0b2ef0 0x7f270c1f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2709d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f270c0b2ef0 0x7f270c1f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37310/0 (socket says 172.21.15.131:37310) 2024-09-17T02:51:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2709d9b640 1 -- 172.21.15.131:0/2513347122 learned_addr learned my addr 172.21.15.131:0/2513347122 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f270ad9d640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f270c0cbed0 0x7f270c166860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f270a59c640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f270c06f700 0x7f270c0cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2709d9b640 1 -- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f270c0cbed0 msgr2=0x7f270c166860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2709d9b640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f270c0cbed0 0x7f270c166860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2709d9b640 1 -- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f270c06f700 msgr2=0x7f270c0cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2709d9b640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f270c06f700 0x7f270c0cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2709d9b640 1 -- 172.21.15.131:0/2513347122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270c1f6fa0 con 0x7f270c0b2ef0 2024-09-17T02:51:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2709d9b640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f270c0b2ef0 0x7f270c1f7a60 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f26fc000c10 tx=0x7f26fc005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:44.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f26faffd640 1 -- 172.21.15.131:0/2513347122 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc0ad050 con 0x7f270c0b2ef0 2024-09-17T02:51:44.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f26faffd640 1 -- 172.21.15.131:0/2513347122 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26fc005520 con 0x7f270c0b2ef0 2024-09-17T02:51:44.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2710c8c640 1 -- 172.21.15.131:0/2513347122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270c0c54d0 con 0x7f270c0b2ef0 2024-09-17T02:51:44.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f26faffd640 1 -- 172.21.15.131:0/2513347122 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26fc004910 con 0x7f270c0b2ef0 2024-09-17T02:51:44.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2710c8c640 1 -- 172.21.15.131:0/2513347122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270c0c5a90 con 0x7f270c0b2ef0 2024-09-17T02:51:44.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f2710c8c640 1 -- 172.21.15.131:0/2513347122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270c0c6da0 con 0x7f270c0b2ef0 2024-09-17T02:51:44.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f26faffd640 1 -- 172.21.15.131:0/2513347122 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f26fc0b0070 con 0x7f270c0b2ef0 2024-09-17T02:51:44.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f26faffd640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26ec073200 0x7f26ec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:44.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f26faffd640 1 -- 172.21.15.131:0/2513347122 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5951+0+0 (secure 0 0 0) 0x7f26fc12ee60 con 0x7f270c0b2ef0 2024-09-17T02:51:44.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.738+0000 7f270a59c640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26ec073200 0x7f26ec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:44.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.742+0000 7f270a59c640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26ec073200 0x7f26ec075710 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2700001570 tx=0x7f2700009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:44.737 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.742+0000 7f26faffd640 1 -- 172.21.15.131:0/2513347122 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f26fc0c2060 con 0x7f270c0b2ef0 2024-09-17T02:51:44.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:44.914+0000 7f2710c8c640 1 -- 172.21.15.131:0/2513347122 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f270c0741b0 con 0x7f270c0b2ef0 2024-09-17T02:51:45.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.550+0000 7f26faffd640 1 -- 172.21.15.131:0/2513347122 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) ==== 169+0+0 (secure 0 0 0) 0x7f26fc0163d0 con 0x7f270c0b2ef0 2024-09-17T02:51:45.543 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-09-17T02:51:45.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 -- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26ec073200 msgr2=0x7f26ec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:45.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26ec073200 0x7f26ec075710 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2700001570 tx=0x7f2700009040 comp rx=0 tx=0).stop 2024-09-17T02:51:45.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 -- 172.21.15.131:0/2513347122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f270c0b2ef0 msgr2=0x7f270c1f7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:45.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f270c0b2ef0 0x7f270c1f7a60 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f26fc000c10 tx=0x7f26fc005040 comp rx=0 tx=0).stop 2024-09-17T02:51:45.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 -- 172.21.15.131:0/2513347122 shutdown_connections 2024-09-17T02:51:45.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f26ec073200 0x7f26ec075710 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f270c0cbed0 0x7f270c166860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f270c0b2ef0 0x7f270c1f7a60 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 --2- 172.21.15.131:0/2513347122 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f270c06f700 0x7f270c0cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 -- 172.21.15.131:0/2513347122 >> 172.21.15.131:0/2513347122 conn(0x7f270c0d05a0 msgr2=0x7f270c142430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:45.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 -- 172.21.15.131:0/2513347122 shutdown_connections 2024-09-17T02:51:45.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.554+0000 7f2710c8c640 1 -- 172.21.15.131:0/2513347122 wait complete. 2024-09-17T02:51:45.583 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T02:51:45.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.750+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/2784038777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bc406dea0 msgr2=0x7f9bc406e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:45.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.750+0000 7f9bc9f7b640 1 --2- 172.21.15.131:0/2784038777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bc406dea0 0x7f9bc406e2b0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f9bc40cdb80 tx=0x7f9bb401c7d0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.750+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/2784038777 shutdown_connections 2024-09-17T02:51:45.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.750+0000 7f9bc9f7b640 1 --2- 172.21.15.131:0/2784038777 >> v2:172.21.15.189:3300/0 conn(0x7f9bc40cab60 0x7f9bc40caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.750+0000 7f9bc9f7b640 1 --2- 172.21.15.131:0/2784038777 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bc406dea0 0x7f9bc406e2b0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.743 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.750+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/2784038777 >> 172.21.15.131:0/2784038777 conn(0x7f9bc406eea0 msgr2=0x7f9bc406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:45.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.750+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/2784038777 shutdown_connections 2024-09-17T02:51:45.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.750+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/2784038777 wait complete. 2024-09-17T02:51:45.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.750+0000 7f9bc9f7b640 1 Processor -- start 2024-09-17T02:51:45.744 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.750+0000 7f9bc9f7b640 1 -- start start 2024-09-17T02:51:45.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc9f7b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc406dea0 0x7f9bc40c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:45.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc9f7b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bc40cab60 0x7f9bc41442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:45.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc9f7b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bc4155240 0x7f9bc4166b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:45.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc9f7b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc40cbd00 con 0x7f9bc4155240 2024-09-17T02:51:45.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc9f7b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc40cbb80 con 0x7f9bc406dea0 2024-09-17T02:51:45.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc9f7b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc40cbe80 con 0x7f9bc40cab60 2024-09-17T02:51:45.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bc4155240 0x7f9bc4166b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:45.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bc4155240 0x7f9bc4166b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37318/0 (socket says 172.21.15.131:37318) 2024-09-17T02:51:45.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc3fff640 1 -- 172.21.15.131:0/3933621602 learned_addr learned my addr 172.21.15.131:0/3933621602 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:45.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bbbfff640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bc40cab60 0x7f9bc41442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc37fe640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc406dea0 0x7f9bc40c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc3fff640 1 -- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bc40cab60 msgr2=0x7f9bc41442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc3fff640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bc40cab60 0x7f9bc41442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc3fff640 1 -- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc406dea0 msgr2=0x7f9bc40c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc3fff640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc406dea0 0x7f9bc40c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc3fff640 1 -- 172.21.15.131:0/3933621602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc40b3140 con 0x7f9bc4155240 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc37fe640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc406dea0 0x7f9bc40c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc3fff640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bc4155240 0x7f9bc4166b70 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00cce0 tx=0x7f9bbc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc0ff9640 1 -- 172.21.15.131:0/3933621602 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00f450 con 0x7f9bc4155240 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc0ff9640 1 -- 172.21.15.131:0/3933621602 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bbc00f6e0 con 0x7f9bc4155240 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/3933621602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc40b33e0 con 0x7f9bc4155240 2024-09-17T02:51:45.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc0ff9640 1 -- 172.21.15.131:0/3933621602 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00fb30 con 0x7f9bc4155240 2024-09-17T02:51:45.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/3933621602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc40b3a50 con 0x7f9bc4155240 2024-09-17T02:51:45.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/3933621602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc40b3620 con 0x7f9bc4155240 2024-09-17T02:51:45.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc0ff9640 1 -- 172.21.15.131:0/3933621602 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9bbc00fcd0 con 0x7f9bc4155240 2024-09-17T02:51:45.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc0ff9640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ba4073230 0x7f9ba4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:45.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc0ff9640 1 -- 172.21.15.131:0/3933621602 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f9bbc094c50 con 0x7f9bc4155240 2024-09-17T02:51:45.750 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.754+0000 7f9bc37fe640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ba4073230 0x7f9ba4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:45.751 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.758+0000 7f9bc37fe640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ba4073230 0x7f9ba4075740 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9bac001630 tx=0x7f9bac009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:45.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.762+0000 7f9bc0ff9640 1 -- 172.21.15.131:0/3933621602 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9bc40b3620 con 0x7f9bc4155240 2024-09-17T02:51:45.834 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:51:45.837 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:51:45.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.934+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/3933621602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f9bc406e2b0 con 0x7f9bc4155240 2024-09-17T02:51:45.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.942+0000 7f9bc0ff9640 1 -- 172.21.15.131:0/3933621602 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v43) ==== 71+0+0 (secure 0 0 0) 0x7f9bbc09a020 con 0x7f9bc4155240 2024-09-17T02:51:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ba4073230 msgr2=0x7f9ba4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ba4073230 0x7f9ba4075740 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9bac001630 tx=0x7f9bac009040 comp rx=0 tx=0).stop 2024-09-17T02:51:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/3933621602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bc4155240 msgr2=0x7f9bc4166b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:45.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bc4155240 0x7f9bc4166b70 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00cce0 tx=0x7f9bbc00a620 comp rx=0 tx=0).stop 2024-09-17T02:51:45.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/3933621602 shutdown_connections 2024-09-17T02:51:45.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9ba4073230 0x7f9ba4075740 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bc4155240 0x7f9bc4166b70 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9bc40cab60 0x7f9bc41442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 --2- 172.21.15.131:0/3933621602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9bc406dea0 0x7f9bc40c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:45.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/3933621602 >> 172.21.15.131:0/3933621602 conn(0x7f9bc406eea0 msgr2=0x7f9bc40733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:45.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/3933621602 shutdown_connections 2024-09-17T02:51:45.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:45.946+0000 7f9bc9f7b640 1 -- 172.21.15.131:0/3933621602 wait complete. 2024-09-17T02:51:45.960 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-09-17T02:51:46.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.122+0000 7fcf0ff50640 1 -- 172.21.15.131:0/4120742843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf08155240 msgr2=0x7fcf08073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:46.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.122+0000 7fcf0ff50640 1 --2- 172.21.15.131:0/4120742843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf08155240 0x7fcf08073430 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400a950 tx=0x7fcf0401db30 comp rx=0 tx=0).stop 2024-09-17T02:51:46.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.122+0000 7fcf0ff50640 1 -- 172.21.15.131:0/4120742843 shutdown_connections 2024-09-17T02:51:46.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.122+0000 7fcf0ff50640 1 --2- 172.21.15.131:0/4120742843 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf08155240 0x7fcf08073430 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.122+0000 7fcf0ff50640 1 -- 172.21.15.131:0/4120742843 >> 172.21.15.131:0/4120742843 conn(0x7fcf0806eea0 msgr2=0x7fcf0806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:46.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.122+0000 7fcf0ff50640 1 -- 172.21.15.131:0/4120742843 shutdown_connections 2024-09-17T02:51:46.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.122+0000 7fcf0ff50640 1 -- 172.21.15.131:0/4120742843 wait complete. 2024-09-17T02:51:46.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0ff50640 1 Processor -- start 2024-09-17T02:51:46.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0ff50640 1 -- start start 2024-09-17T02:51:46.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0ff50640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf08073970 0x7fcf08145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:46.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0ff50640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf08155240 0x7fcf08159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0ff50640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf08153700 0x7fcf0815a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0ff50640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcf08168b80 con 0x7fcf08153700 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0ff50640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcf080cbe40 con 0x7fcf08073970 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0ff50640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcf08168d00 con 0x7fcf08155240 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0e4ed640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf08153700 0x7fcf0815a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0e4ed640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf08153700 0x7fcf0815a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37326/0 (socket says 172.21.15.131:37326) 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0e4ed640 1 -- 172.21.15.131:0/550257644 learned_addr learned my addr 172.21.15.131:0/550257644 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0dcec640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf08073970 0x7fcf08145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0d4eb640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf08155240 0x7fcf08159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0e4ed640 1 -- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf08155240 msgr2=0x7fcf08159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0e4ed640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf08155240 0x7fcf08159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0e4ed640 1 -- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf08073970 msgr2=0x7fcf08145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0e4ed640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf08073970 0x7fcf08145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0e4ed640 1 -- 172.21.15.131:0/550257644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0815bc40 con 0x7fcf08153700 2024-09-17T02:51:46.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0dcec640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf08073970 0x7fcf08145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:51:46.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0d4eb640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf08155240 0x7fcf08159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:46.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcf0e4ed640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf08153700 0x7fcf0815a4b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000cb10 tx=0x7fcf0000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:46.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.126+0000 7fcefe7fc640 1 -- 172.21.15.131:0/550257644 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf00016020 con 0x7fcf08153700 2024-09-17T02:51:46.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.130+0000 7fcefe7fc640 1 -- 172.21.15.131:0/550257644 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf0000ce50 con 0x7fcf08153700 2024-09-17T02:51:46.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.130+0000 7fcf0ff50640 1 -- 172.21.15.131:0/550257644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf08151d50 con 0x7fcf08153700 2024-09-17T02:51:46.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.130+0000 7fcefe7fc640 1 -- 172.21.15.131:0/550257644 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000ada0 con 0x7fcf08153700 2024-09-17T02:51:46.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.130+0000 7fcf0ff50640 1 -- 172.21.15.131:0/550257644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf08152f30 con 0x7fcf08153700 2024-09-17T02:51:46.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.130+0000 7fcf0ff50640 1 -- 172.21.15.131:0/550257644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf08152ca0 con 0x7fcf08153700 2024-09-17T02:51:46.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.130+0000 7fcefe7fc640 1 -- 172.21.15.131:0/550257644 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcf00019020 con 0x7fcf08153700 2024-09-17T02:51:46.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.130+0000 7fcefe7fc640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fceec073230 0x7fceec075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:46.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.130+0000 7fcefe7fc640 1 -- 172.21.15.131:0/550257644 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7fcf000943f0 con 0x7fcf08153700 2024-09-17T02:51:46.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.130+0000 7fcf0dcec640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fceec073230 0x7fceec075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:46.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.130+0000 7fcf0dcec640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fceec073230 0x7fceec075740 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcf040b0000 tx=0x7fcf0401c370 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:46.128 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.134+0000 7fcefe7fc640 1 -- 172.21.15.131:0/550257644 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcf00020020 con 0x7fcf08153700 2024-09-17T02:51:46.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.378+0000 7fcf0ff50640 1 -- 172.21.15.131:0/550257644 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7fcf08073430 con 0x7fcf08153700 2024-09-17T02:51:46.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcefe7fc640 1 -- 172.21.15.131:0/550257644 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v487) ==== 146+0+0 (secure 0 0 0) 0x7fcf0005f410 con 0x7fcf08153700 2024-09-17T02:51:46.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 -- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fceec073230 msgr2=0x7fceec075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:46.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fceec073230 0x7fceec075740 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcf040b0000 tx=0x7fcf0401c370 comp rx=0 tx=0).stop 2024-09-17T02:51:46.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 -- 172.21.15.131:0/550257644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf08153700 msgr2=0x7fcf0815a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:46.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf08153700 0x7fcf0815a4b0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000cb10 tx=0x7fcf0000cc00 comp rx=0 tx=0).stop 2024-09-17T02:51:46.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 -- 172.21.15.131:0/550257644 shutdown_connections 2024-09-17T02:51:46.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fceec073230 0x7fceec075740 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcf08153700 0x7fcf0815a4b0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcf08155240 0x7fcf08159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 --2- 172.21.15.131:0/550257644 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcf08073970 0x7fcf08145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 -- 172.21.15.131:0/550257644 >> 172.21.15.131:0/550257644 conn(0x7fcf0806eea0 msgr2=0x7fcf080b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:46.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 -- 172.21.15.131:0/550257644 shutdown_connections 2024-09-17T02:51:46.550 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.554+0000 7fcf0ff50640 1 -- 172.21.15.131:0/550257644 wait complete. 2024-09-17T02:51:46.570 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-09-17T02:51:46.571 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-17T02:51:46.571 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:51:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.730+0000 7f205bf15640 1 -- 172.21.15.131:0/3643832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2040005260 msgr2=0x7f204000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.730+0000 7f205bf15640 1 --2- 172.21.15.131:0/3643832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2040005260 0x7f204000b9d0 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f204c0016c0 tx=0x7f204c01c5b0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.730+0000 7f205bf15640 1 -- 172.21.15.131:0/3643832 shutdown_connections 2024-09-17T02:51:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.730+0000 7f205bf15640 1 --2- 172.21.15.131:0/3643832 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2040005260 0x7f204000b9d0 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.730+0000 7f205bf15640 1 -- 172.21.15.131:0/3643832 >> 172.21.15.131:0/3643832 conn(0x7f205406eea0 msgr2=0x7f205406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.730+0000 7f205bf15640 1 -- 172.21.15.131:0/3643832 shutdown_connections 2024-09-17T02:51:46.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.730+0000 7f205bf15640 1 -- 172.21.15.131:0/3643832 wait complete. 2024-09-17T02:51:46.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.730+0000 7f205bf15640 1 Processor -- start 2024-09-17T02:51:46.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.730+0000 7f205bf15640 1 -- start start 2024-09-17T02:51:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f205bf15640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2040005260 0x7f20541f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f205bf15640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2054148270 0x7f205415afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f205bf15640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f205415b4f0 0x7f2054157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f205bf15640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f205406dea0 con 0x7f2040005260 2024-09-17T02:51:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f205bf15640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f20540cf320 con 0x7f205415b4f0 2024-09-17T02:51:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f205bf15640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f205406e020 con 0x7f2054148270 2024-09-17T02:51:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f2059cb1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2040005260 0x7f20541f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f2059cb1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2040005260 0x7f20541f2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37344/0 (socket says 172.21.15.131:37344) 2024-09-17T02:51:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f2059cb1640 1 -- 172.21.15.131:0/636366734 learned_addr learned my addr 172.21.15.131:0/636366734 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f205a4b2640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f205415b4f0 0x7f2054157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:46.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f20594b0640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2054148270 0x7f205415afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f2059cb1640 1 -- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2054148270 msgr2=0x7f205415afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f2059cb1640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2054148270 0x7f205415afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f2059cb1640 1 -- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f205415b4f0 msgr2=0x7f2054157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f2059cb1640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f205415b4f0 0x7f2054157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f2059cb1640 1 -- 172.21.15.131:0/636366734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2054156ae0 con 0x7f2040005260 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f205a4b2640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f205415b4f0 0x7f2054157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f2059cb1640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2040005260 0x7f20541f2330 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f2048009900 tx=0x7f2048009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f20467fc640 1 -- 172.21.15.131:0/636366734 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20480112e0 con 0x7f2040005260 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f20467fc640 1 -- 172.21.15.131:0/636366734 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f204800c970 con 0x7f2040005260 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f205bf15640 1 -- 172.21.15.131:0/636366734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2054156dc0 con 0x7f2040005260 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f20467fc640 1 -- 172.21.15.131:0/636366734 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204800cd80 con 0x7f2040005260 2024-09-17T02:51:46.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f205bf15640 1 -- 172.21.15.131:0/636366734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2054157380 con 0x7f2040005260 2024-09-17T02:51:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f205bf15640 1 -- 172.21.15.131:0/636366734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2054075b80 con 0x7f2040005260 2024-09-17T02:51:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.734+0000 7f20467fc640 1 -- 172.21.15.131:0/636366734 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f204800f550 con 0x7f2040005260 2024-09-17T02:51:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.738+0000 7f20467fc640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2038073230 0x7f2038075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.738+0000 7f20467fc640 1 -- 172.21.15.131:0/636366734 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f2048096a30 con 0x7f2040005260 2024-09-17T02:51:46.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.738+0000 7f20594b0640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2038073230 0x7f2038075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:46.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.738+0000 7f20594b0640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2038073230 0x7f2038075740 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2050001e00 tx=0x7f2050001a50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:46.733 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.738+0000 7f20467fc640 1 -- 172.21.15.131:0/636366734 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2048061b70 con 0x7f2040005260 2024-09-17T02:51:46.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.914+0000 7f205bf15640 1 -- 172.21.15.131:0/636366734 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f205415bef0 con 0x7f2040005260 2024-09-17T02:51:46.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.914+0000 7f20467fc640 1 -- 172.21.15.131:0/636366734 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+13497 (secure 0 0 0) 0x7f2048061470 con 0x7f2040005260 2024-09-17T02:51:46.908 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:46.908 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":113,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:51:45.539631+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6835/1272267157":"2024-09-18T02:51:39.491060+0000","172.21.15.131:6833/1272267157":"2024-09-18T02:51:39.491060+0000","172.21.15.131:6835/2930400858":"2024-09-18T02:51:02.657204+0000","172.21.15.131:6833/2930400858":"2024-09-18T02:51:02.657204+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:51:46.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 -- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2038073230 msgr2=0x7f2038075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:46.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2038073230 0x7f2038075740 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2050001e00 tx=0x7f2050001a50 comp rx=0 tx=0).stop 2024-09-17T02:51:46.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 -- 172.21.15.131:0/636366734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2040005260 msgr2=0x7f20541f2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:46.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2040005260 0x7f20541f2330 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f2048009900 tx=0x7f2048009dc0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 -- 172.21.15.131:0/636366734 shutdown_connections 2024-09-17T02:51:46.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2038073230 0x7f2038075740 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f205415b4f0 0x7f2054157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2054148270 0x7f205415afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 --2- 172.21.15.131:0/636366734 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2040005260 0x7f20541f2330 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:46.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 -- 172.21.15.131:0/636366734 >> 172.21.15.131:0/636366734 conn(0x7f205406eea0 msgr2=0x7f205415d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:46.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 -- 172.21.15.131:0/636366734 shutdown_connections 2024-09-17T02:51:46.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:46.918+0000 7f205bf15640 1 -- 172.21.15.131:0/636366734 wait complete. 2024-09-17T02:51:46.931 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T02:40:41.163614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T02:51:46.931 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T02:51:47.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.094+0000 7f98c51b7640 1 -- 172.21.15.131:0/1784649136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98c00cdbf0 msgr2=0x7f98c00ce000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.094+0000 7f98c51b7640 1 --2- 172.21.15.131:0/1784649136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98c00cdbf0 0x7f98c00ce000 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f98c00757f0 tx=0x7f98b001c830 comp rx=0 tx=0).stop 2024-09-17T02:51:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.094+0000 7f98c51b7640 1 -- 172.21.15.131:0/1784649136 shutdown_connections 2024-09-17T02:51:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.094+0000 7f98c51b7640 1 --2- 172.21.15.131:0/1784649136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98c00cdbf0 0x7f98c00ce000 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.094+0000 7f98c51b7640 1 --2- 172.21.15.131:0/1784649136 >> v2:172.21.15.189:3300/0 conn(0x7f98c00c4ac0 0x7f98c00c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.094+0000 7f98c51b7640 1 --2- 172.21.15.131:0/1784649136 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98c006e2c0 0x7f98c00c4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.094+0000 7f98c51b7640 1 -- 172.21.15.131:0/1784649136 >> 172.21.15.131:0/1784649136 conn(0x7f98c006eea0 msgr2=0x7f98c006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.094+0000 7f98c51b7640 1 -- 172.21.15.131:0/1784649136 shutdown_connections 2024-09-17T02:51:47.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.094+0000 7f98c51b7640 1 -- 172.21.15.131:0/1784649136 wait complete. 2024-09-17T02:51:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.094+0000 7f98c51b7640 1 Processor -- start 2024-09-17T02:51:47.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.094+0000 7f98c51b7640 1 -- start start 2024-09-17T02:51:47.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98c51b7640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98c006e2c0 0x7f98c015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:47.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98c51b7640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98c00c4ac0 0x7f98c0166a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:47.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98c51b7640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98c00cdbf0 0x7f98c01fc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:47.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98c51b7640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f98c00ceed0 con 0x7f98c006e2c0 2024-09-17T02:51:47.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98c51b7640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f98c00ced50 con 0x7f98c00cdbf0 2024-09-17T02:51:47.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98c51b7640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f98c00cf050 con 0x7f98c00c4ac0 2024-09-17T02:51:47.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98bed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98c006e2c0 0x7f98c015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:47.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98bed9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98c006e2c0 0x7f98c015ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37366/0 (socket says 172.21.15.131:37366) 2024-09-17T02:51:47.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98bed9d640 1 -- 172.21.15.131:0/264420566 learned_addr learned my addr 172.21.15.131:0/264420566 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:47.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98bf59e640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98c00cdbf0 0x7f98c01fc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:47.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98be59c640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98c00c4ac0 0x7f98c0166a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98bed9d640 1 -- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98c00c4ac0 msgr2=0x7f98c0166a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98bed9d640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98c00c4ac0 0x7f98c0166a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98bed9d640 1 -- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98c00cdbf0 msgr2=0x7f98c01fc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98bed9d640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98c00cdbf0 0x7f98c01fc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98bed9d640 1 -- 172.21.15.131:0/264420566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c015ffc0 con 0x7f98c006e2c0 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98bf59e640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98c00cdbf0 0x7f98c01fc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98be59c640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98c00c4ac0 0x7f98c0166a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98bed9d640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98c006e2c0 0x7f98c015ce10 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f98b800c940 tx=0x7f98b800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98a77fe640 1 -- 172.21.15.131:0/264420566 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8007bb0 con 0x7f98c006e2c0 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98a77fe640 1 -- 172.21.15.131:0/264420566 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98b8007e40 con 0x7f98c006e2c0 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98c51b7640 1 -- 172.21.15.131:0/264420566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c015aa50 con 0x7f98c006e2c0 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98a77fe640 1 -- 172.21.15.131:0/264420566 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b800f690 con 0x7f98c006e2c0 2024-09-17T02:51:47.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98c51b7640 1 -- 172.21.15.131:0/264420566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c015b010 con 0x7f98c006e2c0 2024-09-17T02:51:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98c51b7640 1 -- 172.21.15.131:0/264420566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c00c4ed0 con 0x7f98c006e2c0 2024-09-17T02:51:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98a77fe640 1 -- 172.21.15.131:0/264420566 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f98b800f830 con 0x7f98c006e2c0 2024-09-17T02:51:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98a77fe640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f98ac073200 0x7f98ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.098+0000 7f98a77fe640 1 -- 172.21.15.131:0/264420566 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f98b80948d0 con 0x7f98c006e2c0 2024-09-17T02:51:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.102+0000 7f98be59c640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f98ac073200 0x7f98ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:47.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.102+0000 7f98be59c640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f98ac073200 0x7f98ac075710 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f98c00b4870 tx=0x7f98b40082d0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:47.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.106+0000 7f98a77fe640 1 -- 172.21.15.131:0/264420566 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f98b8014030 con 0x7f98c006e2c0 2024-09-17T02:51:47.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.274+0000 7f98c51b7640 1 -- 172.21.15.131:0/264420566 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98c0073310 con 0x7f98c006e2c0 2024-09-17T02:51:47.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.274+0000 7f98a77fe640 1 -- 172.21.15.131:0/264420566 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7f98b805fa10 con 0x7f98c006e2c0 2024-09-17T02:51:47.268 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-09-17T02:51:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 -- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f98ac073200 msgr2=0x7f98ac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f98ac073200 0x7f98ac075710 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f98c00b4870 tx=0x7f98b40082d0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 -- 172.21.15.131:0/264420566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98c006e2c0 msgr2=0x7f98c015ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98c006e2c0 0x7f98c015ce10 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f98b800c940 tx=0x7f98b800ce00 comp rx=0 tx=0).stop 2024-09-17T02:51:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 -- 172.21.15.131:0/264420566 shutdown_connections 2024-09-17T02:51:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f98ac073200 0x7f98ac075710 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f98c00cdbf0 0x7f98c01fc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f98c00c4ac0 0x7f98c0166a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 --2- 172.21.15.131:0/264420566 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f98c006e2c0 0x7f98c015ce10 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 -- 172.21.15.131:0/264420566 >> 172.21.15.131:0/264420566 conn(0x7f98c006eea0 msgr2=0x7f98c0158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 -- 172.21.15.131:0/264420566 shutdown_connections 2024-09-17T02:51:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.278+0000 7f98c51b7640 1 -- 172.21.15.131:0/264420566 wait complete. 2024-09-17T02:51:47.292 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-09-17T02:51:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.458+0000 7f06f362c640 1 -- 172.21.15.131:0/4190599979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06d8003a60 msgr2=0x7f06d8003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.458+0000 7f06f362c640 1 --2- 172.21.15.131:0/4190599979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06d8003a60 0x7f06d8003ed0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009b30 tx=0x7f06e801d230 comp rx=0 tx=0).stop 2024-09-17T02:51:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.458+0000 7f06f362c640 1 -- 172.21.15.131:0/4190599979 shutdown_connections 2024-09-17T02:51:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.458+0000 7f06f362c640 1 --2- 172.21.15.131:0/4190599979 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06d8003a60 0x7f06d8003ed0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.458+0000 7f06f362c640 1 -- 172.21.15.131:0/4190599979 >> 172.21.15.131:0/4190599979 conn(0x7f06ec0d1b10 msgr2=0x7f06ec0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.458+0000 7f06f362c640 1 -- 172.21.15.131:0/4190599979 shutdown_connections 2024-09-17T02:51:47.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.458+0000 7f06f362c640 1 -- 172.21.15.131:0/4190599979 wait complete. 2024-09-17T02:51:47.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.458+0000 7f06f362c640 1 Processor -- start 2024-09-17T02:51:47.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.458+0000 7f06f362c640 1 -- start start 2024-09-17T02:51:47.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f362c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06d8003a60 0x7f06ec0aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f362c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f06ec0ce170 0x7f06ec0acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f362c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ec1ece90 0x7f06ec1ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f362c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec162c20 con 0x7f06ec1ece90 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f362c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f06ec15fdf0 con 0x7f06d8003a60 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f362c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f06ec162da0 con 0x7f06ec0ce170 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f1bc9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ec1ece90 0x7f06ec1ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f1bc9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ec1ece90 0x7f06ec1ed2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37392/0 (socket says 172.21.15.131:37392) 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f1bc9640 1 -- 172.21.15.131:0/4122773020 learned_addr learned my addr 172.21.15.131:0/4122773020 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f0bc7640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f06ec0ce170 0x7f06ec0acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f13c8640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06d8003a60 0x7f06ec0aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f1bc9640 1 -- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f06ec0ce170 msgr2=0x7f06ec0acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:47.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f1bc9640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f06ec0ce170 0x7f06ec0acaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f1bc9640 1 -- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06d8003a60 msgr2=0x7f06ec0aac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:47.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f1bc9640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06d8003a60 0x7f06ec0aac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:47.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f1bc9640 1 -- 172.21.15.131:0/4122773020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06ec1ec0d0 con 0x7f06ec1ece90 2024-09-17T02:51:47.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f1bc9640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ec1ece90 0x7f06ec1ed2a0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0b72b0 tx=0x7f06e400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:47.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06e1ffb640 1 -- 172.21.15.131:0/4122773020 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4018070 con 0x7f06ec1ece90 2024-09-17T02:51:47.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06e1ffb640 1 -- 172.21.15.131:0/4122773020 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06e4003980 con 0x7f06ec1ece90 2024-09-17T02:51:47.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f362c640 1 -- 172.21.15.131:0/4122773020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06ec1ec3b0 con 0x7f06ec1ece90 2024-09-17T02:51:47.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06e1ffb640 1 -- 172.21.15.131:0/4122773020 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4003db0 con 0x7f06ec1ece90 2024-09-17T02:51:47.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f362c640 1 -- 172.21.15.131:0/4122773020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06ec1ebde0 con 0x7f06ec1ece90 2024-09-17T02:51:47.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06f362c640 1 -- 172.21.15.131:0/4122773020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06ec1ed7e0 con 0x7f06ec1ece90 2024-09-17T02:51:47.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.462+0000 7f06e1ffb640 1 -- 172.21.15.131:0/4122773020 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f06e401b070 con 0x7f06ec1ece90 2024-09-17T02:51:47.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.466+0000 7f06e1ffb640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f06d0073200 0x7f06d0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:47.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.466+0000 7f06e1ffb640 1 -- 172.21.15.131:0/4122773020 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f06e4093a20 con 0x7f06ec1ece90 2024-09-17T02:51:47.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.466+0000 7f06f13c8640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f06d0073200 0x7f06d0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:47.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.466+0000 7f06f13c8640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f06d0073200 0x7f06d0075710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f06e80ae000 tx=0x7f06e8019fd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:47.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.470+0000 7f06e1ffb640 1 -- 172.21.15.131:0/4122773020 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f06e4022020 con 0x7f06ec1ece90 2024-09-17T02:51:47.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:47.702+0000 7f06f362c640 1 -- 172.21.15.131:0/4122773020 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f06ec0b8ab0 con 0x7f06ec1ece90 2024-09-17T02:51:48.558 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.566+0000 7f06e1ffb640 1 -- 172.21.15.131:0/4122773020 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v489) ==== 147+0+0 (secure 0 0 0) 0x7f06e405ec40 con 0x7f06ec1ece90 2024-09-17T02:51:48.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 -- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f06d0073200 msgr2=0x7f06d0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:48.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f06d0073200 0x7f06d0075710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f06e80ae000 tx=0x7f06e8019fd0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 -- 172.21.15.131:0/4122773020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ec1ece90 msgr2=0x7f06ec1ed2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:48.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ec1ece90 0x7f06ec1ed2a0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f06ec0b72b0 tx=0x7f06e400dc10 comp rx=0 tx=0).stop 2024-09-17T02:51:48.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 -- 172.21.15.131:0/4122773020 shutdown_connections 2024-09-17T02:51:48.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f06d0073200 0x7f06d0075710 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f06ec1ece90 0x7f06ec1ed2a0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f06ec0ce170 0x7f06ec0acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 --2- 172.21.15.131:0/4122773020 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f06d8003a60 0x7f06ec0aac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 -- 172.21.15.131:0/4122773020 >> 172.21.15.131:0/4122773020 conn(0x7f06ec0d1b10 msgr2=0x7f06ec1586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:48.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 -- 172.21.15.131:0/4122773020 shutdown_connections 2024-09-17T02:51:48.563 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.570+0000 7f06f362c640 1 -- 172.21.15.131:0/4122773020 wait complete. 2024-09-17T02:51:48.589 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T02:51:48.747 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.754+0000 7f7eadfc4640 1 -- 172.21.15.131:0/1582430811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ea806f6c0 msgr2=0x7f7ea815b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.754+0000 7f7eadfc4640 1 --2- 172.21.15.131:0/1582430811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ea806f6c0 0x7f7ea815b870 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f7e90009840 tx=0x7f7e9001c790 comp rx=0 tx=0).stop 2024-09-17T02:51:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.754+0000 7f7eadfc4640 1 -- 172.21.15.131:0/1582430811 shutdown_connections 2024-09-17T02:51:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.754+0000 7f7eadfc4640 1 --2- 172.21.15.131:0/1582430811 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ea806f6c0 0x7f7ea815b870 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.754+0000 7f7eadfc4640 1 -- 172.21.15.131:0/1582430811 >> 172.21.15.131:0/1582430811 conn(0x7f7ea8073d10 msgr2=0x7f7ea8074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.754+0000 7f7eadfc4640 1 -- 172.21.15.131:0/1582430811 shutdown_connections 2024-09-17T02:51:48.748 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.754+0000 7f7eadfc4640 1 -- 172.21.15.131:0/1582430811 wait complete. 2024-09-17T02:51:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.754+0000 7f7eadfc4640 1 Processor -- start 2024-09-17T02:51:48.749 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.754+0000 7f7eadfc4640 1 -- start start 2024-09-17T02:51:48.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7eadfc4640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ea80cd470 0x7f7ea814fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:48.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7eadfc4640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ea815bdb0 0x7f7ea80afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:48.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7eadfc4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ea8148750 0x7f7ea81fc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:48.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7eadfc4640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea816a190 con 0x7f7ea8148750 2024-09-17T02:51:48.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7eadfc4640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea816a010 con 0x7f7ea80cd470 2024-09-17T02:51:48.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7eadfc4640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea816a310 con 0x7f7ea815bdb0 2024-09-17T02:51:48.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ea8148750 0x7f7ea81fc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:48.752 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ea8148750 0x7f7ea81fc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37412/0 (socket says 172.21.15.131:37412) 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea7fff640 1 -- 172.21.15.131:0/565584480 learned_addr learned my addr 172.21.15.131:0/565584480 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7e9ffff640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ea815bdb0 0x7f7ea80afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea77fe640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ea80cd470 0x7f7ea814fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea7fff640 1 -- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ea815bdb0 msgr2=0x7f7ea80afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea7fff640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ea815bdb0 0x7f7ea80afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea7fff640 1 -- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ea80cd470 msgr2=0x7f7ea814fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea7fff640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ea80cd470 0x7f7ea814fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea7fff640 1 -- 172.21.15.131:0/565584480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea80adc60 con 0x7f7ea8148750 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea7fff640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ea8148750 0x7f7ea81fc320 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000cae0 tx=0x7f7ea000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea4ff9640 1 -- 172.21.15.131:0/565584480 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0016020 con 0x7f7ea8148750 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea4ff9640 1 -- 172.21.15.131:0/565584480 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ea000ab40 con 0x7f7ea8148750 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7eadfc4640 1 -- 172.21.15.131:0/565584480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea80adf40 con 0x7f7ea8148750 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea4ff9640 1 -- 172.21.15.131:0/565584480 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000ce20 con 0x7f7ea8148750 2024-09-17T02:51:48.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7eadfc4640 1 -- 172.21.15.131:0/565584480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea80b0410 con 0x7f7ea8148750 2024-09-17T02:51:48.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7eadfc4640 1 -- 172.21.15.131:0/565584480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea815b870 con 0x7f7ea8148750 2024-09-17T02:51:48.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.758+0000 7f7ea4ff9640 1 -- 172.21.15.131:0/565584480 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7ea0019020 con 0x7f7ea8148750 2024-09-17T02:51:48.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.762+0000 7f7ea4ff9640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e88073200 0x7f7e88075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:48.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.762+0000 7f7ea4ff9640 1 -- 172.21.15.131:0/565584480 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f7ea00948c0 con 0x7f7ea8148750 2024-09-17T02:51:48.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.762+0000 7f7ea77fe640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e88073200 0x7f7e88075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:48.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.762+0000 7f7ea77fe640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e88073200 0x7f7e88075710 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800b810 tx=0x7f7e98008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:48.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.766+0000 7f7ea4ff9640 1 -- 172.21.15.131:0/565584480 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7ea0020030 con 0x7f7ea8148750 2024-09-17T02:51:48.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 -- 172.21.15.131:0/565584480 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f7ea80d0d80 con 0x7f7ea8148750 2024-09-17T02:51:48.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7ea4ff9640 1 -- 172.21.15.131:0/565584480 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v43) ==== 59+0+2294 (secure 0 0 0) 0x7f7ea005fa70 con 0x7f7ea8148750 2024-09-17T02:51:48.923 INFO:teuthology.orchestra.run.smithi131.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T02:51:48.923 INFO:teuthology.orchestra.run.smithi131.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T02:51:48.923 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced client_mount_timeout 30 2024-09-17T02:51:48.923 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_client 20/20 2024-09-17T02:51:48.923 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_mgr 20/20 2024-09-17T02:51:48.923 INFO:teuthology.orchestra.run.smithi131.stdout:mgr advanced debug_ms 1/1 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3855.598857 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3598.912856 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3977.024117 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3823.557728 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4643.710284 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4452.598518 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 6659.946233 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 4539.778076 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds 20/20 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T02:51:48.924 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced debug_ms 2/2 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_frag true 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_debug_scatterstat true 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:mds dev mds_verify_scatter true 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced client_mount_timeout 600 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_client 20/20 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced debug_ms 1/1 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T02:51:48.925 INFO:teuthology.orchestra.run.smithi131.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T02:51:48.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 -- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e88073200 msgr2=0x7f7e88075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:48.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e88073200 0x7f7e88075710 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800b810 tx=0x7f7e98008040 comp rx=0 tx=0).stop 2024-09-17T02:51:48.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 -- 172.21.15.131:0/565584480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ea8148750 msgr2=0x7f7ea81fc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:48.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ea8148750 0x7f7ea81fc320 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f7ea000cae0 tx=0x7f7ea000cbd0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 -- 172.21.15.131:0/565584480 shutdown_connections 2024-09-17T02:51:48.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7e88073200 0x7f7e88075710 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ea8148750 0x7f7ea81fc320 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ea815bdb0 0x7f7ea80afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 --2- 172.21.15.131:0/565584480 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ea80cd470 0x7f7ea814fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:48.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 -- 172.21.15.131:0/565584480 >> 172.21.15.131:0/565584480 conn(0x7f7ea8073d10 msgr2=0x7f7ea806f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:48.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 -- 172.21.15.131:0/565584480 shutdown_connections 2024-09-17T02:51:48.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:48.930+0000 7f7eadfc4640 1 -- 172.21.15.131:0/565584480 wait complete. 2024-09-17T02:51:48.944 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T02:51:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.110+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1282730755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac155240 msgr2=0x7fc7ac073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.110+0000 7fc7b3c28640 1 --2- 172.21.15.131:0/1282730755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac155240 0x7fc7ac073430 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009900 tx=0x7fc79c01c8b0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.110+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1282730755 shutdown_connections 2024-09-17T02:51:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.110+0000 7fc7b3c28640 1 --2- 172.21.15.131:0/1282730755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac155240 0x7fc7ac073430 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.110+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1282730755 >> 172.21.15.131:0/1282730755 conn(0x7fc7ac06eea0 msgr2=0x7fc7ac06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.110+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1282730755 shutdown_connections 2024-09-17T02:51:49.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.110+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1282730755 wait complete. 2024-09-17T02:51:49.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b3c28640 1 Processor -- start 2024-09-17T02:51:49.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b3c28640 1 -- start start 2024-09-17T02:51:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b3c28640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac073970 0x7fc7ac1fa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b3c28640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7ac155240 0x7fc7ac14edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b3c28640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7ac15b2e0 0x7fc7ac153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b3c28640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac16ada0 con 0x7fc7ac073970 2024-09-17T02:51:49.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b3c28640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ac0cb920 con 0x7fc7ac155240 2024-09-17T02:51:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b3c28640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ac16af20 con 0x7fc7ac15b2e0 2024-09-17T02:51:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b19c4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac073970 0x7fc7ac1fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b19c4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac073970 0x7fc7ac1fa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37442/0 (socket says 172.21.15.131:37442) 2024-09-17T02:51:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b19c4640 1 -- 172.21.15.131:0/1954301582 learned_addr learned my addr 172.21.15.131:0/1954301582 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b21c5640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7ac15b2e0 0x7fc7ac153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b11c3640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7ac155240 0x7fc7ac14edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b19c4640 1 -- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7ac15b2e0 msgr2=0x7fc7ac153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b19c4640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7ac15b2e0 0x7fc7ac153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b19c4640 1 -- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7ac155240 msgr2=0x7fc7ac14edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b19c4640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7ac155240 0x7fc7ac14edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b19c4640 1 -- 172.21.15.131:0/1954301582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ac1fab40 con 0x7fc7ac073970 2024-09-17T02:51:49.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b19c4640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac073970 0x7fc7ac1fa600 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800c400 tx=0x7fc7a800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:49.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1954301582 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8017070 con 0x7fc7ac073970 2024-09-17T02:51:49.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1954301582 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7a8003600 con 0x7fc7ac073970 2024-09-17T02:51:49.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1954301582 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a8003a50 con 0x7fc7ac073970 2024-09-17T02:51:49.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1954301582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ac14f330 con 0x7fc7ac073970 2024-09-17T02:51:49.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.114+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1954301582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ac14fe00 con 0x7fc7ac073970 2024-09-17T02:51:49.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.118+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1954301582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ac150800 con 0x7fc7ac073970 2024-09-17T02:51:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.118+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1954301582 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc7a801a070 con 0x7fc7ac073970 2024-09-17T02:51:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.118+0000 7fc7a27fc640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073230 0x7fc790075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.118+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1954301582 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7fc7a8093870 con 0x7fc7ac073970 2024-09-17T02:51:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.118+0000 7fc7b11c3640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073230 0x7fc790075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:49.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.118+0000 7fc7b11c3640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073230 0x7fc790075740 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0af000 tx=0x7fc79c01b0f0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:49.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.122+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1954301582 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc7a8021020 con 0x7fc7ac073970 2024-09-17T02:51:49.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.290+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1954301582 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc7ac073430 con 0x7fc7ac073970 2024-09-17T02:51:49.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7a27fc640 1 -- 172.21.15.131:0/1954301582 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7fc7ac1fab40 con 0x7fc7ac073970 2024-09-17T02:51:49.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073230 msgr2=0x7fc790075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:49.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073230 0x7fc790075740 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc79c0af000 tx=0x7fc79c01b0f0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1954301582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac073970 msgr2=0x7fc7ac1fa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac073970 0x7fc7ac1fa600 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc7a800c400 tx=0x7fc7a800c8c0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1954301582 shutdown_connections 2024-09-17T02:51:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc790073230 0x7fc790075740 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc7ac15b2e0 0x7fc7ac153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc7ac155240 0x7fc7ac14edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 --2- 172.21.15.131:0/1954301582 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc7ac073970 0x7fc7ac1fa600 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1954301582 >> 172.21.15.131:0/1954301582 conn(0x7fc7ac06eea0 msgr2=0x7fc7ac151d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1954301582 shutdown_connections 2024-09-17T02:51:49.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.298+0000 7fc7b3c28640 1 -- 172.21.15.131:0/1954301582 wait complete. 2024-09-17T02:51:49.312 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:51:49.312 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:51:49.383 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:51:49.453 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:51:49.454 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:51:49.454 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:51:49.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:51:49.460 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:51:49.460 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:51:49.466 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:51:49.466 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:51:49.466 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:51:49.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:51:49.506 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:51:49.507 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:51:49.513 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:51:49.513 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T02:51:49.514 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:51:49.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.714+0000 7f09faccd640 1 -- 172.21.15.131:0/3988357687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09f406e2c0 msgr2=0x7f09f40c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:49.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.714+0000 7f09faccd640 1 --2- 172.21.15.131:0/3988357687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09f406e2c0 0x7f09f40c4580 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f09e4009900 tx=0x7f09e401c850 comp rx=0 tx=0).stop 2024-09-17T02:51:49.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.714+0000 7f09faccd640 1 -- 172.21.15.131:0/3988357687 shutdown_connections 2024-09-17T02:51:49.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.714+0000 7f09faccd640 1 --2- 172.21.15.131:0/3988357687 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f09f40cec00 0x7f09f40cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.714+0000 7f09faccd640 1 --2- 172.21.15.131:0/3988357687 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09f406e2c0 0x7f09f40c4580 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.714+0000 7f09faccd640 1 -- 172.21.15.131:0/3988357687 >> 172.21.15.131:0/3988357687 conn(0x7f09f406eea0 msgr2=0x7f09f406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:49.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.714+0000 7f09faccd640 1 -- 172.21.15.131:0/3988357687 shutdown_connections 2024-09-17T02:51:49.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.714+0000 7f09faccd640 1 -- 172.21.15.131:0/3988357687 wait complete. 2024-09-17T02:51:49.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09faccd640 1 Processor -- start 2024-09-17T02:51:49.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09faccd640 1 -- start start 2024-09-17T02:51:49.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09faccd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09f406e2c0 0x7f09f415fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:49.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09faccd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f09f40c4ac0 0x7f09f415d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09faccd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f40cec00 0x7f09f40b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09faccd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f09f4169c60 con 0x7f09f406e2c0 2024-09-17T02:51:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09faccd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f09f4169ae0 con 0x7f09f40cec00 2024-09-17T02:51:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09faccd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f09f4169de0 con 0x7f09f40c4ac0 2024-09-17T02:51:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f8a69640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09f406e2c0 0x7f09f415fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f8a69640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09f406e2c0 0x7f09f415fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37456/0 (socket says 172.21.15.131:37456) 2024-09-17T02:51:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f8a69640 1 -- 172.21.15.131:0/1820758017 learned_addr learned my addr 172.21.15.131:0/1820758017 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f926a640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f40cec00 0x7f09f40b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f3fff640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f09f40c4ac0 0x7f09f415d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f8a69640 1 -- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f09f40c4ac0 msgr2=0x7f09f415d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:49.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f8a69640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f09f40c4ac0 0x7f09f415d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f8a69640 1 -- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f40cec00 msgr2=0x7f09f40b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f8a69640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f40cec00 0x7f09f40b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f8a69640 1 -- 172.21.15.131:0/1820758017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f4148f20 con 0x7f09f406e2c0 2024-09-17T02:51:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f926a640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f40cec00 0x7f09f40b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f3fff640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f09f40c4ac0 0x7f09f415d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f8a69640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09f406e2c0 0x7f09f415fc70 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f09e40b0040 tx=0x7f09e4005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f17fa640 1 -- 172.21.15.131:0/1820758017 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e40ba070 con 0x7f09f406e2c0 2024-09-17T02:51:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f17fa640 1 -- 172.21.15.131:0/1820758017 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09e4004330 con 0x7f09f406e2c0 2024-09-17T02:51:49.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09faccd640 1 -- 172.21.15.131:0/1820758017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f4149200 con 0x7f09f406e2c0 2024-09-17T02:51:49.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09f17fa640 1 -- 172.21.15.131:0/1820758017 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e40b2040 con 0x7f09f406e2c0 2024-09-17T02:51:49.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09faccd640 1 -- 172.21.15.131:0/1820758017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f41497c0 con 0x7f09f406e2c0 2024-09-17T02:51:49.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.718+0000 7f09faccd640 1 -- 172.21.15.131:0/1820758017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f40c4580 con 0x7f09f406e2c0 2024-09-17T02:51:49.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.722+0000 7f09f17fa640 1 -- 172.21.15.131:0/1820758017 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f09e4005280 con 0x7f09f406e2c0 2024-09-17T02:51:49.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.722+0000 7f09f17fa640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f09e0073230 0x7f09e0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:49.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.722+0000 7f09f17fa640 1 -- 172.21.15.131:0/1820758017 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f09e412d9d0 con 0x7f09f406e2c0 2024-09-17T02:51:49.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.722+0000 7f09f3fff640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f09e0073230 0x7f09e0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:49.717 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.722+0000 7f09f3fff640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f09e0073230 0x7f09e0075740 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f09f40c69a0 tx=0x7f09e8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:49.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.726+0000 7f09f17fa640 1 -- 172.21.15.131:0/1820758017 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f09e40164d0 con 0x7f09f406e2c0 2024-09-17T02:51:49.921 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.926+0000 7f09faccd640 1 -- 172.21.15.131:0/1820758017 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09f40c4820 con 0x7f09f406e2c0 2024-09-17T02:51:49.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09f17fa640 1 -- 172.21.15.131:0/1820758017 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+3923 (secure 0 0 0) 0x7f09e40f8b10 con 0x7f09f406e2c0 2024-09-17T02:51:49.922 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:49.922 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":96,"btime":"2024-09-17T02:51:43:515227+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5774,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/4144608586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":4144608586},{"type":"v1","addr":"172.21.15.189:6837","nonce":4144608586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":6373,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/2158132993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":2158132993},{"type":"v1","addr":"172.21.15.189:6835","nonce":2158132993}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":7857,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/1293669493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":1293669493},{"type":"v1","addr":"172.21.15.131:6834","nonce":1293669493}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":81},{"gid":8334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2338820924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":2338820924},{"type":"v1","addr":"172.21.15.131:6835","nonce":2338820924}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[]} 2024-09-17T02:51:49.922 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 96 2024-09-17T02:51:49.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 -- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f09e0073230 msgr2=0x7f09e0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:49.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f09e0073230 0x7f09e0075740 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f09f40c69a0 tx=0x7f09e8009040 comp rx=0 tx=0).stop 2024-09-17T02:51:49.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 -- 172.21.15.131:0/1820758017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09f406e2c0 msgr2=0x7f09f415fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:49.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09f406e2c0 0x7f09f415fc70 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f09e40b0040 tx=0x7f09e4005e70 comp rx=0 tx=0).stop 2024-09-17T02:51:49.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 -- 172.21.15.131:0/1820758017 shutdown_connections 2024-09-17T02:51:49.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f09e0073230 0x7f09e0075740 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f09f40cec00 0x7f09f40b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f09f40c4ac0 0x7f09f415d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 --2- 172.21.15.131:0/1820758017 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f09f406e2c0 0x7f09f415fc70 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:49.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 -- 172.21.15.131:0/1820758017 >> 172.21.15.131:0/1820758017 conn(0x7f09f406eea0 msgr2=0x7f09f40c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:49.925 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 -- 172.21.15.131:0/1820758017 shutdown_connections 2024-09-17T02:51:49.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:49.930+0000 7f09faccd640 1 -- 172.21.15.131:0/1820758017 wait complete. 2024-09-17T02:51:49.945 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T02:51:49.945 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T02:51:49.946 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T02:51:49.946 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:51:49.946 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T02:51:49.946 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T02:51:49.947 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T02:51:49.947 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:51:49.947 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T02:51:49.947 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T02:51:49.948 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T02:51:49.948 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:51:49.948 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T02:51:49.948 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T02:51:49.949 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T02:51:49.949 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:51:49.965 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T02:51:49.965 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T02:51:49.970 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T02:51:49.970 DEBUG:teuthology.orchestra.run.smithi189:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T02:51:49.972 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T02:51:49.972 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T02:51:49.973 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T02:51:49.973 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T02:51:50.015 INFO:tasks.ceph.mds.c:Started 2024-09-17T02:51:50.016 DEBUG:teuthology.parallel:result is None 2024-09-17T02:51:50.016 INFO:tasks.ceph.mds.a:Started 2024-09-17T02:51:50.017 DEBUG:teuthology.parallel:result is None 2024-09-17T02:51:50.017 INFO:tasks.ceph.mds.d:Started 2024-09-17T02:51:50.018 DEBUG:teuthology.parallel:result is None 2024-09-17T02:51:50.019 INFO:tasks.ceph.mds.b:Started 2024-09-17T02:51:50.019 DEBUG:teuthology.parallel:result is None 2024-09-17T02:51:50.019 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T02:51:50.103 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:51:50.103 INFO:tasks.ceph.mds.c.smithi131.stdout:starting mds.c at 2024-09-17T02:51:50.113 INFO:tasks.ceph.mds.b.smithi189.stdout:starting mds.b at 2024-09-17T02:51:50.117 INFO:tasks.ceph.mds.d.smithi189.stdout:starting mds.d at 2024-09-17T02:51:50.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 -- 172.21.15.131:0/969354210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ddc0d2b70 msgr2=0x7f3ddc0d2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:50.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 --2- 172.21.15.131:0/969354210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ddc0d2b70 0x7f3ddc0d2f80 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f3dd4009bc0 tx=0x7f3dd401cba0 comp rx=0 tx=0).stop 2024-09-17T02:51:50.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 -- 172.21.15.131:0/969354210 shutdown_connections 2024-09-17T02:51:50.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 --2- 172.21.15.131:0/969354210 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ddc0d2b70 0x7f3ddc0d2f80 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:50.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 -- 172.21.15.131:0/969354210 >> 172.21.15.131:0/969354210 conn(0x7f3ddc0d1b10 msgr2=0x7f3ddc0d43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:50.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 -- 172.21.15.131:0/969354210 shutdown_connections 2024-09-17T02:51:50.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 -- 172.21.15.131:0/969354210 wait complete. 2024-09-17T02:51:50.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 Processor -- start 2024-09-17T02:51:50.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 -- start start 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3ddc0cf710 0x7f3ddc0cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ddc0d2b70 0x7f3ddc14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ddc15f9a0 0x7f3ddc14e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc16d260 con 0x7f3ddc0d2b70 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc16d0e0 con 0x7f3ddc15f9a0 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3de2bf1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3ddc16d3e0 con 0x7f3ddc0cf710 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.170+0000 7f3ddbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ddc0d2b70 0x7f3ddc14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3ddbfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ddc0d2b70 0x7f3ddc14e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:37506/0 (socket says 172.21.15.131:37506) 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3ddbfff640 1 -- 172.21.15.131:0/4034959955 learned_addr learned my addr 172.21.15.131:0/4034959955 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3de118e640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ddc15f9a0 0x7f3ddc14e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3de098d640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3ddc0cf710 0x7f3ddc0cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:50.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3ddbfff640 1 -- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3ddc0cf710 msgr2=0x7f3ddc0cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3ddbfff640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3ddc0cf710 0x7f3ddc0cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3ddbfff640 1 -- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ddc15f9a0 msgr2=0x7f3ddc14e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3ddbfff640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ddc15f9a0 0x7f3ddc14e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3ddbfff640 1 -- 172.21.15.131:0/4034959955 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc1565a0 con 0x7f3ddc0d2b70 2024-09-17T02:51:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3de118e640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ddc15f9a0 0x7f3ddc14e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:51:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3ddbfff640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ddc0d2b70 0x7f3ddc14e2a0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c3f0 tx=0x7f3dc800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3dd97fa640 1 -- 172.21.15.131:0/4034959955 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8017070 con 0x7f3ddc0d2b70 2024-09-17T02:51:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3dd97fa640 1 -- 172.21.15.131:0/4034959955 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dc80053c0 con 0x7f3ddc0d2b70 2024-09-17T02:51:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3dd97fa640 1 -- 172.21.15.131:0/4034959955 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8003860 con 0x7f3ddc0d2b70 2024-09-17T02:51:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3de2bf1640 1 -- 172.21.15.131:0/4034959955 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc156880 con 0x7f3ddc0d2b70 2024-09-17T02:51:50.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3de2bf1640 1 -- 172.21.15.131:0/4034959955 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc0b4e40 con 0x7f3ddc0d2b70 2024-09-17T02:51:50.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3de2bf1640 1 -- 172.21.15.131:0/4034959955 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ddc0d2f80 con 0x7f3ddc0d2b70 2024-09-17T02:51:50.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3dd97fa640 1 -- 172.21.15.131:0/4034959955 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3dc801a070 con 0x7f3ddc0d2b70 2024-09-17T02:51:50.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3dd97fa640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3dc0073230 0x7f3dc0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:50.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3dd97fa640 1 -- 172.21.15.131:0/4034959955 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5421+0+0 (secure 0 0 0) 0x7f3dc8093410 con 0x7f3ddc0d2b70 2024-09-17T02:51:50.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3de098d640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3dc0073230 0x7f3dc0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:50.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.174+0000 7f3de098d640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3dc0073230 0x7f3dc0075740 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00046c0 tx=0x7f3dd0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:50.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.178+0000 7f3dd97fa640 1 -- 172.21.15.131:0/4034959955 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3dc805e550 con 0x7f3ddc0d2b70 2024-09-17T02:51:50.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:50.350+0000 7f3de2bf1640 1 -- 172.21.15.131:0/4034959955 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3ddc0bb590 con 0x7f3ddc0d2b70 2024-09-17T02:51:51.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.322+0000 7f3dd97fa640 1 -- 172.21.15.131:0/4034959955 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v118) ==== 97+0+0 (secure 0 0 0) 0x7f3dc805de50 con 0x7f3ddc0d2b70 2024-09-17T02:51:51.318 INFO:teuthology.orchestra.run.smithi131.stderr: removed all blocklist entries 2024-09-17T02:51:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.326+0000 7f3de2bf1640 1 -- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3dc0073230 msgr2=0x7f3dc0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.326+0000 7f3de2bf1640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3dc0073230 0x7f3dc0075740 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3dd00046c0 tx=0x7f3dd0009040 comp rx=0 tx=0).stop 2024-09-17T02:51:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.326+0000 7f3de2bf1640 1 -- 172.21.15.131:0/4034959955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ddc0d2b70 msgr2=0x7f3ddc14e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.326+0000 7f3de2bf1640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ddc0d2b70 0x7f3ddc14e2a0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c3f0 tx=0x7f3dc800c8b0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.326+0000 7f3de2bf1640 1 -- 172.21.15.131:0/4034959955 shutdown_connections 2024-09-17T02:51:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.326+0000 7f3de2bf1640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3dc0073230 0x7f3dc0075740 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.326+0000 7f3de2bf1640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ddc15f9a0 0x7f3ddc14e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.326+0000 7f3de2bf1640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3ddc0d2b70 0x7f3ddc14e2a0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.326+0000 7f3de2bf1640 1 --2- 172.21.15.131:0/4034959955 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3ddc0cf710 0x7f3ddc0cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.326+0000 7f3de2bf1640 1 -- 172.21.15.131:0/4034959955 >> 172.21.15.131:0/4034959955 conn(0x7f3ddc0d1b10 msgr2=0x7f3ddc0d23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:51.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.330+0000 7f3de2bf1640 1 -- 172.21.15.131:0/4034959955 shutdown_connections 2024-09-17T02:51:51.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.330+0000 7f3de2bf1640 1 -- 172.21.15.131:0/4034959955 wait complete. 2024-09-17T02:51:51.353 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T02:51:51.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.518+0000 7f1067a01640 1 -- 172.21.15.131:0/2352045633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10600d07f0 msgr2=0x7f106006e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:51.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.518+0000 7f1067a01640 1 --2- 172.21.15.131:0/2352045633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10600d07f0 0x7f106006e3b0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f106006f9e0 tx=0x7f105001c8c0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.518+0000 7f1067a01640 1 -- 172.21.15.131:0/2352045633 shutdown_connections 2024-09-17T02:51:51.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.518+0000 7f1067a01640 1 --2- 172.21.15.131:0/2352045633 >> v2:172.21.15.189:3300/0 conn(0x7f10600b1bd0 0x7f10600b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.518+0000 7f1067a01640 1 --2- 172.21.15.131:0/2352045633 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10600d07f0 0x7f106006e3b0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.518+0000 7f1067a01640 1 -- 172.21.15.131:0/2352045633 >> 172.21.15.131:0/2352045633 conn(0x7f10600b42f0 msgr2=0x7f10600d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:51.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.518+0000 7f1067a01640 1 -- 172.21.15.131:0/2352045633 shutdown_connections 2024-09-17T02:51:51.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.518+0000 7f1067a01640 1 -- 172.21.15.131:0/2352045633 wait complete. 2024-09-17T02:51:51.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1067a01640 1 Processor -- start 2024-09-17T02:51:51.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1067a01640 1 -- start start 2024-09-17T02:51:51.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1067a01640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f106006dfa0 0x7f106014d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:51.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1067a01640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10600b1bd0 0x7f10600b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:51.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1067a01640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f10600ccf20 0x7f10600cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:51.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1067a01640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f106016c2d0 con 0x7f10600b1bd0 2024-09-17T02:51:51.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1067a01640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f106016c150 con 0x7f106006dfa0 2024-09-17T02:51:51.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1067a01640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f106016c450 con 0x7f10600ccf20 2024-09-17T02:51:51.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1064f9c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10600b1bd0 0x7f10600b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:51.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1064f9c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10600b1bd0 0x7f10600b6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41716/0 (socket says 172.21.15.131:41716) 2024-09-17T02:51:51.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1064f9c640 1 -- 172.21.15.131:0/3879974554 learned_addr learned my addr 172.21.15.131:0/3879974554 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:51.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f106579d640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f106006dfa0 0x7f106014d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1064f9c640 1 -- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f10600ccf20 msgr2=0x7f10600cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1064f9c640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f10600ccf20 0x7f10600cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1064f9c640 1 -- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f106006dfa0 msgr2=0x7f106014d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1064f9c640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f106006dfa0 0x7f106014d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1064f9c640 1 -- 172.21.15.131:0/3879974554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1060150260 con 0x7f10600b1bd0 2024-09-17T02:51:51.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1064f9c640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10600b1bd0 0x7f10600b6810 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f10500194d0 tx=0x7f10500a5580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:51.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1055ffb640 1 -- 172.21.15.131:0/3879974554 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10500bc070 con 0x7f10600b1bd0 2024-09-17T02:51:51.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1055ffb640 1 -- 172.21.15.131:0/3879974554 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10500043b0 con 0x7f10600b1bd0 2024-09-17T02:51:51.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1067a01640 1 -- 172.21.15.131:0/3879974554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10600cd870 con 0x7f10600b1bd0 2024-09-17T02:51:51.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1055ffb640 1 -- 172.21.15.131:0/3879974554 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10500b2040 con 0x7f10600b1bd0 2024-09-17T02:51:51.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.522+0000 7f1067a01640 1 -- 172.21.15.131:0/3879974554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10601f9100 con 0x7f10600b1bd0 2024-09-17T02:51:51.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.526+0000 7f1067a01640 1 -- 172.21.15.131:0/3879974554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10600b7120 con 0x7f10600b1bd0 2024-09-17T02:51:51.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.526+0000 7f1055ffb640 1 -- 172.21.15.131:0/3879974554 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1050005290 con 0x7f10600b1bd0 2024-09-17T02:51:51.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.526+0000 7f1055ffb640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1044073200 0x7f1044075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:51.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.526+0000 7f1055ffb640 1 -- 172.21.15.131:0/3879974554 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5249+0+0 (secure 0 0 0) 0x7f105012d8e0 con 0x7f10600b1bd0 2024-09-17T02:51:51.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.526+0000 7f106579d640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1044073200 0x7f1044075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:51.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.526+0000 7f106579d640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1044073200 0x7f1044075710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f105c001650 tx=0x7f105c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:51.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.530+0000 7f1055ffb640 1 -- 172.21.15.131:0/3879974554 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f10600cd870 con 0x7f10600b1bd0 2024-09-17T02:51:51.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.746+0000 7f1067a01640 1 -- 172.21.15.131:0/3879974554 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f10600b1fe0 con 0x7f10600b1bd0 2024-09-17T02:51:51.741 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.746+0000 7f1055ffb640 1 -- 172.21.15.131:0/3879974554 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) ==== 80+0+5304 (secure 0 0 0) 0x7f10500164b0 con 0x7f10600b1bd0 2024-09-17T02:51:51.742 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:51.742 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:51:51.742 INFO:teuthology.orchestra.run.smithi131.stdout: "auth_dump": [ 2024-09-17T02:51:51.742 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.742 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.a", 2024-09-17T02:51:51.742 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm4U1cHhAATu86VrwT2vMjL/cW2SICMw==", 2024-09-17T02:51:51.742 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.742 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:51:51.742 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:51:51.742 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:51:51.742 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.b", 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmVpUSKxAA4l5zGHXg0VRasiyBI/j4ig==", 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.743 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.c", 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmwHuCJRAAlOj+za8ZmqMI55PsShjKDw==", 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mds.d", 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm9f+9MhAAS8z9jZFYon3nrmu9/4JMPg==", 2024-09-17T02:51:51.744 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.0", 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD16+hmV9xkBxAAT+IDf2J4awJfqYcx2xDoNA==", 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.745 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.1", 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD46+hmOhJsGBAAdlA4BSdOPk8BySUTfo1ghg==", 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.2", 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD76+hm3mSoKBAAQPoGFy816yWZQW6vPIK6bA==", 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.746 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.3", 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQD+6+hmpZaPNxAAp0EV6bn5SugDMwA4RnnJwg==", 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.747 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.4", 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAF7OhmgDfVAxAALACK3bk+PK3UhG20b498HQ==", 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.5", 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAI7Ohm6w44FBAA90GkelUAbiNjAwn+6IJWcg==", 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.748 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.6", 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAL7Ohm6S6TIxAArISJW//rmEONG2bC/sfkXA==", 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.749 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "osd.7", 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQAO7OhmICwTMxAArAr7DhcO8A0FDSu7VEBehw==", 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow profile osd", 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile osd", 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.0", 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA==", 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.750 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow r", 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.1", 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmduVoBRAAwr50jSXw6sCyo+5t+3iT7A==", 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:51:51.751 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.2", 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmBBCDABAAMw9581QyaOxJk5fs35o+ng==", 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.752 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.3", 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDy6+hmgznRCRAA1Z8d32ADoSG+7RHtdPGmTQ==", 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow", 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow r", 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow rw", 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow rwx" 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "client.admin", 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmpUfvABAAzclxwjR4E+UxvtkbwVqDXw==", 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.753 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mgr": "allow *", 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow *", 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.x", 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmnWmJGBAA6fzewrm8ELqUAz/MczTOGA==", 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.754 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:51:51.755 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:51.755 INFO:teuthology.orchestra.run.smithi131.stdout: "entity": "mgr.y", 2024-09-17T02:51:51.755 INFO:teuthology.orchestra.run.smithi131.stdout: "key": "AQDx6+hmrTq3ExAAuR8+2bItlpHqqFzlvPxASA==", 2024-09-17T02:51:51.755 INFO:teuthology.orchestra.run.smithi131.stdout: "caps": { 2024-09-17T02:51:51.755 INFO:teuthology.orchestra.run.smithi131.stdout: "mds": "allow *", 2024-09-17T02:51:51.755 INFO:teuthology.orchestra.run.smithi131.stdout: "mon": "allow profile mgr", 2024-09-17T02:51:51.755 INFO:teuthology.orchestra.run.smithi131.stdout: "osd": "allow *" 2024-09-17T02:51:51.755 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.755 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:51.755 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-09-17T02:51:51.755 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:51:51.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 -- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1044073200 msgr2=0x7f1044075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:51.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1044073200 0x7f1044075710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f105c001650 tx=0x7f105c009040 comp rx=0 tx=0).stop 2024-09-17T02:51:51.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 -- 172.21.15.131:0/3879974554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10600b1bd0 msgr2=0x7f10600b6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:51.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10600b1bd0 0x7f10600b6810 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f10500194d0 tx=0x7f10500a5580 comp rx=0 tx=0).stop 2024-09-17T02:51:51.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 -- 172.21.15.131:0/3879974554 shutdown_connections 2024-09-17T02:51:51.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1044073200 0x7f1044075710 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f10600ccf20 0x7f10600cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f10600b1bd0 0x7f10600b6810 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 --2- 172.21.15.131:0/3879974554 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f106006dfa0 0x7f106014d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 -- 172.21.15.131:0/3879974554 >> 172.21.15.131:0/3879974554 conn(0x7f10600b42f0 msgr2=0x7f10600cfed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:51.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 -- 172.21.15.131:0/3879974554 shutdown_connections 2024-09-17T02:51:51.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.750+0000 7f1067a01640 1 -- 172.21.15.131:0/3879974554 wait complete. 2024-09-17T02:51:51.763 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T02:51:51.903 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.910+0000 7fcff2953640 1 -- 172.21.15.131:0/3546345427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfd4003a10 msgr2=0x7fcfd4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:51.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.910+0000 7fcff2953640 1 --2- 172.21.15.131:0/3546345427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfd4003a10 0x7fcfd4003e80 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00c310 tx=0x7fcfdc01cf60 comp rx=0 tx=0).stop 2024-09-17T02:51:51.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.910+0000 7fcff0ef0640 1 -- 172.21.15.131:0/3546345427 reap_dead start 2024-09-17T02:51:51.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.910+0000 7fcff2953640 1 -- 172.21.15.131:0/3546345427 shutdown_connections 2024-09-17T02:51:51.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.910+0000 7fcff2953640 1 -- 172.21.15.131:0/3546345427 >> 172.21.15.131:0/3546345427 conn(0x7fcfec06eea0 msgr2=0x7fcfec06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:51.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.910+0000 7fcff2953640 1 -- 172.21.15.131:0/3546345427 shutdown_connections 2024-09-17T02:51:51.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.910+0000 7fcff2953640 1 -- 172.21.15.131:0/3546345427 wait complete. 2024-09-17T02:51:51.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.910+0000 7fcff2953640 1 Processor -- start 2024-09-17T02:51:51.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.910+0000 7fcff2953640 1 -- start start 2024-09-17T02:51:51.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff2953640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfd4003a10 0x7fcfec152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:51.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff2953640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcfec1fc780 0x7fcfec0c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:51.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff2953640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfec158870 0x7fcfec1f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:51.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff2953640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec169880 con 0x7fcfec158870 2024-09-17T02:51:51.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff2953640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcfec169700 con 0x7fcfd4003a10 2024-09-17T02:51:51.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff2953640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcfec169a00 con 0x7fcfec1fc780 2024-09-17T02:51:51.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff0ef0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfec158870 0x7fcfec1f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:51.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff0ef0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfec158870 0x7fcfec1f8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41738/0 (socket says 172.21.15.131:41738) 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff0ef0640 1 -- 172.21.15.131:0/1394012934 learned_addr learned my addr 172.21.15.131:0/1394012934 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcfeb7fe640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcfec1fc780 0x7fcfec0c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcfebfff640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfd4003a10 0x7fcfec152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff0ef0640 1 -- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcfec1fc780 msgr2=0x7fcfec0c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff0ef0640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcfec1fc780 0x7fcfec0c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff0ef0640 1 -- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfd4003a10 msgr2=0x7fcfec152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff0ef0640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfd4003a10 0x7fcfec152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff0ef0640 1 -- 172.21.15.131:0/1394012934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfec152c60 con 0x7fcfec158870 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff0ef0640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfec158870 0x7fcfec1f8f60 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fcfec075290 tx=0x7fcfe400fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcfe8ff9640 1 -- 172.21.15.131:0/1394012934 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4003d90 con 0x7fcfec158870 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcfe8ff9640 1 -- 172.21.15.131:0/1394012934 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe4017070 con 0x7fcfec158870 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff2953640 1 -- 172.21.15.131:0/1394012934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfec157870 con 0x7fcfec158870 2024-09-17T02:51:51.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcfe8ff9640 1 -- 172.21.15.131:0/1394012934 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe400dd10 con 0x7fcfec158870 2024-09-17T02:51:51.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff2953640 1 -- 172.21.15.131:0/1394012934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfec157b50 con 0x7fcfec158870 2024-09-17T02:51:51.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcff2953640 1 -- 172.21.15.131:0/1394012934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfec0ce880 con 0x7fcfec158870 2024-09-17T02:51:51.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcfe8ff9640 1 -- 172.21.15.131:0/1394012934 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcfe4004030 con 0x7fcfec158870 2024-09-17T02:51:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcfe8ff9640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 0x7fcfd8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.914+0000 7fcfe8ff9640 1 -- 172.21.15.131:0/1394012934 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5249+0+0 (secure 0 0 0) 0x7fcfe4094800 con 0x7fcfec158870 2024-09-17T02:51:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.918+0000 7fcfebfff640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 0x7fcfd8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:51.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.918+0000 7fcfebfff640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 0x7fcfd8075710 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00c310 tx=0x7fcfdc005040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:51.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:51.922+0000 7fcfe8ff9640 1 -- 172.21.15.131:0/1394012934 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcfe405f420 con 0x7fcfec158870 2024-09-17T02:51:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.082+0000 7fcff2953640 1 -- 172.21.15.131:0/1394012934 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcfec1fccd0 con 0x7fcfec158870 2024-09-17T02:51:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.082+0000 7fcfe8ff9640 1 -- 172.21.15.131:0/1394012934 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v118) ==== 84+0+65 (secure 0 0 0) 0x7fcfe4063e50 con 0x7fcfec158870 2024-09-17T02:51:52.076 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:52.076 INFO:teuthology.orchestra.run.smithi131.stdout:[ 2024-09-17T02:51:52.076 INFO:teuthology.orchestra.run.smithi131.stdout: { 2024-09-17T02:51:52.076 INFO:teuthology.orchestra.run.smithi131.stdout: "poolnum": 1, 2024-09-17T02:51:52.076 INFO:teuthology.orchestra.run.smithi131.stdout: "poolname": ".mgr" 2024-09-17T02:51:52.076 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:51:52.076 INFO:teuthology.orchestra.run.smithi131.stdout:] 2024-09-17T02:51:52.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 -- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 msgr2=0x7fcfd8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:52.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 0x7fcfd8075710 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc00c310 tx=0x7fcfdc005040 comp rx=0 tx=0).stop 2024-09-17T02:51:52.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 -- 172.21.15.131:0/1394012934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfec158870 msgr2=0x7fcfec1f8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:52.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfec158870 0x7fcfec1f8f60 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fcfec075290 tx=0x7fcfe400fed0 comp rx=0 tx=0).stop 2024-09-17T02:51:52.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 -- 172.21.15.131:0/1394012934 shutdown_connections 2024-09-17T02:51:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcfd8073200 0x7fcfd8075710 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcfec158870 0x7fcfec1f8f60 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcfec1fc780 0x7fcfec0c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 --2- 172.21.15.131:0/1394012934 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcfd4003a10 0x7fcfec152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 -- 172.21.15.131:0/1394012934 >> 172.21.15.131:0/1394012934 conn(0x7fcfec06eea0 msgr2=0x7fcfec073240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 -- 172.21.15.131:0/1394012934 shutdown_connections 2024-09-17T02:51:52.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.086+0000 7fcff2953640 1 -- 172.21.15.131:0/1394012934 wait complete. 2024-09-17T02:51:52.098 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T02:51:52.098 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T02:51:52.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.254+0000 7fef95d44640 1 -- 172.21.15.131:0/3892385293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef9015aa60 msgr2=0x7fef900b4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:52.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.254+0000 7fef95d44640 1 --2- 172.21.15.131:0/3892385293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef9015aa60 0x7fef900b4420 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fef80009900 tx=0x7fef8001c820 comp rx=0 tx=0).stop 2024-09-17T02:51:52.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.254+0000 7fef95d44640 1 -- 172.21.15.131:0/3892385293 shutdown_connections 2024-09-17T02:51:52.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.254+0000 7fef95d44640 1 --2- 172.21.15.131:0/3892385293 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef900cd370 0x7fef900cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:52.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.254+0000 7fef95d44640 1 --2- 172.21.15.131:0/3892385293 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef9015aa60 0x7fef900b4420 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:52.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.254+0000 7fef95d44640 1 -- 172.21.15.131:0/3892385293 >> 172.21.15.131:0/3892385293 conn(0x7fef900d05a0 msgr2=0x7fef900d09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:52.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.254+0000 7fef95d44640 1 -- 172.21.15.131:0/3892385293 shutdown_connections 2024-09-17T02:51:52.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.254+0000 7fef95d44640 1 -- 172.21.15.131:0/3892385293 wait complete. 2024-09-17T02:51:52.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.254+0000 7fef95d44640 1 Processor -- start 2024-09-17T02:51:52.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.254+0000 7fef95d44640 1 -- start start 2024-09-17T02:51:52.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef95d44640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef9006f700 0x7fef9014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:52.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef95d44640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef900cd370 0x7fef901f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:52.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef95d44640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef9015aa60 0x7fef901f9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef95d44640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fef901679e0 con 0x7fef9006f700 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef95d44640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fef90167860 con 0x7fef9015aa60 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef95d44640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fef90167b60 con 0x7fef900cd370 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef9006f700 0x7fef9014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef9006f700 0x7fef9014cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41748/0 (socket says 172.21.15.131:41748) 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8f7fe640 1 -- 172.21.15.131:0/338567657 learned_addr learned my addr 172.21.15.131:0/338567657 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8f7fe640 1 -- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef900cd370 msgr2=0x7fef901f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8ffff640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef9015aa60 0x7fef901f9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef87fff640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef900cd370 0x7fef901f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8f7fe640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef900cd370 0x7fef901f8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8f7fe640 1 -- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef9015aa60 msgr2=0x7fef901f9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8f7fe640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef9015aa60 0x7fef901f9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:52.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8f7fe640 1 -- 172.21.15.131:0/338567657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef90155530 con 0x7fef9006f700 2024-09-17T02:51:52.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8ffff640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef9015aa60 0x7fef901f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:52.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8f7fe640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef9006f700 0x7fef9014cca0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fef80009850 tx=0x7fef800a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:52.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8cff9640 1 -- 172.21.15.131:0/338567657 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800bc070 con 0x7fef9006f700 2024-09-17T02:51:52.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8cff9640 1 -- 172.21.15.131:0/338567657 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef80004430 con 0x7fef9006f700 2024-09-17T02:51:52.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef95d44640 1 -- 172.21.15.131:0/338567657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef90155810 con 0x7fef9006f700 2024-09-17T02:51:52.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8cff9640 1 -- 172.21.15.131:0/338567657 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800b2040 con 0x7fef9006f700 2024-09-17T02:51:52.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef95d44640 1 -- 172.21.15.131:0/338567657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef901f9ba0 con 0x7fef9006f700 2024-09-17T02:51:52.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef95d44640 1 -- 172.21.15.131:0/338567657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef900cd780 con 0x7fef9006f700 2024-09-17T02:51:52.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8cff9640 1 -- 172.21.15.131:0/338567657 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fef800bf070 con 0x7fef9006f700 2024-09-17T02:51:52.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.258+0000 7fef8cff9640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef70073200 0x7fef70075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:52.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.262+0000 7fef8cff9640 1 -- 172.21.15.131:0/338567657 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5249+0+0 (secure 0 0 0) 0x7fef8012e280 con 0x7fef9006f700 2024-09-17T02:51:52.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.262+0000 7fef87fff640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef70073200 0x7fef70075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:52.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.262+0000 7fef87fff640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef70073200 0x7fef70075710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fef90164950 tx=0x7fef78009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:52.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.266+0000 7fef8cff9640 1 -- 172.21.15.131:0/338567657 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fef800164d0 con 0x7fef9006f700 2024-09-17T02:51:52.423 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:52.430+0000 7fef95d44640 1 -- 172.21.15.131:0/338567657 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fef90157b00 con 0x7fef9006f700 2024-09-17T02:51:53.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.330+0000 7fef8cff9640 1 -- 172.21.15.131:0/338567657 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v119) ==== 138+0+0 (secure 0 0 0) 0x7fef800f8ea0 con 0x7fef9006f700 2024-09-17T02:51:53.323 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' created 2024-09-17T02:51:53.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 -- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef70073200 msgr2=0x7fef70075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:53.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef70073200 0x7fef70075710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fef90164950 tx=0x7fef78009040 comp rx=0 tx=0).stop 2024-09-17T02:51:53.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 -- 172.21.15.131:0/338567657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef9006f700 msgr2=0x7fef9014cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:53.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef9006f700 0x7fef9014cca0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fef80009850 tx=0x7fef800a5770 comp rx=0 tx=0).stop 2024-09-17T02:51:53.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 -- 172.21.15.131:0/338567657 shutdown_connections 2024-09-17T02:51:53.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fef70073200 0x7fef70075710 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:53.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fef9015aa60 0x7fef901f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:53.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fef900cd370 0x7fef901f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:53.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 --2- 172.21.15.131:0/338567657 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fef9006f700 0x7fef9014cca0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:53.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 -- 172.21.15.131:0/338567657 >> 172.21.15.131:0/338567657 conn(0x7fef900d05a0 msgr2=0x7fef900730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:53.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 -- 172.21.15.131:0/338567657 shutdown_connections 2024-09-17T02:51:53.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.334+0000 7fef95d44640 1 -- 172.21.15.131:0/338567657 wait complete. 2024-09-17T02:51:53.358 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T02:51:53.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.534+0000 7f59c2fdf640 1 -- 172.21.15.131:0/2272294326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc151a00 msgr2=0x7f59bc06f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:53.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.534+0000 7f59c2fdf640 1 --2- 172.21.15.131:0/2272294326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc151a00 0x7f59bc06f740 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009c80 tx=0x7f59b401cda0 comp rx=0 tx=0).stop 2024-09-17T02:51:53.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.534+0000 7f59c2fdf640 1 -- 172.21.15.131:0/2272294326 shutdown_connections 2024-09-17T02:51:53.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.534+0000 7f59c2fdf640 1 --2- 172.21.15.131:0/2272294326 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc151a00 0x7f59bc06f740 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:53.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.534+0000 7f59c2fdf640 1 --2- 172.21.15.131:0/2272294326 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59bc0cf680 0x7f59bc15e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:53.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.534+0000 7f59c2fdf640 1 -- 172.21.15.131:0/2272294326 >> 172.21.15.131:0/2272294326 conn(0x7f59bc0d0490 msgr2=0x7f59bc06ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:53.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.534+0000 7f59c2fdf640 1 -- 172.21.15.131:0/2272294326 shutdown_connections 2024-09-17T02:51:53.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.534+0000 7f59c2fdf640 1 -- 172.21.15.131:0/2272294326 wait complete. 2024-09-17T02:51:53.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.534+0000 7f59c2fdf640 1 Processor -- start 2024-09-17T02:51:53.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.534+0000 7f59c2fdf640 1 -- start start 2024-09-17T02:51:53.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c2fdf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59bc0cf680 0x7f59bc1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:53.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c2fdf640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59bc151a00 0x7f59bc13ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:53.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c2fdf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc15eb10 0x7f59bc153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:53.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c2fdf640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc0cfca0 con 0x7f59bc15eb10 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c2fdf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc073950 con 0x7f59bc0cf680 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c2fdf640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f59bc06fc80 con 0x7f59bc151a00 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c157c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc15eb10 0x7f59bc153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c157c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc15eb10 0x7f59bc153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41764/0 (socket says 172.21.15.131:41764) 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c157c640 1 -- 172.21.15.131:0/3882575767 learned_addr learned my addr 172.21.15.131:0/3882575767 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c0d7b640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59bc0cf680 0x7f59bc1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c157c640 1 -- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59bc151a00 msgr2=0x7f59bc13ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59bbfff640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59bc151a00 0x7f59bc13ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c157c640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59bc151a00 0x7f59bc13ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c157c640 1 -- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59bc0cf680 msgr2=0x7f59bc1563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c157c640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59bc0cf680 0x7f59bc1563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c157c640 1 -- 172.21.15.131:0/3882575767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc14ed70 con 0x7f59bc15eb10 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c0d7b640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59bc0cf680 0x7f59bc1563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:53.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c157c640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc15eb10 0x7f59bc153130 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009bd0 tx=0x7f59b4005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:53.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59b97fa640 1 -- 172.21.15.131:0/3882575767 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40bc070 con 0x7f59bc15eb10 2024-09-17T02:51:53.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59b97fa640 1 -- 172.21.15.131:0/3882575767 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59b4004130 con 0x7f59bc15eb10 2024-09-17T02:51:53.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c2fdf640 1 -- 172.21.15.131:0/3882575767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc14f050 con 0x7f59bc15eb10 2024-09-17T02:51:53.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59b97fa640 1 -- 172.21.15.131:0/3882575767 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b40b2040 con 0x7f59bc15eb10 2024-09-17T02:51:53.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c2fdf640 1 -- 172.21.15.131:0/3882575767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc14f610 con 0x7f59bc15eb10 2024-09-17T02:51:53.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c2fdf640 1 -- 172.21.15.131:0/3882575767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59bc15e5d0 con 0x7f59bc15eb10 2024-09-17T02:51:53.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59b97fa640 1 -- 172.21.15.131:0/3882575767 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f59b4004560 con 0x7f59bc15eb10 2024-09-17T02:51:53.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59b97fa640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59a0073230 0x7f59a0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:53.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59b97fa640 1 -- 172.21.15.131:0/3882575767 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5619+0+0 (secure 0 0 0) 0x7f59b412e090 con 0x7f59bc15eb10 2024-09-17T02:51:53.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.538+0000 7f59c0d7b640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59a0073230 0x7f59a0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:53.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.542+0000 7f59c0d7b640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59a0073230 0x7f59a0075740 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f59b0005fd0 tx=0x7f59b0005de0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:53.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.546+0000 7f59b97fa640 1 -- 172.21.15.131:0/3882575767 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f59b4016990 con 0x7f59bc15eb10 2024-09-17T02:51:53.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:53.718+0000 7f59c2fdf640 1 -- 172.21.15.131:0/3882575767 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f59bc0cdca0 con 0x7f59bc15eb10 2024-09-17T02:51:54.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.342+0000 7f59b97fa640 1 -- 172.21.15.131:0/3882575767 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v120) ==== 170+0+0 (secure 0 0 0) 0x7f59b40f9080 con 0x7f59bc15eb10 2024-09-17T02:51:54.337 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' created 2024-09-17T02:51:54.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 -- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59a0073230 msgr2=0x7f59a0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:54.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59a0073230 0x7f59a0075740 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f59b0005fd0 tx=0x7f59b0005de0 comp rx=0 tx=0).stop 2024-09-17T02:51:54.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 -- 172.21.15.131:0/3882575767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc15eb10 msgr2=0x7f59bc153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:54.340 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc15eb10 0x7f59bc153130 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009bd0 tx=0x7f59b4005af0 comp rx=0 tx=0).stop 2024-09-17T02:51:54.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 -- 172.21.15.131:0/3882575767 shutdown_connections 2024-09-17T02:51:54.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f59a0073230 0x7f59a0075740 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:54.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f59bc15eb10 0x7f59bc153130 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:54.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f59bc151a00 0x7f59bc13ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:54.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 --2- 172.21.15.131:0/3882575767 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f59bc0cf680 0x7f59bc1563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:54.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 -- 172.21.15.131:0/3882575767 >> 172.21.15.131:0/3882575767 conn(0x7f59bc0d0490 msgr2=0x7f59bc15b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:54.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 -- 172.21.15.131:0/3882575767 shutdown_connections 2024-09-17T02:51:54.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.346+0000 7f59c2fdf640 1 -- 172.21.15.131:0/3882575767 wait complete. 2024-09-17T02:51:54.381 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T02:51:54.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.542+0000 7f48ba1a6640 1 -- 172.21.15.131:0/1804433054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f489c004810 msgr2=0x7f489c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:54.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.542+0000 7f48ba1a6640 1 --2- 172.21.15.131:0/1804433054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f489c004810 0x7f489c004c20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f48a8009b40 tx=0x7f48a801d270 comp rx=0 tx=0).stop 2024-09-17T02:51:54.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.542+0000 7f48b3fff640 1 -- 172.21.15.131:0/1804433054 reap_dead start 2024-09-17T02:51:54.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.542+0000 7f48ba1a6640 1 -- 172.21.15.131:0/1804433054 shutdown_connections 2024-09-17T02:51:54.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.542+0000 7f48ba1a6640 1 -- 172.21.15.131:0/1804433054 >> 172.21.15.131:0/1804433054 conn(0x7f48b40cf0b0 msgr2=0x7f48b40cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:54.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.542+0000 7f48ba1a6640 1 -- 172.21.15.131:0/1804433054 shutdown_connections 2024-09-17T02:51:54.538 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.542+0000 7f48ba1a6640 1 -- 172.21.15.131:0/1804433054 wait complete. 2024-09-17T02:51:54.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48ba1a6640 1 Processor -- start 2024-09-17T02:51:54.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48ba1a6640 1 -- start start 2024-09-17T02:51:54.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48ba1a6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f489c004810 0x7f48b41567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48ba1a6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f48b40c8580 0x7f48b4159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48ba1a6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48b40c8990 0x7f48b4159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48ba1a6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f48b4162820 con 0x7f48b40c8990 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48ba1a6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f48b40b48d0 con 0x7f489c004810 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48ba1a6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f48b41629a0 con 0x7f48b40c8580 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48b40c8990 0x7f48b4159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48b40c8990 0x7f48b4159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41780/0 (socket says 172.21.15.131:41780) 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b3fff640 1 -- 172.21.15.131:0/4037745624 learned_addr learned my addr 172.21.15.131:0/4037745624 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b37fe640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f489c004810 0x7f48b41567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b2ffd640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f48b40c8580 0x7f48b4159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b3fff640 1 -- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f48b40c8580 msgr2=0x7f48b4159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:54.543 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b3fff640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f48b40c8580 0x7f48b4159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b3fff640 1 -- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f489c004810 msgr2=0x7f48b41567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b3fff640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f489c004810 0x7f48b41567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b3fff640 1 -- 172.21.15.131:0/4037745624 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b4156e00 con 0x7f48b40c8990 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b2ffd640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f48b40c8580 0x7f48b4159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b37fe640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f489c004810 0x7f48b41567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48b3fff640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48b40c8990 0x7f48b4159bb0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f48ac001e40 tx=0x7f48ac010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f489bfff640 1 -- 172.21.15.131:0/4037745624 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac00ee60 con 0x7f48b40c8990 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f489bfff640 1 -- 172.21.15.131:0/4037745624 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48ac013570 con 0x7f48b40c8990 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48ba1a6640 1 -- 172.21.15.131:0/4037745624 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b41570e0 con 0x7f48b40c8990 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f489bfff640 1 -- 172.21.15.131:0/4037745624 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac0139c0 con 0x7f48b40c8990 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.546+0000 7f48ba1a6640 1 -- 172.21.15.131:0/4037745624 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b41576a0 con 0x7f48b40c8990 2024-09-17T02:51:54.544 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.550+0000 7f48ba1a6640 1 -- 172.21.15.131:0/4037745624 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b40b3fb0 con 0x7f48b40c8990 2024-09-17T02:51:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.550+0000 7f489bfff640 1 -- 172.21.15.131:0/4037745624 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f48ac013b60 con 0x7f48b40c8990 2024-09-17T02:51:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.550+0000 7f489bfff640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48a0073200 0x7f48a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.550+0000 7f489bfff640 1 -- 172.21.15.131:0/4037745624 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 6001+0+0 (secure 0 0 0) 0x7f48ac0983a0 con 0x7f48b40c8990 2024-09-17T02:51:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.550+0000 7f48b37fe640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48a0073200 0x7f48a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:54.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.550+0000 7f48b37fe640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48a0073200 0x7f48a0075710 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f48a40065f0 tx=0x7f48a4006bd0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:54.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.554+0000 7f489bfff640 1 -- 172.21.15.131:0/4037745624 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f48ac063290 con 0x7f48b40c8990 2024-09-17T02:51:54.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:54.762+0000 7f48ba1a6640 1 -- 172.21.15.131:0/4037745624 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f48b40cf930 con 0x7f48b40c8990 2024-09-17T02:51:55.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.346+0000 7f489bfff640 1 -- 172.21.15.131:0/4037745624 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v98) ==== 174+0+0 (secure 0 0 0) 0x7f48ac062b90 con 0x7f48b40c8990 2024-09-17T02:51:55.339 INFO:teuthology.orchestra.run.smithi131.stderr:new fs with metadata pool 17 and data pool 18 2024-09-17T02:51:55.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 -- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48a0073200 msgr2=0x7f48a0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48a0073200 0x7f48a0075710 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f48a40065f0 tx=0x7f48a4006bd0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 -- 172.21.15.131:0/4037745624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48b40c8990 msgr2=0x7f48b4159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48b40c8990 0x7f48b4159bb0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f48ac001e40 tx=0x7f48ac010f10 comp rx=0 tx=0).stop 2024-09-17T02:51:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 -- 172.21.15.131:0/4037745624 shutdown_connections 2024-09-17T02:51:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f48a0073200 0x7f48a0075710 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f48b40c8990 0x7f48b4159bb0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f48b40c8580 0x7f48b4159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 --2- 172.21.15.131:0/4037745624 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f489c004810 0x7f48b41567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 -- 172.21.15.131:0/4037745624 >> 172.21.15.131:0/4037745624 conn(0x7f48b40cf0b0 msgr2=0x7f48b40b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 -- 172.21.15.131:0/4037745624 shutdown_connections 2024-09-17T02:51:55.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.350+0000 7f48ba1a6640 1 -- 172.21.15.131:0/4037745624 wait complete. 2024-09-17T02:51:55.389 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:51:55.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.546+0000 7f087a84c640 1 -- 172.21.15.131:0/2884742709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08740cec00 msgr2=0x7f08740cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:55.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.546+0000 7f087a84c640 1 --2- 172.21.15.131:0/2884742709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08740cec00 0x7f08740cf010 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f0874071d30 tx=0x7f086801c770 comp rx=0 tx=0).stop 2024-09-17T02:51:55.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.546+0000 7f087a84c640 1 -- 172.21.15.131:0/2884742709 shutdown_connections 2024-09-17T02:51:55.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.546+0000 7f087a84c640 1 --2- 172.21.15.131:0/2884742709 >> v2:172.21.15.189:3300/0 conn(0x7f08740c4c90 0x7f08740c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.546+0000 7f087a84c640 1 --2- 172.21.15.131:0/2884742709 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f08740cec00 0x7f08740cf010 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.546+0000 7f087a84c640 1 -- 172.21.15.131:0/2884742709 >> 172.21.15.131:0/2884742709 conn(0x7f087406eea0 msgr2=0x7f087406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:55.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.546+0000 7f087a84c640 1 -- 172.21.15.131:0/2884742709 shutdown_connections 2024-09-17T02:51:55.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.546+0000 7f087a84c640 1 -- 172.21.15.131:0/2884742709 wait complete. 2024-09-17T02:51:55.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f087a84c640 1 Processor -- start 2024-09-17T02:51:55.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f087a84c640 1 -- start start 2024-09-17T02:51:55.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f087a84c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087406e2c0 0x7f087414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:55.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f087a84c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08740c4c90 0x7f08740c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:55.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f087a84c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f08740cec00 0x7f08740c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:55.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f087a84c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0874169c60 con 0x7f087406e2c0 2024-09-17T02:51:55.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f087a84c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0874169ae0 con 0x7f08740c4c90 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f087a84c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0874169de0 con 0x7f08740cec00 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0873fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087406e2c0 0x7f087414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0873fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087406e2c0 0x7f087414efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41804/0 (socket says 172.21.15.131:41804) 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0873fff640 1 -- 172.21.15.131:0/3805468664 learned_addr learned my addr 172.21.15.131:0/3805468664 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f08737fe640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08740c4c90 0x7f08740c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0873fff640 1 -- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f08740cec00 msgr2=0x7f08740c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0873fff640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f08740cec00 0x7f08740c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0873fff640 1 -- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08740c4c90 msgr2=0x7f08740c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0873fff640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08740c4c90 0x7f08740c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0873fff640 1 -- 172.21.15.131:0/3805468664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087414fab0 con 0x7f087406e2c0 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f08737fe640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08740c4c90 0x7f08740c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0873fff640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087406e2c0 0x7f087414efc0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f086400c430 tx=0x7f086400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0870ff9640 1 -- 172.21.15.131:0/3805468664 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864011790 con 0x7f087406e2c0 2024-09-17T02:51:55.546 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0870ff9640 1 -- 172.21.15.131:0/3805468664 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0864017070 con 0x7f087406e2c0 2024-09-17T02:51:55.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f087a84c640 1 -- 172.21.15.131:0/3805468664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087414fd90 con 0x7f087406e2c0 2024-09-17T02:51:55.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f0870ff9640 1 -- 172.21.15.131:0/3805468664 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864011ce0 con 0x7f087406e2c0 2024-09-17T02:51:55.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.550+0000 7f087a84c640 1 -- 172.21.15.131:0/3805468664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0874150350 con 0x7f087406e2c0 2024-09-17T02:51:55.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.554+0000 7f087a84c640 1 -- 172.21.15.131:0/3805468664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08740cf010 con 0x7f087406e2c0 2024-09-17T02:51:55.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.554+0000 7f0870ff9640 1 -- 172.21.15.131:0/3805468664 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f086401c460 con 0x7f087406e2c0 2024-09-17T02:51:55.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.554+0000 7f0870ff9640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0860073230 0x7f0860075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:55.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.554+0000 7f0870ff9640 1 -- 172.21.15.131:0/3805468664 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f0864094bf0 con 0x7f087406e2c0 2024-09-17T02:51:55.548 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.554+0000 7f08737fe640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0860073230 0x7f0860075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:55.549 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.554+0000 7f08737fe640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0860073230 0x7f0860075740 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f08680af000 tx=0x7f086801afb0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:55.551 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.558+0000 7f0870ff9640 1 -- 172.21.15.131:0/3805468664 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f086405fc20 con 0x7f087406e2c0 2024-09-17T02:51:55.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.726+0000 7f087a84c640 1 -- 172.21.15.131:0/3805468664 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0874073310 con 0x7f087406e2c0 2024-09-17T02:51:55.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.730+0000 7f0870ff9640 1 -- 172.21.15.131:0/3805468664 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17077 (secure 0 0 0) 0x7f086405f520 con 0x7f087406e2c0 2024-09-17T02:51:55.724 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:55.725 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":121,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:51:54.764405+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:51:52.442187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-17T02:51:53.731428+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:51:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 -- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0860073230 msgr2=0x7f0860075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0860073230 0x7f0860075740 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f08680af000 tx=0x7f086801afb0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 -- 172.21.15.131:0/3805468664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087406e2c0 msgr2=0x7f087414efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:55.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087406e2c0 0x7f087414efc0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f086400c430 tx=0x7f086400c8f0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 -- 172.21.15.131:0/3805468664 shutdown_connections 2024-09-17T02:51:55.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f0860073230 0x7f0860075740 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f08740cec00 0x7f08740c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f08740c4c90 0x7f08740c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 --2- 172.21.15.131:0/3805468664 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f087406e2c0 0x7f087414efc0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 -- 172.21.15.131:0/3805468664 >> 172.21.15.131:0/3805468664 conn(0x7f087406eea0 msgr2=0x7f087415b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:55.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 -- 172.21.15.131:0/3805468664 shutdown_connections 2024-09-17T02:51:55.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.734+0000 7f087a84c640 1 -- 172.21.15.131:0/3805468664 wait complete. 2024-09-17T02:51:55.749 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:51:55.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.914+0000 7f8c8f753640 1 -- 172.21.15.131:0/4138120882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c8815c5b0 msgr2=0x7f8c880cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:55.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.914+0000 7f8c8f753640 1 --2- 172.21.15.131:0/4138120882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c8815c5b0 0x7f8c880cbaf0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009840 tx=0x7f8c8001c820 comp rx=0 tx=0).stop 2024-09-17T02:51:55.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.914+0000 7f8c8f753640 1 -- 172.21.15.131:0/4138120882 shutdown_connections 2024-09-17T02:51:55.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.914+0000 7f8c8f753640 1 --2- 172.21.15.131:0/4138120882 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c8815c5b0 0x7f8c880cbaf0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.914+0000 7f8c8f753640 1 -- 172.21.15.131:0/4138120882 >> 172.21.15.131:0/4138120882 conn(0x7f8c8806eea0 msgr2=0x7f8c8806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:55.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.914+0000 7f8c8f753640 1 -- 172.21.15.131:0/4138120882 shutdown_connections 2024-09-17T02:51:55.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.914+0000 7f8c8f753640 1 -- 172.21.15.131:0/4138120882 wait complete. 2024-09-17T02:51:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.914+0000 7f8c8f753640 1 Processor -- start 2024-09-17T02:51:55.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.914+0000 7f8c8f753640 1 -- start start 2024-09-17T02:51:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8f753640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c880ced00 0x7f8c880c3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:55.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8f753640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c8815c5b0 0x7f8c881523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8f753640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c881528f0 0x7f8c8815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8f753640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88169a20 con 0x7f8c8815c5b0 2024-09-17T02:51:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8f753640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8c881698a0 con 0x7f8c881528f0 2024-09-17T02:51:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8f753640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8c88169ba0 con 0x7f8c880ced00 2024-09-17T02:51:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8ccee640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c8815c5b0 0x7f8c881523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8ccee640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c8815c5b0 0x7f8c881523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41822/0 (socket says 172.21.15.131:41822) 2024-09-17T02:51:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8ccee640 1 -- 172.21.15.131:0/554783528 learned_addr learned my addr 172.21.15.131:0/554783528 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8d4ef640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c880ced00 0x7f8c880c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8dcf0640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c881528f0 0x7f8c8815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8ccee640 1 -- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c880ced00 msgr2=0x7f8c880c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:55.912 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8ccee640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c880ced00 0x7f8c880c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8ccee640 1 -- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c881528f0 msgr2=0x7f8c8815dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8ccee640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c881528f0 0x7f8c8815dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8ccee640 1 -- 172.21.15.131:0/554783528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c881530e0 con 0x7f8c8815c5b0 2024-09-17T02:51:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8d4ef640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c880ced00 0x7f8c880c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8ccee640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c8815c5b0 0x7f8c881523b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400c790 tx=0x7f8c8400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c7dffb640 1 -- 172.21.15.131:0/554783528 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84003b80 con 0x7f8c8815c5b0 2024-09-17T02:51:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c7dffb640 1 -- 172.21.15.131:0/554783528 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c84003d20 con 0x7f8c8815c5b0 2024-09-17T02:51:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8f753640 1 -- 172.21.15.131:0/554783528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c881533c0 con 0x7f8c8815c5b0 2024-09-17T02:51:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c7dffb640 1 -- 172.21.15.131:0/554783528 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8400aa90 con 0x7f8c8815c5b0 2024-09-17T02:51:55.913 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8f753640 1 -- 172.21.15.131:0/554783528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c88153980 con 0x7f8c8815c5b0 2024-09-17T02:51:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c8f753640 1 -- 172.21.15.131:0/554783528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c88075b80 con 0x7f8c8815c5b0 2024-09-17T02:51:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c7dffb640 1 -- 172.21.15.131:0/554783528 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8c8400ac30 con 0x7f8c8815c5b0 2024-09-17T02:51:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c7dffb640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c6c073200 0x7f8c6c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.918+0000 7f8c7dffb640 1 -- 172.21.15.131:0/554783528 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7f8c840941d0 con 0x7f8c8815c5b0 2024-09-17T02:51:55.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.922+0000 7f8c8d4ef640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c6c073200 0x7f8c6c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:55.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.922+0000 7f8c8d4ef640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c6c073200 0x7f8c6c075710 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8c78001400 tx=0x7f8c78009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:55.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:55.926+0000 7f8c7dffb640 1 -- 172.21.15.131:0/554783528 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8c8405f0f0 con 0x7f8c8815c5b0 2024-09-17T02:51:56.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.090+0000 7f8c8f753640 1 -- 172.21.15.131:0/554783528 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c88152e70 con 0x7f8c8815c5b0 2024-09-17T02:51:56.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c7dffb640 1 -- 172.21.15.131:0/554783528 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+17077 (secure 0 0 0) 0x7f8c8405e9f0 con 0x7f8c8815c5b0 2024-09-17T02:51:56.087 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:56.087 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":121,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:51:54.764405+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:51:52.442187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-17T02:51:53.731428+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:51:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 -- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c6c073200 msgr2=0x7f8c6c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c6c073200 0x7f8c6c075710 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8c78001400 tx=0x7f8c78009040 comp rx=0 tx=0).stop 2024-09-17T02:51:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 -- 172.21.15.131:0/554783528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c8815c5b0 msgr2=0x7f8c881523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c8815c5b0 0x7f8c881523b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400c790 tx=0x7f8c8400cc50 comp rx=0 tx=0).stop 2024-09-17T02:51:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 -- 172.21.15.131:0/554783528 shutdown_connections 2024-09-17T02:51:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8c6c073200 0x7f8c6c075710 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8c881528f0 0x7f8c8815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8c8815c5b0 0x7f8c881523b0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:56.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 --2- 172.21.15.131:0/554783528 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8c880ced00 0x7f8c880c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 -- 172.21.15.131:0/554783528 >> 172.21.15.131:0/554783528 conn(0x7f8c8806eea0 msgr2=0x7f8c880afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 -- 172.21.15.131:0/554783528 shutdown_connections 2024-09-17T02:51:56.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.094+0000 7f8c8f753640 1 -- 172.21.15.131:0/554783528 wait complete. 2024-09-17T02:51:56.109 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T02:51:56.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 -- 172.21.15.131:0/3546528807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe250004d50 msgr2=0x7fe250005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:56.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 --2- 172.21.15.131:0/3546528807 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe250004d50 0x7fe250005200 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe254009b30 tx=0x7fe25401d250 comp rx=0 tx=0).stop 2024-09-17T02:51:56.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe262d9d640 1 -- 172.21.15.131:0/3546528807 reap_dead start 2024-09-17T02:51:56.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 -- 172.21.15.131:0/3546528807 shutdown_connections 2024-09-17T02:51:56.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 -- 172.21.15.131:0/3546528807 >> 172.21.15.131:0/3546528807 conn(0x7fe2640cd6a0 msgr2=0x7fe2640cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:56.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 -- 172.21.15.131:0/3546528807 shutdown_connections 2024-09-17T02:51:56.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 -- 172.21.15.131:0/3546528807 wait complete. 2024-09-17T02:51:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 Processor -- start 2024-09-17T02:51:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 -- start start 2024-09-17T02:51:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2640b4620 0x7fe2640b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe26414b910 0x7fe26414bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe26414c260 0x7fe26415dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe2640cf9f0 con 0x7fe26414c260 2024-09-17T02:51:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2640cf870 con 0x7fe2640b4620 2024-09-17T02:51:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe268b91640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe2640cfb70 con 0x7fe26414b910 2024-09-17T02:51:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe262d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe26414c260 0x7fe26415dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe262d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe26414c260 0x7fe26415dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41846/0 (socket says 172.21.15.131:41846) 2024-09-17T02:51:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe262d9d640 1 -- 172.21.15.131:0/2531229552 learned_addr learned my addr 172.21.15.131:0/2531229552 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe26259c640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2640b4620 0x7fe2640b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:56.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.270+0000 7fe261d9b640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe26414b910 0x7fe26414bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe262d9d640 1 -- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe26414b910 msgr2=0x7fe26414bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe262d9d640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe26414b910 0x7fe26414bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe262d9d640 1 -- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2640b4620 msgr2=0x7fe2640b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe262d9d640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2640b4620 0x7fe2640b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe262d9d640 1 -- 172.21.15.131:0/2531229552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2640b4f70 con 0x7fe26414c260 2024-09-17T02:51:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe262d9d640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe26414c260 0x7fe26415dcf0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe2540af040 tx=0x7fe2540add30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe25affd640 1 -- 172.21.15.131:0/2531229552 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2540b1040 con 0x7fe26414c260 2024-09-17T02:51:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe25affd640 1 -- 172.21.15.131:0/2531229552 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe254000e50 con 0x7fe26414c260 2024-09-17T02:51:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe268b91640 1 -- 172.21.15.131:0/2531229552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe264155ff0 con 0x7fe26414c260 2024-09-17T02:51:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe25affd640 1 -- 172.21.15.131:0/2531229552 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe254009730 con 0x7fe26414c260 2024-09-17T02:51:56.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe268b91640 1 -- 172.21.15.131:0/2531229552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2641565b0 con 0x7fe26414c260 2024-09-17T02:51:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe268b91640 1 -- 172.21.15.131:0/2531229552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2640b36b0 con 0x7fe26414c260 2024-09-17T02:51:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe25affd640 1 -- 172.21.15.131:0/2531229552 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe2540bd020 con 0x7fe26414c260 2024-09-17T02:51:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe25affd640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe244073200 0x7fe244075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe25affd640 1 -- 172.21.15.131:0/2531229552 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 6101+0+0 (secure 0 0 0) 0x7fe25412e830 con 0x7fe26414c260 2024-09-17T02:51:56.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.274+0000 7fe26259c640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe244073200 0x7fe244075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:56.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.278+0000 7fe26259c640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe244073200 0x7fe244075710 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe24c001c30 tx=0x7fe24c003980 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:56.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.278+0000 7fe25affd640 1 -- 172.21.15.131:0/2531229552 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe2540c4020 con 0x7fe26414c260 2024-09-17T02:51:56.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:56.482+0000 7fe268b91640 1 -- 172.21.15.131:0/2531229552 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe2640ada20 con 0x7fe26414c260 2024-09-17T02:51:57.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.386+0000 7fe25affd640 1 -- 172.21.15.131:0/2531229552 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v100) ==== 118+0+0 (secure 0 0 0) 0x7fe254100e20 con 0x7fe26414c260 2024-09-17T02:51:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 -- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe244073200 msgr2=0x7fe244075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe244073200 0x7fe244075710 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe24c001c30 tx=0x7fe24c003980 comp rx=0 tx=0).stop 2024-09-17T02:51:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 -- 172.21.15.131:0/2531229552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe26414c260 msgr2=0x7fe26415dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe26414c260 0x7fe26415dcf0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe2540af040 tx=0x7fe2540add30 comp rx=0 tx=0).stop 2024-09-17T02:51:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 -- 172.21.15.131:0/2531229552 shutdown_connections 2024-09-17T02:51:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe244073200 0x7fe244075710 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe26414c260 0x7fe26415dcf0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe26414b910 0x7fe26414bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 --2- 172.21.15.131:0/2531229552 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2640b4620 0x7fe2640b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:57.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 -- 172.21.15.131:0/2531229552 >> 172.21.15.131:0/2531229552 conn(0x7fe2640cd6a0 msgr2=0x7fe2640b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:57.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 -- 172.21.15.131:0/2531229552 shutdown_connections 2024-09-17T02:51:57.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.390+0000 7fe268b91640 1 -- 172.21.15.131:0/2531229552 wait complete. 2024-09-17T02:51:57.408 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T02:51:57.408 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T02:51:57.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.574+0000 7fc206674640 1 -- 172.21.15.131:0/2623175004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2000cec00 msgr2=0x7fc2000cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.574+0000 7fc206674640 1 --2- 172.21.15.131:0/2623175004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2000cec00 0x7fc2000cf010 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fc200071d30 tx=0x7fc1e801c770 comp rx=0 tx=0).stop 2024-09-17T02:51:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.574+0000 7fc206674640 1 -- 172.21.15.131:0/2623175004 shutdown_connections 2024-09-17T02:51:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.574+0000 7fc206674640 1 --2- 172.21.15.131:0/2623175004 >> v2:172.21.15.189:3300/0 conn(0x7fc2000c4c90 0x7fc2000c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.574+0000 7fc206674640 1 --2- 172.21.15.131:0/2623175004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc2000cec00 0x7fc2000cf010 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.574+0000 7fc206674640 1 -- 172.21.15.131:0/2623175004 >> 172.21.15.131:0/2623175004 conn(0x7fc20006eea0 msgr2=0x7fc20006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.574+0000 7fc206674640 1 -- 172.21.15.131:0/2623175004 shutdown_connections 2024-09-17T02:51:57.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.574+0000 7fc206674640 1 -- 172.21.15.131:0/2623175004 wait complete. 2024-09-17T02:51:57.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc206674640 1 Processor -- start 2024-09-17T02:51:57.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc206674640 1 -- start start 2024-09-17T02:51:57.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc206674640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e2c0 0x7fc2001fc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:57.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc206674640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc2000c4c90 0x7fc20015bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:57.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc206674640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2000cec00 0x7fc200156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:57.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc206674640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc200169c60 con 0x7fc20006e2c0 2024-09-17T02:51:57.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc206674640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc200169ae0 con 0x7fc2000cec00 2024-09-17T02:51:57.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc206674640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc200169de0 con 0x7fc2000c4c90 2024-09-17T02:51:57.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e2c0 0x7fc2001fc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:57.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e2c0 0x7fc2001fc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41858/0 (socket says 172.21.15.131:41858) 2024-09-17T02:51:57.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fffff640 1 -- 172.21.15.131:0/3730818494 learned_addr learned my addr 172.21.15.131:0/3730818494 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:57.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc204c11640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2000cec00 0x7fc200156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:57.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fffff640 1 -- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc2000c4c90 msgr2=0x7fc20015bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:57.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1f77fe640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc2000c4c90 0x7fc20015bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:57.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fffff640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc2000c4c90 0x7fc20015bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:57.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fffff640 1 -- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2000cec00 msgr2=0x7fc200156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:57.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fffff640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2000cec00 0x7fc200156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:57.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fffff640 1 -- 172.21.15.131:0/3730818494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc200157b10 con 0x7fc20006e2c0 2024-09-17T02:51:57.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc204c11640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2000cec00 0x7fc200156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:57.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fffff640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e2c0 0x7fc2001fc870 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000c430 tx=0x7fc1f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:57.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fd7fa640 1 -- 172.21.15.131:0/3730818494 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0011790 con 0x7fc20006e2c0 2024-09-17T02:51:57.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fd7fa640 1 -- 172.21.15.131:0/3730818494 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1f0017070 con 0x7fc20006e2c0 2024-09-17T02:51:57.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc206674640 1 -- 172.21.15.131:0/3730818494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc200157df0 con 0x7fc20006e2c0 2024-09-17T02:51:57.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc1fd7fa640 1 -- 172.21.15.131:0/3730818494 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0011ce0 con 0x7fc20006e2c0 2024-09-17T02:51:57.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.578+0000 7fc206674640 1 -- 172.21.15.131:0/3730818494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc20015a040 con 0x7fc20006e2c0 2024-09-17T02:51:57.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.582+0000 7fc206674640 1 -- 172.21.15.131:0/3730818494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2000cf010 con 0x7fc20006e2c0 2024-09-17T02:51:57.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.582+0000 7fc1fd7fa640 1 -- 172.21.15.131:0/3730818494 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc1f001c460 con 0x7fc20006e2c0 2024-09-17T02:51:57.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.582+0000 7fc1fd7fa640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 0x7fc1e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:57.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.582+0000 7fc1fd7fa640 1 -- 172.21.15.131:0/3730818494 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fc1f00941a0 con 0x7fc20006e2c0 2024-09-17T02:51:57.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.582+0000 7fc1f77fe640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 0x7fc1e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:57.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.582+0000 7fc1f77fe640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 0x7fc1e0075710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80af000 tx=0x7fc1e801afb0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:57.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.586+0000 7fc1fd7fa640 1 -- 172.21.15.131:0/3730818494 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc1f005f0f0 con 0x7fc20006e2c0 2024-09-17T02:51:57.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:57.798+0000 7fc206674640 1 -- 172.21.15.131:0/3730818494 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc200073310 con 0x7fc20006e2c0 2024-09-17T02:51:58.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.398+0000 7fc1fd7fa640 1 -- 172.21.15.131:0/3730818494 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v101) ==== 122+0+0 (secure 0 0 0) 0x7fc1f005e9f0 con 0x7fc20006e2c0 2024-09-17T02:51:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 -- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 msgr2=0x7fc1e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 0x7fc1e0075710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80af000 tx=0x7fc1e801afb0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 -- 172.21.15.131:0/3730818494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e2c0 msgr2=0x7fc2001fc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:58.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e2c0 0x7fc2001fc870 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000c430 tx=0x7fc1f000c8f0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 -- 172.21.15.131:0/3730818494 shutdown_connections 2024-09-17T02:51:58.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc1e0073200 0x7fc1e0075710 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc2000cec00 0x7fc200156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc2000c4c90 0x7fc20015bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 --2- 172.21.15.131:0/3730818494 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc20006e2c0 0x7fc2001fc870 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 -- 172.21.15.131:0/3730818494 >> 172.21.15.131:0/3730818494 conn(0x7fc20006eea0 msgr2=0x7fc2000b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:58.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 -- 172.21.15.131:0/3730818494 shutdown_connections 2024-09-17T02:51:58.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.402+0000 7fc206674640 1 -- 172.21.15.131:0/3730818494 wait complete. 2024-09-17T02:51:58.419 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T02:51:58.419 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:51:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 -- 172.21.15.131:0/447155722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58d406dea0 msgr2=0x7f58d406e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 --2- 172.21.15.131:0/447155722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58d406dea0 0x7f58d406e2b0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f58d40cdb80 tx=0x7f58c401c6b0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.579 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 -- 172.21.15.131:0/447155722 shutdown_connections 2024-09-17T02:51:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 --2- 172.21.15.131:0/447155722 >> v2:172.21.15.189:3300/0 conn(0x7f58d40cab60 0x7f58d40caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 --2- 172.21.15.131:0/447155722 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58d406dea0 0x7f58d406e2b0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 -- 172.21.15.131:0/447155722 >> 172.21.15.131:0/447155722 conn(0x7f58d406eea0 msgr2=0x7f58d406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 -- 172.21.15.131:0/447155722 shutdown_connections 2024-09-17T02:51:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 -- 172.21.15.131:0/447155722 wait complete. 2024-09-17T02:51:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 Processor -- start 2024-09-17T02:51:58.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 -- start start 2024-09-17T02:51:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f58d406dea0 0x7f58d4152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58d40cab60 0x7f58d4152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58d4155240 0x7f58d41531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4168430 con 0x7f58d4155240 2024-09-17T02:51:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f58d41682b0 con 0x7f58d40cab60 2024-09-17T02:51:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58db8b9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f58d41685b0 con 0x7f58d406dea0 2024-09-17T02:51:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58d9e56640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58d4155240 0x7f58d41531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:58.583 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58d9e56640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58d4155240 0x7f58d41531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41884/0 (socket says 172.21.15.131:41884) 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.586+0000 7f58d9e56640 1 -- 172.21.15.131:0/4041461284 learned_addr learned my addr 172.21.15.131:0/4041461284 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58d8e54640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58d40cab60 0x7f58d4152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58d9655640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f58d406dea0 0x7f58d4152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58d9e56640 1 -- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f58d406dea0 msgr2=0x7f58d4152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58d9e56640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f58d406dea0 0x7f58d4152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58d9e56640 1 -- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58d40cab60 msgr2=0x7f58d4152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58d9e56640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58d40cab60 0x7f58d4152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58d9e56640 1 -- 172.21.15.131:0/4041461284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d4155bf0 con 0x7f58d4155240 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58d8e54640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58d40cab60 0x7f58d4152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58d9655640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f58d406dea0 0x7f58d4152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58d9e56640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58d4155240 0x7f58d41531c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00ca60 tx=0x7f58cc00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58c9ffb640 1 -- 172.21.15.131:0/4041461284 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc007d90 con 0x7f58d4155240 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58c9ffb640 1 -- 172.21.15.131:0/4041461284 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58cc004c20 con 0x7f58d4155240 2024-09-17T02:51:58.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58db8b9640 1 -- 172.21.15.131:0/4041461284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d413d670 con 0x7f58d4155240 2024-09-17T02:51:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58c9ffb640 1 -- 172.21.15.131:0/4041461284 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc005280 con 0x7f58d4155240 2024-09-17T02:51:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58db8b9640 1 -- 172.21.15.131:0/4041461284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d413dc30 con 0x7f58d4155240 2024-09-17T02:51:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58db8b9640 1 -- 172.21.15.131:0/4041461284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58d413d460 con 0x7f58d4155240 2024-09-17T02:51:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58c9ffb640 1 -- 172.21.15.131:0/4041461284 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f58cc005420 con 0x7f58d4155240 2024-09-17T02:51:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58c9ffb640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58b8073200 0x7f58b8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:58.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58c9ffb640 1 -- 172.21.15.131:0/4041461284 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7f58cc094b10 con 0x7f58d4155240 2024-09-17T02:51:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.590+0000 7f58d9655640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58b8073200 0x7f58b8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.594+0000 7f58d9655640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58b8073200 0x7f58b8075710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f58d000a850 tx=0x7f58d0008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:58.589 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.594+0000 7f58c9ffb640 1 -- 172.21.15.131:0/4041461284 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f58cc014030 con 0x7f58d4155240 2024-09-17T02:51:58.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.798+0000 7f58db8b9640 1 -- 172.21.15.131:0/4041461284 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58d406e2b0 con 0x7f58d4155240 2024-09-17T02:51:58.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.798+0000 7f58c9ffb640 1 -- 172.21.15.131:0/4041461284 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5236 (secure 0 0 0) 0x7f58cc05fa30 con 0x7f58d4155240 2024-09-17T02:51:58.792 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:58.792 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":101,"btime":"2024-09-17T02:51:58:389699+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:51:57.800237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8490":{"gid":8490,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2599844908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2599844908},{"type":"v1","addr":"172.21.15.131:6835","nonce":2599844908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8490,"qdb_cluster":[8490]},"id":8}]} 2024-09-17T02:51:58.792 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 101 2024-09-17T02:51:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 -- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58b8073200 msgr2=0x7f58b8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58b8073200 0x7f58b8075710 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f58d000a850 tx=0x7f58d0008040 comp rx=0 tx=0).stop 2024-09-17T02:51:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 -- 172.21.15.131:0/4041461284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58d4155240 msgr2=0x7f58d41531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58d4155240 0x7f58d41531c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f58cc00ca60 tx=0x7f58cc00cf20 comp rx=0 tx=0).stop 2024-09-17T02:51:58.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 -- 172.21.15.131:0/4041461284 shutdown_connections 2024-09-17T02:51:58.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f58b8073200 0x7f58b8075710 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f58d4155240 0x7f58d41531c0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f58d40cab60 0x7f58d4152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 --2- 172.21.15.131:0/4041461284 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f58d406dea0 0x7f58d4152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 -- 172.21.15.131:0/4041461284 >> 172.21.15.131:0/4041461284 conn(0x7f58d406eea0 msgr2=0x7f58d40733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:58.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 -- 172.21.15.131:0/4041461284 shutdown_connections 2024-09-17T02:51:58.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.802+0000 7f58db8b9640 1 -- 172.21.15.131:0/4041461284 wait complete. 2024-09-17T02:51:58.813 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:51:58.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.974+0000 7ff93dc0d640 1 -- 172.21.15.131:0/2108595635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff924003a90 msgr2=0x7ff924003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:58.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.974+0000 7ff93dc0d640 1 --2- 172.21.15.131:0/2108595635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff924003a90 0x7ff924003f00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff9380c7050 tx=0x7ff92801cee0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.974+0000 7ff937fff640 1 -- 172.21.15.131:0/2108595635 reap_dead start 2024-09-17T02:51:58.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.974+0000 7ff93dc0d640 1 -- 172.21.15.131:0/2108595635 shutdown_connections 2024-09-17T02:51:58.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.974+0000 7ff93dc0d640 1 -- 172.21.15.131:0/2108595635 >> 172.21.15.131:0/2108595635 conn(0x7ff93806eea0 msgr2=0x7ff93806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:58.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.974+0000 7ff93dc0d640 1 -- 172.21.15.131:0/2108595635 shutdown_connections 2024-09-17T02:51:58.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.974+0000 7ff93dc0d640 1 -- 172.21.15.131:0/2108595635 wait complete. 2024-09-17T02:51:58.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.974+0000 7ff93dc0d640 1 Processor -- start 2024-09-17T02:51:58.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.974+0000 7ff93dc0d640 1 -- start start 2024-09-17T02:51:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff93dc0d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff924003a90 0x7ff93814e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff93dc0d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff93813fad0 0x7ff93813fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff93dc0d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff938140420 0x7ff9381f9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff93dc0d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff938168c60 con 0x7ff924003a90 2024-09-17T02:51:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff93dc0d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff938168ae0 con 0x7ff93813fad0 2024-09-17T02:51:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff93dc0d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff938168de0 con 0x7ff938140420 2024-09-17T02:51:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff9377fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff924003a90 0x7ff93814e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff9377fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff924003a90 0x7ff93814e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41912/0 (socket says 172.21.15.131:41912) 2024-09-17T02:51:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff9377fe640 1 -- 172.21.15.131:0/4072487379 learned_addr learned my addr 172.21.15.131:0/4072487379 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:58.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff92effd640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff93813fad0 0x7ff93813fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff937fff640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff938140420 0x7ff9381f9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff9377fe640 1 -- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff938140420 msgr2=0x7ff9381f9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff9377fe640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff938140420 0x7ff9381f9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff9377fe640 1 -- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff93813fad0 msgr2=0x7ff93813fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff9377fe640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff93813fad0 0x7ff93813fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff9377fe640 1 -- 172.21.15.131:0/4072487379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93814e840 con 0x7ff924003a90 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff937fff640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff938140420 0x7ff9381f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff9377fe640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff924003a90 0x7ff93814e300 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7ff92801b5e0 tx=0x7ff928006bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff934ff9640 1 -- 172.21.15.131:0/4072487379 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928001460 con 0x7ff924003a90 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff934ff9640 1 -- 172.21.15.131:0/4072487379 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff928001600 con 0x7ff924003a90 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff93dc0d640 1 -- 172.21.15.131:0/4072487379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9380c9ed0 con 0x7ff924003a90 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff934ff9640 1 -- 172.21.15.131:0/4072487379 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9280064e0 con 0x7ff924003a90 2024-09-17T02:51:58.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff93dc0d640 1 -- 172.21.15.131:0/4072487379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff938157c40 con 0x7ff924003a90 2024-09-17T02:51:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff93dc0d640 1 -- 172.21.15.131:0/4072487379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff938073310 con 0x7ff924003a90 2024-09-17T02:51:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff934ff9640 1 -- 172.21.15.131:0/4072487379 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff928003620 con 0x7ff924003a90 2024-09-17T02:51:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.978+0000 7ff934ff9640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff918073200 0x7ff918075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.982+0000 7ff934ff9640 1 -- 172.21.15.131:0/4072487379 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7ff92812e120 con 0x7ff924003a90 2024-09-17T02:51:58.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.982+0000 7ff92effd640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff918073200 0x7ff918075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:58.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.982+0000 7ff92effd640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff918073200 0x7ff918075710 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff920006fd0 tx=0x7ff920006b30 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:58.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:58.986+0000 7ff934ff9640 1 -- 172.21.15.131:0/4072487379 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff928016c90 con 0x7ff924003a90 2024-09-17T02:51:59.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.154+0000 7ff93dc0d640 1 -- 172.21.15.131:0/4072487379 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9380c4560 con 0x7ff924003a90 2024-09-17T02:51:59.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.154+0000 7ff934ff9640 1 -- 172.21.15.131:0/4072487379 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+17064 (secure 0 0 0) 0x7ff9280f9050 con 0x7ff924003a90 2024-09-17T02:51:59.149 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:59.150 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":122,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:51:56.347847+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:51:52.442187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-17T02:51:53.731428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:51:59.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 -- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff918073200 msgr2=0x7ff918075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff918073200 0x7ff918075710 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff920006fd0 tx=0x7ff920006b30 comp rx=0 tx=0).stop 2024-09-17T02:51:59.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 -- 172.21.15.131:0/4072487379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff924003a90 msgr2=0x7ff93814e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff924003a90 0x7ff93814e300 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7ff92801b5e0 tx=0x7ff928006bc0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 -- 172.21.15.131:0/4072487379 shutdown_connections 2024-09-17T02:51:59.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff918073200 0x7ff918075710 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff938140420 0x7ff9381f9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff93813fad0 0x7ff93813fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 --2- 172.21.15.131:0/4072487379 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff924003a90 0x7ff93814e300 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 -- 172.21.15.131:0/4072487379 >> 172.21.15.131:0/4072487379 conn(0x7ff93806eea0 msgr2=0x7ff93815a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:59.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 -- 172.21.15.131:0/4072487379 shutdown_connections 2024-09-17T02:51:59.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.158+0000 7ff93dc0d640 1 -- 172.21.15.131:0/4072487379 wait complete. 2024-09-17T02:51:59.171 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T02:51:59.171 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:51:59.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.318+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3722790835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c0003a40 msgr2=0x7fd4c0003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.318+0000 7fd4d88ed640 1 --2- 172.21.15.131:0/3722790835 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c0003a40 0x7fd4c0003eb0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4006fd0 tx=0x7fd4c401cf60 comp rx=0 tx=0).stop 2024-09-17T02:51:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.318+0000 7fd4d2d9d640 1 -- 172.21.15.131:0/3722790835 reap_dead start 2024-09-17T02:51:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.318+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3722790835 shutdown_connections 2024-09-17T02:51:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.318+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3722790835 >> 172.21.15.131:0/3722790835 conn(0x7fd4d406eea0 msgr2=0x7fd4d406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.318+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3722790835 shutdown_connections 2024-09-17T02:51:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.318+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3722790835 wait complete. 2024-09-17T02:51:59.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d88ed640 1 Processor -- start 2024-09-17T02:51:59.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d88ed640 1 -- start start 2024-09-17T02:51:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d88ed640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c0003a40 0x7fd4d40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d88ed640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4d414e5c0 0x7fd4d414d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d88ed640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd4d415d040 0x7fd4d415d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d88ed640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4168c60 con 0x7fd4c0003a40 2024-09-17T02:51:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d88ed640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4168ae0 con 0x7fd4d414e5c0 2024-09-17T02:51:59.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d88ed640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d4168de0 con 0x7fd4d415d040 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c0003a40 0x7fd4d40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c0003a40 0x7fd4d40cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41916/0 (socket says 172.21.15.131:41916) 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d259c640 1 -- 172.21.15.131:0/3758244628 learned_addr learned my addr 172.21.15.131:0/3758244628 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d2d9d640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd4d415d040 0x7fd4d415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4cbfff640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4d414e5c0 0x7fd4d414d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d259c640 1 -- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd4d415d040 msgr2=0x7fd4d415d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d259c640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd4d415d040 0x7fd4d415d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d259c640 1 -- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4d414e5c0 msgr2=0x7fd4d414d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d259c640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4d414e5c0 0x7fd4d414d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d259c640 1 -- 172.21.15.131:0/3758244628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d414de80 con 0x7fd4c0003a40 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d2d9d640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd4d415d040 0x7fd4d415d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d259c640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c0003a40 0x7fd4d40cbc00 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fd4c401ca20 tx=0x7fd4c4006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4caffd640 1 -- 172.21.15.131:0/3758244628 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40014e0 con 0x7fd4c0003a40 2024-09-17T02:51:59.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4caffd640 1 -- 172.21.15.131:0/3758244628 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4c40016a0 con 0x7fd4c0003a40 2024-09-17T02:51:59.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4caffd640 1 -- 172.21.15.131:0/3758244628 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40064e0 con 0x7fd4c0003a40 2024-09-17T02:51:59.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3758244628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d40cab60 con 0x7fd4c0003a40 2024-09-17T02:51:59.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3758244628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d40cb120 con 0x7fd4c0003a40 2024-09-17T02:51:59.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.322+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3758244628 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d4073310 con 0x7fd4c0003a40 2024-09-17T02:51:59.319 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.326+0000 7fd4caffd640 1 -- 172.21.15.131:0/3758244628 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fd4c4003620 con 0x7fd4c0003a40 2024-09-17T02:51:59.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.326+0000 7fd4caffd640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd4b4073200 0x7fd4b4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:59.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.326+0000 7fd4caffd640 1 -- 172.21.15.131:0/3758244628 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fd4c412e030 con 0x7fd4c0003a40 2024-09-17T02:51:59.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.326+0000 7fd4cbfff640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd4b4073200 0x7fd4b4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:59.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.326+0000 7fd4cbfff640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd4b4073200 0x7fd4b4075710 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006fd0 tx=0x7fd4bc006b90 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:59.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.330+0000 7fd4caffd640 1 -- 172.21.15.131:0/3758244628 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fd4c4016d10 con 0x7fd4c0003a40 2024-09-17T02:51:59.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.502+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3758244628 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd4d40c6330 con 0x7fd4b4073200 2024-09-17T02:51:59.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.514+0000 7fd4caffd640 1 -- 172.21.15.131:0/3758244628 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252479 (secure 0 0 0) 0x7fd4d40c6330 con 0x7fd4b4073200 2024-09-17T02:51:59.510 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:59.512 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:51:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd4b4073200 msgr2=0x7fd4b4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd4b4073200 0x7fd4b4075710 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc006fd0 tx=0x7fd4bc006b90 comp rx=0 tx=0).stop 2024-09-17T02:51:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3758244628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c0003a40 msgr2=0x7fd4d40cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c0003a40 0x7fd4d40cbc00 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fd4c401ca20 tx=0x7fd4c4006ca0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.513 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3758244628 shutdown_connections 2024-09-17T02:51:59.514 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fd4b4073200 0x7fd4b4075710 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.515 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fd4d415d040 0x7fd4d415d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fd4d414e5c0 0x7fd4d414d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 --2- 172.21.15.131:0/3758244628 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fd4c0003a40 0x7fd4d40cbc00 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3758244628 >> 172.21.15.131:0/3758244628 conn(0x7fd4d406eea0 msgr2=0x7fd4d4155090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3758244628 shutdown_connections 2024-09-17T02:51:59.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.518+0000 7fd4d88ed640 1 -- 172.21.15.131:0/3758244628 wait complete. 2024-09-17T02:51:59.532 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-17T02:51:59.391660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262496,"kb_used_data":12948,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712224,"statfs":{"total":773094113280,"available":772825317376,"internally_reserved":0,"allocated":13258752,"data_stored":6793992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002155"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.381354+0000","last_change":"2024-09-17T02:51:55.378936+0000","last_active":"2024-09-17T02:51:55.381354+0000","last_peered":"2024-09-17T02:51:55.381354+0000","last_clean":"2024-09-17T02:51:55.381354+0000","last_became_active":"2024-09-17T02:51:55.378614+0000","last_became_peered":"2024-09-17T02:51:55.378614+0000","last_unstale":"2024-09-17T02:51:55.381354+0000","last_undegraded":"2024-09-17T02:51:55.381354+0000","last_fullsized":"2024-09-17T02:51:55.381354+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:47.377205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407332+0000","last_change":"2024-09-17T02:51:55.376513+0000","last_active":"2024-09-17T02:51:55.407332+0000","last_peered":"2024-09-17T02:51:55.407332+0000","last_clean":"2024-09-17T02:51:55.407332+0000","last_became_active":"2024-09-17T02:51:55.373289+0000","last_became_peered":"2024-09-17T02:51:55.373289+0000","last_unstale":"2024-09-17T02:51:55.407332+0000","last_undegraded":"2024-09-17T02:51:55.407332+0000","last_fullsized":"2024-09-17T02:51:55.407332+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:50.329245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.421833+0000","last_change":"2024-09-17T02:51:55.418924+0000","last_active":"2024-09-17T02:51:55.421833+0000","last_peered":"2024-09-17T02:51:55.421833+0000","last_clean":"2024-09-17T02:51:55.421833+0000","last_became_active":"2024-09-17T02:51:55.418264+0000","last_became_peered":"2024-09-17T02:51:55.418264+0000","last_unstale":"2024-09-17T02:51:55.421833+0000","last_undegraded":"2024-09-17T02:51:55.421833+0000","last_fullsized":"2024-09-17T02:51:55.421833+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:59.226762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.382323+0000","last_change":"2024-09-17T02:51:55.380263+0000","last_active":"2024-09-17T02:51:55.382323+0000","last_peered":"2024-09-17T02:51:55.382323+0000","last_clean":"2024-09-17T02:51:55.382323+0000","last_became_active":"2024-09-17T02:51:55.379912+0000","last_became_peered":"2024-09-17T02:51:55.379912+0000","last_unstale":"2024-09-17T02:51:55.382323+0000","last_undegraded":"2024-09-17T02:51:55.382323+0000","last_fullsized":"2024-09-17T02:51:55.382323+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:25:16.740036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.376461+0000","last_change":"2024-09-17T02:51:55.371805+0000","last_active":"2024-09-17T02:51:55.376461+0000","last_peered":"2024-09-17T02:51:55.376461+0000","last_clean":"2024-09-17T02:51:55.376461+0000","last_became_active":"2024-09-17T02:51:55.370901+0000","last_became_peered":"2024-09-17T02:51:55.370901+0000","last_unstale":"2024-09-17T02:51:55.376461+0000","last_undegraded":"2024-09-17T02:51:55.376461+0000","last_fullsized":"2024-09-17T02:51:55.376461+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:03.119409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.408621+0000","last_change":"2024-09-17T02:51:55.381571+0000","last_active":"2024-09-17T02:51:55.408621+0000","last_peered":"2024-09-17T02:51:55.408621+0000","last_clean":"2024-09-17T02:51:55.408621+0000","last_became_active":"2024-09-17T02:51:55.380731+0000","last_became_peered":"2024-09-17T02:51:55.380731+0000","last_unstale":"2024-09-17T02:51:55.408621+0000","last_undegraded":"2024-09-17T02:51:55.408621+0000","last_fullsized":"2024-09-17T02:51:55.408621+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:39.528097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.417636+0000","last_change":"2024-09-17T02:51:55.393258+0000","last_active":"2024-09-17T02:51:55.417636+0000","last_peered":"2024-09-17T02:51:55.417636+0000","last_clean":"2024-09-17T02:51:55.417636+0000","last_became_active":"2024-09-17T02:51:55.374759+0000","last_became_peered":"2024-09-17T02:51:55.374759+0000","last_unstale":"2024-09-17T02:51:55.417636+0000","last_undegraded":"2024-09-17T02:51:55.417636+0000","last_fullsized":"2024-09-17T02:51:55.417636+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:37.603100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.401468+0000","last_change":"2024-09-17T02:51:55.381702+0000","last_active":"2024-09-17T02:51:55.401468+0000","last_peered":"2024-09-17T02:51:55.401468+0000","last_clean":"2024-09-17T02:51:55.401468+0000","last_became_active":"2024-09-17T02:51:55.379423+0000","last_became_peered":"2024-09-17T02:51:55.379423+0000","last_unstale":"2024-09-17T02:51:55.401468+0000","last_undegraded":"2024-09-17T02:51:55.401468+0000","last_fullsized":"2024-09-17T02:51:55.401468+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:01.505136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.393398+0000","last_change":"2024-09-17T02:51:55.392112+0000","last_active":"2024-09-17T02:51:55.393398+0000","last_peered":"2024-09-17T02:51:55.393398+0000","last_clean":"2024-09-17T02:51:55.393398+0000","last_became_active":"2024-09-17T02:51:55.391801+0000","last_became_peered":"2024-09-17T02:51:55.391801+0000","last_unstale":"2024-09-17T02:51:55.393398+0000","last_undegraded":"2024-09-17T02:51:55.393398+0000","last_fullsized":"2024-09-17T02:51:55.393398+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:42.403220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.419127+0000","last_change":"2024-09-17T02:51:55.414528+0000","last_active":"2024-09-17T02:51:55.419127+0000","last_peered":"2024-09-17T02:51:55.419127+0000","last_clean":"2024-09-17T02:51:55.419127+0000","last_became_active":"2024-09-17T02:51:55.413841+0000","last_became_peered":"2024-09-17T02:51:55.413841+0000","last_unstale":"2024-09-17T02:51:55.419127+0000","last_undegraded":"2024-09-17T02:51:55.419127+0000","last_fullsized":"2024-09-17T02:51:55.419127+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:25.784396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.365990+0000","last_change":"2024-09-17T02:51:55.363998+0000","last_active":"2024-09-17T02:51:55.365990+0000","last_peered":"2024-09-17T02:51:55.365990+0000","last_clean":"2024-09-17T02:51:55.365990+0000","last_became_active":"2024-09-17T02:51:55.363631+0000","last_became_peered":"2024-09-17T02:51:55.363631+0000","last_unstale":"2024-09-17T02:51:55.365990+0000","last_undegraded":"2024-09-17T02:51:55.365990+0000","last_fullsized":"2024-09-17T02:51:55.365990+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:35.285404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.360676+0000","last_change":"2024-09-17T02:51:55.409803+0000","last_active":"2024-09-17T02:51:56.360676+0000","last_peered":"2024-09-17T02:51:56.360676+0000","last_clean":"2024-09-17T02:51:56.360676+0000","last_became_active":"2024-09-17T02:51:55.397783+0000","last_became_peered":"2024-09-17T02:51:55.397783+0000","last_unstale":"2024-09-17T02:51:56.360676+0000","last_undegraded":"2024-09-17T02:51:56.360676+0000","last_fullsized":"2024-09-17T02:51:56.360676+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:52.805536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.411227+0000","last_change":"2024-09-17T02:51:55.379297+0000","last_active":"2024-09-17T02:51:55.411227+0000","last_peered":"2024-09-17T02:51:55.411227+0000","last_clean":"2024-09-17T02:51:55.411227+0000","last_became_active":"2024-09-17T02:51:55.378499+0000","last_became_peered":"2024-09-17T02:51:55.378499+0000","last_unstale":"2024-09-17T02:51:55.411227+0000","last_undegraded":"2024-09-17T02:51:55.411227+0000","last_fullsized":"2024-09-17T02:51:55.411227+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:58.117312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.360305+0000","last_change":"2024-09-17T02:51:55.397915+0000","last_active":"2024-09-17T02:51:56.360305+0000","last_peered":"2024-09-17T02:51:56.360305+0000","last_clean":"2024-09-17T02:51:56.360305+0000","last_became_active":"2024-09-17T02:51:55.390980+0000","last_became_peered":"2024-09-17T02:51:55.390980+0000","last_unstale":"2024-09-17T02:51:56.360305+0000","last_undegraded":"2024-09-17T02:51:56.360305+0000","last_fullsized":"2024-09-17T02:51:56.360305+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:42.410866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.370778+0000","last_change":"2024-09-17T02:51:55.368786+0000","last_active":"2024-09-17T02:51:55.370778+0000","last_peered":"2024-09-17T02:51:55.370778+0000","last_clean":"2024-09-17T02:51:55.370778+0000","last_became_active":"2024-09-17T02:51:55.368451+0000","last_became_peered":"2024-09-17T02:51:55.368451+0000","last_unstale":"2024-09-17T02:51:55.370778+0000","last_undegraded":"2024-09-17T02:51:55.370778+0000","last_fullsized":"2024-09-17T02:51:55.370778+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:20.498345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.359488+0000","last_change":"2024-09-17T02:51:55.409649+0000","last_active":"2024-09-17T02:51:56.359488+0000","last_peered":"2024-09-17T02:51:56.359488+0000","last_clean":"2024-09-17T02:51:56.359488+0000","last_became_active":"2024-09-17T02:51:55.408928+0000","last_became_peered":"2024-09-17T02:51:55.408928+0000","last_unstale":"2024-09-17T02:51:56.359488+0000","last_undegraded":"2024-09-17T02:51:56.359488+0000","last_fullsized":"2024-09-17T02:51:56.359488+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:19.532519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361378+0000","last_change":"2024-09-17T02:51:55.387322+0000","last_active":"2024-09-17T02:51:56.361378+0000","last_peered":"2024-09-17T02:51:56.361378+0000","last_clean":"2024-09-17T02:51:56.361378+0000","last_became_active":"2024-09-17T02:51:55.380758+0000","last_became_peered":"2024-09-17T02:51:55.380758+0000","last_unstale":"2024-09-17T02:51:56.361378+0000","last_undegraded":"2024-09-17T02:51:56.361378+0000","last_fullsized":"2024-09-17T02:51:56.361378+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:38.638277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361070+0000","last_change":"2024-09-17T02:51:55.398714+0000","last_active":"2024-09-17T02:51:56.361070+0000","last_peered":"2024-09-17T02:51:56.361070+0000","last_clean":"2024-09-17T02:51:56.361070+0000","last_became_active":"2024-09-17T02:51:55.397494+0000","last_became_peered":"2024-09-17T02:51:55.397494+0000","last_unstale":"2024-09-17T02:51:56.361070+0000","last_undegraded":"2024-09-17T02:51:56.361070+0000","last_fullsized":"2024-09-17T02:51:56.361070+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:23.607222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.398530+0000","last_change":"2024-09-17T02:51:55.397902+0000","last_active":"2024-09-17T02:51:55.398530+0000","last_peered":"2024-09-17T02:51:55.398530+0000","last_clean":"2024-09-17T02:51:55.398530+0000","last_became_active":"2024-09-17T02:51:55.397627+0000","last_became_peered":"2024-09-17T02:51:55.397627+0000","last_unstale":"2024-09-17T02:51:55.398530+0000","last_undegraded":"2024-09-17T02:51:55.398530+0000","last_fullsized":"2024-09-17T02:51:55.398530+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:25.309851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.425328+0000","last_change":"2024-09-17T02:51:55.424359+0000","last_active":"2024-09-17T02:51:55.425328+0000","last_peered":"2024-09-17T02:51:55.425328+0000","last_clean":"2024-09-17T02:51:55.425328+0000","last_became_active":"2024-09-17T02:51:55.422988+0000","last_became_peered":"2024-09-17T02:51:55.422988+0000","last_unstale":"2024-09-17T02:51:55.425328+0000","last_undegraded":"2024-09-17T02:51:55.425328+0000","last_fullsized":"2024-09-17T02:51:55.425328+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:53.383522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.383093+0000","last_change":"2024-09-17T02:51:55.376210+0000","last_active":"2024-09-17T02:51:55.383093+0000","last_peered":"2024-09-17T02:51:55.383093+0000","last_clean":"2024-09-17T02:51:55.383093+0000","last_became_active":"2024-09-17T02:51:55.375879+0000","last_became_peered":"2024-09-17T02:51:55.375879+0000","last_unstale":"2024-09-17T02:51:55.383093+0000","last_undegraded":"2024-09-17T02:51:55.383093+0000","last_fullsized":"2024-09-17T02:51:55.383093+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:31.171690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.393117+0000","last_change":"2024-09-17T02:51:55.392110+0000","last_active":"2024-09-17T02:51:55.393117+0000","last_peered":"2024-09-17T02:51:55.393117+0000","last_clean":"2024-09-17T02:51:55.393117+0000","last_became_active":"2024-09-17T02:51:55.391710+0000","last_became_peered":"2024-09-17T02:51:55.391710+0000","last_unstale":"2024-09-17T02:51:55.393117+0000","last_undegraded":"2024-09-17T02:51:55.393117+0000","last_fullsized":"2024-09-17T02:51:55.393117+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:14.229831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.370732+0000","last_change":"2024-09-17T02:51:55.368623+0000","last_active":"2024-09-17T02:51:55.370732+0000","last_peered":"2024-09-17T02:51:55.370732+0000","last_clean":"2024-09-17T02:51:55.370732+0000","last_became_active":"2024-09-17T02:51:55.368272+0000","last_became_peered":"2024-09-17T02:51:55.368272+0000","last_unstale":"2024-09-17T02:51:55.370732+0000","last_undegraded":"2024-09-17T02:51:55.370732+0000","last_fullsized":"2024-09-17T02:51:55.370732+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:31.171690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.393099+0000","last_change":"2024-09-17T02:51:55.392100+0000","last_active":"2024-09-17T02:51:55.393099+0000","last_peered":"2024-09-17T02:51:55.393099+0000","last_clean":"2024-09-17T02:51:55.393099+0000","last_became_active":"2024-09-17T02:51:55.391737+0000","last_became_peered":"2024-09-17T02:51:55.391737+0000","last_unstale":"2024-09-17T02:51:55.393099+0000","last_undegraded":"2024-09-17T02:51:55.393099+0000","last_fullsized":"2024-09-17T02:51:55.393099+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:25.309851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.381404+0000","last_change":"2024-09-17T02:51:55.378538+0000","last_active":"2024-09-17T02:51:55.381404+0000","last_peered":"2024-09-17T02:51:55.381404+0000","last_clean":"2024-09-17T02:51:55.381404+0000","last_became_active":"2024-09-17T02:51:55.378090+0000","last_became_peered":"2024-09-17T02:51:55.378090+0000","last_unstale":"2024-09-17T02:51:55.381404+0000","last_undegraded":"2024-09-17T02:51:55.381404+0000","last_fullsized":"2024-09-17T02:51:55.381404+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:02.410189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.401637+0000","last_change":"2024-09-17T02:51:55.385458+0000","last_active":"2024-09-17T02:51:55.401637+0000","last_peered":"2024-09-17T02:51:55.401637+0000","last_clean":"2024-09-17T02:51:55.401637+0000","last_became_active":"2024-09-17T02:51:55.384568+0000","last_became_peered":"2024-09-17T02:51:55.384568+0000","last_unstale":"2024-09-17T02:51:55.401637+0000","last_undegraded":"2024-09-17T02:51:55.401637+0000","last_fullsized":"2024-09-17T02:51:55.401637+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:59.463669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.376280+0000","last_change":"2024-09-17T02:51:55.372167+0000","last_active":"2024-09-17T02:51:55.376280+0000","last_peered":"2024-09-17T02:51:55.376280+0000","last_clean":"2024-09-17T02:51:55.376280+0000","last_became_active":"2024-09-17T02:51:55.371631+0000","last_became_peered":"2024-09-17T02:51:55.371631+0000","last_unstale":"2024-09-17T02:51:55.376280+0000","last_undegraded":"2024-09-17T02:51:55.376280+0000","last_fullsized":"2024-09-17T02:51:55.376280+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:21.696232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361160+0000","last_change":"2024-09-17T02:51:55.422230+0000","last_active":"2024-09-17T02:51:56.361160+0000","last_peered":"2024-09-17T02:51:56.361160+0000","last_clean":"2024-09-17T02:51:56.361160+0000","last_became_active":"2024-09-17T02:51:55.410789+0000","last_became_peered":"2024-09-17T02:51:55.410789+0000","last_unstale":"2024-09-17T02:51:56.361160+0000","last_undegraded":"2024-09-17T02:51:56.361160+0000","last_fullsized":"2024-09-17T02:51:56.361160+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:03.762220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.421775+0000","last_change":"2024-09-17T02:51:55.420322+0000","last_active":"2024-09-17T02:51:55.421775+0000","last_peered":"2024-09-17T02:51:55.421775+0000","last_clean":"2024-09-17T02:51:55.421775+0000","last_became_active":"2024-09-17T02:51:55.419647+0000","last_became_peered":"2024-09-17T02:51:55.419647+0000","last_unstale":"2024-09-17T02:51:55.421775+0000","last_undegraded":"2024-09-17T02:51:55.421775+0000","last_fullsized":"2024-09-17T02:51:55.421775+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:29.524549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407373+0000","last_change":"2024-09-17T02:51:55.404030+0000","last_active":"2024-09-17T02:51:55.407373+0000","last_peered":"2024-09-17T02:51:55.407373+0000","last_clean":"2024-09-17T02:51:55.407373+0000","last_became_active":"2024-09-17T02:51:55.402207+0000","last_became_peered":"2024-09-17T02:51:55.402207+0000","last_unstale":"2024-09-17T02:51:55.407373+0000","last_undegraded":"2024-09-17T02:51:55.407373+0000","last_fullsized":"2024-09-17T02:51:55.407373+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:28.068092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.393800+0000","last_change":"2024-09-17T02:51:55.393010+0000","last_active":"2024-09-17T02:51:55.393800+0000","last_peered":"2024-09-17T02:51:55.393800+0000","last_clean":"2024-09-17T02:51:55.393800+0000","last_became_active":"2024-09-17T02:51:55.392693+0000","last_became_peered":"2024-09-17T02:51:55.392693+0000","last_unstale":"2024-09-17T02:51:55.393800+0000","last_undegraded":"2024-09-17T02:51:55.393800+0000","last_fullsized":"2024-09-17T02:51:55.393800+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:40.722597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.385699+0000","last_change":"2024-09-17T02:51:55.385244+0000","last_active":"2024-09-17T02:51:55.385699+0000","last_peered":"2024-09-17T02:51:55.385699+0000","last_clean":"2024-09-17T02:51:55.385699+0000","last_became_active":"2024-09-17T02:51:55.384888+0000","last_became_peered":"2024-09-17T02:51:55.384888+0000","last_unstale":"2024-09-17T02:51:55.385699+0000","last_undegraded":"2024-09-17T02:51:55.385699+0000","last_fullsized":"2024-09-17T02:51:55.385699+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.312656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.416048+0000","last_change":"2024-09-17T02:51:55.404830+0000","last_active":"2024-09-17T02:51:55.416048+0000","last_peered":"2024-09-17T02:51:55.416048+0000","last_clean":"2024-09-17T02:51:55.416048+0000","last_became_active":"2024-09-17T02:51:55.403942+0000","last_became_peered":"2024-09-17T02:51:55.403942+0000","last_unstale":"2024-09-17T02:51:55.416048+0000","last_undegraded":"2024-09-17T02:51:55.416048+0000","last_fullsized":"2024-09-17T02:51:55.416048+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:01.987795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"121'4","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.384079+0000","last_change":"2024-09-17T02:51:54.375067+0000","last_active":"2024-09-17T02:51:55.384079+0000","last_peered":"2024-09-17T02:51:55.384079+0000","last_clean":"2024-09-17T02:51:55.384079+0000","last_became_active":"2024-09-17T02:51:54.374709+0000","last_became_peered":"2024-09-17T02:51:54.374709+0000","last_unstale":"2024-09-17T02:51:55.384079+0000","last_undegraded":"2024-09-17T02:51:55.384079+0000","last_fullsized":"2024-09-17T02:51:55.384079+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:05.729763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.418747+0000","last_change":"2024-09-17T02:51:55.414787+0000","last_active":"2024-09-17T02:51:55.418747+0000","last_peered":"2024-09-17T02:51:55.418747+0000","last_clean":"2024-09-17T02:51:55.418747+0000","last_became_active":"2024-09-17T02:51:55.414145+0000","last_became_peered":"2024-09-17T02:51:55.414145+0000","last_unstale":"2024-09-17T02:51:55.418747+0000","last_undegraded":"2024-09-17T02:51:55.418747+0000","last_fullsized":"2024-09-17T02:51:55.418747+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:43.362915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.360702+0000","last_change":"2024-09-17T02:51:54.389172+0000","last_active":"2024-09-17T02:51:56.360702+0000","last_peered":"2024-09-17T02:51:56.360702+0000","last_clean":"2024-09-17T02:51:56.360702+0000","last_became_active":"2024-09-17T02:51:54.388296+0000","last_became_peered":"2024-09-17T02:51:54.388296+0000","last_unstale":"2024-09-17T02:51:56.360702+0000","last_undegraded":"2024-09-17T02:51:56.360702+0000","last_fullsized":"2024-09-17T02:51:56.360702+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:52.024157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.375533+0000","last_change":"2024-09-17T02:51:55.373263+0000","last_active":"2024-09-17T02:51:55.375533+0000","last_peered":"2024-09-17T02:51:55.375533+0000","last_clean":"2024-09-17T02:51:55.375533+0000","last_became_active":"2024-09-17T02:51:55.372892+0000","last_became_peered":"2024-09-17T02:51:55.372892+0000","last_unstale":"2024-09-17T02:51:55.375533+0000","last_undegraded":"2024-09-17T02:51:55.375533+0000","last_fullsized":"2024-09-17T02:51:55.375533+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:50.535681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"121'2","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.387709+0000","last_change":"2024-09-17T02:51:54.377679+0000","last_active":"2024-09-17T02:51:55.387709+0000","last_peered":"2024-09-17T02:51:55.387709+0000","last_clean":"2024-09-17T02:51:55.387709+0000","last_became_active":"2024-09-17T02:51:54.377348+0000","last_became_peered":"2024-09-17T02:51:54.377348+0000","last_unstale":"2024-09-17T02:51:55.387709+0000","last_undegraded":"2024-09-17T02:51:55.387709+0000","last_fullsized":"2024-09-17T02:51:55.387709+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:02.115523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.375417+0000","last_change":"2024-09-17T02:51:55.374491+0000","last_active":"2024-09-17T02:51:55.375417+0000","last_peered":"2024-09-17T02:51:55.375417+0000","last_clean":"2024-09-17T02:51:55.375417+0000","last_became_active":"2024-09-17T02:51:55.374207+0000","last_became_peered":"2024-09-17T02:51:55.374207+0000","last_unstale":"2024-09-17T02:51:55.375417+0000","last_undegraded":"2024-09-17T02:51:55.375417+0000","last_fullsized":"2024-09-17T02:51:55.375417+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:39.318119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"121'2","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.424968+0000","last_change":"2024-09-17T02:51:54.399591+0000","last_active":"2024-09-17T02:51:55.424968+0000","last_peered":"2024-09-17T02:51:55.424968+0000","last_clean":"2024-09-17T02:51:55.424968+0000","last_became_active":"2024-09-17T02:51:54.398462+0000","last_became_peered":"2024-09-17T02:51:54.398462+0000","last_unstale":"2024-09-17T02:51:55.424968+0000","last_undegraded":"2024-09-17T02:51:55.424968+0000","last_fullsized":"2024-09-17T02:51:55.424968+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:53.854955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.401537+0000","last_change":"2024-09-17T02:51:55.382560+0000","last_active":"2024-09-17T02:51:55.401537+0000","last_peered":"2024-09-17T02:51:55.401537+0000","last_clean":"2024-09-17T02:51:55.401537+0000","last_became_active":"2024-09-17T02:51:55.379192+0000","last_became_peered":"2024-09-17T02:51:55.379192+0000","last_unstale":"2024-09-17T02:51:55.401537+0000","last_undegraded":"2024-09-17T02:51:55.401537+0000","last_fullsized":"2024-09-17T02:51:55.401537+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:29.134240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"121'1","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.383777+0000","last_change":"2024-09-17T02:51:54.384242+0000","last_active":"2024-09-17T02:51:55.383777+0000","last_peered":"2024-09-17T02:51:55.383777+0000","last_clean":"2024-09-17T02:51:55.383777+0000","last_became_active":"2024-09-17T02:51:54.383864+0000","last_became_peered":"2024-09-17T02:51:54.383864+0000","last_unstale":"2024-09-17T02:51:55.383777+0000","last_undegraded":"2024-09-17T02:51:55.383777+0000","last_fullsized":"2024-09-17T02:51:55.383777+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:23.756438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.369855+0000","last_change":"2024-09-17T02:51:55.368614+0000","last_active":"2024-09-17T02:51:55.369855+0000","last_peered":"2024-09-17T02:51:55.369855+0000","last_clean":"2024-09-17T02:51:55.369855+0000","last_became_active":"2024-09-17T02:51:55.368268+0000","last_became_peered":"2024-09-17T02:51:55.368268+0000","last_unstale":"2024-09-17T02:51:55.369855+0000","last_undegraded":"2024-09-17T02:51:55.369855+0000","last_fullsized":"2024-09-17T02:51:55.369855+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:59.402992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"121'1","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.413758+0000","last_change":"2024-09-17T02:51:54.397805+0000","last_active":"2024-09-17T02:51:55.413758+0000","last_peered":"2024-09-17T02:51:55.413758+0000","last_clean":"2024-09-17T02:51:55.413758+0000","last_became_active":"2024-09-17T02:51:54.397281+0000","last_became_peered":"2024-09-17T02:51:54.397281+0000","last_unstale":"2024-09-17T02:51:55.413758+0000","last_undegraded":"2024-09-17T02:51:55.413758+0000","last_fullsized":"2024-09-17T02:51:55.413758+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:38.560730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.419356+0000","last_change":"2024-09-17T02:51:55.415382+0000","last_active":"2024-09-17T02:51:55.419356+0000","last_peered":"2024-09-17T02:51:55.419356+0000","last_clean":"2024-09-17T02:51:55.419356+0000","last_became_active":"2024-09-17T02:51:55.414394+0000","last_became_peered":"2024-09-17T02:51:55.414394+0000","last_unstale":"2024-09-17T02:51:55.419356+0000","last_undegraded":"2024-09-17T02:51:55.419356+0000","last_fullsized":"2024-09-17T02:51:55.419356+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:29.622277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.362909+0000","last_change":"2024-09-17T02:51:54.369369+0000","last_active":"2024-09-17T02:51:55.362909+0000","last_peered":"2024-09-17T02:51:55.362909+0000","last_clean":"2024-09-17T02:51:55.362909+0000","last_became_active":"2024-09-17T02:51:54.369043+0000","last_became_peered":"2024-09-17T02:51:54.369043+0000","last_unstale":"2024-09-17T02:51:55.362909+0000","last_undegraded":"2024-09-17T02:51:55.362909+0000","last_fullsized":"2024-09-17T02:51:55.362909+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:10.599327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.403576+0000","last_change":"2024-09-17T02:51:55.381824+0000","last_active":"2024-09-17T02:51:55.403576+0000","last_peered":"2024-09-17T02:51:55.403576+0000","last_clean":"2024-09-17T02:51:55.403576+0000","last_became_active":"2024-09-17T02:51:55.380948+0000","last_became_peered":"2024-09-17T02:51:55.380948+0000","last_unstale":"2024-09-17T02:51:55.403576+0000","last_undegraded":"2024-09-17T02:51:55.403576+0000","last_fullsized":"2024-09-17T02:51:55.403576+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:03.359056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.387327+0000","last_change":"2024-09-17T02:51:54.396926+0000","last_active":"2024-09-17T02:51:55.387327+0000","last_peered":"2024-09-17T02:51:55.387327+0000","last_clean":"2024-09-17T02:51:55.387327+0000","last_became_active":"2024-09-17T02:51:54.396197+0000","last_became_peered":"2024-09-17T02:51:54.396197+0000","last_unstale":"2024-09-17T02:51:55.387327+0000","last_undegraded":"2024-09-17T02:51:55.387327+0000","last_fullsized":"2024-09-17T02:51:55.387327+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:25.132313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.373844+0000","last_change":"2024-09-17T02:51:55.372506+0000","last_active":"2024-09-17T02:51:55.373844+0000","last_peered":"2024-09-17T02:51:55.373844+0000","last_clean":"2024-09-17T02:51:55.373844+0000","last_became_active":"2024-09-17T02:51:55.372198+0000","last_became_peered":"2024-09-17T02:51:55.372198+0000","last_unstale":"2024-09-17T02:51:55.373844+0000","last_undegraded":"2024-09-17T02:51:55.373844+0000","last_fullsized":"2024-09-17T02:51:55.373844+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:26.353572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.365094+0000","last_change":"2024-09-17T02:51:54.384344+0000","last_active":"2024-09-17T02:51:55.365094+0000","last_peered":"2024-09-17T02:51:55.365094+0000","last_clean":"2024-09-17T02:51:55.365094+0000","last_became_active":"2024-09-17T02:51:54.383955+0000","last_became_peered":"2024-09-17T02:51:54.383955+0000","last_unstale":"2024-09-17T02:51:55.365094+0000","last_undegraded":"2024-09-17T02:51:55.365094+0000","last_fullsized":"2024-09-17T02:51:55.365094+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:21.292617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.382144+0000","last_change":"2024-09-17T02:51:55.379732+0000","last_active":"2024-09-17T02:51:55.382144+0000","last_peered":"2024-09-17T02:51:55.382144+0000","last_clean":"2024-09-17T02:51:55.382144+0000","last_became_active":"2024-09-17T02:51:55.379297+0000","last_became_peered":"2024-09-17T02:51:55.379297+0000","last_unstale":"2024-09-17T02:51:55.382144+0000","last_undegraded":"2024-09-17T02:51:55.382144+0000","last_fullsized":"2024-09-17T02:51:55.382144+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:12.080162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.359655+0000","last_change":"2024-09-17T02:51:54.390386+0000","last_active":"2024-09-17T02:51:55.359655+0000","last_peered":"2024-09-17T02:51:55.359655+0000","last_clean":"2024-09-17T02:51:55.359655+0000","last_became_active":"2024-09-17T02:51:54.389933+0000","last_became_peered":"2024-09-17T02:51:54.389933+0000","last_unstale":"2024-09-17T02:51:55.359655+0000","last_undegraded":"2024-09-17T02:51:55.359655+0000","last_fullsized":"2024-09-17T02:51:55.359655+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:03:34.249461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407036+0000","last_change":"2024-09-17T02:51:55.404319+0000","last_active":"2024-09-17T02:51:55.407036+0000","last_peered":"2024-09-17T02:51:55.407036+0000","last_clean":"2024-09-17T02:51:55.407036+0000","last_became_active":"2024-09-17T02:51:55.402307+0000","last_became_peered":"2024-09-17T02:51:55.402307+0000","last_unstale":"2024-09-17T02:51:55.407036+0000","last_undegraded":"2024-09-17T02:51:55.407036+0000","last_fullsized":"2024-09-17T02:51:55.407036+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:58:02.723893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"121'3","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.383585+0000","last_change":"2024-09-17T02:51:54.366554+0000","last_active":"2024-09-17T02:51:55.383585+0000","last_peered":"2024-09-17T02:51:55.383585+0000","last_clean":"2024-09-17T02:51:55.383585+0000","last_became_active":"2024-09-17T02:51:54.366162+0000","last_became_peered":"2024-09-17T02:51:54.366162+0000","last_unstale":"2024-09-17T02:51:55.383585+0000","last_undegraded":"2024-09-17T02:51:55.383585+0000","last_fullsized":"2024-09-17T02:51:55.383585+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:00.221175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.383145+0000","last_change":"2024-09-17T02:51:55.376222+0000","last_active":"2024-09-17T02:51:55.383145+0000","last_peered":"2024-09-17T02:51:55.383145+0000","last_clean":"2024-09-17T02:51:55.383145+0000","last_became_active":"2024-09-17T02:51:55.375846+0000","last_became_peered":"2024-09-17T02:51:55.375846+0000","last_unstale":"2024-09-17T02:51:55.383145+0000","last_undegraded":"2024-09-17T02:51:55.383145+0000","last_fullsized":"2024-09-17T02:51:55.383145+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:20.498345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"121'6","reported_seq":19,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.425174+0000","last_change":"2024-09-17T02:51:54.378308+0000","last_active":"2024-09-17T02:51:55.425174+0000","last_peered":"2024-09-17T02:51:55.425174+0000","last_clean":"2024-09-17T02:51:55.425174+0000","last_became_active":"2024-09-17T02:51:54.377162+0000","last_became_peered":"2024-09-17T02:51:54.377162+0000","last_unstale":"2024-09-17T02:51:55.425174+0000","last_undegraded":"2024-09-17T02:51:55.425174+0000","last_fullsized":"2024-09-17T02:51:55.425174+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:06.691611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.389810+0000","last_change":"2024-09-17T02:51:55.386483+0000","last_active":"2024-09-17T02:51:55.389810+0000","last_peered":"2024-09-17T02:51:55.389810+0000","last_clean":"2024-09-17T02:51:55.389810+0000","last_became_active":"2024-09-17T02:51:55.386102+0000","last_became_peered":"2024-09-17T02:51:55.386102+0000","last_unstale":"2024-09-17T02:51:55.389810+0000","last_undegraded":"2024-09-17T02:51:55.389810+0000","last_fullsized":"2024-09-17T02:51:55.389810+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:26.353572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"121'2","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.415882+0000","last_change":"2024-09-17T02:51:54.393704+0000","last_active":"2024-09-17T02:51:55.415882+0000","last_peered":"2024-09-17T02:51:55.415882+0000","last_clean":"2024-09-17T02:51:55.415882+0000","last_became_active":"2024-09-17T02:51:54.386613+0000","last_became_peered":"2024-09-17T02:51:54.386613+0000","last_unstale":"2024-09-17T02:51:55.415882+0000","last_undegraded":"2024-09-17T02:51:55.415882+0000","last_fullsized":"2024-09-17T02:51:55.415882+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:06.226314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407647+0000","last_change":"2024-09-17T02:51:55.374737+0000","last_active":"2024-09-17T02:51:55.407647+0000","last_peered":"2024-09-17T02:51:55.407647+0000","last_clean":"2024-09-17T02:51:55.407647+0000","last_became_active":"2024-09-17T02:51:55.373464+0000","last_became_peered":"2024-09-17T02:51:55.373464+0000","last_unstale":"2024-09-17T02:51:55.407647+0000","last_undegraded":"2024-09-17T02:51:55.407647+0000","last_fullsized":"2024-09-17T02:51:55.407647+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:55.947251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.392051+0000","last_change":"2024-09-17T02:51:54.385926+0000","last_active":"2024-09-17T02:51:55.392051+0000","last_peered":"2024-09-17T02:51:55.392051+0000","last_clean":"2024-09-17T02:51:55.392051+0000","last_became_active":"2024-09-17T02:51:54.385059+0000","last_became_peered":"2024-09-17T02:51:54.385059+0000","last_unstale":"2024-09-17T02:51:55.392051+0000","last_undegraded":"2024-09-17T02:51:55.392051+0000","last_fullsized":"2024-09-17T02:51:55.392051+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:33:41.467258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.360988+0000","last_change":"2024-09-17T02:51:55.397941+0000","last_active":"2024-09-17T02:51:56.360988+0000","last_peered":"2024-09-17T02:51:56.360988+0000","last_clean":"2024-09-17T02:51:56.360988+0000","last_became_active":"2024-09-17T02:51:55.397151+0000","last_became_peered":"2024-09-17T02:51:55.397151+0000","last_unstale":"2024-09-17T02:51:56.360988+0000","last_undegraded":"2024-09-17T02:51:56.360988+0000","last_fullsized":"2024-09-17T02:51:56.360988+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:29.462526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.362468+0000","last_change":"2024-09-17T02:51:54.370628+0000","last_active":"2024-09-17T02:51:55.362468+0000","last_peered":"2024-09-17T02:51:55.362468+0000","last_clean":"2024-09-17T02:51:55.362468+0000","last_became_active":"2024-09-17T02:51:54.369820+0000","last_became_peered":"2024-09-17T02:51:54.369820+0000","last_unstale":"2024-09-17T02:51:55.362468+0000","last_undegraded":"2024-09-17T02:51:55.362468+0000","last_fullsized":"2024-09-17T02:51:55.362468+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:10.599327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.392961+0000","last_change":"2024-09-17T02:51:55.390665+0000","last_active":"2024-09-17T02:51:55.392961+0000","last_peered":"2024-09-17T02:51:55.392961+0000","last_clean":"2024-09-17T02:51:55.392961+0000","last_became_active":"2024-09-17T02:51:55.390364+0000","last_became_peered":"2024-09-17T02:51:55.390364+0000","last_unstale":"2024-09-17T02:51:55.392961+0000","last_undegraded":"2024-09-17T02:51:55.392961+0000","last_fullsized":"2024-09-17T02:51:55.392961+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:30.741037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361568+0000","last_change":"2024-09-17T02:51:54.389138+0000","last_active":"2024-09-17T02:51:56.361568+0000","last_peered":"2024-09-17T02:51:56.361568+0000","last_clean":"2024-09-17T02:51:56.361568+0000","last_became_active":"2024-09-17T02:51:54.388340+0000","last_became_peered":"2024-09-17T02:51:54.388340+0000","last_unstale":"2024-09-17T02:51:56.361568+0000","last_undegraded":"2024-09-17T02:51:56.361568+0000","last_fullsized":"2024-09-17T02:51:56.361568+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:13:27.090224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.389765+0000","last_change":"2024-09-17T02:51:55.386954+0000","last_active":"2024-09-17T02:51:55.389765+0000","last_peered":"2024-09-17T02:51:55.389765+0000","last_clean":"2024-09-17T02:51:55.389765+0000","last_became_active":"2024-09-17T02:51:55.386631+0000","last_became_peered":"2024-09-17T02:51:55.386631+0000","last_unstale":"2024-09-17T02:51:55.389765+0000","last_undegraded":"2024-09-17T02:51:55.389765+0000","last_fullsized":"2024-09-17T02:51:55.389765+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:34.172743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361071+0000","last_change":"2024-09-17T02:51:54.398592+0000","last_active":"2024-09-17T02:51:56.361071+0000","last_peered":"2024-09-17T02:51:56.361071+0000","last_clean":"2024-09-17T02:51:56.361071+0000","last_became_active":"2024-09-17T02:51:54.397941+0000","last_became_peered":"2024-09-17T02:51:54.397941+0000","last_unstale":"2024-09-17T02:51:56.361071+0000","last_undegraded":"2024-09-17T02:51:56.361071+0000","last_fullsized":"2024-09-17T02:51:56.361071+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:25.022321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.406739+0000","last_change":"2024-09-17T02:51:55.376825+0000","last_active":"2024-09-17T02:51:55.406739+0000","last_peered":"2024-09-17T02:51:55.406739+0000","last_clean":"2024-09-17T02:51:55.406739+0000","last_became_active":"2024-09-17T02:51:55.374321+0000","last_became_peered":"2024-09-17T02:51:55.374321+0000","last_unstale":"2024-09-17T02:51:55.406739+0000","last_undegraded":"2024-09-17T02:51:55.406739+0000","last_fullsized":"2024-09-17T02:51:55.406739+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:14.958103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"121'1","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.383643+0000","last_change":"2024-09-17T02:51:54.374570+0000","last_active":"2024-09-17T02:51:55.383643+0000","last_peered":"2024-09-17T02:51:55.383643+0000","last_clean":"2024-09-17T02:51:55.383643+0000","last_became_active":"2024-09-17T02:51:54.374151+0000","last_became_peered":"2024-09-17T02:51:54.374151+0000","last_unstale":"2024-09-17T02:51:55.383643+0000","last_undegraded":"2024-09-17T02:51:55.383643+0000","last_fullsized":"2024-09-17T02:51:55.383643+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:31.577585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.377377+0000","last_change":"2024-09-17T02:51:55.372551+0000","last_active":"2024-09-17T02:51:55.377377+0000","last_peered":"2024-09-17T02:51:55.377377+0000","last_clean":"2024-09-17T02:51:55.377377+0000","last_became_active":"2024-09-17T02:51:55.372224+0000","last_became_peered":"2024-09-17T02:51:55.372224+0000","last_unstale":"2024-09-17T02:51:55.377377+0000","last_undegraded":"2024-09-17T02:51:55.377377+0000","last_fullsized":"2024-09-17T02:51:55.377377+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:16.522450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.360327+0000","last_change":"2024-09-17T02:51:54.398562+0000","last_active":"2024-09-17T02:51:55.360327+0000","last_peered":"2024-09-17T02:51:55.360327+0000","last_clean":"2024-09-17T02:51:55.360327+0000","last_became_active":"2024-09-17T02:51:54.398210+0000","last_became_peered":"2024-09-17T02:51:54.398210+0000","last_unstale":"2024-09-17T02:51:55.360327+0000","last_undegraded":"2024-09-17T02:51:55.360327+0000","last_fullsized":"2024-09-17T02:51:55.360327+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:39.140524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.409635+0000","last_change":"2024-09-17T02:51:55.383904+0000","last_active":"2024-09-17T02:51:55.409635+0000","last_peered":"2024-09-17T02:51:55.409635+0000","last_clean":"2024-09-17T02:51:55.409635+0000","last_became_active":"2024-09-17T02:51:55.383089+0000","last_became_peered":"2024-09-17T02:51:55.383089+0000","last_unstale":"2024-09-17T02:51:55.409635+0000","last_undegraded":"2024-09-17T02:51:55.409635+0000","last_fullsized":"2024-09-17T02:51:55.409635+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:09.767083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.359400+0000","last_change":"2024-09-17T02:51:54.367975+0000","last_active":"2024-09-17T02:51:56.359400+0000","last_peered":"2024-09-17T02:51:56.359400+0000","last_clean":"2024-09-17T02:51:56.359400+0000","last_became_active":"2024-09-17T02:51:54.365955+0000","last_became_peered":"2024-09-17T02:51:54.365955+0000","last_unstale":"2024-09-17T02:51:56.359400+0000","last_undegraded":"2024-09-17T02:51:56.359400+0000","last_fullsized":"2024-09-17T02:51:56.359400+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:33:58.920111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.359324+0000","last_change":"2024-09-17T02:51:55.388787+0000","last_active":"2024-09-17T02:51:56.359324+0000","last_peered":"2024-09-17T02:51:56.359324+0000","last_clean":"2024-09-17T02:51:56.359324+0000","last_became_active":"2024-09-17T02:51:55.387724+0000","last_became_peered":"2024-09-17T02:51:55.387724+0000","last_unstale":"2024-09-17T02:51:56.359324+0000","last_undegraded":"2024-09-17T02:51:56.359324+0000","last_fullsized":"2024-09-17T02:51:56.359324+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:54.122842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.359056+0000","last_change":"2024-09-17T02:51:54.361303+0000","last_active":"2024-09-17T02:51:55.359056+0000","last_peered":"2024-09-17T02:51:55.359056+0000","last_clean":"2024-09-17T02:51:55.359056+0000","last_became_active":"2024-09-17T02:51:54.360915+0000","last_became_peered":"2024-09-17T02:51:54.360915+0000","last_unstale":"2024-09-17T02:51:55.359056+0000","last_undegraded":"2024-09-17T02:51:55.359056+0000","last_fullsized":"2024-09-17T02:51:55.359056+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:15.518563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.412552+0000","last_change":"2024-09-17T02:51:55.405476+0000","last_active":"2024-09-17T02:51:55.412552+0000","last_peered":"2024-09-17T02:51:55.412552+0000","last_clean":"2024-09-17T02:51:55.412552+0000","last_became_active":"2024-09-17T02:51:55.404359+0000","last_became_peered":"2024-09-17T02:51:55.404359+0000","last_unstale":"2024-09-17T02:51:55.412552+0000","last_undegraded":"2024-09-17T02:51:55.412552+0000","last_fullsized":"2024-09-17T02:51:55.412552+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:14.916860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.387334+0000","last_change":"2024-09-17T02:51:54.398509+0000","last_active":"2024-09-17T02:51:55.387334+0000","last_peered":"2024-09-17T02:51:55.387334+0000","last_clean":"2024-09-17T02:51:55.387334+0000","last_became_active":"2024-09-17T02:51:54.397984+0000","last_became_peered":"2024-09-17T02:51:54.397984+0000","last_unstale":"2024-09-17T02:51:55.387334+0000","last_undegraded":"2024-09-17T02:51:55.387334+0000","last_fullsized":"2024-09-17T02:51:55.387334+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:51.217194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.387401+0000","last_change":"2024-09-17T02:51:55.386639+0000","last_active":"2024-09-17T02:51:55.387401+0000","last_peered":"2024-09-17T02:51:55.387401+0000","last_clean":"2024-09-17T02:51:55.387401+0000","last_became_active":"2024-09-17T02:51:55.386140+0000","last_became_peered":"2024-09-17T02:51:55.386140+0000","last_unstale":"2024-09-17T02:51:55.387401+0000","last_undegraded":"2024-09-17T02:51:55.387401+0000","last_fullsized":"2024-09-17T02:51:55.387401+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:29.529535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.352180+0000","last_change":"2024-09-17T02:51:54.397929+0000","last_active":"2024-09-17T02:51:55.352180+0000","last_peered":"2024-09-17T02:51:55.352180+0000","last_clean":"2024-09-17T02:51:55.352180+0000","last_became_active":"2024-09-17T02:51:54.397540+0000","last_became_peered":"2024-09-17T02:51:54.397540+0000","last_unstale":"2024-09-17T02:51:55.352180+0000","last_undegraded":"2024-09-17T02:51:55.352180+0000","last_fullsized":"2024-09-17T02:51:55.352180+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:15.364789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361406+0000","last_change":"2024-09-17T02:51:55.410609+0000","last_active":"2024-09-17T02:51:56.361406+0000","last_peered":"2024-09-17T02:51:56.361406+0000","last_clean":"2024-09-17T02:51:56.361406+0000","last_became_active":"2024-09-17T02:51:55.398900+0000","last_became_peered":"2024-09-17T02:51:55.398900+0000","last_unstale":"2024-09-17T02:51:56.361406+0000","last_undegraded":"2024-09-17T02:51:56.361406+0000","last_fullsized":"2024-09-17T02:51:56.361406+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:47.994917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"121'2","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.418548+0000","last_change":"2024-09-17T02:51:54.391460+0000","last_active":"2024-09-17T02:51:55.418548+0000","last_peered":"2024-09-17T02:51:55.418548+0000","last_clean":"2024-09-17T02:51:55.418548+0000","last_became_active":"2024-09-17T02:51:54.390634+0000","last_became_peered":"2024-09-17T02:51:54.390634+0000","last_unstale":"2024-09-17T02:51:55.418548+0000","last_undegraded":"2024-09-17T02:51:55.418548+0000","last_fullsized":"2024-09-17T02:51:55.418548+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:36.869075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407123+0000","last_change":"2024-09-17T02:51:55.404238+0000","last_active":"2024-09-17T02:51:55.407123+0000","last_peered":"2024-09-17T02:51:55.407123+0000","last_clean":"2024-09-17T02:51:55.407123+0000","last_became_active":"2024-09-17T02:51:55.402488+0000","last_became_peered":"2024-09-17T02:51:55.402488+0000","last_unstale":"2024-09-17T02:51:55.407123+0000","last_undegraded":"2024-09-17T02:51:55.407123+0000","last_fullsized":"2024-09-17T02:51:55.407123+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:58.349712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.352604+0000","last_change":"2024-09-17T02:51:54.402557+0000","last_active":"2024-09-17T02:51:55.352604+0000","last_peered":"2024-09-17T02:51:55.352604+0000","last_clean":"2024-09-17T02:51:55.352604+0000","last_became_active":"2024-09-17T02:51:54.402264+0000","last_became_peered":"2024-09-17T02:51:54.402264+0000","last_unstale":"2024-09-17T02:51:55.352604+0000","last_undegraded":"2024-09-17T02:51:55.352604+0000","last_fullsized":"2024-09-17T02:51:55.352604+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:25.770612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.367910+0000","last_change":"2024-09-17T02:51:55.367220+0000","last_active":"2024-09-17T02:51:55.367910+0000","last_peered":"2024-09-17T02:51:55.367910+0000","last_clean":"2024-09-17T02:51:55.367910+0000","last_became_active":"2024-09-17T02:51:55.366325+0000","last_became_peered":"2024-09-17T02:51:55.366325+0000","last_unstale":"2024-09-17T02:51:55.367910+0000","last_undegraded":"2024-09-17T02:51:55.367910+0000","last_fullsized":"2024-09-17T02:51:55.367910+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:43.545971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.353644+0000","last_change":"2024-09-17T02:51:54.389344+0000","last_active":"2024-09-17T02:51:55.353644+0000","last_peered":"2024-09-17T02:51:55.353644+0000","last_clean":"2024-09-17T02:51:55.353644+0000","last_became_active":"2024-09-17T02:51:54.388921+0000","last_became_peered":"2024-09-17T02:51:54.388921+0000","last_unstale":"2024-09-17T02:51:55.353644+0000","last_undegraded":"2024-09-17T02:51:55.353644+0000","last_fullsized":"2024-09-17T02:51:55.353644+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:34.705758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.412746+0000","last_change":"2024-09-17T02:51:55.405978+0000","last_active":"2024-09-17T02:51:55.412746+0000","last_peered":"2024-09-17T02:51:55.412746+0000","last_clean":"2024-09-17T02:51:55.412746+0000","last_became_active":"2024-09-17T02:51:55.405177+0000","last_became_peered":"2024-09-17T02:51:55.405177+0000","last_unstale":"2024-09-17T02:51:55.412746+0000","last_undegraded":"2024-09-17T02:51:55.412746+0000","last_fullsized":"2024-09-17T02:51:55.412746+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:47.336476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361302+0000","last_change":"2024-09-17T02:51:54.388465+0000","last_active":"2024-09-17T02:51:56.361302+0000","last_peered":"2024-09-17T02:51:56.361302+0000","last_clean":"2024-09-17T02:51:56.361302+0000","last_became_active":"2024-09-17T02:51:54.384034+0000","last_became_peered":"2024-09-17T02:51:54.384034+0000","last_unstale":"2024-09-17T02:51:56.361302+0000","last_undegraded":"2024-09-17T02:51:56.361302+0000","last_fullsized":"2024-09-17T02:51:56.361302+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:58.728962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.359573+0000","last_change":"2024-09-17T02:51:55.387460+0000","last_active":"2024-09-17T02:51:56.359573+0000","last_peered":"2024-09-17T02:51:56.359573+0000","last_clean":"2024-09-17T02:51:56.359573+0000","last_became_active":"2024-09-17T02:51:55.380416+0000","last_became_peered":"2024-09-17T02:51:55.380416+0000","last_unstale":"2024-09-17T02:51:56.359573+0000","last_undegraded":"2024-09-17T02:51:56.359573+0000","last_fullsized":"2024-09-17T02:51:56.359573+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:26.530831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.353328+0000","last_change":"2024-09-17T02:51:54.386181+0000","last_active":"2024-09-17T02:51:55.353328+0000","last_peered":"2024-09-17T02:51:55.353328+0000","last_clean":"2024-09-17T02:51:55.353328+0000","last_became_active":"2024-09-17T02:51:54.385849+0000","last_became_peered":"2024-09-17T02:51:54.385849+0000","last_unstale":"2024-09-17T02:51:55.353328+0000","last_undegraded":"2024-09-17T02:51:55.353328+0000","last_fullsized":"2024-09-17T02:51:55.353328+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:28.968467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.419365+0000","last_change":"2024-09-17T02:51:55.394327+0000","last_active":"2024-09-17T02:51:55.419365+0000","last_peered":"2024-09-17T02:51:55.419365+0000","last_clean":"2024-09-17T02:51:55.419365+0000","last_became_active":"2024-09-17T02:51:55.392467+0000","last_became_peered":"2024-09-17T02:51:55.392467+0000","last_unstale":"2024-09-17T02:51:55.419365+0000","last_undegraded":"2024-09-17T02:51:55.419365+0000","last_fullsized":"2024-09-17T02:51:55.419365+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:18.014580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.358788+0000","last_change":"2024-09-17T02:51:54.374392+0000","last_active":"2024-09-17T02:51:55.358788+0000","last_peered":"2024-09-17T02:51:55.358788+0000","last_clean":"2024-09-17T02:51:55.358788+0000","last_became_active":"2024-09-17T02:51:54.374037+0000","last_became_peered":"2024-09-17T02:51:54.374037+0000","last_unstale":"2024-09-17T02:51:55.358788+0000","last_undegraded":"2024-09-17T02:51:55.358788+0000","last_fullsized":"2024-09-17T02:51:55.358788+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:20.692346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.403150+0000","last_change":"2024-09-17T02:51:55.402631+0000","last_active":"2024-09-17T02:51:55.403150+0000","last_peered":"2024-09-17T02:51:55.403150+0000","last_clean":"2024-09-17T02:51:55.403150+0000","last_became_active":"2024-09-17T02:51:55.402351+0000","last_became_peered":"2024-09-17T02:51:55.402351+0000","last_unstale":"2024-09-17T02:51:55.403150+0000","last_undegraded":"2024-09-17T02:51:55.403150+0000","last_fullsized":"2024-09-17T02:51:55.403150+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:42.844086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.370416+0000","last_change":"2024-09-17T02:51:54.391400+0000","last_active":"2024-09-17T02:51:55.370416+0000","last_peered":"2024-09-17T02:51:55.370416+0000","last_clean":"2024-09-17T02:51:55.370416+0000","last_became_active":"2024-09-17T02:51:54.390736+0000","last_became_peered":"2024-09-17T02:51:54.390736+0000","last_unstale":"2024-09-17T02:51:55.370416+0000","last_undegraded":"2024-09-17T02:51:55.370416+0000","last_fullsized":"2024-09-17T02:51:55.370416+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:03.876506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.403320+0000","last_change":"2024-09-17T02:51:55.379963+0000","last_active":"2024-09-17T02:51:55.403320+0000","last_peered":"2024-09-17T02:51:55.403320+0000","last_clean":"2024-09-17T02:51:55.403320+0000","last_became_active":"2024-09-17T02:51:55.379030+0000","last_became_peered":"2024-09-17T02:51:55.379030+0000","last_unstale":"2024-09-17T02:51:55.403320+0000","last_undegraded":"2024-09-17T02:51:55.403320+0000","last_fullsized":"2024-09-17T02:51:55.403320+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:27.950945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.352793+0000","last_change":"2024-09-17T02:51:54.375020+0000","last_active":"2024-09-17T02:51:55.352793+0000","last_peered":"2024-09-17T02:51:55.352793+0000","last_clean":"2024-09-17T02:51:55.352793+0000","last_became_active":"2024-09-17T02:51:54.374520+0000","last_became_peered":"2024-09-17T02:51:54.374520+0000","last_unstale":"2024-09-17T02:51:55.352793+0000","last_undegraded":"2024-09-17T02:51:55.352793+0000","last_fullsized":"2024-09-17T02:51:55.352793+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:08.904920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407057+0000","last_change":"2024-09-17T02:51:55.406339+0000","last_active":"2024-09-17T02:51:55.407057+0000","last_peered":"2024-09-17T02:51:55.407057+0000","last_clean":"2024-09-17T02:51:55.407057+0000","last_became_active":"2024-09-17T02:51:55.406000+0000","last_became_peered":"2024-09-17T02:51:55.406000+0000","last_unstale":"2024-09-17T02:51:55.407057+0000","last_undegraded":"2024-09-17T02:51:55.407057+0000","last_fullsized":"2024-09-17T02:51:55.407057+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:03:35.253348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"121'8","reported_seq":21,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.417793+0000","last_change":"2024-09-17T02:51:54.393643+0000","last_active":"2024-09-17T02:51:55.417793+0000","last_peered":"2024-09-17T02:51:55.417793+0000","last_clean":"2024-09-17T02:51:55.417793+0000","last_became_active":"2024-09-17T02:51:54.392509+0000","last_became_peered":"2024-09-17T02:51:54.392509+0000","last_unstale":"2024-09-17T02:51:55.417793+0000","last_undegraded":"2024-09-17T02:51:55.417793+0000","last_fullsized":"2024-09-17T02:51:55.417793+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:00.221175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":301,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.358581+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:51:55.358581+0000","last_peered":"2024-09-17T02:51:55.358581+0000","last_clean":"2024-09-17T02:51:55.358581+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:51:55.358581+0000","last_undegraded":"2024-09-17T02:51:55.358581+0000","last_fullsized":"2024-09-17T02:51:55.358581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509579,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":700218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509579,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2105344,"data_stored":1292198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509580,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1507328,"data_stored":700274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509578,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":700308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509579,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1515520,"data_stored":702412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2101248,"data_stored":1290746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1511424,"data_stored":700904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1515520,"data_stored":706932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:51:59.536 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:51:59.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 -- 172.21.15.131:0/252409824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d806e3e0 msgr2=0x7fa8d815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 --2- 172.21.15.131:0/252409824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d806e3e0 0x7fa8d815bea0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fa8d806a1c0 tx=0x7fa8c801c8e0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 -- 172.21.15.131:0/252409824 shutdown_connections 2024-09-17T02:51:59.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 --2- 172.21.15.131:0/252409824 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d806e3e0 0x7fa8d815bea0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 --2- 172.21.15.131:0/252409824 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa8d815c3e0 0x7fa8d80cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 --2- 172.21.15.131:0/252409824 >> v2:172.21.15.189:3300/0 conn(0x7fa8d80ced00 0x7fa8d806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 -- 172.21.15.131:0/252409824 >> 172.21.15.131:0/252409824 conn(0x7fa8d806eea0 msgr2=0x7fa8d806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:59.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 -- 172.21.15.131:0/252409824 shutdown_connections 2024-09-17T02:51:59.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 -- 172.21.15.131:0/252409824 wait complete. 2024-09-17T02:51:59.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 Processor -- start 2024-09-17T02:51:59.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 -- start start 2024-09-17T02:51:59.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d806e3e0 0x7fa8d81570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:59.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa8d80ced00 0x7fa8d81589e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:59.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8d815c3e0 0x7fa8d8162e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:59.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d80caae0 con 0x7fa8d806e3e0 2024-09-17T02:51:59.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d80ca960 con 0x7fa8d815c3e0 2024-09-17T02:51:59.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dfa36640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa8d80cac60 con 0x7fa8d80ced00 2024-09-17T02:51:59.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dd7d2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d806e3e0 0x7fa8d81570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:59.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dd7d2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d806e3e0 0x7fa8d81570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41940/0 (socket says 172.21.15.131:41940) 2024-09-17T02:51:59.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.722+0000 7fa8dd7d2640 1 -- 172.21.15.131:0/3935603967 learned_addr learned my addr 172.21.15.131:0/3935603967 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:51:59.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8ddfd3640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8d815c3e0 0x7fa8d8162e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dcfd1640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa8d80ced00 0x7fa8d81589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dd7d2640 1 -- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa8d80ced00 msgr2=0x7fa8d81589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dd7d2640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa8d80ced00 0x7fa8d81589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dd7d2640 1 -- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8d815c3e0 msgr2=0x7fa8d8162e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dd7d2640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8d815c3e0 0x7fa8d8162e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dd7d2640 1 -- 172.21.15.131:0/3935603967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d80cb750 con 0x7fa8d806e3e0 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dd7d2640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d806e3e0 0x7fa8d81570f0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400c940 tx=0x7fa8d400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8cdffb640 1 -- 172.21.15.131:0/3935603967 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4004220 con 0x7fa8d806e3e0 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8cdffb640 1 -- 172.21.15.131:0/3935603967 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8d40079b0 con 0x7fa8d806e3e0 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dfa36640 1 -- 172.21.15.131:0/3935603967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d8157940 con 0x7fa8d806e3e0 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8cdffb640 1 -- 172.21.15.131:0/3935603967 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8d40044b0 con 0x7fa8d806e3e0 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dfa36640 1 -- 172.21.15.131:0/3935603967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d8157f00 con 0x7fa8d806e3e0 2024-09-17T02:51:59.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dfa36640 1 -- 172.21.15.131:0/3935603967 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d8158f20 con 0x7fa8d806e3e0 2024-09-17T02:51:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8cdffb640 1 -- 172.21.15.131:0/3935603967 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa8d4003600 con 0x7fa8d806e3e0 2024-09-17T02:51:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8cdffb640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8bc073230 0x7fa8bc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:51:59.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8cdffb640 1 -- 172.21.15.131:0/3935603967 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fa8d4094400 con 0x7fa8d806e3e0 2024-09-17T02:51:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dcfd1640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8bc073230 0x7fa8bc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:51:59.722 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.726+0000 7fa8dcfd1640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8bc073230 0x7fa8bc075740 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80af000 tx=0x7fa8c801b120 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:51:59.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.730+0000 7fa8cdffb640 1 -- 172.21.15.131:0/3935603967 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa8d4014030 con 0x7fa8d806e3e0 2024-09-17T02:51:59.879 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.886+0000 7fa8dfa36640 1 -- 172.21.15.131:0/3935603967 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8d806af50 con 0x7fa8bc073230 2024-09-17T02:51:59.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.898+0000 7fa8cdffb640 1 -- 172.21.15.131:0/3935603967 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252479 (secure 0 0 0) 0x7fa8c80f9a50 con 0x7fa8bc073230 2024-09-17T02:51:59.893 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:51:59.895 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:51:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 -- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8bc073230 msgr2=0x7fa8bc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8bc073230 0x7fa8bc075740 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80af000 tx=0x7fa8c801b120 comp rx=0 tx=0).stop 2024-09-17T02:51:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 -- 172.21.15.131:0/3935603967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d806e3e0 msgr2=0x7fa8d81570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:51:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d806e3e0 0x7fa8d81570f0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400c940 tx=0x7fa8d400ce00 comp rx=0 tx=0).stop 2024-09-17T02:51:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 -- 172.21.15.131:0/3935603967 shutdown_connections 2024-09-17T02:51:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa8bc073230 0x7fa8bc075740 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa8d815c3e0 0x7fa8d8162e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.896 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa8d80ced00 0x7fa8d81589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 --2- 172.21.15.131:0/3935603967 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa8d806e3e0 0x7fa8d81570f0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:51:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 -- 172.21.15.131:0/3935603967 >> 172.21.15.131:0/3935603967 conn(0x7fa8d806eea0 msgr2=0x7fa8d806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:51:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 -- 172.21.15.131:0/3935603967 shutdown_connections 2024-09-17T02:51:59.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:51:59.902+0000 7fa8dfa36640 1 -- 172.21.15.131:0/3935603967 wait complete. 2024-09-17T02:51:59.914 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-17T02:51:59.391660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":111,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262496,"kb_used_data":12948,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712224,"statfs":{"total":773094113280,"available":772825317376,"internally_reserved":0,"allocated":13258752,"data_stored":6793992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002155"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.381354+0000","last_change":"2024-09-17T02:51:55.378936+0000","last_active":"2024-09-17T02:51:55.381354+0000","last_peered":"2024-09-17T02:51:55.381354+0000","last_clean":"2024-09-17T02:51:55.381354+0000","last_became_active":"2024-09-17T02:51:55.378614+0000","last_became_peered":"2024-09-17T02:51:55.378614+0000","last_unstale":"2024-09-17T02:51:55.381354+0000","last_undegraded":"2024-09-17T02:51:55.381354+0000","last_fullsized":"2024-09-17T02:51:55.381354+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:07:47.377205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407332+0000","last_change":"2024-09-17T02:51:55.376513+0000","last_active":"2024-09-17T02:51:55.407332+0000","last_peered":"2024-09-17T02:51:55.407332+0000","last_clean":"2024-09-17T02:51:55.407332+0000","last_became_active":"2024-09-17T02:51:55.373289+0000","last_became_peered":"2024-09-17T02:51:55.373289+0000","last_unstale":"2024-09-17T02:51:55.407332+0000","last_undegraded":"2024-09-17T02:51:55.407332+0000","last_fullsized":"2024-09-17T02:51:55.407332+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:49:50.329245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.421833+0000","last_change":"2024-09-17T02:51:55.418924+0000","last_active":"2024-09-17T02:51:55.421833+0000","last_peered":"2024-09-17T02:51:55.421833+0000","last_clean":"2024-09-17T02:51:55.421833+0000","last_became_active":"2024-09-17T02:51:55.418264+0000","last_became_peered":"2024-09-17T02:51:55.418264+0000","last_unstale":"2024-09-17T02:51:55.421833+0000","last_undegraded":"2024-09-17T02:51:55.421833+0000","last_fullsized":"2024-09-17T02:51:55.421833+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:59.226762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.382323+0000","last_change":"2024-09-17T02:51:55.380263+0000","last_active":"2024-09-17T02:51:55.382323+0000","last_peered":"2024-09-17T02:51:55.382323+0000","last_clean":"2024-09-17T02:51:55.382323+0000","last_became_active":"2024-09-17T02:51:55.379912+0000","last_became_peered":"2024-09-17T02:51:55.379912+0000","last_unstale":"2024-09-17T02:51:55.382323+0000","last_undegraded":"2024-09-17T02:51:55.382323+0000","last_fullsized":"2024-09-17T02:51:55.382323+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:25:16.740036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.376461+0000","last_change":"2024-09-17T02:51:55.371805+0000","last_active":"2024-09-17T02:51:55.376461+0000","last_peered":"2024-09-17T02:51:55.376461+0000","last_clean":"2024-09-17T02:51:55.376461+0000","last_became_active":"2024-09-17T02:51:55.370901+0000","last_became_peered":"2024-09-17T02:51:55.370901+0000","last_unstale":"2024-09-17T02:51:55.376461+0000","last_undegraded":"2024-09-17T02:51:55.376461+0000","last_fullsized":"2024-09-17T02:51:55.376461+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:03.119409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.408621+0000","last_change":"2024-09-17T02:51:55.381571+0000","last_active":"2024-09-17T02:51:55.408621+0000","last_peered":"2024-09-17T02:51:55.408621+0000","last_clean":"2024-09-17T02:51:55.408621+0000","last_became_active":"2024-09-17T02:51:55.380731+0000","last_became_peered":"2024-09-17T02:51:55.380731+0000","last_unstale":"2024-09-17T02:51:55.408621+0000","last_undegraded":"2024-09-17T02:51:55.408621+0000","last_fullsized":"2024-09-17T02:51:55.408621+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:28:39.528097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.417636+0000","last_change":"2024-09-17T02:51:55.393258+0000","last_active":"2024-09-17T02:51:55.417636+0000","last_peered":"2024-09-17T02:51:55.417636+0000","last_clean":"2024-09-17T02:51:55.417636+0000","last_became_active":"2024-09-17T02:51:55.374759+0000","last_became_peered":"2024-09-17T02:51:55.374759+0000","last_unstale":"2024-09-17T02:51:55.417636+0000","last_undegraded":"2024-09-17T02:51:55.417636+0000","last_fullsized":"2024-09-17T02:51:55.417636+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:20:37.603100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.401468+0000","last_change":"2024-09-17T02:51:55.381702+0000","last_active":"2024-09-17T02:51:55.401468+0000","last_peered":"2024-09-17T02:51:55.401468+0000","last_clean":"2024-09-17T02:51:55.401468+0000","last_became_active":"2024-09-17T02:51:55.379423+0000","last_became_peered":"2024-09-17T02:51:55.379423+0000","last_unstale":"2024-09-17T02:51:55.401468+0000","last_undegraded":"2024-09-17T02:51:55.401468+0000","last_fullsized":"2024-09-17T02:51:55.401468+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:44:01.505136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.393398+0000","last_change":"2024-09-17T02:51:55.392112+0000","last_active":"2024-09-17T02:51:55.393398+0000","last_peered":"2024-09-17T02:51:55.393398+0000","last_clean":"2024-09-17T02:51:55.393398+0000","last_became_active":"2024-09-17T02:51:55.391801+0000","last_became_peered":"2024-09-17T02:51:55.391801+0000","last_unstale":"2024-09-17T02:51:55.393398+0000","last_undegraded":"2024-09-17T02:51:55.393398+0000","last_fullsized":"2024-09-17T02:51:55.393398+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:15:42.403220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.419127+0000","last_change":"2024-09-17T02:51:55.414528+0000","last_active":"2024-09-17T02:51:55.419127+0000","last_peered":"2024-09-17T02:51:55.419127+0000","last_clean":"2024-09-17T02:51:55.419127+0000","last_became_active":"2024-09-17T02:51:55.413841+0000","last_became_peered":"2024-09-17T02:51:55.413841+0000","last_unstale":"2024-09-17T02:51:55.419127+0000","last_undegraded":"2024-09-17T02:51:55.419127+0000","last_fullsized":"2024-09-17T02:51:55.419127+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:27:25.784396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.365990+0000","last_change":"2024-09-17T02:51:55.363998+0000","last_active":"2024-09-17T02:51:55.365990+0000","last_peered":"2024-09-17T02:51:55.365990+0000","last_clean":"2024-09-17T02:51:55.365990+0000","last_became_active":"2024-09-17T02:51:55.363631+0000","last_became_peered":"2024-09-17T02:51:55.363631+0000","last_unstale":"2024-09-17T02:51:55.365990+0000","last_undegraded":"2024-09-17T02:51:55.365990+0000","last_fullsized":"2024-09-17T02:51:55.365990+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:35.285404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.360676+0000","last_change":"2024-09-17T02:51:55.409803+0000","last_active":"2024-09-17T02:51:56.360676+0000","last_peered":"2024-09-17T02:51:56.360676+0000","last_clean":"2024-09-17T02:51:56.360676+0000","last_became_active":"2024-09-17T02:51:55.397783+0000","last_became_peered":"2024-09-17T02:51:55.397783+0000","last_unstale":"2024-09-17T02:51:56.360676+0000","last_undegraded":"2024-09-17T02:51:56.360676+0000","last_fullsized":"2024-09-17T02:51:56.360676+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:21:52.805536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.411227+0000","last_change":"2024-09-17T02:51:55.379297+0000","last_active":"2024-09-17T02:51:55.411227+0000","last_peered":"2024-09-17T02:51:55.411227+0000","last_clean":"2024-09-17T02:51:55.411227+0000","last_became_active":"2024-09-17T02:51:55.378499+0000","last_became_peered":"2024-09-17T02:51:55.378499+0000","last_unstale":"2024-09-17T02:51:55.411227+0000","last_undegraded":"2024-09-17T02:51:55.411227+0000","last_fullsized":"2024-09-17T02:51:55.411227+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:58.117312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.360305+0000","last_change":"2024-09-17T02:51:55.397915+0000","last_active":"2024-09-17T02:51:56.360305+0000","last_peered":"2024-09-17T02:51:56.360305+0000","last_clean":"2024-09-17T02:51:56.360305+0000","last_became_active":"2024-09-17T02:51:55.390980+0000","last_became_peered":"2024-09-17T02:51:55.390980+0000","last_unstale":"2024-09-17T02:51:56.360305+0000","last_undegraded":"2024-09-17T02:51:56.360305+0000","last_fullsized":"2024-09-17T02:51:56.360305+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:42.410866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.370778+0000","last_change":"2024-09-17T02:51:55.368786+0000","last_active":"2024-09-17T02:51:55.370778+0000","last_peered":"2024-09-17T02:51:55.370778+0000","last_clean":"2024-09-17T02:51:55.370778+0000","last_became_active":"2024-09-17T02:51:55.368451+0000","last_became_peered":"2024-09-17T02:51:55.368451+0000","last_unstale":"2024-09-17T02:51:55.370778+0000","last_undegraded":"2024-09-17T02:51:55.370778+0000","last_fullsized":"2024-09-17T02:51:55.370778+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:20.498345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.359488+0000","last_change":"2024-09-17T02:51:55.409649+0000","last_active":"2024-09-17T02:51:56.359488+0000","last_peered":"2024-09-17T02:51:56.359488+0000","last_clean":"2024-09-17T02:51:56.359488+0000","last_became_active":"2024-09-17T02:51:55.408928+0000","last_became_peered":"2024-09-17T02:51:55.408928+0000","last_unstale":"2024-09-17T02:51:56.359488+0000","last_undegraded":"2024-09-17T02:51:56.359488+0000","last_fullsized":"2024-09-17T02:51:56.359488+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:45:19.532519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361378+0000","last_change":"2024-09-17T02:51:55.387322+0000","last_active":"2024-09-17T02:51:56.361378+0000","last_peered":"2024-09-17T02:51:56.361378+0000","last_clean":"2024-09-17T02:51:56.361378+0000","last_became_active":"2024-09-17T02:51:55.380758+0000","last_became_peered":"2024-09-17T02:51:55.380758+0000","last_unstale":"2024-09-17T02:51:56.361378+0000","last_undegraded":"2024-09-17T02:51:56.361378+0000","last_fullsized":"2024-09-17T02:51:56.361378+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:47:38.638277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361070+0000","last_change":"2024-09-17T02:51:55.398714+0000","last_active":"2024-09-17T02:51:56.361070+0000","last_peered":"2024-09-17T02:51:56.361070+0000","last_clean":"2024-09-17T02:51:56.361070+0000","last_became_active":"2024-09-17T02:51:55.397494+0000","last_became_peered":"2024-09-17T02:51:55.397494+0000","last_unstale":"2024-09-17T02:51:56.361070+0000","last_undegraded":"2024-09-17T02:51:56.361070+0000","last_fullsized":"2024-09-17T02:51:56.361070+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:23.607222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.398530+0000","last_change":"2024-09-17T02:51:55.397902+0000","last_active":"2024-09-17T02:51:55.398530+0000","last_peered":"2024-09-17T02:51:55.398530+0000","last_clean":"2024-09-17T02:51:55.398530+0000","last_became_active":"2024-09-17T02:51:55.397627+0000","last_became_peered":"2024-09-17T02:51:55.397627+0000","last_unstale":"2024-09-17T02:51:55.398530+0000","last_undegraded":"2024-09-17T02:51:55.398530+0000","last_fullsized":"2024-09-17T02:51:55.398530+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:25.309851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.425328+0000","last_change":"2024-09-17T02:51:55.424359+0000","last_active":"2024-09-17T02:51:55.425328+0000","last_peered":"2024-09-17T02:51:55.425328+0000","last_clean":"2024-09-17T02:51:55.425328+0000","last_became_active":"2024-09-17T02:51:55.422988+0000","last_became_peered":"2024-09-17T02:51:55.422988+0000","last_unstale":"2024-09-17T02:51:55.425328+0000","last_undegraded":"2024-09-17T02:51:55.425328+0000","last_fullsized":"2024-09-17T02:51:55.425328+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:51:53.383522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.383093+0000","last_change":"2024-09-17T02:51:55.376210+0000","last_active":"2024-09-17T02:51:55.383093+0000","last_peered":"2024-09-17T02:51:55.383093+0000","last_clean":"2024-09-17T02:51:55.383093+0000","last_became_active":"2024-09-17T02:51:55.375879+0000","last_became_peered":"2024-09-17T02:51:55.375879+0000","last_unstale":"2024-09-17T02:51:55.383093+0000","last_undegraded":"2024-09-17T02:51:55.383093+0000","last_fullsized":"2024-09-17T02:51:55.383093+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:31.171690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.393117+0000","last_change":"2024-09-17T02:51:55.392110+0000","last_active":"2024-09-17T02:51:55.393117+0000","last_peered":"2024-09-17T02:51:55.393117+0000","last_clean":"2024-09-17T02:51:55.393117+0000","last_became_active":"2024-09-17T02:51:55.391710+0000","last_became_peered":"2024-09-17T02:51:55.391710+0000","last_unstale":"2024-09-17T02:51:55.393117+0000","last_undegraded":"2024-09-17T02:51:55.393117+0000","last_fullsized":"2024-09-17T02:51:55.393117+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:14.229831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.370732+0000","last_change":"2024-09-17T02:51:55.368623+0000","last_active":"2024-09-17T02:51:55.370732+0000","last_peered":"2024-09-17T02:51:55.370732+0000","last_clean":"2024-09-17T02:51:55.370732+0000","last_became_active":"2024-09-17T02:51:55.368272+0000","last_became_peered":"2024-09-17T02:51:55.368272+0000","last_unstale":"2024-09-17T02:51:55.370732+0000","last_undegraded":"2024-09-17T02:51:55.370732+0000","last_fullsized":"2024-09-17T02:51:55.370732+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:31.171690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.393099+0000","last_change":"2024-09-17T02:51:55.392100+0000","last_active":"2024-09-17T02:51:55.393099+0000","last_peered":"2024-09-17T02:51:55.393099+0000","last_clean":"2024-09-17T02:51:55.393099+0000","last_became_active":"2024-09-17T02:51:55.391737+0000","last_became_peered":"2024-09-17T02:51:55.391737+0000","last_unstale":"2024-09-17T02:51:55.393099+0000","last_undegraded":"2024-09-17T02:51:55.393099+0000","last_fullsized":"2024-09-17T02:51:55.393099+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:25.309851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.381404+0000","last_change":"2024-09-17T02:51:55.378538+0000","last_active":"2024-09-17T02:51:55.381404+0000","last_peered":"2024-09-17T02:51:55.381404+0000","last_clean":"2024-09-17T02:51:55.381404+0000","last_became_active":"2024-09-17T02:51:55.378090+0000","last_became_peered":"2024-09-17T02:51:55.378090+0000","last_unstale":"2024-09-17T02:51:55.381404+0000","last_undegraded":"2024-09-17T02:51:55.381404+0000","last_fullsized":"2024-09-17T02:51:55.381404+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:02.410189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.401637+0000","last_change":"2024-09-17T02:51:55.385458+0000","last_active":"2024-09-17T02:51:55.401637+0000","last_peered":"2024-09-17T02:51:55.401637+0000","last_clean":"2024-09-17T02:51:55.401637+0000","last_became_active":"2024-09-17T02:51:55.384568+0000","last_became_peered":"2024-09-17T02:51:55.384568+0000","last_unstale":"2024-09-17T02:51:55.401637+0000","last_undegraded":"2024-09-17T02:51:55.401637+0000","last_fullsized":"2024-09-17T02:51:55.401637+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:54:59.463669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.376280+0000","last_change":"2024-09-17T02:51:55.372167+0000","last_active":"2024-09-17T02:51:55.376280+0000","last_peered":"2024-09-17T02:51:55.376280+0000","last_clean":"2024-09-17T02:51:55.376280+0000","last_became_active":"2024-09-17T02:51:55.371631+0000","last_became_peered":"2024-09-17T02:51:55.371631+0000","last_unstale":"2024-09-17T02:51:55.376280+0000","last_undegraded":"2024-09-17T02:51:55.376280+0000","last_fullsized":"2024-09-17T02:51:55.376280+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:21.696232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361160+0000","last_change":"2024-09-17T02:51:55.422230+0000","last_active":"2024-09-17T02:51:56.361160+0000","last_peered":"2024-09-17T02:51:56.361160+0000","last_clean":"2024-09-17T02:51:56.361160+0000","last_became_active":"2024-09-17T02:51:55.410789+0000","last_became_peered":"2024-09-17T02:51:55.410789+0000","last_unstale":"2024-09-17T02:51:56.361160+0000","last_undegraded":"2024-09-17T02:51:56.361160+0000","last_fullsized":"2024-09-17T02:51:56.361160+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:00:03.762220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.421775+0000","last_change":"2024-09-17T02:51:55.420322+0000","last_active":"2024-09-17T02:51:55.421775+0000","last_peered":"2024-09-17T02:51:55.421775+0000","last_clean":"2024-09-17T02:51:55.421775+0000","last_became_active":"2024-09-17T02:51:55.419647+0000","last_became_peered":"2024-09-17T02:51:55.419647+0000","last_unstale":"2024-09-17T02:51:55.421775+0000","last_undegraded":"2024-09-17T02:51:55.421775+0000","last_fullsized":"2024-09-17T02:51:55.421775+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:29.524549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407373+0000","last_change":"2024-09-17T02:51:55.404030+0000","last_active":"2024-09-17T02:51:55.407373+0000","last_peered":"2024-09-17T02:51:55.407373+0000","last_clean":"2024-09-17T02:51:55.407373+0000","last_became_active":"2024-09-17T02:51:55.402207+0000","last_became_peered":"2024-09-17T02:51:55.402207+0000","last_unstale":"2024-09-17T02:51:55.407373+0000","last_undegraded":"2024-09-17T02:51:55.407373+0000","last_fullsized":"2024-09-17T02:51:55.407373+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:40:28.068092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.393800+0000","last_change":"2024-09-17T02:51:55.393010+0000","last_active":"2024-09-17T02:51:55.393800+0000","last_peered":"2024-09-17T02:51:55.393800+0000","last_clean":"2024-09-17T02:51:55.393800+0000","last_became_active":"2024-09-17T02:51:55.392693+0000","last_became_peered":"2024-09-17T02:51:55.392693+0000","last_unstale":"2024-09-17T02:51:55.393800+0000","last_undegraded":"2024-09-17T02:51:55.393800+0000","last_fullsized":"2024-09-17T02:51:55.393800+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:21:40.722597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.385699+0000","last_change":"2024-09-17T02:51:55.385244+0000","last_active":"2024-09-17T02:51:55.385699+0000","last_peered":"2024-09-17T02:51:55.385699+0000","last_clean":"2024-09-17T02:51:55.385699+0000","last_became_active":"2024-09-17T02:51:55.384888+0000","last_became_peered":"2024-09-17T02:51:55.384888+0000","last_unstale":"2024-09-17T02:51:55.385699+0000","last_undegraded":"2024-09-17T02:51:55.385699+0000","last_fullsized":"2024-09-17T02:51:55.385699+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:49.312656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.416048+0000","last_change":"2024-09-17T02:51:55.404830+0000","last_active":"2024-09-17T02:51:55.416048+0000","last_peered":"2024-09-17T02:51:55.416048+0000","last_clean":"2024-09-17T02:51:55.416048+0000","last_became_active":"2024-09-17T02:51:55.403942+0000","last_became_peered":"2024-09-17T02:51:55.403942+0000","last_unstale":"2024-09-17T02:51:55.416048+0000","last_undegraded":"2024-09-17T02:51:55.416048+0000","last_fullsized":"2024-09-17T02:51:55.416048+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:23:01.987795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1f","version":"121'4","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.384079+0000","last_change":"2024-09-17T02:51:54.375067+0000","last_active":"2024-09-17T02:51:55.384079+0000","last_peered":"2024-09-17T02:51:55.384079+0000","last_clean":"2024-09-17T02:51:55.384079+0000","last_became_active":"2024-09-17T02:51:54.374709+0000","last_became_peered":"2024-09-17T02:51:54.374709+0000","last_unstale":"2024-09-17T02:51:55.384079+0000","last_undegraded":"2024-09-17T02:51:55.384079+0000","last_fullsized":"2024-09-17T02:51:55.384079+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:05.729763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.418747+0000","last_change":"2024-09-17T02:51:55.414787+0000","last_active":"2024-09-17T02:51:55.418747+0000","last_peered":"2024-09-17T02:51:55.418747+0000","last_clean":"2024-09-17T02:51:55.418747+0000","last_became_active":"2024-09-17T02:51:55.414145+0000","last_became_peered":"2024-09-17T02:51:55.414145+0000","last_unstale":"2024-09-17T02:51:55.418747+0000","last_undegraded":"2024-09-17T02:51:55.418747+0000","last_fullsized":"2024-09-17T02:51:55.418747+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:03:43.362915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1e","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.360702+0000","last_change":"2024-09-17T02:51:54.389172+0000","last_active":"2024-09-17T02:51:56.360702+0000","last_peered":"2024-09-17T02:51:56.360702+0000","last_clean":"2024-09-17T02:51:56.360702+0000","last_became_active":"2024-09-17T02:51:54.388296+0000","last_became_peered":"2024-09-17T02:51:54.388296+0000","last_unstale":"2024-09-17T02:51:56.360702+0000","last_undegraded":"2024-09-17T02:51:56.360702+0000","last_fullsized":"2024-09-17T02:51:56.360702+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:10:52.024157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.375533+0000","last_change":"2024-09-17T02:51:55.373263+0000","last_active":"2024-09-17T02:51:55.375533+0000","last_peered":"2024-09-17T02:51:55.375533+0000","last_clean":"2024-09-17T02:51:55.375533+0000","last_became_active":"2024-09-17T02:51:55.372892+0000","last_became_peered":"2024-09-17T02:51:55.372892+0000","last_unstale":"2024-09-17T02:51:55.375533+0000","last_undegraded":"2024-09-17T02:51:55.375533+0000","last_fullsized":"2024-09-17T02:51:55.375533+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:55:50.535681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1d","version":"121'2","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.387709+0000","last_change":"2024-09-17T02:51:54.377679+0000","last_active":"2024-09-17T02:51:55.387709+0000","last_peered":"2024-09-17T02:51:55.387709+0000","last_clean":"2024-09-17T02:51:55.387709+0000","last_became_active":"2024-09-17T02:51:54.377348+0000","last_became_peered":"2024-09-17T02:51:54.377348+0000","last_unstale":"2024-09-17T02:51:55.387709+0000","last_undegraded":"2024-09-17T02:51:55.387709+0000","last_fullsized":"2024-09-17T02:51:55.387709+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:12:02.115523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.375417+0000","last_change":"2024-09-17T02:51:55.374491+0000","last_active":"2024-09-17T02:51:55.375417+0000","last_peered":"2024-09-17T02:51:55.375417+0000","last_clean":"2024-09-17T02:51:55.375417+0000","last_became_active":"2024-09-17T02:51:55.374207+0000","last_became_peered":"2024-09-17T02:51:55.374207+0000","last_unstale":"2024-09-17T02:51:55.375417+0000","last_undegraded":"2024-09-17T02:51:55.375417+0000","last_fullsized":"2024-09-17T02:51:55.375417+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:59:39.318119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"121'2","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.424968+0000","last_change":"2024-09-17T02:51:54.399591+0000","last_active":"2024-09-17T02:51:55.424968+0000","last_peered":"2024-09-17T02:51:55.424968+0000","last_clean":"2024-09-17T02:51:55.424968+0000","last_became_active":"2024-09-17T02:51:54.398462+0000","last_became_peered":"2024-09-17T02:51:54.398462+0000","last_unstale":"2024-09-17T02:51:55.424968+0000","last_undegraded":"2024-09-17T02:51:55.424968+0000","last_fullsized":"2024-09-17T02:51:55.424968+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:51:53.854955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.401537+0000","last_change":"2024-09-17T02:51:55.382560+0000","last_active":"2024-09-17T02:51:55.401537+0000","last_peered":"2024-09-17T02:51:55.401537+0000","last_clean":"2024-09-17T02:51:55.401537+0000","last_became_active":"2024-09-17T02:51:55.379192+0000","last_became_peered":"2024-09-17T02:51:55.379192+0000","last_unstale":"2024-09-17T02:51:55.401537+0000","last_undegraded":"2024-09-17T02:51:55.401537+0000","last_fullsized":"2024-09-17T02:51:55.401537+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:29.134240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1b","version":"121'1","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.383777+0000","last_change":"2024-09-17T02:51:54.384242+0000","last_active":"2024-09-17T02:51:55.383777+0000","last_peered":"2024-09-17T02:51:55.383777+0000","last_clean":"2024-09-17T02:51:55.383777+0000","last_became_active":"2024-09-17T02:51:54.383864+0000","last_became_peered":"2024-09-17T02:51:54.383864+0000","last_unstale":"2024-09-17T02:51:55.383777+0000","last_undegraded":"2024-09-17T02:51:55.383777+0000","last_fullsized":"2024-09-17T02:51:55.383777+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:42:23.756438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.369855+0000","last_change":"2024-09-17T02:51:55.368614+0000","last_active":"2024-09-17T02:51:55.369855+0000","last_peered":"2024-09-17T02:51:55.369855+0000","last_clean":"2024-09-17T02:51:55.369855+0000","last_became_active":"2024-09-17T02:51:55.368268+0000","last_became_peered":"2024-09-17T02:51:55.368268+0000","last_unstale":"2024-09-17T02:51:55.369855+0000","last_undegraded":"2024-09-17T02:51:55.369855+0000","last_fullsized":"2024-09-17T02:51:55.369855+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:59.402992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1a","version":"121'1","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.413758+0000","last_change":"2024-09-17T02:51:54.397805+0000","last_active":"2024-09-17T02:51:55.413758+0000","last_peered":"2024-09-17T02:51:55.413758+0000","last_clean":"2024-09-17T02:51:55.413758+0000","last_became_active":"2024-09-17T02:51:54.397281+0000","last_became_peered":"2024-09-17T02:51:54.397281+0000","last_unstale":"2024-09-17T02:51:55.413758+0000","last_undegraded":"2024-09-17T02:51:55.413758+0000","last_fullsized":"2024-09-17T02:51:55.413758+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:05:38.560730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.419356+0000","last_change":"2024-09-17T02:51:55.415382+0000","last_active":"2024-09-17T02:51:55.419356+0000","last_peered":"2024-09-17T02:51:55.419356+0000","last_clean":"2024-09-17T02:51:55.419356+0000","last_became_active":"2024-09-17T02:51:55.414394+0000","last_became_peered":"2024-09-17T02:51:55.414394+0000","last_unstale":"2024-09-17T02:51:55.419356+0000","last_undegraded":"2024-09-17T02:51:55.419356+0000","last_fullsized":"2024-09-17T02:51:55.419356+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:33:29.622277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.362909+0000","last_change":"2024-09-17T02:51:54.369369+0000","last_active":"2024-09-17T02:51:55.362909+0000","last_peered":"2024-09-17T02:51:55.362909+0000","last_clean":"2024-09-17T02:51:55.362909+0000","last_became_active":"2024-09-17T02:51:54.369043+0000","last_became_peered":"2024-09-17T02:51:54.369043+0000","last_unstale":"2024-09-17T02:51:55.362909+0000","last_undegraded":"2024-09-17T02:51:55.362909+0000","last_fullsized":"2024-09-17T02:51:55.362909+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:10.599327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.403576+0000","last_change":"2024-09-17T02:51:55.381824+0000","last_active":"2024-09-17T02:51:55.403576+0000","last_peered":"2024-09-17T02:51:55.403576+0000","last_clean":"2024-09-17T02:51:55.403576+0000","last_became_active":"2024-09-17T02:51:55.380948+0000","last_became_peered":"2024-09-17T02:51:55.380948+0000","last_unstale":"2024-09-17T02:51:55.403576+0000","last_undegraded":"2024-09-17T02:51:55.403576+0000","last_fullsized":"2024-09-17T02:51:55.403576+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:23:03.359056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.387327+0000","last_change":"2024-09-17T02:51:54.396926+0000","last_active":"2024-09-17T02:51:55.387327+0000","last_peered":"2024-09-17T02:51:55.387327+0000","last_clean":"2024-09-17T02:51:55.387327+0000","last_became_active":"2024-09-17T02:51:54.396197+0000","last_became_peered":"2024-09-17T02:51:54.396197+0000","last_unstale":"2024-09-17T02:51:55.387327+0000","last_undegraded":"2024-09-17T02:51:55.387327+0000","last_fullsized":"2024-09-17T02:51:55.387327+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:09:25.132313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.373844+0000","last_change":"2024-09-17T02:51:55.372506+0000","last_active":"2024-09-17T02:51:55.373844+0000","last_peered":"2024-09-17T02:51:55.373844+0000","last_clean":"2024-09-17T02:51:55.373844+0000","last_became_active":"2024-09-17T02:51:55.372198+0000","last_became_peered":"2024-09-17T02:51:55.372198+0000","last_unstale":"2024-09-17T02:51:55.373844+0000","last_undegraded":"2024-09-17T02:51:55.373844+0000","last_fullsized":"2024-09-17T02:51:55.373844+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:26.353572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.365094+0000","last_change":"2024-09-17T02:51:54.384344+0000","last_active":"2024-09-17T02:51:55.365094+0000","last_peered":"2024-09-17T02:51:55.365094+0000","last_clean":"2024-09-17T02:51:55.365094+0000","last_became_active":"2024-09-17T02:51:54.383955+0000","last_became_peered":"2024-09-17T02:51:54.383955+0000","last_unstale":"2024-09-17T02:51:55.365094+0000","last_undegraded":"2024-09-17T02:51:55.365094+0000","last_fullsized":"2024-09-17T02:51:55.365094+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:01:21.292617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.382144+0000","last_change":"2024-09-17T02:51:55.379732+0000","last_active":"2024-09-17T02:51:55.382144+0000","last_peered":"2024-09-17T02:51:55.382144+0000","last_clean":"2024-09-17T02:51:55.382144+0000","last_became_active":"2024-09-17T02:51:55.379297+0000","last_became_peered":"2024-09-17T02:51:55.379297+0000","last_unstale":"2024-09-17T02:51:55.382144+0000","last_undegraded":"2024-09-17T02:51:55.382144+0000","last_fullsized":"2024-09-17T02:51:55.382144+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:19:12.080162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.359655+0000","last_change":"2024-09-17T02:51:54.390386+0000","last_active":"2024-09-17T02:51:55.359655+0000","last_peered":"2024-09-17T02:51:55.359655+0000","last_clean":"2024-09-17T02:51:55.359655+0000","last_became_active":"2024-09-17T02:51:54.389933+0000","last_became_peered":"2024-09-17T02:51:54.389933+0000","last_unstale":"2024-09-17T02:51:55.359655+0000","last_undegraded":"2024-09-17T02:51:55.359655+0000","last_fullsized":"2024-09-17T02:51:55.359655+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:03:34.249461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407036+0000","last_change":"2024-09-17T02:51:55.404319+0000","last_active":"2024-09-17T02:51:55.407036+0000","last_peered":"2024-09-17T02:51:55.407036+0000","last_clean":"2024-09-17T02:51:55.407036+0000","last_became_active":"2024-09-17T02:51:55.402307+0000","last_became_peered":"2024-09-17T02:51:55.402307+0000","last_unstale":"2024-09-17T02:51:55.407036+0000","last_undegraded":"2024-09-17T02:51:55.407036+0000","last_fullsized":"2024-09-17T02:51:55.407036+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T02:58:02.723893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"121'3","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.383585+0000","last_change":"2024-09-17T02:51:54.366554+0000","last_active":"2024-09-17T02:51:55.383585+0000","last_peered":"2024-09-17T02:51:55.383585+0000","last_clean":"2024-09-17T02:51:55.383585+0000","last_became_active":"2024-09-17T02:51:54.366162+0000","last_became_peered":"2024-09-17T02:51:54.366162+0000","last_unstale":"2024-09-17T02:51:55.383585+0000","last_undegraded":"2024-09-17T02:51:55.383585+0000","last_fullsized":"2024-09-17T02:51:55.383585+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:00.221175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.383145+0000","last_change":"2024-09-17T02:51:55.376222+0000","last_active":"2024-09-17T02:51:55.383145+0000","last_peered":"2024-09-17T02:51:55.383145+0000","last_clean":"2024-09-17T02:51:55.383145+0000","last_became_active":"2024-09-17T02:51:55.375846+0000","last_became_peered":"2024-09-17T02:51:55.375846+0000","last_unstale":"2024-09-17T02:51:55.383145+0000","last_undegraded":"2024-09-17T02:51:55.383145+0000","last_fullsized":"2024-09-17T02:51:55.383145+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:20.498345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.14","version":"121'6","reported_seq":19,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.425174+0000","last_change":"2024-09-17T02:51:54.378308+0000","last_active":"2024-09-17T02:51:55.425174+0000","last_peered":"2024-09-17T02:51:55.425174+0000","last_clean":"2024-09-17T02:51:55.425174+0000","last_became_active":"2024-09-17T02:51:54.377162+0000","last_became_peered":"2024-09-17T02:51:54.377162+0000","last_unstale":"2024-09-17T02:51:55.425174+0000","last_undegraded":"2024-09-17T02:51:55.425174+0000","last_fullsized":"2024-09-17T02:51:55.425174+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:06.691611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.389810+0000","last_change":"2024-09-17T02:51:55.386483+0000","last_active":"2024-09-17T02:51:55.389810+0000","last_peered":"2024-09-17T02:51:55.389810+0000","last_clean":"2024-09-17T02:51:55.389810+0000","last_became_active":"2024-09-17T02:51:55.386102+0000","last_became_peered":"2024-09-17T02:51:55.386102+0000","last_unstale":"2024-09-17T02:51:55.389810+0000","last_undegraded":"2024-09-17T02:51:55.389810+0000","last_fullsized":"2024-09-17T02:51:55.389810+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:55:26.353572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.13","version":"121'2","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.415882+0000","last_change":"2024-09-17T02:51:54.393704+0000","last_active":"2024-09-17T02:51:55.415882+0000","last_peered":"2024-09-17T02:51:55.415882+0000","last_clean":"2024-09-17T02:51:55.415882+0000","last_became_active":"2024-09-17T02:51:54.386613+0000","last_became_peered":"2024-09-17T02:51:54.386613+0000","last_unstale":"2024-09-17T02:51:55.415882+0000","last_undegraded":"2024-09-17T02:51:55.415882+0000","last_fullsized":"2024-09-17T02:51:55.415882+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:51:06.226314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407647+0000","last_change":"2024-09-17T02:51:55.374737+0000","last_active":"2024-09-17T02:51:55.407647+0000","last_peered":"2024-09-17T02:51:55.407647+0000","last_clean":"2024-09-17T02:51:55.407647+0000","last_became_active":"2024-09-17T02:51:55.373464+0000","last_became_peered":"2024-09-17T02:51:55.373464+0000","last_unstale":"2024-09-17T02:51:55.407647+0000","last_undegraded":"2024-09-17T02:51:55.407647+0000","last_fullsized":"2024-09-17T02:51:55.407647+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:55.947251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.392051+0000","last_change":"2024-09-17T02:51:54.385926+0000","last_active":"2024-09-17T02:51:55.392051+0000","last_peered":"2024-09-17T02:51:55.392051+0000","last_clean":"2024-09-17T02:51:55.392051+0000","last_became_active":"2024-09-17T02:51:54.385059+0000","last_became_peered":"2024-09-17T02:51:54.385059+0000","last_unstale":"2024-09-17T02:51:55.392051+0000","last_undegraded":"2024-09-17T02:51:55.392051+0000","last_fullsized":"2024-09-17T02:51:55.392051+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:33:41.467258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.360988+0000","last_change":"2024-09-17T02:51:55.397941+0000","last_active":"2024-09-17T02:51:56.360988+0000","last_peered":"2024-09-17T02:51:56.360988+0000","last_clean":"2024-09-17T02:51:56.360988+0000","last_became_active":"2024-09-17T02:51:55.397151+0000","last_became_peered":"2024-09-17T02:51:55.397151+0000","last_unstale":"2024-09-17T02:51:56.360988+0000","last_undegraded":"2024-09-17T02:51:56.360988+0000","last_fullsized":"2024-09-17T02:51:56.360988+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:45:29.462526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.362468+0000","last_change":"2024-09-17T02:51:54.370628+0000","last_active":"2024-09-17T02:51:55.362468+0000","last_peered":"2024-09-17T02:51:55.362468+0000","last_clean":"2024-09-17T02:51:55.362468+0000","last_became_active":"2024-09-17T02:51:54.369820+0000","last_became_peered":"2024-09-17T02:51:54.369820+0000","last_unstale":"2024-09-17T02:51:55.362468+0000","last_undegraded":"2024-09-17T02:51:55.362468+0000","last_fullsized":"2024-09-17T02:51:55.362468+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:10.599327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.392961+0000","last_change":"2024-09-17T02:51:55.390665+0000","last_active":"2024-09-17T02:51:55.392961+0000","last_peered":"2024-09-17T02:51:55.392961+0000","last_clean":"2024-09-17T02:51:55.392961+0000","last_became_active":"2024-09-17T02:51:55.390364+0000","last_became_peered":"2024-09-17T02:51:55.390364+0000","last_unstale":"2024-09-17T02:51:55.392961+0000","last_undegraded":"2024-09-17T02:51:55.392961+0000","last_fullsized":"2024-09-17T02:51:55.392961+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:08:30.741037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.f","version":"121'1","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361568+0000","last_change":"2024-09-17T02:51:54.389138+0000","last_active":"2024-09-17T02:51:56.361568+0000","last_peered":"2024-09-17T02:51:56.361568+0000","last_clean":"2024-09-17T02:51:56.361568+0000","last_became_active":"2024-09-17T02:51:54.388340+0000","last_became_peered":"2024-09-17T02:51:54.388340+0000","last_unstale":"2024-09-17T02:51:56.361568+0000","last_undegraded":"2024-09-17T02:51:56.361568+0000","last_fullsized":"2024-09-17T02:51:56.361568+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:13:27.090224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.389765+0000","last_change":"2024-09-17T02:51:55.386954+0000","last_active":"2024-09-17T02:51:55.389765+0000","last_peered":"2024-09-17T02:51:55.389765+0000","last_clean":"2024-09-17T02:51:55.389765+0000","last_became_active":"2024-09-17T02:51:55.386631+0000","last_became_peered":"2024-09-17T02:51:55.386631+0000","last_unstale":"2024-09-17T02:51:55.389765+0000","last_undegraded":"2024-09-17T02:51:55.389765+0000","last_fullsized":"2024-09-17T02:51:55.389765+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:47:34.172743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.e","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361071+0000","last_change":"2024-09-17T02:51:54.398592+0000","last_active":"2024-09-17T02:51:56.361071+0000","last_peered":"2024-09-17T02:51:56.361071+0000","last_clean":"2024-09-17T02:51:56.361071+0000","last_became_active":"2024-09-17T02:51:54.397941+0000","last_became_peered":"2024-09-17T02:51:54.397941+0000","last_unstale":"2024-09-17T02:51:56.361071+0000","last_undegraded":"2024-09-17T02:51:56.361071+0000","last_fullsized":"2024-09-17T02:51:56.361071+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:07:25.022321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.406739+0000","last_change":"2024-09-17T02:51:55.376825+0000","last_active":"2024-09-17T02:51:55.406739+0000","last_peered":"2024-09-17T02:51:55.406739+0000","last_clean":"2024-09-17T02:51:55.406739+0000","last_became_active":"2024-09-17T02:51:55.374321+0000","last_became_peered":"2024-09-17T02:51:55.374321+0000","last_unstale":"2024-09-17T02:51:55.406739+0000","last_undegraded":"2024-09-17T02:51:55.406739+0000","last_fullsized":"2024-09-17T02:51:55.406739+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:14.958103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.d","version":"121'1","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.383643+0000","last_change":"2024-09-17T02:51:54.374570+0000","last_active":"2024-09-17T02:51:55.383643+0000","last_peered":"2024-09-17T02:51:55.383643+0000","last_clean":"2024-09-17T02:51:55.383643+0000","last_became_active":"2024-09-17T02:51:54.374151+0000","last_became_peered":"2024-09-17T02:51:54.374151+0000","last_unstale":"2024-09-17T02:51:55.383643+0000","last_undegraded":"2024-09-17T02:51:55.383643+0000","last_fullsized":"2024-09-17T02:51:55.383643+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:31.577585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.377377+0000","last_change":"2024-09-17T02:51:55.372551+0000","last_active":"2024-09-17T02:51:55.377377+0000","last_peered":"2024-09-17T02:51:55.377377+0000","last_clean":"2024-09-17T02:51:55.377377+0000","last_became_active":"2024-09-17T02:51:55.372224+0000","last_became_peered":"2024-09-17T02:51:55.372224+0000","last_unstale":"2024-09-17T02:51:55.377377+0000","last_undegraded":"2024-09-17T02:51:55.377377+0000","last_fullsized":"2024-09-17T02:51:55.377377+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:16.522450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.360327+0000","last_change":"2024-09-17T02:51:54.398562+0000","last_active":"2024-09-17T02:51:55.360327+0000","last_peered":"2024-09-17T02:51:55.360327+0000","last_clean":"2024-09-17T02:51:55.360327+0000","last_became_active":"2024-09-17T02:51:54.398210+0000","last_became_peered":"2024-09-17T02:51:54.398210+0000","last_unstale":"2024-09-17T02:51:55.360327+0000","last_undegraded":"2024-09-17T02:51:55.360327+0000","last_fullsized":"2024-09-17T02:51:55.360327+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:39.140524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.409635+0000","last_change":"2024-09-17T02:51:55.383904+0000","last_active":"2024-09-17T02:51:55.409635+0000","last_peered":"2024-09-17T02:51:55.409635+0000","last_clean":"2024-09-17T02:51:55.409635+0000","last_became_active":"2024-09-17T02:51:55.383089+0000","last_became_peered":"2024-09-17T02:51:55.383089+0000","last_unstale":"2024-09-17T02:51:55.409635+0000","last_undegraded":"2024-09-17T02:51:55.409635+0000","last_fullsized":"2024-09-17T02:51:55.409635+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:09.767083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.b","version":"121'2","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.359400+0000","last_change":"2024-09-17T02:51:54.367975+0000","last_active":"2024-09-17T02:51:56.359400+0000","last_peered":"2024-09-17T02:51:56.359400+0000","last_clean":"2024-09-17T02:51:56.359400+0000","last_became_active":"2024-09-17T02:51:54.365955+0000","last_became_peered":"2024-09-17T02:51:54.365955+0000","last_unstale":"2024-09-17T02:51:56.359400+0000","last_undegraded":"2024-09-17T02:51:56.359400+0000","last_fullsized":"2024-09-17T02:51:56.359400+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:33:58.920111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.359324+0000","last_change":"2024-09-17T02:51:55.388787+0000","last_active":"2024-09-17T02:51:56.359324+0000","last_peered":"2024-09-17T02:51:56.359324+0000","last_clean":"2024-09-17T02:51:56.359324+0000","last_became_active":"2024-09-17T02:51:55.387724+0000","last_became_peered":"2024-09-17T02:51:55.387724+0000","last_unstale":"2024-09-17T02:51:56.359324+0000","last_undegraded":"2024-09-17T02:51:56.359324+0000","last_fullsized":"2024-09-17T02:51:56.359324+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:16:54.122842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.359056+0000","last_change":"2024-09-17T02:51:54.361303+0000","last_active":"2024-09-17T02:51:55.359056+0000","last_peered":"2024-09-17T02:51:55.359056+0000","last_clean":"2024-09-17T02:51:55.359056+0000","last_became_active":"2024-09-17T02:51:54.360915+0000","last_became_peered":"2024-09-17T02:51:54.360915+0000","last_unstale":"2024-09-17T02:51:55.359056+0000","last_undegraded":"2024-09-17T02:51:55.359056+0000","last_fullsized":"2024-09-17T02:51:55.359056+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:45:15.518563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.412552+0000","last_change":"2024-09-17T02:51:55.405476+0000","last_active":"2024-09-17T02:51:55.412552+0000","last_peered":"2024-09-17T02:51:55.412552+0000","last_clean":"2024-09-17T02:51:55.412552+0000","last_became_active":"2024-09-17T02:51:55.404359+0000","last_became_peered":"2024-09-17T02:51:55.404359+0000","last_unstale":"2024-09-17T02:51:55.412552+0000","last_undegraded":"2024-09-17T02:51:55.412552+0000","last_fullsized":"2024-09-17T02:51:55.412552+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:43:14.916860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.387334+0000","last_change":"2024-09-17T02:51:54.398509+0000","last_active":"2024-09-17T02:51:55.387334+0000","last_peered":"2024-09-17T02:51:55.387334+0000","last_clean":"2024-09-17T02:51:55.387334+0000","last_became_active":"2024-09-17T02:51:54.397984+0000","last_became_peered":"2024-09-17T02:51:54.397984+0000","last_unstale":"2024-09-17T02:51:55.387334+0000","last_undegraded":"2024-09-17T02:51:55.387334+0000","last_fullsized":"2024-09-17T02:51:55.387334+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:21:51.217194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.387401+0000","last_change":"2024-09-17T02:51:55.386639+0000","last_active":"2024-09-17T02:51:55.387401+0000","last_peered":"2024-09-17T02:51:55.387401+0000","last_clean":"2024-09-17T02:51:55.387401+0000","last_became_active":"2024-09-17T02:51:55.386140+0000","last_became_peered":"2024-09-17T02:51:55.386140+0000","last_unstale":"2024-09-17T02:51:55.387401+0000","last_undegraded":"2024-09-17T02:51:55.387401+0000","last_fullsized":"2024-09-17T02:51:55.387401+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:32:29.529535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.352180+0000","last_change":"2024-09-17T02:51:54.397929+0000","last_active":"2024-09-17T02:51:55.352180+0000","last_peered":"2024-09-17T02:51:55.352180+0000","last_clean":"2024-09-17T02:51:55.352180+0000","last_became_active":"2024-09-17T02:51:54.397540+0000","last_became_peered":"2024-09-17T02:51:54.397540+0000","last_unstale":"2024-09-17T02:51:55.352180+0000","last_undegraded":"2024-09-17T02:51:55.352180+0000","last_fullsized":"2024-09-17T02:51:55.352180+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:15.364789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361406+0000","last_change":"2024-09-17T02:51:55.410609+0000","last_active":"2024-09-17T02:51:56.361406+0000","last_peered":"2024-09-17T02:51:56.361406+0000","last_clean":"2024-09-17T02:51:56.361406+0000","last_became_active":"2024-09-17T02:51:55.398900+0000","last_became_peered":"2024-09-17T02:51:55.398900+0000","last_unstale":"2024-09-17T02:51:56.361406+0000","last_undegraded":"2024-09-17T02:51:56.361406+0000","last_fullsized":"2024-09-17T02:51:56.361406+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:47.994917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.7","version":"121'2","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.418548+0000","last_change":"2024-09-17T02:51:54.391460+0000","last_active":"2024-09-17T02:51:55.418548+0000","last_peered":"2024-09-17T02:51:55.418548+0000","last_clean":"2024-09-17T02:51:55.418548+0000","last_became_active":"2024-09-17T02:51:54.390634+0000","last_became_peered":"2024-09-17T02:51:54.390634+0000","last_unstale":"2024-09-17T02:51:55.418548+0000","last_undegraded":"2024-09-17T02:51:55.418548+0000","last_fullsized":"2024-09-17T02:51:55.418548+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:52:36.869075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407123+0000","last_change":"2024-09-17T02:51:55.404238+0000","last_active":"2024-09-17T02:51:55.407123+0000","last_peered":"2024-09-17T02:51:55.407123+0000","last_clean":"2024-09-17T02:51:55.407123+0000","last_became_active":"2024-09-17T02:51:55.402488+0000","last_became_peered":"2024-09-17T02:51:55.402488+0000","last_unstale":"2024-09-17T02:51:55.407123+0000","last_undegraded":"2024-09-17T02:51:55.407123+0000","last_fullsized":"2024-09-17T02:51:55.407123+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:14:58.349712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.352604+0000","last_change":"2024-09-17T02:51:54.402557+0000","last_active":"2024-09-17T02:51:55.352604+0000","last_peered":"2024-09-17T02:51:55.352604+0000","last_clean":"2024-09-17T02:51:55.352604+0000","last_became_active":"2024-09-17T02:51:54.402264+0000","last_became_peered":"2024-09-17T02:51:54.402264+0000","last_unstale":"2024-09-17T02:51:55.352604+0000","last_undegraded":"2024-09-17T02:51:55.352604+0000","last_fullsized":"2024-09-17T02:51:55.352604+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:33:25.770612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.367910+0000","last_change":"2024-09-17T02:51:55.367220+0000","last_active":"2024-09-17T02:51:55.367910+0000","last_peered":"2024-09-17T02:51:55.367910+0000","last_clean":"2024-09-17T02:51:55.367910+0000","last_became_active":"2024-09-17T02:51:55.366325+0000","last_became_peered":"2024-09-17T02:51:55.366325+0000","last_unstale":"2024-09-17T02:51:55.367910+0000","last_undegraded":"2024-09-17T02:51:55.367910+0000","last_fullsized":"2024-09-17T02:51:55.367910+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:43.545971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.353644+0000","last_change":"2024-09-17T02:51:54.389344+0000","last_active":"2024-09-17T02:51:55.353644+0000","last_peered":"2024-09-17T02:51:55.353644+0000","last_clean":"2024-09-17T02:51:55.353644+0000","last_became_active":"2024-09-17T02:51:54.388921+0000","last_became_peered":"2024-09-17T02:51:54.388921+0000","last_unstale":"2024-09-17T02:51:55.353644+0000","last_undegraded":"2024-09-17T02:51:55.353644+0000","last_fullsized":"2024-09-17T02:51:55.353644+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:14:34.705758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.412746+0000","last_change":"2024-09-17T02:51:55.405978+0000","last_active":"2024-09-17T02:51:55.412746+0000","last_peered":"2024-09-17T02:51:55.412746+0000","last_clean":"2024-09-17T02:51:55.412746+0000","last_became_active":"2024-09-17T02:51:55.405177+0000","last_became_peered":"2024-09-17T02:51:55.405177+0000","last_unstale":"2024-09-17T02:51:55.412746+0000","last_undegraded":"2024-09-17T02:51:55.412746+0000","last_fullsized":"2024-09-17T02:51:55.412746+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:40:47.336476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.361302+0000","last_change":"2024-09-17T02:51:54.388465+0000","last_active":"2024-09-17T02:51:56.361302+0000","last_peered":"2024-09-17T02:51:56.361302+0000","last_clean":"2024-09-17T02:51:56.361302+0000","last_became_active":"2024-09-17T02:51:54.384034+0000","last_became_peered":"2024-09-17T02:51:54.384034+0000","last_unstale":"2024-09-17T02:51:56.361302+0000","last_undegraded":"2024-09-17T02:51:56.361302+0000","last_fullsized":"2024-09-17T02:51:56.361302+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:20:58.728962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-17T02:51:56.359573+0000","last_change":"2024-09-17T02:51:55.387460+0000","last_active":"2024-09-17T02:51:56.359573+0000","last_peered":"2024-09-17T02:51:56.359573+0000","last_clean":"2024-09-17T02:51:56.359573+0000","last_became_active":"2024-09-17T02:51:55.380416+0000","last_became_peered":"2024-09-17T02:51:55.380416+0000","last_unstale":"2024-09-17T02:51:56.359573+0000","last_undegraded":"2024-09-17T02:51:56.359573+0000","last_fullsized":"2024-09-17T02:51:56.359573+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:55:26.530831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.353328+0000","last_change":"2024-09-17T02:51:54.386181+0000","last_active":"2024-09-17T02:51:55.353328+0000","last_peered":"2024-09-17T02:51:55.353328+0000","last_clean":"2024-09-17T02:51:55.353328+0000","last_became_active":"2024-09-17T02:51:54.385849+0000","last_became_peered":"2024-09-17T02:51:54.385849+0000","last_unstale":"2024-09-17T02:51:55.353328+0000","last_undegraded":"2024-09-17T02:51:55.353328+0000","last_fullsized":"2024-09-17T02:51:55.353328+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:28.968467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.419365+0000","last_change":"2024-09-17T02:51:55.394327+0000","last_active":"2024-09-17T02:51:55.419365+0000","last_peered":"2024-09-17T02:51:55.419365+0000","last_clean":"2024-09-17T02:51:55.419365+0000","last_became_active":"2024-09-17T02:51:55.392467+0000","last_became_peered":"2024-09-17T02:51:55.392467+0000","last_unstale":"2024-09-17T02:51:55.419365+0000","last_undegraded":"2024-09-17T02:51:55.419365+0000","last_fullsized":"2024-09-17T02:51:55.419365+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:26:18.014580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.358788+0000","last_change":"2024-09-17T02:51:54.374392+0000","last_active":"2024-09-17T02:51:55.358788+0000","last_peered":"2024-09-17T02:51:55.358788+0000","last_clean":"2024-09-17T02:51:55.358788+0000","last_became_active":"2024-09-17T02:51:54.374037+0000","last_became_peered":"2024-09-17T02:51:54.374037+0000","last_unstale":"2024-09-17T02:51:55.358788+0000","last_undegraded":"2024-09-17T02:51:55.358788+0000","last_fullsized":"2024-09-17T02:51:55.358788+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:59:20.692346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.403150+0000","last_change":"2024-09-17T02:51:55.402631+0000","last_active":"2024-09-17T02:51:55.403150+0000","last_peered":"2024-09-17T02:51:55.403150+0000","last_clean":"2024-09-17T02:51:55.403150+0000","last_became_active":"2024-09-17T02:51:55.402351+0000","last_became_peered":"2024-09-17T02:51:55.402351+0000","last_unstale":"2024-09-17T02:51:55.403150+0000","last_undegraded":"2024-09-17T02:51:55.403150+0000","last_fullsized":"2024-09-17T02:51:55.403150+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:42.844086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.370416+0000","last_change":"2024-09-17T02:51:54.391400+0000","last_active":"2024-09-17T02:51:55.370416+0000","last_peered":"2024-09-17T02:51:55.370416+0000","last_clean":"2024-09-17T02:51:55.370416+0000","last_became_active":"2024-09-17T02:51:54.390736+0000","last_became_peered":"2024-09-17T02:51:54.390736+0000","last_unstale":"2024-09-17T02:51:55.370416+0000","last_undegraded":"2024-09-17T02:51:55.370416+0000","last_fullsized":"2024-09-17T02:51:55.370416+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:03.876506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.403320+0000","last_change":"2024-09-17T02:51:55.379963+0000","last_active":"2024-09-17T02:51:55.403320+0000","last_peered":"2024-09-17T02:51:55.403320+0000","last_clean":"2024-09-17T02:51:55.403320+0000","last_became_active":"2024-09-17T02:51:55.379030+0000","last_became_peered":"2024-09-17T02:51:55.379030+0000","last_unstale":"2024-09-17T02:51:55.403320+0000","last_undegraded":"2024-09-17T02:51:55.403320+0000","last_fullsized":"2024-09-17T02:51:55.403320+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:02:27.950945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.352793+0000","last_change":"2024-09-17T02:51:54.375020+0000","last_active":"2024-09-17T02:51:55.352793+0000","last_peered":"2024-09-17T02:51:55.352793+0000","last_clean":"2024-09-17T02:51:55.352793+0000","last_became_active":"2024-09-17T02:51:54.374520+0000","last_became_peered":"2024-09-17T02:51:54.374520+0000","last_unstale":"2024-09-17T02:51:55.352793+0000","last_undegraded":"2024-09-17T02:51:55.352793+0000","last_fullsized":"2024-09-17T02:51:55.352793+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:43:08.904920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.407057+0000","last_change":"2024-09-17T02:51:55.406339+0000","last_active":"2024-09-17T02:51:55.407057+0000","last_peered":"2024-09-17T02:51:55.407057+0000","last_clean":"2024-09-17T02:51:55.407057+0000","last_became_active":"2024-09-17T02:51:55.406000+0000","last_became_peered":"2024-09-17T02:51:55.406000+0000","last_unstale":"2024-09-17T02:51:55.407057+0000","last_undegraded":"2024-09-17T02:51:55.407057+0000","last_fullsized":"2024-09-17T02:51:55.407057+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:54.332369+0000","last_clean_scrub_stamp":"2024-09-17T02:51:54.332369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:03:35.253348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"121'8","reported_seq":21,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.417793+0000","last_change":"2024-09-17T02:51:54.393643+0000","last_active":"2024-09-17T02:51:55.417793+0000","last_peered":"2024-09-17T02:51:55.417793+0000","last_clean":"2024-09-17T02:51:55.417793+0000","last_became_active":"2024-09-17T02:51:54.392509+0000","last_became_peered":"2024-09-17T02:51:54.392509+0000","last_unstale":"2024-09-17T02:51:55.417793+0000","last_undegraded":"2024-09-17T02:51:55.417793+0000","last_fullsized":"2024-09-17T02:51:55.417793+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:51:53.328483+0000","last_clean_scrub_stamp":"2024-09-17T02:51:53.328483+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:00.221175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":301,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-17T02:51:55.358581+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:51:55.358581+0000","last_peered":"2024-09-17T02:51:55.358581+0000","last_clean":"2024-09-17T02:51:55.358581+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:51:55.358581+0000","last_undegraded":"2024-09-17T02:51:55.358581+0000","last_fullsized":"2024-09-17T02:51:55.358581+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509579,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1499136,"data_stored":700218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509579,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2105344,"data_stored":1292198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509580,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1507328,"data_stored":700274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509578,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":700308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509579,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32676,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339164,"statfs":{"total":96636764160,"available":96603303936,"internally_reserved":0,"allocated":1515520,"data_stored":702412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2101248,"data_stored":1290746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509580,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1511424,"data_stored":700904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32668,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1515520,"data_stored":706932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:51:59.918 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T02:51:59.919 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T02:52:00.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.126+0000 7ff404ba2640 1 -- 172.21.15.131:0/1372838802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3e80040c0 msgr2=0x7ff3e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:00.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.126+0000 7ff404ba2640 1 --2- 172.21.15.131:0/1372838802 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff3e80040c0 0x7ff3e8004810 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7ff40015c9a0 tx=0x7ff3f401cda0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.126+0000 7ff3fed9d640 1 -- 172.21.15.131:0/1372838802 reap_dead start 2024-09-17T02:52:00.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.126+0000 7ff404ba2640 1 -- 172.21.15.131:0/1372838802 shutdown_connections 2024-09-17T02:52:00.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.126+0000 7ff404ba2640 1 -- 172.21.15.131:0/1372838802 >> 172.21.15.131:0/1372838802 conn(0x7ff4000d0490 msgr2=0x7ff40006ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:00.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.126+0000 7ff404ba2640 1 -- 172.21.15.131:0/1372838802 shutdown_connections 2024-09-17T02:52:00.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.126+0000 7ff404ba2640 1 -- 172.21.15.131:0/1372838802 wait complete. 2024-09-17T02:52:00.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.126+0000 7ff404ba2640 1 Processor -- start 2024-09-17T02:52:00.121 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.126+0000 7ff404ba2640 1 -- start start 2024-09-17T02:52:00.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff404ba2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4000c97f0 0x7ff4000c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:00.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff404ba2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4000c9c00 0x7ff4000c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:00.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff404ba2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4000cb290 0x7ff4000cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff404ba2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff400073780 con 0x7ff4000c9c00 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff404ba2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff40015f500 con 0x7ff4000cb290 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff404ba2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff400073900 con 0x7ff4000c97f0 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fdd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4000c9c00 0x7ff4000c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fdd9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4000c9c00 0x7ff4000c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41956/0 (socket says 172.21.15.131:41956) 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fdd9b640 1 -- 172.21.15.131:0/3235410746 learned_addr learned my addr 172.21.15.131:0/3235410746 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fe59c640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4000c97f0 0x7ff4000c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fed9d640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4000cb290 0x7ff4000cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fdd9b640 1 -- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4000c97f0 msgr2=0x7ff4000c4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fdd9b640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4000c97f0 0x7ff4000c4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fdd9b640 1 -- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4000cb290 msgr2=0x7ff4000cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:00.124 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fdd9b640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4000cb290 0x7ff4000cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fdd9b640 1 -- 172.21.15.131:0/3235410746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40014b630 con 0x7ff4000c9c00 2024-09-17T02:52:00.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fdd9b640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4000c9c00 0x7ff4000c5330 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4000c10 tx=0x7ff3f4005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:00.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3eeffd640 1 -- 172.21.15.131:0/3235410746 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40ab040 con 0x7ff4000c9c00 2024-09-17T02:52:00.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff404ba2640 1 -- 172.21.15.131:0/3235410746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4000c59d0 con 0x7ff4000c9c00 2024-09-17T02:52:00.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3eeffd640 1 -- 172.21.15.131:0/3235410746 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3f40a5bb0 con 0x7ff4000c9c00 2024-09-17T02:52:00.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3eeffd640 1 -- 172.21.15.131:0/3235410746 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f40048d0 con 0x7ff4000c9c00 2024-09-17T02:52:00.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff404ba2640 1 -- 172.21.15.131:0/3235410746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4000c6270 con 0x7ff4000c9c00 2024-09-17T02:52:00.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff404ba2640 1 -- 172.21.15.131:0/3235410746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff400073070 con 0x7ff4000c9c00 2024-09-17T02:52:00.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3eeffd640 1 -- 172.21.15.131:0/3235410746 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff3f40c0030 con 0x7ff4000c9c00 2024-09-17T02:52:00.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3eeffd640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3e0073200 0x7ff3e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:00.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3eeffd640 1 -- 172.21.15.131:0/3235410746 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7ff3f412e920 con 0x7ff4000c9c00 2024-09-17T02:52:00.126 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.130+0000 7ff3fe59c640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3e0073200 0x7ff3e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:00.127 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.134+0000 7ff3fe59c640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3e0073200 0x7ff3e0075710 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0006fd0 tx=0x7ff3f0006f20 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:00.129 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.134+0000 7ff3eeffd640 1 -- 172.21.15.131:0/3235410746 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff3f40c7060 con 0x7ff4000c9c00 2024-09-17T02:52:00.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.354+0000 7ff404ba2640 1 -- 172.21.15.131:0/3235410746 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff40015cf50 con 0x7ff4000c9c00 2024-09-17T02:52:00.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.358+0000 7ff3eeffd640 1 -- 172.21.15.131:0/3235410746 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v36) ==== 190+0+147 (secure 0 0 0) 0x7ff40014b630 con 0x7ff4000c9c00 2024-09-17T02:52:00.351 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-09-17T02:52:00.351 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQDx6+hm3Z6uNxAAm1KlE9btaNWNpQoj027DjA== 2024-09-17T02:52:00.351 INFO:teuthology.orchestra.run.smithi131.stdout: caps mds = "allow" 2024-09-17T02:52:00.352 INFO:teuthology.orchestra.run.smithi131.stdout: caps mon = "allow r" 2024-09-17T02:52:00.352 INFO:teuthology.orchestra.run.smithi131.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T02:52:00.352 INFO:teuthology.orchestra.run.smithi131.stderr:updated caps for client.0 2024-09-17T02:52:00.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.358+0000 7ff404ba2640 1 -- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3e0073200 msgr2=0x7ff3e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:00.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.358+0000 7ff404ba2640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3e0073200 0x7ff3e0075710 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0006fd0 tx=0x7ff3f0006f20 comp rx=0 tx=0).stop 2024-09-17T02:52:00.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.358+0000 7ff404ba2640 1 -- 172.21.15.131:0/3235410746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4000c9c00 msgr2=0x7ff4000c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:00.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.358+0000 7ff404ba2640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4000c9c00 0x7ff4000c5330 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4000c10 tx=0x7ff3f4005600 comp rx=0 tx=0).stop 2024-09-17T02:52:00.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.358+0000 7ff404ba2640 1 -- 172.21.15.131:0/3235410746 shutdown_connections 2024-09-17T02:52:00.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.358+0000 7ff404ba2640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff3e0073200 0x7ff3e0075710 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.358+0000 7ff404ba2640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff4000cb290 0x7ff4000cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.358+0000 7ff404ba2640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff4000c9c00 0x7ff4000c5330 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.358+0000 7ff404ba2640 1 --2- 172.21.15.131:0/3235410746 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff4000c97f0 0x7ff4000c4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.362+0000 7ff404ba2640 1 -- 172.21.15.131:0/3235410746 >> 172.21.15.131:0/3235410746 conn(0x7ff4000d0490 msgr2=0x7ff400151820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:00.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.362+0000 7ff404ba2640 1 -- 172.21.15.131:0/3235410746 shutdown_connections 2024-09-17T02:52:00.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.362+0000 7ff404ba2640 1 -- 172.21.15.131:0/3235410746 wait complete. 2024-09-17T02:52:00.373 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:00.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.530+0000 7fb2a523d640 1 -- 172.21.15.131:0/4159006511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28c004810 msgr2=0x7fb28c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:00.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.530+0000 7fb2a523d640 1 --2- 172.21.15.131:0/4159006511 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb28c004810 0x7fb28c004c20 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00b1a60 tx=0x7fb28801d230 comp rx=0 tx=0).stop 2024-09-17T02:52:00.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.530+0000 7fb29f59e640 1 -- 172.21.15.131:0/4159006511 reap_dead start 2024-09-17T02:52:00.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.530+0000 7fb2a523d640 1 -- 172.21.15.131:0/4159006511 shutdown_connections 2024-09-17T02:52:00.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.530+0000 7fb2a523d640 1 -- 172.21.15.131:0/4159006511 >> 172.21.15.131:0/4159006511 conn(0x7fb2a00cf150 msgr2=0x7fb2a00cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:00.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.530+0000 7fb2a523d640 1 -- 172.21.15.131:0/4159006511 shutdown_connections 2024-09-17T02:52:00.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.530+0000 7fb2a523d640 1 -- 172.21.15.131:0/4159006511 wait complete. 2024-09-17T02:52:00.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2a523d640 1 Processor -- start 2024-09-17T02:52:00.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2a523d640 1 -- start start 2024-09-17T02:52:00.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2a523d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb28c004810 0x7fb2a01f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:00.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2a523d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2a00b7640 0x7fb2a0163880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:00.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2a523d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a00b7a50 0x7fb2a01f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:00.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2a523d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a00b4a30 con 0x7fb2a00b7a50 2024-09-17T02:52:00.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2a523d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a00b48b0 con 0x7fb2a00b7640 2024-09-17T02:52:00.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2a523d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a00cca70 con 0x7fb28c004810 2024-09-17T02:52:00.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb29f59e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a00b7a50 0x7fb2a01f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:00.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb29f59e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a00b7a50 0x7fb2a01f8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:41968/0 (socket says 172.21.15.131:41968) 2024-09-17T02:52:00.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb29f59e640 1 -- 172.21.15.131:0/1828555986 learned_addr learned my addr 172.21.15.131:0/1828555986 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:00.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb29ed9d640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb28c004810 0x7fb2a01f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb29e59c640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2a00b7640 0x7fb2a0163880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb29f59e640 1 -- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb28c004810 msgr2=0x7fb2a01f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb29f59e640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb28c004810 0x7fb2a01f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb29f59e640 1 -- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2a00b7640 msgr2=0x7fb2a0163880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb29f59e640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2a00b7640 0x7fb2a0163880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb29f59e640 1 -- 172.21.15.131:0/1828555986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a00d0b00 con 0x7fb2a00b7a50 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb29f59e640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a00b7a50 0x7fb2a01f8a50 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00c5da0 tx=0x7fb2900012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2977fe640 1 -- 172.21.15.131:0/1828555986 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290011460 con 0x7fb2a00b7a50 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2977fe640 1 -- 172.21.15.131:0/1828555986 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb290017070 con 0x7fb2a00b7a50 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2a523d640 1 -- 172.21.15.131:0/1828555986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a0162dc0 con 0x7fb2a00b7a50 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2977fe640 1 -- 172.21.15.131:0/1828555986 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2900119b0 con 0x7fb2a00b7a50 2024-09-17T02:52:00.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.534+0000 7fb2a523d640 1 -- 172.21.15.131:0/1828555986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a0163080 con 0x7fb2a00b7a50 2024-09-17T02:52:00.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.538+0000 7fb2a523d640 1 -- 172.21.15.131:0/1828555986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a015c980 con 0x7fb2a00b7a50 2024-09-17T02:52:00.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.538+0000 7fb2977fe640 1 -- 172.21.15.131:0/1828555986 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb290011b50 con 0x7fb2a00b7a50 2024-09-17T02:52:00.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.538+0000 7fb2977fe640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb280073200 0x7fb280075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:00.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.538+0000 7fb2977fe640 1 -- 172.21.15.131:0/1828555986 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fb290094920 con 0x7fb2a00b7a50 2024-09-17T02:52:00.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.538+0000 7fb29ed9d640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb280073200 0x7fb280075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:00.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.538+0000 7fb29ed9d640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb280073200 0x7fb280075710 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb2980020b0 tx=0x7fb298001260 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:00.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.542+0000 7fb2977fe640 1 -- 172.21.15.131:0/1828555986 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb29005f7b0 con 0x7fb2a00b7a50 2024-09-17T02:52:00.577 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-09-17T02:52:00.582+0000 7fd29f74d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:52:00.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.734+0000 7fb2a523d640 1 -- 172.21.15.131:0/1828555986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2a00b41a0 con 0x7fb2a00b7a50 2024-09-17T02:52:00.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.734+0000 7fb2977fe640 1 -- 172.21.15.131:0/1828555986 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5236 (secure 0 0 0) 0x7fb29005f0b0 con 0x7fb2a00b7a50 2024-09-17T02:52:00.728 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:00.728 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":101,"btime":"2024-09-17T02:51:58:389699+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:51:57.800237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8490":{"gid":8490,"name":"a","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.131:6835/2599844908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2599844908},{"type":"v1","addr":"172.21.15.131:6835","nonce":2599844908}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8490,"qdb_cluster":[8490]},"id":8}]} 2024-09-17T02:52:00.729 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 101 2024-09-17T02:52:00.730 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 -- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb280073200 msgr2=0x7fb280075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb280073200 0x7fb280075710 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb2980020b0 tx=0x7fb298001260 comp rx=0 tx=0).stop 2024-09-17T02:52:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 -- 172.21.15.131:0/1828555986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a00b7a50 msgr2=0x7fb2a01f8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a00b7a50 0x7fb2a01f8a50 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00c5da0 tx=0x7fb2900012b0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 -- 172.21.15.131:0/1828555986 shutdown_connections 2024-09-17T02:52:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb280073200 0x7fb280075710 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb2a00b7a50 0x7fb2a01f8a50 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb2a00b7640 0x7fb2a0163880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 --2- 172.21.15.131:0/1828555986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb28c004810 0x7fb2a01f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 -- 172.21.15.131:0/1828555986 >> 172.21.15.131:0/1828555986 conn(0x7fb2a00cf150 msgr2=0x7fb2a0155c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 -- 172.21.15.131:0/1828555986 shutdown_connections 2024-09-17T02:52:00.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:00.738+0000 7fb2a523d640 1 -- 172.21.15.131:0/1828555986 wait complete. 2024-09-17T02:52:01.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:51:54.764322+0000', 'modified': '2024-09-17T02:51:57.800237+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8490': {'gid': 8490, 'name': 'a', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.131:6835/2599844908', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 2599844908}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 2599844908}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8490, 'qdb_cluster': [8490]} 2024-09-17T02:52:01.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:52:01.153 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T02:52:01.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.322+0000 7fbbfb19d640 1 -- 172.21.15.131:0/150668136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbdc003a60 msgr2=0x7fbbdc003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.316 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.322+0000 7fbbfb19d640 1 --2- 172.21.15.131:0/150668136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbdc003a60 0x7fbbdc003ed0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009b50 tx=0x7fbbf001d250 comp rx=0 tx=0).stop 2024-09-17T02:52:01.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.322+0000 7fbbfb19d640 1 -- 172.21.15.131:0/150668136 shutdown_connections 2024-09-17T02:52:01.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.322+0000 7fbbfb19d640 1 --2- 172.21.15.131:0/150668136 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbdc003a60 0x7fbbdc003ed0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.322+0000 7fbbfb19d640 1 -- 172.21.15.131:0/150668136 >> 172.21.15.131:0/150668136 conn(0x7fbbf40b5170 msgr2=0x7fbbf40b5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:01.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.322+0000 7fbbfb19d640 1 -- 172.21.15.131:0/150668136 shutdown_connections 2024-09-17T02:52:01.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.322+0000 7fbbfb19d640 1 -- 172.21.15.131:0/150668136 wait complete. 2024-09-17T02:52:01.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbfb19d640 1 Processor -- start 2024-09-17T02:52:01.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbfb19d640 1 -- start start 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbfb19d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbdc003a60 0x7fbbf40999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbfb19d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbbf4155ca0 0x7fbbf4154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbfb19d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbf41552f0 0x7fbbf41f9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbfb19d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf406dca0 con 0x7fbbdc003a60 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbfb19d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf406db20 con 0x7fbbf41552f0 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbfb19d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf4074450 con 0x7fbbf4155ca0 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbf8f39640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbdc003a60 0x7fbbf40999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbf8f39640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbdc003a60 0x7fbbf40999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55078/0 (socket says 172.21.15.131:55078) 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbf8f39640 1 -- 172.21.15.131:0/4156657411 learned_addr learned my addr 172.21.15.131:0/4156657411 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbf973a640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbf41552f0 0x7fbbf41f9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbebfff640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbbf4155ca0 0x7fbbf4154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbf8f39640 1 -- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbbf4155ca0 msgr2=0x7fbbf4154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbbf4155ca0 0x7fbbf4154db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbf8f39640 1 -- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbf41552f0 msgr2=0x7fbbf41f9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbf41552f0 0x7fbbf41f9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbf8f39640 1 -- 172.21.15.131:0/4156657411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf4155700 con 0x7fbbdc003a60 2024-09-17T02:52:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbdc003a60 0x7fbbf40999b0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009b50 tx=0x7fbbf00a5ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbe97fa640 1 -- 172.21.15.131:0/4156657411 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00ad4b0 con 0x7fbbdc003a60 2024-09-17T02:52:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbe97fa640 1 -- 172.21.15.131:0/4156657411 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbf00ad650 con 0x7fbbdc003a60 2024-09-17T02:52:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbfb19d640 1 -- 172.21.15.131:0/4156657411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf415aa00 con 0x7fbbdc003a60 2024-09-17T02:52:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbe97fa640 1 -- 172.21.15.131:0/4156657411 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00ada40 con 0x7fbbdc003a60 2024-09-17T02:52:01.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.326+0000 7fbbfb19d640 1 -- 172.21.15.131:0/4156657411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf40afd30 con 0x7fbbdc003a60 2024-09-17T02:52:01.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbe97fa640 1 -- 172.21.15.131:0/4156657411 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbbf00adbe0 con 0x7fbbdc003a60 2024-09-17T02:52:01.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbe97fa640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbe0073200 0x7fbbe0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbe97fa640 1 -- 172.21.15.131:0/4156657411 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fbbf012f050 con 0x7fbbdc003a60 2024-09-17T02:52:01.323 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbebfff640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbe0073200 0x7fbbe0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbc02468000 1 Processor -- start 2024-09-17T02:52:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbc02468000 1 -- start start 2024-09-17T02:52:01.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbebfff640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbe0073200 0x7fbbe0075710 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40ca840 tx=0x7fbbe4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbc02468000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc07589c0 0x55fdc08a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbc02468000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc08a9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbc02468000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbc02468000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fdc069bb40 con 0x55fdc07323c0 2024-09-17T02:52:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbc02468000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fdc068af70 con 0x55fdc07589c0 2024-09-17T02:52:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbc02468000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fdc068ed80 con 0x55fdc0747570 2024-09-17T02:52:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbf973a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbf973a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08a9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55092/0 (socket says 172.21.15.131:55092) 2024-09-17T02:52:01.326 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbf973a640 1 -- 172.21.15.131:0/1902458914 learned_addr learned my addr 172.21.15.131:0/1902458914 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:01.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/1902458914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc07589c0 0x55fdc08a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbebfff640 1 --2- 172.21.15.131:0/1902458914 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc08a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbf973a640 1 -- 172.21.15.131:0/1902458914 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 msgr2=0x55fdc08a9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:01.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbf973a640 1 --2- 172.21.15.131:0/1902458914 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc08a9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbf973a640 1 -- 172.21.15.131:0/1902458914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc07589c0 msgr2=0x55fdc08a8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:01.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbf973a640 1 --2- 172.21.15.131:0/1902458914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc07589c0 0x55fdc08a8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbf973a640 1 -- 172.21.15.131:0/1902458914 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc063e3e0 con 0x55fdc07323c0 2024-09-17T02:52:01.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.330+0000 7fbbf973a640 1 --2- 172.21.15.131:0/1902458914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08a9a40 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4072ff0 tx=0x7fbbec001fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c11f8098f1f5029 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:01.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbda7fc640 1 -- 172.21.15.131:0/1902458914 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec01a070 con 0x55fdc07323c0 2024-09-17T02:52:01.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbda7fc640 1 -- 172.21.15.131:0/1902458914 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbec003b80 con 0x55fdc07323c0 2024-09-17T02:52:01.327 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbda7fc640 1 -- 172.21.15.131:0/1902458914 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbec005110 con 0x55fdc07323c0 2024-09-17T02:52:01.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 -- 172.21.15.131:0/1902458914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 msgr2=0x55fdc08a9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 --2- 172.21.15.131:0/1902458914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08a9a40 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4072ff0 tx=0x7fbbec001fb0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 -- 172.21.15.131:0/1902458914 shutdown_connections 2024-09-17T02:52:01.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 --2- 172.21.15.131:0/1902458914 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08a9a40 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 --2- 172.21.15.131:0/1902458914 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc08a9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 --2- 172.21.15.131:0/1902458914 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc07589c0 0x55fdc08a8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 -- 172.21.15.131:0/1902458914 >> 172.21.15.131:0/1902458914 conn(0x55fdc0348460 msgr2=0x55fdc088c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:01.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 -- 172.21.15.131:0/1902458914 shutdown_connections 2024-09-17T02:52:01.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 -- 172.21.15.131:0/1902458914 wait complete. 2024-09-17T02:52:01.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 -- start start 2024-09-17T02:52:01.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 Processor -- start 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc088b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc088baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc07589c0 0x55fdc088c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fdc0631520 con 0x55fdc07323c0 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fdc06255f0 con 0x55fdc07589c0 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fdc0656ee0 con 0x55fdc0747570 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbc02468000 10 client.0 _ll_register_callbacks cb 0x55fdc06ac7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbf8f39640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc088b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbf8f39640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc088b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55098/0 (socket says 172.21.15.131:55098) 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbf8f39640 1 -- 172.21.15.131:0/208968420 learned_addr learned my addr 172.21.15.131:0/208968420 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbf973a640 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc07589c0 0x55fdc088c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbebfff640 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc088baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbf8f39640 1 -- 172.21.15.131:0/208968420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 msgr2=0x55fdc088baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:01.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc088baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbf8f39640 1 -- 172.21.15.131:0/208968420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc07589c0 msgr2=0x55fdc088c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc07589c0 0x55fdc088c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbf8f39640 1 -- 172.21.15.131:0/208968420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc0635460 con 0x55fdc07323c0 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbebfff640 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc088baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc088b5b0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0098c40 tx=0x7fbbf00f9380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3fcb2aef6a794e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.334+0000 7fbbd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbbd37fe640 1 -- 172.21.15.131:0/208968420 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf00f9870 con 0x55fdc07323c0 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbbd37fe640 1 -- 172.21.15.131:0/208968420 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fbbf00faba0 con 0x55fdc07323c0 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 -- 172.21.15.131:0/208968420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fdc0623110 con 0x55fdc07323c0 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbbd37fe640 1 -- 172.21.15.131:0/208968420 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7fbbf011b050 con 0x55fdc07323c0 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 10 client.8622 fetch_fsmap learned FSMap version 101 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 -- 172.21.15.131:0/208968420 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55fdc06525b0 con 0x55fdc07323c0 2024-09-17T02:52:01.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbbd37fe640 1 -- 172.21.15.131:0/208968420 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55fdc0623110 con 0x55fdc07323c0 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbbd37fe640 1 -- 172.21.15.131:0/208968420 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7fbbf00b9070 con 0x55fdc07323c0 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 10 client.8622 fetch_fsmap finished waiting for FSMap version 101 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 10 client.8622 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 20 client.8622 populate_metadata read hostname 'smithi131' 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55fdc032d7d0 0x55fdc08ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 4 client.8622 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 -- 172.21.15.131:0/208968420 --> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdc069da90 con 0x55fdc032d7d0 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbbebfff640 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55fdc032d7d0 0x55fdc08ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbbd37fe640 10 client.8622 ms_handle_connect on v2:172.21.15.131:6832/2599844908 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbbebfff640 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55fdc032d7d0 0x55fdc08ba580 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=693b8973e28dcbb8 server_cookie=360d66707b4be8c3 in_seq=0 out_seq=0 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbbd37fe640 1 -- 172.21.15.131:0/208968420 <== mds.0 v2:172.21.15.131:6832/2599844908 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fdc069da90 con 0x55fdc032d7d0 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbbd37fe640 10 client.8622 handle_command_reply: tid=0 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 client.8622 shutdown 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 -- 172.21.15.131:0/208968420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 msgr2=0x55fdc088b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc088b5b0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0098c40 tx=0x7fbbf00f9380 comp rx=0 tx=0).stop 2024-09-17T02:52:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 -- 172.21.15.131:0/208968420 shutdown_connections 2024-09-17T02:52:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55fdc032d7d0 0x55fdc08ba580 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc07589c0 0x55fdc088c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc088baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbbd37fe640 0 client.8622 ms_handle_reset on v2:172.21.15.131:6832/2599844908 2024-09-17T02:52:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 --2- 172.21.15.131:0/208968420 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc088b5b0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 -- 172.21.15.131:0/208968420 >> 172.21.15.131:0/208968420 conn(0x55fdc0348460 msgr2=0x55fdc085d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 -- 172.21.15.131:0/208968420 shutdown_connections 2024-09-17T02:52:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 1 -- 172.21.15.131:0/208968420 wait complete. 2024-09-17T02:52:01.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.338+0000 7fbc02468000 20 client.8622 trim_cache size 0 max 16384 2024-09-17T02:52:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbc02468000 1 Processor -- start 2024-09-17T02:52:01.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbc02468000 1 -- start start 2024-09-17T02:52:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbc02468000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc0348460 0x55fdc08bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbc02468000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08c03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbc02468000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc08ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbc02468000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf4201c10 con 0x55fdc07323c0 2024-09-17T02:52:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbc02468000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fdc069a910 con 0x55fdc0348460 2024-09-17T02:52:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbc02468000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fdc068e090 con 0x55fdc0747570 2024-09-17T02:52:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08c03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08c03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55102/0 (socket says 172.21.15.131:55102) 2024-09-17T02:52:01.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbebfff640 1 -- 172.21.15.131:0/3793817211 learned_addr learned my addr 172.21.15.131:0/3793817211 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/3793817211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc0348460 0x55fdc08bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbebfff640 1 -- 172.21.15.131:0/3793817211 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 msgr2=0x55fdc08ba580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbebfff640 1 --2- 172.21.15.131:0/3793817211 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc08ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbebfff640 1 -- 172.21.15.131:0/3793817211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc0348460 msgr2=0x55fdc08bfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbebfff640 1 --2- 172.21.15.131:0/3793817211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc0348460 0x55fdc08bfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbebfff640 1 -- 172.21.15.131:0/3793817211 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdc0629190 con 0x55fdc07323c0 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/3793817211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc0348460 0x55fdc08bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbebfff640 1 --2- 172.21.15.131:0/3793817211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08c03a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x55fdc0657860 tx=0x7fbbe4017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f63f64cd9ef4d2b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbd9ffb640 1 -- 172.21.15.131:0/3793817211 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe401e040 con 0x55fdc07323c0 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbd9ffb640 1 -- 172.21.15.131:0/3793817211 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbe4003600 con 0x55fdc07323c0 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.358+0000 7fbbd9ffb640 1 -- 172.21.15.131:0/3793817211 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4003a30 con 0x55fdc07323c0 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- 172.21.15.131:0/3793817211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 msgr2=0x55fdc08c03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 --2- 172.21.15.131:0/3793817211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08c03a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x55fdc0657860 tx=0x7fbbe4017040 comp rx=0 tx=0).stop 2024-09-17T02:52:01.355 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- 172.21.15.131:0/3793817211 shutdown_connections 2024-09-17T02:52:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 --2- 172.21.15.131:0/3793817211 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0747570 0x55fdc08ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 --2- 172.21.15.131:0/3793817211 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc08c03a0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 --2- 172.21.15.131:0/3793817211 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc0348460 0x55fdc08bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- 172.21.15.131:0/3793817211 >> 172.21.15.131:0/3793817211 conn(0x55fdc08b5e60 msgr2=0x55fdc08bf750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- 172.21.15.131:0/3793817211 shutdown_connections 2024-09-17T02:52:01.356 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- 172.21.15.131:0/3793817211 wait complete. 2024-09-17T02:52:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- start start 2024-09-17T02:52:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 Processor -- start 2024-09-17T02:52:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0348460 0x55fdc0923020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc0923560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc0747570 0x55fdc0923aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55fdc0656ee0 con 0x55fdc07323c0 2024-09-17T02:52:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55fdc06933b0 con 0x55fdc0747570 2024-09-17T02:52:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55fdc0631520 con 0x55fdc0348460 2024-09-17T02:52:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 10 client.0 _ll_register_callbacks cb 0x7fbbf0005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:52:01.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc0923560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbebfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc0923560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55104/0 (socket says 172.21.15.131:55104) 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbebfff640 1 -- 172.21.15.131:0/46975731 learned_addr learned my addr 172.21.15.131:0/46975731 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbf973a640 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc0747570 0x55fdc0923aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0348460 0x55fdc0923020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbebfff640 1 -- 172.21.15.131:0/46975731 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0348460 msgr2=0x55fdc0923020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbebfff640 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0348460 0x55fdc0923020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbebfff640 1 -- 172.21.15.131:0/46975731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc0747570 msgr2=0x55fdc0923aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbebfff640 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc0747570 0x55fdc0923aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbebfff640 1 -- 172.21.15.131:0/46975731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdc0651ab0 con 0x55fdc07323c0 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0348460 0x55fdc0923020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbebfff640 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc0923560 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4005040 tx=0x7fbbe40038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c915670015a0b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbd37fe640 1 -- 172.21.15.131:0/46975731 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe4003c30 con 0x55fdc07323c0 2024-09-17T02:52:01.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbd37fe640 1 -- 172.21.15.131:0/46975731 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fbbe4003dd0 con 0x55fdc07323c0 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- 172.21.15.131:0/46975731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbe4003600 con 0x55fdc07323c0 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbd37fe640 1 -- 172.21.15.131:0/46975731 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7fbbe403e050 con 0x55fdc07323c0 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 10 client.8628 fetch_fsmap learned FSMap version 101 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- 172.21.15.131:0/46975731 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55fdc0630070 con 0x55fdc07323c0 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbd37fe640 1 -- 172.21.15.131:0/46975731 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbe4003600 con 0x55fdc07323c0 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbd37fe640 1 -- 172.21.15.131:0/46975731 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7fbbe4041020 con 0x55fdc07323c0 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 10 client.8628 fetch_fsmap finished waiting for FSMap version 101 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 10 client.8628 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 20 client.8628 populate_metadata read hostname 'smithi131' 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55fdc0346e80 0x55fdc08df2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 4 client.8628 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbc02468000 1 -- 172.21.15.131:0/46975731 --> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] -- command(tid 0: {"prefix": "status"}) -- 0x55fdc0699540 con 0x55fdc0346e80 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.362+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55fdc0346e80 0x55fdc08df2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbd37fe640 10 client.8628 ms_handle_connect on v2:172.21.15.131:6832/2599844908 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbf8f39640 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55fdc0346e80 0x55fdc08df2a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e67922929bd9877 server_cookie=c62a352ae16b798a in_seq=0 out_seq=0 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbd37fe640 1 -- 172.21.15.131:0/46975731 <== mds.0 v2:172.21.15.131:6832/2599844908 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55fdc0699540 con 0x55fdc0346e80 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbd37fe640 10 client.8628 handle_command_reply: tid=0 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 1 client.8628 shutdown 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 1 -- 172.21.15.131:0/46975731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 msgr2=0x55fdc0923560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc0923560 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbbe4005040 tx=0x7fbbe40038b0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 1 -- 172.21.15.131:0/46975731 shutdown_connections 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55fdc0346e80 0x55fdc08df2a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55fdc0747570 0x55fdc0923aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55fdc07323c0 0x55fdc0923560 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbd37fe640 0 client.8628 ms_handle_reset on v2:172.21.15.131:6832/2599844908 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 1 --2- 172.21.15.131:0/46975731 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55fdc0348460 0x55fdc0923020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 1 -- 172.21.15.131:0/46975731 >> 172.21.15.131:0/46975731 conn(0x55fdc08b5e60 msgr2=0x55fdc08c0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:01.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 1 -- 172.21.15.131:0/46975731 shutdown_connections 2024-09-17T02:52:01.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 1 -- 172.21.15.131:0/46975731 wait complete. 2024-09-17T02:52:01.364 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbc02468000 20 client.8628 trim_cache size 0 max 16384 2024-09-17T02:52:01.364 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:01.364 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:52:01.364 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:52:01.364 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8490, 2024-09-17T02:52:01.364 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:52:01.364 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:52:01.364 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:52:01.365 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 6.0180296789999996, 2024-09-17T02:52:01.365 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 101, 2024-09-17T02:52:01.365 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 122, 2024-09-17T02:52:01.365 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 122, 2024-09-17T02:52:01.365 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 11.253875678 2024-09-17T02:52:01.365 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 -- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbe0073200 msgr2=0x7fbbe0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbe0073200 0x7fbbe0075710 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40ca840 tx=0x7fbbe4009040 comp rx=0 tx=0).stop 2024-09-17T02:52:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 -- 172.21.15.131:0/4156657411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbdc003a60 msgr2=0x7fbbf40999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbdc003a60 0x7fbbf40999b0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009b50 tx=0x7fbbf00a5ed0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 -- 172.21.15.131:0/4156657411 shutdown_connections 2024-09-17T02:52:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbbe0073200 0x7fbbe0075710 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbbf41552f0 0x7fbbf41f9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbbf4155ca0 0x7fbbf4154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 --2- 172.21.15.131:0/4156657411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbbdc003a60 0x7fbbf40999b0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 -- 172.21.15.131:0/4156657411 >> 172.21.15.131:0/4156657411 conn(0x7fbbf40b5170 msgr2=0x7fbbf40d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:01.366 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 -- 172.21.15.131:0/4156657411 shutdown_connections 2024-09-17T02:52:01.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.366+0000 7fbbfb19d640 1 -- 172.21.15.131:0/4156657411 wait complete. 2024-09-17T02:52:01.378 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:52:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.522+0000 7f3a4ccea640 1 -- 172.21.15.131:0/3839048644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a30003a40 msgr2=0x7f3a30003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.522+0000 7f3a4ccea640 1 --2- 172.21.15.131:0/3839048644 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a30003a40 0x7f3a30003eb0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f3a380016f0 tx=0x7f3a3801c7b0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.522+0000 7f3a46d9d640 1 -- 172.21.15.131:0/3839048644 reap_dead start 2024-09-17T02:52:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.522+0000 7f3a4ccea640 1 -- 172.21.15.131:0/3839048644 shutdown_connections 2024-09-17T02:52:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.522+0000 7f3a4ccea640 1 -- 172.21.15.131:0/3839048644 >> 172.21.15.131:0/3839048644 conn(0x7f3a4806eea0 msgr2=0x7f3a4806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 -- 172.21.15.131:0/3839048644 shutdown_connections 2024-09-17T02:52:01.518 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 -- 172.21.15.131:0/3839048644 wait complete. 2024-09-17T02:52:01.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 Processor -- start 2024-09-17T02:52:01.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 -- start start 2024-09-17T02:52:01.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a30003a40 0x7f3a481fac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a481f6c50 0x7f3a48152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4814e1b0 0x7f3a480b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3a48168c60 con 0x7f3a4814e1b0 2024-09-17T02:52:01.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3a48168ae0 con 0x7f3a481f6c50 2024-09-17T02:52:01.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3a48168de0 con 0x7f3a30003a40 2024-09-17T02:52:01.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a46d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4814e1b0 0x7f3a480b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a46d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4814e1b0 0x7f3a480b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55120/0 (socket says 172.21.15.131:55120) 2024-09-17T02:52:01.522 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a46d9d640 1 -- 172.21.15.131:0/655432308 learned_addr learned my addr 172.21.15.131:0/655432308 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a45d9b640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a481f6c50 0x7f3a48152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4659c640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a30003a40 0x7f3a481fac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a46d9d640 1 -- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a30003a40 msgr2=0x7f3a481fac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a46d9d640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a30003a40 0x7f3a481fac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a46d9d640 1 -- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a481f6c50 msgr2=0x7f3a48152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a46d9d640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a481f6c50 0x7f3a48152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a46d9d640 1 -- 172.21.15.131:0/655432308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a481529a0 con 0x7f3a4814e1b0 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a46d9d640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4814e1b0 0x7f3a480b3f40 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f3a480b6130 tx=0x7f3a4000fe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a36ffd640 1 -- 172.21.15.131:0/655432308 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a40004b30 con 0x7f3a4814e1b0 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a36ffd640 1 -- 172.21.15.131:0/655432308 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a40004dc0 con 0x7f3a4814e1b0 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 -- 172.21.15.131:0/655432308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a481f71f0 con 0x7f3a4814e1b0 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a36ffd640 1 -- 172.21.15.131:0/655432308 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4000dd00 con 0x7f3a4814e1b0 2024-09-17T02:52:01.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.526+0000 7f3a4ccea640 1 -- 172.21.15.131:0/655432308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a481f77b0 con 0x7f3a4814e1b0 2024-09-17T02:52:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.530+0000 7f3a4ccea640 1 -- 172.21.15.131:0/655432308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a480c4580 con 0x7f3a4814e1b0 2024-09-17T02:52:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.530+0000 7f3a36ffd640 1 -- 172.21.15.131:0/655432308 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3a4001c020 con 0x7f3a4814e1b0 2024-09-17T02:52:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.530+0000 7f3a36ffd640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a28073200 0x7f3a28075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.530+0000 7f3a36ffd640 1 -- 172.21.15.131:0/655432308 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7f3a400941b0 con 0x7f3a4814e1b0 2024-09-17T02:52:01.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.530+0000 7f3a4659c640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a28073200 0x7f3a28075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:01.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.530+0000 7f3a4659c640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a28073200 0x7f3a28075710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3a380016c0 tx=0x7f3a380a8040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:01.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.534+0000 7f3a36ffd640 1 -- 172.21.15.131:0/655432308 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3a4005efc0 con 0x7f3a4814e1b0 2024-09-17T02:52:01.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.726+0000 7f3a4ccea640 1 -- 172.21.15.131:0/655432308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3a4814ea10 con 0x7f3a4814e1b0 2024-09-17T02:52:01.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.726+0000 7f3a36ffd640 1 -- 172.21.15.131:0/655432308 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) ==== 53+0+73 (secure 0 0 0) 0x7f3a4005e8c0 con 0x7f3a4814e1b0 2024-09-17T02:52:01.721 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:52:01.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 -- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a28073200 msgr2=0x7f3a28075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a28073200 0x7f3a28075710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3a380016c0 tx=0x7f3a380a8040 comp rx=0 tx=0).stop 2024-09-17T02:52:01.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 -- 172.21.15.131:0/655432308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4814e1b0 msgr2=0x7f3a480b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:01.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4814e1b0 0x7f3a480b3f40 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f3a480b6130 tx=0x7f3a4000fe10 comp rx=0 tx=0).stop 2024-09-17T02:52:01.723 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 -- 172.21.15.131:0/655432308 shutdown_connections 2024-09-17T02:52:01.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3a28073200 0x7f3a28075710 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3a4814e1b0 0x7f3a480b3f40 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3a481f6c50 0x7f3a48152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 --2- 172.21.15.131:0/655432308 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3a30003a40 0x7f3a481fac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:01.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 -- 172.21.15.131:0/655432308 >> 172.21.15.131:0/655432308 conn(0x7f3a4806eea0 msgr2=0x7f3a48159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:01.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 -- 172.21.15.131:0/655432308 shutdown_connections 2024-09-17T02:52:01.724 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01.730+0000 7f3a4ccea640 1 -- 172.21.15.131:0/655432308 wait complete. 2024-09-17T02:52:01.741 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:52:01.742 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:52:01.742 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:52:01.742 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:52:01.742 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:01.742 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:52:01.742 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:52:01.742 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:52:01.742 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:01.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:01.742 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:52:01.748 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:52:01.748 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:52:01.748 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:52:01.748 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:01.748 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:52:01.748 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41784sec preferred_lft 41784sec 2024-09-17T02:52:01.749 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:52:01.750 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:01.750 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:52:01.750 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:01.750 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:52:01.750 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:01.750 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:52:01.750 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:01.750 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:52:01.750 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:01.750 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:52:01.751 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:01.751 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:52:01.751 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:01.751 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:52:01.751 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:01.751 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:52:01.751 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:01.751 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:52:01.751 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:01.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:01.752 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:52:01.800 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:52:01.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:01.801 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:52:01.853 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:52:01.853 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:52:01.853 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:52:01.854 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:52:01.854 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:52:01.854 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:52:01.854 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:52:01.854 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:01.854 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:01.854 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:01.854 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:52:01.855 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:52:01.855 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:52:01.855 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:52:01.855 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:52:01.855 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:52:01.855 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:52:01.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:52:01.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:01.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:01.973 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:52:02.018 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:52:02.018 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:52:02.019 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:02.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:02.019 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:52:02.074 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:52:02.075 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:52:02.075 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:02.127 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:02.127 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:52:02.127 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:02.174 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:02.233 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:02.281 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:02.281 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:02.281 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:02.282 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:02.283 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:02.284 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:02.284 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:02.284 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:02.284 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:02.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:02.285 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:02.330 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:52:02.331 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:52:02.376 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:02.389 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:02.435 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:02.435 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:02.435 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:02.435 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:02.435 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:02.435 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:02.435 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:02.435 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:02.435 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:02.436 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:02.437 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:02.437 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:02.437 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:02.437 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:02.437 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:02.437 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:02.437 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:02.437 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:02.437 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:02.437 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:02.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:02.438 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:02.489 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:52:02.490+0000 7f82ef1ee580 -1 init, newargv = 0x7f82dc0ad020 newargc=15 2024-09-17T02:52:02.489 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[51336]: starting ceph client 2024-09-17T02:52:02.491 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[51336]: starting fuse 2024-09-17T02:52:03.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:03.608 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:03.655 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:03.655 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:03.655 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:03.656 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:03.657 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:03.658 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:03.658 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:03.658 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:03.658 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:03.658 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:03.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:03.659 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:03.706 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:52:03.706 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:52:03.707 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:52:03.708 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:03.831 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.51336.asok 2024-09-17T02:52:03.835 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:03.842 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51336.asok 2024-09-17T02:52:03.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:03.843 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51336.asok status 2024-09-17T02:52:04.027 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:04.027 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:52:04.027 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:52:04.027 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:52:04.027 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:52:04.028 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:52:04.028 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:52:04.028 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "51336", 2024-09-17T02:52:04.028 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:52:04.028 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:04.029 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:52:04.029 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:52:04.029 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8643, 2024-09-17T02:52:04.029 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:52:04.029 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:52:04.030 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:52:04.030 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 8643 2024-09-17T02:52:04.030 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:04.030 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:52:04.030 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:52:04.030 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:52:04.031 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3431102630 2024-09-17T02:52:04.031 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:52:04.031 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:04.031 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:52:04.031 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:52:04.032 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:52:04.032 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3431102630 2024-09-17T02:52:04.032 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:04.032 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.8643 192.168.0.1:0/3431102630", 2024-09-17T02:52:04.032 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/3431102630", 2024-09-17T02:52:04.032 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:52:04.033 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 101, 2024-09-17T02:52:04.033 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 122, 2024-09-17T02:52:04.033 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:52:04.033 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:52:04.033 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:52:04.033 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:04.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:04.043 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:52:04.043 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:04.050 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:52:04.050 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:04.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:04.050 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:04.108 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:04.108 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:04.108 DEBUG:teuthology.orchestra.run.smithi131:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:52:04.108 DEBUG:teuthology.orchestra.run.smithi131:> parent = os.path.join(mount_point, "parent") 2024-09-17T02:52:04.108 DEBUG:teuthology.orchestra.run.smithi131:> os.mkdir(parent) 2024-09-17T02:52:04.108 DEBUG:teuthology.orchestra.run.smithi131:> flushed = os.path.join(parent, "flushed") 2024-09-17T02:52:04.108 DEBUG:teuthology.orchestra.run.smithi131:> os.mkdir(flushed) 2024-09-17T02:52:04.108 DEBUG:teuthology.orchestra.run.smithi131:> for f in ["alpha", "bravo", "charlie"]: 2024-09-17T02:52:04.108 DEBUG:teuthology.orchestra.run.smithi131:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-09-17T02:52:04.108 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:04.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:04.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:04.252 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-17T02:52:04.316 INFO:teuthology.orchestra.run.smithi131.stdout:./ 2024-09-17T02:52:04.316 INFO:teuthology.orchestra.run.smithi131.stdout:./parent 2024-09-17T02:52:04.316 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/flushed 2024-09-17T02:52:04.316 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/flushed/charlie 2024-09-17T02:52:04.316 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/flushed/alpha 2024-09-17T02:52:04.316 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/flushed/bravo 2024-09-17T02:52:04.317 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:04.317 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:04.317 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:04.317 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:04.317 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-17T02:52:04.317 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:04.425 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:52:04.427 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:04.435 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:04.435 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:04.435 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:04.435 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:04.435 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-09-17T02:52:04.436 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:04.537 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627776 2024-09-17T02:52:04.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:04.548 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:04.548 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:04.548 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:04.548 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:04.548 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-09-17T02:52:04.548 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:04.650 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627777 2024-09-17T02:52:04.652 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:04.660 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:04.660 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:04.660 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:04.660 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:04.660 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-09-17T02:52:04.660 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:04.762 INFO:teuthology.orchestra.run.smithi131.stdout:1099511628280 2024-09-17T02:52:04.764 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:04.772 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:04.772 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:04.772 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:04.772 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:04.773 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-09-17T02:52:04.773 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:04.874 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627778 2024-09-17T02:52:04.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:04.884 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:04.884 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:04.885 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:04.885 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:04.885 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-09-17T02:52:04.885 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:04.991 INFO:teuthology.orchestra.run.smithi131.stdout:1099511628279 2024-09-17T02:52:04.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:05.001 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:05.001 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:05.047 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:52:05.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.262+0000 7fbfe9045640 1 -- 172.21.15.131:0/1239569119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfcc004810 msgr2=0x7fbfcc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:05.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.262+0000 7fbfe9045640 1 --2- 172.21.15.131:0/1239569119 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfcc004810 0x7fbfcc004c20 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fbfd8009b40 tx=0x7fbfd801d270 comp rx=0 tx=0).stop 2024-09-17T02:52:05.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.262+0000 7fbfe359e640 1 -- 172.21.15.131:0/1239569119 reap_dead start 2024-09-17T02:52:05.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.262+0000 7fbfe9045640 1 -- 172.21.15.131:0/1239569119 shutdown_connections 2024-09-17T02:52:05.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.262+0000 7fbfe9045640 1 -- 172.21.15.131:0/1239569119 >> 172.21.15.131:0/1239569119 conn(0x7fbfe406eea0 msgr2=0x7fbfe406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:05.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.262+0000 7fbfe9045640 1 -- 172.21.15.131:0/1239569119 shutdown_connections 2024-09-17T02:52:05.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.262+0000 7fbfe9045640 1 -- 172.21.15.131:0/1239569119 wait complete. 2024-09-17T02:52:05.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe9045640 1 Processor -- start 2024-09-17T02:52:05.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe9045640 1 -- start start 2024-09-17T02:52:05.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe9045640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfcc004810 0x7fbfe41588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:05.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe9045640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbfe41544b0 0x7fbfe4150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:05.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe9045640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfe415dbe0 0x7fbfe415dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:05.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe9045640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe406f9c0 con 0x7fbfcc004810 2024-09-17T02:52:05.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe9045640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe406e630 con 0x7fbfe415dbe0 2024-09-17T02:52:05.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe9045640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe406fb40 con 0x7fbfe41544b0 2024-09-17T02:52:05.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfcc004810 0x7fbfe41588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:05.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfcc004810 0x7fbfe41588e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55140/0 (socket says 172.21.15.131:55140) 2024-09-17T02:52:05.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe2d9d640 1 -- 172.21.15.131:0/72580308 learned_addr learned my addr 172.21.15.131:0/72580308 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:05.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe359e640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfe415dbe0 0x7fbfe415dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:05.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe2d9d640 1 -- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbfe41544b0 msgr2=0x7fbfe4150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:05.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe259c640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbfe41544b0 0x7fbfe4150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:05.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe2d9d640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbfe41544b0 0x7fbfe4150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe2d9d640 1 -- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfe415dbe0 msgr2=0x7fbfe415dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:05.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe2d9d640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfe415dbe0 0x7fbfe415dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe2d9d640 1 -- 172.21.15.131:0/72580308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe415a1e0 con 0x7fbfcc004810 2024-09-17T02:52:05.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe359e640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfe415dbe0 0x7fbfe415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:05.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe2d9d640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfcc004810 0x7fbfe41588e0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4004ee0 tx=0x7fbfd4001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:05.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfcb7fe640 1 -- 172.21.15.131:0/72580308 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4015020 con 0x7fbfcc004810 2024-09-17T02:52:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfcb7fe640 1 -- 172.21.15.131:0/72580308 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfd4011960 con 0x7fbfcc004810 2024-09-17T02:52:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe9045640 1 -- 172.21.15.131:0/72580308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfe415e530 con 0x7fbfcc004810 2024-09-17T02:52:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfcb7fe640 1 -- 172.21.15.131:0/72580308 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfd4011d90 con 0x7fbfcc004810 2024-09-17T02:52:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.266+0000 7fbfe9045640 1 -- 172.21.15.131:0/72580308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfe416ba60 con 0x7fbfcc004810 2024-09-17T02:52:05.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.270+0000 7fbfe9045640 1 -- 172.21.15.131:0/72580308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfe4073430 con 0x7fbfcc004810 2024-09-17T02:52:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.270+0000 7fbfcb7fe640 1 -- 172.21.15.131:0/72580308 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fbfd4018020 con 0x7fbfcc004810 2024-09-17T02:52:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.270+0000 7fbfcb7fe640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbfd0073200 0x7fbfd0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.270+0000 7fbfcb7fe640 1 -- 172.21.15.131:0/72580308 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fbfd4094e40 con 0x7fbfcc004810 2024-09-17T02:52:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.270+0000 7fbfe259c640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbfd0073200 0x7fbfd0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:05.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.270+0000 7fbfe259c640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbfd0073200 0x7fbfd0075710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80ad000 tx=0x7fbfd801a010 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:05.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.274+0000 7fbfcb7fe640 1 -- 172.21.15.131:0/72580308 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fbfd401f020 con 0x7fbfcc004810 2024-09-17T02:52:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.454+0000 7fbfe9045640 1 -- 172.21.15.131:0/72580308 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbfe406e9d0 con 0x7fbfcc004810 2024-09-17T02:52:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.454+0000 7fbfcb7fe640 1 -- 172.21.15.131:0/72580308 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v122) ==== 80+0+0 (secure 0 0 0) 0x7fbfe415e530 con 0x7fbfcc004810 2024-09-17T02:52:05.449 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:52:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 -- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbfd0073200 msgr2=0x7fbfd0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbfd0073200 0x7fbfd0075710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbfd80ad000 tx=0x7fbfd801a010 comp rx=0 tx=0).stop 2024-09-17T02:52:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 -- 172.21.15.131:0/72580308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfcc004810 msgr2=0x7fbfe41588e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:05.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfcc004810 0x7fbfe41588e0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4004ee0 tx=0x7fbfd4001640 comp rx=0 tx=0).stop 2024-09-17T02:52:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 -- 172.21.15.131:0/72580308 shutdown_connections 2024-09-17T02:52:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fbfd0073200 0x7fbfd0075710 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fbfe415dbe0 0x7fbfe415dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fbfe41544b0 0x7fbfe4150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 --2- 172.21.15.131:0/72580308 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fbfcc004810 0x7fbfe41588e0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 -- 172.21.15.131:0/72580308 >> 172.21.15.131:0/72580308 conn(0x7fbfe406eea0 msgr2=0x7fbfe40b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 -- 172.21.15.131:0/72580308 shutdown_connections 2024-09-17T02:52:05.453 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.458+0000 7fbfe9045640 1 -- 172.21.15.131:0/72580308 wait complete. 2024-09-17T02:52:05.472 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:05.472 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:05.477 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:52:05.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.678+0000 7fcb79242640 1 -- 172.21.15.131:0/1690008015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb740ced00 msgr2=0x7fcb7406dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:05.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.678+0000 7fcb79242640 1 --2- 172.21.15.131:0/1690008015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb740ced00 0x7fcb7406dea0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c009840 tx=0x7fcb5c01c820 comp rx=0 tx=0).stop 2024-09-17T02:52:05.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.678+0000 7fcb79242640 1 -- 172.21.15.131:0/1690008015 shutdown_connections 2024-09-17T02:52:05.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.678+0000 7fcb79242640 1 --2- 172.21.15.131:0/1690008015 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb7406e3e0 0x7fcb7415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.678+0000 7fcb79242640 1 --2- 172.21.15.131:0/1690008015 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb740ced00 0x7fcb7406dea0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.678+0000 7fcb79242640 1 -- 172.21.15.131:0/1690008015 >> 172.21.15.131:0/1690008015 conn(0x7fcb7406eea0 msgr2=0x7fcb7406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:05.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.678+0000 7fcb79242640 1 -- 172.21.15.131:0/1690008015 shutdown_connections 2024-09-17T02:52:05.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.678+0000 7fcb79242640 1 -- 172.21.15.131:0/1690008015 wait complete. 2024-09-17T02:52:05.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb79242640 1 Processor -- start 2024-09-17T02:52:05.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb79242640 1 -- start start 2024-09-17T02:52:05.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb79242640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb7406e3e0 0x7fcb741652e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:05.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb79242640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb740ced00 0x7fcb740cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:05.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb79242640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb7415c3e0 0x7fcb740ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:05.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb79242640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb74168b60 con 0x7fcb740ced00 2024-09-17T02:52:05.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb79242640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcb741689e0 con 0x7fcb7415c3e0 2024-09-17T02:52:05.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb79242640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcb74168ce0 con 0x7fcb7406e3e0 2024-09-17T02:52:05.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb7259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb740ced00 0x7fcb740cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:05.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb7259c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb740ced00 0x7fcb740cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:55166/0 (socket says 172.21.15.131:55166) 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb7259c640 1 -- 172.21.15.131:0/2045063492 learned_addr learned my addr 172.21.15.131:0/2045063492 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb7359e640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb7415c3e0 0x7fcb740ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb72d9d640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb7406e3e0 0x7fcb741652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb7259c640 1 -- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb7406e3e0 msgr2=0x7fcb741652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb7259c640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb7406e3e0 0x7fcb741652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb7259c640 1 -- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb7415c3e0 msgr2=0x7fcb740ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb7259c640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb7415c3e0 0x7fcb740ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb7259c640 1 -- 172.21.15.131:0/2045063492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb741fa560 con 0x7fcb740ced00 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.682+0000 7fcb7259c640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb740ced00 0x7fcb740cdbf0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400d950 tx=0x7fcb6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.686+0000 7fcb6b7fe640 1 -- 172.21.15.131:0/2045063492 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb64017070 con 0x7fcb740ced00 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.686+0000 7fcb6b7fe640 1 -- 172.21.15.131:0/2045063492 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb64009d20 con 0x7fcb740ced00 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.686+0000 7fcb79242640 1 -- 172.21.15.131:0/2045063492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb741fa840 con 0x7fcb740ced00 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.686+0000 7fcb6b7fe640 1 -- 172.21.15.131:0/2045063492 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb640106c0 con 0x7fcb740ced00 2024-09-17T02:52:05.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.686+0000 7fcb79242640 1 -- 172.21.15.131:0/2045063492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb741588b0 con 0x7fcb740ced00 2024-09-17T02:52:05.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.686+0000 7fcb79242640 1 -- 172.21.15.131:0/2045063492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb741fa2d0 con 0x7fcb740ced00 2024-09-17T02:52:05.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.686+0000 7fcb6b7fe640 1 -- 172.21.15.131:0/2045063492 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcb6401a070 con 0x7fcb740ced00 2024-09-17T02:52:05.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.686+0000 7fcb6b7fe640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb54073230 0x7fcb54075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:05.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.686+0000 7fcb6b7fe640 1 -- 172.21.15.131:0/2045063492 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fcb64094910 con 0x7fcb740ced00 2024-09-17T02:52:05.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.686+0000 7fcb72d9d640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb54073230 0x7fcb54075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:05.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.686+0000 7fcb72d9d640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb54073230 0x7fcb54075740 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0af000 tx=0x7fcb5c01b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:05.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.690+0000 7fcb6b7fe640 1 -- 172.21.15.131:0/2045063492 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fcb6405f880 con 0x7fcb740ced00 2024-09-17T02:52:05.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.874+0000 7fcb79242640 1 -- 172.21.15.131:0/2045063492 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcb7415bea0 con 0x7fcb740ced00 2024-09-17T02:52:05.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.874+0000 7fcb6b7fe640 1 -- 172.21.15.131:0/2045063492 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v122) ==== 80+0+0 (secure 0 0 0) 0x7fcb6405f180 con 0x7fcb740ced00 2024-09-17T02:52:05.867 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:52:05.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 -- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb54073230 msgr2=0x7fcb54075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb54073230 0x7fcb54075740 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0af000 tx=0x7fcb5c01b060 comp rx=0 tx=0).stop 2024-09-17T02:52:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 -- 172.21.15.131:0/2045063492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb740ced00 msgr2=0x7fcb740cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb740ced00 0x7fcb740cdbf0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fcb6400d950 tx=0x7fcb6400de10 comp rx=0 tx=0).stop 2024-09-17T02:52:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 -- 172.21.15.131:0/2045063492 shutdown_connections 2024-09-17T02:52:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcb54073230 0x7fcb54075740 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb7415c3e0 0x7fcb740ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb740ced00 0x7fcb740cdbf0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 --2- 172.21.15.131:0/2045063492 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb7406e3e0 0x7fcb741652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 -- 172.21.15.131:0/2045063492 >> 172.21.15.131:0/2045063492 conn(0x7fcb7406eea0 msgr2=0x7fcb7415d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 -- 172.21.15.131:0/2045063492 shutdown_connections 2024-09-17T02:52:05.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:05.878+0000 7fcb79242640 1 -- 172.21.15.131:0/2045063492 wait complete. 2024-09-17T02:52:05.889 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:52:05.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:05.889 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:05.901 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[51336]: fuse finished with error 0 and tester_r 0 2024-09-17T02:52:05.902 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:52:12.004 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:52:12.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:12.005 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:52:12.013 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-09-17T02:52:12.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.226+0000 7f45ba325640 1 -- 172.21.15.131:0/2836540425 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45b4155240 msgr2=0x7f45b4073430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.226+0000 7f45ba325640 1 --2- 172.21.15.131:0/2836540425 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45b4155240 0x7f45b4073430 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009840 tx=0x7f45a401c7e0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.226+0000 7f45ba325640 1 -- 172.21.15.131:0/2836540425 shutdown_connections 2024-09-17T02:52:12.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.226+0000 7f45ba325640 1 --2- 172.21.15.131:0/2836540425 >> v2:172.21.15.189:3300/0 conn(0x7f45b406e5b0 0x7f45b40cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.226+0000 7f45ba325640 1 --2- 172.21.15.131:0/2836540425 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45b4155240 0x7f45b4073430 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.221 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.226+0000 7f45ba325640 1 -- 172.21.15.131:0/2836540425 >> 172.21.15.131:0/2836540425 conn(0x7f45b406eea0 msgr2=0x7f45b406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.226+0000 7f45ba325640 1 -- 172.21.15.131:0/2836540425 shutdown_connections 2024-09-17T02:52:12.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.226+0000 7f45ba325640 1 -- 172.21.15.131:0/2836540425 wait complete. 2024-09-17T02:52:12.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45ba325640 1 Processor -- start 2024-09-17T02:52:12.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45ba325640 1 -- start start 2024-09-17T02:52:12.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45ba325640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45b4073970 0x7f45b414e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45ba325640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b4155240 0x7f45b40cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45ba325640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45b415a060 0x7f45b40cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45ba325640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f45b40cbe70 con 0x7f45b4155240 2024-09-17T02:52:12.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45ba325640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45b40cbcf0 con 0x7f45b4073970 2024-09-17T02:52:12.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45ba325640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f45b4168c80 con 0x7f45b415a060 2024-09-17T02:52:12.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45abfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b4155240 0x7f45b40cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45abfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b4155240 0x7f45b40cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51788/0 (socket says 172.21.15.131:51788) 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45abfff640 1 -- 172.21.15.131:0/3139887873 learned_addr learned my addr 172.21.15.131:0/3139887873 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45b3fff640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45b4073970 0x7f45b414e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45b88c2640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45b415a060 0x7f45b40cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45abfff640 1 -- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45b415a060 msgr2=0x7f45b40cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45abfff640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45b415a060 0x7f45b40cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45abfff640 1 -- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45b4073970 msgr2=0x7f45b414e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45abfff640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45b4073970 0x7f45b414e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45abfff640 1 -- 172.21.15.131:0/3139887873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b414ebd0 con 0x7f45b4155240 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45abfff640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b4155240 0x7f45b40cec40 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f459c00c3f0 tx=0x7f459c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45b17fa640 1 -- 172.21.15.131:0/3139887873 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c017070 con 0x7f45b4155240 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45b17fa640 1 -- 172.21.15.131:0/3139887873 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f459c0053c0 con 0x7f45b4155240 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45b17fa640 1 -- 172.21.15.131:0/3139887873 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c003860 con 0x7f45b4155240 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45ba325640 1 -- 172.21.15.131:0/3139887873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b4152f10 con 0x7f45b4155240 2024-09-17T02:52:12.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.230+0000 7f45ba325640 1 -- 172.21.15.131:0/3139887873 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b40a9510 con 0x7f45b4155240 2024-09-17T02:52:12.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.234+0000 7f45b17fa640 1 -- 172.21.15.131:0/3139887873 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f459c01a070 con 0x7f45b4155240 2024-09-17T02:52:12.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.234+0000 7f45b17fa640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4594073230 0x7f4594075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.234+0000 7f45b17fa640 1 -- 172.21.15.131:0/3139887873 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7f459c093650 con 0x7f45b4155240 2024-09-17T02:52:12.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.234+0000 7f45b3fff640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4594073230 0x7f4594075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.234+0000 7f45c15f0000 1 Processor -- start 2024-09-17T02:52:12.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.234+0000 7f45c15f0000 1 -- start start 2024-09-17T02:52:12.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.234+0000 7f45b3fff640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4594073230 0x7f4594075740 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f45a40af000 tx=0x7f45a401b020 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:12.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc515ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bbffc890 0x55f9bc516530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bc0c0aa0 0x55f9bc516a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f9bc3b0cd0 con 0x55f9bc3a56d0 2024-09-17T02:52:12.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f9bc2c6330 con 0x55f9bc0c0aa0 2024-09-17T02:52:12.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f9bc323bb0 con 0x55f9bbffc890 2024-09-17T02:52:12.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45b3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc515ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45b3fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc515ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51790/0 (socket says 172.21.15.131:51790) 2024-09-17T02:52:12.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45b3fff640 1 -- 172.21.15.131:0/1909101671 learned_addr learned my addr 172.21.15.131:0/1909101671 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:12.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45b88c2640 1 --2- 172.21.15.131:0/1909101671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bc0c0aa0 0x55f9bc516a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45abfff640 1 --2- 172.21.15.131:0/1909101671 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bbffc890 0x55f9bc516530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45b3fff640 1 -- 172.21.15.131:0/1909101671 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bbffc890 msgr2=0x55f9bc516530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:12.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45b3fff640 1 --2- 172.21.15.131:0/1909101671 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bbffc890 0x55f9bc516530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45b3fff640 1 -- 172.21.15.131:0/1909101671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bc0c0aa0 msgr2=0x55f9bc516a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:12.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45b3fff640 1 --2- 172.21.15.131:0/1909101671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bc0c0aa0 0x55f9bc516a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45b3fff640 1 -- 172.21.15.131:0/1909101671 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bc2c7c80 con 0x55f9bc3a56d0 2024-09-17T02:52:12.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45b3fff640 1 --2- 172.21.15.131:0/1909101671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc515ff0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f45a4000c10 tx=0x7f45a40b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc6e212152d4382 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:12.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45a9ffb640 1 -- 172.21.15.131:0/1909101671 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a40b2040 con 0x55f9bc3a56d0 2024-09-17T02:52:12.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45a9ffb640 1 -- 172.21.15.131:0/1909101671 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45a40c7070 con 0x55f9bc3a56d0 2024-09-17T02:52:12.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45a9ffb640 1 -- 172.21.15.131:0/1909101671 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a4003600 con 0x55f9bc3a56d0 2024-09-17T02:52:12.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 -- 172.21.15.131:0/1909101671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 msgr2=0x55f9bc515ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 --2- 172.21.15.131:0/1909101671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc515ff0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f45a4000c10 tx=0x7f45a40b7000 comp rx=0 tx=0).stop 2024-09-17T02:52:12.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 -- 172.21.15.131:0/1909101671 shutdown_connections 2024-09-17T02:52:12.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 --2- 172.21.15.131:0/1909101671 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bc0c0aa0 0x55f9bc516a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 --2- 172.21.15.131:0/1909101671 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bbffc890 0x55f9bc516530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 --2- 172.21.15.131:0/1909101671 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc515ff0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 -- 172.21.15.131:0/1909101671 >> 172.21.15.131:0/1909101671 conn(0x55f9bbfd05c0 msgr2=0x55f9bc4ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:12.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 -- 172.21.15.131:0/1909101671 shutdown_connections 2024-09-17T02:52:12.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 -- 172.21.15.131:0/1909101671 wait complete. 2024-09-17T02:52:12.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 -- start start 2024-09-17T02:52:12.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.238+0000 7f45c15f0000 1 Processor -- start 2024-09-17T02:52:12.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bbffc890 0x55f9bc4d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bc0c0aa0 0x55f9bc4d81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4d8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f9bc316df0 con 0x55f9bc3a56d0 2024-09-17T02:52:12.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55f9bc312fe0 con 0x55f9bbffc890 2024-09-17T02:52:12.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f9bc2b84d0 con 0x55f9bc0c0aa0 2024-09-17T02:52:12.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 10 client.0 _ll_register_callbacks cb 0x55f9bc2967f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b88c2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4d8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b88c2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4d8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51804/0 (socket says 172.21.15.131:51804) 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b88c2640 1 -- 172.21.15.131:0/1996717562 learned_addr learned my addr 172.21.15.131:0/1996717562 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b3fff640 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bbffc890 0x55f9bc4d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45abfff640 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bc0c0aa0 0x55f9bc4d81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b88c2640 1 -- 172.21.15.131:0/1996717562 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bc0c0aa0 msgr2=0x55f9bc4d81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b88c2640 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bc0c0aa0 0x55f9bc4d81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b88c2640 1 -- 172.21.15.131:0/1996717562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bbffc890 msgr2=0x55f9bc4d7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b88c2640 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bbffc890 0x55f9bc4d7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b88c2640 1 -- 172.21.15.131:0/1996717562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bc2b1210 con 0x55f9bc3a56d0 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b88c2640 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4d8730 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00faf0 tx=0x7f45ac00d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6217d44b8c87ad30 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:12.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f4592ffd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:52:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f4592ffd640 1 -- 172.21.15.131:0/1996717562 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00dc60 con 0x55f9bc3a56d0 2024-09-17T02:52:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f4592ffd640 1 -- 172.21.15.131:0/1996717562 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7f45ac00de00 con 0x55f9bc3a56d0 2024-09-17T02:52:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 -- 172.21.15.131:0/1996717562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9bc2ab350 con 0x55f9bc3a56d0 2024-09-17T02:52:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f4592ffd640 1 -- 172.21.15.131:0/1996717562 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f45ac025050 con 0x55f9bc3a56d0 2024-09-17T02:52:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 10 client.8667 fetch_fsmap learned FSMap version 101 2024-09-17T02:52:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 -- 172.21.15.131:0/1996717562 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55f9bc297fc0 con 0x55f9bc3a56d0 2024-09-17T02:52:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f4592ffd640 1 -- 172.21.15.131:0/1996717562 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f9bc2ab350 con 0x55f9bc3a56d0 2024-09-17T02:52:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f4592ffd640 1 -- 172.21.15.131:0/1996717562 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7f45ac028020 con 0x55f9bc3a56d0 2024-09-17T02:52:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 10 client.8667 fetch_fsmap finished waiting for FSMap version 101 2024-09-17T02:52:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 10 client.8667 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-09-17T02:52:12.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 20 client.8667 populate_metadata read hostname 'smithi131' 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55f9bbfb5930 0x55f9bc5275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 4 client.8667 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 -- 172.21.15.131:0/1996717562 --> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9bc325b00 con 0x55f9bbfb5930 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b3fff640 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55f9bbfb5930 0x55f9bc5275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f4592ffd640 10 client.8667 ms_handle_connect on v2:172.21.15.131:6832/2599844908 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45b3fff640 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55f9bbfb5930 0x55f9bc5275b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b1995cced35514 server_cookie=1ec4d86d9fa73b03 in_seq=0 out_seq=0 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f4592ffd640 1 -- 172.21.15.131:0/1996717562 <== mds.0 v2:172.21.15.131:6832/2599844908 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9bc325b00 con 0x55f9bbfb5930 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f4592ffd640 10 client.8667 handle_command_reply: tid=0 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 client.8667 shutdown 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 -- 172.21.15.131:0/1996717562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 msgr2=0x55f9bc4d8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4d8730 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00faf0 tx=0x7f45ac00d3e0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 -- 172.21.15.131:0/1996717562 shutdown_connections 2024-09-17T02:52:12.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55f9bbfb5930 0x55f9bc5275b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4d8730 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bc0c0aa0 0x55f9bc4d81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 --2- 172.21.15.131:0/1996717562 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bbffc890 0x55f9bc4d7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f4592ffd640 0 client.8667 ms_handle_reset on v2:172.21.15.131:6832/2599844908 2024-09-17T02:52:12.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 -- 172.21.15.131:0/1996717562 >> 172.21.15.131:0/1996717562 conn(0x55f9bbfd05c0 msgr2=0x55f9bc4c6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:12.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 -- 172.21.15.131:0/1996717562 shutdown_connections 2024-09-17T02:52:12.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 1 -- 172.21.15.131:0/1996717562 wait complete. 2024-09-17T02:52:12.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.242+0000 7f45c15f0000 20 client.8667 trim_cache size 0 max 16384 2024-09-17T02:52:12.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.258+0000 7f45c15f0000 1 Processor -- start 2024-09-17T02:52:12.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.258+0000 7f45c15f0000 1 -- start start 2024-09-17T02:52:12.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bbffc890 0x55f9bc4c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bc0c0aa0 0x55f9bc4c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4c6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55f9bc4d5d80 con 0x55f9bc3a56d0 2024-09-17T02:52:12.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45b4201090 con 0x55f9bbffc890 2024-09-17T02:52:12.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f45b4158800 con 0x55f9bc0c0aa0 2024-09-17T02:52:12.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45b88c2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45b88c2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51812/0 (socket says 172.21.15.131:51812) 2024-09-17T02:52:12.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45b88c2640 1 -- 172.21.15.131:0/4224193396 learned_addr learned my addr 172.21.15.131:0/4224193396 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:12.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45b88c2640 1 -- 172.21.15.131:0/4224193396 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bc0c0aa0 msgr2=0x55f9bc4c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:12.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45b88c2640 1 --2- 172.21.15.131:0/4224193396 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bc0c0aa0 0x55f9bc4c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45b88c2640 1 -- 172.21.15.131:0/4224193396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bbffc890 msgr2=0x55f9bc4c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:12.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45b88c2640 1 --2- 172.21.15.131:0/4224193396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bbffc890 0x55f9bc4c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45b88c2640 1 -- 172.21.15.131:0/4224193396 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bc370680 con 0x55f9bc3a56d0 2024-09-17T02:52:12.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45b88c2640 1 --2- 172.21.15.131:0/4224193396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4c6360 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f45ac000f90 tx=0x7f45ac004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b6c022e1288bbd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:12.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45a97fa640 1 -- 172.21.15.131:0/4224193396 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0293f0 con 0x55f9bc3a56d0 2024-09-17T02:52:12.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45a97fa640 1 -- 172.21.15.131:0/4224193396 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45ac029590 con 0x55f9bc3a56d0 2024-09-17T02:52:12.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45a97fa640 1 -- 172.21.15.131:0/4224193396 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac0299e0 con 0x55f9bc3a56d0 2024-09-17T02:52:12.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 -- 172.21.15.131:0/4224193396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 msgr2=0x55f9bc4c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 --2- 172.21.15.131:0/4224193396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4c6360 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f45ac000f90 tx=0x7f45ac004ac0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 -- 172.21.15.131:0/4224193396 shutdown_connections 2024-09-17T02:52:12.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 --2- 172.21.15.131:0/4224193396 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc4c6360 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 --2- 172.21.15.131:0/4224193396 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bc0c0aa0 0x55f9bc4c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 --2- 172.21.15.131:0/4224193396 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bbffc890 0x55f9bc4c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 -- 172.21.15.131:0/4224193396 >> 172.21.15.131:0/4224193396 conn(0x55f9bbfd05c0 msgr2=0x55f9bc4d71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:12.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 -- 172.21.15.131:0/4224193396 shutdown_connections 2024-09-17T02:52:12.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 -- 172.21.15.131:0/4224193396 wait complete. 2024-09-17T02:52:12.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 -- start start 2024-09-17T02:52:12.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.262+0000 7f45c15f0000 1 Processor -- start 2024-09-17T02:52:12.261 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bbffc890 0x55f9bc58b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bc0c0aa0 0x55f9bc58ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc58bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41581b0 con 0x55f9bc3a56d0 2024-09-17T02:52:12.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f45b4157b60 con 0x55f9bc0c0aa0 2024-09-17T02:52:12.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55f9bbfd8680 con 0x55f9bbffc890 2024-09-17T02:52:12.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 10 client.0 _ll_register_callbacks cb 0x7f45ac009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:52:12.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b88c2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc58bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b88c2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc58bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51826/0 (socket says 172.21.15.131:51826) 2024-09-17T02:52:12.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b88c2640 1 -- 172.21.15.131:0/4272317718 learned_addr learned my addr 172.21.15.131:0/4272317718 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:12.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45abfff640 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bc0c0aa0 0x55f9bc58ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b3fff640 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bbffc890 0x55f9bc58b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b88c2640 1 -- 172.21.15.131:0/4272317718 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bbffc890 msgr2=0x55f9bc58b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b88c2640 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bbffc890 0x55f9bc58b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b88c2640 1 -- 172.21.15.131:0/4272317718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bc0c0aa0 msgr2=0x55f9bc58ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b88c2640 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bc0c0aa0 0x55f9bc58ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b88c2640 1 -- 172.21.15.131:0/4272317718 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bc2dd0b0 con 0x55f9bc3a56d0 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45abfff640 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bc0c0aa0 0x55f9bc58ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b88c2640 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc58bfa0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f45ac02d040 tx=0x7f45ac02a980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f4521952a3d6a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f4592ffd640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f4592ffd640 1 -- 172.21.15.131:0/4272317718 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac02ad80 con 0x55f9bc3a56d0 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f4592ffd640 1 -- 172.21.15.131:0/4272317718 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7f45ac01e040 con 0x55f9bc3a56d0 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 1 -- 172.21.15.131:0/4272317718 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45ac029590 con 0x55f9bc3a56d0 2024-09-17T02:52:12.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f4592ffd640 1 -- 172.21.15.131:0/4272317718 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f45ac040050 con 0x55f9bc3a56d0 2024-09-17T02:52:12.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 10 client.8673 fetch_fsmap learned FSMap version 101 2024-09-17T02:52:12.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 1 -- 172.21.15.131:0/4272317718 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55f9bc2a9610 con 0x55f9bc3a56d0 2024-09-17T02:52:12.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f4592ffd640 1 -- 172.21.15.131:0/4272317718 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45ac029590 con 0x55f9bc3a56d0 2024-09-17T02:52:12.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f4592ffd640 1 -- 172.21.15.131:0/4272317718 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7f45ac043020 con 0x55f9bc3a56d0 2024-09-17T02:52:12.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 10 client.8673 fetch_fsmap finished waiting for FSMap version 101 2024-09-17T02:52:12.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 10 client.8673 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-09-17T02:52:12.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 20 client.8673 populate_metadata read hostname 'smithi131' 2024-09-17T02:52:12.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55f9bbfa3990 0x55f9bc528590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 4 client.8673 mds_command: new command op to 8490 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-17T02:52:12.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45c15f0000 1 -- 172.21.15.131:0/4272317718 --> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f9bc3215b0 con 0x55f9bbfa3990 2024-09-17T02:52:12.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b3fff640 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55f9bbfa3990 0x55f9bc528590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f4592ffd640 10 client.8673 ms_handle_connect on v2:172.21.15.131:6832/2599844908 2024-09-17T02:52:12.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.266+0000 7f45b3fff640 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55f9bbfa3990 0x55f9bc528590 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=902e6a4cedd280b2 server_cookie=d482fcf0f54fdb23 in_seq=0 out_seq=0 2024-09-17T02:52:12.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f4592ffd640 1 -- 172.21.15.131:0/4272317718 <== mds.0 v2:172.21.15.131:6832/2599844908 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f9bc3215b0 con 0x55f9bbfa3990 2024-09-17T02:52:12.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f4592ffd640 10 client.8673 handle_command_reply: tid=0 2024-09-17T02:52:12.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 1 client.8673 shutdown 2024-09-17T02:52:12.275 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 1 -- 172.21.15.131:0/4272317718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 msgr2=0x55f9bc58bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc58bfa0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f45ac02d040 tx=0x7f45ac02a980 comp rx=0 tx=0).stop 2024-09-17T02:52:12.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 1 -- 172.21.15.131:0/4272317718 shutdown_connections 2024-09-17T02:52:12.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.131:6832/2599844908,v1:172.21.15.131:6835/2599844908] conn(0x55f9bbfa3990 0x55f9bc528590 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55f9bc3a56d0 0x55f9bc58bfa0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55f9bc0c0aa0 0x55f9bc58ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 1 --2- 172.21.15.131:0/4272317718 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55f9bbffc890 0x55f9bc58b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 1 -- 172.21.15.131:0/4272317718 >> 172.21.15.131:0/4272317718 conn(0x55f9bbfd05c0 msgr2=0x55f9bc4ce150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:12.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f4592ffd640 0 client.8673 ms_handle_reset on v2:172.21.15.131:6832/2599844908 2024-09-17T02:52:12.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 1 -- 172.21.15.131:0/4272317718 shutdown_connections 2024-09-17T02:52:12.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 1 -- 172.21.15.131:0/4272317718 wait complete. 2024-09-17T02:52:12.276 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45c15f0000 20 client.8673 trim_cache size 0 max 16384 2024-09-17T02:52:12.277 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:12.277 INFO:teuthology.orchestra.run.smithi131.stdout: "message": "", 2024-09-17T02:52:12.277 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0 2024-09-17T02:52:12.277 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:12.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 -- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4594073230 msgr2=0x7f4594075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4594073230 0x7f4594075740 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f45a40af000 tx=0x7f45a401b020 comp rx=0 tx=0).stop 2024-09-17T02:52:12.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 -- 172.21.15.131:0/3139887873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b4155240 msgr2=0x7f45b40cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b4155240 0x7f45b40cec40 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f459c00c3f0 tx=0x7f459c00c8b0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 -- 172.21.15.131:0/3139887873 shutdown_connections 2024-09-17T02:52:12.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4594073230 0x7f4594075740 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f45b415a060 0x7f45b40cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.278 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f45b4155240 0x7f45b40cec40 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 --2- 172.21.15.131:0/3139887873 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f45b4073970 0x7f45b414e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 -- 172.21.15.131:0/3139887873 >> 172.21.15.131:0/3139887873 conn(0x7f45b406eea0 msgr2=0x7f45b40b0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 -- 172.21.15.131:0/3139887873 shutdown_connections 2024-09-17T02:52:12.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.282+0000 7f45ba325640 1 -- 172.21.15.131:0/3139887873 wait complete. 2024-09-17T02:52:12.296 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:52:12.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.450+0000 7fb65e551640 1 -- 172.21.15.131:0/2102537201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb65806e3e0 msgr2=0x7fb65815bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.450+0000 7fb65e551640 1 --2- 172.21.15.131:0/2102537201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb65806e3e0 0x7fb65815bea0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb65806a1c0 tx=0x7fb64c01c800 comp rx=0 tx=0).stop 2024-09-17T02:52:12.443 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.450+0000 7fb65e551640 1 -- 172.21.15.131:0/2102537201 shutdown_connections 2024-09-17T02:52:12.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.450+0000 7fb65e551640 1 --2- 172.21.15.131:0/2102537201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb65806e3e0 0x7fb65815bea0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.450+0000 7fb65e551640 1 --2- 172.21.15.131:0/2102537201 >> v2:172.21.15.189:3300/0 conn(0x7fb6580ced00 0x7fb65806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.450+0000 7fb65e551640 1 -- 172.21.15.131:0/2102537201 >> 172.21.15.131:0/2102537201 conn(0x7fb65806eea0 msgr2=0x7fb65806f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:12.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.450+0000 7fb65e551640 1 -- 172.21.15.131:0/2102537201 shutdown_connections 2024-09-17T02:52:12.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.450+0000 7fb65e551640 1 -- 172.21.15.131:0/2102537201 wait complete. 2024-09-17T02:52:12.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.450+0000 7fb65e551640 1 Processor -- start 2024-09-17T02:52:12.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.450+0000 7fb65e551640 1 -- start start 2024-09-17T02:52:12.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb65e551640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb65806e3e0 0x7fb6581527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb65e551640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6580ced00 0x7fb658150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb65e551640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb65815c3e0 0x7fb658150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb65e551640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb658166d60 con 0x7fb6580ced00 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb65e551640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb658166be0 con 0x7fb65815c3e0 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb65e551640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb658166ee0 con 0x7fb65806e3e0 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb6577fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6580ced00 0x7fb658150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb6577fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6580ced00 0x7fb658150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51846/0 (socket says 172.21.15.131:51846) 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb6577fe640 1 -- 172.21.15.131:0/1931470593 learned_addr learned my addr 172.21.15.131:0/1931470593 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb65caee640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb65815c3e0 0x7fb658150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb657fff640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb65806e3e0 0x7fb6581527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb6577fe640 1 -- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb65806e3e0 msgr2=0x7fb6581527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb6577fe640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb65806e3e0 0x7fb6581527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb6577fe640 1 -- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb65815c3e0 msgr2=0x7fb658150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb6577fe640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb65815c3e0 0x7fb658150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb6577fe640 1 -- 172.21.15.131:0/1931470593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb658151720 con 0x7fb6580ced00 2024-09-17T02:52:12.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb65caee640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb65815c3e0 0x7fb658150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:12.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb6577fe640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6580ced00 0x7fb658150980 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009790 tx=0x7fb64c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:12.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb654ff9640 1 -- 172.21.15.131:0/1931470593 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0bb070 con 0x7fb6580ced00 2024-09-17T02:52:12.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb654ff9640 1 -- 172.21.15.131:0/1931470593 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb64c005030 con 0x7fb6580ced00 2024-09-17T02:52:12.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb65e551640 1 -- 172.21.15.131:0/1931470593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb658151a00 con 0x7fb6580ced00 2024-09-17T02:52:12.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb654ff9640 1 -- 172.21.15.131:0/1931470593 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0b2040 con 0x7fb6580ced00 2024-09-17T02:52:12.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb65e551640 1 -- 172.21.15.131:0/1931470593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb658151fc0 con 0x7fb6580ced00 2024-09-17T02:52:12.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb65e551640 1 -- 172.21.15.131:0/1931470593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb658151490 con 0x7fb6580ced00 2024-09-17T02:52:12.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb654ff9640 1 -- 172.21.15.131:0/1931470593 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb64c0be070 con 0x7fb6580ced00 2024-09-17T02:52:12.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb654ff9640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb644073230 0x7fb644075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:12.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb654ff9640 1 -- 172.21.15.131:0/1931470593 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fb64c12df90 con 0x7fb6580ced00 2024-09-17T02:52:12.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.454+0000 7fb657fff640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb644073230 0x7fb644075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:12.451 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.458+0000 7fb657fff640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb644073230 0x7fb644075740 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb648005c50 tx=0x7fb648005f50 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:12.454 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.462+0000 7fb654ff9640 1 -- 172.21.15.131:0/1931470593 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb64c0163f0 con 0x7fb6580ced00 2024-09-17T02:52:12.661 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.666+0000 7fb65e551640 1 -- 172.21.15.131:0/1931470593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb65815bea0 con 0x7fb6580ced00 2024-09-17T02:52:12.662 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb654ff9640 1 -- 172.21.15.131:0/1931470593 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) ==== 53+0+73 (secure 0 0 0) 0x7fb64c0f8e20 con 0x7fb6580ced00 2024-09-17T02:52:12.662 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:52:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 -- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb644073230 msgr2=0x7fb644075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb644073230 0x7fb644075740 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb648005c50 tx=0x7fb648005f50 comp rx=0 tx=0).stop 2024-09-17T02:52:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 -- 172.21.15.131:0/1931470593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6580ced00 msgr2=0x7fb658150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6580ced00 0x7fb658150980 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009790 tx=0x7fb64c004040 comp rx=0 tx=0).stop 2024-09-17T02:52:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 -- 172.21.15.131:0/1931470593 shutdown_connections 2024-09-17T02:52:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb644073230 0x7fb644075740 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb65815c3e0 0x7fb658150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb6580ced00 0x7fb658150980 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 --2- 172.21.15.131:0/1931470593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb65806e3e0 0x7fb6581527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 -- 172.21.15.131:0/1931470593 >> 172.21.15.131:0/1931470593 conn(0x7fb65806eea0 msgr2=0x7fb6580cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 -- 172.21.15.131:0/1931470593 shutdown_connections 2024-09-17T02:52:12.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12.670+0000 7fb65e551640 1 -- 172.21.15.131:0/1931470593 wait complete. 2024-09-17T02:52:12.684 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:52:12.684 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:52:12.684 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:52:12.684 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:52:12.684 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:12.685 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:52:12.685 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:52:12.685 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:52:12.685 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:12.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:12.685 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:52:12.691 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:52:12.691 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:52:12.691 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:52:12.691 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:12.691 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:52:12.691 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:12.691 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:12.691 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:12.691 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:52:12.691 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:12.692 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:12.692 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:52:12.692 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:12.692 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:12.692 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:52:12.692 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:12.692 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:52:12.692 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41773sec preferred_lft 41773sec 2024-09-17T02:52:12.692 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:52:12.692 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:12.692 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:52:12.693 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:12.693 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:52:12.693 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:12.693 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:52:12.693 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:12.693 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:52:12.693 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:12.693 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:52:12.693 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:12.693 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:52:12.693 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:12.694 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:52:12.694 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:12.694 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:52:12.694 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:12.694 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:52:12.694 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:12.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:12.694 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:52:12.744 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:52:12.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:12.745 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:52:12.797 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:52:12.797 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:52:12.797 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:52:12.797 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:52:12.798 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:52:12.798 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:52:12.798 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:52:12.798 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:12.798 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:12.798 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:12.798 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:52:12.798 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:52:12.798 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:52:12.799 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:52:12.799 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:52:12.799 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:52:12.799 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:52:12.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:52:12.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:12.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:12.918 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:52:12.966 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:52:12.967 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:52:12.967 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:12.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:12.967 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:52:13.022 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:52:13.022 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:52:13.022 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:13.070 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:13.071 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:52:13.071 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:13.118 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:13.176 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:13.223 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:13.224 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:13.225 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:13.226 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:13.226 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:13.226 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:13.226 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:13.226 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:13.226 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:13.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:13.227 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:13.274 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:52:13.275 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:52:13.320 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:13.334 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:13.383 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:13.383 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:13.383 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:13.383 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:13.383 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:13.383 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:13.383 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:13.383 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:13.383 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:13.383 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:13.384 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:13.385 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:13.385 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:13.385 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:13.385 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:13.385 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:13.385 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:13.385 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:13.385 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:13.385 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:13.385 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:13.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:13.386 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:13.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:52:13.434+0000 7f8a17656580 -1 init, newargv = 0x7f8a0c004f30 newargc=15 2024-09-17T02:52:13.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[51572]: starting ceph client 2024-09-17T02:52:13.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[51572]: starting fuse 2024-09-17T02:52:14.532 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:14.549 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:14.595 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:14.595 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:14.595 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:14.596 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:14.596 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:14.596 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:14.596 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:14.596 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:14.596 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:14.596 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:14.596 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:14.596 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:14.596 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:14.597 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:14.598 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:14.598 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:14.598 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:14.598 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:14.598 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:14.598 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:14.598 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:14.598 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:14.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:14.599 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:14.647 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:52:14.648 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:52:14.648 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:52:14.648 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:52:14.648 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:52:14.648 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:52:14.649 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:14.650 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:52:14.650 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:52:14.650 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:52:14.650 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:52:14.650 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:52:14.650 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:52:14.650 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:52:14.650 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:52:14.650 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:14.650 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:52:14.650 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:14.769 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.51572.asok 2024-09-17T02:52:14.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:14.781 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51572.asok 2024-09-17T02:52:14.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:14.781 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51572.asok status 2024-09-17T02:52:14.956 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:14.956 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:52:14.956 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:52:14.956 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:52:14.956 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:52:14.956 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:52:14.956 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:52:14.956 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "51572", 2024-09-17T02:52:14.956 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:52:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:52:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:52:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8685, 2024-09-17T02:52:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:52:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:52:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:52:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 8685 2024-09-17T02:52:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:52:14.957 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1255054747 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 1255054747 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.8685 192.168.0.1:0/1255054747", 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/1255054747", 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 101, 2024-09-17T02:52:14.958 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 122, 2024-09-17T02:52:14.959 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:52:14.959 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:52:14.959 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:52:14.959 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:14.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:14.973 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:52:14.974 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:14.980 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:52:14.980 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:14.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:14.980 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:15.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:15.034 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-09-17T02:52:15.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:15.092 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-09-17T02:52:15.211 INFO:teuthology.orchestra.run.smithi131.stderr:8+0 records in 2024-09-17T02:52:15.211 INFO:teuthology.orchestra.run.smithi131.stderr:8+0 records out 2024-09-17T02:52:15.211 INFO:teuthology.orchestra.run.smithi131.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0689477 s, 122 MB/s 2024-09-17T02:52:15.213 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:15.213 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:15.213 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:15.213 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:15.213 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-09-17T02:52:15.213 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:15.282 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627779 2024-09-17T02:52:15.285 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:15.292 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:15.292 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:15.292 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:15.292 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:15.292 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-09-17T02:52:15.292 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:15.396 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627780 2024-09-17T02:52:15.399 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:15.405 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:15.405 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:15.457 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:52:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.670+0000 7f7d66283640 1 -- 172.21.15.131:0/1375478751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d600b6930 msgr2=0x7f7d601672a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.670+0000 7f7d66283640 1 --2- 172.21.15.131:0/1375478751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d600b6930 0x7f7d601672a0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009bc0 tx=0x7f7d5801cb70 comp rx=0 tx=0).stop 2024-09-17T02:52:15.663 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.670+0000 7f7d66283640 1 -- 172.21.15.131:0/1375478751 shutdown_connections 2024-09-17T02:52:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.670+0000 7f7d66283640 1 --2- 172.21.15.131:0/1375478751 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d600b6930 0x7f7d601672a0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.670+0000 7f7d66283640 1 -- 172.21.15.131:0/1375478751 >> 172.21.15.131:0/1375478751 conn(0x7f7d600cf0b0 msgr2=0x7f7d600cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.670+0000 7f7d66283640 1 -- 172.21.15.131:0/1375478751 shutdown_connections 2024-09-17T02:52:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.670+0000 7f7d66283640 1 -- 172.21.15.131:0/1375478751 wait complete. 2024-09-17T02:52:15.664 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.670+0000 7f7d66283640 1 Processor -- start 2024-09-17T02:52:15.665 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.670+0000 7f7d66283640 1 -- start start 2024-09-17T02:52:15.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d66283640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d600b6930 0x7f7d6014e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:15.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d66283640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d600cca90 0x7f7d600ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:15.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d66283640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d6015c740 0x7f7d600ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d66283640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6016afd0 con 0x7f7d600cca90 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d66283640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6016ae50 con 0x7f7d6015c740 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d66283640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6016b150 con 0x7f7d600b6930 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d600cca90 0x7f7d600ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d600cca90 0x7f7d600ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51872/0 (socket says 172.21.15.131:51872) 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5f7fe640 1 -- 172.21.15.131:0/2669971526 learned_addr learned my addr 172.21.15.131:0/2669971526 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d64820640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d6015c740 0x7f7d600ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5ffff640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d600b6930 0x7f7d6014e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5f7fe640 1 -- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d600b6930 msgr2=0x7f7d6014e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5f7fe640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d600b6930 0x7f7d6014e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5f7fe640 1 -- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d6015c740 msgr2=0x7f7d600ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:15.668 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5f7fe640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d6015c740 0x7f7d600ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5f7fe640 1 -- 172.21.15.131:0/2669971526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d60150bb0 con 0x7f7d600cca90 2024-09-17T02:52:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5ffff640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d600b6930 0x7f7d6014e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5f7fe640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d600cca90 0x7f7d600ca1f0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400b410 tx=0x7f7d5400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5cff9640 1 -- 172.21.15.131:0/2669971526 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54015030 con 0x7f7d600cca90 2024-09-17T02:52:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5cff9640 1 -- 172.21.15.131:0/2669971526 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d54004020 con 0x7f7d600cca90 2024-09-17T02:52:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d66283640 1 -- 172.21.15.131:0/2669971526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d60150e90 con 0x7f7d600cca90 2024-09-17T02:52:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5cff9640 1 -- 172.21.15.131:0/2669971526 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54004470 con 0x7f7d600cca90 2024-09-17T02:52:15.669 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d66283640 1 -- 172.21.15.131:0/2669971526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d60157db0 con 0x7f7d600cca90 2024-09-17T02:52:15.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d66283640 1 -- 172.21.15.131:0/2669971526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d600d0130 con 0x7f7d600cca90 2024-09-17T02:52:15.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5cff9640 1 -- 172.21.15.131:0/2669971526 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7d54018070 con 0x7f7d600cca90 2024-09-17T02:52:15.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5cff9640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d4c073230 0x7f7d4c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:15.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.674+0000 7f7d5cff9640 1 -- 172.21.15.131:0/2669971526 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7f7d540941f0 con 0x7f7d600cca90 2024-09-17T02:52:15.670 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.678+0000 7f7d5ffff640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d4c073230 0x7f7d4c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:15.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.678+0000 7f7d5ffff640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d4c073230 0x7f7d4c075740 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7d50000f70 tx=0x7f7d50009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:15.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.678+0000 7f7d5cff9640 1 -- 172.21.15.131:0/2669971526 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7d5405f1f0 con 0x7f7d600cca90 2024-09-17T02:52:15.854 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 -- 172.21.15.131:0/2669971526 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7d6006a170 con 0x7f7d600cca90 2024-09-17T02:52:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d5cff9640 1 -- 172.21.15.131:0/2669971526 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v122) ==== 80+0+0 (secure 0 0 0) 0x7f7d5405eaf0 con 0x7f7d600cca90 2024-09-17T02:52:15.855 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:52:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 -- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d4c073230 msgr2=0x7f7d4c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d4c073230 0x7f7d4c075740 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7d50000f70 tx=0x7f7d50009040 comp rx=0 tx=0).stop 2024-09-17T02:52:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 -- 172.21.15.131:0/2669971526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d600cca90 msgr2=0x7f7d600ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d600cca90 0x7f7d600ca1f0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400b410 tx=0x7f7d5400b8d0 comp rx=0 tx=0).stop 2024-09-17T02:52:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 -- 172.21.15.131:0/2669971526 shutdown_connections 2024-09-17T02:52:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7d4c073230 0x7f7d4c075740 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7d6015c740 0x7f7d600ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7d600cca90 0x7f7d600ca1f0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 --2- 172.21.15.131:0/2669971526 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7d600b6930 0x7f7d6014e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 -- 172.21.15.131:0/2669971526 >> 172.21.15.131:0/2669971526 conn(0x7f7d600cf0b0 msgr2=0x7f7d600b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:15.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.862+0000 7f7d66283640 1 -- 172.21.15.131:0/2669971526 shutdown_connections 2024-09-17T02:52:15.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:15.866+0000 7f7d66283640 1 -- 172.21.15.131:0/2669971526 wait complete. 2024-09-17T02:52:15.876 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:15.876 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:15.882 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:52:15.882 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:52:15.928 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:52:16.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.094+0000 7fa5f6bca640 1 -- 172.21.15.131:0/199139392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e00039f0 msgr2=0x7fa5e0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.094+0000 7fa5f6bca640 1 --2- 172.21.15.131:0/199139392 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e00039f0 0x7fa5e0003e20 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fa5e4006f90 tx=0x7fa5e401cf00 comp rx=0 tx=0).stop 2024-09-17T02:52:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.094+0000 7fa5f5167640 1 -- 172.21.15.131:0/199139392 reap_dead start 2024-09-17T02:52:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.094+0000 7fa5f6bca640 1 -- 172.21.15.131:0/199139392 shutdown_connections 2024-09-17T02:52:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.094+0000 7fa5f6bca640 1 -- 172.21.15.131:0/199139392 >> 172.21.15.131:0/199139392 conn(0x7fa5f006eea0 msgr2=0x7fa5f006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.094+0000 7fa5f6bca640 1 -- 172.21.15.131:0/199139392 shutdown_connections 2024-09-17T02:52:16.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.094+0000 7fa5f6bca640 1 -- 172.21.15.131:0/199139392 wait complete. 2024-09-17T02:52:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f6bca640 1 Processor -- start 2024-09-17T02:52:16.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f6bca640 1 -- start start 2024-09-17T02:52:16.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f6bca640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e00039f0 0x7fa5f015b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:16.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f6bca640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f01f7260 0x7fa5f0154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:16.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f6bca640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa5f00b5100 0x7fa5f00b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:16.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f6bca640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f015c230 con 0x7fa5e00039f0 2024-09-17T02:52:16.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f6bca640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa5f015c0b0 con 0x7fa5f01f7260 2024-09-17T02:52:16.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f6bca640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa5f006e7f0 con 0x7fa5f00b5100 2024-09-17T02:52:16.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f4966640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e00039f0 0x7fa5f015b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:16.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f4966640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e00039f0 0x7fa5f015b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51880/0 (socket says 172.21.15.131:51880) 2024-09-17T02:52:16.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f4966640 1 -- 172.21.15.131:0/1499539944 learned_addr learned my addr 172.21.15.131:0/1499539944 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:16.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5effff640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f01f7260 0x7fa5f0154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:16.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f5167640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa5f00b5100 0x7fa5f00b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:16.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f4966640 1 -- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa5f00b5100 msgr2=0x7fa5f00b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:16.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f4966640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa5f00b5100 0x7fa5f00b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f4966640 1 -- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f01f7260 msgr2=0x7fa5f0154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f4966640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f01f7260 0x7fa5f0154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f4966640 1 -- 172.21.15.131:0/1499539944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5f00b5a50 con 0x7fa5e00039f0 2024-09-17T02:52:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f4966640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e00039f0 0x7fa5f015b3a0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa5e401c9c0 tx=0x7fa5e4001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5ed7fa640 1 -- 172.21.15.131:0/1499539944 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40a5b50 con 0x7fa5e00039f0 2024-09-17T02:52:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5ed7fa640 1 -- 172.21.15.131:0/1499539944 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5e40a5de0 con 0x7fa5e00039f0 2024-09-17T02:52:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f6bca640 1 -- 172.21.15.131:0/1499539944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5f01f9c00 con 0x7fa5e00039f0 2024-09-17T02:52:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5ed7fa640 1 -- 172.21.15.131:0/1499539944 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e40062d0 con 0x7fa5e00039f0 2024-09-17T02:52:16.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5f6bca640 1 -- 172.21.15.131:0/1499539944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5f01fa1c0 con 0x7fa5e00039f0 2024-09-17T02:52:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.098+0000 7fa5daffd640 1 -- 172.21.15.131:0/1499539944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c80052a0 con 0x7fa5e00039f0 2024-09-17T02:52:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.102+0000 7fa5ed7fa640 1 -- 172.21.15.131:0/1499539944 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa5e4003600 con 0x7fa5e00039f0 2024-09-17T02:52:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.102+0000 7fa5ed7fa640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa5d4073230 0x7fa5d4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.102+0000 7fa5ed7fa640 1 -- 172.21.15.131:0/1499539944 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fa5e412e550 con 0x7fa5e00039f0 2024-09-17T02:52:16.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.102+0000 7fa5effff640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa5d4073230 0x7fa5d4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:16.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.102+0000 7fa5effff640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa5d4073230 0x7fa5d4075740 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa5f01f9680 tx=0x7fa5dc0091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:16.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.106+0000 7fa5ed7fa640 1 -- 172.21.15.131:0/1499539944 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa5f00b5a50 con 0x7fa5e00039f0 2024-09-17T02:52:16.280 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.286+0000 7fa5daffd640 1 -- 172.21.15.131:0/1499539944 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa5c8005590 con 0x7fa5e00039f0 2024-09-17T02:52:16.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.286+0000 7fa5ed7fa640 1 -- 172.21.15.131:0/1499539944 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v122) ==== 80+0+0 (secure 0 0 0) 0x7fa5e4016cb0 con 0x7fa5e00039f0 2024-09-17T02:52:16.281 INFO:teuthology.orchestra.run.smithi131.stderr:listed 0 entries 2024-09-17T02:52:16.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 -- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa5d4073230 msgr2=0x7fa5d4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:16.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa5d4073230 0x7fa5d4075740 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa5f01f9680 tx=0x7fa5dc0091e0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 -- 172.21.15.131:0/1499539944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e00039f0 msgr2=0x7fa5f015b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:16.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e00039f0 0x7fa5f015b3a0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fa5e401c9c0 tx=0x7fa5e4001cb0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 -- 172.21.15.131:0/1499539944 shutdown_connections 2024-09-17T02:52:16.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa5d4073230 0x7fa5d4075740 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa5f00b5100 0x7fa5f00b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa5f01f7260 0x7fa5f0154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 --2- 172.21.15.131:0/1499539944 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa5e00039f0 0x7fa5f015b3a0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 -- 172.21.15.131:0/1499539944 >> 172.21.15.131:0/1499539944 conn(0x7fa5f006eea0 msgr2=0x7fa5f006dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:16.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 -- 172.21.15.131:0/1499539944 shutdown_connections 2024-09-17T02:52:16.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.290+0000 7fa5daffd640 1 -- 172.21.15.131:0/1499539944 wait complete. 2024-09-17T02:52:16.302 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:52:16.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:16.302 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:16.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[51572]: fuse finished with error 0 and tester_r 0 2024-09-17T02:52:16.315 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:52:16.385 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:52:16.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:16.385 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:52:16.391 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:16.391 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:16.439 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:52:16.440 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi131.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:16.440 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:52:16.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:16.440 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:52:16.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:52:16.487 INFO:teuthology.orchestra.run.smithi131.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-17T02:52:16.487 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:52:16.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.694+0000 7fa045ed3640 1 -- 172.21.15.131:0/1718538399 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa04006dea0 msgr2=0x7fa04006e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:16.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.694+0000 7fa045ed3640 1 --2- 172.21.15.131:0/1718538399 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa04006dea0 0x7fa04006e2b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa028009840 tx=0x7fa02801c7f0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.694+0000 7fa045ed3640 1 -- 172.21.15.131:0/1718538399 shutdown_connections 2024-09-17T02:52:16.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.694+0000 7fa045ed3640 1 --2- 172.21.15.131:0/1718538399 >> v2:172.21.15.189:3300/0 conn(0x7fa0400cab60 0x7fa0400caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.694+0000 7fa045ed3640 1 --2- 172.21.15.131:0/1718538399 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa04006dea0 0x7fa04006e2b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.694+0000 7fa045ed3640 1 -- 172.21.15.131:0/1718538399 >> 172.21.15.131:0/1718538399 conn(0x7fa04006eea0 msgr2=0x7fa04006f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:16.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.694+0000 7fa045ed3640 1 -- 172.21.15.131:0/1718538399 shutdown_connections 2024-09-17T02:52:16.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.694+0000 7fa045ed3640 1 -- 172.21.15.131:0/1718538399 wait complete. 2024-09-17T02:52:16.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa045ed3640 1 Processor -- start 2024-09-17T02:52:16.690 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa045ed3640 1 -- start start 2024-09-17T02:52:16.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa045ed3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa04006dea0 0x7fa0400ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:16.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa045ed3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0400cab60 0x7fa040151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:16.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa045ed3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa040155240 0x7fa0401f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:16.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa045ed3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fa0400cbdd0 con 0x7fa0400cab60 2024-09-17T02:52:16.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa045ed3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fa0400cbc50 con 0x7fa040155240 2024-09-17T02:52:16.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa045ed3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fa040167b70 con 0x7fa04006dea0 2024-09-17T02:52:16.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa037fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0400cab60 0x7fa040151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:16.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa037fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0400cab60 0x7fa040151bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51886/0 (socket says 172.21.15.131:51886) 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa037fff640 1 -- 172.21.15.131:0/220024869 learned_addr learned my addr 172.21.15.131:0/220024869 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa03f7fe640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa04006dea0 0x7fa0400ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa03ffff640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa040155240 0x7fa0401f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa037fff640 1 -- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa04006dea0 msgr2=0x7fa0400ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa037fff640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa04006dea0 0x7fa0400ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa037fff640 1 -- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa040155240 msgr2=0x7fa0401f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa037fff640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa040155240 0x7fa0401f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa037fff640 1 -- 172.21.15.131:0/220024869 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0400ae9a0 con 0x7fa0400cab60 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa037fff640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0400cab60 0x7fa040151bf0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fa028009790 tx=0x7fa0280a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa03cff9640 1 -- 172.21.15.131:0/220024869 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0280bb070 con 0x7fa0400cab60 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa03cff9640 1 -- 172.21.15.131:0/220024869 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa028004470 con 0x7fa0400cab60 2024-09-17T02:52:16.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa045ed3640 1 -- 172.21.15.131:0/220024869 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0400c96f0 con 0x7fa0400cab60 2024-09-17T02:52:16.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa03cff9640 1 -- 172.21.15.131:0/220024869 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0280b2040 con 0x7fa0400cab60 2024-09-17T02:52:16.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.698+0000 7fa045ed3640 1 -- 172.21.15.131:0/220024869 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0400c9cb0 con 0x7fa0400cab60 2024-09-17T02:52:16.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.702+0000 7fa045ed3640 1 -- 172.21.15.131:0/220024869 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa04006e2b0 con 0x7fa0400cab60 2024-09-17T02:52:16.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.702+0000 7fa03cff9640 1 -- 172.21.15.131:0/220024869 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fa028005290 con 0x7fa0400cab60 2024-09-17T02:52:16.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.702+0000 7fa03cff9640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa020073230 0x7fa020075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:16.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.702+0000 7fa03cff9640 1 -- 172.21.15.131:0/220024869 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 6101+0+0 (secure 0 0 0) 0x7fa02812de00 con 0x7fa0400cab60 2024-09-17T02:52:16.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.702+0000 7fa03f7fe640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa020073230 0x7fa020075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:16.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.702+0000 7fa03f7fe640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa020073230 0x7fa020075740 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa0300015f0 tx=0x7fa030009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:16.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.706+0000 7fa03cff9640 1 -- 172.21.15.131:0/220024869 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fa028016410 con 0x7fa0400cab60 2024-09-17T02:52:16.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.914+0000 7fa045ed3640 1 -- 172.21.15.131:0/220024869 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa0400c9480 con 0x7fa0400cab60 2024-09-17T02:52:16.909 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:52:16.914+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:52:16.911 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.918+0000 7fa03cff9640 1 -- 172.21.15.131:0/220024869 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) ==== 161+0+0 (secure 0 0 0) 0x7fa0280f8c00 con 0x7fa0400cab60 2024-09-17T02:52:16.911 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:52:16.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 -- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa020073230 msgr2=0x7fa020075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:16.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa020073230 0x7fa020075740 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa0300015f0 tx=0x7fa030009040 comp rx=0 tx=0).stop 2024-09-17T02:52:16.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 -- 172.21.15.131:0/220024869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0400cab60 msgr2=0x7fa040151bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:16.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0400cab60 0x7fa040151bf0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fa028009790 tx=0x7fa0280a5690 comp rx=0 tx=0).stop 2024-09-17T02:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 -- 172.21.15.131:0/220024869 shutdown_connections 2024-09-17T02:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fa020073230 0x7fa020075740 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fa040155240 0x7fa0401f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fa0400cab60 0x7fa040151bf0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 --2- 172.21.15.131:0/220024869 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fa04006dea0 0x7fa0400ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:16.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 -- 172.21.15.131:0/220024869 >> 172.21.15.131:0/220024869 conn(0x7fa04006eea0 msgr2=0x7fa0400b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:16.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 -- 172.21.15.131:0/220024869 shutdown_connections 2024-09-17T02:52:16.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:16.922+0000 7fa045ed3640 1 -- 172.21.15.131:0/220024869 wait complete. 2024-09-17T02:52:16.928 INFO:tasks.ceph.mds.a.smithi131.stderr: -8794> 2024-09-17T02:52:00.582+0000 7fd29f74d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:52:16.938 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T02:52:16.940 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: chdir = "" 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: auth supported = cephx 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: filestore xattr use omap = true 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: mon clock drift allowed = 1.000 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: osd crush chooseleaf type = 0 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: auth debug = true 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.964 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on old message = true 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on bug = true 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: mon pg warn max object skew = 0 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default size = 2 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow primary affinity = true 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow pg remap = true 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on legacy crush tunables = false 2024-09-17T02:52:16.965 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on crush straw calc version zero = false 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on no sortbitwise = false 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on osd down out interval zero = false 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on too few osds = false 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: mon_allow_pool_size_one = true 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: osd default data pool replay window = 5 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: mon allow pool delete = true 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.966 INFO:teuthology.orchestra.run.smithi131.stdout: mon cluster log file level = debug 2024-09-17T02:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: debug asserts on shutdown = true 2024-09-17T02:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: mon health detail to clog = false 2024-09-17T02:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: mon host = "172.21.15.131,172.21.15.189,[v2:172.21.15.189:3301,v1:172.21.15.189:6790]" 2024-09-17T02:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout:[osd] 2024-09-17T02:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: osd journal size = 100 2024-09-17T02:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub load threshold = 5.0 2024-09-17T02:52:16.967 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub max interval = 600 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: osd mclock profile = high_recovery_ops 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: osd recover clone overlap = true 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: osd recovery max chunk = 1048576 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug shutdown = true 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug op order = true 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify stray on activate = true 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug trim objects = true 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: osd open classes on start = true 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug pg log writeout = true 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: osd deep scrub update digest min age = 30 2024-09-17T02:52:16.968 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: osd map max advance = 10 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: journal zero on create = true 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: filestore ondisk finisher threads = 3 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: filestore apply finisher threads = 3 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: bdev debug aio = true 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug misdirected ops = true 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore block size = 96636764160 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore fsck on mount = True 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluefs = 20 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluestore = 20 2024-09-17T02:52:16.969 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: debug osd = 20 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: debug rocksdb = 10 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd full ratio = 0.9 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd nearfull ratio = 0.8 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: osd failsafe full ratio = 0.95 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: osd objectstore = bluestore 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr] 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: debug mgr = 20 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-17T02:52:16.970 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: mgr/telemetry/nag = false 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout:[mon] 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: debug paxos = 20 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: mon data avail warn = 5 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: mon mgr mkfs grace = 240 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd reporter subtree level = osd 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd prime pg temp = true 2024-09-17T02:52:16.971 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: # 1m isn't quite enough 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: mon_down_mkfs_grace = 2m 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_filestore_osds = false 2024-09-17T02:52:16.972 INFO:teuthology.orchestra.run.smithi131.stdout: mon down mkfs grace = 300 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout:[client] 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout: rgw cache enabled = true 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable ops log = true 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable usage log = true 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = "" 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.a] 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.b] 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.c] 2024-09-17T02:52:16.973 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2024-09-17T02:52:16.974 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2024-09-17T02:52:16.974 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2024-09-17T02:52:16.974 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2024-09-17T02:52:16.974 INFO:teuthology.orchestra.run.smithi131.stdout:[mds] 2024-09-17T02:52:16.974 INFO:teuthology.orchestra.run.smithi131.stdout: mds verify scatter = False 2024-09-17T02:52:16.975 INFO:teuthology.orchestra.run.smithi189.stdout:[global] 2024-09-17T02:52:16.975 INFO:teuthology.orchestra.run.smithi189.stdout: chdir = "" 2024-09-17T02:52:16.975 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T02:52:16.975 INFO:teuthology.orchestra.run.smithi189.stdout: auth supported = cephx 2024-09-17T02:52:16.975 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.975 INFO:teuthology.orchestra.run.smithi189.stdout: filestore xattr use omap = true 2024-09-17T02:52:16.975 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.975 INFO:teuthology.orchestra.run.smithi189.stdout: mon clock drift allowed = 1.000 2024-09-17T02:52:16.975 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.975 INFO:teuthology.orchestra.run.smithi189.stdout: osd crush chooseleaf type = 0 2024-09-17T02:52:16.975 INFO:teuthology.orchestra.run.smithi189.stdout: auth debug = true 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on old message = true 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on bug = true 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: mon pg warn max object skew = 0 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default size = 2 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow primary affinity = true 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow pg remap = true 2024-09-17T02:52:16.976 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on legacy crush tunables = false 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on crush straw calc version zero = false 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on no sortbitwise = false 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on osd down out interval zero = false 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on too few osds = false 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: mon_allow_pool_size_one = true 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: osd default data pool replay window = 5 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: mon allow pool delete = true 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.977 INFO:teuthology.orchestra.run.smithi189.stdout: mon cluster log file level = debug 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: debug asserts on shutdown = true 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: mon health detail to clog = false 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: mon host = "172.21.15.131,172.21.15.189,[v2:172.21.15.189:3301,v1:172.21.15.189:6790]" 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: fsid = f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout:[osd] 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: osd journal size = 100 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub load threshold = 5.0 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub max interval = 600 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: osd mclock profile = high_recovery_ops 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: osd recover clone overlap = true 2024-09-17T02:52:16.978 INFO:teuthology.orchestra.run.smithi189.stdout: osd recovery max chunk = 1048576 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug shutdown = true 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug op order = true 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug verify stray on activate = true 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug trim objects = true 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: osd open classes on start = true 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug pg log writeout = true 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: osd deep scrub update digest min age = 30 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: osd map max advance = 10 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: journal zero on create = true 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.979 INFO:teuthology.orchestra.run.smithi189.stdout: filestore ondisk finisher threads = 3 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: filestore apply finisher threads = 3 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: bdev debug aio = true 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug misdirected ops = true 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore block size = 96636764160 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore fsck on mount = True 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluefs = 20 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluestore = 20 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: debug osd = 20 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: debug rocksdb = 10 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd full ratio = 0.9 2024-09-17T02:52:16.980 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd nearfull ratio = 0.8 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: osd failsafe full ratio = 0.95 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: osd objectstore = bluestore 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr] 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: debug mgr = 20 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: mgr/telemetry/nag = false 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout:[mon] 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:16.981 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: debug paxos = 20 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: mon data avail warn = 5 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: mon mgr mkfs grace = 240 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd reporter subtree level = osd 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd prime pg temp = true 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T02:52:16.982 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: # 1m isn't quite enough 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: mon_down_mkfs_grace = 2m 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_filestore_osds = false 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: mon down mkfs grace = 300 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout:[client] 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: rgw cache enabled = true 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable ops log = true 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable usage log = true 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = "" 2024-09-17T02:52:16.983 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.a] 2024-09-17T02:52:16.984 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.b] 2024-09-17T02:52:16.984 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.c] 2024-09-17T02:52:16.984 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.a] 2024-09-17T02:52:16.984 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.c] 2024-09-17T02:52:16.984 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.b] 2024-09-17T02:52:16.984 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.d] 2024-09-17T02:52:16.984 INFO:teuthology.orchestra.run.smithi189.stdout:[mds] 2024-09-17T02:52:16.984 INFO:teuthology.orchestra.run.smithi189.stdout: mds verify scatter = False 2024-09-17T02:52:16.986 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T02:52:17.021 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:52:17.023 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T02:52:17.042 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-09-17T02:52:17.042 INFO:teuthology.orchestra.run.smithi131.stdout: chdir = "" 2024-09-17T02:52:17.042 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T02:52:17.042 INFO:teuthology.orchestra.run.smithi131.stdout: auth supported = cephx 2024-09-17T02:52:17.042 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.042 INFO:teuthology.orchestra.run.smithi131.stdout: filestore xattr use omap = true 2024-09-17T02:52:17.042 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: mon clock drift allowed = 1.000 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: osd crush chooseleaf type = 0 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: auth debug = true 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on old message = true 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on bug = true 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: mon pg warn max object skew = 0 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default size = 2 2024-09-17T02:52:17.043 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow primary affinity = true 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow pg remap = true 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on legacy crush tunables = false 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on crush straw calc version zero = false 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on no sortbitwise = false 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on osd down out interval zero = false 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on too few osds = false 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: mon_allow_pool_size_one = true 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: osd default data pool replay window = 5 2024-09-17T02:52:17.044 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: mon allow pool delete = true 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: mon cluster log file level = debug 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: debug asserts on shutdown = true 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: mon health detail to clog = false 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: mon host = "172.21.15.131,172.21.15.189,[v2:172.21.15.189:3301,v1:172.21.15.189:6790]" 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout:[osd] 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: osd journal size = 100 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub load threshold = 5.0 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub max interval = 600 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: osd mclock profile = high_recovery_ops 2024-09-17T02:52:17.045 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: osd recover clone overlap = true 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: osd recovery max chunk = 1048576 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug shutdown = true 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug op order = true 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify stray on activate = true 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug trim objects = true 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: osd open classes on start = true 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug pg log writeout = true 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: osd deep scrub update digest min age = 30 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.046 INFO:teuthology.orchestra.run.smithi131.stdout: osd map max advance = 10 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: journal zero on create = true 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: filestore ondisk finisher threads = 3 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: filestore apply finisher threads = 3 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: bdev debug aio = true 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug misdirected ops = true 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore block size = 96636764160 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore fsck on mount = True 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluefs = 20 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluestore = 20 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: debug osd = 20 2024-09-17T02:52:17.047 INFO:teuthology.orchestra.run.smithi131.stdout: debug rocksdb = 10 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd full ratio = 0.9 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd nearfull ratio = 0.8 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: osd failsafe full ratio = 0.95 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: osd objectstore = bluestore 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr] 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: debug mgr = 20 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: mgr/telemetry/nag = false 2024-09-17T02:52:17.048 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout:[mon] 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: debug paxos = 20 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: mon data avail warn = 5 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: mon mgr mkfs grace = 240 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd reporter subtree level = osd 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd prime pg temp = true 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T02:52:17.049 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: # 1m isn't quite enough 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: mon_down_mkfs_grace = 2m 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_filestore_osds = false 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: mon down mkfs grace = 300 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout:[client] 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: rgw cache enabled = true 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable ops log = true 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable usage log = true 2024-09-17T02:52:17.050 INFO:teuthology.orchestra.run.smithi131.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = "" 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.a] 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.b] 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.c] 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout:[mds] 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout: mds verify scatter = False 2024-09-17T02:52:17.051 INFO:teuthology.orchestra.run.smithi131.stdout: mds debug scatterstat = False 2024-09-17T02:52:17.058 INFO:teuthology.orchestra.run.smithi189.stdout:[global] 2024-09-17T02:52:17.058 INFO:teuthology.orchestra.run.smithi189.stdout: chdir = "" 2024-09-17T02:52:17.058 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T02:52:17.058 INFO:teuthology.orchestra.run.smithi189.stdout: auth supported = cephx 2024-09-17T02:52:17.058 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: filestore xattr use omap = true 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: mon clock drift allowed = 1.000 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: osd crush chooseleaf type = 0 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: auth debug = true 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on old message = true 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on bug = true 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: mon pg warn max object skew = 0 2024-09-17T02:52:17.059 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default size = 2 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow primary affinity = true 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow pg remap = true 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on legacy crush tunables = false 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on crush straw calc version zero = false 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on no sortbitwise = false 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on osd down out interval zero = false 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on too few osds = false 2024-09-17T02:52:17.060 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: mon_allow_pool_size_one = true 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: osd default data pool replay window = 5 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: mon allow pool delete = true 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: mon cluster log file level = debug 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: debug asserts on shutdown = true 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: mon health detail to clog = false 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: mon host = "172.21.15.131,172.21.15.189,[v2:172.21.15.189:3301,v1:172.21.15.189:6790]" 2024-09-17T02:52:17.061 INFO:teuthology.orchestra.run.smithi189.stdout: fsid = f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout:[osd] 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: osd journal size = 100 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub load threshold = 5.0 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub max interval = 600 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: osd mclock profile = high_recovery_ops 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: osd recover clone overlap = true 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: osd recovery max chunk = 1048576 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug shutdown = true 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug op order = true 2024-09-17T02:52:17.062 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug verify stray on activate = true 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug trim objects = true 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: osd open classes on start = true 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug pg log writeout = true 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: osd deep scrub update digest min age = 30 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: osd map max advance = 10 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: journal zero on create = true 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: filestore ondisk finisher threads = 3 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: filestore apply finisher threads = 3 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.063 INFO:teuthology.orchestra.run.smithi189.stdout: bdev debug aio = true 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug misdirected ops = true 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore block size = 96636764160 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore fsck on mount = True 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluefs = 20 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluestore = 20 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: debug osd = 20 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: debug rocksdb = 10 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd full ratio = 0.9 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd nearfull ratio = 0.8 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: osd failsafe full ratio = 0.95 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: osd objectstore = bluestore 2024-09-17T02:52:17.064 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr] 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout: debug mgr = 20 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout: mgr/telemetry/nag = false 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout:[mon] 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-17T02:52:17.065 INFO:teuthology.orchestra.run.smithi189.stdout: debug paxos = 20 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: mon data avail warn = 5 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: mon mgr mkfs grace = 240 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd reporter subtree level = osd 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd prime pg temp = true 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T02:52:17.066 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: # 1m isn't quite enough 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: mon_down_mkfs_grace = 2m 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_filestore_osds = false 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: mon down mkfs grace = 300 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout:[client] 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: rgw cache enabled = true 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable ops log = true 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable usage log = true 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T02:52:17.067 INFO:teuthology.orchestra.run.smithi189.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T02:52:17.068 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = "" 2024-09-17T02:52:17.068 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.a] 2024-09-17T02:52:17.068 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.b] 2024-09-17T02:52:17.068 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.c] 2024-09-17T02:52:17.068 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.a] 2024-09-17T02:52:17.068 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.c] 2024-09-17T02:52:17.068 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.b] 2024-09-17T02:52:17.068 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.d] 2024-09-17T02:52:17.068 INFO:teuthology.orchestra.run.smithi189.stdout:[mds] 2024-09-17T02:52:17.068 INFO:teuthology.orchestra.run.smithi189.stdout: mds verify scatter = False 2024-09-17T02:52:17.068 INFO:teuthology.orchestra.run.smithi189.stdout: mds debug scatterstat = False 2024-09-17T02:52:17.069 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-09-17T02:52:17.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a29ff0080 1 Processor -- start 2024-09-17T02:52:17.156 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a29ff0080 1 -- start start 2024-09-17T02:52:17.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a29ff0080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55795db27110 0x55795dc02480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a29ff0080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55795dbf9330 0x55795db503b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a29ff0080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 0x55795dbf4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a29ff0080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55795da73a00 con 0x55795dbfbb40 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a29ff0080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55795dad1f40 con 0x55795db27110 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a29ff0080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55795db54d60 con 0x55795dbf9330 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a242c8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 0x55795dbf4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a242c8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 0x55795dbf4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51912/0 (socket says 172.21.15.131:51912) 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a242c8640 1 -- 172.21.15.131:0/1200859597 learned_addr learned my addr 172.21.15.131:0/1200859597 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a232c6640 1 --2- 172.21.15.131:0/1200859597 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55795dbf9330 0x55795db503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.162+0000 7f4a23ac7640 1 --2- 172.21.15.131:0/1200859597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55795db27110 0x55795dc02480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a242c8640 1 -- 172.21.15.131:0/1200859597 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55795dbf9330 msgr2=0x55795db503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a242c8640 1 --2- 172.21.15.131:0/1200859597 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55795dbf9330 0x55795db503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a242c8640 1 -- 172.21.15.131:0/1200859597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55795db27110 msgr2=0x55795dc02480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a242c8640 1 --2- 172.21.15.131:0/1200859597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55795db27110 0x55795dc02480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a242c8640 1 -- 172.21.15.131:0/1200859597 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55795da4cc10 con 0x55795dbfbb40 2024-09-17T02:52:17.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a242c8640 1 --2- 172.21.15.131:0/1200859597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 0x55795dbf4740 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400a3d0 tx=0x7f4a1400a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bba94e2de58aff server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a222c4640 1 -- 172.21.15.131:0/1200859597 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14014620 con 0x55795dbfbb40 2024-09-17T02:52:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a222c4640 1 -- 172.21.15.131:0/1200859597 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a1400f030 con 0x55795dbfbb40 2024-09-17T02:52:17.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a222c4640 1 -- 172.21.15.131:0/1200859597 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a14014b30 con 0x55795dbfbb40 2024-09-17T02:52:17.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1200859597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 msgr2=0x55795dbf4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:17.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 --2- 172.21.15.131:0/1200859597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 0x55795dbf4740 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400a3d0 tx=0x7f4a1400a4c0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1200859597 shutdown_connections 2024-09-17T02:52:17.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 --2- 172.21.15.131:0/1200859597 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 0x55795dbf4740 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 --2- 172.21.15.131:0/1200859597 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55795dbf9330 0x55795db503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 --2- 172.21.15.131:0/1200859597 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55795db27110 0x55795dc02480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1200859597 >> 172.21.15.131:0/1200859597 conn(0x55795db11d40 msgr2=0x55795db51500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1200859597 shutdown_connections 2024-09-17T02:52:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1200859597 wait complete. 2024-09-17T02:52:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 Processor -- start 2024-09-17T02:52:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 -- start start 2024-09-17T02:52:17.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55795db27110 0x55795dbf2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55795dbf9330 0x55795dbf5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 0x55795dbf63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55795dc01aa0 con 0x55795dbfbb40 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55795dc01800 con 0x55795dbf9330 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a29ff0080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55795dc01d40 con 0x55795db27110 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a242c8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 0x55795dbf63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a242c8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 0x55795dbf63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51928/0 (socket says 172.21.15.131:51928) 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a242c8640 1 -- 172.21.15.131:0/1561679376 learned_addr learned my addr 172.21.15.131:0/1561679376 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a23ac7640 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55795db27110 0x55795dbf2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a232c6640 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55795dbf9330 0x55795dbf5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.166+0000 7f4a242c8640 1 -- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55795db27110 msgr2=0x55795dbf2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a242c8640 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55795db27110 0x55795dbf2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a242c8640 1 -- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55795dbf9330 msgr2=0x55795dbf5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a242c8640 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55795dbf9330 0x55795dbf5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a242c8640 1 -- 172.21.15.131:0/1561679376 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55795dbfe120 con 0x55795dbfbb40 2024-09-17T02:52:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a232c6640 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55795dbf9330 0x55795dbf5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a242c8640 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 0x55795dbf63f0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400a9c0 tx=0x7f4a14014650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a212c2640 1 -- 172.21.15.131:0/1561679376 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1401c420 con 0x55795dbfbb40 2024-09-17T02:52:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a212c2640 1 -- 172.21.15.131:0/1561679376 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a14021070 con 0x55795dbfbb40 2024-09-17T02:52:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1561679376 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55795db473c0 con 0x55795dbfbb40 2024-09-17T02:52:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a212c2640 1 -- 172.21.15.131:0/1561679376 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a1401c970 con 0x55795dbfbb40 2024-09-17T02:52:17.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1561679376 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55795db50e50 con 0x55795dbfbb40 2024-09-17T02:52:17.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a212c2640 1 -- 172.21.15.131:0/1561679376 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4a1401cb10 con 0x55795dbfbb40 2024-09-17T02:52:17.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a212c2640 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4a10071c10 0x7f4a10074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a212c2640 1 -- 172.21.15.131:0/1561679376 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f4a140944d0 con 0x55795dbfbb40 2024-09-17T02:52:17.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a29ff0080 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55795dbf3760 0x55795db47d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1561679376 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) -- 0x55795dbf3300 con 0x55795dbf3760 2024-09-17T02:52:17.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a23ac7640 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4a10071c10 0x7f4a10074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a232c6640 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55795dbf3760 0x55795db47d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.170+0000 7f4a232c6640 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55795dbf3760 0x55795db47d70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:17.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.174+0000 7f4a23ac7640 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4a10071c10 0x7f4a10074120 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0ac000 tx=0x7f4a1c019bc0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a232c6640 1 -- 172.21.15.131:0/1561679376 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v123'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f4a1800f040 con 0x55795dbf3760 2024-09-17T02:52:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55795dbf3760 msgr2=0x55795db47d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a29ff0080 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55795dbf3760 0x55795db47d70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4a10071c10 msgr2=0x7f4a10074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a29ff0080 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4a10071c10 0x7f4a10074120 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c0ac000 tx=0x7f4a1c019bc0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1561679376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 msgr2=0x55795dbf63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a29ff0080 1 --2- 172.21.15.131:0/1561679376 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55795dbfbb40 0x55795dbf63f0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f4a1400a9c0 tx=0x7f4a14014650 comp rx=0 tx=0).stop 2024-09-17T02:52:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a242c8640 1 -- 172.21.15.131:0/1561679376 reap_dead start 2024-09-17T02:52:17.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1561679376 shutdown_connections 2024-09-17T02:52:17.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1561679376 >> 172.21.15.131:0/1561679376 conn(0x55795db11d40 msgr2=0x55795dbfef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:17.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1561679376 shutdown_connections 2024-09-17T02:52:17.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.182+0000 7f4a29ff0080 1 -- 172.21.15.131:0/1561679376 wait complete. 2024-09-17T02:52:17.182 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T02:52:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.342+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2591122365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c8815d390 msgr2=0x7f3c880cca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.342+0000 7f3c8f9a3640 1 --2- 172.21.15.131:0/2591122365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c8815d390 0x7f3c880cca80 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009840 tx=0x7f3c7c01c820 comp rx=0 tx=0).stop 2024-09-17T02:52:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.342+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2591122365 shutdown_connections 2024-09-17T02:52:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.342+0000 7f3c8f9a3640 1 --2- 172.21.15.131:0/2591122365 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c8815d390 0x7f3c880cca80 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.342+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2591122365 >> 172.21.15.131:0/2591122365 conn(0x7f3c880b1c40 msgr2=0x7f3c880b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.342+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2591122365 shutdown_connections 2024-09-17T02:52:17.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.342+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2591122365 wait complete. 2024-09-17T02:52:17.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8f9a3640 1 Processor -- start 2024-09-17T02:52:17.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8f9a3640 1 -- start start 2024-09-17T02:52:17.341 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8f9a3640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3c8815a0c0 0x7f3c88150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8f9a3640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c8815d390 0x7f3c881fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8f9a3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c8815d7a0 0x7f3c880c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8f9a3640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8816b4d0 con 0x7f3c8815d7a0 2024-09-17T02:52:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8f9a3640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8816b350 con 0x7f3c8815d390 2024-09-17T02:52:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8f9a3640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8816b650 con 0x7f3c8815a0c0 2024-09-17T02:52:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8df40640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c8815d7a0 0x7f3c880c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8df40640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c8815d7a0 0x7f3c880c5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51942/0 (socket says 172.21.15.131:51942) 2024-09-17T02:52:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8df40640 1 -- 172.21.15.131:0/2021578493 learned_addr learned my addr 172.21.15.131:0/2021578493 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8cf3e640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c8815d390 0x7f3c881fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8d73f640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3c8815a0c0 0x7f3c88150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8df40640 1 -- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3c8815a0c0 msgr2=0x7f3c88150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:17.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8df40640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3c8815a0c0 0x7f3c88150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8df40640 1 -- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c8815d390 msgr2=0x7f3c881fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:17.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8df40640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c8815d390 0x7f3c881fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8df40640 1 -- 172.21.15.131:0/2021578493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c88070f30 con 0x7f3c8815d7a0 2024-09-17T02:52:17.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8d73f640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3c8815a0c0 0x7f3c88150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:17.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8df40640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c8815d7a0 0x7f3c880c5890 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000c3f0 tx=0x7f3c8000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:17.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c85ffb640 1 -- 172.21.15.131:0/2021578493 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80017070 con 0x7f3c8815d7a0 2024-09-17T02:52:17.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c85ffb640 1 -- 172.21.15.131:0/2021578493 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c800053c0 con 0x7f3c8815d7a0 2024-09-17T02:52:17.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c85ffb640 1 -- 172.21.15.131:0/2021578493 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80003860 con 0x7f3c8815d7a0 2024-09-17T02:52:17.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2021578493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c881fabc0 con 0x7f3c8815d7a0 2024-09-17T02:52:17.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.346+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2021578493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c881fb0d0 con 0x7f3c8815d7a0 2024-09-17T02:52:17.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.350+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2021578493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c881fae40 con 0x7f3c8815d7a0 2024-09-17T02:52:17.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.350+0000 7f3c85ffb640 1 -- 172.21.15.131:0/2021578493 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3c8001a070 con 0x7f3c8815d7a0 2024-09-17T02:52:17.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.350+0000 7f3c85ffb640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3c74073230 0x7f3c74075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:17.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.350+0000 7f3c85ffb640 1 -- 172.21.15.131:0/2021578493 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f3c80093790 con 0x7f3c8815d7a0 2024-09-17T02:52:17.344 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.350+0000 7f3c8d73f640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3c74073230 0x7f3c74075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:17.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.350+0000 7f3c8d73f640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3c74073230 0x7f3c74075740 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3c780015b0 tx=0x7f3c78009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:17.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.354+0000 7f3c85ffb640 1 -- 172.21.15.131:0/2021578493 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3c8005e690 con 0x7f3c8815d7a0 2024-09-17T02:52:17.547 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.554+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2021578493 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f3c880cad90 con 0x7f3c8815d7a0 2024-09-17T02:52:17.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.922+0000 7f3c85ffb640 1 -- 172.21.15.131:0/2021578493 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v103) ==== 162+0+0 (secure 0 0 0) 0x7f3c8005df90 con 0x7f3c8815d7a0 2024-09-17T02:52:17.915 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T02:52:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.922+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3c74073230 msgr2=0x7f3c74075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.922+0000 7f3c8f9a3640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3c74073230 0x7f3c74075740 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3c780015b0 tx=0x7f3c78009040 comp rx=0 tx=0).stop 2024-09-17T02:52:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.922+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2021578493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c8815d7a0 msgr2=0x7f3c880c5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.922+0000 7f3c8f9a3640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c8815d7a0 0x7f3c880c5890 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000c3f0 tx=0x7f3c8000c8b0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.922+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2021578493 shutdown_connections 2024-09-17T02:52:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.922+0000 7f3c8f9a3640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3c74073230 0x7f3c74075740 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.922+0000 7f3c8f9a3640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3c8815d7a0 0x7f3c880c5890 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.922+0000 7f3c8f9a3640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3c8815d390 0x7f3c881fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.922+0000 7f3c8f9a3640 1 --2- 172.21.15.131:0/2021578493 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3c8815a0c0 0x7f3c88150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.922+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2021578493 >> 172.21.15.131:0/2021578493 conn(0x7f3c880b1c40 msgr2=0x7f3c880b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.926+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2021578493 shutdown_connections 2024-09-17T02:52:17.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:17.926+0000 7f3c8f9a3640 1 -- 172.21.15.131:0/2021578493 wait complete. 2024-09-17T02:52:17.946 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.094+0000 7ff97da1d640 1 -- 172.21.15.131:0/1974124871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff978074350 msgr2=0x7ff9780cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.094+0000 7ff97da1d640 1 --2- 172.21.15.131:0/1974124871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff978074350 0x7ff9780cd990 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7ff968009900 tx=0x7ff96801c8e0 comp rx=0 tx=0).stop 2024-09-17T02:52:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.094+0000 7ff97da1d640 1 -- 172.21.15.131:0/1974124871 shutdown_connections 2024-09-17T02:52:18.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.094+0000 7ff97da1d640 1 --2- 172.21.15.131:0/1974124871 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff978074350 0x7ff9780cd990 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.094+0000 7ff97da1d640 1 --2- 172.21.15.131:0/1974124871 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff97806d550 0x7ff978073e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.094+0000 7ff97da1d640 1 -- 172.21.15.131:0/1974124871 >> 172.21.15.131:0/1974124871 conn(0x7ff9780b2dc0 msgr2=0x7ff9780b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.094+0000 7ff97da1d640 1 -- 172.21.15.131:0/1974124871 shutdown_connections 2024-09-17T02:52:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.094+0000 7ff97da1d640 1 -- 172.21.15.131:0/1974124871 wait complete. 2024-09-17T02:52:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff97da1d640 1 Processor -- start 2024-09-17T02:52:18.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff97da1d640 1 -- start start 2024-09-17T02:52:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff97da1d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff978074350 0x7ff97813a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff97da1d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff97815d790 0x7ff9781440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff97da1d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff97806d710 0x7ff978144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff97da1d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff978169f30 con 0x7ff978074350 2024-09-17T02:52:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff97da1d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff978169db0 con 0x7ff97806d710 2024-09-17T02:52:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff97da1d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff97816a0b0 con 0x7ff97815d790 2024-09-17T02:52:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff976ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff978074350 0x7ff97813a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff976ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff978074350 0x7ff97813a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51956/0 (socket says 172.21.15.131:51956) 2024-09-17T02:52:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff976ffd640 1 -- 172.21.15.131:0/4114463039 learned_addr learned my addr 172.21.15.131:0/4114463039 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff9767fc640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff97815d790 0x7ff9781440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:18.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff976ffd640 1 -- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff97815d790 msgr2=0x7ff9781440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff976ffd640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff97815d790 0x7ff9781440c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff976ffd640 1 -- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff97806d710 msgr2=0x7ff978144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff9777fe640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff97806d710 0x7ff978144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff976ffd640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff97806d710 0x7ff978144600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff976ffd640 1 -- 172.21.15.131:0/4114463039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97813b6f0 con 0x7ff978074350 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff976ffd640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff978074350 0x7ff97813a240 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7ff97000ba10 tx=0x7ff97000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff95ffff640 1 -- 172.21.15.131:0/4114463039 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff970019070 con 0x7ff978074350 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff95ffff640 1 -- 172.21.15.131:0/4114463039 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff97000ccc0 con 0x7ff978074350 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff97da1d640 1 -- 172.21.15.131:0/4114463039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97813b9d0 con 0x7ff978074350 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff95ffff640 1 -- 172.21.15.131:0/4114463039 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9700146a0 con 0x7ff978074350 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff97da1d640 1 -- 172.21.15.131:0/4114463039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97813feb0 con 0x7ff978074350 2024-09-17T02:52:18.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.098+0000 7ff97da1d640 1 -- 172.21.15.131:0/4114463039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff978140110 con 0x7ff978074350 2024-09-17T02:52:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.102+0000 7ff95ffff640 1 -- 172.21.15.131:0/4114463039 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff97001c070 con 0x7ff978074350 2024-09-17T02:52:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.102+0000 7ff95ffff640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff964073200 0x7ff964075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:18.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.102+0000 7ff95ffff640 1 -- 172.21.15.131:0/4114463039 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7ff970094040 con 0x7ff978074350 2024-09-17T02:52:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.102+0000 7ff9767fc640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff964073200 0x7ff964075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:18.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.102+0000 7ff9767fc640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff964073200 0x7ff964075710 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff97815df20 tx=0x7ff96c008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:18.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.106+0000 7ff95ffff640 1 -- 172.21.15.131:0/4114463039 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff97005e8c0 con 0x7ff978074350 2024-09-17T02:52:18.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.294+0000 7ff97da1d640 1 -- 172.21.15.131:0/4114463039 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff978073e10 con 0x7ff978074350 2024-09-17T02:52:18.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.294+0000 7ff95ffff640 1 -- 172.21.15.131:0/4114463039 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5232 (secure 0 0 0) 0x7ff9700632f0 con 0x7ff978074350 2024-09-17T02:52:18.289 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:18.289 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":104,"btime":"2024-09-17T02:52:17:929763+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:52:17.929730+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8712":{"gid":8712,"name":"a","rank":0,"incarnation":104,"state":"up:replay","state_seq":1,"addr":"172.21.15.131:6835/2981077586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2981077586},{"type":"v1","addr":"172.21.15.131:6835","nonce":2981077586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-17T02:52:18.290 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 104 2024-09-17T02:52:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 -- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff964073200 msgr2=0x7ff964075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff964073200 0x7ff964075710 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff97815df20 tx=0x7ff96c008040 comp rx=0 tx=0).stop 2024-09-17T02:52:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 -- 172.21.15.131:0/4114463039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff978074350 msgr2=0x7ff97813a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff978074350 0x7ff97813a240 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7ff97000ba10 tx=0x7ff97000bed0 comp rx=0 tx=0).stop 2024-09-17T02:52:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 -- 172.21.15.131:0/4114463039 shutdown_connections 2024-09-17T02:52:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff964073200 0x7ff964075710 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff97806d710 0x7ff978144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:18.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff97815d790 0x7ff9781440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 --2- 172.21.15.131:0/4114463039 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff978074350 0x7ff97813a240 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 -- 172.21.15.131:0/4114463039 >> 172.21.15.131:0/4114463039 conn(0x7ff9780b2dc0 msgr2=0x7ff97815c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 -- 172.21.15.131:0/4114463039 shutdown_connections 2024-09-17T02:52:18.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:18.298+0000 7ff97da1d640 1 -- 172.21.15.131:0/4114463039 wait complete. 2024-09-17T02:52:18.713 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:51:54.764322+0000', 'modified': '2024-09-17T02:52:17.929730+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8712': {'gid': 8712, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/2981077586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 2981077586}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 2981077586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:52:18.713 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8712:up:replay 2024-09-17T02:52:19.714 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:19.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.886+0000 7f9c15bdd640 1 -- 172.21.15.131:0/2187454480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf8003b20 msgr2=0x7f9bf8003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:19.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.886+0000 7f9c15bdd640 1 --2- 172.21.15.131:0/2187454480 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf8003b20 0x7f9bf8003f90 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f9c00001df0 tx=0x7f9c0001cb10 comp rx=0 tx=0).stop 2024-09-17T02:52:19.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.886+0000 7f9c0ffff640 1 -- 172.21.15.131:0/2187454480 reap_dead start 2024-09-17T02:52:19.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.886+0000 7f9c15bdd640 1 -- 172.21.15.131:0/2187454480 shutdown_connections 2024-09-17T02:52:19.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.886+0000 7f9c15bdd640 1 -- 172.21.15.131:0/2187454480 >> 172.21.15.131:0/2187454480 conn(0x7f9c1006eed0 msgr2=0x7f9c1006f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:19.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.886+0000 7f9c15bdd640 1 -- 172.21.15.131:0/2187454480 shutdown_connections 2024-09-17T02:52:19.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.886+0000 7f9c15bdd640 1 -- 172.21.15.131:0/2187454480 wait complete. 2024-09-17T02:52:19.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c15bdd640 1 Processor -- start 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c15bdd640 1 -- start start 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c15bdd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf8003b20 0x7f9c10165450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c15bdd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c1013e2c0 0x7f9c100b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c15bdd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c1013e6d0 0x7f9c100b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c15bdd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10167bf0 con 0x7f9bf8003b20 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c15bdd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9c10167a70 con 0x7f9c1013e6d0 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c15bdd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9c10167d70 con 0x7f9c1013e2c0 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf8003b20 0x7f9c10165450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf8003b20 0x7f9c10165450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:51970/0 (socket says 172.21.15.131:51970) 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0f7fe640 1 -- 172.21.15.131:0/1746524911 learned_addr learned my addr 172.21.15.131:0/1746524911 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0ffff640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c1013e6d0 0x7f9c100b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0effd640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c1013e2c0 0x7f9c100b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:19.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0f7fe640 1 -- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c1013e2c0 msgr2=0x7f9c100b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0f7fe640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c1013e2c0 0x7f9c100b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0f7fe640 1 -- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c1013e6d0 msgr2=0x7f9c100b2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0f7fe640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c1013e6d0 0x7f9c100b2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0f7fe640 1 -- 172.21.15.131:0/1746524911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1013eea0 con 0x7f9bf8003b20 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0ffff640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c1013e6d0 0x7f9c100b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c0f7fe640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf8003b20 0x7f9c10165450 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f9c0001b210 tx=0x7f9c00003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9bf7fff640 1 -- 172.21.15.131:0/1746524911 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c000a5960 con 0x7f9bf8003b20 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9bf7fff640 1 -- 172.21.15.131:0/1746524911 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c000a5bf0 con 0x7f9bf8003b20 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c15bdd640 1 -- 172.21.15.131:0/1746524911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c101fa1e0 con 0x7f9bf8003b20 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9bf7fff640 1 -- 172.21.15.131:0/1746524911 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c000019f0 con 0x7f9bf8003b20 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c15bdd640 1 -- 172.21.15.131:0/1746524911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c101fa7a0 con 0x7f9bf8003b20 2024-09-17T02:52:19.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.890+0000 7f9c15bdd640 1 -- 172.21.15.131:0/1746524911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c10074350 con 0x7f9bf8003b20 2024-09-17T02:52:19.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.894+0000 7f9bf7fff640 1 -- 172.21.15.131:0/1746524911 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9c00006070 con 0x7f9bf8003b20 2024-09-17T02:52:19.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.894+0000 7f9bf7fff640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bfc073200 0x7f9bfc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:19.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.894+0000 7f9bf7fff640 1 -- 172.21.15.131:0/1746524911 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f9c0012e380 con 0x7f9bf8003b20 2024-09-17T02:52:19.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.894+0000 7f9c0effd640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bfc073200 0x7f9bfc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:19.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.894+0000 7f9c0effd640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bfc073200 0x7f9bfc075710 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9c100b5b20 tx=0x7f9c040091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:19.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:19.898+0000 7f9bf7fff640 1 -- 172.21.15.131:0/1746524911 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9c00016870 con 0x7f9bf8003b20 2024-09-17T02:52:20.102 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 -- 172.21.15.131:0/1746524911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c100cc4f0 con 0x7f9bf8003b20 2024-09-17T02:52:20.103 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9bf7fff640 1 -- 172.21.15.131:0/1746524911 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) ==== 95+0+5232 (secure 0 0 0) 0x7f9c000f91c0 con 0x7f9bf8003b20 2024-09-17T02:52:20.103 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:20.103 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":106,"btime":"2024-09-17T02:52:19:936576+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:52:18.942075+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8712":{"gid":8712,"name":"a","rank":0,"incarnation":104,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.131:6835/2981077586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2981077586},{"type":"v1","addr":"172.21.15.131:6835","nonce":2981077586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-17T02:52:20.104 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 106 2024-09-17T02:52:20.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 -- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bfc073200 msgr2=0x7f9bfc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:20.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bfc073200 0x7f9bfc075710 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f9c100b5b20 tx=0x7f9c040091e0 comp rx=0 tx=0).stop 2024-09-17T02:52:20.105 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 -- 172.21.15.131:0/1746524911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf8003b20 msgr2=0x7f9c10165450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf8003b20 0x7f9c10165450 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f9c0001b210 tx=0x7f9c00003f50 comp rx=0 tx=0).stop 2024-09-17T02:52:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 -- 172.21.15.131:0/1746524911 shutdown_connections 2024-09-17T02:52:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9bfc073200 0x7f9bfc075710 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9c1013e6d0 0x7f9c100b2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9c1013e2c0 0x7f9c100b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 --2- 172.21.15.131:0/1746524911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9bf8003b20 0x7f9c10165450 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 -- 172.21.15.131:0/1746524911 >> 172.21.15.131:0/1746524911 conn(0x7f9c1006eed0 msgr2=0x7f9c100cba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 -- 172.21.15.131:0/1746524911 shutdown_connections 2024-09-17T02:52:20.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:20.110+0000 7f9c15bdd640 1 -- 172.21.15.131:0/1746524911 wait complete. 2024-09-17T02:52:20.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:51:54.764322+0000', 'modified': '2024-09-17T02:52:18.942075+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8712': {'gid': 8712, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.131:6835/2981077586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 2981077586}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 2981077586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:52:20.530 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8712:up:rejoin 2024-09-17T02:52:21.530 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:21.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.706+0000 7fe8be8b5640 1 -- 172.21.15.131:0/822497356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b80b61d0 msgr2=0x7fe8b8073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:21.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.706+0000 7fe8be8b5640 1 --2- 172.21.15.131:0/822497356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b80b61d0 0x7fe8b8073e10 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009bc0 tx=0x7fe8b001cb70 comp rx=0 tx=0).stop 2024-09-17T02:52:21.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.706+0000 7fe8be8b5640 1 -- 172.21.15.131:0/822497356 shutdown_connections 2024-09-17T02:52:21.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.706+0000 7fe8be8b5640 1 --2- 172.21.15.131:0/822497356 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b80b61d0 0x7fe8b8073e10 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:21.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.706+0000 7fe8be8b5640 1 --2- 172.21.15.131:0/822497356 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe8b8155980 0x7fe8b8071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:21.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.706+0000 7fe8be8b5640 1 -- 172.21.15.131:0/822497356 >> 172.21.15.131:0/822497356 conn(0x7fe8b80b2dc0 msgr2=0x7fe8b80b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:21.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.706+0000 7fe8be8b5640 1 -- 172.21.15.131:0/822497356 shutdown_connections 2024-09-17T02:52:21.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.706+0000 7fe8be8b5640 1 -- 172.21.15.131:0/822497356 wait complete. 2024-09-17T02:52:21.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.706+0000 7fe8be8b5640 1 Processor -- start 2024-09-17T02:52:21.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.706+0000 7fe8be8b5640 1 -- start start 2024-09-17T02:52:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8be8b5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe8b8155980 0x7fe8b8153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8be8b5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8b815c360 0x7fe8b81589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8be8b5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b815ac60 0x7fe8b80d1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8be8b5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b8169360 con 0x7fe8b815ac60 2024-09-17T02:52:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8be8b5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b81691e0 con 0x7fe8b815c360 2024-09-17T02:52:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8be8b5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe8b81694e0 con 0x7fe8b8155980 2024-09-17T02:52:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8bce52640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b815ac60 0x7fe8b80d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8bce52640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b815ac60 0x7fe8b80d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35894/0 (socket says 172.21.15.131:35894) 2024-09-17T02:52:21.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8bce52640 1 -- 172.21.15.131:0/4162532082 learned_addr learned my addr 172.21.15.131:0/4162532082 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8b77fe640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8b815c360 0x7fe8b81589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8b7fff640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe8b8155980 0x7fe8b8153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8bce52640 1 -- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe8b8155980 msgr2=0x7fe8b8153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8bce52640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe8b8155980 0x7fe8b8153e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8bce52640 1 -- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8b815c360 msgr2=0x7fe8b81589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8bce52640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8b815c360 0x7fe8b81589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8bce52640 1 -- 172.21.15.131:0/4162532082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b814fa60 con 0x7fe8b815ac60 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8b77fe640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8b815c360 0x7fe8b81589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8bce52640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b815ac60 0x7fe8b80d1100 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fe8b001a7a0 tx=0x7fe8b0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8b4ff9640 1 -- 172.21.15.131:0/4162532082 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00bb070 con 0x7fe8b815ac60 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8b4ff9640 1 -- 172.21.15.131:0/4162532082 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8b0004e00 con 0x7fe8b815ac60 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8be8b5640 1 -- 172.21.15.131:0/4162532082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b814fd00 con 0x7fe8b815ac60 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8b4ff9640 1 -- 172.21.15.131:0/4162532082 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00b2040 con 0x7fe8b815ac60 2024-09-17T02:52:21.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8be8b5640 1 -- 172.21.15.131:0/4162532082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b81502c0 con 0x7fe8b815ac60 2024-09-17T02:52:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8be8b5640 1 -- 172.21.15.131:0/4162532082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b8073e10 con 0x7fe8b815ac60 2024-09-17T02:52:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8b4ff9640 1 -- 172.21.15.131:0/4162532082 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe8b00be070 con 0x7fe8b815ac60 2024-09-17T02:52:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8b4ff9640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8a4073230 0x7fe8a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:21.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8b4ff9640 1 -- 172.21.15.131:0/4162532082 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7fe8b012e2e0 con 0x7fe8b815ac60 2024-09-17T02:52:21.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.710+0000 7fe8b7fff640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8a4073230 0x7fe8a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:21.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.714+0000 7fe8b7fff640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8a4073230 0x7fe8a4075740 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80058d0 tx=0x7fe8a8005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:21.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.714+0000 7fe8b4ff9640 1 -- 172.21.15.131:0/4162532082 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe8b0016790 con 0x7fe8b815ac60 2024-09-17T02:52:21.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.902+0000 7fe8be8b5640 1 -- 172.21.15.131:0/4162532082 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8b80d0ee0 con 0x7fe8b815ac60 2024-09-17T02:52:21.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8b4ff9640 1 -- 172.21.15.131:0/4162532082 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5239 (secure 0 0 0) 0x7fe8b00f9320 con 0x7fe8b815ac60 2024-09-17T02:52:21.899 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:21.899 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":107,"btime":"2024-09-17T02:52:20:947699+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:52:20.947683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8712":{"gid":8712,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/2981077586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2981077586},{"type":"v1","addr":"172.21.15.131:6835","nonce":2981077586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8712,"qdb_cluster":[8712]},"id":8}]} 2024-09-17T02:52:21.899 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 107 2024-09-17T02:52:21.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 -- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8a4073230 msgr2=0x7fe8a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:21.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8a4073230 0x7fe8a4075740 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80058d0 tx=0x7fe8a8005860 comp rx=0 tx=0).stop 2024-09-17T02:52:21.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 -- 172.21.15.131:0/4162532082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b815ac60 msgr2=0x7fe8b80d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:21.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b815ac60 0x7fe8b80d1100 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fe8b001a7a0 tx=0x7fe8b0003f60 comp rx=0 tx=0).stop 2024-09-17T02:52:21.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 -- 172.21.15.131:0/4162532082 shutdown_connections 2024-09-17T02:52:21.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe8a4073230 0x7fe8a4075740 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:21.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe8b815ac60 0x7fe8b80d1100 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:21.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe8b815c360 0x7fe8b81589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:21.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 --2- 172.21.15.131:0/4162532082 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe8b8155980 0x7fe8b8153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:21.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 -- 172.21.15.131:0/4162532082 >> 172.21.15.131:0/4162532082 conn(0x7fe8b80b2dc0 msgr2=0x7fe8b80b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:21.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 -- 172.21.15.131:0/4162532082 shutdown_connections 2024-09-17T02:52:21.902 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:21.906+0000 7fe8be8b5640 1 -- 172.21.15.131:0/4162532082 wait complete. 2024-09-17T02:52:22.323 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:51:54.764322+0000', 'modified': '2024-09-17T02:52:20.947683+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8712}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8712': {'gid': 8712, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.131:6835/2981077586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 2981077586}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 2981077586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8712, 'qdb_cluster': [8712]} 2024-09-17T02:52:22.323 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:52:22.323 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T02:52:22.488 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.493+0000 7f9fccd5e640 1 -- 172.21.15.131:0/3936114054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fc81605b0 msgr2=0x7f9fc80b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.493+0000 7f9fccd5e640 1 --2- 172.21.15.131:0/3936114054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fc81605b0 0x7f9fc80b3c80 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f9fc815cbb0 tx=0x7f9fb801c620 comp rx=0 tx=0).stop 2024-09-17T02:52:22.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.493+0000 7f9fccd5e640 1 -- 172.21.15.131:0/3936114054 shutdown_connections 2024-09-17T02:52:22.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.493+0000 7f9fccd5e640 1 --2- 172.21.15.131:0/3936114054 >> v2:172.21.15.189:3300/0 conn(0x7f9fc80b41c0 0x7f9fc80d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.493+0000 7f9fccd5e640 1 --2- 172.21.15.131:0/3936114054 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fc81605b0 0x7f9fc80b3c80 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.493+0000 7f9fccd5e640 1 -- 172.21.15.131:0/3936114054 >> 172.21.15.131:0/3936114054 conn(0x7f9fc806d440 msgr2=0x7f9fc80cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:22.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.493+0000 7f9fccd5e640 1 -- 172.21.15.131:0/3936114054 shutdown_connections 2024-09-17T02:52:22.489 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.493+0000 7f9fccd5e640 1 -- 172.21.15.131:0/3936114054 wait complete. 2024-09-17T02:52:22.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fccd5e640 1 Processor -- start 2024-09-17T02:52:22.490 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fccd5e640 1 -- start start 2024-09-17T02:52:22.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fccd5e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9fc80b41c0 0x7f9fc815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fccd5e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fc81601a0 0x7f9fc815f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fccd5e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fc81605b0 0x7f9fc8147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fccd5e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc8153750 con 0x7f9fc81605b0 2024-09-17T02:52:22.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fccd5e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc81535d0 con 0x7f9fc81601a0 2024-09-17T02:52:22.492 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fccd5e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc81538d0 con 0x7f9fc80b41c0 2024-09-17T02:52:22.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fc6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fc81605b0 0x7f9fc8147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fc6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fc81605b0 0x7f9fc8147d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35916/0 (socket says 172.21.15.131:35916) 2024-09-17T02:52:22.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/4084094174 learned_addr learned my addr 172.21.15.131:0/4084094174 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:22.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fc659c640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9fc80b41c0 0x7f9fc815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fc5d9b640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fc81601a0 0x7f9fc815f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9fc80b41c0 msgr2=0x7f9fc815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fc6d9d640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9fc80b41c0 0x7f9fc815ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fc81601a0 msgr2=0x7f9fc815f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fc6d9d640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fc81601a0 0x7f9fc815f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/4084094174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc815f780 con 0x7f9fc81605b0 2024-09-17T02:52:22.493 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fc6d9d640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fc81605b0 0x7f9fc8147d10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000ca60 tx=0x7f9fc000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:22.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fbeffd640 1 -- 172.21.15.131:0/4084094174 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0007d90 con 0x7f9fc81605b0 2024-09-17T02:52:22.494 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fbeffd640 1 -- 172.21.15.131:0/4084094174 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fc0004c20 con 0x7f9fc81605b0 2024-09-17T02:52:22.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fccd5e640 1 -- 172.21.15.131:0/4084094174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc814dc00 con 0x7f9fc81605b0 2024-09-17T02:52:22.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fbeffd640 1 -- 172.21.15.131:0/4084094174 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0005280 con 0x7f9fc81605b0 2024-09-17T02:52:22.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.497+0000 7f9fccd5e640 1 -- 172.21.15.131:0/4084094174 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc80c4b50 con 0x7f9fc81605b0 2024-09-17T02:52:22.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fbeffd640 1 -- 172.21.15.131:0/4084094174 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9fc0005420 con 0x7f9fc81605b0 2024-09-17T02:52:22.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fbeffd640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fa8073200 0x7f9fa8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.495 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fbeffd640 1 -- 172.21.15.131:0/4084094174 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f9fc0094bb0 con 0x7f9fc81605b0 2024-09-17T02:52:22.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fc659c640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fa8073200 0x7f9fa8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fd4029000 1 Processor -- start 2024-09-17T02:52:22.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fd4029000 1 -- start start 2024-09-17T02:52:22.496 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fc659c640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fa8073200 0x7f9fa8075710 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00098e0 tx=0x7f9fb0009060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:22.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555634eb0460 0x5556353f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556352a0950 0x5556353f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555635289ca0 0x5556353f2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fd4029000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5556352c6410 con 0x555635289ca0 2024-09-17T02:52:22.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fd4029000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x555635203b40 con 0x5556352a0950 2024-09-17T02:52:22.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fd4029000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5556351f2f70 con 0x555634eb0460 2024-09-17T02:52:22.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fc6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555635289ca0 0x5556353f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fc6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555635289ca0 0x5556353f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35930/0 (socket says 172.21.15.131:35930) 2024-09-17T02:52:22.500 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/957972449 learned_addr learned my addr 172.21.15.131:0/957972449 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fc659c640 1 --2- 172.21.15.131:0/957972449 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555634eb0460 0x5556353f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.501+0000 7f9fc5d9b640 1 --2- 172.21.15.131:0/957972449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556352a0950 0x5556353f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/957972449 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555634eb0460 msgr2=0x5556353f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fc6d9d640 1 --2- 172.21.15.131:0/957972449 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555634eb0460 0x5556353f16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/957972449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556352a0950 msgr2=0x5556353f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fc6d9d640 1 --2- 172.21.15.131:0/957972449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556352a0950 0x5556353f1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/957972449 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556351ba5b0 con 0x555635289ca0 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fc5d9b640 1 --2- 172.21.15.131:0/957972449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556352a0950 0x5556353f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fc659c640 1 --2- 172.21.15.131:0/957972449 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555634eb0460 0x5556353f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fc6d9d640 1 --2- 172.21.15.131:0/957972449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555635289ca0 0x5556353f2120 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0092e70 tx=0x7f9fc005ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace63dea4c3f622e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fa7fff640 1 -- 172.21.15.131:0/957972449 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0084070 con 0x555635289ca0 2024-09-17T02:52:22.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fa7fff640 1 -- 172.21.15.131:0/957972449 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fc0068c90 con 0x555635289ca0 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fa7fff640 1 -- 172.21.15.131:0/957972449 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc001d040 con 0x555635289ca0 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 -- 172.21.15.131:0/957972449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555635289ca0 msgr2=0x5556353f2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 --2- 172.21.15.131:0/957972449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555635289ca0 0x5556353f2120 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0092e70 tx=0x7f9fc005ee30 comp rx=0 tx=0).stop 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 -- 172.21.15.131:0/957972449 shutdown_connections 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 --2- 172.21.15.131:0/957972449 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555635289ca0 0x5556353f2120 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 --2- 172.21.15.131:0/957972449 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556352a0950 0x5556353f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 --2- 172.21.15.131:0/957972449 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555634eb0460 0x5556353f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 -- 172.21.15.131:0/957972449 >> 172.21.15.131:0/957972449 conn(0x555635376d90 msgr2=0x5556353a6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 -- 172.21.15.131:0/957972449 shutdown_connections 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 -- 172.21.15.131:0/957972449 wait complete. 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 -- start start 2024-09-17T02:52:22.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 Processor -- start 2024-09-17T02:52:22.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x5556353f8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 0x5556353f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5556352a0950 0x5556353f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55563518d5f0 con 0x555634eb0460 2024-09-17T02:52:22.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5556351f6d80 con 0x555635289ca0 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x555635199520 con 0x5556352a0950 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fc659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x5556353f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fd4029000 10 client.0 _ll_register_callbacks cb 0x55563519d830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fc659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x5556353f8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35940/0 (socket says 172.21.15.131:35940) 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.505+0000 7f9fc659c640 1 -- 172.21.15.131:0/3632195399 learned_addr learned my addr 172.21.15.131:0/3632195399 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fc5d9b640 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 0x5556353f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fc6d9d640 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5556352a0950 0x5556353f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fc659c640 1 -- 172.21.15.131:0/3632195399 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5556352a0950 msgr2=0x5556353f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fc659c640 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5556352a0950 0x5556353f93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fc659c640 1 -- 172.21.15.131:0/3632195399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 msgr2=0x5556353f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fc659c640 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 0x5556353f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fc659c640 1 -- 172.21.15.131:0/3632195399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555635191190 con 0x555634eb0460 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fc5d9b640 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 0x5556353f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fc659c640 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x5556353f8920 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0015700 tx=0x7f9fb0004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af4470c4af84c6a server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fa57fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3632195399 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0013720 con 0x555634eb0460 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3632195399 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f9fb00138c0 con 0x555634eb0460 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 -- 172.21.15.131:0/3632195399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55563518b110 con 0x555634eb0460 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3632195399 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f9fb002e050 con 0x555634eb0460 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 10 client.8766 fetch_fsmap learned FSMap version 107 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 -- 172.21.15.131:0/3632195399 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x555635277f10 con 0x555634eb0460 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3632195399 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55563518b110 con 0x555634eb0460 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3632195399 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 107) ==== 3502+0+0 (secure 0 0 0) 0x7f9fb0031020 con 0x555634eb0460 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 10 client.8766 fetch_fsmap finished waiting for FSMap version 107 2024-09-17T02:52:22.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 10 client.8766 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T02:52:22.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 20 client.8766 populate_metadata read hostname 'smithi131' 2024-09-17T02:52:22.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x555634e957d0 0x555635402d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 4 client.8766 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:52:22.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 -- 172.21.15.131:0/3632195399 --> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555635205a90 con 0x555634e957d0 2024-09-17T02:52:22.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fc5d9b640 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x555634e957d0 0x555635402d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fa57fa640 10 client.8766 ms_handle_connect on v2:172.21.15.131:6832/2981077586 2024-09-17T02:52:22.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fc5d9b640 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x555634e957d0 0x555635402d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2737ff4afdfde69 server_cookie=48f9cc0ceba1779 in_seq=0 out_seq=0 2024-09-17T02:52:22.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3632195399 <== mds.0 v2:172.21.15.131:6832/2981077586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555635205a90 con 0x555634e957d0 2024-09-17T02:52:22.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fa57fa640 10 client.8766 handle_command_reply: tid=0 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 client.8766 shutdown 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 -- 172.21.15.131:0/3632195399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 msgr2=0x5556353f8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x5556353f8920 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0015700 tx=0x7f9fb0004690 comp rx=0 tx=0).stop 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 -- 172.21.15.131:0/3632195399 shutdown_connections 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x555634e957d0 0x555635402d00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5556352a0950 0x5556353f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 0x5556353f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3632195399 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x5556353f8920 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fa57fa640 0 client.8766 ms_handle_reset on v2:172.21.15.131:6832/2981077586 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 -- 172.21.15.131:0/3632195399 >> 172.21.15.131:0/3632195399 conn(0x555635376d90 msgr2=0x5556353ad440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 -- 172.21.15.131:0/3632195399 shutdown_connections 2024-09-17T02:52:22.510 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 1 -- 172.21.15.131:0/3632195399 wait complete. 2024-09-17T02:52:22.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.509+0000 7f9fd4029000 20 client.8766 trim_cache size 0 max 16384 2024-09-17T02:52:22.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fd4029000 1 Processor -- start 2024-09-17T02:52:22.523 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fd4029000 1 -- start start 2024-09-17T02:52:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x555635467a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555635289ca0 0x555635467f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556352a0950 0x5556354684b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fd4029000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc8203a60 con 0x555634eb0460 2024-09-17T02:52:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fd4029000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc8202a90 con 0x5556352a0950 2024-09-17T02:52:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fd4029000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9fc81f7720 con 0x555635289ca0 2024-09-17T02:52:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fc659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x555635467a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fc659c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x555635467a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35946/0 (socket says 172.21.15.131:35946) 2024-09-17T02:52:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fc659c640 1 -- 172.21.15.131:0/912795932 learned_addr learned my addr 172.21.15.131:0/912795932 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:22.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fc659c640 1 -- 172.21.15.131:0/912795932 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555635289ca0 msgr2=0x555635467f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:22.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fc659c640 1 --2- 172.21.15.131:0/912795932 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555635289ca0 0x555635467f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fc6d9d640 1 --2- 172.21.15.131:0/912795932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556352a0950 0x5556354684b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fc659c640 1 -- 172.21.15.131:0/912795932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556352a0950 msgr2=0x5556354684b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:22.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fc659c640 1 --2- 172.21.15.131:0/912795932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556352a0950 0x5556354684b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fc659c640 1 -- 172.21.15.131:0/912795932 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563524edb0 con 0x555634eb0460 2024-09-17T02:52:22.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fc659c640 1 --2- 172.21.15.131:0/912795932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x555635467a30 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000ec70 tx=0x7f9fb0013db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8d753121dfbb05 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:22.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fa77fe640 1 -- 172.21.15.131:0/912795932 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0036020 con 0x555634eb0460 2024-09-17T02:52:22.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fa77fe640 1 -- 172.21.15.131:0/912795932 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fb0026070 con 0x555634eb0460 2024-09-17T02:52:22.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.529+0000 7f9fa77fe640 1 -- 172.21.15.131:0/912795932 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb00216c0 con 0x555634eb0460 2024-09-17T02:52:22.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 -- 172.21.15.131:0/912795932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 msgr2=0x555635467a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 --2- 172.21.15.131:0/912795932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x555635467a30 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f9fb000ec70 tx=0x7f9fb0013db0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 -- 172.21.15.131:0/912795932 shutdown_connections 2024-09-17T02:52:22.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 --2- 172.21.15.131:0/912795932 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5556352a0950 0x5556354684b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 --2- 172.21.15.131:0/912795932 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555635289ca0 0x555635467f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 --2- 172.21.15.131:0/912795932 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x555634eb0460 0x555635467a30 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 -- 172.21.15.131:0/912795932 >> 172.21.15.131:0/912795932 conn(0x555635376d90 msgr2=0x5556353ef620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:22.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 -- 172.21.15.131:0/912795932 shutdown_connections 2024-09-17T02:52:22.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 -- 172.21.15.131:0/912795932 wait complete. 2024-09-17T02:52:22.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 -- start start 2024-09-17T02:52:22.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 Processor -- start 2024-09-17T02:52:22.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555634eb0460 0x55563546b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 0x55563546bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556352a0950 0x55563546c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc8202440 con 0x5556352a0950 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5556351beee0 con 0x555635289ca0 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5556352b2750 con 0x555634eb0460 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 10 client.0 _ll_register_callbacks cb 0x7f9fb0006e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556352a0950 0x55563546c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556352a0950 0x55563546c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35954/0 (socket says 172.21.15.131:35954) 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/3739031935 learned_addr learned my addr 172.21.15.131:0/3739031935 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc659c640 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555634eb0460 0x55563546b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc5d9b640 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 0x55563546bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/3739031935 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555634eb0460 msgr2=0x55563546b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc6d9d640 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555634eb0460 0x55563546b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/3739031935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 msgr2=0x55563546bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc6d9d640 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 0x55563546bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.534 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc6d9d640 1 -- 172.21.15.131:0/3739031935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556351b9ab0 con 0x5556352a0950 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc5d9b640 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 0x55563546bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fc6d9d640 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556352a0950 0x55563546c220 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0092f50 tx=0x7f9fc0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460cadcb3c9a6c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fa57fa640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3739031935 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc001d070 con 0x5556352a0950 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3739031935 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f9fc005e580 con 0x5556352a0950 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.533+0000 7f9fd4029000 1 -- 172.21.15.131:0/3739031935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb0026070 con 0x5556352a0950 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3739031935 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f9fc0084040 con 0x5556352a0950 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 10 client.8772 fetch_fsmap learned FSMap version 107 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 -- 172.21.15.131:0/3739031935 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x555635198070 con 0x5556352a0950 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3739031935 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fb0026070 con 0x5556352a0950 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3739031935 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 107) ==== 3502+0+0 (secure 0 0 0) 0x7f9fc007f020 con 0x5556352a0950 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 10 client.8772 fetch_fsmap finished waiting for FSMap version 107 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 10 client.8772 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T02:52:22.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 20 client.8772 populate_metadata read hostname 'smithi131' 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x555634e83830 0x555635427ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 4 client.8772 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 -- 172.21.15.131:0/3739031935 --> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] -- command(tid 0: {"prefix": "status"}) -- 0x555635201540 con 0x555634e83830 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fc659c640 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x555634e83830 0x555635427ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fa57fa640 10 client.8772 ms_handle_connect on v2:172.21.15.131:6832/2981077586 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fc659c640 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x555634e83830 0x555635427ab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a050d407b52a60 server_cookie=cc76cc9de50f61e4 in_seq=0 out_seq=0 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fa57fa640 1 -- 172.21.15.131:0/3739031935 <== mds.0 v2:172.21.15.131:6832/2981077586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555635201540 con 0x555634e83830 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fa57fa640 10 client.8772 handle_command_reply: tid=0 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 client.8772 shutdown 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 -- 172.21.15.131:0/3739031935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556352a0950 msgr2=0x55563546c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556352a0950 0x55563546c220 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0092f50 tx=0x7f9fc0007f30 comp rx=0 tx=0).stop 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 -- 172.21.15.131:0/3739031935 shutdown_connections 2024-09-17T02:52:22.536 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x555634e83830 0x555635427ab0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5556352a0950 0x55563546c220 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x555635289ca0 0x55563546bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 --2- 172.21.15.131:0/3739031935 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x555634eb0460 0x55563546b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fa57fa640 0 client.8772 ms_handle_reset on v2:172.21.15.131:6832/2981077586 2024-09-17T02:52:22.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 -- 172.21.15.131:0/3739031935 >> 172.21.15.131:0/3739031935 conn(0x555635376d90 msgr2=0x5556353effe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:22.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 -- 172.21.15.131:0/3739031935 shutdown_connections 2024-09-17T02:52:22.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 1 -- 172.21.15.131:0/3739031935 wait complete. 2024-09-17T02:52:22.537 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fd4029000 20 client.8772 trim_cache size 0 max 16384 2024-09-17T02:52:22.537 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8712, 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 4.599701949, 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 107, 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 123, 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 123, 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 5.5081191570000003 2024-09-17T02:52:22.538 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:22.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 -- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fa8073200 msgr2=0x7f9fa8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fa8073200 0x7f9fa8075710 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00098e0 tx=0x7f9fb0009060 comp rx=0 tx=0).stop 2024-09-17T02:52:22.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 -- 172.21.15.131:0/4084094174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fc81605b0 msgr2=0x7f9fc8147d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fc81605b0 0x7f9fc8147d10 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000ca60 tx=0x7f9fc000cf20 comp rx=0 tx=0).stop 2024-09-17T02:52:22.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 -- 172.21.15.131:0/4084094174 shutdown_connections 2024-09-17T02:52:22.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9fa8073200 0x7f9fa8075710 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9fc81605b0 0x7f9fc8147d10 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9fc81601a0 0x7f9fc815f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 --2- 172.21.15.131:0/4084094174 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9fc80b41c0 0x7f9fc815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 -- 172.21.15.131:0/4084094174 >> 172.21.15.131:0/4084094174 conn(0x7f9fc806d440 msgr2=0x7f9fc80b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:22.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 -- 172.21.15.131:0/4084094174 shutdown_connections 2024-09-17T02:52:22.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.537+0000 7f9fccd5e640 1 -- 172.21.15.131:0/4084094174 wait complete. 2024-09-17T02:52:22.551 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:52:22.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.701+0000 7fad3438c640 1 -- 172.21.15.131:0/1009003556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad2c0d2dd0 msgr2=0x7fad2c0d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.701+0000 7fad3438c640 1 --2- 172.21.15.131:0/1009003556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad2c0d2dd0 0x7fad2c0d31e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0ce940 tx=0x7fad2801c6e0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.701+0000 7fad3438c640 1 -- 172.21.15.131:0/1009003556 shutdown_connections 2024-09-17T02:52:22.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.701+0000 7fad3438c640 1 --2- 172.21.15.131:0/1009003556 >> v2:172.21.15.189:3300/0 conn(0x7fad2c0b5e40 0x7fad2c0b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.701+0000 7fad3438c640 1 --2- 172.21.15.131:0/1009003556 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad2c0d2dd0 0x7fad2c0d31e0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.701+0000 7fad3438c640 1 -- 172.21.15.131:0/1009003556 >> 172.21.15.131:0/1009003556 conn(0x7fad2c0d0e40 msgr2=0x7fad2c0d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:22.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.701+0000 7fad3438c640 1 -- 172.21.15.131:0/1009003556 shutdown_connections 2024-09-17T02:52:22.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.701+0000 7fad3438c640 1 -- 172.21.15.131:0/1009003556 wait complete. 2024-09-17T02:52:22.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad3438c640 1 Processor -- start 2024-09-17T02:52:22.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad3438c640 1 -- start start 2024-09-17T02:52:22.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad3438c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad2c0b5e40 0x7fad2c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad3438c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fad2c0d2dd0 0x7fad2c1f9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad3438c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad2c15c3f0 0x7fad2c151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad3438c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c0b3940 con 0x7fad2c15c3f0 2024-09-17T02:52:22.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad3438c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fad2c0b37c0 con 0x7fad2c0b5e40 2024-09-17T02:52:22.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad3438c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fad2c0b3ac0 con 0x7fad2c0d2dd0 2024-09-17T02:52:22.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad32929640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad2c15c3f0 0x7fad2c151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad32929640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad2c15c3f0 0x7fad2c151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35980/0 (socket says 172.21.15.131:35980) 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad32929640 1 -- 172.21.15.131:0/1151926868 learned_addr learned my addr 172.21.15.131:0/1151926868 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad31927640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fad2c0d2dd0 0x7fad2c1f9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad32128640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad2c0b5e40 0x7fad2c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad32929640 1 -- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fad2c0d2dd0 msgr2=0x7fad2c1f9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad32929640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fad2c0d2dd0 0x7fad2c1f9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad32929640 1 -- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad2c0b5e40 msgr2=0x7fad2c1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad32929640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad2c0b5e40 0x7fad2c1529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad32929640 1 -- 172.21.15.131:0/1151926868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2c0cab40 con 0x7fad2c15c3f0 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad32929640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad2c15c3f0 0x7fad2c151340 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fad2400ca80 tx=0x7fad2400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad22ffd640 1 -- 172.21.15.131:0/1151926868 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24007d90 con 0x7fad2c15c3f0 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad22ffd640 1 -- 172.21.15.131:0/1151926868 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad24003a90 con 0x7fad2c15c3f0 2024-09-17T02:52:22.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad3438c640 1 -- 172.21.15.131:0/1151926868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2c151ee0 con 0x7fad2c15c3f0 2024-09-17T02:52:22.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad22ffd640 1 -- 172.21.15.131:0/1151926868 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad24004020 con 0x7fad2c15c3f0 2024-09-17T02:52:22.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad3438c640 1 -- 172.21.15.131:0/1151926868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad2c1524a0 con 0x7fad2c15c3f0 2024-09-17T02:52:22.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.705+0000 7fad3438c640 1 -- 172.21.15.131:0/1151926868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad2c0b6250 con 0x7fad2c15c3f0 2024-09-17T02:52:22.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.709+0000 7fad22ffd640 1 -- 172.21.15.131:0/1151926868 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fad240041c0 con 0x7fad2c15c3f0 2024-09-17T02:52:22.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.709+0000 7fad22ffd640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad10073200 0x7fad10075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:22.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.709+0000 7fad22ffd640 1 -- 172.21.15.131:0/1151926868 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7fad24094e30 con 0x7fad2c15c3f0 2024-09-17T02:52:22.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.709+0000 7fad32128640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad10073200 0x7fad10075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:22.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.709+0000 7fad32128640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad10073200 0x7fad10075710 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00a750 tx=0x7fad1c008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:22.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.713+0000 7fad22ffd640 1 -- 172.21.15.131:0/1151926868 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fad24014070 con 0x7fad2c15c3f0 2024-09-17T02:52:22.906 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 -- 172.21.15.131:0/1151926868 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fad2c0b6dc0 con 0x7fad2c15c3f0 2024-09-17T02:52:22.907 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad22ffd640 1 -- 172.21.15.131:0/1151926868 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v107) ==== 53+0+73 (secure 0 0 0) 0x7fad2405fd00 con 0x7fad2c15c3f0 2024-09-17T02:52:22.907 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:52:22.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 -- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad10073200 msgr2=0x7fad10075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad10073200 0x7fad10075710 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fad1c00a750 tx=0x7fad1c008040 comp rx=0 tx=0).stop 2024-09-17T02:52:22.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 -- 172.21.15.131:0/1151926868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad2c15c3f0 msgr2=0x7fad2c151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:22.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad2c15c3f0 0x7fad2c151340 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fad2400ca80 tx=0x7fad2400cf40 comp rx=0 tx=0).stop 2024-09-17T02:52:22.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 -- 172.21.15.131:0/1151926868 shutdown_connections 2024-09-17T02:52:22.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fad10073200 0x7fad10075710 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fad2c15c3f0 0x7fad2c151340 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fad2c0d2dd0 0x7fad2c1f9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 --2- 172.21.15.131:0/1151926868 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fad2c0b5e40 0x7fad2c1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:22.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 -- 172.21.15.131:0/1151926868 >> 172.21.15.131:0/1151926868 conn(0x7fad2c0d0e40 msgr2=0x7fad2c0b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:22.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 -- 172.21.15.131:0/1151926868 shutdown_connections 2024-09-17T02:52:22.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:22.913+0000 7fad3438c640 1 -- 172.21.15.131:0/1151926868 wait complete. 2024-09-17T02:52:22.928 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:52:22.928 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:52:22.928 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:52:22.928 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:52:22.928 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:22.928 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:52:22.928 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:52:22.928 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:52:22.928 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:22.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:22.929 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:52:22.933 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-09-17T02:52:22.937+0000 7f0f95025640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:52:22.934 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:52:22.934 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:22.935 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41763sec preferred_lft 41763sec 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:22.936 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:52:22.937 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:22.937 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:52:22.937 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:22.937 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:52:22.937 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:22.937 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:52:22.937 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:22.937 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:52:22.937 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:22.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:22.937 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:52:22.988 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:52:22.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:22.990 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:52:23.041 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:52:23.041 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:52:23.041 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:52:23.041 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:52:23.041 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:52:23.041 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:52:23.041 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:52:23.042 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:23.042 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:23.042 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:23.042 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:52:23.042 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:52:23.042 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:52:23.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:52:23.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:52:23.042 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:52:23.042 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:52:23.125 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:52:23.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:23.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:23.157 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:52:23.203 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:52:23.203 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:52:23.203 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:23.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:23.204 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:52:23.259 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:52:23.259 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:52:23.259 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:23.307 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:23.307 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:52:23.307 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:23.354 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:23.412 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:23.459 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:23.460 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:23.460 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:23.460 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:23.460 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:23.460 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:23.460 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:23.460 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:23.460 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:23.460 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:23.460 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:23.460 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:23.461 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:23.461 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:23.461 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:23.461 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:23.461 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:23.461 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:23.461 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:23.461 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:23.462 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:23.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:23.464 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:23.511 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:52:23.511 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:52:23.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:23.570 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:23.619 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:23.619 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:23.619 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:23.619 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:23.619 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:23.619 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:23.619 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:23.619 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:23.620 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:23.621 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:23.621 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:23.621 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:23.621 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:23.621 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:23.621 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:23.621 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:23.621 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:23.621 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:23.621 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:23.621 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:23.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:23.622 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:23.668 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:52:23.673+0000 7f14ee7f1580 -1 init, newargv = 0x55959f9f89c0 newargc=15 2024-09-17T02:52:23.668 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[52008]: starting ceph client 2024-09-17T02:52:23.674 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[52008]: starting fuse 2024-09-17T02:52:24.772 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:24.788 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:24.835 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:24.835 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:24.835 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:24.835 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:24.836 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:24.837 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:24.838 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:24.838 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:24.838 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:24.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:24.839 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:24.887 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:52:24.887 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:52:24.887 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:52:24.887 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:24.888 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:52:24.889 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:52:24.889 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:52:24.889 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:52:24.889 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:52:24.889 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:52:24.889 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:52:24.889 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:52:24.889 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:24.889 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:52:24.889 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:25.013 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.52008.asok 2024-09-17T02:52:25.017 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:25.025 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52008.asok 2024-09-17T02:52:25.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:25.026 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52008.asok status 2024-09-17T02:52:25.209 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:52:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:52:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:52:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:52:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:52:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:52:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "52008", 2024-09-17T02:52:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:52:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:52:25.210 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8784, 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 8784 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3942631780 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:25.211 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 3942631780 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.8784 192.168.0.1:0/3942631780", 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/3942631780", 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 107, 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 123, 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:52:25.212 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:25.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:25.226 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:52:25.226 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:25.232 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:52:25.232 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:25.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:25.233 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:25.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:25.286 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-17T02:52:25.355 INFO:teuthology.orchestra.run.smithi131.stdout:./ 2024-09-17T02:52:25.355 INFO:teuthology.orchestra.run.smithi131.stdout:./parent 2024-09-17T02:52:25.355 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/flushed 2024-09-17T02:52:25.355 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/flushed/charlie 2024-09-17T02:52:25.355 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/flushed/alpha 2024-09-17T02:52:25.355 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/unflushed 2024-09-17T02:52:25.355 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/unflushed/jfile 2024-09-17T02:52:25.356 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:25.356 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:25.356 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:25.357 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:25.357 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-17T02:52:25.357 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:25.463 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:52:25.466 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:25.473 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:25.473 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:25.473 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:25.474 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:25.474 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-09-17T02:52:25.474 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:25.575 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627776 2024-09-17T02:52:25.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:25.585 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:25.585 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:25.586 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:25.586 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:25.586 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-09-17T02:52:25.586 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:25.694 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627777 2024-09-17T02:52:25.697 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:25.704 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:25.704 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:25.704 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:25.704 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:25.704 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-09-17T02:52:25.704 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:25.806 INFO:teuthology.orchestra.run.smithi131.stdout:1099511628280 2024-09-17T02:52:25.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:25.815 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:25.815 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:25.815 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:25.815 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:25.815 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-09-17T02:52:25.815 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:25.917 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627778 2024-09-17T02:52:25.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:25.927 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:25.927 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:25.928 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:25.928 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:25.928 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-09-17T02:52:25.928 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:26.033 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627779 2024-09-17T02:52:26.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:26.044 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:26.044 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:26.044 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:26.044 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:26.045 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-09-17T02:52:26.045 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:26.150 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627780 2024-09-17T02:52:26.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:26.159 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:26.159 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:26.207 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:52:26.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 -- 172.21.15.131:0/3589109165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6524071c70 msgr2=0x7f6524073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:26.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 --2- 172.21.15.131:0/3589109165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6524071c70 0x7f6524073e10 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f6524072500 tx=0x7f650c01c7d0 comp rx=0 tx=0).stop 2024-09-17T02:52:26.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 -- 172.21.15.131:0/3589109165 shutdown_connections 2024-09-17T02:52:26.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 --2- 172.21.15.131:0/3589109165 >> v2:172.21.15.189:3300/0 conn(0x7f6524074350 0x7f65240d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:26.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 --2- 172.21.15.131:0/3589109165 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f6524071c70 0x7f6524073e10 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:26.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 -- 172.21.15.131:0/3589109165 >> 172.21.15.131:0/3589109165 conn(0x7f65240b2dc0 msgr2=0x7f65240b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:26.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 -- 172.21.15.131:0/3589109165 shutdown_connections 2024-09-17T02:52:26.415 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 -- 172.21.15.131:0/3589109165 wait complete. 2024-09-17T02:52:26.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 Processor -- start 2024-09-17T02:52:26.416 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 -- start start 2024-09-17T02:52:26.418 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6524071c70 0x7f652415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6524074350 0x7f652414faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65240b0640 0x7f652415b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f6524168960 con 0x7f65240b0640 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f65241687e0 con 0x7f6524071c70 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f652a666640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f6524168ae0 con 0x7f6524074350 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f6528c03640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65240b0640 0x7f652415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f6528c03640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65240b0640 0x7f652415b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:35986/0 (socket says 172.21.15.131:35986) 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f6528c03640 1 -- 172.21.15.131:0/3654649924 learned_addr learned my addr 172.21.15.131:0/3654649924 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.421+0000 7f6523fff640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6524071c70 0x7f652415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f65237fe640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6524074350 0x7f652414faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6528c03640 1 -- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6524074350 msgr2=0x7f652414faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:26.419 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6528c03640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6524074350 0x7f652414faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:26.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6528c03640 1 -- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6524071c70 msgr2=0x7f652415a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:26.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6528c03640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6524071c70 0x7f652415a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:26.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6528c03640 1 -- 172.21.15.131:0/3654649924 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6524160010 con 0x7f65240b0640 2024-09-17T02:52:26.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6528c03640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65240b0640 0x7f652415b480 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f651400e970 tx=0x7f651400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:26.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6520ff9640 1 -- 172.21.15.131:0/3654649924 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514004d30 con 0x7f65240b0640 2024-09-17T02:52:26.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6520ff9640 1 -- 172.21.15.131:0/3654649924 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f651400cb20 con 0x7f65240b0640 2024-09-17T02:52:26.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f652a666640 1 -- 172.21.15.131:0/3654649924 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6524150030 con 0x7f65240b0640 2024-09-17T02:52:26.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6520ff9640 1 -- 172.21.15.131:0/3654649924 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514005020 con 0x7f65240b0640 2024-09-17T02:52:26.420 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f652a666640 1 -- 172.21.15.131:0/3654649924 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6524158520 con 0x7f65240b0640 2024-09-17T02:52:26.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f652a666640 1 -- 172.21.15.131:0/3654649924 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f652415a610 con 0x7f65240b0640 2024-09-17T02:52:26.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6520ff9640 1 -- 172.21.15.131:0/3654649924 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f65140051c0 con 0x7f65240b0640 2024-09-17T02:52:26.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6520ff9640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6504073200 0x7f6504075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:26.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6520ff9640 1 -- 172.21.15.131:0/3654649924 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f6514094bf0 con 0x7f65240b0640 2024-09-17T02:52:26.421 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.425+0000 7f6523fff640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6504073200 0x7f6504075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:26.422 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.429+0000 7f6523fff640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6504073200 0x7f6504075710 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f651c000f70 tx=0x7f651c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:26.425 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.429+0000 7f6520ff9640 1 -- 172.21.15.131:0/3654649924 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f6514016030 con 0x7f65240b0640 2024-09-17T02:52:26.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.613+0000 7f652a666640 1 -- 172.21.15.131:0/3654649924 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f65240b5670 con 0x7f65240b0640 2024-09-17T02:52:26.607 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.613+0000 7f6520ff9640 1 -- 172.21.15.131:0/3654649924 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v123) ==== 80+0+124 (secure 0 0 0) 0x7f651405fb50 con 0x7f65240b0640 2024-09-17T02:52:26.608 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6835/2599844908 2024-09-18T02:52:16.917982+0000 2024-09-17T02:52:26.608 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6832/2599844908 2024-09-18T02:52:16.917982+0000 2024-09-17T02:52:26.608 INFO:teuthology.orchestra.run.smithi131.stderr:listed 2 entries 2024-09-17T02:52:26.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 -- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6504073200 msgr2=0x7f6504075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:26.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6504073200 0x7f6504075710 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f651c000f70 tx=0x7f651c009040 comp rx=0 tx=0).stop 2024-09-17T02:52:26.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 -- 172.21.15.131:0/3654649924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65240b0640 msgr2=0x7f652415b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:26.610 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65240b0640 0x7f652415b480 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f651400e970 tx=0x7f651400ee30 comp rx=0 tx=0).stop 2024-09-17T02:52:26.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 -- 172.21.15.131:0/3654649924 shutdown_connections 2024-09-17T02:52:26.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6504073200 0x7f6504075710 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:26.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f65240b0640 0x7f652415b480 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:26.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f6524074350 0x7f652414faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:26.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 --2- 172.21.15.131:0/3654649924 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f6524071c70 0x7f652415a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:26.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 -- 172.21.15.131:0/3654649924 >> 172.21.15.131:0/3654649924 conn(0x7f65240b2dc0 msgr2=0x7f65240b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:26.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 -- 172.21.15.131:0/3654649924 shutdown_connections 2024-09-17T02:52:26.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.617+0000 7f652a666640 1 -- 172.21.15.131:0/3654649924 wait complete. 2024-09-17T02:52:26.630 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:26.630 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:26.636 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:52:26.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.849+0000 7f8017dfa640 1 -- 172.21.15.131:0/2851119105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ffc004e70 msgr2=0x7f7ffc0052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:26.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.849+0000 7f8017dfa640 1 --2- 172.21.15.131:0/2851119105 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ffc004e70 0x7f7ffc0052c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f80100709b0 tx=0x7f8008020d50 comp rx=0 tx=0).stop 2024-09-17T02:52:26.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.849+0000 7f8016397640 1 -- 172.21.15.131:0/2851119105 reap_dead start 2024-09-17T02:52:26.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.849+0000 7f8017dfa640 1 -- 172.21.15.131:0/2851119105 shutdown_connections 2024-09-17T02:52:26.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.849+0000 7f8017dfa640 1 -- 172.21.15.131:0/2851119105 >> 172.21.15.131:0/2851119105 conn(0x7f80100cff40 msgr2=0x7f80100d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:26.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.849+0000 7f8017dfa640 1 -- 172.21.15.131:0/2851119105 shutdown_connections 2024-09-17T02:52:26.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.849+0000 7f8017dfa640 1 -- 172.21.15.131:0/2851119105 wait complete. 2024-09-17T02:52:26.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.849+0000 7f8017dfa640 1 Processor -- start 2024-09-17T02:52:26.845 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.849+0000 7f8017dfa640 1 -- start start 2024-09-17T02:52:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8017dfa640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc004e70 0x7f8010154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8017dfa640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80101f7390 0x7f80101f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8017dfa640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8010152390 0x7f80101527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8017dfa640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f80100cb4d0 con 0x7f8010152390 2024-09-17T02:52:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8017dfa640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f80100cb350 con 0x7f7ffc004e70 2024-09-17T02:52:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8017dfa640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f801006eef0 con 0x7f80101f7390 2024-09-17T02:52:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8016397640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8010152390 0x7f80101527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8016397640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8010152390 0x7f80101527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:36018/0 (socket says 172.21.15.131:36018) 2024-09-17T02:52:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8016397640 1 -- 172.21.15.131:0/2388856231 learned_addr learned my addr 172.21.15.131:0/2388856231 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8015b96640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc004e70 0x7f8010154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:26.848 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8015395640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80101f7390 0x7f80101f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8016397640 1 -- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80101f7390 msgr2=0x7f80101f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8016397640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80101f7390 0x7f80101f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8016397640 1 -- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc004e70 msgr2=0x7f8010154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8016397640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc004e70 0x7f8010154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8016397640 1 -- 172.21.15.131:0/2388856231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8010152ce0 con 0x7f8010152390 2024-09-17T02:52:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8015395640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80101f7390 0x7f80101f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8016397640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8010152390 0x7f80101527a0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8008006040 tx=0x7f80080020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f80067fc640 1 -- 172.21.15.131:0/2388856231 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008001c60 con 0x7f8010152390 2024-09-17T02:52:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f80067fc640 1 -- 172.21.15.131:0/2388856231 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8008004e40 con 0x7f8010152390 2024-09-17T02:52:26.849 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f80067fc640 1 -- 172.21.15.131:0/2388856231 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008001880 con 0x7f8010152390 2024-09-17T02:52:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8017dfa640 1 -- 172.21.15.131:0/2388856231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80101551b0 con 0x7f8010152390 2024-09-17T02:52:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8017dfa640 1 -- 172.21.15.131:0/2388856231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80101f37f0 con 0x7f8010152390 2024-09-17T02:52:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.853+0000 7f8017dfa640 1 -- 172.21.15.131:0/2388856231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801006f450 con 0x7f8010152390 2024-09-17T02:52:26.850 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.857+0000 7f80067fc640 1 -- 172.21.15.131:0/2388856231 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f80080bb030 con 0x7f8010152390 2024-09-17T02:52:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.857+0000 7f80067fc640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7ff40733a0 0x7f7ff40758b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.857+0000 7f80067fc640 1 -- 172.21.15.131:0/2388856231 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f80081315d0 con 0x7f8010152390 2024-09-17T02:52:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.857+0000 7f8015b96640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7ff40733a0 0x7f7ff40758b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:26.851 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.857+0000 7f8015b96640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7ff40733a0 0x7f7ff40758b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f800c005fd0 tx=0x7f800c005f40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:26.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:26.861+0000 7f80067fc640 1 -- 172.21.15.131:0/2388856231 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8010152ce0 con 0x7f8010152390 2024-09-17T02:52:27.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.041+0000 7f8017dfa640 1 -- 172.21.15.131:0/2388856231 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8010155410 con 0x7f8010152390 2024-09-17T02:52:27.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.041+0000 7f80067fc640 1 -- 172.21.15.131:0/2388856231 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v123) ==== 80+0+124 (secure 0 0 0) 0x7f800801ab00 con 0x7f8010152390 2024-09-17T02:52:27.038 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6835/2599844908 2024-09-18T02:52:16.917982+0000 2024-09-17T02:52:27.038 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6832/2599844908 2024-09-18T02:52:16.917982+0000 2024-09-17T02:52:27.038 INFO:teuthology.orchestra.run.smithi131.stderr:listed 2 entries 2024-09-17T02:52:27.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 -- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7ff40733a0 msgr2=0x7f7ff40758b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:27.040 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7ff40733a0 0x7f7ff40758b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f800c005fd0 tx=0x7f800c005f40 comp rx=0 tx=0).stop 2024-09-17T02:52:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 -- 172.21.15.131:0/2388856231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8010152390 msgr2=0x7f80101527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8010152390 0x7f80101527a0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f8008006040 tx=0x7f80080020e0 comp rx=0 tx=0).stop 2024-09-17T02:52:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 -- 172.21.15.131:0/2388856231 shutdown_connections 2024-09-17T02:52:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7ff40733a0 0x7f7ff40758b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8010152390 0x7f80101527a0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f80101f7390 0x7f80101f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 --2- 172.21.15.131:0/2388856231 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ffc004e70 0x7f8010154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 -- 172.21.15.131:0/2388856231 >> 172.21.15.131:0/2388856231 conn(0x7f80100cff40 msgr2=0x7f801014f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 -- 172.21.15.131:0/2388856231 shutdown_connections 2024-09-17T02:52:27.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:27.045+0000 7f8017dfa640 1 -- 172.21.15.131:0/2388856231 wait complete. 2024-09-17T02:52:27.059 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:52:27.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:27.060 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:27.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[52008]: fuse finished with error 0 and tester_r 0 2024-09-17T02:52:27.071 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:52:33.172 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:52:33.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:33.173 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:52:33.181 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:33.386 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 -- 172.21.15.131:0/590963283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad8003af0 msgr2=0x7f2ad8003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 --2- 172.21.15.131:0/590963283 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad8003af0 0x7f2ad8003f60 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8006fd0 tx=0x7f2ae801ce70 comp rx=0 tx=0).stop 2024-09-17T02:52:33.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af5cb9640 1 -- 172.21.15.131:0/590963283 reap_dead start 2024-09-17T02:52:33.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 -- 172.21.15.131:0/590963283 shutdown_connections 2024-09-17T02:52:33.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 -- 172.21.15.131:0/590963283 >> 172.21.15.131:0/590963283 conn(0x7f2af006d440 msgr2=0x7f2af00cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:33.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 -- 172.21.15.131:0/590963283 shutdown_connections 2024-09-17T02:52:33.387 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 -- 172.21.15.131:0/590963283 wait complete. 2024-09-17T02:52:33.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 Processor -- start 2024-09-17T02:52:33.388 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 -- start start 2024-09-17T02:52:33.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad8003af0 0x7f2af014ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2af01535a0 0x7f2af00c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2af00c7e60 0x7f2af01f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2af0152f10 con 0x7f2ad8003af0 2024-09-17T02:52:33.391 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2af0152d90 con 0x7f2af00c7e60 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af771c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2af0153090 con 0x7f2af01535a0 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af54b8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad8003af0 0x7f2af014ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af54b8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad8003af0 0x7f2af014ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40500/0 (socket says 172.21.15.131:40500) 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.393+0000 7f2af54b8640 1 -- 172.21.15.131:0/906637986 learned_addr learned my addr 172.21.15.131:0/906637986 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af5cb9640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2af00c7e60 0x7f2af01f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af4cb7640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2af01535a0 0x7f2af00c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af54b8640 1 -- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2af01535a0 msgr2=0x7f2af00c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af54b8640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2af01535a0 0x7f2af00c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af54b8640 1 -- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2af00c7e60 msgr2=0x7f2af01f9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af54b8640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2af00c7e60 0x7f2af01f9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af54b8640 1 -- 172.21.15.131:0/906637986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2af00c8270 con 0x7f2ad8003af0 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af4cb7640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2af01535a0 0x7f2af00c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:33.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af5cb9640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2af00c7e60 0x7f2af01f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af54b8640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad8003af0 0x7f2af014ae50 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f2ae801c930 tx=0x7f2ae80a5bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2aedffb640 1 -- 172.21.15.131:0/906637986 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8006030 con 0x7f2ad8003af0 2024-09-17T02:52:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2aedffb640 1 -- 172.21.15.131:0/906637986 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ae80a9070 con 0x7f2ad8003af0 2024-09-17T02:52:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af771c640 1 -- 172.21.15.131:0/906637986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2af0153ef0 con 0x7f2ad8003af0 2024-09-17T02:52:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2aedffb640 1 -- 172.21.15.131:0/906637986 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8003600 con 0x7f2ad8003af0 2024-09-17T02:52:33.393 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af771c640 1 -- 172.21.15.131:0/906637986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2af01544b0 con 0x7f2ad8003af0 2024-09-17T02:52:33.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af771c640 1 -- 172.21.15.131:0/906637986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2af00cce40 con 0x7f2ad8003af0 2024-09-17T02:52:33.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2aedffb640 1 -- 172.21.15.131:0/906637986 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2ae80037a0 con 0x7f2ad8003af0 2024-09-17T02:52:33.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2aedffb640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2adc073230 0x7f2adc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2aedffb640 1 -- 172.21.15.131:0/906637986 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f2ae812e0b0 con 0x7f2ad8003af0 2024-09-17T02:52:33.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.397+0000 7f2af4cb7640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2adc073230 0x7f2adc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.394 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.401+0000 7f2af4cb7640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2adc073230 0x7f2adc075740 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2af015ffd0 tx=0x7f2ae4009880 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:33.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.405+0000 7f2aedffb640 1 -- 172.21.15.131:0/906637986 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2af00c8270 con 0x7f2ad8003af0 2024-09-17T02:52:33.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.601+0000 7f2af771c640 1 -- 172.21.15.131:0/906637986 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2af014b9d0 con 0x7f2ad8003af0 2024-09-17T02:52:33.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.601+0000 7f2aedffb640 1 -- 172.21.15.131:0/906637986 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5239 (secure 0 0 0) 0x7f2ae80b5070 con 0x7f2ad8003af0 2024-09-17T02:52:33.598 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:33.598 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":107,"btime":"2024-09-17T02:52:20:947699+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:52:20.947683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8712},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8712":{"gid":8712,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/2981077586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2981077586},{"type":"v1","addr":"172.21.15.131:6835","nonce":2981077586}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8712,"qdb_cluster":[8712]},"id":8}]} 2024-09-17T02:52:33.598 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 107 2024-09-17T02:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 -- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2adc073230 msgr2=0x7f2adc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2adc073230 0x7f2adc075740 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2af015ffd0 tx=0x7f2ae4009880 comp rx=0 tx=0).stop 2024-09-17T02:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 -- 172.21.15.131:0/906637986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad8003af0 msgr2=0x7f2af014ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad8003af0 0x7f2af014ae50 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f2ae801c930 tx=0x7f2ae80a5bf0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 -- 172.21.15.131:0/906637986 shutdown_connections 2024-09-17T02:52:33.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2adc073230 0x7f2adc075740 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2af00c7e60 0x7f2af01f9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2af01535a0 0x7f2af00c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 --2- 172.21.15.131:0/906637986 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ad8003af0 0x7f2af014ae50 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 -- 172.21.15.131:0/906637986 >> 172.21.15.131:0/906637986 conn(0x7f2af006d440 msgr2=0x7f2af0068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:33.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 -- 172.21.15.131:0/906637986 shutdown_connections 2024-09-17T02:52:33.601 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.605+0000 7f2af771c640 1 -- 172.21.15.131:0/906637986 wait complete. 2024-09-17T02:52:33.619 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-09-17T02:52:33.794 INFO:teuthology.orchestra.run.smithi131.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-09-17T02:52:33.794 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-09-17T02:52:33.795 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-09-17T02:52:33.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 Processor -- start 2024-09-17T02:52:33.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 -- start start 2024-09-17T02:52:33.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ac71d29dd0 0x55ac71d58580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac71d2e110 0x55ac71e08da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 0x55ac71d571d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ac71c7aa00 con 0x55ac71e00330 2024-09-17T02:52:33.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ac71cd8f40 con 0x55ac71d2e110 2024-09-17T02:52:33.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55ac71e02400 con 0x55ac71d29dd0 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5550c0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 0x55ac71d571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5550c0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 0x55ac71d571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40512/0 (socket says 172.21.15.131:40512) 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5550c0640 1 -- 172.21.15.131:0/1959435168 learned_addr learned my addr 172.21.15.131:0/1959435168 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5540be640 1 --2- 172.21.15.131:0/1959435168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac71d2e110 0x55ac71e08da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5548bf640 1 --2- 172.21.15.131:0/1959435168 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ac71d29dd0 0x55ac71d58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5550c0640 1 -- 172.21.15.131:0/1959435168 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ac71d29dd0 msgr2=0x55ac71d58580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5550c0640 1 --2- 172.21.15.131:0/1959435168 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ac71d29dd0 0x55ac71d58580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5550c0640 1 -- 172.21.15.131:0/1959435168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac71d2e110 msgr2=0x55ac71e08da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5550c0640 1 --2- 172.21.15.131:0/1959435168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac71d2e110 0x55ac71e08da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5550c0640 1 -- 172.21.15.131:0/1959435168 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac71c53c10 con 0x55ac71e00330 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5550c0640 1 --2- 172.21.15.131:0/1959435168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 0x55ac71d571d0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fc54c001e10 tx=0x7fc54c00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e58753a6997579 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5530bc640 1 -- 172.21.15.131:0/1959435168 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c00e840 con 0x55ac71e00330 2024-09-17T02:52:33.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5530bc640 1 -- 172.21.15.131:0/1959435168 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc54c00e9e0 con 0x55ac71e00330 2024-09-17T02:52:33.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc5530bc640 1 -- 172.21.15.131:0/1959435168 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c00ee10 con 0x55ac71e00330 2024-09-17T02:52:33.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 -- 172.21.15.131:0/1959435168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 msgr2=0x55ac71d571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 --2- 172.21.15.131:0/1959435168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 0x55ac71d571d0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fc54c001e10 tx=0x7fc54c00b740 comp rx=0 tx=0).stop 2024-09-17T02:52:33.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 -- 172.21.15.131:0/1959435168 shutdown_connections 2024-09-17T02:52:33.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 --2- 172.21.15.131:0/1959435168 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 0x55ac71d571d0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 --2- 172.21.15.131:0/1959435168 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac71d2e110 0x55ac71e08da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 --2- 172.21.15.131:0/1959435168 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ac71d29dd0 0x55ac71d58580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 -- 172.21.15.131:0/1959435168 >> 172.21.15.131:0/1959435168 conn(0x55ac71d18d40 msgr2=0x55ac71e05da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:33.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 -- 172.21.15.131:0/1959435168 shutdown_connections 2024-09-17T02:52:33.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.869+0000 7fc55ade8080 1 -- 172.21.15.131:0/1959435168 wait complete. 2024-09-17T02:52:33.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc55ade8080 1 Processor -- start 2024-09-17T02:52:33.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc55ade8080 1 -- start start 2024-09-17T02:52:33.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc55ade8080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac71d29dd0 0x55ac71dfd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc55ade8080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ac71d2e110 0x55ac71de59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc55ade8080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 0x55ac71dfb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc55ade8080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55ac71e083c0 con 0x55ac71e00330 2024-09-17T02:52:33.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc55ade8080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55ac71e0ab20 con 0x55ac71d29dd0 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc55ade8080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55ac71e08660 con 0x55ac71d2e110 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5550c0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 0x55ac71dfb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5550c0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 0x55ac71dfb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40522/0 (socket says 172.21.15.131:40522) 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5550c0640 1 -- 172.21.15.131:0/4135427348 learned_addr learned my addr 172.21.15.131:0/4135427348 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5548bf640 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac71d29dd0 0x55ac71dfd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5540be640 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ac71d2e110 0x55ac71de59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5550c0640 1 -- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ac71d2e110 msgr2=0x55ac71de59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5550c0640 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55ac71d2e110 0x55ac71de59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5550c0640 1 -- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac71d29dd0 msgr2=0x55ac71dfd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5550c0640 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55ac71d29dd0 0x55ac71dfd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5550c0640 1 -- 172.21.15.131:0/4135427348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac71de8870 con 0x55ac71e00330 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5550c0640 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 0x55ac71dfb060 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x55ac71e0a050 tx=0x7fc54c017630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:33.873 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5520ba640 1 -- 172.21.15.131:0/4135427348 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c027070 con 0x55ac71e00330 2024-09-17T02:52:33.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5520ba640 1 -- 172.21.15.131:0/4135427348 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc54c0177a0 con 0x55ac71e00330 2024-09-17T02:52:33.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc55ade8080 1 -- 172.21.15.131:0/4135427348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ac71de64e0 con 0x55ac71e00330 2024-09-17T02:52:33.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5520ba640 1 -- 172.21.15.131:0/4135427348 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c017c30 con 0x55ac71e00330 2024-09-17T02:52:33.874 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc55ade8080 1 -- 172.21.15.131:0/4135427348 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac71de6a20 con 0x55ac71e00330 2024-09-17T02:52:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.873+0000 7fc5520ba640 1 -- 172.21.15.131:0/4135427348 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc54c02a070 con 0x55ac71e00330 2024-09-17T02:52:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc5520ba640 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc534071be0 0x7fc5340740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc5520ba640 1 -- 172.21.15.131:0/4135427348 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7fc54c0948e0 con 0x55ac71e00330 2024-09-17T02:52:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55ac71dfbab0 0x55ac71de7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc5548bf640 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc534071be0 0x7fc5340740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 -- 172.21.15.131:0/4135427348 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55ac71de7940 con 0x55ac71dfbab0 2024-09-17T02:52:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc5540be640 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55ac71dfbab0 0x55ac71de7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc5548bf640 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc534071be0 0x7fc5340740f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc54400f940 tx=0x7fc544004c10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc5540be640 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55ac71dfbab0 0x55ac71de7400 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc5540be640 1 -- 172.21.15.131:0/4135427348 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7fc53c0acaa0 con 0x55ac71dfbab0 2024-09-17T02:52:33.876 INFO:teuthology.orchestra.run.smithi131.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-09-17T02:52:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 -- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55ac71dfbab0 msgr2=0x55ac71de7400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55ac71dfbab0 0x55ac71de7400 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 -- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc534071be0 msgr2=0x7fc5340740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc534071be0 0x7fc5340740f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc54400f940 tx=0x7fc544004c10 comp rx=0 tx=0).stop 2024-09-17T02:52:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 -- 172.21.15.131:0/4135427348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 msgr2=0x55ac71dfb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 --2- 172.21.15.131:0/4135427348 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55ac71e00330 0x55ac71dfb060 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x55ac71e0a050 tx=0x7fc54c017630 comp rx=0 tx=0).stop 2024-09-17T02:52:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc5550c0640 1 -- 172.21.15.131:0/4135427348 reap_dead start 2024-09-17T02:52:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 -- 172.21.15.131:0/4135427348 shutdown_connections 2024-09-17T02:52:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 -- 172.21.15.131:0/4135427348 >> 172.21.15.131:0/4135427348 conn(0x55ac71d18d40 msgr2=0x55ac71e02ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 -- 172.21.15.131:0/4135427348 shutdown_connections 2024-09-17T02:52:33.877 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.877+0000 7fc55ade8080 1 -- 172.21.15.131:0/4135427348 wait complete. 2024-09-17T02:52:33.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:52:33.879 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-09-17T02:52:33.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba12a5a080 1 Processor -- start 2024-09-17T02:52:33.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba12a5a080 1 -- start start 2024-09-17T02:52:33.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba12a5a080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ce8272e20 0x558ce81fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba12a5a080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558ce828a940 0x558ce81b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba12a5a080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828ad50 0x558ce81b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba12a5a080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558ce8103a00 con 0x558ce828ad50 2024-09-17T02:52:33.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba12a5a080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558ce8161f40 con 0x558ce8272e20 2024-09-17T02:52:33.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba12a5a080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x558ce8273440 con 0x558ce828a940 2024-09-17T02:52:33.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba0cd32640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828ad50 0x558ce81b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba0cd32640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828ad50 0x558ce81b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40548/0 (socket says 172.21.15.131:40548) 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba0cd32640 1 -- 172.21.15.131:0/790552615 learned_addr learned my addr 172.21.15.131:0/790552615 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba0cd32640 1 -- 172.21.15.131:0/790552615 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558ce828a940 msgr2=0x558ce81b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba0cd32640 1 --2- 172.21.15.131:0/790552615 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558ce828a940 0x558ce81b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba0cd32640 1 -- 172.21.15.131:0/790552615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ce8272e20 msgr2=0x558ce81fdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba0cd32640 1 --2- 172.21.15.131:0/790552615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ce8272e20 0x558ce81fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.941+0000 7fba0cd32640 1 -- 172.21.15.131:0/790552615 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce80dcc10 con 0x558ce828ad50 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0cd32640 1 --2- 172.21.15.131:0/790552615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828ad50 0x558ce81b4420 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c4e0 tx=0x7fb9fc00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb201ece2723b30 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0ad2e640 1 -- 172.21.15.131:0/790552615 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc00a9f0 con 0x558ce828ad50 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0ad2e640 1 -- 172.21.15.131:0/790552615 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9fc00ab90 con 0x558ce828ad50 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0ad2e640 1 -- 172.21.15.131:0/790552615 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc008430 con 0x558ce828ad50 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 -- 172.21.15.131:0/790552615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828ad50 msgr2=0x558ce81b4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 --2- 172.21.15.131:0/790552615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828ad50 0x558ce81b4420 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c4e0 tx=0x7fb9fc00c9a0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 -- 172.21.15.131:0/790552615 shutdown_connections 2024-09-17T02:52:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 --2- 172.21.15.131:0/790552615 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828ad50 0x558ce81b4420 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 --2- 172.21.15.131:0/790552615 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558ce828a940 0x558ce81b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 --2- 172.21.15.131:0/790552615 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ce8272e20 0x558ce81fdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 -- 172.21.15.131:0/790552615 >> 172.21.15.131:0/790552615 conn(0x558ce81a1d40 msgr2=0x558ce81f74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 -- 172.21.15.131:0/790552615 shutdown_connections 2024-09-17T02:52:33.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 -- 172.21.15.131:0/790552615 wait complete. 2024-09-17T02:52:33.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 Processor -- start 2024-09-17T02:52:33.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 -- start start 2024-09-17T02:52:33.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558ce8272e20 0x558ce8288700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828a940 0x558ce826ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ce828ad50 0x558ce81fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558ce826fd80 con 0x558ce828a940 2024-09-17T02:52:33.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558ce81d8040 con 0x558ce828ad50 2024-09-17T02:52:33.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba12a5a080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x558ce81d82e0 con 0x558ce8272e20 2024-09-17T02:52:33.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0bd30640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828a940 0x558ce826ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0bd30640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828a940 0x558ce826ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40564/0 (socket says 172.21.15.131:40564) 2024-09-17T02:52:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0bd30640 1 -- 172.21.15.131:0/1746221682 learned_addr learned my addr 172.21.15.131:0/1746221682 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0c531640 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558ce8272e20 0x558ce8288700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0cd32640 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ce828ad50 0x558ce81fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0bd30640 1 -- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558ce8272e20 msgr2=0x558ce8288700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0bd30640 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558ce8272e20 0x558ce8288700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0bd30640 1 -- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ce828ad50 msgr2=0x558ce81fdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0bd30640 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558ce828ad50 0x558ce81fdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0bd30640 1 -- 172.21.15.131:0/1746221682 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce81d8580 con 0x558ce828a940 2024-09-17T02:52:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0c531640 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558ce8272e20 0x558ce8288700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.945+0000 7fba0bd30640 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828a940 0x558ce826ef00 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fba0001c180 tx=0x7fba000a76d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba09d2c640 1 -- 172.21.15.131:0/1746221682 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000b0680 con 0x558ce828a940 2024-09-17T02:52:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba09d2c640 1 -- 172.21.15.131:0/1746221682 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba000b0840 con 0x558ce828a940 2024-09-17T02:52:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba12a5a080 1 -- 172.21.15.131:0/1746221682 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558ce81d8820 con 0x558ce828a940 2024-09-17T02:52:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba09d2c640 1 -- 172.21.15.131:0/1746221682 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000b0c90 con 0x558ce828a940 2024-09-17T02:52:33.945 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba12a5a080 1 -- 172.21.15.131:0/1746221682 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce81d8d60 con 0x558ce828a940 2024-09-17T02:52:33.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba09d2c640 1 -- 172.21.15.131:0/1746221682 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fba000bb420 con 0x558ce828a940 2024-09-17T02:52:33.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba09d2c640 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9f8071be0 0x7fb9f80740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba09d2c640 1 -- 172.21.15.131:0/1746221682 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7fba00130aa0 con 0x558ce828a940 2024-09-17T02:52:33.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba12a5a080 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x558ce8289810 0x558ce8289c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:33.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba12a5a080 1 -- 172.21.15.131:0/1746221682 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x558ce828a160 con 0x558ce8289810 2024-09-17T02:52:33.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba0c531640 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9f8071be0 0x7fb9f80740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba0cd32640 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x558ce8289810 0x558ce8289c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:33.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba0c531640 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9f8071be0 0x7fb9f80740f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fba04005fd0 tx=0x7fba04007040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:33.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.949+0000 7fba0cd32640 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x558ce8289810 0x558ce8289c20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba0cd32640 1 -- 172.21.15.131:0/1746221682 <== osd.6 v2:172.21.15.189:6802/952142225 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fb9fc00a8f0 con 0x558ce8289810 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba12a5a080 1 -- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x558ce8289810 msgr2=0x558ce8289c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba12a5a080 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x558ce8289810 0x558ce8289c20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba12a5a080 1 -- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9f8071be0 msgr2=0x7fb9f80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba12a5a080 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9f8071be0 0x7fb9f80740f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fba04005fd0 tx=0x7fba04007040 comp rx=0 tx=0).stop 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba12a5a080 1 -- 172.21.15.131:0/1746221682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828a940 msgr2=0x558ce826ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba12a5a080 1 --2- 172.21.15.131:0/1746221682 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558ce828a940 0x558ce826ef00 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fba0001c180 tx=0x7fba000a76d0 comp rx=0 tx=0).stop 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba0cd32640 1 -- 172.21.15.131:0/1746221682 reap_dead start 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba12a5a080 1 -- 172.21.15.131:0/1746221682 shutdown_connections 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba12a5a080 1 -- 172.21.15.131:0/1746221682 >> 172.21.15.131:0/1746221682 conn(0x558ce81a1d40 msgr2=0x558ce81e24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba12a5a080 1 -- 172.21.15.131:0/1746221682 shutdown_connections 2024-09-17T02:52:33.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:33.953+0000 7fba12a5a080 1 -- 172.21.15.131:0/1746221682 wait complete. 2024-09-17T02:52:33.952 INFO:teuthology.orchestra.run.smithi131.stdout:mytag123 2024-09-17T02:52:33.952 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-09-17T02:52:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 Processor -- start 2024-09-17T02:52:34.011 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 -- start start 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55aca224f8f0 0x55aca230e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca229abe0 0x55aca2297b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aca230ff40 0x55aca2298070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55aca219ea00 con 0x55aca229abe0 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aca21fcf40 con 0x55aca230ff40 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55aca2327a20 con 0x55aca224f8f0 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc169d2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca229abe0 0x55aca2297b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc169d2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca229abe0 0x55aca2297b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40580/0 (socket says 172.21.15.131:40580) 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc169d2640 1 -- 172.21.15.131:0/2285051281 learned_addr learned my addr 172.21.15.131:0/2285051281 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc179d4640 1 --2- 172.21.15.131:0/2285051281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aca230ff40 0x55aca2298070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc169d2640 1 -- 172.21.15.131:0/2285051281 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55aca224f8f0 msgr2=0x55aca230e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.014 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc169d2640 1 --2- 172.21.15.131:0/2285051281 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55aca224f8f0 0x55aca230e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc169d2640 1 -- 172.21.15.131:0/2285051281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aca230ff40 msgr2=0x55aca2298070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc169d2640 1 --2- 172.21.15.131:0/2285051281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aca230ff40 0x55aca2298070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc169d2640 1 -- 172.21.15.131:0/2285051281 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca2177c10 con 0x55aca229abe0 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc169d2640 1 --2- 172.21.15.131:0/2285051281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca229abe0 0x55aca2297b30 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00b580 tx=0x7fcc0c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d4561dad74a2e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc159d0640 1 -- 172.21.15.131:0/2285051281 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c018070 con 0x55aca229abe0 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc159d0640 1 -- 172.21.15.131:0/2285051281 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc0c009860 con 0x55aca229abe0 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc159d0640 1 -- 172.21.15.131:0/2285051281 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c009c70 con 0x55aca229abe0 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2285051281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca229abe0 msgr2=0x55aca2297b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 --2- 172.21.15.131:0/2285051281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca229abe0 0x55aca2297b30 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c00b580 tx=0x7fcc0c00ba40 comp rx=0 tx=0).stop 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2285051281 shutdown_connections 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 --2- 172.21.15.131:0/2285051281 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aca230ff40 0x55aca2298070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 --2- 172.21.15.131:0/2285051281 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca229abe0 0x55aca2297b30 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 --2- 172.21.15.131:0/2285051281 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55aca224f8f0 0x55aca230e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2285051281 >> 172.21.15.131:0/2285051281 conn(0x55aca223cd40 msgr2=0x55aca22951d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2285051281 shutdown_connections 2024-09-17T02:52:34.015 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.017+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2285051281 wait complete. 2024-09-17T02:52:34.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc1d6fc080 1 Processor -- start 2024-09-17T02:52:34.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc1d6fc080 1 -- start start 2024-09-17T02:52:34.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc1d6fc080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aca224f8f0 0x55aca2307d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc1d6fc080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55aca229abe0 0x55aca22f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc1d6fc080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca230ff40 0x55aca2295220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc1d6fc080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55aca22959f0 con 0x55aca230ff40 2024-09-17T02:52:34.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc1d6fc080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55aca2297660 con 0x55aca224f8f0 2024-09-17T02:52:34.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc1d6fc080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55aca22962d0 con 0x55aca229abe0 2024-09-17T02:52:34.019 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc179d4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca230ff40 0x55aca2295220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc179d4640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca230ff40 0x55aca2295220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40592/0 (socket says 172.21.15.131:40592) 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc179d4640 1 -- 172.21.15.131:0/2315199069 learned_addr learned my addr 172.21.15.131:0/2315199069 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc169d2640 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55aca229abe0 0x55aca22f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc171d3640 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aca224f8f0 0x55aca2307d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc179d4640 1 -- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55aca229abe0 msgr2=0x55aca22f9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc179d4640 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55aca229abe0 0x55aca22f9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc179d4640 1 -- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aca224f8f0 msgr2=0x55aca2307d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc179d4640 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aca224f8f0 0x55aca2307d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc179d4640 1 -- 172.21.15.131:0/2315199069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aca2280200 con 0x55aca230ff40 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc171d3640 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55aca224f8f0 0x55aca2307d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc179d4640 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca230ff40 0x55aca2295220 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fcc1001b370 tx=0x7fcc10004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc149ce640 1 -- 172.21.15.131:0/2315199069 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc100bb020 con 0x55aca230ff40 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc149ce640 1 -- 172.21.15.131:0/2315199069 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc100b1040 con 0x55aca230ff40 2024-09-17T02:52:34.020 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc149ce640 1 -- 172.21.15.131:0/2315199069 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc100b0070 con 0x55aca230ff40 2024-09-17T02:52:34.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2315199069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aca22804a0 con 0x55aca230ff40 2024-09-17T02:52:34.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.021+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2315199069 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aca22809e0 con 0x55aca230ff40 2024-09-17T02:52:34.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc149ce640 1 -- 172.21.15.131:0/2315199069 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcc100be020 con 0x55aca230ff40 2024-09-17T02:52:34.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc149ce640 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc04071be0 0x7fcc040740f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc149ce640 1 -- 172.21.15.131:0/2315199069 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7fcc1012e140 con 0x55aca230ff40 2024-09-17T02:52:34.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55aca2308f50 0x55aca2322490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2315199069 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x55aca2281120 con 0x55aca2308f50 2024-09-17T02:52:34.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc171d3640 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc04071be0 0x7fcc040740f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.021 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc169d2640 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55aca2308f50 0x55aca2322490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc169d2640 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55aca2308f50 0x55aca2322490 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.022 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc171d3640 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc04071be0 0x7fcc040740f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800b450 tx=0x7fcc08005560 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc169d2640 1 -- 172.21.15.131:0/2315199069 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fcc0c01b040 con 0x55aca2308f50 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2315199069 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55aca2308f50 msgr2=0x55aca2322490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55aca2308f50 0x55aca2322490 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc04071be0 msgr2=0x7fcc040740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcc04071be0 0x7fcc040740f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800b450 tx=0x7fcc08005560 comp rx=0 tx=0).stop 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2315199069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca230ff40 msgr2=0x55aca2295220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 --2- 172.21.15.131:0/2315199069 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55aca230ff40 0x55aca2295220 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fcc1001b370 tx=0x7fcc10004570 comp rx=0 tx=0).stop 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc179d4640 1 -- 172.21.15.131:0/2315199069 reap_dead start 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2315199069 shutdown_connections 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2315199069 >> 172.21.15.131:0/2315199069 conn(0x55aca223cd40 msgr2=0x55aca2299f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2315199069 shutdown_connections 2024-09-17T02:52:34.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.025+0000 7fcc1d6fc080 1 -- 172.21.15.131:0/2315199069 wait complete. 2024-09-17T02:52:34.026 INFO:teuthology.orchestra.run.smithi131.stdout:mytag123 2024-09-17T02:52:34.027 DEBUG:teuthology.orchestra.run.smithi131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-09-17T02:52:34.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 Processor -- start 2024-09-17T02:52:34.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 -- start start 2024-09-17T02:52:34.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 0x560e2698ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e26a1c6e0 0x560e26acc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e26a1fb50 0x560e26acc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560e268b1a00 con 0x560e26965650 2024-09-17T02:52:34.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e2690ff40 con 0x560e26a1fb50 2024-09-17T02:52:34.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x560e26a1cd00 con 0x560e26a1c6e0 2024-09-17T02:52:34.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20fd78b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 0x560e2698ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20fd78b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 0x560e2698ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40602/0 (socket says 172.21.15.131:40602) 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20fd78b640 1 -- 172.21.15.131:0/1356945714 learned_addr learned my addr 172.21.15.131:0/1356945714 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20fcf8a640 1 --2- 172.21.15.131:0/1356945714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e26a1c6e0 0x560e26acc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20fdf8c640 1 --2- 172.21.15.131:0/1356945714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e26a1fb50 0x560e26acc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20fd78b640 1 -- 172.21.15.131:0/1356945714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e26a1c6e0 msgr2=0x560e26acc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20fd78b640 1 --2- 172.21.15.131:0/1356945714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e26a1c6e0 0x560e26acc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20fd78b640 1 -- 172.21.15.131:0/1356945714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e26a1fb50 msgr2=0x560e26acc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20fd78b640 1 --2- 172.21.15.131:0/1356945714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e26a1fb50 0x560e26acc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20fd78b640 1 -- 172.21.15.131:0/1356945714 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e2688ac10 con 0x560e26965650 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20fd78b640 1 --2- 172.21.15.131:0/1356945714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 0x560e2698ee80 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f20f0006f10 tx=0x7f20f000b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c4ee8586fbfc84 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20ef7fe640 1 -- 172.21.15.131:0/1356945714 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f000e840 con 0x560e26965650 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20ef7fe640 1 -- 172.21.15.131:0/1356945714 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20f000ead0 con 0x560e26965650 2024-09-17T02:52:34.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f20ef7fe640 1 -- 172.21.15.131:0/1356945714 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f000d390 con 0x560e26965650 2024-09-17T02:52:34.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 -- 172.21.15.131:0/1356945714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 msgr2=0x560e2698ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 --2- 172.21.15.131:0/1356945714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 0x560e2698ee80 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f20f0006f10 tx=0x7f20f000b680 comp rx=0 tx=0).stop 2024-09-17T02:52:34.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 -- 172.21.15.131:0/1356945714 shutdown_connections 2024-09-17T02:52:34.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 --2- 172.21.15.131:0/1356945714 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e26a1fb50 0x560e26acc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 --2- 172.21.15.131:0/1356945714 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e26a1c6e0 0x560e26acc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 --2- 172.21.15.131:0/1356945714 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 0x560e2698ee80 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 -- 172.21.15.131:0/1356945714 >> 172.21.15.131:0/1356945714 conn(0x560e2694fd40 msgr2=0x560e26964ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 -- 172.21.15.131:0/1356945714 shutdown_connections 2024-09-17T02:52:34.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.093+0000 7f2103cb4080 1 -- 172.21.15.131:0/1356945714 wait complete. 2024-09-17T02:52:34.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f2103cb4080 1 Processor -- start 2024-09-17T02:52:34.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f2103cb4080 1 -- start start 2024-09-17T02:52:34.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f2103cb4080 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 0x560e26a1ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f2103cb4080 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e26a1c6e0 0x560e2698ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f2103cb4080 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e26a1fb50 0x560e2698f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f2103cb4080 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x560e26991790 con 0x560e26965650 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f2103cb4080 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x560e269914f0 con 0x560e26a1fb50 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f2103cb4080 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x560e26991a30 con 0x560e26a1c6e0 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20fd78b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 0x560e26a1ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20fd78b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 0x560e26a1ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40606/0 (socket says 172.21.15.131:40606) 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20fd78b640 1 -- 172.21.15.131:0/2445773754 learned_addr learned my addr 172.21.15.131:0/2445773754 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20fdf8c640 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e26a1fb50 0x560e2698f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20fcf8a640 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e26a1c6e0 0x560e2698ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20fd78b640 1 -- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e26a1c6e0 msgr2=0x560e2698ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20fd78b640 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x560e26a1c6e0 0x560e2698ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20fd78b640 1 -- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e26a1fb50 msgr2=0x560e2698f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20fd78b640 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x560e26a1fb50 0x560e2698f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20fd78b640 1 -- 172.21.15.131:0/2445773754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e26ad4430 con 0x560e26965650 2024-09-17T02:52:34.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20fd78b640 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 0x560e26a1ae40 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f20f0017620 tx=0x7f20f0017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20ee7fc640 1 -- 172.21.15.131:0/2445773754 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0025070 con 0x560e26965650 2024-09-17T02:52:34.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20ee7fc640 1 -- 172.21.15.131:0/2445773754 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20f0017980 con 0x560e26965650 2024-09-17T02:52:34.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f2103cb4080 1 -- 172.21.15.131:0/2445773754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560e26ad1420 con 0x560e26965650 2024-09-17T02:52:34.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f20ee7fc640 1 -- 172.21.15.131:0/2445773754 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f0017dd0 con 0x560e26965650 2024-09-17T02:52:34.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.097+0000 7f2103cb4080 1 -- 172.21.15.131:0/2445773754 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e26ad1960 con 0x560e26965650 2024-09-17T02:52:34.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f20ee7fc640 1 -- 172.21.15.131:0/2445773754 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f20f0028070 con 0x560e26965650 2024-09-17T02:52:34.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f20ee7fc640 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f20e0071c10 0x7f20e0074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f20ee7fc640 1 -- 172.21.15.131:0/2445773754 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f20f0094290 con 0x560e26965650 2024-09-17T02:52:34.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f20fcf8a640 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f20e0071c10 0x7f20e0074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x560e26ad1e00 0x560e26ad2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 -- 172.21.15.131:0/2445773754 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) -- 0x560e26ad2750 con 0x560e26ad1e00 2024-09-17T02:52:34.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f20fdf8c640 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x560e26ad1e00 0x560e26ad2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f20fcf8a640 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f20e0071c10 0x7f20e0074120 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f20f40041c0 tx=0x7f20f4007bb0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f20fdf8c640 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x560e26ad1e00 0x560e26ad2210 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f20fdf8c640 1 -- 172.21.15.131:0/2445773754 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f20f8002050 con 0x560e26ad1e00 2024-09-17T02:52:34.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 -- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x560e26ad1e00 msgr2=0x560e26ad2210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x560e26ad1e00 0x560e26ad2210 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 -- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f20e0071c10 msgr2=0x7f20e0074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f20e0071c10 0x7f20e0074120 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f20f40041c0 tx=0x7f20f4007bb0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 -- 172.21.15.131:0/2445773754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 msgr2=0x560e26a1ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 --2- 172.21.15.131:0/2445773754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x560e26965650 0x560e26a1ae40 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f20f0017620 tx=0x7f20f0017650 comp rx=0 tx=0).stop 2024-09-17T02:52:34.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f20fdf8c640 1 -- 172.21.15.131:0/2445773754 reap_dead start 2024-09-17T02:52:34.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 -- 172.21.15.131:0/2445773754 shutdown_connections 2024-09-17T02:52:34.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 -- 172.21.15.131:0/2445773754 >> 172.21.15.131:0/2445773754 conn(0x560e2694fd40 msgr2=0x560e269649c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 -- 172.21.15.131:0/2445773754 shutdown_connections 2024-09-17T02:52:34.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.101+0000 7f2103cb4080 1 -- 172.21.15.131:0/2445773754 wait complete. 2024-09-17T02:52:34.103 INFO:teuthology.orchestra.run.smithi131.stdout:mytag123 2024-09-17T02:52:34.104 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-09-17T02:52:34.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.257+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/206330058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec806eeb0 msgr2=0x7f7ec806f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.257+0000 7f7ecdfbf640 1 --2- 172.21.15.131:0/206330058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec806eeb0 0x7f7ec806f2c0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009840 tx=0x7f7ebc01c820 comp rx=0 tx=0).stop 2024-09-17T02:52:34.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.257+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/206330058 shutdown_connections 2024-09-17T02:52:34.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.257+0000 7f7ecdfbf640 1 --2- 172.21.15.131:0/206330058 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ec80cdc40 0x7f7ec80ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.257+0000 7f7ecdfbf640 1 --2- 172.21.15.131:0/206330058 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec806eeb0 0x7f7ec806f2c0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.257+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/206330058 >> 172.21.15.131:0/206330058 conn(0x7f7ec80b1c40 msgr2=0x7f7ec80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.257+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/206330058 shutdown_connections 2024-09-17T02:52:34.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.257+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/206330058 wait complete. 2024-09-17T02:52:34.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ecdfbf640 1 Processor -- start 2024-09-17T02:52:34.254 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ecdfbf640 1 -- start start 2024-09-17T02:52:34.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ecdfbf640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ec806eeb0 0x7f7ec8151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ecdfbf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec80cdc40 0x7f7ec8155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ecdfbf640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec815ac00 0x7f7ec8156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ecdfbf640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec8168db0 con 0x7f7ec80cdc40 2024-09-17T02:52:34.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ecdfbf640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec8168c30 con 0x7f7ec815ac00 2024-09-17T02:52:34.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ecdfbf640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec8168f30 con 0x7f7ec806eeb0 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ec6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec80cdc40 0x7f7ec8155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ec6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec80cdc40 0x7f7ec8155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40624/0 (socket says 172.21.15.131:40624) 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ec6ffd640 1 -- 172.21.15.131:0/853746456 learned_addr learned my addr 172.21.15.131:0/853746456 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ec6ffd640 1 -- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ec806eeb0 msgr2=0x7f7ec8151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ec806eeb0 0x7f7ec8151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ec7fff640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec815ac00 0x7f7ec8156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ec6ffd640 1 -- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec815ac00 msgr2=0x7f7ec8156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec815ac00 0x7f7ec8156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ec6ffd640 1 -- 172.21.15.131:0/853746456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec815a050 con 0x7f7ec80cdc40 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec80cdc40 0x7f7ec8155f10 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc01aed0 tx=0x7f7ebc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7eaffff640 1 -- 172.21.15.131:0/853746456 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0bc070 con 0x7f7ec80cdc40 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7eaffff640 1 -- 172.21.15.131:0/853746456 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ebc003600 con 0x7f7ec80cdc40 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/853746456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec80adae0 con 0x7f7ec80cdc40 2024-09-17T02:52:34.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7eaffff640 1 -- 172.21.15.131:0/853746456 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0b2040 con 0x7f7ec80cdc40 2024-09-17T02:52:34.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.261+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/853746456 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec81fcf40 con 0x7f7ec80cdc40 2024-09-17T02:52:34.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7eaffff640 1 -- 172.21.15.131:0/853746456 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7ebc0bf070 con 0x7f7ec80cdc40 2024-09-17T02:52:34.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7eaffff640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7eb4073230 0x7f7eb4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ed528a000 1 Processor -- start 2024-09-17T02:52:34.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ed528a000 1 -- start start 2024-09-17T02:52:34.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7eaffff640 1 -- 172.21.15.131:0/853746456 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f7ebc12dd90 con 0x7f7ec80cdc40 2024-09-17T02:52:34.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ec77fe640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7eb4073230 0x7f7eb4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ec77fe640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7eb4073230 0x7f7eb4075740 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8001530 tx=0x7f7eb8007a40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f4e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 0x558c72f4ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72f4f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ed528a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558c72dedcd0 con 0x558c72de26d0 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ed528a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558c72d03330 con 0x558c72afdaa0 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ed528a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x558c72d60bb0 con 0x558c72a39890 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ec77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f4e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ec77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f4e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40634/0 (socket says 172.21.15.131:40634) 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ec77fe640 1 -- 172.21.15.131:0/1590842215 learned_addr learned my addr 172.21.15.131:0/1590842215 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/1590842215 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 0x558c72f4ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ec7fff640 1 --2- 172.21.15.131:0/1590842215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72f4f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ec77fe640 1 -- 172.21.15.131:0/1590842215 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 msgr2=0x558c72f4ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ec77fe640 1 --2- 172.21.15.131:0/1590842215 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 0x558c72f4ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ec77fe640 1 -- 172.21.15.131:0/1590842215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 msgr2=0x558c72f4f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.265+0000 7f7ec77fe640 1 --2- 172.21.15.131:0/1590842215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72f4f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ec77fe640 1 -- 172.21.15.131:0/1590842215 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c72d04c80 con 0x558c72de26d0 2024-09-17T02:52:34.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ec77fe640 1 --2- 172.21.15.131:0/1590842215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f4e9f0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8016840 tx=0x7f7eb800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9181b5ed6f997ff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ead7fa640 1 -- 172.21.15.131:0/1590842215 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8024070 con 0x558c72de26d0 2024-09-17T02:52:34.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ead7fa640 1 -- 172.21.15.131:0/1590842215 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7eb80162e0 con 0x558c72de26d0 2024-09-17T02:52:34.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ead7fa640 1 -- 172.21.15.131:0/1590842215 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8013b10 con 0x558c72de26d0 2024-09-17T02:52:34.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 -- 172.21.15.131:0/1590842215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 msgr2=0x558c72f4e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 --2- 172.21.15.131:0/1590842215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f4e9f0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8016840 tx=0x7f7eb800f000 comp rx=0 tx=0).stop 2024-09-17T02:52:34.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 -- 172.21.15.131:0/1590842215 shutdown_connections 2024-09-17T02:52:34.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 --2- 172.21.15.131:0/1590842215 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72f4f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 --2- 172.21.15.131:0/1590842215 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 0x558c72f4ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 --2- 172.21.15.131:0/1590842215 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f4e9f0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 -- 172.21.15.131:0/1590842215 >> 172.21.15.131:0/1590842215 conn(0x558c72a0d5c0 msgr2=0x558c72f0bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 -- 172.21.15.131:0/1590842215 shutdown_connections 2024-09-17T02:52:34.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 -- 172.21.15.131:0/1590842215 wait complete. 2024-09-17T02:52:34.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 -- start start 2024-09-17T02:52:34.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 Processor -- start 2024-09-17T02:52:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 0x558c72f0a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72f0af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558c72d53df0 con 0x558c72de26d0 2024-09-17T02:52:34.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558c72d4ffe0 con 0x558c72afdaa0 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x558c72cf54d0 con 0x558c72a39890 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ed528a000 10 client.0 _ll_register_callbacks cb 0x558c72cd37f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ec7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ec7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40640/0 (socket says 172.21.15.131:40640) 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.269+0000 7f7ec7fff640 1 -- 172.21.15.131:0/2260650870 learned_addr learned my addr 172.21.15.131:0/2260650870 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ec77fe640 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 0x558c72f0a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72f0af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ec7fff640 1 -- 172.21.15.131:0/2260650870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 msgr2=0x558c72f0a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ec7fff640 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 0x558c72f0a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ec7fff640 1 -- 172.21.15.131:0/2260650870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 msgr2=0x558c72f0af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ec7fff640 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72f0af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ec7fff640 1 -- 172.21.15.131:0/2260650870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c72cee210 con 0x558c72de26d0 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72f0af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ec7fff640 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f115f0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000e8b0 tx=0x7f7ec000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38596fe88e1f0fce server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ea67fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ea67fc640 1 -- 172.21.15.131:0/2260650870 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec0018020 con 0x558c72de26d0 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ea67fc640 1 -- 172.21.15.131:0/2260650870 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f7ec000cb80 con 0x558c72de26d0 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 -- 172.21.15.131:0/2260650870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558c72ce8350 con 0x558c72de26d0 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ea67fc640 1 -- 172.21.15.131:0/2260650870 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f7ec0022050 con 0x558c72de26d0 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 10 client.8856 fetch_fsmap learned FSMap version 107 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 -- 172.21.15.131:0/2260650870 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x558c72cd4fc0 con 0x558c72de26d0 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ea67fc640 1 -- 172.21.15.131:0/2260650870 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558c72ce8350 con 0x558c72de26d0 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ea67fc640 1 -- 172.21.15.131:0/2260650870 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 107) ==== 3502+0+0 (secure 0 0 0) 0x7f7ec0025020 con 0x558c72de26d0 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 10 client.8856 fetch_fsmap finished waiting for FSMap version 107 2024-09-17T02:52:34.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 10 client.8856 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 20 client.8856 populate_metadata read hostname 'smithi131' 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x558c729f2930 0x558c72f55e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 4 client.8856 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 -- 172.21.15.131:0/2260650870 --> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c72d62b00 con 0x558c729f2930 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ec77fe640 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x558c729f2930 0x558c72f55e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ec77fe640 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x558c729f2930 0x558c72f55e80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a6a39aafe388fc5 server_cookie=46e811bac028debd in_seq=0 out_seq=0 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ea67fc640 10 client.8856 ms_handle_connect on v2:172.21.15.131:6832/2981077586 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ea67fc640 1 -- 172.21.15.131:0/2260650870 <== mds.0 v2:172.21.15.131:6832/2981077586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c72d62b00 con 0x558c729f2930 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ea67fc640 10 client.8856 handle_command_reply: tid=0 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 client.8856 shutdown 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 -- 172.21.15.131:0/2260650870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 msgr2=0x558c72f115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f115f0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f7ec000e8b0 tx=0x7f7ec000ed70 comp rx=0 tx=0).stop 2024-09-17T02:52:34.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 -- 172.21.15.131:0/2260650870 shutdown_connections 2024-09-17T02:52:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x558c729f2930 0x558c72f55e80 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72de26d0 0x558c72f115f0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ea67fc640 0 client.8856 ms_handle_reset on v2:172.21.15.131:6832/2981077586 2024-09-17T02:52:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72f0af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 --2- 172.21.15.131:0/2260650870 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 0x558c72f0a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 -- 172.21.15.131:0/2260650870 >> 172.21.15.131:0/2260650870 conn(0x558c72a0d5c0 msgr2=0x558c72f03d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 -- 172.21.15.131:0/2260650870 shutdown_connections 2024-09-17T02:52:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 1 -- 172.21.15.131:0/2260650870 wait complete. 2024-09-17T02:52:34.273 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.273+0000 7f7ed528a000 20 client.8856 trim_cache size 0 max 16384 2024-09-17T02:52:34.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ed528a000 1 Processor -- start 2024-09-17T02:52:34.284 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ed528a000 1 -- start start 2024-09-17T02:52:34.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 0x558c72fbffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72afdaa0 0x558c72fc0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72de26d0 0x558c72fc0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ed528a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec8205d60 con 0x558c72afdaa0 2024-09-17T02:52:34.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ed528a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec8204d80 con 0x558c72de26d0 2024-09-17T02:52:34.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ed528a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec8158d70 con 0x558c72a39890 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ec6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72afdaa0 0x558c72fc0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ec6ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72afdaa0 0x558c72fc0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40642/0 (socket says 172.21.15.131:40642) 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ec6ffd640 1 -- 172.21.15.131:0/2269038510 learned_addr learned my addr 172.21.15.131:0/2269038510 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ec6ffd640 1 -- 172.21.15.131:0/2269038510 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 msgr2=0x558c72fbffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/2269038510 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 0x558c72fbffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ec6ffd640 1 -- 172.21.15.131:0/2269038510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72de26d0 msgr2=0x558c72fc0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/2269038510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72de26d0 0x558c72fc0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.289+0000 7f7ec6ffd640 1 -- 172.21.15.131:0/2269038510 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c72dad680 con 0x558c72afdaa0 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/2269038510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72afdaa0 0x558c72fc0520 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0f83c0 tx=0x7f7ebc0f4400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67faac0154b6a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7eacff9640 1 -- 172.21.15.131:0/2269038510 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0f7ba0 con 0x558c72afdaa0 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7eacff9640 1 -- 172.21.15.131:0/2269038510 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ebc0f7f40 con 0x558c72afdaa0 2024-09-17T02:52:34.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7eacff9640 1 -- 172.21.15.131:0/2269038510 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0f8b30 con 0x558c72afdaa0 2024-09-17T02:52:34.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 -- 172.21.15.131:0/2269038510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72afdaa0 msgr2=0x558c72fc0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.288 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 --2- 172.21.15.131:0/2269038510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72afdaa0 0x558c72fc0520 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0f83c0 tx=0x7f7ebc0f4400 comp rx=0 tx=0).stop 2024-09-17T02:52:34.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 -- 172.21.15.131:0/2269038510 shutdown_connections 2024-09-17T02:52:34.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 --2- 172.21.15.131:0/2269038510 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72de26d0 0x558c72fc0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 --2- 172.21.15.131:0/2269038510 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72afdaa0 0x558c72fc0520 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 --2- 172.21.15.131:0/2269038510 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72a39890 0x558c72fbffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 -- 172.21.15.131:0/2269038510 >> 172.21.15.131:0/2269038510 conn(0x558c72a0d5c0 msgr2=0x558c72f60fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 -- 172.21.15.131:0/2269038510 shutdown_connections 2024-09-17T02:52:34.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 -- 172.21.15.131:0/2269038510 wait complete. 2024-09-17T02:52:34.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 -- start start 2024-09-17T02:52:34.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 Processor -- start 2024-09-17T02:52:34.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72a39890 0x558c72fc3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72fc4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72de26d0 0x558c72fc4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x558c72d4aee0 con 0x558c72a39890 2024-09-17T02:52:34.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x558c72d18610 con 0x558c72afdaa0 2024-09-17T02:52:34.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x558c72d53100 con 0x558c72de26d0 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ed528a000 10 client.0 _ll_register_callbacks cb 0x7f7ec0008040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72a39890 0x558c72fc3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72a39890 0x558c72fc3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40652/0 (socket says 172.21.15.131:40652) 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec77fe640 1 -- 172.21.15.131:0/3638020040 learned_addr learned my addr 172.21.15.131:0/3638020040 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72fc4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec77fe640 1 -- 172.21.15.131:0/3638020040 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72de26d0 msgr2=0x558c72fc4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec77fe640 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72de26d0 0x558c72fc4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec77fe640 1 -- 172.21.15.131:0/3638020040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 msgr2=0x558c72fc4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec77fe640 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72fc4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec77fe640 1 -- 172.21.15.131:0/3638020040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c72d1a0b0 con 0x558c72a39890 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72fc4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ec77fe640 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72a39890 0x558c72fc3c00 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007f70 tx=0x7f7eb80162e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f4754988af3a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ea67fc640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.293+0000 7f7ea67fc640 1 -- 172.21.15.131:0/3638020040 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb8024040 con 0x558c72a39890 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ed528a000 1 -- 172.21.15.131:0/3638020040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ebc0f7f40 con 0x558c72a39890 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ea67fc640 1 -- 172.21.15.131:0/3638020040 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f7eb8005200 con 0x558c72a39890 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ea67fc640 1 -- 172.21.15.131:0/3638020040 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f7eb8038050 con 0x558c72a39890 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ed528a000 10 client.8862 fetch_fsmap learned FSMap version 107 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ed528a000 1 -- 172.21.15.131:0/3638020040 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x558c72ce6610 con 0x558c72a39890 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ea67fc640 1 -- 172.21.15.131:0/3638020040 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ebc0f7f40 con 0x558c72a39890 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ea67fc640 1 -- 172.21.15.131:0/3638020040 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 107) ==== 3502+0+0 (secure 0 0 0) 0x7f7eb803b020 con 0x558c72a39890 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ed528a000 10 client.8862 fetch_fsmap finished waiting for FSMap version 107 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ed528a000 10 client.8862 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ed528a000 20 client.8862 populate_metadata read hostname 'smithi131' 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ed528a000 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x558c729e0990 0x558c72f05430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ed528a000 4 client.8862 mds_command: new command op to 8712 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-17T02:52:34.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ed528a000 1 -- 172.21.15.131:0/3638020040 --> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c72d5e5b0 con 0x558c729e0990 2024-09-17T02:52:34.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x558c729e0990 0x558c72f05430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ea67fc640 10 client.8862 ms_handle_connect on v2:172.21.15.131:6832/2981077586 2024-09-17T02:52:34.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.297+0000 7f7ec6ffd640 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x558c729e0990 0x558c72f05430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9908f7c4536394d2 server_cookie=8362151fab2b8bea in_seq=0 out_seq=0 2024-09-17T02:52:34.310 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.313+0000 7f7ea67fc640 1 -- 172.21.15.131:0/3638020040 <== mds.0 v2:172.21.15.131:6832/2981077586 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c72d5e5b0 con 0x558c729e0990 2024-09-17T02:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.313+0000 7f7ea67fc640 10 client.8862 handle_command_reply: tid=0 2024-09-17T02:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.313+0000 7f7ed528a000 1 client.8862 shutdown 2024-09-17T02:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ed528a000 1 -- 172.21.15.131:0/3638020040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72a39890 msgr2=0x558c72fc3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ed528a000 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72a39890 0x558c72fc3c00 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8007f70 tx=0x7f7eb80162e0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ed528a000 1 -- 172.21.15.131:0/3638020040 shutdown_connections 2024-09-17T02:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ed528a000 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.131:6832/2981077586,v1:172.21.15.131:6835/2981077586] conn(0x558c729e0990 0x558c72f05430 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ed528a000 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x558c72de26d0 0x558c72fc4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ed528a000 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x558c72afdaa0 0x558c72fc4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ea67fc640 0 client.8862 ms_handle_reset on v2:172.21.15.131:6832/2981077586 2024-09-17T02:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ed528a000 1 --2- 172.21.15.131:0/3638020040 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x558c72a39890 0x558c72fc3c00 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ed528a000 1 -- 172.21.15.131:0/3638020040 >> 172.21.15.131:0/3638020040 conn(0x558c72a0d5c0 msgr2=0x558c72f105a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ed528a000 1 -- 172.21.15.131:0/3638020040 shutdown_connections 2024-09-17T02:52:34.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ed528a000 1 -- 172.21.15.131:0/3638020040 wait complete. 2024-09-17T02:52:34.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ed528a000 20 client.8862 trim_cache size 0 max 16384 2024-09-17T02:52:34.312 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:34.312 INFO:teuthology.orchestra.run.smithi131.stdout: "message": "", 2024-09-17T02:52:34.312 INFO:teuthology.orchestra.run.smithi131.stdout: "return_code": 0 2024-09-17T02:52:34.312 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:34.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7eb4073230 msgr2=0x7f7eb4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7eb4073230 0x7f7eb4075740 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7eb8001530 tx=0x7f7eb8007a40 comp rx=0 tx=0).stop 2024-09-17T02:52:34.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/853746456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec80cdc40 msgr2=0x7f7ec8155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec80cdc40 0x7f7ec8155f10 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc01aed0 tx=0x7f7ebc004040 comp rx=0 tx=0).stop 2024-09-17T02:52:34.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/853746456 shutdown_connections 2024-09-17T02:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7eb4073230 0x7f7eb4075740 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ec815ac00 0x7f7ec8156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ec80cdc40 0x7f7ec8155f10 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 --2- 172.21.15.131:0/853746456 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ec806eeb0 0x7f7ec8151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/853746456 >> 172.21.15.131:0/853746456 conn(0x7f7ec80b1c40 msgr2=0x7f7ec81557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/853746456 shutdown_connections 2024-09-17T02:52:34.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.317+0000 7f7ecdfbf640 1 -- 172.21.15.131:0/853746456 wait complete. 2024-09-17T02:52:34.330 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:52:34.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.473+0000 7f1f67320640 1 -- 172.21.15.131:0/3524095357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f600d3f40 msgr2=0x7f1f6006e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.473+0000 7f1f67320640 1 --2- 172.21.15.131:0/3524095357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f600d3f40 0x7f1f6006e5d0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009bc0 tx=0x7f1f5801cba0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.473+0000 7f1f67320640 1 -- 172.21.15.131:0/3524095357 shutdown_connections 2024-09-17T02:52:34.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.473+0000 7f1f67320640 1 --2- 172.21.15.131:0/3524095357 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f600d3f40 0x7f1f6006e5d0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.473+0000 7f1f67320640 1 --2- 172.21.15.131:0/3524095357 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1f6015e740 0x7f1f600d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.473+0000 7f1f67320640 1 -- 172.21.15.131:0/3524095357 >> 172.21.15.131:0/3524095357 conn(0x7f1f600b5ee0 msgr2=0x7f1f600b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.473+0000 7f1f67320640 1 -- 172.21.15.131:0/3524095357 shutdown_connections 2024-09-17T02:52:34.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.473+0000 7f1f67320640 1 -- 172.21.15.131:0/3524095357 wait complete. 2024-09-17T02:52:34.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.473+0000 7f1f67320640 1 Processor -- start 2024-09-17T02:52:34.469 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.473+0000 7f1f67320640 1 -- start start 2024-09-17T02:52:34.472 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f67320640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1f600d11e0 0x7f1f6014e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f67320640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f600d3f40 0x7f1f60155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f67320640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f6015e740 0x7f1f60155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f67320640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6016bd30 con 0x7f1f600d3f40 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f67320640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1f600b36e0 con 0x7f1f6015e740 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f67320640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f1f6016beb0 con 0x7f1f600d11e0 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f648bb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f600d3f40 0x7f1f60155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f648bb640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f600d3f40 0x7f1f60155530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40670/0 (socket says 172.21.15.131:40670) 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f648bb640 1 -- 172.21.15.131:0/773839506 learned_addr learned my addr 172.21.15.131:0/773839506 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f658bd640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f6015e740 0x7f1f60155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f650bc640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1f600d11e0 0x7f1f6014e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f648bb640 1 -- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1f600d11e0 msgr2=0x7f1f6014e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f648bb640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1f600d11e0 0x7f1f6014e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f648bb640 1 -- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f6015e740 msgr2=0x7f1f60155a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f648bb640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f6015e740 0x7f1f60155a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f648bb640 1 -- 172.21.15.131:0/773839506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6014ed90 con 0x7f1f600d3f40 2024-09-17T02:52:34.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f648bb640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f600d3f40 0x7f1f60155530 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c430 tx=0x7f1f5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f55ffb640 1 -- 172.21.15.131:0/773839506 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c011790 con 0x7f1f600d3f40 2024-09-17T02:52:34.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f55ffb640 1 -- 172.21.15.131:0/773839506 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f5c017070 con 0x7f1f600d3f40 2024-09-17T02:52:34.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f67320640 1 -- 172.21.15.131:0/773839506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f601537e0 con 0x7f1f600d3f40 2024-09-17T02:52:34.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f55ffb640 1 -- 172.21.15.131:0/773839506 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c011cc0 con 0x7f1f600d3f40 2024-09-17T02:52:34.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f67320640 1 -- 172.21.15.131:0/773839506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f601691f0 con 0x7f1f600d3f40 2024-09-17T02:52:34.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.477+0000 7f1f67320640 1 -- 172.21.15.131:0/773839506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f600d0ca0 con 0x7f1f600d3f40 2024-09-17T02:52:34.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.481+0000 7f1f55ffb640 1 -- 172.21.15.131:0/773839506 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1f5c01c460 con 0x7f1f600d3f40 2024-09-17T02:52:34.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.481+0000 7f1f55ffb640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1f44073200 0x7f1f44075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.481+0000 7f1f55ffb640 1 -- 172.21.15.131:0/773839506 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 6187+0+0 (secure 0 0 0) 0x7f1f5c094ac0 con 0x7f1f600d3f40 2024-09-17T02:52:34.475 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.481+0000 7f1f650bc640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1f44073200 0x7f1f44075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.476 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.481+0000 7f1f650bc640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1f44073200 0x7f1f44075710 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1f500058d0 tx=0x7f1f50005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.485+0000 7f1f55ffb640 1 -- 172.21.15.131:0/773839506 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f1f5c05fb00 con 0x7f1f600d3f40 2024-09-17T02:52:34.667 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.673+0000 7f1f67320640 1 -- 172.21.15.131:0/773839506 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1f600b8820 con 0x7f1f600d3f40 2024-09-17T02:52:34.668 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:52:34.673+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:52:34.671 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.677+0000 7f1f55ffb640 1 -- 172.21.15.131:0/773839506 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) ==== 161+0+0 (secure 0 0 0) 0x7f1f5c05f400 con 0x7f1f600d3f40 2024-09-17T02:52:34.671 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:52:34.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 -- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1f44073200 msgr2=0x7f1f44075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1f44073200 0x7f1f44075710 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1f500058d0 tx=0x7f1f50005860 comp rx=0 tx=0).stop 2024-09-17T02:52:34.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 -- 172.21.15.131:0/773839506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f600d3f40 msgr2=0x7f1f60155530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f600d3f40 0x7f1f60155530 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c00c430 tx=0x7f1f5c00c8f0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.675 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 -- 172.21.15.131:0/773839506 shutdown_connections 2024-09-17T02:52:34.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1f44073200 0x7f1f44075710 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1f6015e740 0x7f1f60155a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1f600d3f40 0x7f1f60155530 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 --2- 172.21.15.131:0/773839506 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1f600d11e0 0x7f1f6014e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 -- 172.21.15.131:0/773839506 >> 172.21.15.131:0/773839506 conn(0x7f1f600b5ee0 msgr2=0x7f1f600b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 -- 172.21.15.131:0/773839506 shutdown_connections 2024-09-17T02:52:34.676 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.681+0000 7f1f67320640 1 -- 172.21.15.131:0/773839506 wait complete. 2024-09-17T02:52:34.678 INFO:tasks.ceph.mds.a.smithi131.stderr: -2641> 2024-09-17T02:52:22.937+0000 7f0f95025640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:52:34.695 DEBUG:teuthology.orchestra.run.smithi131:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force --yes-i-really-really-mean-it 2024-09-17T02:52:34.741 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:52:34.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 1 -- start start 2024-09-17T02:52:34.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 1 Processor -- start 2024-09-17T02:52:34.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d311a80 0x55a75e011080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30f200 0x55a75e010b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30ed80 0x55a75e010580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a75e02e000 con 0x55a75d30ed80 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a75e02e180 con 0x55a75d311a80 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55a75e02e300 con 0x55a75d30f200 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba40f2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30ed80 0x55a75e010580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba40f2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30ed80 0x55a75e010580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40700/0 (socket says 172.21.15.131:40700) 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba40f2640 1 -- 172.21.15.131:0/3356961045 learned_addr learned my addr 172.21.15.131:0/3356961045 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba38f1640 1 --2- 172.21.15.131:0/3356961045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d311a80 0x55a75e011080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/3356961045 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30f200 0x55a75e010b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba40f2640 1 -- 172.21.15.131:0/3356961045 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30f200 msgr2=0x55a75e010b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba40f2640 1 --2- 172.21.15.131:0/3356961045 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30f200 0x55a75e010b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba40f2640 1 -- 172.21.15.131:0/3356961045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d311a80 msgr2=0x55a75e011080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba40f2640 1 --2- 172.21.15.131:0/3356961045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d311a80 0x55a75e011080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba40f2640 1 -- 172.21.15.131:0/3356961045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75e008b40 con 0x55a75d30ed80 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/3356961045 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30f200 0x55a75e010b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba40f2640 1 --2- 172.21.15.131:0/3356961045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30ed80 0x55a75e010580 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x55a75d34a720 tx=0x55a75d349b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=484d5c617ca3e972 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba10ec640 1 -- 172.21.15.131:0/3356961045 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a75d2631e0 con 0x55a75d30ed80 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba10ec640 1 -- 172.21.15.131:0/3356961045 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a75d2808c0 con 0x55a75d30ed80 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba10ec640 1 -- 172.21.15.131:0/3356961045 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a75d2636c0 con 0x55a75d30ed80 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 1 -- 172.21.15.131:0/3356961045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a75e0090e0 con 0x55a75d30ed80 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 1 -- 172.21.15.131:0/3356961045 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a75e0092c0 con 0x55a75d30ed80 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 4 waiting for MDS map... 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba10ec640 1 -- 172.21.15.131:0/3356961045 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x55a75e00ea00 con 0x55a75d30ed80 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba10ec640 1 -- 172.21.15.131:0/3356961045 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 108) ==== 2893+0+0 (secure 0 0 0) 0x55a75d283180 con 0x55a75d30ed80 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 4 Got MDS map 108 2024-09-17T02:52:34.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.769+0000 7f7ba5132400 10 main: JournalTool::main 2024-09-17T02:52:34.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 4 main: JournalTool: connecting to RADOS... 2024-09-17T02:52:34.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 Processor -- start 2024-09-17T02:52:34.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 -- start start 2024-09-17T02:52:34.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30e480 0x55a75e011600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d30e000 0x55a75e011b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d310880 0x55a75e012100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a75e02e480 con 0x55a75d310880 2024-09-17T02:52:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a75e02e600 con 0x55a75d30e000 2024-09-17T02:52:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55a75e02e780 con 0x55a75d30e480 2024-09-17T02:52:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba30f0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d310880 0x55a75e012100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba30f0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d310880 0x55a75e012100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40716/0 (socket says 172.21.15.131:40716) 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba30f0640 1 -- 172.21.15.131:0/4157821637 learned_addr learned my addr 172.21.15.131:0/4157821637 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba38f1640 1 --2- 172.21.15.131:0/4157821637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d30e000 0x55a75e011b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/4157821637 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30e480 0x55a75e011600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba30f0640 1 -- 172.21.15.131:0/4157821637 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30e480 msgr2=0x55a75e011600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/4157821637 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30e480 0x55a75e011600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba30f0640 1 -- 172.21.15.131:0/4157821637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d30e000 msgr2=0x55a75e011b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/4157821637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d30e000 0x55a75e011b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba30f0640 1 -- 172.21.15.131:0/4157821637 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75e008960 con 0x55a75d310880 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/4157821637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d310880 0x55a75e012100 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x55a75e178bd0 tx=0x55a75e184c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7d7956655fbacd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7b9f0e8640 1 -- 172.21.15.131:0/4157821637 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a75d2631e0 con 0x55a75d310880 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7b9f0e8640 1 -- 172.21.15.131:0/4157821637 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a75d280c40 con 0x55a75d310880 2024-09-17T02:52:34.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7b9f0e8640 1 -- 172.21.15.131:0/4157821637 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a75d2636c0 con 0x55a75d310880 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 -- 172.21.15.131:0/4157821637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d310880 msgr2=0x55a75e012100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 --2- 172.21.15.131:0/4157821637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d310880 0x55a75e012100 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x55a75e178bd0 tx=0x55a75e184c00 comp rx=0 tx=0).stop 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 -- 172.21.15.131:0/4157821637 shutdown_connections 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 --2- 172.21.15.131:0/4157821637 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d310880 0x55a75e012100 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 --2- 172.21.15.131:0/4157821637 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d30e000 0x55a75e011b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 --2- 172.21.15.131:0/4157821637 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30e480 0x55a75e011600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 -- 172.21.15.131:0/4157821637 >> 172.21.15.131:0/4157821637 conn(0x55a75d30e900 msgr2=0x55a75e020800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 -- 172.21.15.131:0/4157821637 shutdown_connections 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.773+0000 7f7ba5132400 1 -- 172.21.15.131:0/4157821637 wait complete. 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba5132400 1 Processor -- start 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba5132400 1 -- start start 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30e000 0x55a75e011b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d30e480 0x55a75e011600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d310880 0x55a75e012100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba5132400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a75e02fe00 con 0x55a75d30e000 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba5132400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a75e02fc80 con 0x55a75d30e480 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba5132400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55a75e02fb00 con 0x55a75d310880 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba30f0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30e000 0x55a75e011b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba30f0640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30e000 0x55a75e011b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40718/0 (socket says 172.21.15.131:40718) 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba30f0640 1 -- 172.21.15.131:0/2836365593 learned_addr learned my addr 172.21.15.131:0/2836365593 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba38f1640 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d30e480 0x55a75e011600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d310880 0x55a75e012100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba30f0640 1 -- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d310880 msgr2=0x55a75e012100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d310880 0x55a75e012100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba30f0640 1 -- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d30e480 msgr2=0x55a75e011600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d30e480 0x55a75e011600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba30f0640 1 -- 172.21.15.131:0/2836365593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75e009e00 con 0x55a75d30e000 2024-09-17T02:52:34.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30e000 0x55a75e011b80 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x55a75e179e90 tx=0x55a75e184e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7b9d8e5640 1 -- 172.21.15.131:0/2836365593 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a75d2631e0 con 0x55a75d30e000 2024-09-17T02:52:34.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba5132400 1 -- 172.21.15.131:0/2836365593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a75e008d20 con 0x55a75d30e000 2024-09-17T02:52:34.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7b9d8e5640 1 -- 172.21.15.131:0/2836365593 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a75d280000 con 0x55a75d30e000 2024-09-17T02:52:34.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7b9d8e5640 1 -- 172.21.15.131:0/2836365593 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a75d2636c0 con 0x55a75d30e000 2024-09-17T02:52:34.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba5132400 1 -- 172.21.15.131:0/2836365593 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a75e008960 con 0x55a75d30e000 2024-09-17T02:52:34.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7ba5132400 4 main: JournalTool: resolving pool 17 2024-09-17T02:52:34.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.777+0000 7f7b9d8e5640 1 -- 172.21.15.131:0/2836365593 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x55a75d584c00 con 0x55a75d30e000 2024-09-17T02:52:34.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7b9d8e5640 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55a75d310400 0x55a75e012680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7b9d8e5640 1 -- 172.21.15.131:0/2836365593 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x55a75d594600 con 0x55a75d30e000 2024-09-17T02:52:34.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 4 main: JournalTool: creating IoCtx.. 2024-09-17T02:52:34.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 4 main: Executing for rank 0 2024-09-17T02:52:34.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba38f1640 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55a75d310400 0x55a75e012680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 1 -- start start 2024-09-17T02:52:34.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 1 Processor -- start 2024-09-17T02:52:34.780 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba38f1640 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55a75d310400 0x55a75e012680 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55a75d34b230 tx=0x55a75d5f5800 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30fb00 0x55a75e012c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30f680 0x55a75e013180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d311600 0x55a75e013700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55a75e02f800 con 0x55a75d30fb00 2024-09-17T02:52:34.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55a75e02f680 con 0x55a75d311600 2024-09-17T02:52:34.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55a75e02e480 con 0x55a75d30f680 2024-09-17T02:52:34.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba40f2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30fb00 0x55a75e012c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba40f2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30fb00 0x55a75e012c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40720/0 (socket says 172.21.15.131:40720) 2024-09-17T02:52:34.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba40f2640 1 -- 172.21.15.131:0/2685344201 learned_addr learned my addr 172.21.15.131:0/2685344201 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.781 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d311600 0x55a75e013700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba38f1640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30f680 0x55a75e013180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba40f2640 1 -- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30f680 msgr2=0x55a75e013180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba40f2640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30f680 0x55a75e013180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba40f2640 1 -- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d311600 msgr2=0x55a75e013700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba40f2640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d311600 0x55a75e013700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba40f2640 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75d6ce780 con 0x55a75d30fb00 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba40f2640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30fb00 0x55a75e012c00 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x55a75d577260 tx=0x55a75e164240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3946b4253ca00164 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7b9a8df640 1 -- 172.21.15.131:0/2685344201 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a75d263520 con 0x55a75d30fb00 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7b9a8df640 1 -- 172.21.15.131:0/2685344201 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a75d280e00 con 0x55a75d30fb00 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7b9a8df640 1 -- 172.21.15.131:0/2685344201 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a75d263040 con 0x55a75d30fb00 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a75d6ce960 con 0x55a75d30fb00 2024-09-17T02:52:34.782 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a75d6ceb40 con 0x55a75d30fb00 2024-09-17T02:52:34.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7ba5132400 4 waiting for MDS map... 2024-09-17T02:52:34.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.781+0000 7f7b9a8df640 1 -- 172.21.15.131:0/2685344201 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x55a75d6dce00 con 0x55a75d30fb00 2024-09-17T02:52:34.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.785+0000 7f7b9a8df640 1 -- 172.21.15.131:0/2685344201 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 108) ==== 2893+0+0 (secure 0 0 0) 0x55a75d283500 con 0x55a75d30fb00 2024-09-17T02:52:34.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.785+0000 7f7ba5132400 4 Got MDS map 108 2024-09-17T02:52:34.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.785+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55a75d311180 0x55a75d6f6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.785+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55a75d6f0400 con 0x55a75d311180 2024-09-17T02:52:34.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.785+0000 7f7ba38f1640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55a75d311180 0x55a75d6f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.785+0000 7f7ba38f1640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55a75d311180 0x55a75d6f6000 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=b18cdd9ae821231e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.789+0000 7f7ba38f1640 1 -- 172.21.15.131:0/2685344201 <== osd.6 v2:172.21.15.189:6802/952142225 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v124'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55a75d336000 con 0x55a75d311180 2024-09-17T02:52:34.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.789+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55a75d6fa000 0x55a75d6f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.789+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55a75d6f0800 con 0x55a75d6fa000 2024-09-17T02:52:34.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.789+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55a75d6fa000 0x55a75d6f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.789+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55a75d6fa000 0x55a75d6f6580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=32ba3bd8f6760f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba30f0640 1 -- 172.21.15.131:0/2685344201 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v124'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55a75d336500 con 0x55a75d6fa000 2024-09-17T02:52:34.787 INFO:teuthology.orchestra.run.smithi131.stdout:writing EResetJournal entry 2024-09-17T02:52:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55a75d6f0c00 con 0x55a75d6fa000 2024-09-17T02:52:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55a75d6fa480 0x55a75d6f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55a75d6f1000 con 0x55a75d6fa480 2024-09-17T02:52:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55a75d6f1400 con 0x55a75d311180 2024-09-17T02:52:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55a75d6fa900 0x55a75d6f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55a75d6f1800 con 0x55a75d6fa900 2024-09-17T02:52:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55a75d6fad80 0x55a75d6f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55a75d6f1c00 con 0x55a75d6fad80 2024-09-17T02:52:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55a75d712000 con 0x55a75d311180 2024-09-17T02:52:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba40f2640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55a75d6fa480 0x55a75d6f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba38f1640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55a75d6fa900 0x55a75d6f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55a75d6fad80 0x55a75d6f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba38f1640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55a75d6fa900 0x55a75d6f7080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=b8201d20a8331899 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba40f2640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55a75d6fa480 0x55a75d6f6b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=f4914fe88de8dfb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.793+0000 7f7ba30f0640 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55a75d6fad80 0x55a75d6f7600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=8f3dab6af68dac67 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.797+0000 7f7ba38f1640 1 -- 172.21.15.131:0/2685344201 <== osd.6 v2:172.21.15.189:6802/952142225 2 ==== osd_op_reply(5 200.00000003 [delete] v124'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55a75d336000 con 0x55a75d311180 2024-09-17T02:52:34.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.797+0000 7f7ba38f1640 1 -- 172.21.15.131:0/2685344201 <== osd.6 v2:172.21.15.189:6802/952142225 3 ==== osd_op_reply(8 200.00000006 [delete] v124'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55a75d336000 con 0x55a75d311180 2024-09-17T02:52:34.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.797+0000 7f7ba30f0640 1 -- 172.21.15.131:0/2685344201 <== osd.5 v2:172.21.15.189:6826/3175183672 2 ==== osd_op_reply(3 200.00000001 [delete] v124'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55a75d336500 con 0x55a75d6fa000 2024-09-17T02:52:34.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.797+0000 7f7ba30f0640 1 -- 172.21.15.131:0/2685344201 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== osd_op_reply(7 200.00000005 [delete] v124'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55a75d336500 con 0x55a75d6fad80 2024-09-17T02:52:34.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.797+0000 7f7ba38f1640 1 -- 172.21.15.131:0/2685344201 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== osd_op_reply(6 200.00000004 [delete] v124'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55a75d336000 con 0x55a75d6fa900 2024-09-17T02:52:34.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.797+0000 7f7ba40f2640 1 -- 172.21.15.131:0/2685344201 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(4 200.00000002 [delete] v124'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55a75d336780 con 0x55a75d6fa480 2024-09-17T02:52:34.793 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.797+0000 7f7b990dc640 1 -- 172.21.15.131:0/2685344201 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55a75d6b0800 con 0x55a75d6fa000 2024-09-17T02:52:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba30f0640 1 -- 172.21.15.131:0/2685344201 <== osd.5 v2:172.21.15.189:6826/3175183672 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v124'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55a75d336500 con 0x55a75d6fa000 2024-09-17T02:52:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 4 Successfully wrote new journal pointer and header for rank 8:0 2024-09-17T02:52:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55a75d6fa900 msgr2=0x55a75d6f7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55a75d6fa900 0x55a75d6f7080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.798 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55a75d6fa480 msgr2=0x55a75d6f6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55a75d6fa480 0x55a75d6f6b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55a75d6fad80 msgr2=0x55a75d6f7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55a75d6fad80 0x55a75d6f7600 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55a75d6fa000 msgr2=0x55a75d6f6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55a75d6fa000 0x55a75d6f6580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55a75d311180 msgr2=0x55a75d6f6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba30f0640 1 -- 172.21.15.131:0/2685344201 reap_dead start 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55a75d311180 0x55a75d6f6000 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba30f0640 1 -- 172.21.15.131:0/2685344201 reap_dead start 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30fb00 msgr2=0x55a75e012c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30fb00 0x55a75e012c00 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x55a75d577260 tx=0x55a75e164240 comp rx=0 tx=0).stop 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 shutdown_connections 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2685344201 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30fb00 0x55a75e012c00 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.799 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 >> 172.21.15.131:0/2685344201 conn(0x55a75d310d00 msgr2=0x55a75e021800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 shutdown_connections 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2685344201 wait complete. 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55a75d310400 msgr2=0x55a75e012680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55a75d310400 0x55a75e012680 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55a75d34b230 tx=0x55a75d5f5800 comp rx=0 tx=0).stop 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2836365593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30e000 msgr2=0x55a75e011b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30e000 0x55a75e011b80 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x55a75e179e90 tx=0x55a75e184e40 comp rx=0 tx=0).stop 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2836365593 shutdown_connections 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55a75d310400 0x55a75e012680 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d310880 0x55a75e012100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d30e480 0x55a75e011600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/2836365593 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30e000 0x55a75e011b80 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2836365593 >> 172.21.15.131:0/2836365593 conn(0x55a75d30e900 msgr2=0x55a75e020800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2836365593 shutdown_connections 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/2836365593 wait complete. 2024-09-17T02:52:34.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/3356961045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30ed80 msgr2=0x55a75e010580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/3356961045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30ed80 0x55a75e010580 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x55a75d34a720 tx=0x55a75d349b00 comp rx=0 tx=0).stop 2024-09-17T02:52:34.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/3356961045 shutdown_connections 2024-09-17T02:52:34.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/3356961045 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55a75d30ed80 0x55a75e010580 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/3356961045 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55a75d30f200 0x55a75e010b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 --2- 172.21.15.131:0/3356961045 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55a75d311a80 0x55a75e011080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/3356961045 >> 172.21.15.131:0/3356961045 conn(0x55a75d30ff80 msgr2=0x55a75e020000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/3356961045 shutdown_connections 2024-09-17T02:52:34.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.801+0000 7f7ba5132400 1 -- 172.21.15.131:0/3356961045 wait complete. 2024-09-17T02:52:34.804 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force', '--yes-i-really-really-mean-it'] in time 0:00:00.109020, result: writing EResetJournal entry 2024-09-17T02:52:34.804 DEBUG:teuthology.orchestra.run.smithi131:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-09-17T02:52:34.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.909+0000 7f778ab138c0 1 -- start start 2024-09-17T02:52:34.905 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.909+0000 7f778ab138c0 1 Processor -- start 2024-09-17T02:52:34.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.909+0000 7f778ab138c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34f680 0x56325d0acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.909+0000 7f778ab138c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c34fb00 0x56325d0ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.909+0000 7f778ab138c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c34ff80 0x56325d0ac000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.909+0000 7f778ab138c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56325d0d2180 con 0x56325c34f680 2024-09-17T02:52:34.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.909+0000 7f778ab138c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56325c32f800 con 0x56325c34ff80 2024-09-17T02:52:34.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.909+0000 7f778ab138c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56325c32f380 con 0x56325c34fb00 2024-09-17T02:52:34.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f77892d2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34f680 0x56325d0acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f77892d2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34f680 0x56325d0acb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40734/0 (socket says 172.21.15.131:40734) 2024-09-17T02:52:34.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f77892d2640 1 -- 172.21.15.131:0/1021365911 learned_addr learned my addr 172.21.15.131:0/1021365911 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.908 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1021365911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c34ff80 0x56325d0ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f7788ad1640 1 --2- 172.21.15.131:0/1021365911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c34fb00 0x56325d0ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f77892d2640 1 -- 172.21.15.131:0/1021365911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c34fb00 msgr2=0x56325d0ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f77892d2640 1 --2- 172.21.15.131:0/1021365911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c34fb00 0x56325d0ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f77892d2640 1 -- 172.21.15.131:0/1021365911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c34ff80 msgr2=0x56325d0ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f77892d2640 1 --2- 172.21.15.131:0/1021365911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c34ff80 0x56325d0ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f77892d2640 1 -- 172.21.15.131:0/1021365911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632582c21e0 con 0x56325c34f680 2024-09-17T02:52:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f77892d2640 1 --2- 172.21.15.131:0/1021365911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34f680 0x56325d0acb00 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x56325d0ba0c0 tx=0x56325c3ec360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a2b2b10079edbd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.909 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f7786acd640 1 -- 172.21.15.131:0/1021365911 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56325c305520 con 0x56325c34f680 2024-09-17T02:52:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f7786acd640 1 -- 172.21.15.131:0/1021365911 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56325c3228c0 con 0x56325c34f680 2024-09-17T02:52:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f7786acd640 1 -- 172.21.15.131:0/1021365911 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56325c3051e0 con 0x56325c34f680 2024-09-17T02:52:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 1 -- 172.21.15.131:0/1021365911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632582c32c0 con 0x56325c34f680 2024-09-17T02:52:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 1 -- 172.21.15.131:0/1021365911 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5632582c3a40 con 0x56325c34f680 2024-09-17T02:52:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 4 waiting for MDS map... 2024-09-17T02:52:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f7786acd640 1 -- 172.21.15.131:0/1021365911 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x56325d200600 con 0x56325c34f680 2024-09-17T02:52:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f7786acd640 1 -- 172.21.15.131:0/1021365911 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 108) ==== 2893+0+0 (secure 0 0 0) 0x56325c325180 con 0x56325c34f680 2024-09-17T02:52:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 4 Got MDS map 108 2024-09-17T02:52:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 4 datascan.main: Using metadata pool output 2024-09-17T02:52:34.910 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 4 datascan.main: connecting to RADOS... 2024-09-17T02:52:34.914 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 1 Processor -- start 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 1 -- start start 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34ed80 0x56325d0ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c34e900 0x56325d0ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c350880 0x56325d0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56325c32f980 con 0x56325c34ed80 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56325d0d2d80 con 0x56325c350880 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f778ab138c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56325d0d2f00 con 0x56325c34e900 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f7789ad3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34ed80 0x56325d0ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f7789ad3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34ed80 0x56325d0ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40746/0 (socket says 172.21.15.131:40746) 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.913+0000 7f7789ad3640 1 -- 172.21.15.131:0/1608396518 learned_addr learned my addr 172.21.15.131:0/1608396518 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 -- 172.21.15.131:0/1608396518 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c34e900 msgr2=0x56325d0ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.915 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1608396518 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c34e900 0x56325d0ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 -- 172.21.15.131:0/1608396518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c350880 msgr2=0x56325d0adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1608396518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c350880 0x56325d0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 -- 172.21.15.131:0/1608396518 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632582c3c20 con 0x56325c34ed80 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1608396518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34ed80 0x56325d0ad080 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x56325d13d800 tx=0x56325c44e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f9af72eaf59aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7784ac9640 1 -- 172.21.15.131:0/1608396518 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56325c305380 con 0x56325c34ed80 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7784ac9640 1 -- 172.21.15.131:0/1608396518 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56325c322fc0 con 0x56325c34ed80 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7784ac9640 1 -- 172.21.15.131:0/1608396518 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56325c305520 con 0x56325c34ed80 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 -- 172.21.15.131:0/1608396518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34ed80 msgr2=0x56325d0ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1608396518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34ed80 0x56325d0ad080 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x56325d13d800 tx=0x56325c44e5a0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 -- 172.21.15.131:0/1608396518 shutdown_connections 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1608396518 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c350880 0x56325d0adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1608396518 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c34e900 0x56325d0ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1608396518 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34ed80 0x56325d0ad080 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 -- 172.21.15.131:0/1608396518 >> 172.21.15.131:0/1608396518 conn(0x56325c34f200 msgr2=0x56325c606000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 -- 172.21.15.131:0/1608396518 shutdown_connections 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 -- 172.21.15.131:0/1608396518 wait complete. 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 Processor -- start 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 -- start start 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34e900 0x56325d0ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c34ed80 0x56325d0ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c350880 0x56325d0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x56325d0d3200 con 0x56325c34e900 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x56325d0d3080 con 0x56325c34ed80 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f778ab138c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x56325d0d3380 con 0x56325c350880 2024-09-17T02:52:34.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34e900 0x56325d0ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34e900 0x56325d0ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40760/0 (socket says 172.21.15.131:40760) 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 learned_addr learned my addr 172.21.15.131:0/1357721243 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c350880 msgr2=0x56325d0adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c350880 0x56325d0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c34ed80 msgr2=0x56325d0ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c34ed80 0x56325d0ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632582c34a0 con 0x56325c34e900 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.917+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34e900 0x56325d0ad600 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x56325d0bb6e0 tx=0x56325d220cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f77832c6640 1 -- 172.21.15.131:0/1357721243 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56325c3051e0 con 0x56325c34e900 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f77832c6640 1 -- 172.21.15.131:0/1357721243 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56325c322e00 con 0x56325c34e900 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5632582c23c0 con 0x56325c34e900 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f77832c6640 1 -- 172.21.15.131:0/1357721243 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56325c304820 con 0x56325c34e900 2024-09-17T02:52:34.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632582c3c20 con 0x56325c34e900 2024-09-17T02:52:34.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f778ab138c0 4 datascan.init: resolving metadata pool 17 2024-09-17T02:52:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f77832c6640 1 -- 172.21.15.131:0/1357721243 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x56325c614c00 con 0x56325c34e900 2024-09-17T02:52:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f77832c6640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x56325c351a80 0x56325d0ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f77832c6640 1 -- 172.21.15.131:0/1357721243 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x56325c618600 con 0x56325c34e900 2024-09-17T02:52:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f778ab138c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-17T02:52:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f778ab138c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-09-17T02:52:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f778ab138c0 4 datascan.main: opening data pool 'cephfs_data' 2024-09-17T02:52:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x56325c350d00 0x56325d0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325d1fbc00 con 0x56325c350d00 2024-09-17T02:52:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f77892d2640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x56325c351a80 0x56325d0ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f7788ad1640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x56325c350d00 0x56325d0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f77892d2640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x56325c351a80 0x56325d0ae100 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x56325d0ba5a0 tx=0x56325c3ead80 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.921+0000 7f7788ad1640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x56325c350d00 0x56325d0ae680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 1 ==== osd_op_reply(1 [pgnls start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f77842c8640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x56325c728000 0x56325c730000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a000 con 0x56325c728000 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x56325c728000 0x56325c730000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x56325c728000 0x56325c730000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(2 [pgnls start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c728000 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f7784ac9640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x56325c728480 0x56325c730580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a400 con 0x56325c728480 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f77892d2640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x56325c728480 0x56325c730580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f77892d2640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x56325c728480 0x56325c730580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== osd_op_reply(3 [pgnls start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728480 2024-09-17T02:52:34.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a800 con 0x56325c350d00 2024-09-17T02:52:34.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 2 ==== osd_op_reply(4 [pgnls start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f7784ac9640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x56325c728900 0x56325c730b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a800 con 0x56325c728900 2024-09-17T02:52:34.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f7788ad1640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x56325c728900 0x56325c730b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.924 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.925+0000 7f7788ad1640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x56325c728900 0x56325c730b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(5 [pgnls start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f77842c8640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x56325c729200 0x56325c731600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73ac00 con 0x56325c729200 2024-09-17T02:52:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x56325c729200 0x56325c731600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x56325c729200 0x56325c731600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.1 v2:172.21.15.131:6824/1616383728 1 ==== osd_op_reply(6 [pgnls start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729200 2024-09-17T02:52:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f7784ac9640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x56325c728d80 0x56325c731080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325d1fbc00 con 0x56325c728d80 2024-09-17T02:52:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f77892d2640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x56325c728d80 0x56325c731080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.926 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f77892d2640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x56325c728d80 0x56325c731080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== osd_op_reply(7 [pgnls start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728d80 2024-09-17T02:52:34.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.929+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b000 con 0x56325c728d80 2024-09-17T02:52:34.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.4 v2:172.21.15.189:6818/1965940492 2 ==== osd_op_reply(8 [pgnls start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728d80 2024-09-17T02:52:34.927 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73bc00 con 0x56325c350d00 2024-09-17T02:52:34.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 3 ==== osd_op_reply(9 [pgnls start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b400 con 0x56325c350d00 2024-09-17T02:52:34.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 4 ==== osd_op_reply(10 [pgnls start_epoch 124 out=49b] v122'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a800 con 0x56325c729200 2024-09-17T02:52:34.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.1 v2:172.21.15.131:6824/1616383728 2 ==== osd_op_reply(11 [pgnls start_epoch 124 out=49b] v122'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729200 2024-09-17T02:52:34.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b800 con 0x56325c350d00 2024-09-17T02:52:34.928 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 5 ==== osd_op_reply(12 [pgnls start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7784ac9640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x56325c729b00 0x56325c732100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73bc00 con 0x56325c729b00 2024-09-17T02:52:34.929 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7788ad1640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x56325c729b00 0x56325c732100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7788ad1640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x56325c729b00 0x56325c732100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== osd_op_reply(13 [pgnls start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c729b00 2024-09-17T02:52:34.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756000 con 0x56325c728900 2024-09-17T02:52:34.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 2 ==== osd_op_reply(14 [pgnls start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.930 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.933+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b400 con 0x56325c728d80 2024-09-17T02:52:34.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.4 v2:172.21.15.189:6818/1965940492 3 ==== osd_op_reply(15 [pgnls start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728d80 2024-09-17T02:52:34.931 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756400 con 0x56325c729b00 2024-09-17T02:52:34.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.2 v2:172.21.15.131:6800/163801004 2 ==== osd_op_reply(16 [pgnls start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c729b00 2024-09-17T02:52:34.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756000 con 0x56325c728480 2024-09-17T02:52:34.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.0 v2:172.21.15.131:6808/1154840320 2 ==== osd_op_reply(17 [pgnls start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728480 2024-09-17T02:52:34.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f77842c8640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x56325c729680 0x56325c731b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:34.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756800 con 0x56325c729680 2024-09-17T02:52:34.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x56325c729680 0x56325c731b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:34.932 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f7789ad3640 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x56325c729680 0x56325c731b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:34.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== osd_op_reply(18 [pgnls start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729680 2024-09-17T02:52:34.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756400 con 0x56325c350d00 2024-09-17T02:52:34.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 6 ==== osd_op_reply(19 [pgnls start_epoch 124 out=81b] v123'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.933 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.937+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a400 con 0x56325c350d00 2024-09-17T02:52:34.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.941+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 7 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.941+0000 7f778ab138c0 10 datascan.operator(): handling object 1099511627778.0 2024-09-17T02:52:34.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.941+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b000 con 0x56325c350d00 2024-09-17T02:52:34.936 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.941+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 8 ==== osd_op_reply(21 10000000002.00000000 [call] v124'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.941+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325d1fbc00 con 0x56325c350d00 2024-09-17T02:52:34.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.941+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 9 ==== osd_op_reply(22 [pgnls start_epoch 124 out=49b] v122'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.941+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c757000 con 0x56325c729680 2024-09-17T02:52:34.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.7 v2:172.21.15.189:6810/3586470400 2 ==== osd_op_reply(23 [pgnls start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729680 2024-09-17T02:52:34.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756c00 con 0x56325c729200 2024-09-17T02:52:34.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.1 v2:172.21.15.131:6824/1616383728 3 ==== osd_op_reply(24 [pgnls start_epoch 124 out=49b] v122'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729200 2024-09-17T02:52:34.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b800 con 0x56325c729680 2024-09-17T02:52:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.7 v2:172.21.15.189:6810/3586470400 3 ==== osd_op_reply(25 [pgnls start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729680 2024-09-17T02:52:34.939 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756800 con 0x56325c728900 2024-09-17T02:52:34.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 3 ==== osd_op_reply(26 [pgnls start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73bc00 con 0x56325c729b00 2024-09-17T02:52:34.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.2 v2:172.21.15.131:6800/163801004 3 ==== osd_op_reply(27 [pgnls start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c729b00 2024-09-17T02:52:34.940 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a000 con 0x56325c728d80 2024-09-17T02:52:34.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.4 v2:172.21.15.189:6818/1965940492 4 ==== osd_op_reply(28 [pgnls start_epoch 124 out=49b] v122'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728d80 2024-09-17T02:52:34.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756400 con 0x56325c729680 2024-09-17T02:52:34.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.7 v2:172.21.15.189:6810/3586470400 4 ==== osd_op_reply(29 [pgnls start_epoch 124 out=49b] v122'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729680 2024-09-17T02:52:34.941 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.945+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73ac00 con 0x56325c728900 2024-09-17T02:52:34.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.949+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 4 ==== osd_op_reply(30 [pgnls start_epoch 124 out=81b] v122'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.942 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.949+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x56325c756000 con 0x56325c728900 2024-09-17T02:52:34.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.949+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 5 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.943 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.949+0000 7f778ab138c0 10 datascan.operator(): handling object 1099511628279.0 2024-09-17T02:52:34.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.949+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56325c757400 con 0x56325c728900 2024-09-17T02:52:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.953+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 6 ==== osd_op_reply(32 100000001f7.00000000 [call] v124'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.953+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c757800 con 0x56325c728900 2024-09-17T02:52:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.953+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 7 ==== osd_op_reply(33 [pgnls start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.947 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.953+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a800 con 0x56325c350d00 2024-09-17T02:52:34.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.953+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 10 ==== osd_op_reply(34 [pgnls start_epoch 124 out=49b] v122'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.948 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.953+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756800 con 0x56325c728000 2024-09-17T02:52:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.953+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.3 v2:172.21.15.131:6816/4047536388 2 ==== osd_op_reply(35 [pgnls start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c728000 2024-09-17T02:52:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.953+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b400 con 0x56325c728900 2024-09-17T02:52:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.953+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 8 ==== osd_op_reply(36 [pgnls start_epoch 124 out=49b] v122'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.949 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.953+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c757000 con 0x56325c728900 2024-09-17T02:52:34.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.957+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 9 ==== osd_op_reply(37 [pgnls start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.957+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73bc00 con 0x56325c728000 2024-09-17T02:52:34.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.957+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.3 v2:172.21.15.131:6816/4047536388 3 ==== osd_op_reply(38 [pgnls start_epoch 124 out=49b] v122'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c728000 2024-09-17T02:52:34.950 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.957+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325d1fbc00 con 0x56325c728900 2024-09-17T02:52:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.957+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 10 ==== osd_op_reply(39 [pgnls start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.951 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.957+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73ac00 con 0x56325c729200 2024-09-17T02:52:34.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.957+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.1 v2:172.21.15.131:6824/1616383728 4 ==== osd_op_reply(40 [pgnls start_epoch 124 out=81b] v122'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729200 2024-09-17T02:52:34.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.957+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x56325c757c00 con 0x56325c729200 2024-09-17T02:52:34.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.957+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.1 v2:172.21.15.131:6824/1616383728 5 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729200 2024-09-17T02:52:34.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.957+0000 7f778ab138c0 10 datascan.operator(): handling object 1099511627780.1 2024-09-17T02:52:34.952 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.957+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b800 con 0x56325c729680 2024-09-17T02:52:34.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.7 v2:172.21.15.189:6810/3586470400 5 ==== osd_op_reply(42 10000000004.00000000 [call] v124'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729680 2024-09-17T02:52:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73bc00 con 0x56325c350d00 2024-09-17T02:52:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 11 ==== osd_op_reply(43 [pgnls start_epoch 124 out=49b] v122'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a800 con 0x56325c729200 2024-09-17T02:52:34.955 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.1 v2:172.21.15.131:6824/1616383728 6 ==== osd_op_reply(44 [pgnls start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729200 2024-09-17T02:52:34.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a400 con 0x56325c728000 2024-09-17T02:52:34.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.3 v2:172.21.15.131:6816/4047536388 4 ==== osd_op_reply(45 [pgnls start_epoch 124 out=81b] v123'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56325c3d8500 con 0x56325c728000 2024-09-17T02:52:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x56325c756c00 con 0x56325c728000 2024-09-17T02:52:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.3 v2:172.21.15.131:6816/4047536388 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x56325c3d8500 con 0x56325c728000 2024-09-17T02:52:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f778ab138c0 10 datascan.operator(): handling object 1099511628280.0 2024-09-17T02:52:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56325c756400 con 0x56325c728000 2024-09-17T02:52:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.3 v2:172.21.15.131:6816/4047536388 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v124'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56325c3d8500 con 0x56325c728000 2024-09-17T02:52:34.957 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.961+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756800 con 0x56325c728900 2024-09-17T02:52:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.965+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 11 ==== osd_op_reply(48 [pgnls start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.958 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.965+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b000 con 0x56325c729680 2024-09-17T02:52:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.965+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.7 v2:172.21.15.189:6810/3586470400 6 ==== osd_op_reply(49 [pgnls start_epoch 124 out=81b] v124'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729680 2024-09-17T02:52:34.959 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.965+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b800 con 0x56325c729680 2024-09-17T02:52:34.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.965+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.7 v2:172.21.15.189:6810/3586470400 7 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729680 2024-09-17T02:52:34.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.965+0000 7f778ab138c0 10 datascan.operator(): handling object 1099511627780.0 2024-09-17T02:52:34.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.965+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a800 con 0x56325c729680 2024-09-17T02:52:34.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.7 v2:172.21.15.189:6810/3586470400 8 ==== osd_op_reply(51 10000000004.00000000 [call] v124'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729680 2024-09-17T02:52:34.963 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a400 con 0x56325c728480 2024-09-17T02:52:34.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.0 v2:172.21.15.131:6808/1154840320 3 ==== osd_op_reply(52 [pgnls start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728480 2024-09-17T02:52:34.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b000 con 0x56325c729b00 2024-09-17T02:52:34.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.2 v2:172.21.15.131:6800/163801004 4 ==== osd_op_reply(53 [pgnls start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c729b00 2024-09-17T02:52:34.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756000 con 0x56325c728000 2024-09-17T02:52:34.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.3 v2:172.21.15.131:6816/4047536388 7 ==== osd_op_reply(54 [pgnls start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c728000 2024-09-17T02:52:34.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73ac00 con 0x56325c728000 2024-09-17T02:52:34.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.3 v2:172.21.15.131:6816/4047536388 8 ==== osd_op_reply(55 [pgnls start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c728000 2024-09-17T02:52:34.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c757400 con 0x56325c728480 2024-09-17T02:52:34.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.0 v2:172.21.15.131:6808/1154840320 4 ==== osd_op_reply(56 [pgnls start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728480 2024-09-17T02:52:34.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c76a000 con 0x56325c728900 2024-09-17T02:52:34.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 12 ==== osd_op_reply(57 [pgnls start_epoch 124 out=49b] v122'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.969+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c757800 con 0x56325c729680 2024-09-17T02:52:34.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.7 v2:172.21.15.189:6810/3586470400 9 ==== osd_op_reply(58 [pgnls start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729680 2024-09-17T02:52:34.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c76a400 con 0x56325c728d80 2024-09-17T02:52:34.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.4 v2:172.21.15.189:6818/1965940492 5 ==== osd_op_reply(59 [pgnls start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728d80 2024-09-17T02:52:34.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c76a000 con 0x56325c729680 2024-09-17T02:52:34.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.7 v2:172.21.15.189:6810/3586470400 10 ==== osd_op_reply(60 [pgnls start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729680 2024-09-17T02:52:34.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c76a800 con 0x56325c728480 2024-09-17T02:52:34.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.0 v2:172.21.15.131:6808/1154840320 5 ==== osd_op_reply(61 [pgnls start_epoch 124 out=49b] v122'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728480 2024-09-17T02:52:34.967 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b400 con 0x56325c728900 2024-09-17T02:52:34.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.5 v2:172.21.15.189:6826/3175183672 13 ==== osd_op_reply(62 [pgnls start_epoch 124 out=49b] v122'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c728900 2024-09-17T02:52:34.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c76ac00 con 0x56325c728000 2024-09-17T02:52:34.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.3 v2:172.21.15.131:6816/4047536388 9 ==== osd_op_reply(63 [pgnls start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c728000 2024-09-17T02:52:34.968 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73bc00 con 0x56325c728480 2024-09-17T02:52:34.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.0 v2:172.21.15.131:6808/1154840320 6 ==== osd_op_reply(64 [pgnls start_epoch 124 out=49b] v122'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728480 2024-09-17T02:52:34.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c76b000 con 0x56325c728d80 2024-09-17T02:52:34.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.4 v2:172.21.15.189:6818/1965940492 6 ==== osd_op_reply(65 [pgnls start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728d80 2024-09-17T02:52:34.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.973+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b000 con 0x56325c728480 2024-09-17T02:52:34.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.0 v2:172.21.15.131:6808/1154840320 7 ==== osd_op_reply(66 [pgnls start_epoch 124 out=49b] v122'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728480 2024-09-17T02:52:34.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c76b400 con 0x56325c729680 2024-09-17T02:52:34.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.7 v2:172.21.15.189:6810/3586470400 11 ==== osd_op_reply(67 [pgnls start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c729680 2024-09-17T02:52:34.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c757000 con 0x56325c728000 2024-09-17T02:52:34.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.3 v2:172.21.15.131:6816/4047536388 10 ==== osd_op_reply(68 [pgnls start_epoch 124 out=49b] v122'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c728000 2024-09-17T02:52:34.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c76b800 con 0x56325c728480 2024-09-17T02:52:34.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.0 v2:172.21.15.131:6808/1154840320 8 ==== osd_op_reply(69 [pgnls start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728480 2024-09-17T02:52:34.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c756800 con 0x56325c729b00 2024-09-17T02:52:34.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.2 v2:172.21.15.131:6800/163801004 5 ==== osd_op_reply(70 [pgnls start_epoch 124 out=49b] v122'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c729b00 2024-09-17T02:52:34.972 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c76bc00 con 0x56325c728480 2024-09-17T02:52:34.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f77892d2640 1 -- 172.21.15.131:0/1357721243 <== osd.0 v2:172.21.15.131:6808/1154840320 9 ==== osd_op_reply(71 [pgnls start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8780 con 0x56325c728480 2024-09-17T02:52:34.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325d1fbc00 con 0x56325c350d00 2024-09-17T02:52:34.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.6 v2:172.21.15.189:6802/952142225 12 ==== osd_op_reply(72 [pgnls start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c350d00 2024-09-17T02:52:34.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7784ac9640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73b800 con 0x56325c729b00 2024-09-17T02:52:34.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 <== osd.2 v2:172.21.15.131:6800/163801004 6 ==== osd_op_reply(73 [pgnls start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8000 con 0x56325c729b00 2024-09-17T02:52:34.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f77842c8640 1 -- 172.21.15.131:0/1357721243 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x56325c73a800 con 0x56325c728000 2024-09-17T02:52:34.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7789ad3640 1 -- 172.21.15.131:0/1357721243 <== osd.3 v2:172.21.15.131:6816/4047536388 11 ==== osd_op_reply(74 [pgnls start_epoch 124 out=49b] v122'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56325c3d8500 con 0x56325c728000 2024-09-17T02:52:34.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x56325c728480 msgr2=0x56325c730580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x56325c728480 0x56325c730580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.975 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x56325c729200 msgr2=0x56325c731600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x56325c729200 0x56325c731600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x56325c729b00 msgr2=0x56325c732100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x56325c729b00 0x56325c732100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x56325c728000 msgr2=0x56325c730000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x56325c728000 0x56325c730000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 reap_dead start 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x56325c728d80 msgr2=0x56325c731080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x56325c728d80 0x56325c731080 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x56325c728900 msgr2=0x56325c730b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x56325c728900 0x56325c730b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x56325c350d00 msgr2=0x56325d0ae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x56325c350d00 0x56325d0ae680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x56325c729680 msgr2=0x56325c731b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x56325c729680 0x56325c731b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f7788ad1640 1 -- 172.21.15.131:0/1357721243 reap_dead start 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x56325c351a80 msgr2=0x56325d0ae100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x56325c351a80 0x56325d0ae100 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x56325d0ba5a0 tx=0x56325c3ead80 comp rx=0 tx=0).stop 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34e900 msgr2=0x56325d0ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.977+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34e900 0x56325d0ad600 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x56325d0bb6e0 tx=0x56325d220cc0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 shutdown_connections 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x56325c351a80 0x56325d0ae100 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1357721243 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34e900 0x56325d0ad600 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 >> 172.21.15.131:0/1357721243 conn(0x56325c34f200 msgr2=0x56325c606000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 shutdown_connections 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 -- 172.21.15.131:0/1357721243 wait complete. 2024-09-17T02:52:34.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 -- 172.21.15.131:0/1021365911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34f680 msgr2=0x56325d0acb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:34.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1021365911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34f680 0x56325d0acb00 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x56325d0ba0c0 tx=0x56325c3ec360 comp rx=0 tx=0).stop 2024-09-17T02:52:34.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 -- 172.21.15.131:0/1021365911 shutdown_connections 2024-09-17T02:52:34.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1021365911 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x56325c34ff80 0x56325d0ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1021365911 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x56325c34fb00 0x56325d0ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 --2- 172.21.15.131:0/1021365911 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x56325c34f680 0x56325d0acb00 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:34.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 -- 172.21.15.131:0/1021365911 >> 172.21.15.131:0/1021365911 conn(0x56325c350400 msgr2=0x5632582cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:34.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 -- 172.21.15.131:0/1021365911 shutdown_connections 2024-09-17T02:52:34.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:34.981+0000 7f778ab138c0 1 -- 172.21.15.131:0/1021365911 wait complete. 2024-09-17T02:52:34.981 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.176285, result: 2024-09-17T02:52:34.981 DEBUG:teuthology.orchestra.run.smithi131:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-09-17T02:52:35.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.045+0000 7f82cbc588c0 1 -- start start 2024-09-17T02:52:35.042 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.045+0000 7f82cbc588c0 1 Processor -- start 2024-09-17T02:52:35.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7841a80 0x55d3f793b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f783f200 0x55d3f793ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.044 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f783ed80 0x55d3f793bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d3f85be480 con 0x55d3f783ed80 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d3f85be600 con 0x55d3f7841a80 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d3f85be780 con 0x55d3f783f200 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cac18640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f783ed80 0x55d3f793bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cac18640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f783ed80 0x55d3f793bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40780/0 (socket says 172.21.15.131:40780) 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cac18640 1 -- 172.21.15.131:0/3907381693 learned_addr learned my addr 172.21.15.131:0/3907381693 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82ca417640 1 --2- 172.21.15.131:0/3907381693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7841a80 0x55d3f793b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82c9c16640 1 --2- 172.21.15.131:0/3907381693 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f783f200 0x55d3f793ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cac18640 1 -- 172.21.15.131:0/3907381693 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f783f200 msgr2=0x55d3f793ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cac18640 1 --2- 172.21.15.131:0/3907381693 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f783f200 0x55d3f793ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cac18640 1 -- 172.21.15.131:0/3907381693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7841a80 msgr2=0x55d3f793b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cac18640 1 --2- 172.21.15.131:0/3907381693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7841a80 0x55d3f793b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cac18640 1 -- 172.21.15.131:0/3907381693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f37b21e0 con 0x55d3f783ed80 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cac18640 1 --2- 172.21.15.131:0/3907381693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f783ed80 0x55d3f793bb80 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x55d3f8617e30 tx=0x55d3f78dd1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be445008f4b68df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82c7c12640 1 -- 172.21.15.131:0/3907381693 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d3f77f56c0 con 0x55d3f783ed80 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82c7c12640 1 -- 172.21.15.131:0/3907381693 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d3f7812fc0 con 0x55d3f783ed80 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82c7c12640 1 -- 172.21.15.131:0/3907381693 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d3f77f5520 con 0x55d3f783ed80 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3907381693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f37b32c0 con 0x55d3f783ed80 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3907381693 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55d3f37b3c20 con 0x55d3f783ed80 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 4 waiting for MDS map... 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82c7c12640 1 -- 172.21.15.131:0/3907381693 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x55d3f8702600 con 0x55d3f783ed80 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82c7c12640 1 -- 172.21.15.131:0/3907381693 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 108) ==== 2893+0+0 (secure 0 0 0) 0x55d3f7815180 con 0x55d3f783ed80 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 4 Got MDS map 108 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 4 datascan.main: Using metadata pool output 2024-09-17T02:52:35.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 4 datascan.main: connecting to RADOS... 2024-09-17T02:52:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 1 Processor -- start 2024-09-17T02:52:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.049+0000 7f82cbc588c0 1 -- start start 2024-09-17T02:52:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f7840880 0x55d3f793b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7840d00 0x55d3f793a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7841180 0x55d3f793c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d3f781f980 con 0x55d3f7841180 2024-09-17T02:52:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d3f85bef00 con 0x55d3f7840d00 2024-09-17T02:52:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d3f85bf080 con 0x55d3f7840880 2024-09-17T02:52:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c9c16640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7841180 0x55d3f793c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c9c16640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7841180 0x55d3f793c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40786/0 (socket says 172.21.15.131:40786) 2024-09-17T02:52:35.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c9c16640 1 -- 172.21.15.131:0/3406249780 learned_addr learned my addr 172.21.15.131:0/3406249780 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82ca417640 1 --2- 172.21.15.131:0/3406249780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7840d00 0x55d3f793a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c9c16640 1 -- 172.21.15.131:0/3406249780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f7840880 msgr2=0x55d3f793b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c9c16640 1 --2- 172.21.15.131:0/3406249780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f7840880 0x55d3f793b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c9c16640 1 -- 172.21.15.131:0/3406249780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7840d00 msgr2=0x55d3f793a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c9c16640 1 --2- 172.21.15.131:0/3406249780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7840d00 0x55d3f793a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c9c16640 1 -- 172.21.15.131:0/3406249780 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f37b3e00 con 0x55d3f7841180 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82ca417640 1 --2- 172.21.15.131:0/3406249780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7840d00 0x55d3f793a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c9c16640 1 --2- 172.21.15.131:0/3406249780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7841180 0x55d3f793c100 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x55d3f870f110 tx=0x55d3f871cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bcd6b75878057e server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c5c0e640 1 -- 172.21.15.131:0/3406249780 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d3f77f56c0 con 0x55d3f7841180 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c5c0e640 1 -- 172.21.15.131:0/3406249780 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d3f7812e00 con 0x55d3f7841180 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c5c0e640 1 -- 172.21.15.131:0/3406249780 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d3f77f5520 con 0x55d3f7841180 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3406249780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7841180 msgr2=0x55d3f793c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/3406249780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7841180 0x55d3f793c100 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x55d3f870f110 tx=0x55d3f871cc00 comp rx=0 tx=0).stop 2024-09-17T02:52:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3406249780 shutdown_connections 2024-09-17T02:52:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/3406249780 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7841180 0x55d3f793c100 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/3406249780 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7840d00 0x55d3f793a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/3406249780 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f7840880 0x55d3f793b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3406249780 >> 172.21.15.131:0/3406249780 conn(0x55d3f783e900 msgr2=0x55d3f85ba800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3406249780 shutdown_connections 2024-09-17T02:52:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3406249780 wait complete. 2024-09-17T02:52:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 Processor -- start 2024-09-17T02:52:35.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- start start 2024-09-17T02:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7840880 0x55d3f793d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7840d00 0x55d3f793cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f7841180 0x55d3f793c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55d3f85bf380 con 0x55d3f7840880 2024-09-17T02:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55d3f85bf200 con 0x55d3f7840d00 2024-09-17T02:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82cbc588c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55d3f85bf500 con 0x55d3f7841180 2024-09-17T02:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.053+0000 7f82c9c16640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7840880 0x55d3f793d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c9c16640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7840880 0x55d3f793d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40798/0 (socket says 172.21.15.131:40798) 2024-09-17T02:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 learned_addr learned my addr 172.21.15.131:0/2061646834 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82ca417640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7840d00 0x55d3f793cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f7841180 msgr2=0x55d3f793c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c9c16640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f7841180 0x55d3f793c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7840d00 msgr2=0x55d3f793cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c9c16640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7840d00 0x55d3f793cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3f37b21e0 con 0x55d3f7840880 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c9c16640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7840880 0x55d3f793d180 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x55d3f8710570 tx=0x55d3f871cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c440b640 1 -- 172.21.15.131:0/2061646834 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d3f77f5040 con 0x55d3f7840880 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c440b640 1 -- 172.21.15.131:0/2061646834 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d3f7812c40 con 0x55d3f7840880 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c440b640 1 -- 172.21.15.131:0/2061646834 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d3f77f56c0 con 0x55d3f7840880 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d3f37b32c0 con 0x55d3f7840880 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3f37b3e00 con 0x55d3f7840880 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82cbc588c0 4 datascan.init: resolving metadata pool 17 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c440b640 1 -- 172.21.15.131:0/2061646834 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x55d3f7b10c00 con 0x55d3f7840880 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c440b640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55d3f783ff80 0x55d3f793d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c440b640 1 -- 172.21.15.131:0/2061646834 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x55d3f7b0ea00 con 0x55d3f7840880 2024-09-17T02:52:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82cbc588c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-17T02:52:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82cbc588c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-09-17T02:52:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82cbc588c0 4 datascan.main: opening data pool 'cephfs_data' 2024-09-17T02:52:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55d3f7841600 0x55d3f793c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b12800 con 0x55d3f7841600 2024-09-17T02:52:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82ca417640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55d3f783ff80 0x55d3f793d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.057+0000 7f82c9c16640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55d3f7841600 0x55d3f793c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82ca417640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55d3f783ff80 0x55d3f793d700 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55d3f78cd800 tx=0x55d3f793e600 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82c9c16640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55d3f7841600 0x55d3f793c100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv35 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55d3f7c26000 0x55d3f793a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b12c00 con 0x55d3f7c26000 2024-09-17T02:52:35.058 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82cac18640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55d3f7c26000 0x55d3f793a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82cac18640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55d3f7c26000 0x55d3f793a000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55d3f7c26480 0x55d3f793b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13000 con 0x55d3f7c26480 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82ca417640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55d3f7c26480 0x55d3f793b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82ca417640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55d3f7c26480 0x55d3f793b600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82c540d640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55d3f7c29180 0x55d3f7c3e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13c00 con 0x55d3f7c29180 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82c9c16640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55d3f7c29180 0x55d3f7c3e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.061+0000 7f82c9c16640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55d3f7c29180 0x55d3f7c3e000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c29180 2024-09-17T02:52:35.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c000 con 0x55d3f7c26000 2024-09-17T02:52:35.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c400 con 0x55d3f7c26480 2024-09-17T02:52:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4cc00 con 0x55d3f7841600 2024-09-17T02:52:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82c540d640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55d3f7c29600 0x55d3f7c3e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c800 con 0x55d3f7c29600 2024-09-17T02:52:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82cac18640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55d3f7c29600 0x55d3f7c3e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82cac18640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55d3f7c29600 0x55d3f7c3e580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.1 v2:172.21.15.131:6824/1616383728 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29600 2024-09-17T02:52:35.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82c5c0e640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55d3f7c50000 0x55d3f7c3f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4d000 con 0x55d3f7c50000 2024-09-17T02:52:35.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82ca417640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55d3f7c50000 0x55d3f7c3f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.065+0000 7f82ca417640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55d3f7c50000 0x55d3f7c3f080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c50000 2024-09-17T02:52:35.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b12c00 con 0x55d3f7c50000 2024-09-17T02:52:35.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.4 v2:172.21.15.189:6818/1965940492 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c50000 2024-09-17T02:52:35.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13000 con 0x55d3f7c26480 2024-09-17T02:52:35.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f8701400 con 0x55d3f7c26480 2024-09-17T02:52:35.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 124 out=49b] v122'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b12c00 con 0x55d3f7c29600 2024-09-17T02:52:35.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.1 v2:172.21.15.131:6824/1616383728 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 124 out=49b] v122'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29600 2024-09-17T02:52:35.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f8701000 con 0x55d3f7c26480 2024-09-17T02:52:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82c5c0e640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55d3f7c50480 0x55d3f7c3f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f378dc00 con 0x55d3f7c50480 2024-09-17T02:52:35.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82c9c16640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55d3f7c50480 0x55d3f7c3f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.069+0000 7f82c9c16640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55d3f7c50480 0x55d3f7c3f600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c50480 2024-09-17T02:52:35.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f378dc00 con 0x55d3f7841600 2024-09-17T02:52:35.066 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c000 con 0x55d3f7c50000 2024-09-17T02:52:35.067 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.4 v2:172.21.15.189:6818/1965940492 3 ==== osd_op_reply(17 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c50000 2024-09-17T02:52:35.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b12800 con 0x55d3f7c50480 2024-09-17T02:52:35.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.2 v2:172.21.15.131:6800/163801004 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c50480 2024-09-17T02:52:35.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c400 con 0x55d3f7c26000 2024-09-17T02:52:35.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82c540d640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55d3f7c29a80 0x55d3f7c3eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13c00 con 0x55d3f7c29a80 2024-09-17T02:52:35.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82cac18640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55d3f7c29a80 0x55d3f7c3eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82cac18640 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55d3f7c29a80 0x55d3f7c3eb00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== osd_op_reply(20 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29a80 2024-09-17T02:52:35.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.073+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 124 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c800 con 0x55d3f7c26480 2024-09-17T02:52:35.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 6 ==== osd_op_reply(21 [pgnls-filter start_epoch 124 out=81b] v124'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13400 con 0x55d3f7c26480 2024-09-17T02:52:35.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.071 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4cc00 con 0x55d3f7c29180 2024-09-17T02:52:35.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.3 v2:172.21.15.131:6816/4047536388 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c29180 2024-09-17T02:52:35.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c000 con 0x55d3f7c26000 2024-09-17T02:52:35.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c800 con 0x55d3f7c26480 2024-09-17T02:52:35.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.072 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13000 con 0x55d3f7841600 2024-09-17T02:52:35.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4d000 con 0x55d3f7c29600 2024-09-17T02:52:35.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.1 v2:172.21.15.131:6824/1616383728 3 ==== osd_op_reply(27 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29600 2024-09-17T02:52:35.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.077+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13400 con 0x55d3f7c50000 2024-09-17T02:52:35.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.4 v2:172.21.15.189:6818/1965940492 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c50000 2024-09-17T02:52:35.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4d400 con 0x55d3f7c50000 2024-09-17T02:52:35.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.4 v2:172.21.15.189:6818/1965940492 5 ==== osd_op_reply(29 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c50000 2024-09-17T02:52:35.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f8701400 con 0x55d3f7c26480 2024-09-17T02:52:35.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 9 ==== osd_op_reply(30 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4d800 con 0x55d3f7c26480 2024-09-17T02:52:35.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 10 ==== osd_op_reply(31 [pgnls-filter start_epoch 124 out=49b] v122'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f8701000 con 0x55d3f7c29600 2024-09-17T02:52:35.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.1 v2:172.21.15.131:6824/1616383728 4 ==== osd_op_reply(32 [pgnls-filter start_epoch 124 out=49b] v122'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29600 2024-09-17T02:52:35.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4dc00 con 0x55d3f7c26480 2024-09-17T02:52:35.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 11 ==== osd_op_reply(33 [pgnls-filter start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f8701000 con 0x55d3f7c50480 2024-09-17T02:52:35.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.2 v2:172.21.15.131:6800/163801004 3 ==== osd_op_reply(34 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c50480 2024-09-17T02:52:35.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.081+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f8701000 con 0x55d3f7841600 2024-09-17T02:52:35.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c400 con 0x55d3f7c50000 2024-09-17T02:52:35.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.4 v2:172.21.15.189:6818/1965940492 6 ==== osd_op_reply(36 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c50000 2024-09-17T02:52:35.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b12800 con 0x55d3f7c50480 2024-09-17T02:52:35.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.2 v2:172.21.15.131:6800/163801004 4 ==== osd_op_reply(37 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c50480 2024-09-17T02:52:35.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c000 con 0x55d3f7c26000 2024-09-17T02:52:35.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13000 con 0x55d3f7c29a80 2024-09-17T02:52:35.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.7 v2:172.21.15.189:6810/3586470400 2 ==== osd_op_reply(39 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29a80 2024-09-17T02:52:35.080 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4d000 con 0x55d3f7c26480 2024-09-17T02:52:35.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 12 ==== osd_op_reply(40 [pgnls-filter start_epoch 124 out=49b] v124'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4cc00 con 0x55d3f7c26480 2024-09-17T02:52:35.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 13 ==== osd_op_reply(41 [pgnls-filter start_epoch 124 out=49b] v122'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.085+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b12c00 con 0x55d3f7c29a80 2024-09-17T02:52:35.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.7 v2:172.21.15.189:6810/3586470400 3 ==== osd_op_reply(42 [pgnls-filter start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29a80 2024-09-17T02:52:35.082 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f378dc00 con 0x55d3f7c29600 2024-09-17T02:52:35.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.1 v2:172.21.15.131:6824/1616383728 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 124 out=49b] v122'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29600 2024-09-17T02:52:35.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13c00 con 0x55d3f7c29a80 2024-09-17T02:52:35.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.7 v2:172.21.15.189:6810/3586470400 4 ==== osd_op_reply(44 [pgnls-filter start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29a80 2024-09-17T02:52:35.083 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c800 con 0x55d3f7841600 2024-09-17T02:52:35.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 6 ==== osd_op_reply(45 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6c000 con 0x55d3f7c50480 2024-09-17T02:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.2 v2:172.21.15.131:6800/163801004 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c50480 2024-09-17T02:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4d400 con 0x55d3f7c50000 2024-09-17T02:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.4 v2:172.21.15.189:6818/1965940492 7 ==== osd_op_reply(47 [pgnls-filter start_epoch 124 out=49b] v122'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c50000 2024-09-17T02:52:35.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6c400 con 0x55d3f7c29a80 2024-09-17T02:52:35.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.089+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.7 v2:172.21.15.189:6810/3586470400 5 ==== osd_op_reply(48 [pgnls-filter start_epoch 124 out=49b] v122'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29a80 2024-09-17T02:52:35.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4d800 con 0x55d3f7841600 2024-09-17T02:52:35.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 7 ==== osd_op_reply(49 [pgnls-filter start_epoch 124 out=81b] v124'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-09-17T02:52:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 10 datascan.operator(): handling object 100000001f7.0 2024-09-17T02:52:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6dc00 con 0x55d3f7841600 2024-09-17T02:52:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 8 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-09-17T02:52:35.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-09-17T02:52:35.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-09-17T02:52:35.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6c000 con 0x55d3f7841600 2024-09-17T02:52:35.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 9 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-09-17T02:52:35.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c800 con 0x55d3f7c50000 2024-09-17T02:52:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.4 v2:172.21.15.189:6818/1965940492 8 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv14 ondisk = 0) ==== 164+0+524 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c50000 2024-09-17T02:52:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-09-17T02:52:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-09-17T02:52:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 20 datascan.get_frag_of: frag is * 2024-09-17T02:52:35.089 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.093+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4d800 con 0x55d3f7841600 2024-09-17T02:52:35.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 10 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82cbc588c0 10 datascan.find_or_create_dirfrag: read_version = 8 2024-09-17T02:52:35.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82cbc588c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-09-17T02:52:35.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b12800 con 0x55d3f7841600 2024-09-17T02:52:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 11 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82cbc588c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-09-17T02:52:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82cbc588c0 20 datascan.read_dentry: bravo_head not found in result 2024-09-17T02:52:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82cbc588c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-09-17T02:52:35.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55d3f8701000 con 0x55d3f7841600 2024-09-17T02:52:35.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 12 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v124'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82cbc588c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-09-17T02:52:35.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.097+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6dc00 con 0x55d3f7841600 2024-09-17T02:52:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 13 ==== osd_op_reply(56 [pgnls-filter start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.094 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4dc00 con 0x55d3f7c26480 2024-09-17T02:52:35.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 14 ==== osd_op_reply(57 [pgnls-filter start_epoch 124 out=49b] v122'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6c800 con 0x55d3f7c29180 2024-09-17T02:52:35.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.3 v2:172.21.15.131:6816/4047536388 3 ==== osd_op_reply(58 [pgnls-filter start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c29180 2024-09-17T02:52:35.095 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c400 con 0x55d3f7841600 2024-09-17T02:52:35.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 14 ==== osd_op_reply(59 [pgnls-filter start_epoch 124 out=49b] v122'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6cc00 con 0x55d3f7841600 2024-09-17T02:52:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 15 ==== osd_op_reply(60 [pgnls-filter start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f8701400 con 0x55d3f7c29180 2024-09-17T02:52:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.3 v2:172.21.15.131:6816/4047536388 4 ==== osd_op_reply(61 [pgnls-filter start_epoch 124 out=49b] v122'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c29180 2024-09-17T02:52:35.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.101+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6d000 con 0x55d3f7841600 2024-09-17T02:52:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13400 con 0x55d3f7c29600 2024-09-17T02:52:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.1 v2:172.21.15.131:6824/1616383728 6 ==== osd_op_reply(63 [pgnls-filter start_epoch 124 out=49b] v122'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29600 2024-09-17T02:52:35.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6d400 con 0x55d3f7c26480 2024-09-17T02:52:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 15 ==== osd_op_reply(64 [pgnls-filter start_epoch 124 out=49b] v122'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13000 con 0x55d3f7c29600 2024-09-17T02:52:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.1 v2:172.21.15.131:6824/1616383728 7 ==== osd_op_reply(65 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29600 2024-09-17T02:52:35.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6d800 con 0x55d3f7c29180 2024-09-17T02:52:35.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.3 v2:172.21.15.131:6816/4047536388 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 124 out=49b] v124'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c29180 2024-09-17T02:52:35.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b12c00 con 0x55d3f7841600 2024-09-17T02:52:35.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 17 ==== osd_op_reply(67 [pgnls-filter start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.100 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6c400 con 0x55d3f7c29a80 2024-09-17T02:52:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.7 v2:172.21.15.189:6810/3586470400 6 ==== osd_op_reply(68 [pgnls-filter start_epoch 124 out=49b] v124'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29a80 2024-09-17T02:52:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13c00 con 0x55d3f7c26000 2024-09-17T02:52:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.101 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.105+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6c000 con 0x55d3f7c50480 2024-09-17T02:52:35.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.2 v2:172.21.15.131:6800/163801004 6 ==== osd_op_reply(70 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c50480 2024-09-17T02:52:35.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13400 con 0x55d3f7c29180 2024-09-17T02:52:35.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.3 v2:172.21.15.131:6816/4047536388 6 ==== osd_op_reply(71 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c29180 2024-09-17T02:52:35.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6dc00 con 0x55d3f7c29180 2024-09-17T02:52:35.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.3 v2:172.21.15.131:6816/4047536388 7 ==== osd_op_reply(72 [pgnls-filter start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c29180 2024-09-17T02:52:35.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13800 con 0x55d3f7c26000 2024-09-17T02:52:35.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.106 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6c800 con 0x55d3f7841600 2024-09-17T02:52:35.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 124 out=49b] v122'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4c000 con 0x55d3f7c29a80 2024-09-17T02:52:35.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.7 v2:172.21.15.189:6810/3586470400 7 ==== osd_op_reply(75 [pgnls-filter start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29a80 2024-09-17T02:52:35.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f8701000 con 0x55d3f7c50000 2024-09-17T02:52:35.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.4 v2:172.21.15.189:6818/1965940492 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c50000 2024-09-17T02:52:35.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c4d800 con 0x55d3f7c29a80 2024-09-17T02:52:35.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.7 v2:172.21.15.189:6810/3586470400 8 ==== osd_op_reply(77 [pgnls-filter start_epoch 124 out=49b] v122'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29a80 2024-09-17T02:52:35.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.109+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b12800 con 0x55d3f7c26000 2024-09-17T02:52:35.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 8 ==== osd_op_reply(78 [pgnls-filter start_epoch 124 out=49b] v122'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.107 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f378dc00 con 0x55d3f7841600 2024-09-17T02:52:35.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.5 v2:172.21.15.189:6826/3175183672 19 ==== osd_op_reply(79 [pgnls-filter start_epoch 124 out=49b] v122'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7841600 2024-09-17T02:52:35.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b12c00 con 0x55d3f7c29180 2024-09-17T02:52:35.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.3 v2:172.21.15.131:6816/4047536388 8 ==== osd_op_reply(80 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c29180 2024-09-17T02:52:35.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c78000 con 0x55d3f7c26000 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 9 ==== osd_op_reply(81 [pgnls-filter start_epoch 124 out=49b] v122'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7b13400 con 0x55d3f7c50000 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.4 v2:172.21.15.189:6818/1965940492 10 ==== osd_op_reply(82 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c50000 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c78400 con 0x55d3f7c26000 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 10 ==== osd_op_reply(83 [pgnls-filter start_epoch 124 out=49b] v122'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6cc00 con 0x55d3f7c29a80 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.7 v2:172.21.15.189:6810/3586470400 9 ==== osd_op_reply(84 [pgnls-filter start_epoch 124 out=49b] v122'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c29a80 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c78800 con 0x55d3f7c29180 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.3 v2:172.21.15.131:6816/4047536388 9 ==== osd_op_reply(85 [pgnls-filter start_epoch 124 out=49b] v122'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c29180 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6d000 con 0x55d3f7c26000 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 11 ==== osd_op_reply(86 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.113+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c78c00 con 0x55d3f7c50480 2024-09-17T02:52:35.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.2 v2:172.21.15.131:6800/163801004 7 ==== osd_op_reply(87 [pgnls-filter start_epoch 124 out=49b] v122'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c50480 2024-09-17T02:52:35.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6d800 con 0x55d3f7c26000 2024-09-17T02:52:35.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cac18640 1 -- 172.21.15.131:0/2061646834 <== osd.0 v2:172.21.15.131:6808/1154840320 12 ==== osd_op_reply(88 [pgnls-filter start_epoch 124 out=49b] v122'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8500 con 0x55d3f7c26000 2024-09-17T02:52:35.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c79000 con 0x55d3f7c26480 2024-09-17T02:52:35.113 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 <== osd.6 v2:172.21.15.189:6802/952142225 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 124 out=49b] v122'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8780 con 0x55d3f7c26480 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82c540d640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c6c000 con 0x55d3f7c50480 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.2 v2:172.21.15.131:6800/163801004 8 ==== osd_op_reply(90 [pgnls-filter start_epoch 124 out=49b] v122'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c50480 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82c5c0e640 1 -- 172.21.15.131:0/2061646834 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 124 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55d3f7c79400 con 0x55d3f7c29180 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82c9c16640 1 -- 172.21.15.131:0/2061646834 <== osd.3 v2:172.21.15.131:6816/4047536388 10 ==== osd_op_reply(91 [pgnls-filter start_epoch 124 out=49b] v122'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55d3f78c8000 con 0x55d3f7c29180 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55d3f7c26000 msgr2=0x55d3f793a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55d3f7c26000 0x55d3f793a000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55d3f7c29600 msgr2=0x55d3f7c3e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55d3f7c29600 0x55d3f7c3e580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55d3f7c50480 msgr2=0x55d3f7c3f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55d3f7c50480 0x55d3f7c3f600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55d3f7c29180 msgr2=0x55d3f7c3e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55d3f7c29180 0x55d3f7c3e000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 reap_dead start 2024-09-17T02:52:35.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55d3f7c50000 msgr2=0x55d3f7c3f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55d3f7c50000 0x55d3f7c3f080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 reap_dead start 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55d3f7841600 msgr2=0x55d3f793c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55d3f7841600 0x55d3f793c100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55d3f7c26480 msgr2=0x55d3f793b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55d3f7c26480 0x55d3f793b600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55d3f7c29a80 msgr2=0x55d3f7c3eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55d3f7c29a80 0x55d3f7c3eb00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55d3f783ff80 msgr2=0x55d3f793d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55d3f783ff80 0x55d3f793d700 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55d3f78cd800 tx=0x55d3f793e600 comp rx=0 tx=0).stop 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7840880 msgr2=0x55d3f793d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/2061646834 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f7840880 0x55d3f793d180 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x55d3f8710570 tx=0x55d3f871cde0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82ca417640 1 -- 172.21.15.131:0/2061646834 reap_dead start 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 shutdown_connections 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 >> 172.21.15.131:0/2061646834 conn(0x55d3f783e900 msgr2=0x55d3f85ba800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 shutdown_connections 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/2061646834 wait complete. 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3907381693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f783ed80 msgr2=0x55d3f793bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/3907381693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f783ed80 0x55d3f793bb80 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x55d3f8617e30 tx=0x55d3f78dd1a0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3907381693 shutdown_connections 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/3907381693 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55d3f783ed80 0x55d3f793bb80 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/3907381693 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55d3f783f200 0x55d3f793ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 --2- 172.21.15.131:0/3907381693 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55d3f7841a80 0x55d3f793b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3907381693 >> 172.21.15.131:0/3907381693 conn(0x55d3f7840400 msgr2=0x55d3f85ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3907381693 shutdown_connections 2024-09-17T02:52:35.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.117+0000 7f82cbc588c0 1 -- 172.21.15.131:0/3907381693 wait complete. 2024-09-17T02:52:35.119 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.137389, result: 2024-09-17T02:52:35.119 DEBUG:teuthology.orchestra.run.smithi131:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-09-17T02:52:35.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac737968c0 1 -- start start 2024-09-17T02:52:35.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac737968c0 1 Processor -- start 2024-09-17T02:52:35.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac737968c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634b680 0x55c3c6442580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac737968c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634bb00 0x55c3c6442000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac737968c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634bf80 0x55c3c6442b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac737968c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c3c70b2180 con 0x55c3c634b680 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac737968c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c3c70b2300 con 0x55c3c634bb00 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac737968c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55c3c70b2480 con 0x55c3c634bf80 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac71f55640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634b680 0x55c3c6442580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac71f55640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634b680 0x55c3c6442580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40808/0 (socket says 172.21.15.131:40808) 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac71f55640 1 -- 172.21.15.131:0/2097123872 learned_addr learned my addr 172.21.15.131:0/2097123872 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac71754640 1 --2- 172.21.15.131:0/2097123872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634bb00 0x55c3c6442000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac72756640 1 --2- 172.21.15.131:0/2097123872 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634bf80 0x55c3c6442b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac71f55640 1 -- 172.21.15.131:0/2097123872 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634bf80 msgr2=0x55c3c6442b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:35.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac71f55640 1 --2- 172.21.15.131:0/2097123872 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634bf80 0x55c3c6442b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac71f55640 1 -- 172.21.15.131:0/2097123872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634bb00 msgr2=0x55c3c6442000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac71f55640 1 --2- 172.21.15.131:0/2097123872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634bb00 0x55c3c6442000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac71f55640 1 -- 172.21.15.131:0/2097123872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c22be1e0 con 0x55c3c634b680 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac71754640 1 --2- 172.21.15.131:0/2097123872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634bb00 0x55c3c6442000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.181+0000 7fac71f55640 1 --2- 172.21.15.131:0/2097123872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634b680 0x55c3c6442580 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x55c3c71f00c0 tx=0x55c3c63e91a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dad1eb6cac43ea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac6f750640 1 -- 172.21.15.131:0/2097123872 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c3c6301520 con 0x55c3c634b680 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac6f750640 1 -- 172.21.15.131:0/2097123872 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c3c631e8c0 con 0x55c3c634b680 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac6f750640 1 -- 172.21.15.131:0/2097123872 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c3c63011e0 con 0x55c3c634b680 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 -- 172.21.15.131:0/2097123872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c22bf4a0 con 0x55c3c634b680 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 -- 172.21.15.131:0/2097123872 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55c3c22bfa40 con 0x55c3c634b680 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 4 waiting for MDS map... 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac6f750640 1 -- 172.21.15.131:0/2097123872 <== mon.0 v2:172.21.15.131:3300/0 4 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x55c3c71fe600 con 0x55c3c634b680 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac6f750640 1 -- 172.21.15.131:0/2097123872 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 108) ==== 2893+0+0 (secure 0 0 0) 0x55c3c6321180 con 0x55c3c634b680 2024-09-17T02:52:35.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 4 Got MDS map 108 2024-09-17T02:52:35.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 4 datascan.main: Using metadata pool output 2024-09-17T02:52:35.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 4 datascan.main: connecting to RADOS... 2024-09-17T02:52:35.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 Processor -- start 2024-09-17T02:52:35.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 -- start start 2024-09-17T02:52:35.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634ad80 0x55c3c6443080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634c880 0x55c3c6443b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c3c70b2600 con 0x55c3c634a900 2024-09-17T02:52:35.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c3c70b2780 con 0x55c3c634ad80 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55c3c70b2900 con 0x55c3c634c880 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac71f55640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac71f55640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40814/0 (socket says 172.21.15.131:40814) 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac71f55640 1 -- 172.21.15.131:0/1018433338 learned_addr learned my addr 172.21.15.131:0/1018433338 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac71f55640 1 -- 172.21.15.131:0/1018433338 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634c880 msgr2=0x55c3c6443b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac72756640 1 --2- 172.21.15.131:0/1018433338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634ad80 0x55c3c6443080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac71f55640 1 --2- 172.21.15.131:0/1018433338 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634c880 0x55c3c6443b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac71f55640 1 -- 172.21.15.131:0/1018433338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634ad80 msgr2=0x55c3c6443080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac71f55640 1 --2- 172.21.15.131:0/1018433338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634ad80 0x55c3c6443080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac71f55640 1 -- 172.21.15.131:0/1018433338 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c22bfc20 con 0x55c3c634a900 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac71f55640 1 --2- 172.21.15.131:0/1018433338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x55c3c71f1530 tx=0x55c3c644a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c6b634baaf8b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac6d74c640 1 -- 172.21.15.131:0/1018433338 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c3c6301040 con 0x55c3c634a900 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac6d74c640 1 -- 172.21.15.131:0/1018433338 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c3c631efc0 con 0x55c3c634a900 2024-09-17T02:52:35.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac6d74c640 1 -- 172.21.15.131:0/1018433338 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c3c6301520 con 0x55c3c634a900 2024-09-17T02:52:35.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 -- 172.21.15.131:0/1018433338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 msgr2=0x55c3c6443600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.185+0000 7fac737968c0 1 --2- 172.21.15.131:0/1018433338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x55c3c71f1530 tx=0x55c3c644a540 comp rx=0 tx=0).stop 2024-09-17T02:52:35.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 -- 172.21.15.131:0/1018433338 shutdown_connections 2024-09-17T02:52:35.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 --2- 172.21.15.131:0/1018433338 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634c880 0x55c3c6443b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 --2- 172.21.15.131:0/1018433338 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 --2- 172.21.15.131:0/1018433338 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634ad80 0x55c3c6443080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 -- 172.21.15.131:0/1018433338 >> 172.21.15.131:0/1018433338 conn(0x55c3c634b200 msgr2=0x55c3c65fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:35.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 -- 172.21.15.131:0/1018433338 shutdown_connections 2024-09-17T02:52:35.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 -- 172.21.15.131:0/1018433338 wait complete. 2024-09-17T02:52:35.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 Processor -- start 2024-09-17T02:52:35.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 -- start start 2024-09-17T02:52:35.188 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634ad80 0x55c3c6443080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634c880 0x55c3c6443b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x55c3c70b3c80 con 0x55c3c634a900 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x55c3c70b3b00 con 0x55c3c634c880 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x55c3c70b3980 con 0x55c3c634ad80 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac72756640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac72756640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40816/0 (socket says 172.21.15.131:40816) 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 learned_addr learned my addr 172.21.15.131:0/2586463499 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634ad80 msgr2=0x55c3c6443080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac71754640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634c880 0x55c3c6443b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac72756640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634ad80 0x55c3c6443080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634c880 msgr2=0x55c3c6443b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac72756640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634c880 0x55c3c6443b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.189 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3c22be1e0 con 0x55c3c634a900 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac72756640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x55c3c66126f0 tx=0x55c3c7214c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac6bf49640 1 -- 172.21.15.131:0/2586463499 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c3c6301040 con 0x55c3c634a900 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac6bf49640 1 -- 172.21.15.131:0/2586463499 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c3c631ec40 con 0x55c3c634a900 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c3c22bf4a0 con 0x55c3c634a900 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac6bf49640 1 -- 172.21.15.131:0/2586463499 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c3c6301520 con 0x55c3c634a900 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3c22bfe00 con 0x55c3c634a900 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.189+0000 7fac737968c0 4 datascan.init: resolving metadata pool 17 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac6bf49640 1 -- 172.21.15.131:0/2586463499 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x55c3c6610c00 con 0x55c3c634a900 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac6bf49640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55c3c634da80 0x55c3c6444100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac6bf49640 1 -- 172.21.15.131:0/2586463499 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x55c3c661ea00 con 0x55c3c634a900 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac737968c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac737968c0 4 datascan.main: resolving metadata pool 17 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55c3c634cd00 0x55c3c6444680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.190 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c634cd00 2024-09-17T02:52:35.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac71754640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55c3c634cd00 0x55c3c6444680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac71f55640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55c3c634da80 0x55c3c6444100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac71754640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55c3c634cd00 0x55c3c6444680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== osd_op_reply(1 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac71754640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55c3c671e000 0x55c3c6722000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671e000 2024-09-17T02:52:35.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac71f55640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55c3c634da80 0x55c3c6444100 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x55c3c71f1530 tx=0x55c3c671c0c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac72756640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55c3c671e000 0x55c3c6722000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.191 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac72756640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55c3c671e000 0x55c3c6722000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== osd_op_reply(2 [pgnls start_epoch 124 out=177b] v123'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c634cd00 2024-09-17T02:52:35.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 2 ==== osd_op_reply(3 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac71754640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55c3c671e480 0x55c3c6722580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.193+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671e480 2024-09-17T02:52:35.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.197+0000 7fac71f55640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55c3c671e480 0x55c3c6722580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.197+0000 7fac71f55640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55c3c671e480 0x55c3c6722580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.197+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== osd_op_reply(4 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671e480 2024-09-17T02:52:35.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.197+0000 7fac71f55640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55c3c671ed80 0x55c3c6723080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.193 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.197+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671ed80 2024-09-17T02:52:35.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.197+0000 7fac71754640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55c3c671ed80 0x55c3c6723080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.197+0000 7fac71754640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55c3c671ed80 0x55c3c6723080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.197+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 1 ==== osd_op_reply(5 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.197+0000 7fac71754640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55c3c671e900 0x55c3c6722b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.194 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.197+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671e900 2024-09-17T02:52:35.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.197+0000 7fac72756640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55c3c671e900 0x55c3c6722b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac72756640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55c3c671e900 0x55c3c6722b00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== osd_op_reply(6 [pgnls start_epoch 124 out=149b] v124'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac72756640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55c3c671f200 0x55c3c6723600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671f200 2024-09-17T02:52:35.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac71f55640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55c3c671f200 0x55c3c6723600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.195 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac71f55640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55c3c671f200 0x55c3c6723600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== osd_op_reply(7 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671f200 2024-09-17T02:52:35.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671e900 2024-09-17T02:52:35.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 2 ==== osd_op_reply(8 [pgnls start_epoch 124 out=73b] v123'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac72756640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55c3c671f680 0x55c3c6723b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.196 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671f680 2024-09-17T02:52:35.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac71754640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55c3c671f680 0x55c3c6723b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac71754640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55c3c671f680 0x55c3c6723b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.1 v2:172.21.15.131:6824/1616383728 1 ==== osd_op_reply(9 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671f680 2024-09-17T02:52:35.197 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.201+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671e480 2024-09-17T02:52:35.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.7 v2:172.21.15.189:6810/3586470400 2 ==== osd_op_reply(10 [pgnls start_epoch 124 out=81b] v123'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671e480 2024-09-17T02:52:35.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671f680 2024-09-17T02:52:35.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.1 v2:172.21.15.131:6824/1616383728 2 ==== osd_op_reply(11 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671f680 2024-09-17T02:52:35.198 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671e000 2024-09-17T02:52:35.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 2 ==== osd_op_reply(12 [pgnls start_epoch 124 out=49b] v124'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.199 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c634cd00 2024-09-17T02:52:35.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 3 ==== osd_op_reply(13 [pgnls start_epoch 124 out=77b] v123'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac71754640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55c3c671ff80 0x55c3c6724680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671ff80 2024-09-17T02:52:35.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac72756640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55c3c671ff80 0x55c3c6724680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac72756640 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55c3c671ff80 0x55c3c6724680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.200 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== osd_op_reply(14 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671ff80 2024-09-17T02:52:35.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671ed80 2024-09-17T02:52:35.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 2 ==== osd_op_reply(15 [pgnls start_epoch 124 out=73b] v123'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671ed80 2024-09-17T02:52:35.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 3 ==== osd_op_reply(16 [pgnls start_epoch 124 out=49b] v124'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.205+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671e900 2024-09-17T02:52:35.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 3 ==== osd_op_reply(17 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671e000 2024-09-17T02:52:35.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 3 ==== osd_op_reply(18 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671e480 2024-09-17T02:52:35.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.7 v2:172.21.15.189:6810/3586470400 3 ==== osd_op_reply(19 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671e480 2024-09-17T02:52:35.204 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671f200 2024-09-17T02:52:35.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.3 v2:172.21.15.131:6816/4047536388 2 ==== osd_op_reply(20 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671f200 2024-09-17T02:52:35.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c634cd00 2024-09-17T02:52:35.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 4 ==== osd_op_reply(21 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671f200 2024-09-17T02:52:35.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.3 v2:172.21.15.131:6816/4047536388 3 ==== osd_op_reply(22 [pgnls start_epoch 124 out=97b] v123'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671f200 2024-09-17T02:52:35.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c634cd00 2024-09-17T02:52:35.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 5 ==== osd_op_reply(23 [pgnls start_epoch 124 out=74b] v121'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671f680 2024-09-17T02:52:35.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.1 v2:172.21.15.131:6824/1616383728 3 ==== osd_op_reply(24 [pgnls start_epoch 124 out=73b] v123'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671f680 2024-09-17T02:52:35.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.209+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c634cd00 2024-09-17T02:52:35.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 6 ==== osd_op_reply(25 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671e000 2024-09-17T02:52:35.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 4 ==== osd_op_reply(26 [pgnls start_epoch 124 out=98b] v123'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671ed80 2024-09-17T02:52:35.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 4 ==== osd_op_reply(27 [pgnls start_epoch 124 out=76b] v124'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671f200 2024-09-17T02:52:35.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.3 v2:172.21.15.131:6816/4047536388 4 ==== osd_op_reply(28 [pgnls start_epoch 124 out=49b] v124'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671f200 2024-09-17T02:52:35.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671e900 2024-09-17T02:52:35.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 4 ==== osd_op_reply(29 [pgnls start_epoch 124 out=105b] v124'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671ff80 2024-09-17T02:52:35.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.2 v2:172.21.15.131:6800/163801004 2 ==== osd_op_reply(30 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671ff80 2024-09-17T02:52:35.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671ed80 2024-09-17T02:52:35.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 5 ==== osd_op_reply(31 [pgnls start_epoch 124 out=73b] v124'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.209 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.213+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c634cd00 2024-09-17T02:52:35.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 7 ==== osd_op_reply(32 [pgnls start_epoch 124 out=101b] v124'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-09-17T02:52:35.210 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620c00 con 0x55c3c671e000 2024-09-17T02:52:35.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-09-17T02:52:35.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621000 con 0x55c3c671e000 2024-09-17T02:52:35.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.211 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-09-17T02:52:35.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621400 con 0x55c3c671e000 2024-09-17T02:52:35.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv14 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-09-17T02:52:35.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621800 con 0x55c3c671e000 2024-09-17T02:52:35.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-09-17T02:52:35.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620c00 con 0x55c3c671e000 2024-09-17T02:52:35.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-09-17T02:52:35.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-17T02:52:35.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-09-17T02:52:35.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-17T02:52:35.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-09-17T02:52:35.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-17T02:52:35.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-09-17T02:52:35.214 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.217+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621000 con 0x55c3c671e900 2024-09-17T02:52:35.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 5 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv37 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-09-17T02:52:35.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621400 con 0x55c3c671e900 2024-09-17T02:52:35.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-09-17T02:52:35.215 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621800 con 0x55c3c671e480 2024-09-17T02:52:35.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.7 v2:172.21.15.189:6810/3586470400 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671e480 2024-09-17T02:52:35.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-09-17T02:52:35.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-17T02:52:35.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-09-17T02:52:35.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621c00 con 0x55c3c671ed80 2024-09-17T02:52:35.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 6 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-09-17T02:52:35.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620800 con 0x55c3c671f200 2024-09-17T02:52:35.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.221+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.3 v2:172.21.15.131:6816/4047536388 5 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671f200 2024-09-17T02:52:35.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-09-17T02:52:35.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-17T02:52:35.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-09-17T02:52:35.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-17T02:52:35.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-09-17T02:52:35.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621c00 con 0x55c3c671f680 2024-09-17T02:52:35.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.1 v2:172.21.15.131:6824/1616383728 4 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671f680 2024-09-17T02:52:35.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-09-17T02:52:35.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-17T02:52:35.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-09-17T02:52:35.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299c00 con 0x55c3c671e000 2024-09-17T02:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 10 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-09-17T02:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-17T02:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-09-17T02:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e900 2024-09-17T02:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-09-17T02:52:35.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299800 con 0x55c3c671e900 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 20 datascan.scan_links: injected ino detected 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299c00 con 0x55c3c634cd00 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 8 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c634cd00 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 9 ==== osd_op_reply(48 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.222 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.225+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e000 2024-09-17T02:52:35.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 11 ==== osd_op_reply(49 [pgnls start_epoch 124 out=177b] v123'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c634cd00 2024-09-17T02:52:35.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 10 ==== osd_op_reply(50 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e480 2024-09-17T02:52:35.223 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.7 v2:172.21.15.189:6810/3586470400 5 ==== osd_op_reply(51 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671e480 2024-09-17T02:52:35.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671ed80 2024-09-17T02:52:35.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 7 ==== osd_op_reply(52 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.224 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e900 2024-09-17T02:52:35.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 9 ==== osd_op_reply(53 [pgnls start_epoch 124 out=149b] v124'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671f200 2024-09-17T02:52:35.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.3 v2:172.21.15.131:6816/4047536388 6 ==== osd_op_reply(54 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671f200 2024-09-17T02:52:35.225 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.229+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e900 2024-09-17T02:52:35.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 10 ==== osd_op_reply(55 [pgnls start_epoch 124 out=73b] v123'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671f680 2024-09-17T02:52:35.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.1 v2:172.21.15.131:6824/1616383728 5 ==== osd_op_reply(56 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671f680 2024-09-17T02:52:35.226 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e480 2024-09-17T02:52:35.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.7 v2:172.21.15.189:6810/3586470400 6 ==== osd_op_reply(57 [pgnls start_epoch 124 out=81b] v123'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671e480 2024-09-17T02:52:35.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671f680 2024-09-17T02:52:35.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.1 v2:172.21.15.131:6824/1616383728 6 ==== osd_op_reply(58 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671f680 2024-09-17T02:52:35.227 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e000 2024-09-17T02:52:35.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 12 ==== osd_op_reply(59 [pgnls start_epoch 124 out=49b] v124'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c634cd00 2024-09-17T02:52:35.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 11 ==== osd_op_reply(60 [pgnls start_epoch 124 out=77b] v123'10 uv10 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.228 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671ff80 2024-09-17T02:52:35.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.2 v2:172.21.15.131:6800/163801004 3 ==== osd_op_reply(61 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671ff80 2024-09-17T02:52:35.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671ed80 2024-09-17T02:52:35.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 8 ==== osd_op_reply(62 [pgnls start_epoch 124 out=73b] v123'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.229 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.233+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671ed80 2024-09-17T02:52:35.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 9 ==== osd_op_reply(63 [pgnls start_epoch 124 out=49b] v124'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.230 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e900 2024-09-17T02:52:35.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 11 ==== osd_op_reply(64 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e000 2024-09-17T02:52:35.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 13 ==== osd_op_reply(65 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.231 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e480 2024-09-17T02:52:35.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.7 v2:172.21.15.189:6810/3586470400 7 ==== osd_op_reply(66 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671e480 2024-09-17T02:52:35.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671f200 2024-09-17T02:52:35.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.3 v2:172.21.15.131:6816/4047536388 7 ==== osd_op_reply(67 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671f200 2024-09-17T02:52:35.232 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c634cd00 2024-09-17T02:52:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 12 ==== osd_op_reply(68 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671f200 2024-09-17T02:52:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.3 v2:172.21.15.131:6816/4047536388 8 ==== osd_op_reply(69 [pgnls start_epoch 124 out=97b] v123'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671f200 2024-09-17T02:52:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c634cd00 2024-09-17T02:52:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 13 ==== osd_op_reply(70 [pgnls start_epoch 124 out=74b] v121'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.237+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671f680 2024-09-17T02:52:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.1 v2:172.21.15.131:6824/1616383728 7 ==== osd_op_reply(71 [pgnls start_epoch 124 out=73b] v123'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671f680 2024-09-17T02:52:35.234 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c634cd00 2024-09-17T02:52:35.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 14 ==== osd_op_reply(72 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e000 2024-09-17T02:52:35.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 14 ==== osd_op_reply(73 [pgnls start_epoch 124 out=98b] v123'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.235 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671ed80 2024-09-17T02:52:35.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 10 ==== osd_op_reply(74 [pgnls start_epoch 124 out=76b] v124'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671f200 2024-09-17T02:52:35.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.3 v2:172.21.15.131:6816/4047536388 9 ==== osd_op_reply(75 [pgnls start_epoch 124 out=49b] v124'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671f200 2024-09-17T02:52:35.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671e900 2024-09-17T02:52:35.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 12 ==== osd_op_reply(76 [pgnls start_epoch 124 out=105b] v124'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671ff80 2024-09-17T02:52:35.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.2 v2:172.21.15.131:6800/163801004 4 ==== osd_op_reply(77 [pgnls start_epoch 124 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671ff80 2024-09-17T02:52:35.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c671ed80 2024-09-17T02:52:35.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 11 ==== osd_op_reply(78 [pgnls start_epoch 124 out=73b] v124'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.237 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.241+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 124 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299400 con 0x55c3c634cd00 2024-09-17T02:52:35.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 15 ==== osd_op_reply(79 [pgnls start_epoch 124 out=101b] v124'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-09-17T02:52:35.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620c00 con 0x55c3c671e000 2024-09-17T02:52:35.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 15 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-09-17T02:52:35.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621000 con 0x55c3c671e000 2024-09-17T02:52:35.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 16 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-09-17T02:52:35.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621400 con 0x55c3c671e000 2024-09-17T02:52:35.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 17 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv14 ondisk = 0) ==== 164+0+1047 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-09-17T02:52:35.240 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6752000 con 0x55c3c671e000 2024-09-17T02:52:35.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 18 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-09-17T02:52:35.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6752400 con 0x55c3c671e000 2024-09-17T02:52:35.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.245+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 19 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-09-17T02:52:35.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-17T02:52:35.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-09-17T02:52:35.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-17T02:52:35.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-09-17T02:52:35.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-17T02:52:35.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-09-17T02:52:35.242 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6752800 con 0x55c3c671e900 2024-09-17T02:52:35.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 13 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv37 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-09-17T02:52:35.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6752c00 con 0x55c3c671e900 2024-09-17T02:52:35.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 14 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-09-17T02:52:35.243 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6753000 con 0x55c3c671e480 2024-09-17T02:52:35.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.7 v2:172.21.15.189:6810/3586470400 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) ==== 164+0+523 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671e480 2024-09-17T02:52:35.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-09-17T02:52:35.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-17T02:52:35.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-09-17T02:52:35.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6753400 con 0x55c3c671ed80 2024-09-17T02:52:35.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.6 v2:172.21.15.189:6802/952142225 12 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671ed80 2024-09-17T02:52:35.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-09-17T02:52:35.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6753800 con 0x55c3c671f200 2024-09-17T02:52:35.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac71f55640 1 -- 172.21.15.131:0/2586463499 <== osd.3 v2:172.21.15.131:6816/4047536388 10 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4780 con 0x55c3c671f200 2024-09-17T02:52:35.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-09-17T02:52:35.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-17T02:52:35.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-09-17T02:52:35.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-17T02:52:35.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-09-17T02:52:35.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.249+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6753c00 con 0x55c3c671f680 2024-09-17T02:52:35.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.1 v2:172.21.15.131:6824/1616383728 8 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c671f680 2024-09-17T02:52:35.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-09-17T02:52:35.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-17T02:52:35.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-09-17T02:52:35.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6752000 con 0x55c3c671e000 2024-09-17T02:52:35.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 20 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-09-17T02:52:35.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-17T02:52:35.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-09-17T02:52:35.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6752400 con 0x55c3c671e900 2024-09-17T02:52:35.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-09-17T02:52:35.248 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620c00 con 0x55c3c671e900 2024-09-17T02:52:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) ==== 164+0+1545 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 20 datascan.scan_links: injected ino detected 2024-09-17T02:52:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-09-17T02:52:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-09-17T02:52:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-17T02:52:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-09-17T02:52:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-17T02:52:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-09-17T02:52:35.249 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621000 con 0x55c3c634cd00 2024-09-17T02:52:35.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 16 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) ==== 154+0+524 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-09-17T02:52:35.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-09-17T02:52:35.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-09-17T02:52:35.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: processing 1 injected_inos 2024-09-17T02:52:35.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-09-17T02:52:35.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-09-17T02:52:35.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6621400 con 0x55c3c671e900 2024-09-17T02:52:35.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-09-17T02:52:35.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 20 datascan.scan_links: first is now 2 2024-09-17T02:52:35.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.253+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55c3c2299800 con 0x55c3c671e900 2024-09-17T02:52:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.257+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.5 v2:172.21.15.189:6826/3175183672 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v124'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e900 2024-09-17T02:52:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.257+0000 7fac737968c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-09-17T02:52:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.257+0000 7fac737968c0 10 datascan.scan_links: updating inotable 2024-09-17T02:52:35.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.257+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6620c00 con 0x55c3c671e000 2024-09-17T02:52:35.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.257+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.257+0000 7fac737968c0 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-09-17T02:52:35.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.257+0000 7fac737968c0 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-09-17T02:52:35.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.257+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6752800 con 0x55c3c671e000 2024-09-17T02:52:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac72756640 1 -- 172.21.15.131:0/2586463499 <== osd.4 v2:172.21.15.189:6818/1965940492 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v124'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x55c3c63d4500 con 0x55c3c671e000 2024-09-17T02:52:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 10 datascan.scan_links: updating snaptable 2024-09-17T02:52:35.255 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) -- 0x55c3c6752c00 con 0x55c3c634cd00 2024-09-17T02:52:35.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 <== osd.0 v2:172.21.15.131:6808/1154840320 17 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x55c3c63d4000 con 0x55c3c634cd00 2024-09-17T02:52:35.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55c3c634cd00 msgr2=0x55c3c6444680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x55c3c634cd00 0x55c3c6444680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55c3c671f680 msgr2=0x55c3c6723b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x55c3c671f680 0x55c3c6723b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.257 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55c3c671ff80 msgr2=0x55c3c6724680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x55c3c671ff80 0x55c3c6724680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55c3c671f200 msgr2=0x55c3c6723600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x55c3c671f200 0x55c3c6723600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55c3c671e000 msgr2=0x55c3c6722000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x55c3c671e000 0x55c3c6722000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55c3c671e900 msgr2=0x55c3c6722b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x55c3c671e900 0x55c3c6722b00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55c3c671ed80 msgr2=0x55c3c6723080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x55c3c671ed80 0x55c3c6723080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55c3c671e480 msgr2=0x55c3c6722580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x55c3c671e480 0x55c3c6722580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.258 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 reap_dead start 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 reap_dead start 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 reap_dead start 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac71754640 1 -- 172.21.15.131:0/2586463499 reap_dead start 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55c3c634da80 msgr2=0x55c3c6444100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55c3c634da80 0x55c3c6444100 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x55c3c71f1530 tx=0x55c3c671c0c0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 msgr2=0x55c3c6443600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x55c3c66126f0 tx=0x55c3c7214c00 comp rx=0 tx=0).stop 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 shutdown_connections 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x55c3c634da80 0x55c3c6444100 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2586463499 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634a900 0x55c3c6443600 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 >> 172.21.15.131:0/2586463499 conn(0x55c3c634b200 msgr2=0x55c3c65fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 shutdown_connections 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2586463499 wait complete. 2024-09-17T02:52:35.259 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2097123872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634b680 msgr2=0x55c3c6442580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2097123872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634b680 0x55c3c6442580 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x55c3c71f00c0 tx=0x55c3c63e91a0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2097123872 shutdown_connections 2024-09-17T02:52:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2097123872 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x55c3c634bf80 0x55c3c6442b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2097123872 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x55c3c634bb00 0x55c3c6442000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 --2- 172.21.15.131:0/2097123872 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x55c3c634b680 0x55c3c6442580 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2097123872 >> 172.21.15.131:0/2097123872 conn(0x55c3c634c400 msgr2=0x55c3c22c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2097123872 shutdown_connections 2024-09-17T02:52:35.260 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.261+0000 7fac737968c0 1 -- 172.21.15.131:0/2097123872 wait complete. 2024-09-17T02:52:35.263 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.144240, result: 2024-09-17T02:52:35.264 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T02:52:35.266 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T02:52:35.286 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-09-17T02:52:35.286 INFO:teuthology.orchestra.run.smithi131.stdout: chdir = "" 2024-09-17T02:52:35.286 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T02:52:35.286 INFO:teuthology.orchestra.run.smithi131.stdout: auth supported = cephx 2024-09-17T02:52:35.286 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.286 INFO:teuthology.orchestra.run.smithi131.stdout: filestore xattr use omap = true 2024-09-17T02:52:35.286 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.286 INFO:teuthology.orchestra.run.smithi131.stdout: mon clock drift allowed = 1.000 2024-09-17T02:52:35.286 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.286 INFO:teuthology.orchestra.run.smithi131.stdout: osd crush chooseleaf type = 0 2024-09-17T02:52:35.286 INFO:teuthology.orchestra.run.smithi131.stdout: auth debug = true 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on old message = true 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on bug = true 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: mon pg warn max object skew = 0 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default size = 2 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow primary affinity = true 2024-09-17T02:52:35.287 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow pg remap = true 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on legacy crush tunables = false 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on crush straw calc version zero = false 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on no sortbitwise = false 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on osd down out interval zero = false 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on too few osds = false 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: mon_allow_pool_size_one = true 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: osd default data pool replay window = 5 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.288 INFO:teuthology.orchestra.run.smithi131.stdout: mon allow pool delete = true 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: mon cluster log file level = debug 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: debug asserts on shutdown = true 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: mon health detail to clog = false 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: mon host = "172.21.15.131,172.21.15.189,[v2:172.21.15.189:3301,v1:172.21.15.189:6790]" 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout:[osd] 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: osd journal size = 100 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub load threshold = 5.0 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub max interval = 600 2024-09-17T02:52:35.289 INFO:teuthology.orchestra.run.smithi131.stdout: osd mclock profile = high_recovery_ops 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: osd recover clone overlap = true 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: osd recovery max chunk = 1048576 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug shutdown = true 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug op order = true 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify stray on activate = true 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug trim objects = true 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: osd open classes on start = true 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug pg log writeout = true 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: osd deep scrub update digest min age = 30 2024-09-17T02:52:35.290 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: osd map max advance = 10 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: journal zero on create = true 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: filestore ondisk finisher threads = 3 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: filestore apply finisher threads = 3 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: bdev debug aio = true 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug misdirected ops = true 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore block size = 96636764160 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore fsck on mount = True 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluefs = 20 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluestore = 20 2024-09-17T02:52:35.291 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: debug osd = 20 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: debug rocksdb = 10 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd full ratio = 0.9 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd nearfull ratio = 0.8 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: osd failsafe full ratio = 0.95 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: osd objectstore = bluestore 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr] 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: debug mgr = 20 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:35.292 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: mgr/telemetry/nag = false 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout:[mon] 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: debug paxos = 20 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: mon data avail warn = 5 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: mon mgr mkfs grace = 240 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd reporter subtree level = osd 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd prime pg temp = true 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:35.293 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: # 1m isn't quite enough 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: mon_down_mkfs_grace = 2m 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_filestore_osds = false 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: mon down mkfs grace = 300 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.294 INFO:teuthology.orchestra.run.smithi131.stdout:[client] 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout: rgw cache enabled = true 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable ops log = true 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable usage log = true 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = "" 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.a] 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.b] 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.c] 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2024-09-17T02:52:35.295 INFO:teuthology.orchestra.run.smithi131.stdout:[mds] 2024-09-17T02:52:35.296 INFO:teuthology.orchestra.run.smithi131.stdout: mds verify scatter = True 2024-09-17T02:52:35.296 INFO:teuthology.orchestra.run.smithi131.stdout: mds debug scatterstat = False 2024-09-17T02:52:35.298 INFO:teuthology.orchestra.run.smithi189.stdout:[global] 2024-09-17T02:52:35.298 INFO:teuthology.orchestra.run.smithi189.stdout: chdir = "" 2024-09-17T02:52:35.298 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: auth supported = cephx 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: filestore xattr use omap = true 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: mon clock drift allowed = 1.000 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: osd crush chooseleaf type = 0 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: auth debug = true 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on old message = true 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on bug = true 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T02:52:35.299 INFO:teuthology.orchestra.run.smithi189.stdout: mon pg warn max object skew = 0 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default size = 2 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow primary affinity = true 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow pg remap = true 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on legacy crush tunables = false 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on crush straw calc version zero = false 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on no sortbitwise = false 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on osd down out interval zero = false 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on too few osds = false 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T02:52:35.300 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: mon_allow_pool_size_one = true 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: osd default data pool replay window = 5 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: mon allow pool delete = true 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: mon cluster log file level = debug 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: debug asserts on shutdown = true 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: mon health detail to clog = false 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: mon host = "172.21.15.131,172.21.15.189,[v2:172.21.15.189:3301,v1:172.21.15.189:6790]" 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: fsid = f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:52:35.301 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout:[osd] 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: osd journal size = 100 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub load threshold = 5.0 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub max interval = 600 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: osd mclock profile = high_recovery_ops 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: osd recover clone overlap = true 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: osd recovery max chunk = 1048576 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug shutdown = true 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug op order = true 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug verify stray on activate = true 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug trim objects = true 2024-09-17T02:52:35.302 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: osd open classes on start = true 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug pg log writeout = true 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: osd deep scrub update digest min age = 30 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: osd map max advance = 10 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: journal zero on create = true 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: filestore ondisk finisher threads = 3 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: filestore apply finisher threads = 3 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: bdev debug aio = true 2024-09-17T02:52:35.303 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug misdirected ops = true 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore block size = 96636764160 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore fsck on mount = True 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluefs = 20 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluestore = 20 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: debug osd = 20 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: debug rocksdb = 10 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd full ratio = 0.9 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd nearfull ratio = 0.8 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: osd failsafe full ratio = 0.95 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: osd objectstore = bluestore 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr] 2024-09-17T02:52:35.304 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: debug mgr = 20 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: mgr/telemetry/nag = false 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout:[mon] 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: debug paxos = 20 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: mon data avail warn = 5 2024-09-17T02:52:35.305 INFO:teuthology.orchestra.run.smithi189.stdout: mon mgr mkfs grace = 240 2024-09-17T02:52:35.306 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:35.306 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd reporter subtree level = osd 2024-09-17T02:52:35.306 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd prime pg temp = true 2024-09-17T02:52:35.306 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:35.306 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.306 INFO:teuthology.orchestra.run.smithi189.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T02:52:35.308 INFO:teuthology.orchestra.run.smithi189.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T02:52:35.308 INFO:teuthology.orchestra.run.smithi189.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T02:52:35.308 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.308 INFO:teuthology.orchestra.run.smithi189.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T02:52:35.308 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T02:52:35.308 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T02:52:35.308 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.308 INFO:teuthology.orchestra.run.smithi189.stdout: # 1m isn't quite enough 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout: mon_down_mkfs_grace = 2m 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_filestore_osds = false 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout: mon down mkfs grace = 300 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout:[client] 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout: rgw cache enabled = true 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable ops log = true 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable usage log = true 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = "" 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.a] 2024-09-17T02:52:35.309 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.b] 2024-09-17T02:52:35.310 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.c] 2024-09-17T02:52:35.310 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.a] 2024-09-17T02:52:35.310 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.c] 2024-09-17T02:52:35.310 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.b] 2024-09-17T02:52:35.310 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.d] 2024-09-17T02:52:35.310 INFO:teuthology.orchestra.run.smithi189.stdout:[mds] 2024-09-17T02:52:35.310 INFO:teuthology.orchestra.run.smithi189.stdout: mds verify scatter = True 2024-09-17T02:52:35.310 INFO:teuthology.orchestra.run.smithi189.stdout: mds debug scatterstat = False 2024-09-17T02:52:35.312 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T02:52:35.339 DEBUG:teuthology.orchestra.run.smithi189:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T02:52:35.362 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-09-17T02:52:35.362 INFO:teuthology.orchestra.run.smithi131.stdout: chdir = "" 2024-09-17T02:52:35.362 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T02:52:35.362 INFO:teuthology.orchestra.run.smithi131.stdout: auth supported = cephx 2024-09-17T02:52:35.362 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.362 INFO:teuthology.orchestra.run.smithi131.stdout: filestore xattr use omap = true 2024-09-17T02:52:35.362 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: mon clock drift allowed = 1.000 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: osd crush chooseleaf type = 0 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: auth debug = true 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on old message = true 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: ms die on bug = true 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: mon pg warn max object skew = 0 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.363 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default size = 2 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow primary affinity = true 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd allow pg remap = true 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on legacy crush tunables = false 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on crush straw calc version zero = false 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on no sortbitwise = false 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on osd down out interval zero = false 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: mon warn on too few osds = false 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: mon_allow_pool_size_one = true 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T02:52:35.364 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: osd default data pool replay window = 5 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: mon allow pool delete = true 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: mon cluster log file level = debug 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: debug asserts on shutdown = true 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: mon health detail to clog = false 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: mon host = "172.21.15.131,172.21.15.189,[v2:172.21.15.189:3301,v1:172.21.15.189:6790]" 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout:[osd] 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: osd journal size = 100 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.365 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub load threshold = 5.0 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: osd scrub max interval = 600 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: osd mclock profile = high_recovery_ops 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: osd recover clone overlap = true 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: osd recovery max chunk = 1048576 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug shutdown = true 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug op order = true 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug verify stray on activate = true 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug trim objects = true 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: osd open classes on start = true 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug pg log writeout = true 2024-09-17T02:52:35.366 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: osd deep scrub update digest min age = 30 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: osd map max advance = 10 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: journal zero on create = true 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: filestore ondisk finisher threads = 3 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: filestore apply finisher threads = 3 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: bdev debug aio = true 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: osd debug misdirected ops = true 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore block size = 96636764160 2024-09-17T02:52:35.367 INFO:teuthology.orchestra.run.smithi131.stdout: bluestore fsck on mount = True 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluefs = 20 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: debug bluestore = 20 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: debug osd = 20 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: debug rocksdb = 10 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd full ratio = 0.9 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd nearfull ratio = 0.8 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: osd failsafe full ratio = 0.95 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: osd objectstore = bluestore 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout:[mgr] 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: debug mgr = 20 2024-09-17T02:52:35.368 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: mgr/telemetry/nag = false 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout:[mon] 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: debug ms = 1 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: debug mon = 20 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: debug paxos = 20 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: debug auth = 20 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: mon data avail warn = 5 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: mon mgr mkfs grace = 240 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:35.369 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd reporter subtree level = osd 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: mon osd prime pg temp = true 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: # 1m isn't quite enough 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: mon_down_mkfs_grace = 2m 2024-09-17T02:52:35.370 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: mon_warn_on_filestore_osds = false 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: mon down mkfs grace = 300 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout:[client] 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: rgw cache enabled = true 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable ops log = true 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: rgw enable usage log = true 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout: pid file = "" 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.a] 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.b] 2024-09-17T02:52:35.371 INFO:teuthology.orchestra.run.smithi131.stdout:[mon.c] 2024-09-17T02:52:35.372 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.a] 2024-09-17T02:52:35.372 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.c] 2024-09-17T02:52:35.372 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.b] 2024-09-17T02:52:35.372 INFO:teuthology.orchestra.run.smithi131.stdout:[mds.d] 2024-09-17T02:52:35.372 INFO:teuthology.orchestra.run.smithi131.stdout:[mds] 2024-09-17T02:52:35.372 INFO:teuthology.orchestra.run.smithi131.stdout: mds verify scatter = True 2024-09-17T02:52:35.372 INFO:teuthology.orchestra.run.smithi131.stdout: mds debug scatterstat = True 2024-09-17T02:52:35.383 INFO:teuthology.orchestra.run.smithi189.stdout:[global] 2024-09-17T02:52:35.383 INFO:teuthology.orchestra.run.smithi189.stdout: chdir = "" 2024-09-17T02:52:35.383 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T02:52:35.383 INFO:teuthology.orchestra.run.smithi189.stdout: auth supported = cephx 2024-09-17T02:52:35.383 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.383 INFO:teuthology.orchestra.run.smithi189.stdout: filestore xattr use omap = true 2024-09-17T02:52:35.383 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.383 INFO:teuthology.orchestra.run.smithi189.stdout: mon clock drift allowed = 1.000 2024-09-17T02:52:35.383 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: osd crush chooseleaf type = 0 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: auth debug = true 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on old message = true 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: ms die on bug = true 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: mon pg warn max object skew = 0 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default size = 2 2024-09-17T02:52:35.384 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow primary affinity = true 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd allow pg remap = true 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on legacy crush tunables = false 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on crush straw calc version zero = false 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on no sortbitwise = false 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on osd down out interval zero = false 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: mon warn on too few osds = false 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: mon_allow_pool_size_one = true 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T02:52:35.385 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: osd default data pool replay window = 5 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: mon allow pool delete = true 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: mon cluster log file level = debug 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: debug asserts on shutdown = true 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: mon health detail to clog = false 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: mon host = "172.21.15.131,172.21.15.189,[v2:172.21.15.189:3301,v1:172.21.15.189:6790]" 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: fsid = f03002b8-e723-4674-9385-a37ff83176c9 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout:[osd] 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: osd journal size = 100 2024-09-17T02:52:35.386 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub load threshold = 5.0 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: osd scrub max interval = 600 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: osd mclock profile = high_recovery_ops 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: osd recover clone overlap = true 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: osd recovery max chunk = 1048576 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug shutdown = true 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug op order = true 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug verify stray on activate = true 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug trim objects = true 2024-09-17T02:52:35.387 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: osd open classes on start = true 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug pg log writeout = true 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: osd deep scrub update digest min age = 30 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: osd map max advance = 10 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: journal zero on create = true 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: filestore ondisk finisher threads = 3 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: filestore apply finisher threads = 3 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: bdev debug aio = true 2024-09-17T02:52:35.388 INFO:teuthology.orchestra.run.smithi189.stdout: osd debug misdirected ops = true 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore block size = 96636764160 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: bluestore fsck on mount = True 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluefs = 20 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: debug bluestore = 20 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: debug osd = 20 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: debug rocksdb = 10 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd full ratio = 0.9 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd nearfull ratio = 0.8 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: osd failsafe full ratio = 0.95 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: osd objectstore = bluestore 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout:[mgr] 2024-09-17T02:52:35.389 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:35.390 INFO:teuthology.orchestra.run.smithi189.stdout: debug mgr = 20 2024-09-17T02:52:35.390 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-17T02:52:35.390 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-17T02:52:35.390 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:35.390 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:35.390 INFO:teuthology.orchestra.run.smithi189.stdout: mgr/telemetry/nag = false 2024-09-17T02:52:35.390 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.390 INFO:teuthology.orchestra.run.smithi189.stdout:[mon] 2024-09-17T02:52:35.390 INFO:teuthology.orchestra.run.smithi189.stdout: debug ms = 1 2024-09-17T02:52:35.390 INFO:teuthology.orchestra.run.smithi189.stdout: debug mon = 20 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: debug paxos = 20 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: debug auth = 20 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: mon data avail warn = 5 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: mon mgr mkfs grace = 240 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min pgs per osd = 4 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd reporter subtree level = osd 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: mon osd prime pg temp = true 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: mon reweight min bytes per osd = 10 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T02:52:35.391 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: # 1m isn't quite enough 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: mon_down_mkfs_grace = 2m 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: mon_warn_on_filestore_osds = false 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: mon down mkfs grace = 300 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout:[client] 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: rgw cache enabled = true 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable ops log = true 2024-09-17T02:52:35.392 INFO:teuthology.orchestra.run.smithi189.stdout: rgw enable usage log = true 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout: pid file = "" 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.a] 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.b] 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout:[mon.c] 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.a] 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.c] 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.b] 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout:[mds.d] 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout:[mds] 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout: mds verify scatter = True 2024-09-17T02:52:35.393 INFO:teuthology.orchestra.run.smithi189.stdout: mds debug scatterstat = True 2024-09-17T02:52:35.394 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-17T02:52:35.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.573+0000 7f9290d91640 1 -- 172.21.15.131:0/2039497925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c0cfdd0 msgr2=0x7f928c0d1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.573+0000 7f9290d91640 1 --2- 172.21.15.131:0/2039497925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c0cfdd0 0x7f928c0d1fe0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9284009bc0 tx=0x7f928401cb10 comp rx=0 tx=0).stop 2024-09-17T02:52:35.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.573+0000 7f9290d91640 1 -- 172.21.15.131:0/2039497925 shutdown_connections 2024-09-17T02:52:35.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.573+0000 7f9290d91640 1 --2- 172.21.15.131:0/2039497925 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c0cfdd0 0x7f928c0d1fe0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.573+0000 7f9290d91640 1 --2- 172.21.15.131:0/2039497925 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f928c0d11a0 0x7f928c0cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.573+0000 7f9290d91640 1 -- 172.21.15.131:0/2039497925 >> 172.21.15.131:0/2039497925 conn(0x7f928c09d700 msgr2=0x7f928c0d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:35.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.573+0000 7f9290d91640 1 -- 172.21.15.131:0/2039497925 shutdown_connections 2024-09-17T02:52:35.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.573+0000 7f9290d91640 1 -- 172.21.15.131:0/2039497925 wait complete. 2024-09-17T02:52:35.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9290d91640 1 Processor -- start 2024-09-17T02:52:35.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9290d91640 1 -- start start 2024-09-17T02:52:35.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9290d91640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c0cfdd0 0x7f928c151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9290d91640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f928c0d0d90 0x7f928c153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9290d91640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f928c0d11a0 0x7f928c15cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9290d91640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f928c16be20 con 0x7f928c0d11a0 2024-09-17T02:52:35.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9290d91640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f928c16bca0 con 0x7f928c0cfdd0 2024-09-17T02:52:35.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9290d91640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f928c16bfa0 con 0x7f928c0d0d90 2024-09-17T02:52:35.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f928ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f928c0d11a0 0x7f928c15cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f928ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f928c0d11a0 0x7f928c15cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40820/0 (socket says 172.21.15.131:40820) 2024-09-17T02:52:35.573 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f928ad9d640 1 -- 172.21.15.131:0/2372585446 learned_addr learned my addr 172.21.15.131:0/2372585446 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f928a59c640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c0cfdd0 0x7f928c151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9283fff640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f928c0d0d90 0x7f928c153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f928ad9d640 1 -- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f928c0d0d90 msgr2=0x7f928c153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f928ad9d640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f928c0d0d90 0x7f928c153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f928ad9d640 1 -- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c0cfdd0 msgr2=0x7f928c151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f928ad9d640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c0cfdd0 0x7f928c151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f928ad9d640 1 -- 172.21.15.131:0/2372585446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928c151be0 con 0x7f928c0d11a0 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f928a59c640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c0cfdd0 0x7f928c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f928ad9d640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f928c0d11a0 0x7f928c15cfc0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f928401a740 tx=0x7f9284009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9282ffd640 1 -- 172.21.15.131:0/2372585446 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92840bb070 con 0x7f928c0d11a0 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9290d91640 1 -- 172.21.15.131:0/2372585446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928c159ae0 con 0x7f928c0d11a0 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9282ffd640 1 -- 172.21.15.131:0/2372585446 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9284004d40 con 0x7f928c0d11a0 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9282ffd640 1 -- 172.21.15.131:0/2372585446 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92840b2040 con 0x7f928c0d11a0 2024-09-17T02:52:35.574 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9290d91640 1 -- 172.21.15.131:0/2372585446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928c15a0a0 con 0x7f928c0d11a0 2024-09-17T02:52:35.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.577+0000 7f9290d91640 1 -- 172.21.15.131:0/2372585446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f928c151950 con 0x7f928c0d11a0 2024-09-17T02:52:35.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.581+0000 7f9282ffd640 1 -- 172.21.15.131:0/2372585446 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9284004260 con 0x7f928c0d11a0 2024-09-17T02:52:35.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.581+0000 7f9282ffd640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f926c073200 0x7f926c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:35.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.581+0000 7f9282ffd640 1 -- 172.21.15.131:0/2372585446 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7f928412e7e0 con 0x7f928c0d11a0 2024-09-17T02:52:35.575 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.581+0000 7f928a59c640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f926c073200 0x7f926c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:35.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.581+0000 7f928a59c640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f926c073200 0x7f926c075710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f927c0016a0 tx=0x7f927c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:35.578 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.585+0000 7f9282ffd640 1 -- 172.21.15.131:0/2372585446 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f92840c4020 con 0x7f928c0d11a0 2024-09-17T02:52:35.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:35.789+0000 7f9290d91640 1 -- 172.21.15.131:0/2372585446 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f928c0d1fe0 con 0x7f928c0d11a0 2024-09-17T02:52:36.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.069+0000 7f9282ffd640 1 -- 172.21.15.131:0/2372585446 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v110) ==== 162+0+0 (secure 0 0 0) 0x7f9284016790 con 0x7f928c0d11a0 2024-09-17T02:52:36.066 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-17T02:52:36.068 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 -- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f926c073200 msgr2=0x7f926c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:36.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f926c073200 0x7f926c075710 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f927c0016a0 tx=0x7f927c009040 comp rx=0 tx=0).stop 2024-09-17T02:52:36.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 -- 172.21.15.131:0/2372585446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f928c0d11a0 msgr2=0x7f928c15cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:36.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f928c0d11a0 0x7f928c15cfc0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f928401a740 tx=0x7f9284009260 comp rx=0 tx=0).stop 2024-09-17T02:52:36.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 -- 172.21.15.131:0/2372585446 shutdown_connections 2024-09-17T02:52:36.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f926c073200 0x7f926c075710 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f928c0d11a0 0x7f928c15cfc0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f928c0d0d90 0x7f928c153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 --2- 172.21.15.131:0/2372585446 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f928c0cfdd0 0x7f928c151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 -- 172.21.15.131:0/2372585446 >> 172.21.15.131:0/2372585446 conn(0x7f928c09d700 msgr2=0x7f928c158910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:36.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 -- 172.21.15.131:0/2372585446 shutdown_connections 2024-09-17T02:52:36.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.073+0000 7f9290d91640 1 -- 172.21.15.131:0/2372585446 wait complete. 2024-09-17T02:52:36.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:36.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.249+0000 7f2ed5f19640 1 -- 172.21.15.131:0/4069247999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed00b61d0 msgr2=0x7f2ed0073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:36.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.249+0000 7f2ed5f19640 1 --2- 172.21.15.131:0/4069247999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed00b61d0 0x7f2ed0073e10 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8009c80 tx=0x7f2ec801cd70 comp rx=0 tx=0).stop 2024-09-17T02:52:36.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.249+0000 7f2ed5f19640 1 -- 172.21.15.131:0/4069247999 shutdown_connections 2024-09-17T02:52:36.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.249+0000 7f2ed5f19640 1 --2- 172.21.15.131:0/4069247999 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed00b61d0 0x7f2ed0073e10 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.249+0000 7f2ed5f19640 1 --2- 172.21.15.131:0/4069247999 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ed0155980 0x7f2ed0071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.249+0000 7f2ed5f19640 1 -- 172.21.15.131:0/4069247999 >> 172.21.15.131:0/4069247999 conn(0x7f2ed00b2dc0 msgr2=0x7f2ed00b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.249+0000 7f2ed5f19640 1 -- 172.21.15.131:0/4069247999 shutdown_connections 2024-09-17T02:52:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.249+0000 7f2ed5f19640 1 -- 172.21.15.131:0/4069247999 wait complete. 2024-09-17T02:52:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ed5f19640 1 Processor -- start 2024-09-17T02:52:36.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ed5f19640 1 -- start start 2024-09-17T02:52:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ed5f19640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ed0155980 0x7f2ed01456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ed5f19640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed015c360 0x7f2ed015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ed5f19640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed014e420 0x7f2ed01544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ed5f19640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed0169360 con 0x7f2ed014e420 2024-09-17T02:52:36.250 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ed5f19640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed01691e0 con 0x7f2ed015c360 2024-09-17T02:52:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ed5f19640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed01694e0 con 0x7f2ed0155980 2024-09-17T02:52:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ecffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed014e420 0x7f2ed01544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ecffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed014e420 0x7f2ed01544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40838/0 (socket says 172.21.15.131:40838) 2024-09-17T02:52:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ecffff640 1 -- 172.21.15.131:0/532472317 learned_addr learned my addr 172.21.15.131:0/532472317 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ecf7fe640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ed0155980 0x7f2ed01456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2eceffd640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed015c360 0x7f2ed015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ecffff640 1 -- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ed0155980 msgr2=0x7f2ed01456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ecffff640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ed0155980 0x7f2ed01456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ecffff640 1 -- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed015c360 msgr2=0x7f2ed015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:36.251 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ecffff640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed015c360 0x7f2ed015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ecffff640 1 -- 172.21.15.131:0/532472317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed01461e0 con 0x7f2ed014e420 2024-09-17T02:52:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ecffff640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed014e420 0x7f2ed01544b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f2ec801b420 tx=0x7f2ec8009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2eb7fff640 1 -- 172.21.15.131:0/532472317 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec80bb070 con 0x7f2ed014e420 2024-09-17T02:52:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2eb7fff640 1 -- 172.21.15.131:0/532472317 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ec8004d40 con 0x7f2ed014e420 2024-09-17T02:52:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2eb7fff640 1 -- 172.21.15.131:0/532472317 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec80b2040 con 0x7f2ed014e420 2024-09-17T02:52:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ed5f19640 1 -- 172.21.15.131:0/532472317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed0156560 con 0x7f2ed014e420 2024-09-17T02:52:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.253+0000 7f2ed5f19640 1 -- 172.21.15.131:0/532472317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed014dc20 con 0x7f2ed014e420 2024-09-17T02:52:36.252 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.257+0000 7f2ed5f19640 1 -- 172.21.15.131:0/532472317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed0073e10 con 0x7f2ed014e420 2024-09-17T02:52:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.257+0000 7f2eb7fff640 1 -- 172.21.15.131:0/532472317 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2ec80ad850 con 0x7f2ed014e420 2024-09-17T02:52:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.257+0000 7f2eb7fff640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ebc073200 0x7f2ebc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.257+0000 7f2eb7fff640 1 -- 172.21.15.131:0/532472317 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7f2ec812e3d0 con 0x7f2ed014e420 2024-09-17T02:52:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.257+0000 7f2ecf7fe640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ebc073200 0x7f2ebc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:36.253 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.257+0000 7f2ecf7fe640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ebc073200 0x7f2ebc075710 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40058d0 tx=0x7f2ec4005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:36.256 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.261+0000 7f2eb7fff640 1 -- 172.21.15.131:0/532472317 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2ed01461e0 con 0x7f2ed014e420 2024-09-17T02:52:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.461+0000 7f2ed5f19640 1 -- 172.21.15.131:0/532472317 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ed00d0ee0 con 0x7f2ed014e420 2024-09-17T02:52:36.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2eb7fff640 1 -- 172.21.15.131:0/532472317 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5232 (secure 0 0 0) 0x7f2ec8016990 con 0x7f2ed014e420 2024-09-17T02:52:36.459 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:36.459 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":110,"btime":"2024-09-17T02:52:36:070976+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:52:36.070939+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8880":{"gid":8880,"name":"a","rank":0,"incarnation":110,"state":"up:replay","state_seq":1,"addr":"172.21.15.131:6835/4011259083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4011259083},{"type":"v1","addr":"172.21.15.131:6835","nonce":4011259083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-17T02:52:36.459 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 110 2024-09-17T02:52:36.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 -- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ebc073200 msgr2=0x7f2ebc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:36.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ebc073200 0x7f2ebc075710 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40058d0 tx=0x7f2ec4005860 comp rx=0 tx=0).stop 2024-09-17T02:52:36.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 -- 172.21.15.131:0/532472317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed014e420 msgr2=0x7f2ed01544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:36.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed014e420 0x7f2ed01544b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f2ec801b420 tx=0x7f2ec8009350 comp rx=0 tx=0).stop 2024-09-17T02:52:36.461 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 -- 172.21.15.131:0/532472317 shutdown_connections 2024-09-17T02:52:36.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2ebc073200 0x7f2ebc075710 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ed014e420 0x7f2ed01544b0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ed015c360 0x7f2ed015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 --2- 172.21.15.131:0/532472317 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ed0155980 0x7f2ed01456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:36.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 -- 172.21.15.131:0/532472317 >> 172.21.15.131:0/532472317 conn(0x7f2ed00b2dc0 msgr2=0x7f2ed00b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:36.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 -- 172.21.15.131:0/532472317 shutdown_connections 2024-09-17T02:52:36.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:36.465+0000 7f2ed5f19640 1 -- 172.21.15.131:0/532472317 wait complete. 2024-09-17T02:52:36.883 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:51:54.764322+0000', 'modified': '2024-09-17T02:52:36.070939+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 124, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8880}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8880': {'gid': 8880, 'name': 'a', 'rank': 0, 'incarnation': 110, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.131:6835/4011259083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 4011259083}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 4011259083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:52:36.883 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8880:up:replay 2024-09-17T02:52:37.884 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:38.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.049+0000 7f75a47e8640 1 -- 172.21.15.131:0/878793005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c0d0440 msgr2=0x7f759c15ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:38.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.049+0000 7f75a47e8640 1 --2- 172.21.15.131:0/878793005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c0d0440 0x7f759c15ba80 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f7590009900 tx=0x7f759001c700 comp rx=0 tx=0).stop 2024-09-17T02:52:38.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.049+0000 7f75a47e8640 1 -- 172.21.15.131:0/878793005 shutdown_connections 2024-09-17T02:52:38.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.049+0000 7f75a47e8640 1 --2- 172.21.15.131:0/878793005 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c0d0440 0x7f759c15ba80 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:38.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.049+0000 7f75a47e8640 1 --2- 172.21.15.131:0/878793005 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f759c0cc300 0x7f759c0cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:38.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.049+0000 7f75a47e8640 1 -- 172.21.15.131:0/878793005 >> 172.21.15.131:0/878793005 conn(0x7f759c06eed0 msgr2=0x7f759c06f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:38.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.049+0000 7f75a47e8640 1 -- 172.21.15.131:0/878793005 shutdown_connections 2024-09-17T02:52:38.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.049+0000 7f75a47e8640 1 -- 172.21.15.131:0/878793005 wait complete. 2024-09-17T02:52:38.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a47e8640 1 Processor -- start 2024-09-17T02:52:38.047 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a47e8640 1 -- start start 2024-09-17T02:52:38.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a47e8640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f759c0cc300 0x7f759c13e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a47e8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c0d0440 0x7f759c0b2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a47e8640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f759c15bfc0 0x7f759c144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a47e8640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f759c1669e0 con 0x7f759c0d0440 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a47e8640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f759c166860 con 0x7f759c0cc300 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a47e8640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f759c166b60 con 0x7f759c15bfc0 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a1d83640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c0d0440 0x7f759c0b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a1d83640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c0d0440 0x7f759c0b2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40854/0 (socket says 172.21.15.131:40854) 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a1d83640 1 -- 172.21.15.131:0/2902724912 learned_addr learned my addr 172.21.15.131:0/2902724912 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a2584640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f759c0cc300 0x7f759c13e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a2d85640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f759c15bfc0 0x7f759c144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a1d83640 1 -- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f759c15bfc0 msgr2=0x7f759c144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a1d83640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f759c15bfc0 0x7f759c144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a1d83640 1 -- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f759c0cc300 msgr2=0x7f759c13e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:38.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a1d83640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f759c0cc300 0x7f759c13e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a1d83640 1 -- 172.21.15.131:0/2902724912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c13c7e0 con 0x7f759c0d0440 2024-09-17T02:52:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a2584640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f759c0cc300 0x7f759c13e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a1d83640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c0d0440 0x7f759c0b2300 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f759001cbe0 tx=0x7f75900a6c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f759affd640 1 -- 172.21.15.131:0/2902724912 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75900ac070 con 0x7f759c0d0440 2024-09-17T02:52:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f759affd640 1 -- 172.21.15.131:0/2902724912 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7590004140 con 0x7f759c0d0440 2024-09-17T02:52:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a47e8640 1 -- 172.21.15.131:0/2902724912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c157860 con 0x7f759c0d0440 2024-09-17T02:52:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f759affd640 1 -- 172.21.15.131:0/2902724912 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590005070 con 0x7f759c0d0440 2024-09-17T02:52:38.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.053+0000 7f75a47e8640 1 -- 172.21.15.131:0/2902724912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c0b03f0 con 0x7f759c0d0440 2024-09-17T02:52:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.057+0000 7f75a47e8640 1 -- 172.21.15.131:0/2902724912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759c1572a0 con 0x7f759c0d0440 2024-09-17T02:52:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.057+0000 7f759affd640 1 -- 172.21.15.131:0/2902724912 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f75900af070 con 0x7f759c0d0440 2024-09-17T02:52:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.057+0000 7f759affd640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073200 0x7f7588075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.057+0000 7f759affd640 1 -- 172.21.15.131:0/2902724912 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7f759012e9a0 con 0x7f759c0d0440 2024-09-17T02:52:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.057+0000 7f75a2584640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073200 0x7f7588075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:38.052 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.057+0000 7f75a2584640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073200 0x7f7588075710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f758c0058d0 tx=0x7f758c005ca0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:38.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.061+0000 7f759affd640 1 -- 172.21.15.131:0/2902724912 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f75900b6070 con 0x7f759c0d0440 2024-09-17T02:52:38.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.269+0000 7f75a47e8640 1 -- 172.21.15.131:0/2902724912 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f759c06f460 con 0x7f759c0d0440 2024-09-17T02:52:38.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f759affd640 1 -- 172.21.15.131:0/2902724912 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5232 (secure 0 0 0) 0x7f759c0d0850 con 0x7f759c0d0440 2024-09-17T02:52:38.266 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:38.267 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":112,"btime":"2024-09-17T02:52:38:076898+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:52:37.082214+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8880":{"gid":8880,"name":"a","rank":0,"incarnation":110,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.131:6835/4011259083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4011259083},{"type":"v1","addr":"172.21.15.131:6835","nonce":4011259083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-17T02:52:38.267 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 112 2024-09-17T02:52:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 -- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073200 msgr2=0x7f7588075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073200 0x7f7588075710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f758c0058d0 tx=0x7f758c005ca0 comp rx=0 tx=0).stop 2024-09-17T02:52:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 -- 172.21.15.131:0/2902724912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c0d0440 msgr2=0x7f759c0b2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c0d0440 0x7f759c0b2300 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f759001cbe0 tx=0x7f75900a6c20 comp rx=0 tx=0).stop 2024-09-17T02:52:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 -- 172.21.15.131:0/2902724912 shutdown_connections 2024-09-17T02:52:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7588073200 0x7f7588075710 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f759c15bfc0 0x7f759c144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:38.269 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f759c0d0440 0x7f759c0b2300 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:38.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 --2- 172.21.15.131:0/2902724912 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f759c0cc300 0x7f759c13e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:38.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 -- 172.21.15.131:0/2902724912 >> 172.21.15.131:0/2902724912 conn(0x7f759c06eed0 msgr2=0x7f759c159fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:38.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 -- 172.21.15.131:0/2902724912 shutdown_connections 2024-09-17T02:52:38.270 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:38.273+0000 7f75a47e8640 1 -- 172.21.15.131:0/2902724912 wait complete. 2024-09-17T02:52:38.692 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:51:54.764322+0000', 'modified': '2024-09-17T02:52:37.082214+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 124, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8880}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8880': {'gid': 8880, 'name': 'a', 'rank': 0, 'incarnation': 110, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.131:6835/4011259083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 4011259083}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 4011259083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-17T02:52:38.692 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8880:up:rejoin 2024-09-17T02:52:39.693 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.861+0000 7fe42c180640 1 -- 172.21.15.131:0/1983528544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424158e20 msgr2=0x7fe4240b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.861+0000 7fe42c180640 1 --2- 172.21.15.131:0/1983528544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424158e20 0x7fe4240b3cd0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009bc0 tx=0x7fe41c01cb10 comp rx=0 tx=0).stop 2024-09-17T02:52:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.861+0000 7fe42c180640 1 -- 172.21.15.131:0/1983528544 shutdown_connections 2024-09-17T02:52:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.861+0000 7fe42c180640 1 --2- 172.21.15.131:0/1983528544 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4240b4210 0x7fe424162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:39.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.861+0000 7fe42c180640 1 --2- 172.21.15.131:0/1983528544 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424158e20 0x7fe4240b3cd0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:39.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.861+0000 7fe42c180640 1 -- 172.21.15.131:0/1983528544 >> 172.21.15.131:0/1983528544 conn(0x7fe42406d440 msgr2=0x7fe4240cecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:39.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.861+0000 7fe42c180640 1 -- 172.21.15.131:0/1983528544 shutdown_connections 2024-09-17T02:52:39.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.861+0000 7fe42c180640 1 -- 172.21.15.131:0/1983528544 wait complete. 2024-09-17T02:52:39.858 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42c180640 1 Processor -- start 2024-09-17T02:52:39.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42c180640 1 -- start start 2024-09-17T02:52:39.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42c180640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4240b4210 0x7fe4241476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42c180640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424158e20 0x7fe4240b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42c180640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe424162850 0x7fe42406efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42c180640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe42416bd50 con 0x7fe424158e20 2024-09-17T02:52:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42c180640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe42416bbd0 con 0x7fe424162850 2024-09-17T02:52:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42c180640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe42416bed0 con 0x7fe4240b4210 2024-09-17T02:52:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42971b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424158e20 0x7fe4240b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42971b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424158e20 0x7fe4240b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40862/0 (socket says 172.21.15.131:40862) 2024-09-17T02:52:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42971b640 1 -- 172.21.15.131:0/34600093 learned_addr learned my addr 172.21.15.131:0/34600093 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:39.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42971b640 1 -- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4240b4210 msgr2=0x7fe4241476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe429f1c640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4240b4210 0x7fe4241476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42a71d640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe424162850 0x7fe42406efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42971b640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4240b4210 0x7fe4241476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42971b640 1 -- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe424162850 msgr2=0x7fe42406efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42971b640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe424162850 0x7fe42406efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42971b640 1 -- 172.21.15.131:0/34600093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4240b2680 con 0x7fe424158e20 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42971b640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424158e20 0x7fe4240b30f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fe41800d6a0 tx=0x7fe41800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe4227fc640 1 -- 172.21.15.131:0/34600093 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418017070 con 0x7fe424158e20 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe4227fc640 1 -- 172.21.15.131:0/34600093 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe418009ab0 con 0x7fe424158e20 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe4227fc640 1 -- 172.21.15.131:0/34600093 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe418010460 con 0x7fe424158e20 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42c180640 1 -- 172.21.15.131:0/34600093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4240c5150 con 0x7fe424158e20 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42c180640 1 -- 172.21.15.131:0/34600093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4240c53f0 con 0x7fe424158e20 2024-09-17T02:52:39.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.865+0000 7fe42c180640 1 -- 172.21.15.131:0/34600093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe424162310 con 0x7fe424158e20 2024-09-17T02:52:39.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.869+0000 7fe4227fc640 1 -- 172.21.15.131:0/34600093 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe41801a070 con 0x7fe424158e20 2024-09-17T02:52:39.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.869+0000 7fe4227fc640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 0x7fe410075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:39.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.869+0000 7fe4227fc640 1 -- 172.21.15.131:0/34600093 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7fe418094690 con 0x7fe424158e20 2024-09-17T02:52:39.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.869+0000 7fe429f1c640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 0x7fe410075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:39.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.869+0000 7fe429f1c640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 0x7fe410075710 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0af000 tx=0x7fe41c01b350 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:39.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:39.873+0000 7fe4227fc640 1 -- 172.21.15.131:0/34600093 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe41805f3d0 con 0x7fe424158e20 2024-09-17T02:52:40.081 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.085+0000 7fe42c180640 1 -- 172.21.15.131:0/34600093 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4240d01a0 con 0x7fe424158e20 2024-09-17T02:52:40.084 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.089+0000 7fe4227fc640 1 -- 172.21.15.131:0/34600093 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5239 (secure 0 0 0) 0x7fe41805ecd0 con 0x7fe424158e20 2024-09-17T02:52:40.085 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:40.085 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":113,"btime":"2024-09-17T02:52:39:086790+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:52:39.086771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8880":{"gid":8880,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/4011259083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4011259083},{"type":"v1","addr":"172.21.15.131:6835","nonce":4011259083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8880,"qdb_cluster":[8880]},"id":8}]} 2024-09-17T02:52:40.085 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 113 2024-09-17T02:52:40.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 -- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 msgr2=0x7fe410075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 0x7fe410075710 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0af000 tx=0x7fe41c01b350 comp rx=0 tx=0).stop 2024-09-17T02:52:40.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 -- 172.21.15.131:0/34600093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424158e20 msgr2=0x7fe4240b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424158e20 0x7fe4240b30f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fe41800d6a0 tx=0x7fe41800db60 comp rx=0 tx=0).stop 2024-09-17T02:52:40.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 -- 172.21.15.131:0/34600093 shutdown_connections 2024-09-17T02:52:40.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe410073200 0x7fe410075710 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe424162850 0x7fe42406efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe424158e20 0x7fe4240b30f0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 --2- 172.21.15.131:0/34600093 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe4240b4210 0x7fe4241476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 -- 172.21.15.131:0/34600093 >> 172.21.15.131:0/34600093 conn(0x7fe42406d440 msgr2=0x7fe4241584d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:40.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 -- 172.21.15.131:0/34600093 shutdown_connections 2024-09-17T02:52:40.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.093+0000 7fe42c180640 1 -- 172.21.15.131:0/34600093 wait complete. 2024-09-17T02:52:40.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T02:51:54.764322+0000', 'modified': '2024-09-17T02:52:39.086771+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 124, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8880}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8880': {'gid': 8880, 'name': 'a', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.131:6835/4011259083', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.131:6832', 'nonce': 4011259083}, {'type': 'v1', 'addr': '172.21.15.131:6835', 'nonce': 4011259083}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8880, 'qdb_cluster': [8880]} 2024-09-17T02:52:40.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T02:52:40.512 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T02:52:40.681 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.685+0000 7f1c4154e640 1 -- 172.21.15.131:0/3612484130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c3c0743c0 msgr2=0x7f1c3c0747d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.685+0000 7f1c4154e640 1 --2- 172.21.15.131:0/3612484130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c3c0743c0 0x7f1c3c0747d0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c071720 tx=0x7f1c3401cd80 comp rx=0 tx=0).stop 2024-09-17T02:52:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.685+0000 7f1c4154e640 1 -- 172.21.15.131:0/3612484130 shutdown_connections 2024-09-17T02:52:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.685+0000 7f1c4154e640 1 --2- 172.21.15.131:0/3612484130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c3c0743c0 0x7f1c3c0747d0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.685+0000 7f1c4154e640 1 --2- 172.21.15.131:0/3612484130 >> v2:172.21.15.189:3300/0 conn(0x7f1c3c06f780 0x7f1c3c06fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.685+0000 7f1c4154e640 1 -- 172.21.15.131:0/3612484130 >> 172.21.15.131:0/3612484130 conn(0x7f1c3c0cff40 msgr2=0x7f1c3c0d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.685+0000 7f1c4154e640 1 -- 172.21.15.131:0/3612484130 shutdown_connections 2024-09-17T02:52:40.682 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.685+0000 7f1c4154e640 1 -- 172.21.15.131:0/3612484130 wait complete. 2024-09-17T02:52:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c4154e640 1 Processor -- start 2024-09-17T02:52:40.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c4154e640 1 -- start start 2024-09-17T02:52:40.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c4154e640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1c3c06f780 0x7f1c3c147520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c4154e640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c3c0743c0 0x7f1c3c146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c4154e640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c3c15b000 0x7f1c3c1420e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.686 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c4154e640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c151710 con 0x7f1c3c15b000 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c4154e640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c151590 con 0x7f1c3c0743c0 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c4154e640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f1c3c151890 con 0x7f1c3c06f780 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c3b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c3c15b000 0x7f1c3c1420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c3b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c3c15b000 0x7f1c3c1420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40884/0 (socket says 172.21.15.131:40884) 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c3b7fe640 1 -- 172.21.15.131:0/991887599 learned_addr learned my addr 172.21.15.131:0/991887599 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c3a7fc640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c3c0743c0 0x7f1c3c146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c3affd640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1c3c06f780 0x7f1c3c147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c3b7fe640 1 -- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1c3c06f780 msgr2=0x7f1c3c147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c3b7fe640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1c3c06f780 0x7f1c3c147520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c3b7fe640 1 -- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c3c0743c0 msgr2=0x7f1c3c146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c3b7fe640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c3c0743c0 0x7f1c3c146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.687 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c3b7fe640 1 -- 172.21.15.131:0/991887599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c3c143050 con 0x7f1c3c15b000 2024-09-17T02:52:40.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c3b7fe640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c3c15b000 0x7f1c3c1420e0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009bd0 tx=0x7f1c34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:40.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c237fe640 1 -- 172.21.15.131:0/991887599 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c340bb070 con 0x7f1c3c15b000 2024-09-17T02:52:40.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c237fe640 1 -- 172.21.15.131:0/991887599 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c34004b40 con 0x7f1c3c15b000 2024-09-17T02:52:40.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c4154e640 1 -- 172.21.15.131:0/991887599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c3c14c2b0 con 0x7f1c3c15b000 2024-09-17T02:52:40.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c237fe640 1 -- 172.21.15.131:0/991887599 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c340b2040 con 0x7f1c3c15b000 2024-09-17T02:52:40.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.689+0000 7f1c4154e640 1 -- 172.21.15.131:0/991887599 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c3c1fc920 con 0x7f1c3c15b000 2024-09-17T02:52:40.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.693+0000 7f1c237fe640 1 -- 172.21.15.131:0/991887599 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f1c340be070 con 0x7f1c3c15b000 2024-09-17T02:52:40.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.693+0000 7f1c237fe640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1c28073230 0x7f1c28075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.693+0000 7f1c237fe640 1 -- 172.21.15.131:0/991887599 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7f1c3412e4c0 con 0x7f1c3c15b000 2024-09-17T02:52:40.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.693+0000 7f1c3affd640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1c28073230 0x7f1c28075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.693+0000 7f1c48819000 1 Processor -- start 2024-09-17T02:52:40.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.693+0000 7f1c48819000 1 -- start start 2024-09-17T02:52:40.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.693+0000 7f1c3affd640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1c28073230 0x7f1c28075740 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1c300058d0 tx=0x7f1c30005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef0b0460 0x5603ef5f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef4a0950 0x5603ef5f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef5f2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5603ef4c6410 con 0x5603ef489ca0 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603ef403b40 con 0x5603ef4a0950 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5603ef3f2f70 con 0x5603ef0b0460 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef5f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3b7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef5f2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40892/0 (socket says 172.21.15.131:40892) 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3b7fe640 1 -- 172.21.15.131:0/3409266726 learned_addr learned my addr 172.21.15.131:0/3409266726 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3a7fc640 1 --2- 172.21.15.131:0/3409266726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef4a0950 0x5603ef5f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3affd640 1 --2- 172.21.15.131:0/3409266726 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef0b0460 0x5603ef5f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3b7fe640 1 -- 172.21.15.131:0/3409266726 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef0b0460 msgr2=0x5603ef5f16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3b7fe640 1 --2- 172.21.15.131:0/3409266726 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef0b0460 0x5603ef5f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3b7fe640 1 -- 172.21.15.131:0/3409266726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef4a0950 msgr2=0x5603ef5f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3b7fe640 1 --2- 172.21.15.131:0/3409266726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef4a0950 0x5603ef5f1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3b7fe640 1 -- 172.21.15.131:0/3409266726 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ef3ba5b0 con 0x5603ef489ca0 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3a7fc640 1 --2- 172.21.15.131:0/3409266726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef4a0950 0x5603ef5f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c3b7fe640 1 --2- 172.21.15.131:0/3409266726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef5f2120 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f1c340f89c0 tx=0x7f1c340f8a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135fbe1897f3d21b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c20ff9640 1 -- 172.21.15.131:0/3409266726 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c340a5cc0 con 0x5603ef489ca0 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c20ff9640 1 -- 172.21.15.131:0/3409266726 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c340f87d0 con 0x5603ef489ca0 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c20ff9640 1 -- 172.21.15.131:0/3409266726 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c3412cdd0 con 0x5603ef489ca0 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 -- 172.21.15.131:0/3409266726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 msgr2=0x5603ef5f2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 --2- 172.21.15.131:0/3409266726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef5f2120 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f1c340f89c0 tx=0x7f1c340f8a90 comp rx=0 tx=0).stop 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 -- 172.21.15.131:0/3409266726 shutdown_connections 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 --2- 172.21.15.131:0/3409266726 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef5f2120 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 --2- 172.21.15.131:0/3409266726 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef4a0950 0x5603ef5f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 --2- 172.21.15.131:0/3409266726 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef0b0460 0x5603ef5f16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 -- 172.21.15.131:0/3409266726 >> 172.21.15.131:0/3409266726 conn(0x5603ef576d90 msgr2=0x5603ef5a6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:40.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 -- 172.21.15.131:0/3409266726 shutdown_connections 2024-09-17T02:52:40.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 -- 172.21.15.131:0/3409266726 wait complete. 2024-09-17T02:52:40.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 -- start start 2024-09-17T02:52:40.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.697+0000 7f1c48819000 1 Processor -- start 2024-09-17T02:52:40.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef5f8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef489ca0 0x5603ef5f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef4a0950 0x5603ef5f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x5603ef38d5f0 con 0x5603ef0b0460 2024-09-17T02:52:40.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603ef3f6d80 con 0x5603ef489ca0 2024-09-17T02:52:40.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5603ef399520 con 0x5603ef4a0950 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef5f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 10 client.0 _ll_register_callbacks cb 0x5603ef39d830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef5f8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40900/0 (socket says 172.21.15.131:40900) 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3affd640 1 -- 172.21.15.131:0/1318460973 learned_addr learned my addr 172.21.15.131:0/1318460973 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3a7fc640 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef489ca0 0x5603ef5f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3affd640 1 -- 172.21.15.131:0/1318460973 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef4a0950 msgr2=0x5603ef5f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3affd640 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef4a0950 0x5603ef5f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3affd640 1 -- 172.21.15.131:0/1318460973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef489ca0 msgr2=0x5603ef5f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3affd640 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef489ca0 0x5603ef5f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3affd640 1 -- 172.21.15.131:0/1318460973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ef391190 con 0x5603ef0b0460 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3affd640 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef5f8920 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f1c30011720 tx=0x7f1c300046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b67a1ee73ff18b server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:52:40.699 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c19ffb640 1 -- 172.21.15.131:0/1318460973 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30022070 con 0x5603ef0b0460 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c19ffb640 1 -- 172.21.15.131:0/1318460973 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7f1c30010920 con 0x5603ef0b0460 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 -- 172.21.15.131:0/1318460973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5603ef38b110 con 0x5603ef0b0460 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c19ffb640 1 -- 172.21.15.131:0/1318460973 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f1c3002d050 con 0x5603ef0b0460 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 10 client.8967 fetch_fsmap learned FSMap version 113 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 -- 172.21.15.131:0/1318460973 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x5603ef477f10 con 0x5603ef0b0460 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c19ffb640 1 -- 172.21.15.131:0/1318460973 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5603ef38b110 con 0x5603ef0b0460 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c19ffb640 1 -- 172.21.15.131:0/1318460973 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 113) ==== 3502+0+0 (secure 0 0 0) 0x7f1c30030020 con 0x5603ef0b0460 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 10 client.8967 fetch_fsmap finished waiting for FSMap version 113 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 10 client.8967 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 20 client.8967 populate_metadata read hostname 'smithi131' 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.131:6832/4011259083,v1:172.21.15.131:6835/4011259083] conn(0x5603ef0957d0 0x5603ef602d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 4 client.8967 mds_command: new command op to 8880 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T02:52:40.700 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 -- 172.21.15.131:0/1318460973 --> [v2:172.21.15.131:6832/4011259083,v1:172.21.15.131:6835/4011259083] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603ef405a90 con 0x5603ef0957d0 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3a7fc640 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.131:6832/4011259083,v1:172.21.15.131:6835/4011259083] conn(0x5603ef0957d0 0x5603ef602d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c3a7fc640 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.131:6832/4011259083,v1:172.21.15.131:6835/4011259083] conn(0x5603ef0957d0 0x5603ef602d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76a3e3c0a8ed5abb server_cookie=da5300d91570ba29 in_seq=0 out_seq=0 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c19ffb640 10 client.8967 ms_handle_connect on v2:172.21.15.131:6832/4011259083 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c19ffb640 1 -- 172.21.15.131:0/1318460973 <== mds.0 v2:172.21.15.131:6832/4011259083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5603ef405a90 con 0x5603ef0957d0 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c19ffb640 10 client.8967 handle_command_reply: tid=0 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 client.8967 shutdown 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 -- 172.21.15.131:0/1318460973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 msgr2=0x5603ef5f8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef5f8920 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f1c30011720 tx=0x7f1c300046d0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 -- 172.21.15.131:0/1318460973 shutdown_connections 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.131:6832/4011259083,v1:172.21.15.131:6835/4011259083] conn(0x5603ef0957d0 0x5603ef602d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef4a0950 0x5603ef5f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef489ca0 0x5603ef5f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.701 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 --2- 172.21.15.131:0/1318460973 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef5f8920 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 -- 172.21.15.131:0/1318460973 >> 172.21.15.131:0/1318460973 conn(0x5603ef576d90 msgr2=0x5603ef5ad440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c19ffb640 0 client.8967 ms_handle_reset on v2:172.21.15.131:6832/4011259083 2024-09-17T02:52:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 -- 172.21.15.131:0/1318460973 shutdown_connections 2024-09-17T02:52:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 1 -- 172.21.15.131:0/1318460973 wait complete. 2024-09-17T02:52:40.702 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.701+0000 7f1c48819000 20 client.8967 trim_cache size 0 max 16384 2024-09-17T02:52:40.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c48819000 1 Processor -- start 2024-09-17T02:52:40.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c48819000 1 -- start start 2024-09-17T02:52:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c48819000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef663620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c48819000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef489ca0 0x5603ef663b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c48819000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef4a0950 0x5603ef6640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c48819000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c205d50 con 0x5603ef0b0460 2024-09-17T02:52:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c48819000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c204d90 con 0x5603ef4a0950 2024-09-17T02:52:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c48819000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f1c3c204120 con 0x5603ef489ca0 2024-09-17T02:52:40.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c3affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef663620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c3affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef663620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40906/0 (socket says 172.21.15.131:40906) 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c3affd640 1 -- 172.21.15.131:0/34369938 learned_addr learned my addr 172.21.15.131:0/34369938 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c3b7fe640 1 --2- 172.21.15.131:0/34369938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef4a0950 0x5603ef6640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c3affd640 1 -- 172.21.15.131:0/34369938 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef489ca0 msgr2=0x5603ef663b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c3affd640 1 --2- 172.21.15.131:0/34369938 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef489ca0 0x5603ef663b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c3affd640 1 -- 172.21.15.131:0/34369938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef4a0950 msgr2=0x5603ef6640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c3affd640 1 --2- 172.21.15.131:0/34369938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef4a0950 0x5603ef6640a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.721+0000 7f1c3affd640 1 -- 172.21.15.131:0/34369938 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ef44edb0 con 0x5603ef0b0460 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c3affd640 1 --2- 172.21.15.131:0/34369938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef663620 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f1c3002b040 tx=0x7f1c3000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332cdf32c1305589 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c1bfff640 1 -- 172.21.15.131:0/34369938 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30035020 con 0x5603ef0b0460 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c1bfff640 1 -- 172.21.15.131:0/34369938 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c30017040 con 0x5603ef0b0460 2024-09-17T02:52:40.719 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c1bfff640 1 -- 172.21.15.131:0/34369938 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c30022070 con 0x5603ef0b0460 2024-09-17T02:52:40.720 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 -- 172.21.15.131:0/34369938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 msgr2=0x5603ef663620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 --2- 172.21.15.131:0/34369938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef663620 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f1c3002b040 tx=0x7f1c3000ef40 comp rx=0 tx=0).stop 2024-09-17T02:52:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 -- 172.21.15.131:0/34369938 shutdown_connections 2024-09-17T02:52:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 --2- 172.21.15.131:0/34369938 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef4a0950 0x5603ef6640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 --2- 172.21.15.131:0/34369938 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef489ca0 0x5603ef663b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 --2- 172.21.15.131:0/34369938 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef0b0460 0x5603ef663620 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 -- 172.21.15.131:0/34369938 >> 172.21.15.131:0/34369938 conn(0x5603ef576d90 msgr2=0x5603ef5ef620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 -- 172.21.15.131:0/34369938 shutdown_connections 2024-09-17T02:52:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 -- 172.21.15.131:0/34369938 wait complete. 2024-09-17T02:52:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 -- start start 2024-09-17T02:52:40.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 Processor -- start 2024-09-17T02:52:40.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef0b0460 0x5603ef6675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef667ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.725 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef4a0950 0x5603ef668020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f1c3c204740 con 0x5603ef489ca0 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x5603ef3fb3b0 con 0x5603ef0b0460 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x5603ef4b2750 con 0x5603ef4a0950 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c48819000 10 client.0 _ll_register_callbacks cb 0x7f1c300096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c3a7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef667ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c3a7fc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef667ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40916/0 (socket says 172.21.15.131:40916) 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.725+0000 7f1c3a7fc640 1 -- 172.21.15.131:0/2836198827 learned_addr learned my addr 172.21.15.131:0/2836198827 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c3affd640 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef0b0460 0x5603ef6675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c3b7fe640 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef4a0950 0x5603ef668020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c3a7fc640 1 -- 172.21.15.131:0/2836198827 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef4a0950 msgr2=0x5603ef668020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c3a7fc640 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef4a0950 0x5603ef668020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c3a7fc640 1 -- 172.21.15.131:0/2836198827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef0b0460 msgr2=0x5603ef6675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c3a7fc640 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef0b0460 0x5603ef6675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.726 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c3a7fc640 1 -- 172.21.15.131:0/2836198827 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ef3b9ab0 con 0x5603ef489ca0 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c3affd640 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef0b0460 0x5603ef6675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c3a7fc640 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef667ae0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c003620 tx=0x7f1c2c004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7509c2f5d397fd68 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.131:3300/0 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c19ffb640 1 -- 172.21.15.131:0/2836198827 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c020070 con 0x5603ef489ca0 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c19ffb640 1 -- 172.21.15.131:0/2836198827 <== mon.0 v2:172.21.15.131:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7f1c2c017cf0 con 0x5603ef489ca0 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 -- 172.21.15.131:0/2836198827 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c30017040 con 0x5603ef489ca0 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c19ffb640 1 -- 172.21.15.131:0/2836198827 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7f1c2c02b050 con 0x5603ef489ca0 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 10 client.8973 fetch_fsmap learned FSMap version 113 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 -- 172.21.15.131:0/2836198827 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) -- 0x5603ef398070 con 0x5603ef489ca0 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c19ffb640 1 -- 172.21.15.131:0/2836198827 <== mon.0 v2:172.21.15.131:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c30017040 con 0x5603ef489ca0 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c19ffb640 1 -- 172.21.15.131:0/2836198827 <== mon.0 v2:172.21.15.131:3300/0 5 ==== fsmap(e 113) ==== 3502+0+0 (secure 0 0 0) 0x7f1c2c02e020 con 0x5603ef489ca0 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 10 client.8973 fetch_fsmap finished waiting for FSMap version 113 2024-09-17T02:52:40.727 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 10 client.8973 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 20 client.8973 populate_metadata read hostname 'smithi131' 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.131:6832/4011259083,v1:172.21.15.131:6835/4011259083] conn(0x5603ef083830 0x5603ef6237f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 4 client.8973 mds_command: new command op to 8880 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 -- 172.21.15.131:0/2836198827 --> [v2:172.21.15.131:6832/4011259083,v1:172.21.15.131:6835/4011259083] -- command(tid 0: {"prefix": "status"}) -- 0x5603ef401540 con 0x5603ef083830 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c3affd640 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.131:6832/4011259083,v1:172.21.15.131:6835/4011259083] conn(0x5603ef083830 0x5603ef6237f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c19ffb640 10 client.8973 ms_handle_connect on v2:172.21.15.131:6832/4011259083 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c3affd640 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.131:6832/4011259083,v1:172.21.15.131:6835/4011259083] conn(0x5603ef083830 0x5603ef6237f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f48f3b7ffcdd7ffe server_cookie=189a6e51a3183498 in_seq=0 out_seq=0 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c19ffb640 1 -- 172.21.15.131:0/2836198827 <== mds.0 v2:172.21.15.131:6832/4011259083 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x5603ef401540 con 0x5603ef083830 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c19ffb640 10 client.8973 handle_command_reply: tid=0 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 client.8973 shutdown 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 -- 172.21.15.131:0/2836198827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 msgr2=0x5603ef667ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef667ae0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c003620 tx=0x7f1c2c004f20 comp rx=0 tx=0).stop 2024-09-17T02:52:40.728 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 -- 172.21.15.131:0/2836198827 shutdown_connections 2024-09-17T02:52:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.131:6832/4011259083,v1:172.21.15.131:6835/4011259083] conn(0x5603ef083830 0x5603ef6237f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x5603ef4a0950 0x5603ef668020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x5603ef489ca0 0x5603ef667ae0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c19ffb640 0 client.8973 ms_handle_reset on v2:172.21.15.131:6832/4011259083 2024-09-17T02:52:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 --2- 172.21.15.131:0/2836198827 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x5603ef0b0460 0x5603ef6675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 -- 172.21.15.131:0/2836198827 >> 172.21.15.131:0/2836198827 conn(0x5603ef576d90 msgr2=0x5603ef6026a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 -- 172.21.15.131:0/2836198827 shutdown_connections 2024-09-17T02:52:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 1 -- 172.21.15.131:0/2836198827 wait complete. 2024-09-17T02:52:40.729 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c48819000 20 client.8973 trim_cache size 0 max 16384 2024-09-17T02:52:40.729 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:40.729 INFO:teuthology.orchestra.run.smithi131.stdout: "cluster_fsid": "f03002b8-e723-4674-9385-a37ff83176c9", 2024-09-17T02:52:40.730 INFO:teuthology.orchestra.run.smithi131.stdout: "whoami": 0, 2024-09-17T02:52:40.730 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8880, 2024-09-17T02:52:40.730 INFO:teuthology.orchestra.run.smithi131.stdout: "want_state": "up:active", 2024-09-17T02:52:40.730 INFO:teuthology.orchestra.run.smithi131.stdout: "state": "up:active", 2024-09-17T02:52:40.730 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs", 2024-09-17T02:52:40.730 INFO:teuthology.orchestra.run.smithi131.stdout: "rank_uptime": 4.6572174239999997, 2024-09-17T02:52:40.730 INFO:teuthology.orchestra.run.smithi131.stdout: "mdsmap_epoch": 113, 2024-09-17T02:52:40.730 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch": 124, 2024-09-17T02:52:40.730 INFO:teuthology.orchestra.run.smithi131.stdout: "osdmap_epoch_barrier": 124, 2024-09-17T02:52:40.730 INFO:teuthology.orchestra.run.smithi131.stdout: "uptime": 5.9808773459999998 2024-09-17T02:52:40.730 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:40.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 -- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1c28073230 msgr2=0x7f1c28075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1c28073230 0x7f1c28075740 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1c300058d0 tx=0x7f1c30005860 comp rx=0 tx=0).stop 2024-09-17T02:52:40.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 -- 172.21.15.131:0/991887599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c3c15b000 msgr2=0x7f1c3c1420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.731 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c3c15b000 0x7f1c3c1420e0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009bd0 tx=0x7f1c34004270 comp rx=0 tx=0).stop 2024-09-17T02:52:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 -- 172.21.15.131:0/991887599 shutdown_connections 2024-09-17T02:52:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f1c28073230 0x7f1c28075740 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f1c3c15b000 0x7f1c3c1420e0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f1c3c0743c0 0x7f1c3c146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 --2- 172.21.15.131:0/991887599 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f1c3c06f780 0x7f1c3c147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 -- 172.21.15.131:0/991887599 >> 172.21.15.131:0/991887599 conn(0x7f1c3c0cff40 msgr2=0x7f1c3c0d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 -- 172.21.15.131:0/991887599 shutdown_connections 2024-09-17T02:52:40.732 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.729+0000 7f1c4154e640 1 -- 172.21.15.131:0/991887599 wait complete. 2024-09-17T02:52:40.744 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T02:52:40.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.893+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3047360552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b40b6550 msgr2=0x7fe5b40d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.893+0000 7fe5baf3c640 1 --2- 172.21.15.131:0/3047360552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b40b6550 0x7fe5b40d31e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009bc0 tx=0x7fe5ac01cba0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.893+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3047360552 shutdown_connections 2024-09-17T02:52:40.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.893+0000 7fe5baf3c640 1 --2- 172.21.15.131:0/3047360552 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b40b6550 0x7fe5b40d31e0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.893+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3047360552 >> 172.21.15.131:0/3047360552 conn(0x7fe5b40d0e40 msgr2=0x7fe5b40d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:40.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.893+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3047360552 shutdown_connections 2024-09-17T02:52:40.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.893+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3047360552 wait complete. 2024-09-17T02:52:40.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.893+0000 7fe5baf3c640 1 Processor -- start 2024-09-17T02:52:40.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.893+0000 7fe5baf3c640 1 -- start start 2024-09-17T02:52:40.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5baf3c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5b40d2dd0 0x7fe5b4150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5baf3c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe5b415c3f0 0x7fe5b40cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5baf3c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b40ce320 0x7fe5b40cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5baf3c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b416d890 con 0x7fe5b40ce320 2024-09-17T02:52:40.891 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5baf3c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b416d710 con 0x7fe5b40d2dd0 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5baf3c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b416da10 con 0x7fe5b415c3f0 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b94d9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b40ce320 0x7fe5b40cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b94d9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b40ce320 0x7fe5b40cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40940/0 (socket says 172.21.15.131:40940) 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b94d9640 1 -- 172.21.15.131:0/3160190424 learned_addr learned my addr 172.21.15.131:0/3160190424 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b8cd8640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5b40d2dd0 0x7fe5b4150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b3fff640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe5b415c3f0 0x7fe5b40cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b94d9640 1 -- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe5b415c3f0 msgr2=0x7fe5b40cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b94d9640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe5b415c3f0 0x7fe5b40cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b94d9640 1 -- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5b40d2dd0 msgr2=0x7fe5b4150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b94d9640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5b40d2dd0 0x7fe5b4150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b94d9640 1 -- 172.21.15.131:0/3160190424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b41540a0 con 0x7fe5b40ce320 2024-09-17T02:52:40.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b3fff640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe5b415c3f0 0x7fe5b40cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:40.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b94d9640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b40ce320 0x7fe5b40cc7b0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0b0040 tx=0x7fe5ac0042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:40.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b17fa640 1 -- 172.21.15.131:0/3160190424 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0bb070 con 0x7fe5b40ce320 2024-09-17T02:52:40.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b17fa640 1 -- 172.21.15.131:0/3160190424 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5ac0047b0 con 0x7fe5b40ce320 2024-09-17T02:52:40.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b17fa640 1 -- 172.21.15.131:0/3160190424 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0b2040 con 0x7fe5b40ce320 2024-09-17T02:52:40.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3160190424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b40cb6c0 con 0x7fe5b40ce320 2024-09-17T02:52:40.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3160190424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b41fa210 con 0x7fe5b40ce320 2024-09-17T02:52:40.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3160190424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b40d31e0 con 0x7fe5b40ce320 2024-09-17T02:52:40.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b17fa640 1 -- 172.21.15.131:0/3160190424 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe5ac0a5d10 con 0x7fe5b40ce320 2024-09-17T02:52:40.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b17fa640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe5a0073230 0x7fe5a0075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:40.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b17fa640 1 -- 172.21.15.131:0/3160190424 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7fe5ac12e480 con 0x7fe5b40ce320 2024-09-17T02:52:40.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.897+0000 7fe5b8cd8640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe5a0073230 0x7fe5a0075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:40.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.901+0000 7fe5b8cd8640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe5a0073230 0x7fe5a0075740 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000f70 tx=0x7fe5a4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:40.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:40.901+0000 7fe5b17fa640 1 -- 172.21.15.131:0/3160190424 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe5ac016790 con 0x7fe5b40ce320 2024-09-17T02:52:41.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.097+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3160190424 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe5b40b6dc0 con 0x7fe5b40ce320 2024-09-17T02:52:41.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.097+0000 7fe5b17fa640 1 -- 172.21.15.131:0/3160190424 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) ==== 53+0+73 (secure 0 0 0) 0x7fe5ac0f9260 con 0x7fe5b40ce320 2024-09-17T02:52:41.093 INFO:teuthology.orchestra.run.smithi131.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T02:52:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe5a0073230 msgr2=0x7fe5a0075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe5a0073230 0x7fe5a0075740 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000f70 tx=0x7fe5a4009040 comp rx=0 tx=0).stop 2024-09-17T02:52:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3160190424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b40ce320 msgr2=0x7fe5b40cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b40ce320 0x7fe5b40cc7b0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0b0040 tx=0x7fe5ac0042d0 comp rx=0 tx=0).stop 2024-09-17T02:52:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3160190424 shutdown_connections 2024-09-17T02:52:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe5a0073230 0x7fe5a0075740 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:41.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe5b40ce320 0x7fe5b40cc7b0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe5b415c3f0 0x7fe5b40cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 --2- 172.21.15.131:0/3160190424 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe5b40d2dd0 0x7fe5b4150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3160190424 >> 172.21.15.131:0/3160190424 conn(0x7fe5b40d0e40 msgr2=0x7fe5b4154890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3160190424 shutdown_connections 2024-09-17T02:52:41.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41.101+0000 7fe5baf3c640 1 -- 172.21.15.131:0/3160190424 wait complete. 2024-09-17T02:52:41.115 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T02:52:41.115 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T02:52:41.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi131.front.sepia.ceph.com 2024-09-17T02:52:41.115 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T02:52:41.115 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:41.115 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T02:52:41.115 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T02:52:41.115 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T02:52:41.115 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:41.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:41.116 DEBUG:teuthology.orchestra.run.smithi131:> ip addr 2024-09-17T02:52:41.122 INFO:teuthology.orchestra.run.smithi131.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T02:52:41.122 INFO:teuthology.orchestra.run.smithi131.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T02:52:41.122 INFO:teuthology.orchestra.run.smithi131.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T02:52:41.122 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:41.122 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 ::1/128 scope host 2024-09-17T02:52:41.122 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:41.122 INFO:teuthology.orchestra.run.smithi131.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:41.122 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:18 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:41.122 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f0 2024-09-17T02:52:41.122 INFO:teuthology.orchestra.run.smithi131.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:41.122 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:d8:fb:19 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout: altname enp7s0f1 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f8 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0c:c4:7a:88:72:f9 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.21.15.131/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft 41745sec preferred_lft 41745sec 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::ec4:7aff:fe88:72f9/64 scope link 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 02:42:1b:73:f8:72 brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T02:52:41.123 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-17T02:52:41.124 INFO:teuthology.orchestra.run.smithi131.stdout: valid_lft forever preferred_lft forever 2024-09-17T02:52:41.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:41.125 DEBUG:teuthology.orchestra.run.smithi131:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T02:52:41.176 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:52:41.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:41.178 DEBUG:teuthology.orchestra.run.smithi131:> route 2024-09-17T02:52:41.230 INFO:teuthology.orchestra.run.smithi131.stdout:Kernel IP routing table 2024-09-17T02:52:41.230 INFO:teuthology.orchestra.run.smithi131.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T02:52:41.231 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-17T02:52:41.231 INFO:teuthology.orchestra.run.smithi131.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T02:52:41.231 INFO:teuthology.orchestra.run.smithi131.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-17T02:52:41.231 INFO:teuthology.orchestra.run.smithi131.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-17T02:52:41.231 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T02:52:41.231 INFO:teuthology.orchestra.run.smithi131.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:41.231 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:41.231 INFO:teuthology.orchestra.run.smithi131.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-17T02:52:41.231 INFO:teuthology.orchestra.run.smithi131.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T02:52:41.231 DEBUG:teuthology.orchestra.run.smithi131:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T02:52:41.232 DEBUG:teuthology.orchestra.run.smithi131:> set -e 2024-09-17T02:52:41.232 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T02:52:41.232 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T02:52:41.232 DEBUG:teuthology.orchestra.run.smithi131:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T02:52:41.232 DEBUG:teuthology.orchestra.run.smithi131:> ') 2024-09-17T02:52:41.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T02:52:41.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:41.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:41.352 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:52:41.358 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:52:41.358 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:52:41.359 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:41.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:41.359 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link set brx.0 up 2024-09-17T02:52:41.414 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T02:52:41.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:52:41.415 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:41.462 INFO:teuthology.orchestra.run.smithi131.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T02:52:41.462 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T02:52:41.462 DEBUG:teuthology.orchestra.run.smithi131:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:41.510 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:41.568 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:41.620 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:41.621 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:41.621 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:41.621 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:41.621 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:41.621 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:41.621 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:41.621 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:41.621 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:41.621 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:41.621 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:41.621 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:41.622 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:41.623 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:41.623 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:41.623 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:41.623 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:41.623 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:41.623 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:41.623 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:41.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:41.624 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:41.674 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T02:52:41.674 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T02:52:41.685 INFO:tasks.ceph.mds.a.smithi131.stderr:2024-09-17T02:52:41.689+0000 7f1ce50dd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:52:41.687 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:41.701 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:41.747 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:41.748 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:41.749 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:41.749 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:41.749 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:41.749 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:41.749 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:41.749 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:41.749 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:41.749 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:41.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:41.749 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:41.797 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:2024-09-17T02:52:41.801+0000 7fda8e5ce580 -1 init, newargv = 0x7fda700ad020 newargc=15 2024-09-17T02:52:41.797 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[52708]: starting ceph client 2024-09-17T02:52:41.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[52708]: starting fuse 2024-09-17T02:52:42.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo modprobe fuse 2024-09-17T02:52:42.912 DEBUG:teuthology.orchestra.run.smithi131:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T02:52:42.963 INFO:teuthology.orchestra.run.smithi131.stdout:/sys 2024-09-17T02:52:42.964 INFO:teuthology.orchestra.run.smithi131.stdout:/proc 2024-09-17T02:52:42.964 INFO:teuthology.orchestra.run.smithi131.stdout:/dev 2024-09-17T02:52:42.964 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/pts 2024-09-17T02:52:42.964 INFO:teuthology.orchestra.run.smithi131.stdout:/run 2024-09-17T02:52:42.964 INFO:teuthology.orchestra.run.smithi131.stdout:/ 2024-09-17T02:52:42.964 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/security 2024-09-17T02:52:42.964 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/shm 2024-09-17T02:52:42.964 INFO:teuthology.orchestra.run.smithi131.stdout:/run/lock 2024-09-17T02:52:42.964 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/cgroup 2024-09-17T02:52:42.964 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/pstore 2024-09-17T02:52:42.964 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/bpf 2024-09-17T02:52:42.965 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T02:52:42.965 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/hugepages 2024-09-17T02:52:42.965 INFO:teuthology.orchestra.run.smithi131.stdout:/dev/mqueue 2024-09-17T02:52:42.965 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/debug 2024-09-17T02:52:42.965 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/tracing 2024-09-17T02:52:42.965 INFO:teuthology.orchestra.run.smithi131.stdout:/proc/fs/nfsd 2024-09-17T02:52:42.965 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/fs/fuse/connections 2024-09-17T02:52:42.965 INFO:teuthology.orchestra.run.smithi131.stdout:/sys/kernel/config 2024-09-17T02:52:42.965 INFO:teuthology.orchestra.run.smithi131.stdout:/run/credentials/systemd-sysusers.service 2024-09-17T02:52:42.965 INFO:teuthology.orchestra.run.smithi131.stdout:/run/rpc_pipefs 2024-09-17T02:52:42.966 INFO:teuthology.orchestra.run.smithi131.stdout:/run/user/1000 2024-09-17T02:52:42.966 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph 2024-09-17T02:52:42.966 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T02:52:42.966 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T02:52:42.966 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T02:52:42.966 INFO:teuthology.orchestra.run.smithi131.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T02:52:42.966 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns 2024-09-17T02:52:42.966 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:42.966 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:42.966 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:42.966 INFO:teuthology.orchestra.run.smithi131.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:42.967 INFO:teuthology.orchestra.run.smithi131.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:42.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:42.967 DEBUG:teuthology.orchestra.run.smithi131:> ls /sys/fs/fuse/connections 2024-09-17T02:52:43.015 INFO:teuthology.orchestra.run.smithi131.stdout:45 2024-09-17T02:52:43.015 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-17T02:52:43.015 DEBUG:teuthology.orchestra.run.smithi131:> sudo stdin-killer -- python3 -c ' 2024-09-17T02:52:43.015 DEBUG:teuthology.orchestra.run.smithi131:> import glob 2024-09-17T02:52:43.015 DEBUG:teuthology.orchestra.run.smithi131:> import re 2024-09-17T02:52:43.015 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:43.015 DEBUG:teuthology.orchestra.run.smithi131:> import subprocess 2024-09-17T02:52:43.016 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:43.016 DEBUG:teuthology.orchestra.run.smithi131:> def _find_admin_socket(client_name): 2024-09-17T02:52:43.016 DEBUG:teuthology.orchestra.run.smithi131:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T02:52:43.016 DEBUG:teuthology.orchestra.run.smithi131:> files = glob.glob(asok_path) 2024-09-17T02:52:43.016 DEBUG:teuthology.orchestra.run.smithi131:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T02:52:43.016 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:43.016 DEBUG:teuthology.orchestra.run.smithi131:> # Given a non-glob path, it better be there 2024-09-17T02:52:43.016 DEBUG:teuthology.orchestra.run.smithi131:> if "*" not in asok_path: 2024-09-17T02:52:43.016 DEBUG:teuthology.orchestra.run.smithi131:> assert(len(files) == 1) 2024-09-17T02:52:43.016 DEBUG:teuthology.orchestra.run.smithi131:> return files[0] 2024-09-17T02:52:43.016 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:43.017 DEBUG:teuthology.orchestra.run.smithi131:> for f in files: 2024-09-17T02:52:43.017 DEBUG:teuthology.orchestra.run.smithi131:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T02:52:43.017 DEBUG:teuthology.orchestra.run.smithi131:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T02:52:43.017 DEBUG:teuthology.orchestra.run.smithi131:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T02:52:43.017 DEBUG:teuthology.orchestra.run.smithi131:> contents = proc_f.read() 2024-09-17T02:52:43.017 DEBUG:teuthology.orchestra.run.smithi131:> if mountpoint in contents: 2024-09-17T02:52:43.017 DEBUG:teuthology.orchestra.run.smithi131:> return f 2024-09-17T02:52:43.017 DEBUG:teuthology.orchestra.run.smithi131:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T02:52:43.017 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:43.017 DEBUG:teuthology.orchestra.run.smithi131:> print(_find_admin_socket("client.0")) 2024-09-17T02:52:43.017 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:43.140 INFO:teuthology.orchestra.run.smithi131.stdout:/var/run/ceph/ceph-client.0.52708.asok 2024-09-17T02:52:43.143 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:43.152 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52708.asok 2024-09-17T02:52:43.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:43.152 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52708.asok status 2024-09-17T02:52:43.333 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:43.333 INFO:teuthology.orchestra.run.smithi131.stdout: "metadata": { 2024-09-17T02:52:43.333 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T02:52:43.333 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T02:52:43.333 INFO:teuthology.orchestra.run.smithi131.stdout: "entity_id": "0", 2024-09-17T02:52:43.333 INFO:teuthology.orchestra.run.smithi131.stdout: "hostname": "smithi131", 2024-09-17T02:52:43.333 INFO:teuthology.orchestra.run.smithi131.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T02:52:43.333 INFO:teuthology.orchestra.run.smithi131.stdout: "pid": "52708", 2024-09-17T02:52:43.333 INFO:teuthology.orchestra.run.smithi131.stdout: "root": "/" 2024-09-17T02:52:43.333 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_count": 0, 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "dentry_pinned_count": 0, 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "id": 8985, 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "inst": { 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "name": { 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "client", 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "num": 8985 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2145880714 2024-09-17T02:52:43.334 INFO:teuthology.orchestra.run.smithi131.stdout: } 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": { 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: "type": "v1", 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: "addr": "192.168.0.1:0", 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: "nonce": 2145880714 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: }, 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: "inst_str": "client.8985 192.168.0.1:0/2145880714", 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: "addr_str": "192.168.0.1:0/2145880714", 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: "inode_count": 1, 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: "mds_epoch": 113, 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch": 124, 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: "osd_epoch_barrier": 0, 2024-09-17T02:52:43.335 INFO:teuthology.orchestra.run.smithi131.stdout: "blocklisted": false, 2024-09-17T02:52:43.336 INFO:teuthology.orchestra.run.smithi131.stdout: "fs_name": "cephfs" 2024-09-17T02:52:43.336 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:43.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:43.349 DEBUG:teuthology.orchestra.run.smithi131:> stat --file-system '--printf=%T 2024-09-17T02:52:43.349 DEBUG:teuthology.orchestra.run.smithi131:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:43.355 INFO:teuthology.orchestra.run.smithi131.stdout:fuseblk 2024-09-17T02:52:43.355 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:43.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:43.355 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:43.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:43.410 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-17T02:52:43.480 INFO:teuthology.orchestra.run.smithi131.stdout:./ 2024-09-17T02:52:43.480 INFO:teuthology.orchestra.run.smithi131.stdout:./parent 2024-09-17T02:52:43.480 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/flushed 2024-09-17T02:52:43.480 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/flushed/charlie 2024-09-17T02:52:43.480 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/flushed/alpha 2024-09-17T02:52:43.481 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/flushed/bravo 2024-09-17T02:52:43.481 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/unflushed 2024-09-17T02:52:43.481 INFO:teuthology.orchestra.run.smithi131.stdout:./parent/unflushed/jfile 2024-09-17T02:52:43.482 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:43.482 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:43.482 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:43.482 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:43.482 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-17T02:52:43.482 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:43.593 INFO:teuthology.orchestra.run.smithi131.stdout:1 2024-09-17T02:52:43.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:43.602 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:43.603 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:43.603 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:43.603 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:43.603 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-09-17T02:52:43.603 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:43.713 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627776 2024-09-17T02:52:43.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:43.723 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:43.723 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:43.723 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:43.723 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:43.723 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-09-17T02:52:43.723 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:43.827 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627777 2024-09-17T02:52:43.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:43.837 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:43.837 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:43.837 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:43.838 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:43.838 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-09-17T02:52:43.838 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:43.941 INFO:teuthology.orchestra.run.smithi131.stdout:1099511628280 2024-09-17T02:52:43.944 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:43.951 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:43.951 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:43.951 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:43.951 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:43.951 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-09-17T02:52:43.952 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:44.053 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627778 2024-09-17T02:52:44.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:44.063 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:44.063 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:44.063 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:44.063 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:44.063 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-09-17T02:52:44.063 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:44.173 INFO:teuthology.orchestra.run.smithi131.stdout:1099511628279 2024-09-17T02:52:44.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:44.182 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:44.183 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:44.183 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:44.183 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:44.183 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-09-17T02:52:44.183 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:44.290 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627779 2024-09-17T02:52:44.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:44.299 DEBUG:teuthology.orchestra.run.smithi131:> stdin-killer -- python3 -c ' 2024-09-17T02:52:44.299 DEBUG:teuthology.orchestra.run.smithi131:> import os 2024-09-17T02:52:44.299 DEBUG:teuthology.orchestra.run.smithi131:> import stat 2024-09-17T02:52:44.299 DEBUG:teuthology.orchestra.run.smithi131:> 2024-09-17T02:52:44.300 DEBUG:teuthology.orchestra.run.smithi131:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-09-17T02:52:44.300 DEBUG:teuthology.orchestra.run.smithi131:> ' 2024-09-17T02:52:44.405 INFO:teuthology.orchestra.run.smithi131.stdout:1099511627780 2024-09-17T02:52:44.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T02:52:44.415 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T02:52:44.415 DEBUG:teuthology.orchestra.run.smithi131:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:52:44.540 DEBUG:teuthology.orchestra.run.smithi189:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T02:52:44.617 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T02:52:44.617 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:52:44.618 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:44.618 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:44.624 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:44.624 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:44.671 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T02:52:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.885+0000 7f2ed133d640 1 -- 172.21.15.131:0/1270632087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ecc15c4b0 msgr2=0x7f2ecc160800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.885+0000 7f2ed133d640 1 --2- 172.21.15.131:0/1270632087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ecc15c4b0 0x7f2ecc160800 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009840 tx=0x7f2ec001c820 comp rx=0 tx=0).stop 2024-09-17T02:52:44.881 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.885+0000 7f2ed133d640 1 -- 172.21.15.131:0/1270632087 shutdown_connections 2024-09-17T02:52:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.885+0000 7f2ed133d640 1 --2- 172.21.15.131:0/1270632087 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ecc0cae60 0x7f2ecc0cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.885+0000 7f2ed133d640 1 --2- 172.21.15.131:0/1270632087 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ecc15c4b0 0x7f2ecc160800 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.885+0000 7f2ed133d640 1 -- 172.21.15.131:0/1270632087 >> 172.21.15.131:0/1270632087 conn(0x7f2ecc0cff40 msgr2=0x7f2ecc0d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.885+0000 7f2ed133d640 1 -- 172.21.15.131:0/1270632087 shutdown_connections 2024-09-17T02:52:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.885+0000 7f2ed133d640 1 -- 172.21.15.131:0/1270632087 wait complete. 2024-09-17T02:52:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ed133d640 1 Processor -- start 2024-09-17T02:52:44.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ed133d640 1 -- start start 2024-09-17T02:52:44.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ed133d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ecc0b5d00 0x7f2ecc157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:44.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ed133d640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ecc0cae60 0x7f2ecc1fa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ed133d640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ecc15c4b0 0x7f2ecc1506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ed133d640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc169da0 con 0x7f2ecc0b5d00 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ed133d640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f2ecc169c20 con 0x7f2ecc15c4b0 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ed133d640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f2ecc169f20 con 0x7f2ecc0cae60 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ecaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ecc0b5d00 0x7f2ecc157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ecaffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ecc0b5d00 0x7f2ecc157140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39828/0 (socket says 172.21.15.131:39828) 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ecaffd640 1 -- 172.21.15.131:0/1955301003 learned_addr learned my addr 172.21.15.131:0/1955301003 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ecb7fe640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ecc15c4b0 0x7f2ecc1506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ecaffd640 1 -- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ecc0cae60 msgr2=0x7f2ecc1fa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2eca7fc640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ecc0cae60 0x7f2ecc1fa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ecaffd640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ecc0cae60 0x7f2ecc1fa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ecaffd640 1 -- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ecc15c4b0 msgr2=0x7f2ecc1506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:44.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ecaffd640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ecc15c4b0 0x7f2ecc1506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ecaffd640 1 -- 172.21.15.131:0/1955301003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ecc150be0 con 0x7f2ecc0b5d00 2024-09-17T02:52:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ecb7fe640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ecc15c4b0 0x7f2ecc1506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ecaffd640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ecc0b5d00 0x7f2ecc157140 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c400 tx=0x7f2ebc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2eb37fe640 1 -- 172.21.15.131:0/1955301003 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc011790 con 0x7f2ecc0b5d00 2024-09-17T02:52:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2eb37fe640 1 -- 172.21.15.131:0/1955301003 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ebc017070 con 0x7f2ecc0b5d00 2024-09-17T02:52:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ed133d640 1 -- 172.21.15.131:0/1955301003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ecc15cdb0 con 0x7f2ecc0b5d00 2024-09-17T02:52:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2eb37fe640 1 -- 172.21.15.131:0/1955301003 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc011cc0 con 0x7f2ecc0b5d00 2024-09-17T02:52:44.887 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.889+0000 7f2ed133d640 1 -- 172.21.15.131:0/1955301003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ecc156940 con 0x7f2ecc0b5d00 2024-09-17T02:52:44.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.893+0000 7f2ed133d640 1 -- 172.21.15.131:0/1955301003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ecc0cb270 con 0x7f2ecc0b5d00 2024-09-17T02:52:44.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.893+0000 7f2eb37fe640 1 -- 172.21.15.131:0/1955301003 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2ebc01c460 con 0x7f2ecc0b5d00 2024-09-17T02:52:44.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.893+0000 7f2eb37fe640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2eb8073230 0x7f2eb8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:44.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.893+0000 7f2eb37fe640 1 -- 172.21.15.131:0/1955301003 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7f2ebc0941a0 con 0x7f2ecc0b5d00 2024-09-17T02:52:44.888 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.893+0000 7f2eca7fc640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2eb8073230 0x7f2eb8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:44.889 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.893+0000 7f2eca7fc640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2eb8073230 0x7f2eb8075740 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00af000 tx=0x7f2ec001b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:44.892 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:44.897+0000 7f2eb37fe640 1 -- 172.21.15.131:0/1955301003 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f2ebc05f010 con 0x7f2ecc0b5d00 2024-09-17T02:52:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.081+0000 7f2ed133d640 1 -- 172.21.15.131:0/1955301003 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2ecc06de50 con 0x7f2ecc0b5d00 2024-09-17T02:52:45.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.081+0000 7f2eb37fe640 1 -- 172.21.15.131:0/1955301003 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v124) ==== 80+0+248 (secure 0 0 0) 0x7f2ebc05e910 con 0x7f2ecc0b5d00 2024-09-17T02:52:45.076 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6835/2981077586 2024-09-18T02:52:34.677555+0000 2024-09-17T02:52:45.076 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6832/2981077586 2024-09-18T02:52:34.677555+0000 2024-09-17T02:52:45.076 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6835/2599844908 2024-09-18T02:52:16.917982+0000 2024-09-17T02:52:45.076 INFO:teuthology.orchestra.run.smithi131.stdout:172.21.15.131:6832/2599844908 2024-09-18T02:52:16.917982+0000 2024-09-17T02:52:45.076 INFO:teuthology.orchestra.run.smithi131.stderr:listed 4 entries 2024-09-17T02:52:45.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.081+0000 7f2ed133d640 1 -- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2eb8073230 msgr2=0x7f2eb8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:45.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.081+0000 7f2ed133d640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2eb8073230 0x7f2eb8075740 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2ec00af000 tx=0x7f2ec001b060 comp rx=0 tx=0).stop 2024-09-17T02:52:45.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.081+0000 7f2ed133d640 1 -- 172.21.15.131:0/1955301003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ecc0b5d00 msgr2=0x7f2ecc157140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:45.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.081+0000 7f2ed133d640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ecc0b5d00 0x7f2ecc157140 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c400 tx=0x7f2ebc00c8c0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.078 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.081+0000 7f2ed133d640 1 -- 172.21.15.131:0/1955301003 shutdown_connections 2024-09-17T02:52:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.081+0000 7f2ed133d640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2eb8073230 0x7f2eb8075740 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.081+0000 7f2ed133d640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f2ecc15c4b0 0x7f2ecc1506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.081+0000 7f2ed133d640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f2ecc0cae60 0x7f2ecc1fa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.081+0000 7f2ed133d640 1 --2- 172.21.15.131:0/1955301003 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f2ecc0b5d00 0x7f2ecc157140 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.085+0000 7f2ed133d640 1 -- 172.21.15.131:0/1955301003 >> 172.21.15.131:0/1955301003 conn(0x7f2ecc0cff40 msgr2=0x7f2ecc0afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.085+0000 7f2ed133d640 1 -- 172.21.15.131:0/1955301003 shutdown_connections 2024-09-17T02:52:45.079 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.085+0000 7f2ed133d640 1 -- 172.21.15.131:0/1955301003 wait complete. 2024-09-17T02:52:45.097 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi131.front.sepia.ceph.com... 2024-09-17T02:52:45.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:45.098 DEBUG:teuthology.orchestra.run.smithi131:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T02:52:45.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi131.stderr:ceph-fuse[52708]: fuse finished with error 0 and tester_r 0 2024-09-17T02:52:45.110 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:52:45.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:45.111 DEBUG:teuthology.orchestra.run.smithi131:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T02:52:45.202 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:52:45.202 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:52:45.202 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:45.209 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:52:45.209 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:52:45.209 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:45.210 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:45.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:52:45.251 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T02:52:45.252 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:52:45.252 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:45.257 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:52:45.258 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:45.457 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.461+0000 7fe2eca31640 1 -- 172.21.15.131:0/381540200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e80d2dd0 msgr2=0x7fe2e80d31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:45.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.461+0000 7fe2eca31640 1 --2- 172.21.15.131:0/381540200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e80d2dd0 0x7fe2e80d31e0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009900 tx=0x7fe2dc01c8e0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.461+0000 7fe2eca31640 1 -- 172.21.15.131:0/381540200 shutdown_connections 2024-09-17T02:52:45.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.461+0000 7fe2eca31640 1 --2- 172.21.15.131:0/381540200 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2e80b5e40 0x7fe2e80b6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.461+0000 7fe2eca31640 1 --2- 172.21.15.131:0/381540200 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e80d2dd0 0x7fe2e80d31e0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.461+0000 7fe2eca31640 1 -- 172.21.15.131:0/381540200 >> 172.21.15.131:0/381540200 conn(0x7fe2e80d0e40 msgr2=0x7fe2e80d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:45.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.461+0000 7fe2eca31640 1 -- 172.21.15.131:0/381540200 shutdown_connections 2024-09-17T02:52:45.458 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.461+0000 7fe2eca31640 1 -- 172.21.15.131:0/381540200 wait complete. 2024-09-17T02:52:45.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2eca31640 1 Processor -- start 2024-09-17T02:52:45.459 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2eca31640 1 -- start start 2024-09-17T02:52:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2eca31640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2e80b5e40 0x7fe2e81515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2eca31640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e80d2dd0 0x7fe2e8161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2eca31640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2e815c3f0 0x7fe2e8161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2eca31640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e80b3990 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2eca31640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e80b3810 con 0x7fe2e815c3f0 2024-09-17T02:52:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2eca31640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe2e80b3b10 con 0x7fe2e80b5e40 2024-09-17T02:52:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e80d2dd0 0x7fe2e8161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e5d9b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e80d2dd0 0x7fe2e8161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39842/0 (socket says 172.21.15.131:39842) 2024-09-17T02:52:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e5d9b640 1 -- 172.21.15.131:0/419412565 learned_addr learned my addr 172.21.15.131:0/419412565 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:45.462 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e659c640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2e80b5e40 0x7fe2e81515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e6d9d640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2e815c3f0 0x7fe2e8161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e5d9b640 1 -- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2e80b5e40 msgr2=0x7fe2e81515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e5d9b640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2e80b5e40 0x7fe2e81515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e5d9b640 1 -- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2e815c3f0 msgr2=0x7fe2e8161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e5d9b640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2e815c3f0 0x7fe2e8161580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e5d9b640 1 -- 172.21.15.131:0/419412565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e80ce540 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e6d9d640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2e815c3f0 0x7fe2e8161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2e5d9b640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e80d2dd0 0x7fe2e8161040 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009850 tx=0x7fe2dc0a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2d6ffd640 1 -- 172.21.15.131:0/419412565 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0b2040 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2d6ffd640 1 -- 172.21.15.131:0/419412565 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2dc0043b0 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2eca31640 1 -- 172.21.15.131:0/419412565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e80b98b0 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2d6ffd640 1 -- 172.21.15.131:0/419412565 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc005280 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.463 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.465+0000 7fe2eca31640 1 -- 172.21.15.131:0/419412565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e80b95d0 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.469+0000 7fe2eca31640 1 -- 172.21.15.131:0/419412565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e80b6250 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.469+0000 7fe2d6ffd640 1 -- 172.21.15.131:0/419412565 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe2dc005420 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.469+0000 7fe2d6ffd640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe2c8073230 0x7fe2c8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:45.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.469+0000 7fe2d6ffd640 1 -- 172.21.15.131:0/419412565 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7fe2dc0be070 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.464 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.469+0000 7fe2e659c640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe2c8073230 0x7fe2c8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:45.465 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.469+0000 7fe2e659c640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe2c8073230 0x7fe2c8075740 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8000f70 tx=0x7fe2d8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:45.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.473+0000 7fe2d6ffd640 1 -- 172.21.15.131:0/419412565 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fe2dc0164d0 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.673 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.677+0000 7fe2eca31640 1 -- 172.21.15.131:0/419412565 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2e80b6dc0 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2d6ffd640 1 -- 172.21.15.131:0/419412565 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5239 (secure 0 0 0) 0x7fe2dc0f9020 con 0x7fe2e80d2dd0 2024-09-17T02:52:45.675 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:45.675 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":113,"btime":"2024-09-17T02:52:39:086790+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:52:39.086771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8880":{"gid":8880,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/4011259083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4011259083},{"type":"v1","addr":"172.21.15.131:6835","nonce":4011259083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8880,"qdb_cluster":[8880]},"id":8}]} 2024-09-17T02:52:45.675 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 113 2024-09-17T02:52:45.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 -- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe2c8073230 msgr2=0x7fe2c8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:45.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe2c8073230 0x7fe2c8075740 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8000f70 tx=0x7fe2d8009040 comp rx=0 tx=0).stop 2024-09-17T02:52:45.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 -- 172.21.15.131:0/419412565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e80d2dd0 msgr2=0x7fe2e8161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:45.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e80d2dd0 0x7fe2e8161040 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc009850 tx=0x7fe2dc0a5620 comp rx=0 tx=0).stop 2024-09-17T02:52:45.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 -- 172.21.15.131:0/419412565 shutdown_connections 2024-09-17T02:52:45.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe2c8073230 0x7fe2c8075740 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe2e815c3f0 0x7fe2e8161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe2e80d2dd0 0x7fe2e8161040 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 --2- 172.21.15.131:0/419412565 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe2e80b5e40 0x7fe2e81515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.677 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 -- 172.21.15.131:0/419412565 >> 172.21.15.131:0/419412565 conn(0x7fe2e80d0e40 msgr2=0x7fe2e8162050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:45.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 -- 172.21.15.131:0/419412565 shutdown_connections 2024-09-17T02:52:45.678 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.681+0000 7fe2eca31640 1 -- 172.21.15.131:0/419412565 wait complete. 2024-09-17T02:52:45.697 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:45.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.861+0000 7f63e8fe2640 1 -- 172.21.15.131:0/1768908305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c8004810 msgr2=0x7f63c8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:45.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.861+0000 7f63e8fe2640 1 --2- 172.21.15.131:0/1768908305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c8004810 0x7f63c8004c20 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f63d000c260 tx=0x7f63d001f6c0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.861+0000 7f63e8fe2640 1 -- 172.21.15.131:0/1768908305 shutdown_connections 2024-09-17T02:52:45.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.861+0000 7f63e8fe2640 1 --2- 172.21.15.131:0/1768908305 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63c8004810 0x7f63c8004c20 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.861+0000 7f63e8fe2640 1 -- 172.21.15.131:0/1768908305 >> 172.21.15.131:0/1768908305 conn(0x7f63e406eed0 msgr2=0x7f63e406f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:45.855 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.861+0000 7f63e8fe2640 1 -- 172.21.15.131:0/1768908305 shutdown_connections 2024-09-17T02:52:45.856 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.861+0000 7f63e8fe2640 1 -- 172.21.15.131:0/1768908305 wait complete. 2024-09-17T02:52:45.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.861+0000 7f63e8fe2640 1 Processor -- start 2024-09-17T02:52:45.857 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.861+0000 7f63e8fe2640 1 -- start start 2024-09-17T02:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e8fe2640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f63c8004810 0x7f63e413a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e8fe2640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63e4143d00 0x7f63e40c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e8fe2640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e41536c0 0x7f63e4153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e8fe2640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f63e40cff00 con 0x7f63e41536c0 2024-09-17T02:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e8fe2640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f63e415bea0 con 0x7f63e4143d00 2024-09-17T02:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e8fe2640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f63e40d0080 con 0x7f63c8004810 2024-09-17T02:52:45.859 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e41536c0 0x7f63e4153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e2d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e41536c0 0x7f63e4153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39850/0 (socket says 172.21.15.131:39850) 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e2d9d640 1 -- 172.21.15.131:0/856664030 learned_addr learned my addr 172.21.15.131:0/856664030 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e259c640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f63c8004810 0x7f63e413a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e1d9b640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63e4143d00 0x7f63e40c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e2d9d640 1 -- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f63c8004810 msgr2=0x7f63e413a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e2d9d640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f63c8004810 0x7f63e413a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e2d9d640 1 -- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63e4143d00 msgr2=0x7f63e40c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e2d9d640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63e4143d00 0x7f63e40c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e2d9d640 1 -- 172.21.15.131:0/856664030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e40b3b00 con 0x7f63e41536c0 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e1d9b640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63e4143d00 0x7f63e40c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e2d9d640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e41536c0 0x7f63e4153ad0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f63e415cb60 tx=0x7f63dc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63daffd640 1 -- 172.21.15.131:0/856664030 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc004b30 con 0x7f63e41536c0 2024-09-17T02:52:45.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63daffd640 1 -- 172.21.15.131:0/856664030 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63dc004cf0 con 0x7f63e41536c0 2024-09-17T02:52:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e8fe2640 1 -- 172.21.15.131:0/856664030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e4154010 con 0x7f63e41536c0 2024-09-17T02:52:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63daffd640 1 -- 172.21.15.131:0/856664030 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc003be0 con 0x7f63e41536c0 2024-09-17T02:52:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e8fe2640 1 -- 172.21.15.131:0/856664030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e40c7420 con 0x7f63e41536c0 2024-09-17T02:52:45.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e8fe2640 1 -- 172.21.15.131:0/856664030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e40d05e0 con 0x7f63e41536c0 2024-09-17T02:52:45.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63daffd640 1 -- 172.21.15.131:0/856664030 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f63dc004030 con 0x7f63e41536c0 2024-09-17T02:52:45.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63daffd640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f63cc073200 0x7f63cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:45.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63daffd640 1 -- 172.21.15.131:0/856664030 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7f63dc096980 con 0x7f63e41536c0 2024-09-17T02:52:45.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.865+0000 7f63e259c640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f63cc073200 0x7f63cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:45.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.869+0000 7f63e259c640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f63cc073200 0x7f63cc075710 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f63e406b700 tx=0x7f63d4006b30 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:45.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:45.873+0000 7f63daffd640 1 -- 172.21.15.131:0/856664030 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f63e4154010 con 0x7f63e41536c0 2024-09-17T02:52:46.039 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:52:46.042 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:52:46.069 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.073+0000 7f63e8fe2640 1 -- 172.21.15.131:0/856664030 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63e415b640 con 0x7f63e41536c0 2024-09-17T02:52:46.070 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63daffd640 1 -- 172.21.15.131:0/856664030 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5239 (secure 0 0 0) 0x7f63dc061880 con 0x7f63e41536c0 2024-09-17T02:52:46.071 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:46.071 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":113,"btime":"2024-09-17T02:52:39:086790+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:52:39.086771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8880":{"gid":8880,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/4011259083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4011259083},{"type":"v1","addr":"172.21.15.131:6835","nonce":4011259083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8880,"qdb_cluster":[8880]},"id":8}]} 2024-09-17T02:52:46.071 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 113 2024-09-17T02:52:46.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 -- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f63cc073200 msgr2=0x7f63cc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f63cc073200 0x7f63cc075710 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f63e406b700 tx=0x7f63d4006b30 comp rx=0 tx=0).stop 2024-09-17T02:52:46.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 -- 172.21.15.131:0/856664030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e41536c0 msgr2=0x7f63e4153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e41536c0 0x7f63e4153ad0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f63e415cb60 tx=0x7f63dc00c6a0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 -- 172.21.15.131:0/856664030 shutdown_connections 2024-09-17T02:52:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f63cc073200 0x7f63cc075710 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f63e41536c0 0x7f63e4153ad0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f63e4143d00 0x7f63e40c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 --2- 172.21.15.131:0/856664030 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f63c8004810 0x7f63e413a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 -- 172.21.15.131:0/856664030 >> 172.21.15.131:0/856664030 conn(0x7f63e406eed0 msgr2=0x7f63e40b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 -- 172.21.15.131:0/856664030 shutdown_connections 2024-09-17T02:52:46.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.077+0000 7f63e8fe2640 1 -- 172.21.15.131:0/856664030 wait complete. 2024-09-17T02:52:46.094 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:52:46.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.297+0000 7fba06020640 1 -- 172.21.15.131:0/2447393635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba000cc300 msgr2=0x7fba000cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.297+0000 7fba06020640 1 --2- 172.21.15.131:0/2447393635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba000cc300 0x7fba000cff00 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009900 tx=0x7fb9f801c8e0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.297+0000 7fba06020640 1 -- 172.21.15.131:0/2447393635 shutdown_connections 2024-09-17T02:52:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.297+0000 7fba06020640 1 --2- 172.21.15.131:0/2447393635 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba000d0440 0x7fba0015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.297+0000 7fba06020640 1 --2- 172.21.15.131:0/2447393635 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba000cc300 0x7fba000cff00 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.297+0000 7fba06020640 1 -- 172.21.15.131:0/2447393635 >> 172.21.15.131:0/2447393635 conn(0x7fba0006eed0 msgr2=0x7fba0006f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.297+0000 7fba06020640 1 -- 172.21.15.131:0/2447393635 shutdown_connections 2024-09-17T02:52:46.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.297+0000 7fba06020640 1 -- 172.21.15.131:0/2447393635 wait complete. 2024-09-17T02:52:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.297+0000 7fba06020640 1 Processor -- start 2024-09-17T02:52:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.297+0000 7fba06020640 1 -- start start 2024-09-17T02:52:46.299 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fba06020640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba000cc300 0x7fba00152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:46.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fba06020640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba000d0440 0x7fba000c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:46.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fba06020640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba0015bfc0 0x7fba000c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:46.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fba06020640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fba00168970 con 0x7fba000d0440 2024-09-17T02:52:46.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fba06020640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fba001687f0 con 0x7fba0015bfc0 2024-09-17T02:52:46.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fba06020640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fba00168af0 con 0x7fba000cc300 2024-09-17T02:52:46.300 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9feffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba000d0440 0x7fba000c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:46.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9feffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba000d0440 0x7fba000c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39860/0 (socket says 172.21.15.131:39860) 2024-09-17T02:52:46.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9feffd640 1 -- 172.21.15.131:0/1671104602 learned_addr learned my addr 172.21.15.131:0/1671104602 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:46.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9fffff640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba0015bfc0 0x7fba000c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:46.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9ff7fe640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba000cc300 0x7fba00152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:46.301 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9feffd640 1 -- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba000cc300 msgr2=0x7fba00152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9feffd640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba000cc300 0x7fba00152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9feffd640 1 -- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba0015bfc0 msgr2=0x7fba000c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9feffd640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba0015bfc0 0x7fba000c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9feffd640 1 -- 172.21.15.131:0/1671104602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba001562e0 con 0x7fba000d0440 2024-09-17T02:52:46.302 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9fffff640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba0015bfc0 0x7fba000c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:52:46.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9ff7fe640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba000cc300 0x7fba00152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:52:46.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9feffd640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba000d0440 0x7fba000c92c0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80058d0 tx=0x7fb9e800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:46.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9f7fff640 1 -- 172.21.15.131:0/1671104602 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8017070 con 0x7fba000d0440 2024-09-17T02:52:46.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9f7fff640 1 -- 172.21.15.131:0/1671104602 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9e8009d20 con 0x7fba000d0440 2024-09-17T02:52:46.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fba06020640 1 -- 172.21.15.131:0/1671104602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba00152860 con 0x7fba000d0440 2024-09-17T02:52:46.303 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9f7fff640 1 -- 172.21.15.131:0/1671104602 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e80106c0 con 0x7fba000d0440 2024-09-17T02:52:46.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fba06020640 1 -- 172.21.15.131:0/1671104602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba00153970 con 0x7fba000d0440 2024-09-17T02:52:46.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fba06020640 1 -- 172.21.15.131:0/1671104602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba00153400 con 0x7fba000d0440 2024-09-17T02:52:46.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.301+0000 7fb9f7fff640 1 -- 172.21.15.131:0/1671104602 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb9e801a070 con 0x7fba000d0440 2024-09-17T02:52:46.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.305+0000 7fb9f7fff640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9e0073200 0x7fb9e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:46.304 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.305+0000 7fb9f7fff640 1 -- 172.21.15.131:0/1671104602 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7fb9e80946a0 con 0x7fba000d0440 2024-09-17T02:52:46.305 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.305+0000 7fb9ff7fe640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9e0073200 0x7fb9e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:46.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.305+0000 7fb9ff7fe640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9e0073200 0x7fb9e0075710 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80af000 tx=0x7fb9f801b120 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:46.306 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.309+0000 7fb9f7fff640 1 -- 172.21.15.131:0/1671104602 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fb9e805f480 con 0x7fba000d0440 2024-09-17T02:52:46.468 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.473+0000 7fba06020640 1 -- 172.21.15.131:0/1671104602 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba00068e50 con 0x7fba000d0440 2024-09-17T02:52:46.471 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fb9f7fff640 1 -- 172.21.15.131:0/1671104602 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+17327 (secure 0 0 0) 0x7fb9e805ed80 con 0x7fba000d0440 2024-09-17T02:52:46.471 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:46.471 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":124,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:52:34.677596+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:51:52.442187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-17T02:51:53.731428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6835/2981077586":"2024-09-18T02:52:34.677555+0000","172.21.15.131:6832/2981077586":"2024-09-18T02:52:34.677555+0000","172.21.15.131:6835/2599844908":"2024-09-18T02:52:16.917982+0000","172.21.15.131:6832/2599844908":"2024-09-18T02:52:16.917982+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:52:46.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 -- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9e0073200 msgr2=0x7fb9e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9e0073200 0x7fb9e0075710 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80af000 tx=0x7fb9f801b120 comp rx=0 tx=0).stop 2024-09-17T02:52:46.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 -- 172.21.15.131:0/1671104602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba000d0440 msgr2=0x7fba000c92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba000d0440 0x7fba000c92c0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fb9e80058d0 tx=0x7fb9e800dde0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 -- 172.21.15.131:0/1671104602 shutdown_connections 2024-09-17T02:52:46.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb9e0073200 0x7fb9e0075710 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba0015bfc0 0x7fba000c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba000d0440 0x7fba000c92c0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 --2- 172.21.15.131:0/1671104602 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba000cc300 0x7fba00152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 -- 172.21.15.131:0/1671104602 >> 172.21.15.131:0/1671104602 conn(0x7fba0006eed0 msgr2=0x7fba0015cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:46.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 -- 172.21.15.131:0/1671104602 shutdown_connections 2024-09-17T02:52:46.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.477+0000 7fba06020640 1 -- 172.21.15.131:0/1671104602 wait complete. 2024-09-17T02:52:46.492 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T02:52:46.493 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:46.649 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.653+0000 7feb75bd6640 1 -- 172.21.15.131:0/3665515862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb58004810 msgr2=0x7feb58004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.653+0000 7feb75bd6640 1 --2- 172.21.15.131:0/3665515862 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb58004810 0x7feb58004c20 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7feb640016f0 tx=0x7feb6401c7b0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.653+0000 7feb6ffff640 1 -- 172.21.15.131:0/3665515862 reap_dead start 2024-09-17T02:52:46.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.653+0000 7feb75bd6640 1 -- 172.21.15.131:0/3665515862 shutdown_connections 2024-09-17T02:52:46.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.653+0000 7feb75bd6640 1 -- 172.21.15.131:0/3665515862 >> 172.21.15.131:0/3665515862 conn(0x7feb7006eed0 msgr2=0x7feb7006f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:46.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.653+0000 7feb75bd6640 1 -- 172.21.15.131:0/3665515862 shutdown_connections 2024-09-17T02:52:46.650 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.653+0000 7feb75bd6640 1 -- 172.21.15.131:0/3665515862 wait complete. 2024-09-17T02:52:46.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb75bd6640 1 Processor -- start 2024-09-17T02:52:46.651 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb75bd6640 1 -- start start 2024-09-17T02:52:46.653 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb75bd6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb58004810 0x7feb70160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb75bd6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb7015cb90 0x7feb70141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb75bd6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feb70160c00 0x7feb70161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb75bd6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7feb701647e0 con 0x7feb7015cb90 2024-09-17T02:52:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb75bd6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7feb70164660 con 0x7feb58004810 2024-09-17T02:52:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb75bd6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7feb70164960 con 0x7feb70160c00 2024-09-17T02:52:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb6effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb7015cb90 0x7feb70141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb6effd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb7015cb90 0x7feb70141ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39888/0 (socket says 172.21.15.131:39888) 2024-09-17T02:52:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb6effd640 1 -- 172.21.15.131:0/3973383965 learned_addr learned my addr 172.21.15.131:0/3973383965 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb6f7fe640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb58004810 0x7feb70160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb6effd640 1 -- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feb70160c00 msgr2=0x7feb70161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.654 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb6effd640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feb70160c00 0x7feb70161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb6effd640 1 -- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb58004810 msgr2=0x7feb70160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb6effd640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb58004810 0x7feb70160060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb6effd640 1 -- 172.21.15.131:0/3973383965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb70161550 con 0x7feb7015cb90 2024-09-17T02:52:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb6f7fe640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb58004810 0x7feb70160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb6effd640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb7015cb90 0x7feb70141ba0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7feb64006040 tx=0x7feb640013b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb57fff640 1 -- 172.21.15.131:0/3973383965 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb640a5610 con 0x7feb7015cb90 2024-09-17T02:52:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb57fff640 1 -- 172.21.15.131:0/3973383965 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb640a58a0 con 0x7feb7015cb90 2024-09-17T02:52:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb75bd6640 1 -- 172.21.15.131:0/3973383965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7015d2e0 con 0x7feb7015cb90 2024-09-17T02:52:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb57fff640 1 -- 172.21.15.131:0/3973383965 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb640a5cd0 con 0x7feb7015cb90 2024-09-17T02:52:46.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.657+0000 7feb75bd6640 1 -- 172.21.15.131:0/3973383965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7015d8a0 con 0x7feb7015cb90 2024-09-17T02:52:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.661+0000 7feb75bd6640 1 -- 172.21.15.131:0/3973383965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7015ba80 con 0x7feb7015cb90 2024-09-17T02:52:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.661+0000 7feb57fff640 1 -- 172.21.15.131:0/3973383965 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7feb64003600 con 0x7feb7015cb90 2024-09-17T02:52:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.661+0000 7feb57fff640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb5c073230 0x7feb5c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.661+0000 7feb57fff640 1 -- 172.21.15.131:0/3973383965 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7feb6412e3d0 con 0x7feb7015cb90 2024-09-17T02:52:46.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.661+0000 7feb6f7fe640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb5c073230 0x7feb5c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:46.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.661+0000 7feb6f7fe640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb5c073230 0x7feb5c075740 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7feb60001890 tx=0x7feb600014c0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:46.660 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.665+0000 7feb57fff640 1 -- 172.21.15.131:0/3973383965 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7feb70161550 con 0x7feb7015cb90 2024-09-17T02:52:46.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.873+0000 7feb75bd6640 1 -- 172.21.15.131:0/3973383965 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb700afb90 con 0x7feb7015cb90 2024-09-17T02:52:46.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.873+0000 7feb57fff640 1 -- 172.21.15.131:0/3973383965 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5239 (secure 0 0 0) 0x7feb64016510 con 0x7feb7015cb90 2024-09-17T02:52:46.868 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:46.868 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":113,"btime":"2024-09-17T02:52:39:086790+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:52:39.086771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8880":{"gid":8880,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/4011259083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4011259083},{"type":"v1","addr":"172.21.15.131:6835","nonce":4011259083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8880,"qdb_cluster":[8880]},"id":8}]} 2024-09-17T02:52:46.868 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 113 2024-09-17T02:52:46.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 -- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb5c073230 msgr2=0x7feb5c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb5c073230 0x7feb5c075740 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7feb60001890 tx=0x7feb600014c0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 -- 172.21.15.131:0/3973383965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb7015cb90 msgr2=0x7feb70141ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb7015cb90 0x7feb70141ba0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7feb64006040 tx=0x7feb640013b0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 -- 172.21.15.131:0/3973383965 shutdown_connections 2024-09-17T02:52:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7feb5c073230 0x7feb5c075740 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7feb70160c00 0x7feb70161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7feb7015cb90 0x7feb70141ba0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 --2- 172.21.15.131:0/3973383965 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7feb58004810 0x7feb70160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 -- 172.21.15.131:0/3973383965 >> 172.21.15.131:0/3973383965 conn(0x7feb7006eed0 msgr2=0x7feb700caf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 -- 172.21.15.131:0/3973383965 shutdown_connections 2024-09-17T02:52:46.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:46.877+0000 7feb75bd6640 1 -- 172.21.15.131:0/3973383965 wait complete. 2024-09-17T02:52:46.891 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:52:47.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.053+0000 7f8641e7a640 1 -- 172.21.15.131:0/2784823256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f863c074660 msgr2=0x7f863c0caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.053+0000 7f8641e7a640 1 --2- 172.21.15.131:0/2784823256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f863c074660 0x7f863c0caed0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f863400adc0 tx=0x7f863401dc90 comp rx=0 tx=0).stop 2024-09-17T02:52:47.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.053+0000 7f8641e7a640 1 -- 172.21.15.131:0/2784823256 shutdown_connections 2024-09-17T02:52:47.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.053+0000 7f8641e7a640 1 --2- 172.21.15.131:0/2784823256 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f863c074660 0x7f863c0caed0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.053+0000 7f8641e7a640 1 --2- 172.21.15.131:0/2784823256 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f863c0d0310 0x7f863c074120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.053+0000 7f8641e7a640 1 -- 172.21.15.131:0/2784823256 >> 172.21.15.131:0/2784823256 conn(0x7f863c06eed0 msgr2=0x7f863c06f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:47.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.053+0000 7f8641e7a640 1 -- 172.21.15.131:0/2784823256 shutdown_connections 2024-09-17T02:52:47.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.053+0000 7f8641e7a640 1 -- 172.21.15.131:0/2784823256 wait complete. 2024-09-17T02:52:47.050 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8641e7a640 1 Processor -- start 2024-09-17T02:52:47.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8641e7a640 1 -- start start 2024-09-17T02:52:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8641e7a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f863c074660 0x7f863c15ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8641e7a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f863c0cff00 0x7f863c15c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8641e7a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f863c0d0310 0x7f863c15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.053 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8641e7a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f863c167ff0 con 0x7f863c0d0310 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8641e7a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f863c167e70 con 0x7f863c074660 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8641e7a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f863c168170 con 0x7f863c0cff00 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f863bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f863c0d0310 0x7f863c15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f863bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f863c0d0310 0x7f863c15cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39912/0 (socket says 172.21.15.131:39912) 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f863bfff640 1 -- 172.21.15.131:0/2779795091 learned_addr learned my addr 172.21.15.131:0/2779795091 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f863b7fe640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f863c074660 0x7f863c15ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f863bfff640 1 -- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f863c0cff00 msgr2=0x7f863c15c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f863bfff640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f863c0cff00 0x7f863c15c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f863bfff640 1 -- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f863c074660 msgr2=0x7f863c15ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f863bfff640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f863c074660 0x7f863c15ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f863bfff640 1 -- 172.21.15.131:0/2779795091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c0b2a40 con 0x7f863c0d0310 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f863bfff640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f863c0d0310 0x7f863c15cc10 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f863400ace0 tx=0x7f863400a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:47.054 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8623fff640 1 -- 172.21.15.131:0/2779795091 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86340b3040 con 0x7f863c0d0310 2024-09-17T02:52:47.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8623fff640 1 -- 172.21.15.131:0/2779795091 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8634004310 con 0x7f863c0d0310 2024-09-17T02:52:47.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8641e7a640 1 -- 172.21.15.131:0/2779795091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c0b5c00 con 0x7f863c0d0310 2024-09-17T02:52:47.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8623fff640 1 -- 172.21.15.131:0/2779795091 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634004760 con 0x7f863c0d0310 2024-09-17T02:52:47.055 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.057+0000 7f8641e7a640 1 -- 172.21.15.131:0/2779795091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c0b2760 con 0x7f863c0d0310 2024-09-17T02:52:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.061+0000 7f8641e7a640 1 -- 172.21.15.131:0/2779795091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863c0caed0 con 0x7f863c0d0310 2024-09-17T02:52:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.061+0000 7f8623fff640 1 -- 172.21.15.131:0/2779795091 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f86340c0060 con 0x7f863c0d0310 2024-09-17T02:52:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.061+0000 7f8623fff640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8628073200 0x7f8628075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.061+0000 7f8623fff640 1 -- 172.21.15.131:0/2779795091 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7f863412e900 con 0x7f863c0d0310 2024-09-17T02:52:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.061+0000 7f863b7fe640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8628073200 0x7f8628075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:47.056 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.061+0000 7f863b7fe640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8628073200 0x7f8628075710 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f862c0016d0 tx=0x7f862c009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:47.059 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.065+0000 7f8623fff640 1 -- 172.21.15.131:0/2779795091 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8634017880 con 0x7f863c0d0310 2024-09-17T02:52:47.267 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.273+0000 7f8641e7a640 1 -- 172.21.15.131:0/2779795091 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f863c06f660 con 0x7f863c0d0310 2024-09-17T02:52:47.268 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.273+0000 7f8623fff640 1 -- 172.21.15.131:0/2779795091 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) ==== 95+0+5239 (secure 0 0 0) 0x7f86340f9880 con 0x7f863c0d0310 2024-09-17T02:52:47.269 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:47.269 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":113,"btime":"2024-09-17T02:52:39:086790+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T02:51:54.764322+0000","modified":"2024-09-17T02:52:39.086771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":124,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8880},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8880":{"gid":8880,"name":"a","rank":0,"incarnation":110,"state":"up:active","state_seq":4,"addr":"172.21.15.131:6835/4011259083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":4011259083},{"type":"v1","addr":"172.21.15.131:6835","nonce":4011259083}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8880,"qdb_cluster":[8880]},"id":8}]} 2024-09-17T02:52:47.269 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 113 2024-09-17T02:52:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 -- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8628073200 msgr2=0x7f8628075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8628073200 0x7f8628075710 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f862c0016d0 tx=0x7f862c009040 comp rx=0 tx=0).stop 2024-09-17T02:52:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 -- 172.21.15.131:0/2779795091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f863c0d0310 msgr2=0x7f863c15cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f863c0d0310 0x7f863c15cc10 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f863400ace0 tx=0x7f863400a3d0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 -- 172.21.15.131:0/2779795091 shutdown_connections 2024-09-17T02:52:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8628073200 0x7f8628075710 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f863c0d0310 0x7f863c15cc10 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f863c0cff00 0x7f863c15c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 --2- 172.21.15.131:0/2779795091 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f863c074660 0x7f863c15ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.271 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 -- 172.21.15.131:0/2779795091 >> 172.21.15.131:0/2779795091 conn(0x7f863c06eed0 msgr2=0x7f863c1509b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 -- 172.21.15.131:0/2779795091 shutdown_connections 2024-09-17T02:52:47.272 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.277+0000 7f8641e7a640 1 -- 172.21.15.131:0/2779795091 wait complete. 2024-09-17T02:52:47.291 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:52:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.437+0000 7f53ffbfd640 1 -- 172.21.15.131:0/3328272964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f80cd090 msgr2=0x7f53f815c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.437+0000 7f53ffbfd640 1 --2- 172.21.15.131:0/3328272964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f80cd090 0x7f53f815c5e0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f53f8073d00 tx=0x7f53e801c840 comp rx=0 tx=0).stop 2024-09-17T02:52:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.437+0000 7f53ffbfd640 1 -- 172.21.15.131:0/3328272964 shutdown_connections 2024-09-17T02:52:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.437+0000 7f53ffbfd640 1 --2- 172.21.15.131:0/3328272964 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f53f80cb6c0 0x7f53f80cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.437+0000 7f53ffbfd640 1 --2- 172.21.15.131:0/3328272964 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f80cd090 0x7f53f815c5e0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.437+0000 7f53ffbfd640 1 -- 172.21.15.131:0/3328272964 >> 172.21.15.131:0/3328272964 conn(0x7f53f806eed0 msgr2=0x7f53f806f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.437+0000 7f53ffbfd640 1 -- 172.21.15.131:0/3328272964 shutdown_connections 2024-09-17T02:52:47.432 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.437+0000 7f53ffbfd640 1 -- 172.21.15.131:0/3328272964 wait complete. 2024-09-17T02:52:47.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.437+0000 7f53ffbfd640 1 Processor -- start 2024-09-17T02:52:47.433 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.437+0000 7f53ffbfd640 1 -- start start 2024-09-17T02:52:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ffbfd640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f80ca5c0 0x7f53f81505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ffbfd640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f53f80cb6c0 0x7f53f80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ffbfd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f80c9c60 0x7f53f8157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ffbfd640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f53f81699f0 con 0x7f53f80c9c60 2024-09-17T02:52:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ffbfd640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f53f8169870 con 0x7f53f80ca5c0 2024-09-17T02:52:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ffbfd640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f53f8169b70 con 0x7f53f80cb6c0 2024-09-17T02:52:47.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fe19a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f80c9c60 0x7f53f8157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fe19a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f80c9c60 0x7f53f8157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39930/0 (socket says 172.21.15.131:39930) 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fe19a640 1 -- 172.21.15.131:0/2253617153 learned_addr learned my addr 172.21.15.131:0/2253617153 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fd999640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f80ca5c0 0x7f53f81505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fd198640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f53f80cb6c0 0x7f53f80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fe19a640 1 -- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f53f80cb6c0 msgr2=0x7f53f80b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fe19a640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f53f80cb6c0 0x7f53f80b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fe19a640 1 -- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f80ca5c0 msgr2=0x7f53f81505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fe19a640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f80ca5c0 0x7f53f81505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fe19a640 1 -- 172.21.15.131:0/2253617153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f815f8e0 con 0x7f53f80c9c60 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fd999640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f80ca5c0 0x7f53f81505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53fe19a640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f80c9c60 0x7f53f8157de0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f53f000b9e0 tx=0x7f53f000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ee7fc640 1 -- 172.21.15.131:0/2253617153 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0019070 con 0x7f53f80c9c60 2024-09-17T02:52:47.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ee7fc640 1 -- 172.21.15.131:0/2253617153 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53f000cc10 con 0x7f53f80c9c60 2024-09-17T02:52:47.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ee7fc640 1 -- 172.21.15.131:0/2253617153 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f00146a0 con 0x7f53f80c9c60 2024-09-17T02:52:47.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ffbfd640 1 -- 172.21.15.131:0/2253617153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f815fbc0 con 0x7f53f80c9c60 2024-09-17T02:52:47.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ffbfd640 1 -- 172.21.15.131:0/2253617153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f80c9740 con 0x7f53f80c9c60 2024-09-17T02:52:47.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ffbfd640 1 -- 172.21.15.131:0/2253617153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f80cbad0 con 0x7f53f80c9c60 2024-09-17T02:52:47.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ee7fc640 1 -- 172.21.15.131:0/2253617153 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f53f001c070 con 0x7f53f80c9c60 2024-09-17T02:52:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ee7fc640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f53dc073200 0x7f53dc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.441+0000 7f53ee7fc640 1 -- 172.21.15.131:0/2253617153 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7f53f0094210 con 0x7f53f80c9c60 2024-09-17T02:52:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.445+0000 7f53fd999640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f53dc073200 0x7f53dc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:47.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.445+0000 7f53fd999640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f53dc073200 0x7f53dc075710 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f53f4001630 tx=0x7f53f4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:47.442 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.449+0000 7f53ee7fc640 1 -- 172.21.15.131:0/2253617153 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f53f005eff0 con 0x7f53f80c9c60 2024-09-17T02:52:47.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.605+0000 7f53ffbfd640 1 -- 172.21.15.131:0/2253617153 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53f815fd90 con 0x7f53f80c9c60 2024-09-17T02:52:47.600 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.605+0000 7f53ee7fc640 1 -- 172.21.15.131:0/2253617153 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+17327 (secure 0 0 0) 0x7f53f005e8f0 con 0x7f53f80c9c60 2024-09-17T02:52:47.600 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:47.600 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":124,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:52:34.677596+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-09-17T02:51:52.442187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-09-17T02:51:53.731428+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6835/2981077586":"2024-09-18T02:52:34.677555+0000","172.21.15.131:6832/2981077586":"2024-09-18T02:52:34.677555+0000","172.21.15.131:6835/2599844908":"2024-09-18T02:52:16.917982+0000","172.21.15.131:6832/2599844908":"2024-09-18T02:52:16.917982+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:52:47.602 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 -- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f53dc073200 msgr2=0x7f53dc075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f53dc073200 0x7f53dc075710 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f53f4001630 tx=0x7f53f4009040 comp rx=0 tx=0).stop 2024-09-17T02:52:47.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 -- 172.21.15.131:0/2253617153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f80c9c60 msgr2=0x7f53f8157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f80c9c60 0x7f53f8157de0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f53f000b9e0 tx=0x7f53f000bea0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 -- 172.21.15.131:0/2253617153 shutdown_connections 2024-09-17T02:52:47.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f53dc073200 0x7f53dc075710 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f53f80c9c60 0x7f53f8157de0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f53f80cb6c0 0x7f53f80b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 --2- 172.21.15.131:0/2253617153 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f53f80ca5c0 0x7f53f81505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 -- 172.21.15.131:0/2253617153 >> 172.21.15.131:0/2253617153 conn(0x7f53f806eed0 msgr2=0x7f53f806f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:47.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 -- 172.21.15.131:0/2253617153 shutdown_connections 2024-09-17T02:52:47.603 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.609+0000 7f53ffbfd640 1 -- 172.21.15.131:0/2253617153 wait complete. 2024-09-17T02:52:47.622 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T02:52:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 -- 172.21.15.131:0/3748566354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8005160 msgr2=0x7f3fe800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 --2- 172.21.15.131:0/3748566354 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8005160 0x7f3fe800b9d0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc072c00 tx=0x7f3ff401d2e0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f4001b63640 1 -- 172.21.15.131:0/3748566354 reap_dead start 2024-09-17T02:52:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 -- 172.21.15.131:0/3748566354 shutdown_connections 2024-09-17T02:52:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 -- 172.21.15.131:0/3748566354 >> 172.21.15.131:0/3748566354 conn(0x7f3ffc0cff40 msgr2=0x7f3ffc0d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 -- 172.21.15.131:0/3748566354 shutdown_connections 2024-09-17T02:52:47.767 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 -- 172.21.15.131:0/3748566354 wait complete. 2024-09-17T02:52:47.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 Processor -- start 2024-09-17T02:52:47.768 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 -- start start 2024-09-17T02:52:47.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8005160 0x7f3ffc14b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3ffc14f140 0x7f3ffc14f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ffc14d1e0 0x7f3ffc14fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc15b7a0 con 0x7f3fe8005160 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc15b620 con 0x7f3ffc14d1e0 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f40035c6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f3ffc15b920 con 0x7f3ffc14f140 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f4001362640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8005160 0x7f3ffc14b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f4001362640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8005160 0x7f3ffc14b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:39952/0 (socket says 172.21.15.131:39952) 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f4001362640 1 -- 172.21.15.131:0/3422972603 learned_addr learned my addr 172.21.15.131:0/3422972603 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f4000b61640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3ffc14f140 0x7f3ffc14f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.773+0000 7f4001b63640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ffc14d1e0 0x7f3ffc14fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f4001362640 1 -- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3ffc14f140 msgr2=0x7f3ffc14f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f4001362640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3ffc14f140 0x7f3ffc14f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f4001362640 1 -- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ffc14d1e0 msgr2=0x7f3ffc14fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f4001362640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ffc14d1e0 0x7f3ffc14fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f4001362640 1 -- 172.21.15.131:0/3422972603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc14bd90 con 0x7f3fe8005160 2024-09-17T02:52:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f4001362640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8005160 0x7f3ffc14b850 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009fc0 tx=0x7f3ff800ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f3ff1ffb640 1 -- 172.21.15.131:0/3422972603 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff801e070 con 0x7f3fe8005160 2024-09-17T02:52:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f3ff1ffb640 1 -- 172.21.15.131:0/3422972603 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ff8011b90 con 0x7f3fe8005160 2024-09-17T02:52:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f3ff1ffb640 1 -- 172.21.15.131:0/3422972603 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff800c450 con 0x7f3fe8005160 2024-09-17T02:52:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f40035c6640 1 -- 172.21.15.131:0/3422972603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc0b3050 con 0x7f3fe8005160 2024-09-17T02:52:47.772 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f40035c6640 1 -- 172.21.15.131:0/3422972603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc14d6c0 con 0x7f3fe8005160 2024-09-17T02:52:47.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f40035c6640 1 -- 172.21.15.131:0/3422972603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ffc0747d0 con 0x7f3fe8005160 2024-09-17T02:52:47.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f3ff1ffb640 1 -- 172.21.15.131:0/3422972603 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3ff8021070 con 0x7f3fe8005160 2024-09-17T02:52:47.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f3ff1ffb640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fe0073200 0x7f3fe0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:47.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f3ff1ffb640 1 -- 172.21.15.131:0/3422972603 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 6273+0+0 (secure 0 0 0) 0x7f3ff8096a60 con 0x7f3fe8005160 2024-09-17T02:52:47.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f4000b61640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fe0073200 0x7f3fe0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:47.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.777+0000 7f4000b61640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fe0073200 0x7f3fe0075710 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc15a730 tx=0x7f3fec0093a0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:47.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.781+0000 7f3ff1ffb640 1 -- 172.21.15.131:0/3422972603 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3ff8028020 con 0x7f3fe8005160 2024-09-17T02:52:47.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.973+0000 7f40035c6640 1 -- 172.21.15.131:0/3422972603 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f3ffc0ca350 con 0x7f3fe8005160 2024-09-17T02:52:47.967 INFO:tasks.ceph.mon.a.smithi131.stderr:2024-09-17T02:52:47.973+0000 7f3a3683c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T02:52:47.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f3ff1ffb640 1 -- 172.21.15.131:0/3422972603 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) ==== 161+0+0 (secure 0 0 0) 0x7f3ff8061a40 con 0x7f3fe8005160 2024-09-17T02:52:47.970 INFO:teuthology.orchestra.run.smithi131.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T02:52:47.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 -- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fe0073200 msgr2=0x7f3fe0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fe0073200 0x7f3fe0075710 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc15a730 tx=0x7f3fec0093a0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 -- 172.21.15.131:0/3422972603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8005160 msgr2=0x7f3ffc14b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:47.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8005160 0x7f3ffc14b850 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009fc0 tx=0x7f3ff800ead0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 -- 172.21.15.131:0/3422972603 shutdown_connections 2024-09-17T02:52:47.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3fe0073200 0x7f3fe0075710 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f3ffc14d1e0 0x7f3ffc14fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f3ffc14f140 0x7f3ffc14f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 --2- 172.21.15.131:0/3422972603 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3fe8005160 0x7f3ffc14b850 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:47.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 -- 172.21.15.131:0/3422972603 >> 172.21.15.131:0/3422972603 conn(0x7f3ffc0cff40 msgr2=0x7f3ffc150cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:47.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 -- 172.21.15.131:0/3422972603 shutdown_connections 2024-09-17T02:52:47.974 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:47.977+0000 7f40035c6640 1 -- 172.21.15.131:0/3422972603 wait complete. 2024-09-17T02:52:47.980 INFO:tasks.ceph.mds.a.smithi131.stderr: -1386> 2024-09-17T02:52:41.689+0000 7f1ce50dd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T02:52:47.994 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T02:52:48.056 INFO:tasks.ceph.mds.a.smithi131.stdout:starting mds.a at 2024-09-17T02:52:48.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.149+0000 7f36ae04a640 1 -- 172.21.15.131:0/862937699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3694003a40 msgr2=0x7f3694003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:48.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.149+0000 7f36ae04a640 1 --2- 172.21.15.131:0/862937699 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3694003a40 0x7f3694003eb0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f3698009b60 tx=0x7f369801d200 comp rx=0 tx=0).stop 2024-09-17T02:52:48.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.149+0000 7f36a7fff640 1 -- 172.21.15.131:0/862937699 reap_dead start 2024-09-17T02:52:48.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.149+0000 7f36ae04a640 1 -- 172.21.15.131:0/862937699 shutdown_connections 2024-09-17T02:52:48.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.149+0000 7f36ae04a640 1 -- 172.21.15.131:0/862937699 >> 172.21.15.131:0/862937699 conn(0x7f36a808bfe0 msgr2=0x7f36a80d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:48.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.149+0000 7f36ae04a640 1 -- 172.21.15.131:0/862937699 shutdown_connections 2024-09-17T02:52:48.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.149+0000 7f36ae04a640 1 -- 172.21.15.131:0/862937699 wait complete. 2024-09-17T02:52:48.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.149+0000 7f36ae04a640 1 Processor -- start 2024-09-17T02:52:48.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.149+0000 7f36ae04a640 1 -- start start 2024-09-17T02:52:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36ae04a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3694003a40 0x7f36a806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36ae04a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f36a81f8df0 0x7f36a8158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36ae04a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36a80b5cd0 0x7f36a80b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36ae04a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f36a80d1150 con 0x7f3694003a40 2024-09-17T02:52:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36ae04a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f36a80d0fd0 con 0x7f36a80b5cd0 2024-09-17T02:52:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36ae04a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f36a8168e20 con 0x7f36a81f8df0 2024-09-17T02:52:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3694003a40 0x7f36a806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3694003a40 0x7f36a806fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40000/0 (socket says 172.21.15.131:40000) 2024-09-17T02:52:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a77fe640 1 -- 172.21.15.131:0/1092341386 learned_addr learned my addr 172.21.15.131:0/1092341386 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a7fff640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36a80b5cd0 0x7f36a80b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:48.148 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f369effd640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f36a81f8df0 0x7f36a8158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a77fe640 1 -- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f36a81f8df0 msgr2=0x7f36a8158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a77fe640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f36a81f8df0 0x7f36a8158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a77fe640 1 -- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36a80b5cd0 msgr2=0x7f36a80b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a77fe640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36a80b5cd0 0x7f36a80b60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a77fe640 1 -- 172.21.15.131:0/1092341386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a8158cc0 con 0x7f3694003a40 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f369effd640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f36a81f8df0 0x7f36a8158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a77fe640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3694003a40 0x7f36a806fa20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f3698009b60 tx=0x7f3698005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a4ff9640 1 -- 172.21.15.131:0/1092341386 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36980b1040 con 0x7f3694003a40 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a4ff9640 1 -- 172.21.15.131:0/1092341386 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36980bb070 con 0x7f3694003a40 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a4ff9640 1 -- 172.21.15.131:0/1092341386 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698004290 con 0x7f3694003a40 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36ae04a640 1 -- 172.21.15.131:0/1092341386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a80b6d50 con 0x7f3694003a40 2024-09-17T02:52:48.149 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36ae04a640 1 -- 172.21.15.131:0/1092341386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a80b54d0 con 0x7f3694003a40 2024-09-17T02:52:48.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36a4ff9640 1 -- 172.21.15.131:0/1092341386 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f36980046d0 con 0x7f3694003a40 2024-09-17T02:52:48.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.153+0000 7f36ae04a640 1 -- 172.21.15.131:0/1092341386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a80d2430 con 0x7f3694003a40 2024-09-17T02:52:48.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.157+0000 7f36a4ff9640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3688073200 0x7f3688075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:48.151 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.157+0000 7f36a4ff9640 1 -- 172.21.15.131:0/1092341386 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6359+0+0 (secure 0 0 0) 0x7f369812f580 con 0x7f3694003a40 2024-09-17T02:52:48.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.157+0000 7f369effd640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3688073200 0x7f3688075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:48.152 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.157+0000 7f369effd640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3688073200 0x7f3688075710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3690001890 tx=0x7f3690001c70 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:48.154 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.161+0000 7f36a4ff9640 1 -- 172.21.15.131:0/1092341386 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f3698016fa0 con 0x7f3694003a40 2024-09-17T02:52:48.358 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.365+0000 7f36ae04a640 1 -- 172.21.15.131:0/1092341386 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f36a815bfa0 con 0x7f3694003a40 2024-09-17T02:52:48.973 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.977+0000 7f36a4ff9640 1 -- 172.21.15.131:0/1092341386 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v115) ==== 104+0+0 (secure 0 0 0) 0x7f36980fa310 con 0x7f3694003a40 2024-09-17T02:52:48.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 -- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3688073200 msgr2=0x7f3688075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:48.976 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3688073200 0x7f3688075710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3690001890 tx=0x7f3690001c70 comp rx=0 tx=0).stop 2024-09-17T02:52:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 -- 172.21.15.131:0/1092341386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3694003a40 msgr2=0x7f36a806fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3694003a40 0x7f36a806fa20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f3698009b60 tx=0x7f3698005540 comp rx=0 tx=0).stop 2024-09-17T02:52:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 -- 172.21.15.131:0/1092341386 shutdown_connections 2024-09-17T02:52:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3688073200 0x7f3688075710 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f36a80b5cd0 0x7f36a80b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f36a81f8df0 0x7f36a8158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 --2- 172.21.15.131:0/1092341386 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f3694003a40 0x7f36a806fa20 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 -- 172.21.15.131:0/1092341386 >> 172.21.15.131:0/1092341386 conn(0x7f36a808bfe0 msgr2=0x7f36a8068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 -- 172.21.15.131:0/1092341386 shutdown_connections 2024-09-17T02:52:48.977 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:48.981+0000 7f36ae04a640 1 -- 172.21.15.131:0/1092341386 wait complete. 2024-09-17T02:52:49.003 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T02:52:49.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.173+0000 7f48821a6640 1 -- 172.21.15.131:0/1167063175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487c0b34f0 msgr2=0x7f487c0afce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:49.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.173+0000 7f48821a6640 1 --2- 172.21.15.131:0/1167063175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487c0b34f0 0x7f487c0afce0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f487c0737c0 tx=0x7f486c01c8c0 comp rx=0 tx=0).stop 2024-09-17T02:52:49.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.173+0000 7f48821a6640 1 -- 172.21.15.131:0/1167063175 shutdown_connections 2024-09-17T02:52:49.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.173+0000 7f48821a6640 1 --2- 172.21.15.131:0/1167063175 >> v2:172.21.15.189:3300/0 conn(0x7f487c15b970 0x7f487c0aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:49.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.173+0000 7f48821a6640 1 --2- 172.21.15.131:0/1167063175 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487c0b34f0 0x7f487c0afce0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:49.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.173+0000 7f48821a6640 1 -- 172.21.15.131:0/1167063175 >> 172.21.15.131:0/1167063175 conn(0x7f487c0b6640 msgr2=0x7f487c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:49.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.173+0000 7f48821a6640 1 -- 172.21.15.131:0/1167063175 shutdown_connections 2024-09-17T02:52:49.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.173+0000 7f48821a6640 1 -- 172.21.15.131:0/1167063175 wait complete. 2024-09-17T02:52:49.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.173+0000 7f48821a6640 1 Processor -- start 2024-09-17T02:52:49.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.173+0000 7f48821a6640 1 -- start start 2024-09-17T02:52:49.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f48821a6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f487c0b34f0 0x7f487c1f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f48821a6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f487c15b560 0x7f487c1593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f48821a6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487c14efb0 0x7f487c15c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f48821a6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f487c1690f0 con 0x7f487c14efb0 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f48821a6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f487c168f70 con 0x7f487c15b560 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f48821a6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f487c169270 con 0x7f487c0b34f0 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487c14efb0 0x7f487c15c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487c14efb0 0x7f487c15c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40028/0 (socket says 172.21.15.131:40028) 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487bfff640 1 -- 172.21.15.131:0/216933394 learned_addr learned my addr 172.21.15.131:0/216933394 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487b7fe640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f487c0b34f0 0x7f487c1f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487affd640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f487c15b560 0x7f487c1593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487bfff640 1 -- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f487c0b34f0 msgr2=0x7f487c1f9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:49.173 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487bfff640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f487c0b34f0 0x7f487c1f9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487bfff640 1 -- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f487c15b560 msgr2=0x7f487c1593d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487bfff640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f487c15b560 0x7f487c1593d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487bfff640 1 -- 172.21.15.131:0/216933394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487c15e650 con 0x7f487c14efb0 2024-09-17T02:52:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487affd640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f487c15b560 0x7f487c1593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f487bfff640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487c14efb0 0x7f487c15c5a0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f487400cd10 tx=0x7f487400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f4863fff640 1 -- 172.21.15.131:0/216933394 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487400f450 con 0x7f487c14efb0 2024-09-17T02:52:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f4863fff640 1 -- 172.21.15.131:0/216933394 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f487400f700 con 0x7f487c14efb0 2024-09-17T02:52:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f48821a6640 1 -- 172.21.15.131:0/216933394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487c15e930 con 0x7f487c14efb0 2024-09-17T02:52:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f4863fff640 1 -- 172.21.15.131:0/216933394 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487400fb50 con 0x7f487c14efb0 2024-09-17T02:52:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f48821a6640 1 -- 172.21.15.131:0/216933394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487c15eef0 con 0x7f487c14efb0 2024-09-17T02:52:49.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f48821a6640 1 -- 172.21.15.131:0/216933394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f487c15b970 con 0x7f487c14efb0 2024-09-17T02:52:49.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.177+0000 7f4863fff640 1 -- 172.21.15.131:0/216933394 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f487400fcf0 con 0x7f487c14efb0 2024-09-17T02:52:49.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.181+0000 7f4863fff640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4868073230 0x7f4868075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:49.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.181+0000 7f4863fff640 1 -- 172.21.15.131:0/216933394 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 6359+0+0 (secure 0 0 0) 0x7f48740950a0 con 0x7f487c14efb0 2024-09-17T02:52:49.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.181+0000 7f487b7fe640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4868073230 0x7f4868075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:49.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.181+0000 7f487b7fe640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4868073230 0x7f4868075740 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f486c0af000 tx=0x7f486c01b100 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:49.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.185+0000 7f4863fff640 1 -- 172.21.15.131:0/216933394 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4874014030 con 0x7f487c14efb0 2024-09-17T02:52:49.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:49.357+0000 7f48821a6640 1 -- 172.21.15.131:0/216933394 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f487c155770 con 0x7f487c14efb0 2024-09-17T02:52:50.134 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.137+0000 7f4863fff640 1 -- 172.21.15.131:0/216933394 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) ==== 181+0+0 (secure 0 0 0) 0x7f487405f7f0 con 0x7f487c14efb0 2024-09-17T02:52:50.134 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_metadata' removed 2024-09-17T02:52:50.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 -- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4868073230 msgr2=0x7f4868075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:50.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4868073230 0x7f4868075740 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f486c0af000 tx=0x7f486c01b100 comp rx=0 tx=0).stop 2024-09-17T02:52:50.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 -- 172.21.15.131:0/216933394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487c14efb0 msgr2=0x7f487c15c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:50.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487c14efb0 0x7f487c15c5a0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f487400cd10 tx=0x7f487400a620 comp rx=0 tx=0).stop 2024-09-17T02:52:50.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 -- 172.21.15.131:0/216933394 shutdown_connections 2024-09-17T02:52:50.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4868073230 0x7f4868075740 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:50.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f487c14efb0 0x7f487c15c5a0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:50.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f487c15b560 0x7f487c1593d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:50.137 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 --2- 172.21.15.131:0/216933394 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f487c0b34f0 0x7f487c1f9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 -- 172.21.15.131:0/216933394 >> 172.21.15.131:0/216933394 conn(0x7f487c0b6640 msgr2=0x7f487c0b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 -- 172.21.15.131:0/216933394 shutdown_connections 2024-09-17T02:52:50.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.141+0000 7f48821a6640 1 -- 172.21.15.131:0/216933394 wait complete. 2024-09-17T02:52:50.162 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T02:52:50.328 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.333+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3712819465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f80cfad0 msgr2=0x7f75f806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.333+0000 7f75fdbe6640 1 --2- 172.21.15.131:0/3712819465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f80cfad0 0x7f75f806ec10 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009900 tx=0x7f75e801c820 comp rx=0 tx=0).stop 2024-09-17T02:52:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.333+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3712819465 shutdown_connections 2024-09-17T02:52:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.333+0000 7f75fdbe6640 1 --2- 172.21.15.131:0/3712819465 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75f8073480 0x7f75f80b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.333+0000 7f75fdbe6640 1 --2- 172.21.15.131:0/3712819465 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f80cfad0 0x7f75f806ec10 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.333+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3712819465 >> 172.21.15.131:0/3712819465 conn(0x7f75f806d010 msgr2=0x7f75f806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.333+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3712819465 shutdown_connections 2024-09-17T02:52:50.329 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.333+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3712819465 wait complete. 2024-09-17T02:52:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75fdbe6640 1 Processor -- start 2024-09-17T02:52:50.330 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75fdbe6640 1 -- start start 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75fdbe6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75f806f150 0x7f75f8153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75fdbe6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f8073480 0x7f75f8157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75fdbe6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f80cfad0 0x7f75f814c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75fdbe6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f75f80b3cb0 con 0x7f75f80cfad0 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75fdbe6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f75f80b3b30 con 0x7f75f8073480 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75fdbe6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f75f80b3e30 con 0x7f75f806f150 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f80cfad0 0x7f75f814c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f80cfad0 0x7f75f814c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:40046/0 (socket says 172.21.15.131:40046) 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f7fff640 1 -- 172.21.15.131:0/3274571947 learned_addr learned my addr 172.21.15.131:0/3274571947 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f77fe640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75f806f150 0x7f75f8153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f6ffd640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f8073480 0x7f75f8157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:50.333 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f7fff640 1 -- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75f806f150 msgr2=0x7f75f8153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f7fff640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75f806f150 0x7f75f8153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f7fff640 1 -- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f8073480 msgr2=0x7f75f8157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f7fff640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f8073480 0x7f75f8157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f7fff640 1 -- 172.21.15.131:0/3274571947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f8157540 con 0x7f75f80cfad0 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f6ffd640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f8073480 0x7f75f8157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f77fe640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75f806f150 0x7f75f8153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75f7fff640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f80cfad0 0x7f75f814c350 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f75f000e9a0 tx=0x7f75f000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75dffff640 1 -- 172.21.15.131:0/3274571947 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0004d30 con 0x7f75f80cfad0 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75dffff640 1 -- 172.21.15.131:0/3274571947 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75f000cb20 con 0x7f75f80cfad0 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3274571947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f8156240 con 0x7f75f80cfad0 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75dffff640 1 -- 172.21.15.131:0/3274571947 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0005020 con 0x7f75f80cfad0 2024-09-17T02:52:50.334 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3274571947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f8156800 con 0x7f75f80cfad0 2024-09-17T02:52:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.337+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3274571947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75f80b43d0 con 0x7f75f80cfad0 2024-09-17T02:52:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.341+0000 7f75dffff640 1 -- 172.21.15.131:0/3274571947 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f75f00051c0 con 0x7f75f80cfad0 2024-09-17T02:52:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.341+0000 7f75dffff640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f75e4073230 0x7f75e4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:50.335 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.341+0000 7f75dffff640 1 -- 172.21.15.131:0/3274571947 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5921+0+0 (secure 0 0 0) 0x7f75f0094b70 con 0x7f75f80cfad0 2024-09-17T02:52:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.341+0000 7f75f77fe640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f75e4073230 0x7f75e4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:50.336 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.341+0000 7f75f77fe640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f75e4073230 0x7f75e4075740 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75e80af000 tx=0x7f75e801b060 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:50.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.345+0000 7f75dffff640 1 -- 172.21.15.131:0/3274571947 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f75f0016030 con 0x7f75f80cfad0 2024-09-17T02:52:50.511 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:50.517+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3274571947 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f75f8154420 con 0x7f75f80cfad0 2024-09-17T02:52:51.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.165+0000 7f75dffff640 1 -- 172.21.15.131:0/3274571947 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v127) ==== 169+0+0 (secure 0 0 0) 0x7f75f005fb40 con 0x7f75f80cfad0 2024-09-17T02:52:51.159 INFO:teuthology.orchestra.run.smithi131.stderr:pool 'cephfs_data' removed 2024-09-17T02:52:51.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f75e4073230 msgr2=0x7f75e4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:51.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f75e4073230 0x7f75e4075740 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f75e80af000 tx=0x7f75e801b060 comp rx=0 tx=0).stop 2024-09-17T02:52:51.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3274571947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f80cfad0 msgr2=0x7f75f814c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:51.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f80cfad0 0x7f75f814c350 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f75f000e9a0 tx=0x7f75f000ee60 comp rx=0 tx=0).stop 2024-09-17T02:52:51.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3274571947 shutdown_connections 2024-09-17T02:52:51.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f75e4073230 0x7f75e4075740 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f75f80cfad0 0x7f75f814c350 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f75f8073480 0x7f75f8157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 --2- 172.21.15.131:0/3274571947 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f75f806f150 0x7f75f8153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3274571947 >> 172.21.15.131:0/3274571947 conn(0x7f75f806d010 msgr2=0x7f75f815cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:51.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3274571947 shutdown_connections 2024-09-17T02:52:51.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.169+0000 7f75fdbe6640 1 -- 172.21.15.131:0/3274571947 wait complete. 2024-09-17T02:52:51.191 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T02:52:51.345 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.349+0000 7f9080370640 1 -- 172.21.15.131:0/1846933507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f907815d0e0 msgr2=0x7f907806ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:51.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.349+0000 7f9080370640 1 --2- 172.21.15.131:0/1846933507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f907815d0e0 0x7f907806ce40 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f90780b0d30 tx=0x7f906801c830 comp rx=0 tx=0).stop 2024-09-17T02:52:51.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.349+0000 7f9080370640 1 -- 172.21.15.131:0/1846933507 shutdown_connections 2024-09-17T02:52:51.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.349+0000 7f9080370640 1 --2- 172.21.15.131:0/1846933507 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f907815d0e0 0x7f907806ce40 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.349+0000 7f9080370640 1 --2- 172.21.15.131:0/1846933507 >> v2:172.21.15.189:3300/0 conn(0x7f9078145e70 0x7f907815cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.349+0000 7f9080370640 1 -- 172.21.15.131:0/1846933507 >> 172.21.15.131:0/1846933507 conn(0x7f907806df50 msgr2=0x7f907806e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:51.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.349+0000 7f9080370640 1 -- 172.21.15.131:0/1846933507 shutdown_connections 2024-09-17T02:52:51.346 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.349+0000 7f9080370640 1 -- 172.21.15.131:0/1846933507 wait complete. 2024-09-17T02:52:51.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f9080370640 1 Processor -- start 2024-09-17T02:52:51.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f9080370640 1 -- start start 2024-09-17T02:52:51.350 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f9080370640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f90781457e0 0x7f907813a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f9080370640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9078145e70 0x7f907813c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f9080370640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f907815d0e0 0x7f907815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f9080370640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9078169f10 con 0x7f9078145e70 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f9080370640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9078169d90 con 0x7f907815d0e0 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f9080370640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f907816a090 con 0x7f90781457e0 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f907d90b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9078145e70 0x7f907813c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f907d90b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9078145e70 0x7f907813c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44954/0 (socket says 172.21.15.131:44954) 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f907d90b640 1 -- 172.21.15.131:0/1876755411 learned_addr learned my addr 172.21.15.131:0/1876755411 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f907e90d640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f907815d0e0 0x7f907815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.353+0000 7f907e10c640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f90781457e0 0x7f907813a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f907d90b640 1 -- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f90781457e0 msgr2=0x7f907813a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:51.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f907d90b640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f90781457e0 0x7f907813a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f907d90b640 1 -- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f907815d0e0 msgr2=0x7f907815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f907d90b640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f907815d0e0 0x7f907815ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f907d90b640 1 -- 172.21.15.131:0/1876755411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907813cbe0 con 0x7f9078145e70 2024-09-17T02:52:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f907e10c640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f90781457e0 0x7f907813a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f907d90b640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9078145e70 0x7f907813c6a0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e990 tx=0x7f906c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f9076ffd640 1 -- 172.21.15.131:0/1876755411 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c004d30 con 0x7f9078145e70 2024-09-17T02:52:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f9076ffd640 1 -- 172.21.15.131:0/1876755411 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f906c00cb00 con 0x7f9078145e70 2024-09-17T02:52:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f9080370640 1 -- 172.21.15.131:0/1876755411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907813b0f0 con 0x7f9078145e70 2024-09-17T02:52:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f9076ffd640 1 -- 172.21.15.131:0/1876755411 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c005020 con 0x7f9078145e70 2024-09-17T02:52:51.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f9080370640 1 -- 172.21.15.131:0/1876755411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907813b6b0 con 0x7f9078145e70 2024-09-17T02:52:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f9080370640 1 -- 172.21.15.131:0/1876755411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907813ae60 con 0x7f9078145e70 2024-09-17T02:52:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f9076ffd640 1 -- 172.21.15.131:0/1876755411 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f906c0051c0 con 0x7f9078145e70 2024-09-17T02:52:51.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f9076ffd640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9064073200 0x7f9064075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f9076ffd640 1 -- 172.21.15.131:0/1876755411 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f906c094ca0 con 0x7f9078145e70 2024-09-17T02:52:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.357+0000 7f907e10c640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9064073200 0x7f9064075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:51.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.361+0000 7f907e10c640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9064073200 0x7f9064075710 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f907000a850 tx=0x7f9070008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:51.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.361+0000 7f9076ffd640 1 -- 172.21.15.131:0/1876755411 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f906c016030 con 0x7f9078145e70 2024-09-17T02:52:51.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.533+0000 7f9080370640 1 -- 172.21.15.131:0/1876755411 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f907813b8f0 con 0x7f9078145e70 2024-09-17T02:52:51.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.541+0000 7f9076ffd640 1 -- 172.21.15.131:0/1876755411 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v45) ==== 71+0+0 (secure 0 0 0) 0x7f907813b8f0 con 0x7f9078145e70 2024-09-17T02:52:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 -- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9064073200 msgr2=0x7f9064075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9064073200 0x7f9064075710 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f907000a850 tx=0x7f9070008040 comp rx=0 tx=0).stop 2024-09-17T02:52:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 -- 172.21.15.131:0/1876755411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9078145e70 msgr2=0x7f907813c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9078145e70 0x7f907813c6a0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e990 tx=0x7f906c00ee50 comp rx=0 tx=0).stop 2024-09-17T02:52:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 -- 172.21.15.131:0/1876755411 shutdown_connections 2024-09-17T02:52:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9064073200 0x7f9064075710 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f907815d0e0 0x7f907815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9078145e70 0x7f907813c6a0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 --2- 172.21.15.131:0/1876755411 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f90781457e0 0x7f907813a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 -- 172.21.15.131:0/1876755411 >> 172.21.15.131:0/1876755411 conn(0x7f907806df50 msgr2=0x7f90780b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:51.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 -- 172.21.15.131:0/1876755411 shutdown_connections 2024-09-17T02:52:51.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.545+0000 7f9080370640 1 -- 172.21.15.131:0/1876755411 wait complete. 2024-09-17T02:52:51.558 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-09-17T02:52:51.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.713+0000 7f8cdc281640 1 -- 172.21.15.131:0/1755694557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc0004c50 msgr2=0x7f8cc00050c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:51.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.713+0000 7f8cdc281640 1 --2- 172.21.15.131:0/1755694557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc0004c50 0x7f8cc00050c0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009ee0 tx=0x7f8cd001d5e0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.713+0000 7f8cdc281640 1 -- 172.21.15.131:0/1755694557 shutdown_connections 2024-09-17T02:52:51.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.713+0000 7f8cdc281640 1 --2- 172.21.15.131:0/1755694557 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc0004c50 0x7f8cc00050c0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.708 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.713+0000 7f8cdc281640 1 -- 172.21.15.131:0/1755694557 >> 172.21.15.131:0/1755694557 conn(0x7f8cd406eed0 msgr2=0x7f8cd406f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:51.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.713+0000 7f8cdc281640 1 -- 172.21.15.131:0/1755694557 shutdown_connections 2024-09-17T02:52:51.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.713+0000 7f8cdc281640 1 -- 172.21.15.131:0/1755694557 wait complete. 2024-09-17T02:52:51.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.713+0000 7f8cdc281640 1 Processor -- start 2024-09-17T02:52:51.709 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.713+0000 7f8cdc281640 1 -- start start 2024-09-17T02:52:51.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cdc281640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc0004c50 0x7f8cd40c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:51.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cdc281640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd40c9b30 0x7f8cd41593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:51.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cdc281640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8cd40b4c20 0x7f8cd4159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:51.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cdc281640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd40cff70 con 0x7f8cc0004c50 2024-09-17T02:52:51.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cdc281640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd40cb420 con 0x7f8cd40c9b30 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cdc281640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd40d00f0 con 0x7f8cd40b4c20 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cda01d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc0004c50 0x7f8cd40c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cda01d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc0004c50 0x7f8cd40c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44982/0 (socket says 172.21.15.131:44982) 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cda01d640 1 -- 172.21.15.131:0/1947425118 learned_addr learned my addr 172.21.15.131:0/1947425118 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cd981c640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd40c9b30 0x7f8cd41593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cda81e640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8cd40b4c20 0x7f8cd4159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cda01d640 1 -- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8cd40b4c20 msgr2=0x7f8cd4159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cda01d640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8cd40b4c20 0x7f8cd4159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cda01d640 1 -- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd40c9b30 msgr2=0x7f8cd41593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cda01d640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd40c9b30 0x7f8cd41593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cda01d640 1 -- 172.21.15.131:0/1947425118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd40b33d0 con 0x7f8cc0004c50 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cda81e640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8cd40b4c20 0x7f8cd4159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cda01d640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc0004c50 0x7f8cd40c95f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f8cd001d410 tx=0x7f8cd0009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:51.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8ccaffd640 1 -- 172.21.15.131:0/1947425118 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0003980 con 0x7f8cc0004c50 2024-09-17T02:52:51.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8ccaffd640 1 -- 172.21.15.131:0/1947425118 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cd0003b40 con 0x7f8cc0004c50 2024-09-17T02:52:51.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8ccaffd640 1 -- 172.21.15.131:0/1947425118 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd00aa8b0 con 0x7f8cc0004c50 2024-09-17T02:52:51.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cdc281640 1 -- 172.21.15.131:0/1947425118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd40b36b0 con 0x7f8cc0004c50 2024-09-17T02:52:51.714 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cdc281640 1 -- 172.21.15.131:0/1947425118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd40b3c70 con 0x7f8cc0004c50 2024-09-17T02:52:51.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8cdc281640 1 -- 172.21.15.131:0/1947425118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd40b3eb0 con 0x7f8cc0004c50 2024-09-17T02:52:51.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.717+0000 7f8ccaffd640 1 -- 172.21.15.131:0/1947425118 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f8cd00aa580 con 0x7f8cc0004c50 2024-09-17T02:52:51.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.721+0000 7f8ccaffd640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8cb8073200 0x7f8cb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:51.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.721+0000 7f8ccaffd640 1 -- 172.21.15.131:0/1947425118 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f8cd012f7e0 con 0x7f8cc0004c50 2024-09-17T02:52:51.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.721+0000 7f8cd981c640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8cb8073200 0x7f8cb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:51.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.721+0000 7f8cd981c640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8cb8073200 0x7f8cb8075710 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8cd415c300 tx=0x7f8cc4009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:51.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.725+0000 7f8ccaffd640 1 -- 172.21.15.131:0/1947425118 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f8cd40b3eb0 con 0x7f8cc0004c50 2024-09-17T02:52:51.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:51.945+0000 7f8cdc281640 1 -- 172.21.15.131:0/1947425118 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f8cd4073b70 con 0x7f8cc0004c50 2024-09-17T02:52:52.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8ccaffd640 1 -- 172.21.15.131:0/1947425118 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v542) ==== 144+0+0 (secure 0 0 0) 0x7f8cd00ae020 con 0x7f8cc0004c50 2024-09-17T02:52:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8cdc281640 1 -- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8cb8073200 msgr2=0x7f8cb8075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8cdc281640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8cb8073200 0x7f8cb8075710 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8cd415c300 tx=0x7f8cc4009040 comp rx=0 tx=0).stop 2024-09-17T02:52:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8cdc281640 1 -- 172.21.15.131:0/1947425118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc0004c50 msgr2=0x7f8cd40c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8cdc281640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc0004c50 0x7f8cd40c95f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f8cd001d410 tx=0x7f8cd0009200 comp rx=0 tx=0).stop 2024-09-17T02:52:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8cdc281640 1 -- 172.21.15.131:0/1947425118 shutdown_connections 2024-09-17T02:52:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8cdc281640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f8cb8073200 0x7f8cb8075710 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8cdc281640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f8cd40b4c20 0x7f8cd4159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8cdc281640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f8cd40c9b30 0x7f8cd41593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8cdc281640 1 --2- 172.21.15.131:0/1947425118 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f8cc0004c50 0x7f8cd40c95f0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8cdc281640 1 -- 172.21.15.131:0/1947425118 >> 172.21.15.131:0/1947425118 conn(0x7f8cd406eed0 msgr2=0x7f8cd4074480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:52.174 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.177+0000 7f8cdc281640 1 -- 172.21.15.131:0/1947425118 shutdown_connections 2024-09-17T02:52:52.175 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:52.181+0000 7f8cdc281640 1 -- 172.21.15.131:0/1947425118 wait complete. 2024-09-17T02:52:52.197 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-09-17T02:52:52.198 INFO:tasks.cephfs_test_runner: 2024-09-17T02:52:52.198 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-17T02:52:52.198 INFO:tasks.cephfs_test_runner:Ran 7 tests in 676.918s 2024-09-17T02:52:52.198 INFO:tasks.cephfs_test_runner: 2024-09-17T02:52:52.198 INFO:tasks.cephfs_test_runner:OK 2024-09-17T02:52:52.198 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-17T02:52:52.208 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-17T02:52:52.217 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-17T02:52:52.218 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:52.218 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:52.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:52:52.224 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-17T02:52:52.224 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:52.271 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336432k,nr_inodes=4084108,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18847 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T02:52:52.271 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:52:52.271 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:52.277 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:52:52.278 DEBUG:teuthology.orchestra.run.smithi189:> set -ex 2024-09-17T02:52:52.278 DEBUG:teuthology.orchestra.run.smithi189:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T02:52:52.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336476k,nr_inodes=4084119,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15883 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T02:52:52.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:52.325 DEBUG:teuthology.orchestra.run.smithi189:> ip netns list 2024-09-17T02:52:52.373 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T02:52:52.373 INFO:teuthology.orchestra.run.smithi189.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T02:52:52.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:52.374 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T02:52:52.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:52.430 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T02:52:52.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:52.501 DEBUG:teuthology.orchestra.run.smithi189:> sudo ip link delete ceph-brx 2024-09-17T02:52:52.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:52.674 DEBUG:teuthology.orchestra.run.smithi131:> ip netns list 2024-09-17T02:52:52.681 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T02:52:52.681 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T02:52:52.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:52.681 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T02:52:52.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:52.736 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T02:52:52.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T02:52:52.803 DEBUG:teuthology.orchestra.run.smithi131:> sudo ip link delete ceph-brx 2024-09-17T02:52:52.971 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T02:52:52.981 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T02:52:52.981 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:52:53.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.145+0000 7f97a3a65640 1 -- 172.21.15.131:0/386859130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f979c15aba0 msgr2=0x7f979c0ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.145+0000 7f97a3a65640 1 --2- 172.21.15.131:0/386859130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f979c15aba0 0x7f979c0ccfa0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f979c0d0c40 tx=0x7f979401c680 comp rx=0 tx=0).stop 2024-09-17T02:52:53.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.145+0000 7f97a3a65640 1 -- 172.21.15.131:0/386859130 shutdown_connections 2024-09-17T02:52:53.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.145+0000 7f97a3a65640 1 --2- 172.21.15.131:0/386859130 >> v2:172.21.15.189:3300/0 conn(0x7f979c0ca5c0 0x7f979c0ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.145+0000 7f97a3a65640 1 --2- 172.21.15.131:0/386859130 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f979c15aba0 0x7f979c0ccfa0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.145+0000 7f97a3a65640 1 -- 172.21.15.131:0/386859130 >> 172.21.15.131:0/386859130 conn(0x7f979c06eed0 msgr2=0x7f979c06f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:53.140 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.145+0000 7f97a3a65640 1 -- 172.21.15.131:0/386859130 shutdown_connections 2024-09-17T02:52:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.145+0000 7f97a3a65640 1 -- 172.21.15.131:0/386859130 wait complete. 2024-09-17T02:52:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.145+0000 7f97a3a65640 1 Processor -- start 2024-09-17T02:52:53.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.145+0000 7f97a3a65640 1 -- start start 2024-09-17T02:52:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a3a65640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f979c073760 0x7f979c1537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a3a65640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f979c0ca5c0 0x7f979c1f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a3a65640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f979c15aba0 0x7f979c0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a3a65640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f979c0cb390 con 0x7f979c073760 2024-09-17T02:52:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a3a65640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f979c0cb210 con 0x7f979c0ca5c0 2024-09-17T02:52:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a3a65640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f979c167c40 con 0x7f979c15aba0 2024-09-17T02:52:53.144 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a1801640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f979c073760 0x7f979c1537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a1801640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f979c073760 0x7f979c1537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:44998/0 (socket says 172.21.15.131:44998) 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a1801640 1 -- 172.21.15.131:0/1599598415 learned_addr learned my addr 172.21.15.131:0/1599598415 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a1000640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f979c0ca5c0 0x7f979c1f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a1801640 1 -- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f979c15aba0 msgr2=0x7f979c0c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a1801640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f979c15aba0 0x7f979c0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a1801640 1 -- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f979c0ca5c0 msgr2=0x7f979c1f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a1801640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f979c0ca5c0 0x7f979c1f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a1801640 1 -- 172.21.15.131:0/1599598415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c1f6dc0 con 0x7f979c073760 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a1000640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f979c0ca5c0 0x7f979c1f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a1801640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f979c073760 0x7f979c1537e0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f9794000c10 tx=0x7f9794005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f9791ffb640 1 -- 172.21.15.131:0/1599598415 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97940b2040 con 0x7f979c073760 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f9791ffb640 1 -- 172.21.15.131:0/1599598415 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97940042f0 con 0x7f979c073760 2024-09-17T02:52:53.145 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a3a65640 1 -- 172.21.15.131:0/1599598415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979c1f72c0 con 0x7f979c073760 2024-09-17T02:52:53.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f9791ffb640 1 -- 172.21.15.131:0/1599598415 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794005270 con 0x7f979c073760 2024-09-17T02:52:53.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a3a65640 1 -- 172.21.15.131:0/1599598415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979c152590 con 0x7f979c073760 2024-09-17T02:52:53.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f97a3a65640 1 -- 172.21.15.131:0/1599598415 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979c0ca9d0 con 0x7f979c073760 2024-09-17T02:52:53.146 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.149+0000 7f9791ffb640 1 -- 172.21.15.131:0/1599598415 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9794005410 con 0x7f979c073760 2024-09-17T02:52:53.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.153+0000 7f9791ffb640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9780073200 0x7f9780075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.153+0000 7f9791ffb640 1 -- 172.21.15.131:0/1599598415 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f979412e080 con 0x7f979c073760 2024-09-17T02:52:53.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.153+0000 7f97a1000640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9780073200 0x7f9780075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:53.147 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.153+0000 7f97a1000640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9780073200 0x7f9780075710 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f978c005fd0 tx=0x7f978c005dc0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:53.150 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.157+0000 7f9791ffb640 1 -- 172.21.15.131:0/1599598415 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f97940163f0 con 0x7f979c073760 2024-09-17T02:52:53.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.325+0000 7f97a3a65640 1 -- 172.21.15.131:0/1599598415 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f979c0d0c70 con 0x7f9780073200 2024-09-17T02:52:53.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.325+0000 7f9791ffb640 1 -- 172.21.15.131:0/1599598415 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13510 (secure 0 0 0) 0x7f979c0d0c70 con 0x7f9780073200 2024-09-17T02:52:53.322 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:53.322 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:52:53.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 -- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9780073200 msgr2=0x7f9780075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9780073200 0x7f9780075710 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f978c005fd0 tx=0x7f978c005dc0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 -- 172.21.15.131:0/1599598415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f979c073760 msgr2=0x7f979c1537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f979c073760 0x7f979c1537e0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f9794000c10 tx=0x7f9794005db0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 -- 172.21.15.131:0/1599598415 shutdown_connections 2024-09-17T02:52:53.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9780073200 0x7f9780075710 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f979c15aba0 0x7f979c0c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f979c0ca5c0 0x7f979c1f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 --2- 172.21.15.131:0/1599598415 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f979c073760 0x7f979c1537e0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 -- 172.21.15.131:0/1599598415 >> 172.21.15.131:0/1599598415 conn(0x7f979c06eed0 msgr2=0x7f979c06f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:53.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 -- 172.21.15.131:0/1599598415 shutdown_connections 2024-09-17T02:52:53.325 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.329+0000 7f97a3a65640 1 -- 172.21.15.131:0/1599598415 wait complete. 2024-09-17T02:52:53.344 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-09-17T02:52:51.414225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":279308,"kb_used_data":29780,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695412,"statfs":{"total":773094113280,"available":772808101888,"internally_reserved":0,"allocated":30494720,"data_stored":23816673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390143,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-249,"num_read_kb":-251,"num_write":-169,"num_write_kb":-8288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002578"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":314,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T02:52:51.171022+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:52:51.171022+0000","last_peered":"2024-09-17T02:52:51.171022+0000","last_clean":"2024-09-17T02:52:51.171022+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:52:51.171022+0000","last_undegraded":"2024-09-17T02:52:51.171022+0000","last_fullsized":"2024-09-17T02:52:51.171022+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509590,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36816,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":5758976,"data_stored":4927458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2162688,"data_stored":1323527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509591,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1564672,"data_stored":733154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1568768,"data_stored":733161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509590,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":5767168,"data_stored":4927470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":6204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":6352896,"data_stored":5517826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509591,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36816,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":5758976,"data_stored":4927458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1560576,"data_stored":726619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:52:53.345 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:52:53.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.505+0000 7f9b42cf6640 1 -- 172.21.15.131:0/55381800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b3c06e660 msgr2=0x7f9b3c15f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.501 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.505+0000 7f9b42cf6640 1 --2- 172.21.15.131:0/55381800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b3c06e660 0x7f9b3c15f9a0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009840 tx=0x7f9b2c01c790 comp rx=0 tx=0).stop 2024-09-17T02:52:53.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.505+0000 7f9b42cf6640 1 -- 172.21.15.131:0/55381800 shutdown_connections 2024-09-17T02:52:53.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.505+0000 7f9b42cf6640 1 --2- 172.21.15.131:0/55381800 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9b3c0afd70 0x7f9b3c0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.505+0000 7f9b42cf6640 1 --2- 172.21.15.131:0/55381800 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b3c06e660 0x7f9b3c15f9a0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.505+0000 7f9b42cf6640 1 -- 172.21.15.131:0/55381800 >> 172.21.15.131:0/55381800 conn(0x7f9b3c0cd940 msgr2=0x7f9b3c0cdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:53.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.505+0000 7f9b42cf6640 1 -- 172.21.15.131:0/55381800 shutdown_connections 2024-09-17T02:52:53.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.505+0000 7f9b42cf6640 1 -- 172.21.15.131:0/55381800 wait complete. 2024-09-17T02:52:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b42cf6640 1 Processor -- start 2024-09-17T02:52:53.503 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b42cf6640 1 -- start start 2024-09-17T02:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b42cf6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b3c0afd70 0x7f9b3c1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b42cf6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b3c15c5e0 0x7f9b3c158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b42cf6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9b3c1558c0 0x7f9b3c1588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b42cf6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c168c50 con 0x7f9b3c15c5e0 2024-09-17T02:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b42cf6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c168ad0 con 0x7f9b3c0afd70 2024-09-17T02:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b42cf6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3c168dd0 con 0x7f9b3c1558c0 2024-09-17T02:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b3bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b3c15c5e0 0x7f9b3c158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:53.506 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b3bfff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b3c15c5e0 0x7f9b3c158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45022/0 (socket says 172.21.15.131:45022) 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b3bfff640 1 -- 172.21.15.131:0/3271431068 learned_addr learned my addr 172.21.15.131:0/3271431068 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b40a92640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b3c0afd70 0x7f9b3c1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b41293640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9b3c1558c0 0x7f9b3c1588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b3bfff640 1 -- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9b3c1558c0 msgr2=0x7f9b3c1588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b3bfff640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9b3c1558c0 0x7f9b3c1588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b3bfff640 1 -- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b3c0afd70 msgr2=0x7f9b3c1525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b3bfff640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b3c0afd70 0x7f9b3c1525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b3bfff640 1 -- 172.21.15.131:0/3271431068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3c152af0 con 0x7f9b3c15c5e0 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b3bfff640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b3c15c5e0 0x7f9b3c158390 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c01d030 tx=0x7f9b2c0a6590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b397fa640 1 -- 172.21.15.131:0/3271431068 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0ac030 con 0x7f9b3c15c5e0 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b397fa640 1 -- 172.21.15.131:0/3271431068 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b2c0054b0 con 0x7f9b3c15c5e0 2024-09-17T02:52:53.507 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b42cf6640 1 -- 172.21.15.131:0/3271431068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b3c0b6670 con 0x7f9b3c15c5e0 2024-09-17T02:52:53.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b397fa640 1 -- 172.21.15.131:0/3271431068 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c004930 con 0x7f9b3c15c5e0 2024-09-17T02:52:53.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.509+0000 7f9b42cf6640 1 -- 172.21.15.131:0/3271431068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3c0b6390 con 0x7f9b3c15c5e0 2024-09-17T02:52:53.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.513+0000 7f9b42cf6640 1 -- 172.21.15.131:0/3271431068 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b3c0b5e20 con 0x7f9b3c15c5e0 2024-09-17T02:52:53.508 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.513+0000 7f9b397fa640 1 -- 172.21.15.131:0/3271431068 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f9b2c0af070 con 0x7f9b3c15c5e0 2024-09-17T02:52:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.513+0000 7f9b397fa640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9b28073230 0x7f9b28075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.513+0000 7f9b397fa640 1 -- 172.21.15.131:0/3271431068 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f9b2c12e410 con 0x7f9b3c15c5e0 2024-09-17T02:52:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.513+0000 7f9b40a92640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9b28073230 0x7f9b28075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:53.509 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.513+0000 7f9b40a92640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9b28073230 0x7f9b28075740 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9b34002210 tx=0x7f9b34001a00 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:53.512 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.517+0000 7f9b397fa640 1 -- 172.21.15.131:0/3271431068 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f9b2c016410 con 0x7f9b3c15c5e0 2024-09-17T02:52:53.679 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.685+0000 7f9b42cf6640 1 -- 172.21.15.131:0/3271431068 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b3c15f9a0 con 0x7f9b28073230 2024-09-17T02:52:53.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.685+0000 7f9b397fa640 1 -- 172.21.15.131:0/3271431068 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13510 (secure 0 0 0) 0x7f9b3c15f9a0 con 0x7f9b28073230 2024-09-17T02:52:53.681 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:53.681 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:52:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 -- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9b28073230 msgr2=0x7f9b28075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9b28073230 0x7f9b28075740 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9b34002210 tx=0x7f9b34001a00 comp rx=0 tx=0).stop 2024-09-17T02:52:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 -- 172.21.15.131:0/3271431068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b3c15c5e0 msgr2=0x7f9b3c158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b3c15c5e0 0x7f9b3c158390 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c01d030 tx=0x7f9b2c0a6590 comp rx=0 tx=0).stop 2024-09-17T02:52:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 -- 172.21.15.131:0/3271431068 shutdown_connections 2024-09-17T02:52:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f9b28073230 0x7f9b28075740 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f9b3c1558c0 0x7f9b3c1588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f9b3c15c5e0 0x7f9b3c158390 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 --2- 172.21.15.131:0/3271431068 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f9b3c0afd70 0x7f9b3c1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 -- 172.21.15.131:0/3271431068 >> 172.21.15.131:0/3271431068 conn(0x7f9b3c0cd940 msgr2=0x7f9b3c06d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:53.683 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 -- 172.21.15.131:0/3271431068 shutdown_connections 2024-09-17T02:52:53.684 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.689+0000 7f9b42cf6640 1 -- 172.21.15.131:0/3271431068 wait complete. 2024-09-17T02:52:53.703 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-17T02:52:53.414873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":279308,"kb_used_data":29780,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695412,"statfs":{"total":773094113280,"available":772808101888,"internally_reserved":0,"allocated":30494720,"data_stored":23816673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-177,"num_write_kb":-8294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002468"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":314,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T02:52:51.171022+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:52:51.171022+0000","last_peered":"2024-09-17T02:52:51.171022+0000","last_clean":"2024-09-17T02:52:51.171022+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:52:51.171022+0000","last_undegraded":"2024-09-17T02:52:51.171022+0000","last_fullsized":"2024-09-17T02:52:51.171022+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509590,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36816,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":5758976,"data_stored":4927458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2162688,"data_stored":1323527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509591,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1564672,"data_stored":733154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1568768,"data_stored":733161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509590,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":5767168,"data_stored":4927470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":6204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":6352896,"data_stored":5517826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509591,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36816,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":5758976,"data_stored":4927458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1560576,"data_stored":726619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:52:53.704 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T02:52:53.704 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:52:53.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.865+0000 7f776071a640 1 -- 172.21.15.131:0/401973896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7758160c30 msgr2=0x7f77580d26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.865+0000 7f776071a640 1 --2- 172.21.15.131:0/401973896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7758160c30 0x7f77580d26b0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f7750009c80 tx=0x7f775001cda0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.865+0000 7f776071a640 1 -- 172.21.15.131:0/401973896 shutdown_connections 2024-09-17T02:52:53.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.865+0000 7f776071a640 1 --2- 172.21.15.131:0/401973896 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7758160c30 0x7f77580d26b0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.865+0000 7f776071a640 1 --2- 172.21.15.131:0/401973896 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7758160820 0x7f77580d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.865+0000 7f776071a640 1 -- 172.21.15.131:0/401973896 >> 172.21.15.131:0/401973896 conn(0x7f775806d240 msgr2=0x7f775806d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:53.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.865+0000 7f776071a640 1 -- 172.21.15.131:0/401973896 shutdown_connections 2024-09-17T02:52:53.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.865+0000 7f776071a640 1 -- 172.21.15.131:0/401973896 wait complete. 2024-09-17T02:52:53.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.865+0000 7f776071a640 1 Processor -- start 2024-09-17T02:52:53.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.865+0000 7f776071a640 1 -- start start 2024-09-17T02:52:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f776071a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77580cebf0 0x7f77580c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f776071a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7758160820 0x7f77580b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f776071a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7758160c30 0x7f77580c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f776071a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7758169ec0 con 0x7f77580cebf0 2024-09-17T02:52:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f776071a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7758169d40 con 0x7f7758160820 2024-09-17T02:52:53.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f776071a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f775816a040 con 0x7f7758160c30 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f775e4b6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77580cebf0 0x7f77580c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f775e4b6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77580cebf0 0x7f77580c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45034/0 (socket says 172.21.15.131:45034) 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f775e4b6640 1 -- 172.21.15.131:0/908484740 learned_addr learned my addr 172.21.15.131:0/908484740 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f775ecb7640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7758160c30 0x7f77580c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f775dcb5640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7758160820 0x7f77580b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f775e4b6640 1 -- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7758160c30 msgr2=0x7f77580c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f775e4b6640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7758160c30 0x7f77580c5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f775e4b6640 1 -- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7758160820 msgr2=0x7f77580b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f775e4b6640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7758160820 0x7f77580b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f775e4b6640 1 -- 172.21.15.131:0/908484740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77581f7120 con 0x7f77580cebf0 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f775e4b6640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77580cebf0 0x7f77580c7cb0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f775400c3f0 tx=0x7f775400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f774effd640 1 -- 172.21.15.131:0/908484740 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754017070 con 0x7f77580cebf0 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f774effd640 1 -- 172.21.15.131:0/908484740 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77540053c0 con 0x7f77580cebf0 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f776071a640 1 -- 172.21.15.131:0/908484740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77581f7400 con 0x7f77580cebf0 2024-09-17T02:52:53.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f774effd640 1 -- 172.21.15.131:0/908484740 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754003860 con 0x7f77580cebf0 2024-09-17T02:52:53.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f776071a640 1 -- 172.21.15.131:0/908484740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77580c82b0 con 0x7f77580cebf0 2024-09-17T02:52:53.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f776071a640 1 -- 172.21.15.131:0/908484740 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77581f7920 con 0x7f77580cebf0 2024-09-17T02:52:53.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.869+0000 7f774effd640 1 -- 172.21.15.131:0/908484740 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f775401a070 con 0x7f77580cebf0 2024-09-17T02:52:53.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.873+0000 7f774effd640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f773c073200 0x7f773c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:53.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.873+0000 7f774effd640 1 -- 172.21.15.131:0/908484740 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f7754093470 con 0x7f77580cebf0 2024-09-17T02:52:53.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.873+0000 7f775dcb5640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f773c073200 0x7f773c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:53.867 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.873+0000 7f775dcb5640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f773c073200 0x7f773c075710 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7748005fd0 tx=0x7f7748005d10 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:53.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:53.877+0000 7f774effd640 1 -- 172.21.15.131:0/908484740 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f77581f7920 con 0x7f77580cebf0 2024-09-17T02:52:54.028 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.033+0000 7f776071a640 1 -- 172.21.15.131:0/908484740 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77580c9010 con 0x7f773c073200 2024-09-17T02:52:54.029 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.033+0000 7f774effd640 1 -- 172.21.15.131:0/908484740 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13510 (secure 0 0 0) 0x7f77580c9010 con 0x7f773c073200 2024-09-17T02:52:54.030 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:54.030 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:52:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 -- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f773c073200 msgr2=0x7f773c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f773c073200 0x7f773c075710 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7748005fd0 tx=0x7f7748005d10 comp rx=0 tx=0).stop 2024-09-17T02:52:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 -- 172.21.15.131:0/908484740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77580cebf0 msgr2=0x7f77580c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77580cebf0 0x7f77580c7cb0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f775400c3f0 tx=0x7f775400c8b0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 -- 172.21.15.131:0/908484740 shutdown_connections 2024-09-17T02:52:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f773c073200 0x7f773c075710 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7758160c30 0x7f77580c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7758160820 0x7f77580b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.032 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 --2- 172.21.15.131:0/908484740 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f77580cebf0 0x7f77580c7cb0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 -- 172.21.15.131:0/908484740 >> 172.21.15.131:0/908484740 conn(0x7f775806d240 msgr2=0x7f775815e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:54.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 -- 172.21.15.131:0/908484740 shutdown_connections 2024-09-17T02:52:54.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.037+0000 7f776071a640 1 -- 172.21.15.131:0/908484740 wait complete. 2024-09-17T02:52:54.051 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-17T02:52:53.414873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":279308,"kb_used_data":29780,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695412,"statfs":{"total":773094113280,"available":772808101888,"internally_reserved":0,"allocated":30494720,"data_stored":23816673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392376,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-177,"num_write_kb":-8294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002468"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":314,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T02:52:51.171022+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:52:51.171022+0000","last_peered":"2024-09-17T02:52:51.171022+0000","last_clean":"2024-09-17T02:52:51.171022+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:52:51.171022+0000","last_undegraded":"2024-09-17T02:52:51.171022+0000","last_fullsized":"2024-09-17T02:52:51.171022+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509590,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36816,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":5758976,"data_stored":4927458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509590,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2162688,"data_stored":1323527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509591,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32720,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339120,"statfs":{"total":96636764160,"available":96603258880,"internally_reserved":0,"allocated":1564672,"data_stored":733154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509589,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32724,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1568768,"data_stored":733161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509590,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":5767168,"data_stored":4927470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":6204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":6352896,"data_stored":5517826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509591,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36816,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335024,"statfs":{"total":96636764160,"available":96599064576,"internally_reserved":0,"allocated":5758976,"data_stored":4927458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32712,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339128,"statfs":{"total":96636764160,"available":96603267072,"internally_reserved":0,"allocated":1560576,"data_stored":726619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:52:54.051 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T02:52:54.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.205+0000 7f5acd438640 1 -- 172.21.15.131:0/1805731342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa80040f0 msgr2=0x7f5aa8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:54.201 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.205+0000 7f5acd438640 1 --2- 172.21.15.131:0/1805731342 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5aa80040f0 0x7f5aa8004810 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800c610 tx=0x7f5ab801f680 comp rx=0 tx=0).stop 2024-09-17T02:52:54.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.205+0000 7f5ac77fe640 1 -- 172.21.15.131:0/1805731342 reap_dead start 2024-09-17T02:52:54.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.205+0000 7f5acd438640 1 -- 172.21.15.131:0/1805731342 shutdown_connections 2024-09-17T02:52:54.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.205+0000 7f5acd438640 1 -- 172.21.15.131:0/1805731342 >> 172.21.15.131:0/1805731342 conn(0x7f5ac80d0e40 msgr2=0x7f5ac80d1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:54.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.205+0000 7f5acd438640 1 -- 172.21.15.131:0/1805731342 shutdown_connections 2024-09-17T02:52:54.202 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.205+0000 7f5acd438640 1 -- 172.21.15.131:0/1805731342 wait complete. 2024-09-17T02:52:54.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5acd438640 1 Processor -- start 2024-09-17T02:52:54.203 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5acd438640 1 -- start start 2024-09-17T02:52:54.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5acd438640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5aa80040f0 0x7f5ac8159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:54.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5acd438640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5ac815a5a0 0x7f5ac80b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:54.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5acd438640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ac80b8de0 0x7f5ac81f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:54.205 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5acd438640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8160350 con 0x7f5ac80b8de0 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5acd438640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac80d2730 con 0x7f5aa80040f0 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5acd438640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac81604d0 con 0x7f5ac815a5a0 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5ac77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ac80b8de0 0x7f5ac81f8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5ac77fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ac80b8de0 0x7f5ac81f8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45042/0 (socket says 172.21.15.131:45042) 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5ac77fe640 1 -- 172.21.15.131:0/4061117547 learned_addr learned my addr 172.21.15.131:0/4061117547 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5ac6ffd640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5aa80040f0 0x7f5ac8159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5abe7fc640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5ac815a5a0 0x7f5ac80b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5ac77fe640 1 -- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5ac815a5a0 msgr2=0x7f5ac80b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5ac77fe640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5ac815a5a0 0x7f5ac80b88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5ac77fe640 1 -- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5aa80040f0 msgr2=0x7f5ac8159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5ac77fe640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5aa80040f0 0x7f5ac8159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5ac77fe640 1 -- 172.21.15.131:0/4061117547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac80b91f0 con 0x7f5ac80b8de0 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5ac77fe640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ac80b8de0 0x7f5ac81f8fd0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80ba470 tx=0x7f5ac000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:54.206 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5abffff640 1 -- 172.21.15.131:0/4061117547 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000f040 con 0x7f5ac80b8de0 2024-09-17T02:52:54.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5abffff640 1 -- 172.21.15.131:0/4061117547 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ac0018070 con 0x7f5ac80b8de0 2024-09-17T02:52:54.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5acd438640 1 -- 172.21.15.131:0/4061117547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac80cd9f0 con 0x7f5ac80b8de0 2024-09-17T02:52:54.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5abffff640 1 -- 172.21.15.131:0/4061117547 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000c900 con 0x7f5ac80b8de0 2024-09-17T02:52:54.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.209+0000 7f5acd438640 1 -- 172.21.15.131:0/4061117547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac80cdfb0 con 0x7f5ac80b8de0 2024-09-17T02:52:54.207 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.213+0000 7f5acd438640 1 -- 172.21.15.131:0/4061117547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac80b6170 con 0x7f5ac80b8de0 2024-09-17T02:52:54.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.213+0000 7f5abffff640 1 -- 172.21.15.131:0/4061117547 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f5ac000caa0 con 0x7f5ac80b8de0 2024-09-17T02:52:54.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.213+0000 7f5abffff640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5aac073200 0x7f5aac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:54.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.213+0000 7f5abffff640 1 -- 172.21.15.131:0/4061117547 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f5ac0093d00 con 0x7f5ac80b8de0 2024-09-17T02:52:54.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.213+0000 7f5ac6ffd640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5aac073200 0x7f5aac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:54.208 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.213+0000 7f5ac6ffd640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5aac073200 0x7f5aac075710 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009ab0 tx=0x7f5ab00091e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:54.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.217+0000 7f5abffff640 1 -- 172.21.15.131:0/4061117547 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f5ac80b91f0 con 0x7f5ac80b8de0 2024-09-17T02:52:54.380 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.385+0000 7f5acd438640 1 -- 172.21.15.131:0/4061117547 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ac80cd780 con 0x7f5ac80b8de0 2024-09-17T02:52:54.381 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.385+0000 7f5abffff640 1 -- 172.21.15.131:0/4061117547 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) ==== 74+0+13629 (secure 0 0 0) 0x7f5ac005ede0 con 0x7f5ac80b8de0 2024-09-17T02:52:54.381 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:54.381 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":127,"fsid":"f03002b8-e723-4674-9385-a37ff83176c9","created":"2024-09-17T02:40:26.203384+0000","modified":"2024-09-17T02:52:51.143113+0000","last_up_change":"2024-09-17T02:40:40.942146+0000","last_in_change":"2024-09-17T02:40:30.823064+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T02:40:41.163614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5ed37453-7b40-4153-8943-f14db057d578","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6808","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6809","nonce":1154840320}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6810","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6811","nonce":1154840320}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6814","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6815","nonce":1154840320}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6812","nonce":1154840320},{"type":"v1","addr":"172.21.15.131:6813","nonce":1154840320}]},"public_addr":"172.21.15.131:6809/1154840320","cluster_addr":"172.21.15.131:6811/1154840320","heartbeat_back_addr":"172.21.15.131:6815/1154840320","heartbeat_front_addr":"172.21.15.131:6813/1154840320","state":["exists","up"]},{"osd":1,"uuid":"fdcf031e-6ed3-418a-b404-6fda63045e6a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6824","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6825","nonce":1616383728}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6826","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6827","nonce":1616383728}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6830","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6831","nonce":1616383728}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6828","nonce":1616383728},{"type":"v1","addr":"172.21.15.131:6829","nonce":1616383728}]},"public_addr":"172.21.15.131:6825/1616383728","cluster_addr":"172.21.15.131:6827/1616383728","heartbeat_back_addr":"172.21.15.131:6831/1616383728","heartbeat_front_addr":"172.21.15.131:6829/1616383728","state":["exists","up"]},{"osd":2,"uuid":"d004f0b0-433d-46db-a270-e6f5f1f6e851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6800","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6801","nonce":163801004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6802","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6803","nonce":163801004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6806","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6807","nonce":163801004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6804","nonce":163801004},{"type":"v1","addr":"172.21.15.131:6805","nonce":163801004}]},"public_addr":"172.21.15.131:6801/163801004","cluster_addr":"172.21.15.131:6803/163801004","heartbeat_back_addr":"172.21.15.131:6807/163801004","heartbeat_front_addr":"172.21.15.131:6805/163801004","state":["exists","up"]},{"osd":3,"uuid":"ae5eb561-b2d1-4e1f-93db-a2076d46bf5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6816","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6817","nonce":4047536388}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6818","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6819","nonce":4047536388}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6822","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6823","nonce":4047536388}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6820","nonce":4047536388},{"type":"v1","addr":"172.21.15.131:6821","nonce":4047536388}]},"public_addr":"172.21.15.131:6817/4047536388","cluster_addr":"172.21.15.131:6819/4047536388","heartbeat_back_addr":"172.21.15.131:6823/4047536388","heartbeat_front_addr":"172.21.15.131:6821/4047536388","state":["exists","up"]},{"osd":4,"uuid":"a25e80d7-ac62-4a4c-a40f-95d4b43d985b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6818","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6819","nonce":1965940492}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6820","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6821","nonce":1965940492}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6824","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6825","nonce":1965940492}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6822","nonce":1965940492},{"type":"v1","addr":"172.21.15.189:6823","nonce":1965940492}]},"public_addr":"172.21.15.189:6819/1965940492","cluster_addr":"172.21.15.189:6821/1965940492","heartbeat_back_addr":"172.21.15.189:6825/1965940492","heartbeat_front_addr":"172.21.15.189:6823/1965940492","state":["exists","up"]},{"osd":5,"uuid":"1246ef1d-3575-4bbd-9cad-c79513551797","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6826","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6827","nonce":3175183672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6828","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6829","nonce":3175183672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6832","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6833","nonce":3175183672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6830","nonce":3175183672},{"type":"v1","addr":"172.21.15.189:6831","nonce":3175183672}]},"public_addr":"172.21.15.189:6827/3175183672","cluster_addr":"172.21.15.189:6829/3175183672","heartbeat_back_addr":"172.21.15.189:6833/3175183672","heartbeat_front_addr":"172.21.15.189:6831/3175183672","state":["exists","up"]},{"osd":6,"uuid":"f660a25e-255b-49d0-872a-d35c13154d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6802","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6803","nonce":952142225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6804","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6805","nonce":952142225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6808","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6809","nonce":952142225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6806","nonce":952142225},{"type":"v1","addr":"172.21.15.189:6807","nonce":952142225}]},"public_addr":"172.21.15.189:6803/952142225","cluster_addr":"172.21.15.189:6805/952142225","heartbeat_back_addr":"172.21.15.189:6809/952142225","heartbeat_front_addr":"172.21.15.189:6807/952142225","state":["exists","up"]},{"osd":7,"uuid":"92b097eb-7f27-4093-b098-45aebfa15244","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6810","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6811","nonce":3586470400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6812","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6813","nonce":3586470400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6816","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6817","nonce":3586470400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6814","nonce":3586470400},{"type":"v1","addr":"172.21.15.189:6815","nonce":3586470400}]},"public_addr":"172.21.15.189:6811/3586470400","cluster_addr":"172.21.15.189:6813/3586470400","heartbeat_back_addr":"172.21.15.189:6817/3586470400","heartbeat_front_addr":"172.21.15.189:6815/3586470400","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.302357+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.855188+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.202195+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.564603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.994267+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.741577+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.076436+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T02:40:37.275492+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6832/4011259083":"2024-09-18T02:52:47.976234+0000","172.21.15.131:6835/2981077586":"2024-09-18T02:52:34.677555+0000","172.21.15.131:6832/2981077586":"2024-09-18T02:52:34.677555+0000","172.21.15.131:6835/4011259083":"2024-09-18T02:52:47.976234+0000","172.21.15.131:6835/2599844908":"2024-09-18T02:52:16.917982+0000","172.21.15.131:6832/2599844908":"2024-09-18T02:52:16.917982+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T02:52:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 -- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5aac073200 msgr2=0x7f5aac075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5aac073200 0x7f5aac075710 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009ab0 tx=0x7f5ab00091e0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 -- 172.21.15.131:0/4061117547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ac80b8de0 msgr2=0x7f5ac81f8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ac80b8de0 0x7f5ac81f8fd0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f5ac80ba470 tx=0x7f5ac000ec20 comp rx=0 tx=0).stop 2024-09-17T02:52:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 -- 172.21.15.131:0/4061117547 shutdown_connections 2024-09-17T02:52:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f5aac073200 0x7f5aac075710 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.384 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f5ac80b8de0 0x7f5ac81f8fd0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f5ac815a5a0 0x7f5ac80b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 --2- 172.21.15.131:0/4061117547 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f5aa80040f0 0x7f5ac8159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 -- 172.21.15.131:0/4061117547 >> 172.21.15.131:0/4061117547 conn(0x7f5ac80d0e40 msgr2=0x7f5ac80b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 -- 172.21.15.131:0/4061117547 shutdown_connections 2024-09-17T02:52:54.385 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:54.389+0000 7f5acd438640 1 -- 172.21.15.131:0/4061117547 wait complete. 2024-09-17T02:52:55.404 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T02:52:55.404 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T02:52:55.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.581+0000 7f62992cc640 1 -- 172.21.15.131:0/509353629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62940ce210 msgr2=0x7f62940ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:55.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.581+0000 7f62992cc640 1 --2- 172.21.15.131:0/509353629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62940ce210 0x7f62940ce620 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f628c009bc0 tx=0x7f628c01cb70 comp rx=0 tx=0).stop 2024-09-17T02:52:55.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.581+0000 7f62992cc640 1 -- 172.21.15.131:0/509353629 shutdown_connections 2024-09-17T02:52:55.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.581+0000 7f62992cc640 1 --2- 172.21.15.131:0/509353629 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62940ce210 0x7f62940ce620 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.581+0000 7f62992cc640 1 --2- 172.21.15.131:0/509353629 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f62941524e0 0x7f629415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.576 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.581+0000 7f62992cc640 1 -- 172.21.15.131:0/509353629 >> 172.21.15.131:0/509353629 conn(0x7f629406d010 msgr2=0x7f629406d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:55.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.581+0000 7f62992cc640 1 -- 172.21.15.131:0/509353629 shutdown_connections 2024-09-17T02:52:55.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.581+0000 7f62992cc640 1 -- 172.21.15.131:0/509353629 wait complete. 2024-09-17T02:52:55.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.581+0000 7f62992cc640 1 Processor -- start 2024-09-17T02:52:55.577 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.581+0000 7f62992cc640 1 -- start start 2024-09-17T02:52:55.580 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62992cc640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62940ce210 0x7f62940cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62992cc640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f62940d02e0 0x7f6294148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62992cc640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62941524e0 0x7f62940b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62992cc640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f629406f3e0 con 0x7f62941524e0 2024-09-17T02:52:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62992cc640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f629406f260 con 0x7f62940ce210 2024-09-17T02:52:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62992cc640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f629406f560 con 0x7f62940d02e0 2024-09-17T02:52:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62937fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62941524e0 0x7f62940b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62937fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62941524e0 0x7f62940b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45068/0 (socket says 172.21.15.131:45068) 2024-09-17T02:52:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62937fe640 1 -- 172.21.15.131:0/2949096158 learned_addr learned my addr 172.21.15.131:0/2949096158 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f6292ffd640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62940ce210 0x7f62940cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:55.581 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f628bfff640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f62940d02e0 0x7f6294148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62937fe640 1 -- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f62940d02e0 msgr2=0x7f6294148bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62937fe640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f62940d02e0 0x7f6294148bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62937fe640 1 -- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62940ce210 msgr2=0x7f62940cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62937fe640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f62940ce210 0x7f62940cb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62937fe640 1 -- 172.21.15.131:0/2949096158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629414cce0 con 0x7f62941524e0 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f628bfff640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f62940d02e0 0x7f6294148bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62937fe640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62941524e0 0x7f62940b19e0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f628c009b10 tx=0x7f628c005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f628b7fe640 1 -- 172.21.15.131:0/2949096158 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c0bc070 con 0x7f62941524e0 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f628b7fe640 1 -- 172.21.15.131:0/2949096158 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f628c0042f0 con 0x7f62941524e0 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62992cc640 1 -- 172.21.15.131:0/2949096158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629414cfc0 con 0x7f62941524e0 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f628b7fe640 1 -- 172.21.15.131:0/2949096158 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c0b2040 con 0x7f62941524e0 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62992cc640 1 -- 172.21.15.131:0/2949096158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f629414d580 con 0x7f62941524e0 2024-09-17T02:52:55.582 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f62992cc640 1 -- 172.21.15.131:0/2949096158 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f629406faa0 con 0x7f62941524e0 2024-09-17T02:52:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.585+0000 7f628b7fe640 1 -- 172.21.15.131:0/2949096158 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f628c004700 con 0x7f62941524e0 2024-09-17T02:52:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.589+0000 7f628b7fe640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6274073230 0x7f6274075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.589+0000 7f628b7fe640 1 -- 172.21.15.131:0/2949096158 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f628c12e830 con 0x7f62941524e0 2024-09-17T02:52:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.589+0000 7f628b7fe640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x7f6274079860 0x7f627407bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.589+0000 7f628b7fe640 1 -- 172.21.15.131:0/2949096158 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f628c0a5d30 con 0x7f6274079860 2024-09-17T02:52:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.589+0000 7f628b7fe640 1 -- 172.21.15.131:0/2949096158 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f628c12ecc0 con 0x7f62941524e0 2024-09-17T02:52:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.589+0000 7f6292ffd640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6274073230 0x7f6274075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.589+0000 7f628bfff640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x7f6274079860 0x7f627407bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:55.584 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.589+0000 7f628bfff640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x7f6274079860 0x7f627407bcf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:55.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.589+0000 7f6292ffd640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6274073230 0x7f6274075740 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6284005fd0 tx=0x7f628400c460 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:55.585 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.589+0000 7f628b7fe640 1 -- 172.21.15.131:0/2949096158 <== osd.0 v2:172.21.15.131:6808/1154840320 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f628c0a5d30 con 0x7f6274079860 2024-09-17T02:52:55.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62992cc640 1 -- 172.21.15.131:0/2949096158 --> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f629414ca50 con 0x7f6274079860 2024-09-17T02:52:55.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f628b7fe640 1 -- 172.21.15.131:0/2949096158 <== osd.0 v2:172.21.15.131:6808/1154840320 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f629414ca50 con 0x7f6274079860 2024-09-17T02:52:55.597 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:55.597 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T02:52:55.597 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62992cc640 1 -- 172.21.15.131:0/2949096158 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x7f6274079860 msgr2=0x7f627407bcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62992cc640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.131:6808/1154840320,v1:172.21.15.131:6809/1154840320] conn(0x7f6274079860 0x7f627407bcf0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62992cc640 1 -- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6274073230 msgr2=0x7f6274075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62992cc640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f6274073230 0x7f6274075740 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6284005fd0 tx=0x7f628400c460 comp rx=0 tx=0).stop 2024-09-17T02:52:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62992cc640 1 -- 172.21.15.131:0/2949096158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62941524e0 msgr2=0x7f62940b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62992cc640 1 --2- 172.21.15.131:0/2949096158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f62941524e0 0x7f62940b19e0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f628c009b10 tx=0x7f628c005b10 comp rx=0 tx=0).stop 2024-09-17T02:52:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62937fe640 1 -- 172.21.15.131:0/2949096158 reap_dead start 2024-09-17T02:52:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62992cc640 1 -- 172.21.15.131:0/2949096158 shutdown_connections 2024-09-17T02:52:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62992cc640 1 -- 172.21.15.131:0/2949096158 >> 172.21.15.131:0/2949096158 conn(0x7f629406d010 msgr2=0x7f629414bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62992cc640 1 -- 172.21.15.131:0/2949096158 shutdown_connections 2024-09-17T02:52:55.598 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.601+0000 7f62992cc640 1 -- 172.21.15.131:0/2949096158 wait complete. 2024-09-17T02:52:55.616 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T02:52:55.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.761+0000 7f091c4b6640 1 -- 172.21.15.131:0/3506742200 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0914160be0 msgr2=0x7f09140caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:55.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.761+0000 7f091c4b6640 1 --2- 172.21.15.131:0/3506742200 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0914160be0 0x7f09140caf70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0904009840 tx=0x7f090401c6d0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.761+0000 7f091c4b6640 1 -- 172.21.15.131:0/3506742200 shutdown_connections 2024-09-17T02:52:55.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.761+0000 7f091c4b6640 1 --2- 172.21.15.131:0/3506742200 >> v2:172.21.15.189:3300/0 conn(0x7f091406fa10 0x7f09140cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.761+0000 7f091c4b6640 1 --2- 172.21.15.131:0/3506742200 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f0914160be0 0x7f09140caf70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.761+0000 7f091c4b6640 1 -- 172.21.15.131:0/3506742200 >> 172.21.15.131:0/3506742200 conn(0x7f09140cff40 msgr2=0x7f09140d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:55.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.761+0000 7f091c4b6640 1 -- 172.21.15.131:0/3506742200 shutdown_connections 2024-09-17T02:52:55.757 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.761+0000 7f091c4b6640 1 -- 172.21.15.131:0/3506742200 wait complete. 2024-09-17T02:52:55.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091c4b6640 1 Processor -- start 2024-09-17T02:52:55.758 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091c4b6640 1 -- start start 2024-09-17T02:52:55.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091c4b6640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f091406eef0 0x7f091414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:55.760 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091c4b6640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f091406fa10 0x7f09140b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091c4b6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0914160be0 0x7f0914153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091c4b6640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f0914169c30 con 0x7f0914160be0 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091c4b6640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f0914169ab0 con 0x7f091406eef0 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091c4b6640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f0914169db0 con 0x7f091406fa10 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091aa53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0914160be0 0x7f0914153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091aa53640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0914160be0 0x7f0914153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45084/0 (socket says 172.21.15.131:45084) 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091aa53640 1 -- 172.21.15.131:0/3953682918 learned_addr learned my addr 172.21.15.131:0/3953682918 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091a252640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f091406eef0 0x7f091414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f0919a51640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f091406fa10 0x7f09140b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091aa53640 1 -- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f091406fa10 msgr2=0x7f09140b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091aa53640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f091406fa10 0x7f09140b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091aa53640 1 -- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f091406eef0 msgr2=0x7f091414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:55.761 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091aa53640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f091406eef0 0x7f091414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091aa53640 1 -- 172.21.15.131:0/3953682918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09140cb7f0 con 0x7f0914160be0 2024-09-17T02:52:55.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091a252640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f091406eef0 0x7f091414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:52:55.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091aa53640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0914160be0 0x7f0914153240 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f090c00e940 tx=0x7f090c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:55.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f090affd640 1 -- 172.21.15.131:0/3953682918 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c00cc60 con 0x7f0914160be0 2024-09-17T02:52:55.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f090affd640 1 -- 172.21.15.131:0/3953682918 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f090c00ce00 con 0x7f0914160be0 2024-09-17T02:52:55.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091c4b6640 1 -- 172.21.15.131:0/3953682918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0914156e20 con 0x7f0914160be0 2024-09-17T02:52:55.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f090affd640 1 -- 172.21.15.131:0/3953682918 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c004280 con 0x7f0914160be0 2024-09-17T02:52:55.762 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091c4b6640 1 -- 172.21.15.131:0/3953682918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0914156b40 con 0x7f0914160be0 2024-09-17T02:52:55.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.765+0000 7f091c4b6640 1 -- 172.21.15.131:0/3953682918 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f091406f300 con 0x7f0914160be0 2024-09-17T02:52:55.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.769+0000 7f090affd640 1 -- 172.21.15.131:0/3953682918 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f090c003600 con 0x7f0914160be0 2024-09-17T02:52:55.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.769+0000 7f090affd640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08f8073230 0x7f08f8075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:55.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.769+0000 7f090affd640 1 -- 172.21.15.131:0/3953682918 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f090c0949b0 con 0x7f0914160be0 2024-09-17T02:52:55.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.769+0000 7f091a252640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08f8073230 0x7f08f8075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:55.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.769+0000 7f091a252640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08f8073230 0x7f08f8075740 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0910002180 tx=0x7f0910009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:55.766 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.773+0000 7f090affd640 1 -- 172.21.15.131:0/3953682918 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f0914156e20 con 0x7f0914160be0 2024-09-17T02:52:55.934 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.937+0000 7f091c4b6640 1 -- 172.21.15.131:0/3953682918 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f09141557a0 con 0x7f08f8073230 2024-09-17T02:52:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f090affd640 1 -- 172.21.15.131:0/3953682918 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f09141557a0 con 0x7f08f8073230 2024-09-17T02:52:55.935 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T02:52:55.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 -- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08f8073230 msgr2=0x7f08f8075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:55.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08f8073230 0x7f08f8075740 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0910002180 tx=0x7f0910009040 comp rx=0 tx=0).stop 2024-09-17T02:52:55.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 -- 172.21.15.131:0/3953682918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0914160be0 msgr2=0x7f0914153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:55.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0914160be0 0x7f0914153240 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f090c00e940 tx=0x7f090c00ed40 comp rx=0 tx=0).stop 2024-09-17T02:52:55.937 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 -- 172.21.15.131:0/3953682918 shutdown_connections 2024-09-17T02:52:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f08f8073230 0x7f08f8075740 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f0914160be0 0x7f0914153240 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f091406fa10 0x7f09140b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 --2- 172.21.15.131:0/3953682918 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f091406eef0 0x7f091414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 -- 172.21.15.131:0/3953682918 >> 172.21.15.131:0/3953682918 conn(0x7f09140cff40 msgr2=0x7f0914155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 -- 172.21.15.131:0/3953682918 shutdown_connections 2024-09-17T02:52:55.938 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:55.941+0000 7f091c4b6640 1 -- 172.21.15.131:0/3953682918 wait complete. 2024-09-17T02:52:55.957 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T02:52:55.957 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T02:52:56.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 -- 172.21.15.131:0/2761973754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f620806f950 msgr2=0x7f62080d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.110 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 --2- 172.21.15.131:0/2761973754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f620806f950 0x7f62080d02e0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f62080cead0 tx=0x7f61fc01c770 comp rx=0 tx=0).stop 2024-09-17T02:52:56.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 -- 172.21.15.131:0/2761973754 shutdown_connections 2024-09-17T02:52:56.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 --2- 172.21.15.131:0/2761973754 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f620806f950 0x7f62080d02e0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 --2- 172.21.15.131:0/2761973754 >> v2:172.21.15.189:3300/0 conn(0x7f620815d040 0x7f62080d0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 --2- 172.21.15.131:0/2761973754 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f620806f540 0x7f620815cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 -- 172.21.15.131:0/2761973754 >> 172.21.15.131:0/2761973754 conn(0x7f620806d010 msgr2=0x7f620806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:56.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 -- 172.21.15.131:0/2761973754 shutdown_connections 2024-09-17T02:52:56.111 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 -- 172.21.15.131:0/2761973754 wait complete. 2024-09-17T02:52:56.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 Processor -- start 2024-09-17T02:52:56.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 -- start start 2024-09-17T02:52:56.114 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f620806f540 0x7f62081f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f620806f950 0x7f62080b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f620815d040 0x7f6208141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f62080cf490 con 0x7f620806f950 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f62080cf310 con 0x7f620806f540 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f620deb1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f62080cf610 con 0x7f620815d040 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f6206ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f620806f950 0x7f62080b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f6206ffd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f620806f950 0x7f62080b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45088/0 (socket says 172.21.15.131:45088) 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f6206ffd640 1 -- 172.21.15.131:0/468434004 learned_addr learned my addr 172.21.15.131:0/468434004 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f62077fe640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f620806f540 0x7f62081f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f6207fff640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f620815d040 0x7f6208141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f6206ffd640 1 -- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f620815d040 msgr2=0x7f6208141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.117+0000 7f6206ffd640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f620815d040 0x7f6208141820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.115 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f6206ffd640 1 -- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f620806f540 msgr2=0x7f62081f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f6206ffd640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f620806f540 0x7f62081f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f6206ffd640 1 -- 172.21.15.131:0/468434004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62080cda30 con 0x7f620806f950 2024-09-17T02:52:56.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f6207fff640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f620815d040 0x7f6208141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:56.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f6206ffd640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f620806f950 0x7f62080b3120 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009850 tx=0x7f61fc0a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:56.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f61effff640 1 -- 172.21.15.131:0/468434004 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0ba020 con 0x7f620806f950 2024-09-17T02:52:56.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f61effff640 1 -- 172.21.15.131:0/468434004 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61fc0043b0 con 0x7f620806f950 2024-09-17T02:52:56.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f620deb1640 1 -- 172.21.15.131:0/468434004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62081f8d60 con 0x7f620806f950 2024-09-17T02:52:56.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f61effff640 1 -- 172.21.15.131:0/468434004 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0b2040 con 0x7f620806f950 2024-09-17T02:52:56.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f620deb1640 1 -- 172.21.15.131:0/468434004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62081f9250 con 0x7f620806f950 2024-09-17T02:52:56.116 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f620deb1640 1 -- 172.21.15.131:0/468434004 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f620815cb00 con 0x7f620806f950 2024-09-17T02:52:56.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f61effff640 1 -- 172.21.15.131:0/468434004 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f61fc005290 con 0x7f620806f950 2024-09-17T02:52:56.117 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f61effff640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f61f4073230 0x7f61f4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f61effff640 1 -- 172.21.15.131:0/468434004 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f61fc12eb10 con 0x7f620806f950 2024-09-17T02:52:56.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f61effff640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x7f61f4079860 0x7f61f407bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f61effff640 1 -- 172.21.15.131:0/468434004 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61fc0c3e50 con 0x7f61f4079860 2024-09-17T02:52:56.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f61effff640 1 -- 172.21.15.131:0/468434004 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f61fc12efa0 con 0x7f620806f950 2024-09-17T02:52:56.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f62077fe640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f61f4073230 0x7f61f4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f6207fff640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x7f61f4079860 0x7f61f407bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f6207fff640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x7f61f4079860 0x7f61f407bcf0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:56.118 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.121+0000 7f62077fe640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f61f4073230 0x7f61f4075740 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f61f8005fd0 tx=0x7f61f800c460 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:56.119 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.125+0000 7f61effff640 1 -- 172.21.15.131:0/468434004 <== osd.1 v2:172.21.15.131:6824/1616383728 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61fc0c3e50 con 0x7f61f4079860 2024-09-17T02:52:56.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f620deb1640 1 -- 172.21.15.131:0/468434004 --> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f62081f9000 con 0x7f61f4079860 2024-09-17T02:52:56.131 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f61effff640 1 -- 172.21.15.131:0/468434004 <== osd.1 v2:172.21.15.131:6824/1616383728 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f62081f9000 con 0x7f61f4079860 2024-09-17T02:52:56.132 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:56.132 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T02:52:56.132 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:56.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f620deb1640 1 -- 172.21.15.131:0/468434004 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x7f61f4079860 msgr2=0x7f61f407bcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f620deb1640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.131:6824/1616383728,v1:172.21.15.131:6825/1616383728] conn(0x7f61f4079860 0x7f61f407bcf0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f620deb1640 1 -- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f61f4073230 msgr2=0x7f61f4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f620deb1640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f61f4073230 0x7f61f4075740 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f61f8005fd0 tx=0x7f61f800c460 comp rx=0 tx=0).stop 2024-09-17T02:52:56.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f620deb1640 1 -- 172.21.15.131:0/468434004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f620806f950 msgr2=0x7f62080b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f620deb1640 1 --2- 172.21.15.131:0/468434004 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f620806f950 0x7f62080b3120 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009850 tx=0x7f61fc0a5690 comp rx=0 tx=0).stop 2024-09-17T02:52:56.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f6207fff640 1 -- 172.21.15.131:0/468434004 reap_dead start 2024-09-17T02:52:56.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f620deb1640 1 -- 172.21.15.131:0/468434004 shutdown_connections 2024-09-17T02:52:56.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f620deb1640 1 -- 172.21.15.131:0/468434004 >> 172.21.15.131:0/468434004 conn(0x7f620806d010 msgr2=0x7f62081432b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:56.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f620deb1640 1 -- 172.21.15.131:0/468434004 shutdown_connections 2024-09-17T02:52:56.133 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.137+0000 7f620deb1640 1 -- 172.21.15.131:0/468434004 wait complete. 2024-09-17T02:52:56.149 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T02:52:56.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.289+0000 7f1722a84640 1 -- 172.21.15.131:0/2879210572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17080039f0 msgr2=0x7f1708003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.289+0000 7f1722a84640 1 --2- 172.21.15.131:0/2879210572 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f17080039f0 0x7f1708003e20 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f170c00c670 tx=0x7f170c01d300 comp rx=0 tx=0).stop 2024-09-17T02:52:56.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.289+0000 7f1721021640 1 -- 172.21.15.131:0/2879210572 reap_dead start 2024-09-17T02:52:56.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.289+0000 7f1722a84640 1 -- 172.21.15.131:0/2879210572 shutdown_connections 2024-09-17T02:52:56.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.289+0000 7f1722a84640 1 -- 172.21.15.131:0/2879210572 >> 172.21.15.131:0/2879210572 conn(0x7f171c06df50 msgr2=0x7f171c06e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:56.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.289+0000 7f1722a84640 1 -- 172.21.15.131:0/2879210572 shutdown_connections 2024-09-17T02:52:56.286 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.289+0000 7f1722a84640 1 -- 172.21.15.131:0/2879210572 wait complete. 2024-09-17T02:52:56.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1722a84640 1 Processor -- start 2024-09-17T02:52:56.287 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1722a84640 1 -- start start 2024-09-17T02:52:56.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1722a84640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17080039f0 0x7f171c14ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1722a84640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f171c06ce40 0x7f171c06d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1722a84640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c166130 0x7f171c166540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1722a84640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f171c0d17d0 con 0x7f171c166130 2024-09-17T02:52:56.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1722a84640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f171c0d1650 con 0x7f17080039f0 2024-09-17T02:52:56.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1722a84640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f171c0d1950 con 0x7f171c06ce40 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1721021640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c166130 0x7f171c166540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1721021640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c166130 0x7f171c166540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45114/0 (socket says 172.21.15.131:45114) 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1721021640 1 -- 172.21.15.131:0/1265568128 learned_addr learned my addr 172.21.15.131:0/1265568128 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1720820640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17080039f0 0x7f171c14ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f171bfff640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f171c06ce40 0x7f171c06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1721021640 1 -- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f171c06ce40 msgr2=0x7f171c06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1721021640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f171c06ce40 0x7f171c06d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1721021640 1 -- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17080039f0 msgr2=0x7f171c14ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1721021640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17080039f0 0x7f171c14ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1721021640 1 -- 172.21.15.131:0/1265568128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f171c15ed30 con 0x7f171c166130 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1721021640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c166130 0x7f171c166540 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f1714000c10 tx=0x7f1714001720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f17197fa640 1 -- 172.21.15.131:0/1265568128 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714011660 con 0x7f171c166130 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f17197fa640 1 -- 172.21.15.131:0/1265568128 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1714017070 con 0x7f171c166130 2024-09-17T02:52:56.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1722a84640 1 -- 172.21.15.131:0/1265568128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f171c06d790 con 0x7f171c166130 2024-09-17T02:52:56.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f17197fa640 1 -- 172.21.15.131:0/1265568128 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1714011bd0 con 0x7f171c166130 2024-09-17T02:52:56.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.293+0000 7f1722a84640 1 -- 172.21.15.131:0/1265568128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f171c165650 con 0x7f171c166130 2024-09-17T02:52:56.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.297+0000 7f1722a84640 1 -- 172.21.15.131:0/1265568128 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f171c15e7f0 con 0x7f171c166130 2024-09-17T02:52:56.292 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.297+0000 7f17197fa640 1 -- 172.21.15.131:0/1265568128 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f171401c460 con 0x7f171c166130 2024-09-17T02:52:56.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.297+0000 7f17197fa640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f16fc073230 0x7f16fc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.297+0000 7f17197fa640 1 -- 172.21.15.131:0/1265568128 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f17140950e0 con 0x7f171c166130 2024-09-17T02:52:56.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.297+0000 7f1720820640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f16fc073230 0x7f16fc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.293 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.297+0000 7f1720820640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f16fc073230 0x7f16fc075740 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f170c0ad000 tx=0x7f170c01a140 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:56.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.301+0000 7f17197fa640 1 -- 172.21.15.131:0/1265568128 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f171c06d790 con 0x7f171c166130 2024-09-17T02:52:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.457+0000 7f1722a84640 1 -- 172.21.15.131:0/1265568128 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f171c155d70 con 0x7f16fc073230 2024-09-17T02:52:56.452 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.457+0000 7f17197fa640 1 -- 172.21.15.131:0/1265568128 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f171c155d70 con 0x7f16fc073230 2024-09-17T02:52:56.453 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T02:52:56.455 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 -- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f16fc073230 msgr2=0x7f16fc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f16fc073230 0x7f16fc075740 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f170c0ad000 tx=0x7f170c01a140 comp rx=0 tx=0).stop 2024-09-17T02:52:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 -- 172.21.15.131:0/1265568128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c166130 msgr2=0x7f171c166540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c166130 0x7f171c166540 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f1714000c10 tx=0x7f1714001720 comp rx=0 tx=0).stop 2024-09-17T02:52:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 -- 172.21.15.131:0/1265568128 shutdown_connections 2024-09-17T02:52:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f16fc073230 0x7f16fc075740 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f171c166130 0x7f171c166540 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f171c06ce40 0x7f171c06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 --2- 172.21.15.131:0/1265568128 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f17080039f0 0x7f171c14ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 -- 172.21.15.131:0/1265568128 >> 172.21.15.131:0/1265568128 conn(0x7f171c06df50 msgr2=0x7f171c0d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 -- 172.21.15.131:0/1265568128 shutdown_connections 2024-09-17T02:52:56.456 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.461+0000 7f1722a84640 1 -- 172.21.15.131:0/1265568128 wait complete. 2024-09-17T02:52:56.474 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T02:52:56.475 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T02:52:56.632 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.637+0000 7fb99a370640 1 -- 172.21.15.131:0/3850446735 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb9940b65c0 msgr2=0x7fb9940d3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.637+0000 7fb99a370640 1 --2- 172.21.15.131:0/3850446735 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb9940b65c0 0x7fb9940d3300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009bc0 tx=0x7fb98c01cba0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.637+0000 7fb99a370640 1 -- 172.21.15.131:0/3850446735 shutdown_connections 2024-09-17T02:52:56.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.637+0000 7fb99a370640 1 --2- 172.21.15.131:0/3850446735 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb9940b65c0 0x7fb9940d3300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.637+0000 7fb99a370640 1 --2- 172.21.15.131:0/3850446735 >> v2:172.21.15.189:3300/0 conn(0x7fb99415e7a0 0x7fb994162210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.637+0000 7fb99a370640 1 -- 172.21.15.131:0/3850446735 >> 172.21.15.131:0/3850446735 conn(0x7fb9940d0eb0 msgr2=0x7fb9940d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:56.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.637+0000 7fb99a370640 1 -- 172.21.15.131:0/3850446735 shutdown_connections 2024-09-17T02:52:56.633 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.637+0000 7fb99a370640 1 -- 172.21.15.131:0/3850446735 wait complete. 2024-09-17T02:52:56.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99a370640 1 Processor -- start 2024-09-17T02:52:56.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99a370640 1 -- start start 2024-09-17T02:52:56.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99a370640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9940d2ef0 0x7fb9940c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99a370640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb99415e7a0 0x7fb9940cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99a370640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb99415d9c0 0x7fb99415bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99a370640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fb99416bfd0 con 0x7fb9940d2ef0 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99a370640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fb99416be50 con 0x7fb99415e7a0 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99a370640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fb99416c150 con 0x7fb99415d9c0 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb993fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9940d2ef0 0x7fb9940c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb993fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9940d2ef0 0x7fb9940c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45124/0 (socket says 172.21.15.131:45124) 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb993fff640 1 -- 172.21.15.131:0/1032906257 learned_addr learned my addr 172.21.15.131:0/1032906257 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb9937fe640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb99415e7a0 0x7fb9940cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99890d640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb99415d9c0 0x7fb99415bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb993fff640 1 -- 172.21.15.131:0/1032906257 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb99415d9c0 msgr2=0x7fb99415bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb993fff640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fb99415d9c0 0x7fb99415bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.637 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb993fff640 1 -- 172.21.15.131:0/1032906257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb99415e7a0 msgr2=0x7fb9940cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb993fff640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fb99415e7a0 0x7fb9940cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb993fff640 1 -- 172.21.15.131:0/1032906257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9940cb9b0 con 0x7fb9940d2ef0 2024-09-17T02:52:56.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb993fff640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9940d2ef0 0x7fb9940c9550 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fb98800e7a0 tx=0x7fb98800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:56.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb990ff9640 1 -- 172.21.15.131:0/1032906257 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98800ccc0 con 0x7fb9940d2ef0 2024-09-17T02:52:56.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99a370640 1 -- 172.21.15.131:0/1032906257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9940ca080 con 0x7fb9940d2ef0 2024-09-17T02:52:56.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb990ff9640 1 -- 172.21.15.131:0/1032906257 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb988010430 con 0x7fb9940d2ef0 2024-09-17T02:52:56.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb990ff9640 1 -- 172.21.15.131:0/1032906257 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988010860 con 0x7fb9940d2ef0 2024-09-17T02:52:56.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99a370640 1 -- 172.21.15.131:0/1032906257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb99415c590 con 0x7fb9940d2ef0 2024-09-17T02:52:56.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.641+0000 7fb99a370640 1 -- 172.21.15.131:0/1032906257 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb9940d3300 con 0x7fb9940d2ef0 2024-09-17T02:52:56.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.645+0000 7fb990ff9640 1 -- 172.21.15.131:0/1032906257 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fb988010a00 con 0x7fb9940d2ef0 2024-09-17T02:52:56.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.645+0000 7fb990ff9640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb980073200 0x7fb980075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.645+0000 7fb990ff9640 1 -- 172.21.15.131:0/1032906257 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7fb9880957c0 con 0x7fb9940d2ef0 2024-09-17T02:52:56.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.645+0000 7fb990ff9640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x7fb980079830 0x7fb98007bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.645+0000 7fb990ff9640 1 -- 172.21.15.131:0/1032906257 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb98801bec0 con 0x7fb980079830 2024-09-17T02:52:56.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.645+0000 7fb990ff9640 1 -- 172.21.15.131:0/1032906257 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7fb988095c50 con 0x7fb9940d2ef0 2024-09-17T02:52:56.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.645+0000 7fb9937fe640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb980073200 0x7fb980075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.645+0000 7fb99890d640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x7fb980079830 0x7fb98007bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.645+0000 7fb99890d640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x7fb980079830 0x7fb98007bcc0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:56.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.645+0000 7fb9937fe640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb980073200 0x7fb980075710 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb9941fb8e0 tx=0x7fb984009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:56.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.645+0000 7fb990ff9640 1 -- 172.21.15.131:0/1032906257 <== osd.2 v2:172.21.15.131:6800/163801004 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb98801bec0 con 0x7fb980079830 2024-09-17T02:52:56.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99a370640 1 -- 172.21.15.131:0/1032906257 --> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb9941616b0 con 0x7fb980079830 2024-09-17T02:52:56.655 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb990ff9640 1 -- 172.21.15.131:0/1032906257 <== osd.2 v2:172.21.15.131:6800/163801004 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb9941616b0 con 0x7fb980079830 2024-09-17T02:52:56.655 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:56.655 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T02:52:56.655 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:56.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99a370640 1 -- 172.21.15.131:0/1032906257 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x7fb980079830 msgr2=0x7fb98007bcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.656 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99a370640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.131:6800/163801004,v1:172.21.15.131:6801/163801004] conn(0x7fb980079830 0x7fb98007bcc0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99a370640 1 -- 172.21.15.131:0/1032906257 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb980073200 msgr2=0x7fb980075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99a370640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fb980073200 0x7fb980075710 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb9941fb8e0 tx=0x7fb984009040 comp rx=0 tx=0).stop 2024-09-17T02:52:56.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99a370640 1 -- 172.21.15.131:0/1032906257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9940d2ef0 msgr2=0x7fb9940c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99a370640 1 --2- 172.21.15.131:0/1032906257 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fb9940d2ef0 0x7fb9940c9550 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fb98800e7a0 tx=0x7fb98800ec60 comp rx=0 tx=0).stop 2024-09-17T02:52:56.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99890d640 1 -- 172.21.15.131:0/1032906257 reap_dead start 2024-09-17T02:52:56.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99a370640 1 -- 172.21.15.131:0/1032906257 shutdown_connections 2024-09-17T02:52:56.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99a370640 1 -- 172.21.15.131:0/1032906257 >> 172.21.15.131:0/1032906257 conn(0x7fb9940d0eb0 msgr2=0x7fb9940cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:56.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99a370640 1 -- 172.21.15.131:0/1032906257 shutdown_connections 2024-09-17T02:52:56.657 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.661+0000 7fb99a370640 1 -- 172.21.15.131:0/1032906257 wait complete. 2024-09-17T02:52:56.674 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T02:52:56.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.821+0000 7f29e404c640 1 -- 172.21.15.131:0/1320086158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc0cdb10 msgr2=0x7f29dc0cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.821+0000 7f29e404c640 1 --2- 172.21.15.131:0/1320086158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc0cdb10 0x7f29dc0cbd90 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f29cc009900 tx=0x7f29cc01c700 comp rx=0 tx=0).stop 2024-09-17T02:52:56.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.821+0000 7f29e404c640 1 -- 172.21.15.131:0/1320086158 shutdown_connections 2024-09-17T02:52:56.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.821+0000 7f29e404c640 1 --2- 172.21.15.131:0/1320086158 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc0cdb10 0x7f29dc0cbd90 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.821+0000 7f29e404c640 1 --2- 172.21.15.131:0/1320086158 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29dc15f960 0x7f29dc0ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.821+0000 7f29e404c640 1 -- 172.21.15.131:0/1320086158 >> 172.21.15.131:0/1320086158 conn(0x7f29dc0cff40 msgr2=0x7f29dc0d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:56.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.821+0000 7f29e404c640 1 -- 172.21.15.131:0/1320086158 shutdown_connections 2024-09-17T02:52:56.815 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.821+0000 7f29e404c640 1 -- 172.21.15.131:0/1320086158 wait complete. 2024-09-17T02:52:56.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.821+0000 7f29e404c640 1 Processor -- start 2024-09-17T02:52:56.816 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.821+0000 7f29e404c640 1 -- start start 2024-09-17T02:52:56.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e404c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc0cc2d0 0x7f29dc13d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.819 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e404c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29dc0cdb10 0x7f29dc0b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e404c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc15f960 0x7f29dc06f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e404c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc168cc0 con 0x7f29dc0cc2d0 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e404c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f29dc168b40 con 0x7f29dc15f960 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e404c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f29dc168e40 con 0x7f29dc0cdb10 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e1de8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc0cc2d0 0x7f29dc13d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e1de8640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc0cc2d0 0x7f29dc13d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45130/0 (socket says 172.21.15.131:45130) 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e1de8640 1 -- 172.21.15.131:0/444876164 learned_addr learned my addr 172.21.15.131:0/444876164 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e25e9640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc15f960 0x7f29dc06f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e15e7640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29dc0cdb10 0x7f29dc0b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e1de8640 1 -- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29dc0cdb10 msgr2=0x7f29dc0b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e1de8640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29dc0cdb10 0x7f29dc0b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e1de8640 1 -- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc15f960 msgr2=0x7f29dc06f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e1de8640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc15f960 0x7f29dc06f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e1de8640 1 -- 172.21.15.131:0/444876164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29dc1f6f30 con 0x7f29dc0cc2d0 2024-09-17T02:52:56.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e25e9640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc15f960 0x7f29dc06f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:56.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e1de8640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc0cc2d0 0x7f29dc13d8a0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c970 tx=0x7f29d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:56.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29d27fc640 1 -- 172.21.15.131:0/444876164 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8004220 con 0x7f29dc0cc2d0 2024-09-17T02:52:56.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29d27fc640 1 -- 172.21.15.131:0/444876164 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29d80079b0 con 0x7f29dc0cc2d0 2024-09-17T02:52:56.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29d27fc640 1 -- 172.21.15.131:0/444876164 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d80044b0 con 0x7f29dc0cc2d0 2024-09-17T02:52:56.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e404c640 1 -- 172.21.15.131:0/444876164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29dc1f7210 con 0x7f29dc0cc2d0 2024-09-17T02:52:56.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e404c640 1 -- 172.21.15.131:0/444876164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29dc1f77d0 con 0x7f29dc0cc2d0 2024-09-17T02:52:56.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e404c640 1 -- 172.21.15.131:0/444876164 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29dc15b1d0 con 0x7f29dc0cc2d0 2024-09-17T02:52:56.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29d27fc640 1 -- 172.21.15.131:0/444876164 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f29d8003600 con 0x7f29dc0cc2d0 2024-09-17T02:52:56.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29d27fc640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29c0073200 0x7f29c0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:56.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29d27fc640 1 -- 172.21.15.131:0/444876164 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f29d8094320 con 0x7f29dc0cc2d0 2024-09-17T02:52:56.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.825+0000 7f29e15e7640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29c0073200 0x7f29c0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:56.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.829+0000 7f29e15e7640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29c0073200 0x7f29c0075710 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0af000 tx=0x7f29cc01af40 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:56.825 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.829+0000 7f29d27fc640 1 -- 172.21.15.131:0/444876164 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f29d8014030 con 0x7f29dc0cc2d0 2024-09-17T02:52:56.984 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.989+0000 7f29e404c640 1 -- 172.21.15.131:0/444876164 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f29dc13b1a0 con 0x7f29c0073200 2024-09-17T02:52:56.985 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.989+0000 7f29d27fc640 1 -- 172.21.15.131:0/444876164 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f29cc0041c0 con 0x7f29c0073200 2024-09-17T02:52:56.985 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 -- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29c0073200 msgr2=0x7f29c0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29c0073200 0x7f29c0075710 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0af000 tx=0x7f29cc01af40 comp rx=0 tx=0).stop 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 -- 172.21.15.131:0/444876164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc0cc2d0 msgr2=0x7f29dc13d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc0cc2d0 0x7f29dc13d8a0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c970 tx=0x7f29d800ce30 comp rx=0 tx=0).stop 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 -- 172.21.15.131:0/444876164 shutdown_connections 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f29c0073200 0x7f29c0075710 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f29dc15f960 0x7f29dc06f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f29dc0cdb10 0x7f29dc0b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 --2- 172.21.15.131:0/444876164 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f29dc0cc2d0 0x7f29dc13d8a0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 -- 172.21.15.131:0/444876164 >> 172.21.15.131:0/444876164 conn(0x7f29dc0cff40 msgr2=0x7f29dc13f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 -- 172.21.15.131:0/444876164 shutdown_connections 2024-09-17T02:52:56.988 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:56.993+0000 7f29e404c640 1 -- 172.21.15.131:0/444876164 wait complete. 2024-09-17T02:52:57.006 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T02:52:57.006 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T02:52:57.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.165+0000 7f4739ce9640 1 -- 172.21.15.131:0/3073818535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47340d2ef0 msgr2=0x7f47340d3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.165+0000 7f4739ce9640 1 --2- 172.21.15.131:0/3073818535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47340d2ef0 0x7f47340d3300 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f471c009840 tx=0x7f471c01c820 comp rx=0 tx=0).stop 2024-09-17T02:52:57.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.165+0000 7f4739ce9640 1 -- 172.21.15.131:0/3073818535 shutdown_connections 2024-09-17T02:52:57.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.165+0000 7f4739ce9640 1 --2- 172.21.15.131:0/3073818535 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f47340b5eb0 0x7f47340b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.165+0000 7f4739ce9640 1 --2- 172.21.15.131:0/3073818535 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47340d2ef0 0x7f47340d3300 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.165+0000 7f4739ce9640 1 -- 172.21.15.131:0/3073818535 >> 172.21.15.131:0/3073818535 conn(0x7f47340d0eb0 msgr2=0x7f47340d12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:57.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.165+0000 7f4739ce9640 1 -- 172.21.15.131:0/3073818535 shutdown_connections 2024-09-17T02:52:57.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.165+0000 7f4739ce9640 1 -- 172.21.15.131:0/3073818535 wait complete. 2024-09-17T02:52:57.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f4739ce9640 1 Processor -- start 2024-09-17T02:52:57.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f4739ce9640 1 -- start start 2024-09-17T02:52:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f4739ce9640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340b5eb0 0x7f4734160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f4739ce9640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47340d2ef0 0x7f47341597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f4739ce9640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f473415e7a0 0x7f47340c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f4739ce9640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f473416c0c0 con 0x7f47340d2ef0 2024-09-17T02:52:57.167 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f4739ce9640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f473416bf40 con 0x7f47340b5eb0 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f4739ce9640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f473416c240 con 0x7f473415e7a0 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f472affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47340d2ef0 0x7f47341597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f472affd640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47340d2ef0 0x7f47341597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45148/0 (socket says 172.21.15.131:45148) 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f472affd640 1 -- 172.21.15.131:0/517380464 learned_addr learned my addr 172.21.15.131:0/517380464 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f4733fff640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f473415e7a0 0x7f47340c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f47337fe640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340b5eb0 0x7f4734160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f472affd640 1 -- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f473415e7a0 msgr2=0x7f47340c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f472affd640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f473415e7a0 0x7f47340c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f472affd640 1 -- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340b5eb0 msgr2=0x7f4734160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f472affd640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340b5eb0 0x7f4734160bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f472affd640 1 -- 172.21.15.131:0/517380464 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47341619c0 con 0x7f47340d2ef0 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f47337fe640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f47340b5eb0 0x7f4734160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:57.168 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.169+0000 7f4733fff640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f473415e7a0 0x7f47340c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:57.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f472affd640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47340d2ef0 0x7f47341597f0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f471c009790 tx=0x7f471c0a6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:57.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4730ff9640 1 -- 172.21.15.131:0/517380464 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c004040 con 0x7f47340d2ef0 2024-09-17T02:52:57.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4730ff9640 1 -- 172.21.15.131:0/517380464 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f471c0042d0 con 0x7f47340d2ef0 2024-09-17T02:52:57.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4739ce9640 1 -- 172.21.15.131:0/517380464 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47340cc190 con 0x7f47340d2ef0 2024-09-17T02:52:57.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4730ff9640 1 -- 172.21.15.131:0/517380464 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c003600 con 0x7f47340d2ef0 2024-09-17T02:52:57.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4739ce9640 1 -- 172.21.15.131:0/517380464 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4734159d30 con 0x7f47340d2ef0 2024-09-17T02:52:57.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4739ce9640 1 -- 172.21.15.131:0/517380464 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f47340cc850 con 0x7f47340d2ef0 2024-09-17T02:52:57.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4730ff9640 1 -- 172.21.15.131:0/517380464 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f471c0c1020 con 0x7f47340d2ef0 2024-09-17T02:52:57.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4730ff9640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4714073200 0x7f4714075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4730ff9640 1 -- 172.21.15.131:0/517380464 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f471c12ee00 con 0x7f47340d2ef0 2024-09-17T02:52:57.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4730ff9640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x7f4714079830 0x7f471407bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.170 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4730ff9640 1 -- 172.21.15.131:0/517380464 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f471c0bee50 con 0x7f4714079830 2024-09-17T02:52:57.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4730ff9640 1 -- 172.21.15.131:0/517380464 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7f471c131500 con 0x7f47340d2ef0 2024-09-17T02:52:57.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f47337fe640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4714073200 0x7f4714075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4733fff640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x7f4714079830 0x7f471407bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f4733fff640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x7f4714079830 0x7f471407bcc0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:57.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.173+0000 7f47337fe640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4714073200 0x7f4714075710 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4724001400 tx=0x7f4724009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:57.171 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.177+0000 7f4730ff9640 1 -- 172.21.15.131:0/517380464 <== osd.3 v2:172.21.15.131:6816/4047536388 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f471c0bee50 con 0x7f4714079830 2024-09-17T02:52:57.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4739ce9640 1 -- 172.21.15.131:0/517380464 --> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4734161730 con 0x7f4714079830 2024-09-17T02:52:57.183 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4730ff9640 1 -- 172.21.15.131:0/517380464 <== osd.3 v2:172.21.15.131:6816/4047536388 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4734161730 con 0x7f4714079830 2024-09-17T02:52:57.184 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:57.184 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T02:52:57.184 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:57.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4739ce9640 1 -- 172.21.15.131:0/517380464 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x7f4714079830 msgr2=0x7f471407bcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4739ce9640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.131:6816/4047536388,v1:172.21.15.131:6817/4047536388] conn(0x7f4714079830 0x7f471407bcc0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4739ce9640 1 -- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4714073200 msgr2=0x7f4714075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4739ce9640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4714073200 0x7f4714075710 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4724001400 tx=0x7f4724009040 comp rx=0 tx=0).stop 2024-09-17T02:52:57.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4739ce9640 1 -- 172.21.15.131:0/517380464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47340d2ef0 msgr2=0x7f47341597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4739ce9640 1 --2- 172.21.15.131:0/517380464 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f47340d2ef0 0x7f47341597f0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f471c009790 tx=0x7f471c0a6600 comp rx=0 tx=0).stop 2024-09-17T02:52:57.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4733fff640 1 -- 172.21.15.131:0/517380464 reap_dead start 2024-09-17T02:52:57.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4739ce9640 1 -- 172.21.15.131:0/517380464 shutdown_connections 2024-09-17T02:52:57.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4739ce9640 1 -- 172.21.15.131:0/517380464 >> 172.21.15.131:0/517380464 conn(0x7f47340d0eb0 msgr2=0x7f47340b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:57.185 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4739ce9640 1 -- 172.21.15.131:0/517380464 shutdown_connections 2024-09-17T02:52:57.186 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.189+0000 7f4739ce9640 1 -- 172.21.15.131:0/517380464 wait complete. 2024-09-17T02:52:57.204 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T02:52:57.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.353+0000 7fc5aec8b640 1 -- 172.21.15.131:0/2318077232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc590004d50 msgr2=0x7fc5900051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.353+0000 7fc5aec8b640 1 --2- 172.21.15.131:0/2318077232 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc590004d50 0x7fc5900051a0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009e00 tx=0x7fc5a001d150 comp rx=0 tx=0).stop 2024-09-17T02:52:57.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.353+0000 7fc5ad228640 1 -- 172.21.15.131:0/2318077232 reap_dead start 2024-09-17T02:52:57.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.353+0000 7fc5aec8b640 1 -- 172.21.15.131:0/2318077232 shutdown_connections 2024-09-17T02:52:57.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.353+0000 7fc5aec8b640 1 -- 172.21.15.131:0/2318077232 >> 172.21.15.131:0/2318077232 conn(0x7fc5a806d440 msgr2=0x7fc5a80cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.353+0000 7fc5aec8b640 1 -- 172.21.15.131:0/2318077232 shutdown_connections 2024-09-17T02:52:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.353+0000 7fc5aec8b640 1 -- 172.21.15.131:0/2318077232 wait complete. 2024-09-17T02:52:57.348 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.353+0000 7fc5aec8b640 1 Processor -- start 2024-09-17T02:52:57.349 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.353+0000 7fc5aec8b640 1 -- start start 2024-09-17T02:52:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aec8b640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a81f7780 0x7fc5a81463e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aec8b640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a81f7b90 0x7fc5a8146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aec8b640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5a80c68d0 0x7fc5a80c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aec8b640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a80d00d0 con 0x7fc5a81f7b90 2024-09-17T02:52:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aec8b640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a80d1d30 con 0x7fc5a81f7780 2024-09-17T02:52:57.351 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aec8b640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a80d0250 con 0x7fc5a80c68d0 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a81f7b90 0x7fc5a8146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a81f7b90 0x7fc5a8146920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45156/0 (socket says 172.21.15.131:45156) 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a7fff640 1 -- 172.21.15.131:0/434387769 learned_addr learned my addr 172.21.15.131:0/434387769 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aca27640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a81f7780 0x7fc5a81463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5ad228640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5a80c68d0 0x7fc5a80c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a7fff640 1 -- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5a80c68d0 msgr2=0x7fc5a80c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a7fff640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5a80c68d0 0x7fc5a80c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a7fff640 1 -- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a81f7780 msgr2=0x7fc5a81463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a7fff640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a81f7780 0x7fc5a81463e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a7fff640 1 -- 172.21.15.131:0/434387769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a81f5010 con 0x7fc5a81f7b90 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aca27640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a81f7780 0x7fc5a81463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a7fff640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a81f7b90 0x7fc5a8146920 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e990 tx=0x7fc59c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:57.352 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a57fa640 1 -- 172.21.15.131:0/434387769 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00cbe0 con 0x7fc5a81f7b90 2024-09-17T02:52:57.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a57fa640 1 -- 172.21.15.131:0/434387769 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc59c005020 con 0x7fc5a81f7b90 2024-09-17T02:52:57.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aec8b640 1 -- 172.21.15.131:0/434387769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a80c7670 con 0x7fc5a81f7b90 2024-09-17T02:52:57.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a57fa640 1 -- 172.21.15.131:0/434387769 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c005450 con 0x7fc5a81f7b90 2024-09-17T02:52:57.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aec8b640 1 -- 172.21.15.131:0/434387769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a81f5840 con 0x7fc5a81f7b90 2024-09-17T02:52:57.353 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aec8b640 1 -- 172.21.15.131:0/434387769 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a80b3c80 con 0x7fc5a81f7b90 2024-09-17T02:52:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a57fa640 1 -- 172.21.15.131:0/434387769 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fc59c0047b0 con 0x7fc5a81f7b90 2024-09-17T02:52:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a57fa640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc594073200 0x7fc594075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5a57fa640 1 -- 172.21.15.131:0/434387769 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7fc59c094b80 con 0x7fc5a81f7b90 2024-09-17T02:52:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.357+0000 7fc5aca27640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc594073200 0x7fc594075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.354 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.361+0000 7fc5aca27640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc594073200 0x7fc594075710 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc598006740 tx=0x7fc598001200 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:57.357 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.361+0000 7fc5a57fa640 1 -- 172.21.15.131:0/434387769 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fc59c016030 con 0x7fc5a81f7b90 2024-09-17T02:52:57.516 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.521+0000 7fc5aec8b640 1 -- 172.21.15.131:0/434387769 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fc5a80b8180 con 0x7fc594073200 2024-09-17T02:52:57.517 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.521+0000 7fc5a57fa640 1 -- 172.21.15.131:0/434387769 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc5a80b8180 con 0x7fc594073200 2024-09-17T02:52:57.517 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T02:52:57.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 -- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc594073200 msgr2=0x7fc594075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc594073200 0x7fc594075710 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fc598006740 tx=0x7fc598001200 comp rx=0 tx=0).stop 2024-09-17T02:52:57.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 -- 172.21.15.131:0/434387769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a81f7b90 msgr2=0x7fc5a8146920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.519 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a81f7b90 0x7fc5a8146920 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e990 tx=0x7fc59c00ee50 comp rx=0 tx=0).stop 2024-09-17T02:52:57.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 -- 172.21.15.131:0/434387769 shutdown_connections 2024-09-17T02:52:57.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fc594073200 0x7fc594075710 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.520 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fc5a80c68d0 0x7fc5a80c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fc5a81f7b90 0x7fc5a8146920 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 --2- 172.21.15.131:0/434387769 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fc5a81f7780 0x7fc5a81463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 -- 172.21.15.131:0/434387769 >> 172.21.15.131:0/434387769 conn(0x7fc5a806d440 msgr2=0x7fc5a8068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:57.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 -- 172.21.15.131:0/434387769 shutdown_connections 2024-09-17T02:52:57.521 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.525+0000 7fc5aec8b640 1 -- 172.21.15.131:0/434387769 wait complete. 2024-09-17T02:52:57.538 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T02:52:57.538 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T02:52:57.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.693+0000 7fbaa0aef640 1 -- 172.21.15.131:0/540298441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c0d09f0 msgr2=0x7fba9c0cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.693+0000 7fbaa0aef640 1 --2- 172.21.15.131:0/540298441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c0d09f0 0x7fba9c0cca30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fba9c15dc70 tx=0x7fba9401c930 comp rx=0 tx=0).stop 2024-09-17T02:52:57.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.693+0000 7fbaa0aef640 1 -- 172.21.15.131:0/540298441 shutdown_connections 2024-09-17T02:52:57.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.693+0000 7fbaa0aef640 1 --2- 172.21.15.131:0/540298441 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c0d09f0 0x7fba9c0cca30 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.693+0000 7fbaa0aef640 1 --2- 172.21.15.131:0/540298441 >> v2:172.21.15.189:3300/0 conn(0x7fba9c06f540 0x7fba9c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.688 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.693+0000 7fbaa0aef640 1 -- 172.21.15.131:0/540298441 >> 172.21.15.131:0/540298441 conn(0x7fba9c06d010 msgr2=0x7fba9c06d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:57.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.693+0000 7fbaa0aef640 1 -- 172.21.15.131:0/540298441 shutdown_connections 2024-09-17T02:52:57.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.693+0000 7fbaa0aef640 1 -- 172.21.15.131:0/540298441 wait complete. 2024-09-17T02:52:57.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.693+0000 7fbaa0aef640 1 Processor -- start 2024-09-17T02:52:57.689 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.693+0000 7fbaa0aef640 1 -- start start 2024-09-17T02:52:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fbaa0aef640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba9c06f540 0x7fba9c13b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fbaa0aef640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c06f950 0x7fba9c155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.692 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fbaa0aef640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba9c15edf0 0x7fba9c15f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fbaa0aef640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c16a0c0 con 0x7fba9c06f950 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fbaa0aef640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c169f40 con 0x7fba9c06f540 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fbaa0aef640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fba9c16a240 con 0x7fba9c15edf0 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba93fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c06f950 0x7fba9c155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba93fff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c06f950 0x7fba9c155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45170/0 (socket says 172.21.15.131:45170) 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba93fff640 1 -- 172.21.15.131:0/1208833793 learned_addr learned my addr 172.21.15.131:0/1208833793 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba9a59c640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba9c06f540 0x7fba9c13b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba93fff640 1 -- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba9c15edf0 msgr2=0x7fba9c15f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba93fff640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fba9c15edf0 0x7fba9c15f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba93fff640 1 -- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba9c06f540 msgr2=0x7fba9c13b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba93fff640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fba9c06f540 0x7fba9c13b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba93fff640 1 -- 172.21.15.131:0/1208833793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c15f740 con 0x7fba9c06f950 2024-09-17T02:52:57.693 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba93fff640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c06f950 0x7fba9c155e70 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fba8400c530 tx=0x7fba8400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba92ffd640 1 -- 172.21.15.131:0/1208833793 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84016020 con 0x7fba9c06f950 2024-09-17T02:52:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba92ffd640 1 -- 172.21.15.131:0/1208833793 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba84012940 con 0x7fba9c06f950 2024-09-17T02:52:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fba92ffd640 1 -- 172.21.15.131:0/1208833793 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba84012d90 con 0x7fba9c06f950 2024-09-17T02:52:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fbaa0aef640 1 -- 172.21.15.131:0/1208833793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9c13ad50 con 0x7fba9c06f950 2024-09-17T02:52:57.694 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fbaa0aef640 1 -- 172.21.15.131:0/1208833793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9c0b14b0 con 0x7fba9c06f950 2024-09-17T02:52:57.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.697+0000 7fbaa0aef640 1 -- 172.21.15.131:0/1208833793 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba9c15cb00 con 0x7fba9c06f950 2024-09-17T02:52:57.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.701+0000 7fba92ffd640 1 -- 172.21.15.131:0/1208833793 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fba84019020 con 0x7fba9c06f950 2024-09-17T02:52:57.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.701+0000 7fba92ffd640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba7c073230 0x7fba7c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.701+0000 7fba92ffd640 1 -- 172.21.15.131:0/1208833793 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7fba84094680 con 0x7fba9c06f950 2024-09-17T02:52:57.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.701+0000 7fba92ffd640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x7fba7c079860 0x7fba7c07bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.701+0000 7fba92ffd640 1 -- 172.21.15.131:0/1208833793 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba8401ce50 con 0x7fba7c079860 2024-09-17T02:52:57.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.701+0000 7fba92ffd640 1 -- 172.21.15.131:0/1208833793 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7fba84094b10 con 0x7fba9c06f950 2024-09-17T02:52:57.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.701+0000 7fba9a59c640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba7c073230 0x7fba7c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.695 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.701+0000 7fba9ad9d640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x7fba7c079860 0x7fba7c07bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.701+0000 7fba9ad9d640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x7fba7c079860 0x7fba7c07bcf0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:57.696 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.701+0000 7fba9a59c640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba7c073230 0x7fba7c075740 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fba8c005d10 tx=0x7fba8c005c80 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:57.698 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.705+0000 7fba92ffd640 1 -- 172.21.15.131:0/1208833793 <== osd.4 v2:172.21.15.189:6818/1965940492 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba8401ce50 con 0x7fba7c079860 2024-09-17T02:52:57.710 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fbaa0aef640 1 -- 172.21.15.131:0/1208833793 --> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fba9c1563b0 con 0x7fba7c079860 2024-09-17T02:52:57.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fba92ffd640 1 -- 172.21.15.131:0/1208833793 <== osd.4 v2:172.21.15.189:6818/1965940492 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fba9c1563b0 con 0x7fba7c079860 2024-09-17T02:52:57.711 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:57.711 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T02:52:57.711 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fbaa0aef640 1 -- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x7fba7c079860 msgr2=0x7fba7c07bcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fbaa0aef640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:6818/1965940492,v1:172.21.15.189:6819/1965940492] conn(0x7fba7c079860 0x7fba7c07bcf0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fbaa0aef640 1 -- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba7c073230 msgr2=0x7fba7c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fbaa0aef640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fba7c073230 0x7fba7c075740 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fba8c005d10 tx=0x7fba8c005c80 comp rx=0 tx=0).stop 2024-09-17T02:52:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fbaa0aef640 1 -- 172.21.15.131:0/1208833793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c06f950 msgr2=0x7fba9c155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fbaa0aef640 1 --2- 172.21.15.131:0/1208833793 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fba9c06f950 0x7fba9c155e70 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7fba8400c530 tx=0x7fba8400c9f0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fba9ad9d640 1 -- 172.21.15.131:0/1208833793 reap_dead start 2024-09-17T02:52:57.712 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fbaa0aef640 1 -- 172.21.15.131:0/1208833793 shutdown_connections 2024-09-17T02:52:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fbaa0aef640 1 -- 172.21.15.131:0/1208833793 >> 172.21.15.131:0/1208833793 conn(0x7fba9c06d010 msgr2=0x7fba9c15c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fbaa0aef640 1 -- 172.21.15.131:0/1208833793 shutdown_connections 2024-09-17T02:52:57.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.717+0000 7fbaa0aef640 1 -- 172.21.15.131:0/1208833793 wait complete. 2024-09-17T02:52:57.730 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T02:52:57.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.869+0000 7f3787508640 1 -- 172.21.15.131:0/3171673968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f378015ef80 msgr2=0x7f378006f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.869+0000 7f3787508640 1 --2- 172.21.15.131:0/3171673968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f378015ef80 0x7f378006f780 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f3780072f60 tx=0x7f377c01c8c0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.869+0000 7f3787508640 1 -- 172.21.15.131:0/3171673968 shutdown_connections 2024-09-17T02:52:57.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.869+0000 7f3787508640 1 --2- 172.21.15.131:0/3171673968 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f37800cdb10 0x7f37800cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.865 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.869+0000 7f3787508640 1 --2- 172.21.15.131:0/3171673968 >> v2:172.21.15.189:3300/0 conn(0x7f378015f390 0x7f378006fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.869+0000 7f3787508640 1 --2- 172.21.15.131:0/3171673968 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f378015ef80 0x7f378006f780 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.869+0000 7f3787508640 1 -- 172.21.15.131:0/3171673968 >> 172.21.15.131:0/3171673968 conn(0x7f37800cff40 msgr2=0x7f37800d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:57.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.869+0000 7f3787508640 1 -- 172.21.15.131:0/3171673968 shutdown_connections 2024-09-17T02:52:57.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.869+0000 7f3787508640 1 -- 172.21.15.131:0/3171673968 wait complete. 2024-09-17T02:52:57.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3787508640 1 Processor -- start 2024-09-17T02:52:57.866 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3787508640 1 -- start start 2024-09-17T02:52:57.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3787508640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37800cdb10 0x7f3780139010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3787508640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f378015ef80 0x7f378013b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3787508640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f378015f390 0x7f378013be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.869 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3787508640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f3780074640 con 0x7f378015ef80 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3787508640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f37800744c0 con 0x7f37800cdb10 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3787508640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f37800747c0 con 0x7f378015f390 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3784aa3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f378015ef80 0x7f378013b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3784aa3640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f378015ef80 0x7f378013b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45188/0 (socket says 172.21.15.131:45188) 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3784aa3640 1 -- 172.21.15.131:0/2478539026 learned_addr learned my addr 172.21.15.131:0/2478539026 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f37852a4640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37800cdb10 0x7f3780139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3785aa5640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f378015f390 0x7f378013be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3784aa3640 1 -- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f378015f390 msgr2=0x7f378013be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3784aa3640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f378015f390 0x7f378013be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3784aa3640 1 -- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37800cdb10 msgr2=0x7f3780139010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3784aa3640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37800cdb10 0x7f3780139010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3784aa3640 1 -- 172.21.15.131:0/2478539026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37800b5d70 con 0x7f378015ef80 2024-09-17T02:52:57.870 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3784aa3640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f378015ef80 0x7f378013b910 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f377000c970 tx=0x7f377000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:57.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3775ffb640 1 -- 172.21.15.131:0/2478539026 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770014070 con 0x7f378015ef80 2024-09-17T02:52:57.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3775ffb640 1 -- 172.21.15.131:0/2478539026 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37700077a0 con 0x7f378015ef80 2024-09-17T02:52:57.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3787508640 1 -- 172.21.15.131:0/2478539026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37800b4fd0 con 0x7f378015ef80 2024-09-17T02:52:57.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3775ffb640 1 -- 172.21.15.131:0/2478539026 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770007bf0 con 0x7f378015ef80 2024-09-17T02:52:57.871 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.873+0000 7f3787508640 1 -- 172.21.15.131:0/2478539026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37800b5570 con 0x7f378015ef80 2024-09-17T02:52:57.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.877+0000 7f3787508640 1 -- 172.21.15.131:0/2478539026 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37800c5120 con 0x7f378015ef80 2024-09-17T02:52:57.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.877+0000 7f3775ffb640 1 -- 172.21.15.131:0/2478539026 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f3770017070 con 0x7f378015ef80 2024-09-17T02:52:57.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.877+0000 7f3775ffb640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3764073230 0x7f3764075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:57.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.877+0000 7f3775ffb640 1 -- 172.21.15.131:0/2478539026 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f37700943f0 con 0x7f378015ef80 2024-09-17T02:52:57.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.877+0000 7f37852a4640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3764073230 0x7f3764075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:57.872 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.877+0000 7f37852a4640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3764073230 0x7f3764075740 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f377c0af000 tx=0x7f377c01b100 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:57.875 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:57.881+0000 7f3775ffb640 1 -- 172.21.15.131:0/2478539026 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f377005f4c0 con 0x7f378015ef80 2024-09-17T02:52:58.033 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.037+0000 7f3787508640 1 -- 172.21.15.131:0/2478539026 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f3780074cf0 con 0x7f3764073230 2024-09-17T02:52:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3775ffb640 1 -- 172.21.15.131:0/2478539026 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3780074cf0 con 0x7f3764073230 2024-09-17T02:52:58.034 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T02:52:58.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 -- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3764073230 msgr2=0x7f3764075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3764073230 0x7f3764075740 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f377c0af000 tx=0x7f377c01b100 comp rx=0 tx=0).stop 2024-09-17T02:52:58.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 -- 172.21.15.131:0/2478539026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f378015ef80 msgr2=0x7f378013b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f378015ef80 0x7f378013b910 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f377000c970 tx=0x7f377000ce30 comp rx=0 tx=0).stop 2024-09-17T02:52:58.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 -- 172.21.15.131:0/2478539026 shutdown_connections 2024-09-17T02:52:58.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f3764073230 0x7f3764075740 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.036 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f378015f390 0x7f378013be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f378015ef80 0x7f378013b910 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 --2- 172.21.15.131:0/2478539026 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f37800cdb10 0x7f3780139010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 -- 172.21.15.131:0/2478539026 >> 172.21.15.131:0/2478539026 conn(0x7f37800cff40 msgr2=0x7f378006eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 -- 172.21.15.131:0/2478539026 shutdown_connections 2024-09-17T02:52:58.037 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.041+0000 7f3787508640 1 -- 172.21.15.131:0/2478539026 wait complete. 2024-09-17T02:52:58.055 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T02:52:58.055 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T02:52:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.217+0000 7fcb10224640 1 -- 172.21.15.131:0/1939679555 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb080d0950 msgr2=0x7fcb08072d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.217+0000 7fcb10224640 1 --2- 172.21.15.131:0/1939679555 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb080d0950 0x7fcb08072d70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009c80 tx=0x7fcb0001cce0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.217+0000 7fcb10224640 1 -- 172.21.15.131:0/1939679555 shutdown_connections 2024-09-17T02:52:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.217+0000 7fcb10224640 1 --2- 172.21.15.131:0/1939679555 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb080d0950 0x7fcb08072d70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.217+0000 7fcb10224640 1 --2- 172.21.15.131:0/1939679555 >> v2:172.21.15.189:3300/0 conn(0x7fcb080b3e80 0x7fcb080d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.212 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.217+0000 7fcb10224640 1 -- 172.21.15.131:0/1939679555 >> 172.21.15.131:0/1939679555 conn(0x7fcb0806d010 msgr2=0x7fcb0806d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:58.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.217+0000 7fcb10224640 1 -- 172.21.15.131:0/1939679555 shutdown_connections 2024-09-17T02:52:58.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.217+0000 7fcb10224640 1 -- 172.21.15.131:0/1939679555 wait complete. 2024-09-17T02:52:58.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.217+0000 7fcb10224640 1 Processor -- start 2024-09-17T02:52:58.213 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.217+0000 7fcb10224640 1 -- start start 2024-09-17T02:52:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb10224640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb080b3a70 0x7fcb0806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb10224640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb080b3e80 0x7fcb0806f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb10224640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb080d0950 0x7fcb081fa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb10224640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fcb08168bf0 con 0x7fcb080b3e80 2024-09-17T02:52:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb10224640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0815c320 con 0x7fcb080d0950 2024-09-17T02:52:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb10224640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fcb08168d70 con 0x7fcb080b3a70 2024-09-17T02:52:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb0d7bf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb080b3e80 0x7fcb0806f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb0d7bf640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb080b3e80 0x7fcb0806f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45190/0 (socket says 172.21.15.131:45190) 2024-09-17T02:52:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb0d7bf640 1 -- 172.21.15.131:0/1812643074 learned_addr learned my addr 172.21.15.131:0/1812643074 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:58.216 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb0e7c1640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb080d0950 0x7fcb081fa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb0dfc0640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb080b3a70 0x7fcb0806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb0d7bf640 1 -- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb080b3a70 msgr2=0x7fcb0806ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb0d7bf640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fcb080b3a70 0x7fcb0806ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb0d7bf640 1 -- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb080d0950 msgr2=0x7fcb081fa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb0d7bf640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fcb080d0950 0x7fcb081fa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb0d7bf640 1 -- 172.21.15.131:0/1812643074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb080510a0 con 0x7fcb080b3e80 2024-09-17T02:52:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb0d7bf640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb080b3e80 0x7fcb0806f150 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800e7f0 tx=0x7fcaf800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcafe7fc640 1 -- 172.21.15.131:0/1812643074 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf800ccc0 con 0x7fcb080b3e80 2024-09-17T02:52:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcafe7fc640 1 -- 172.21.15.131:0/1812643074 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaf800ce80 con 0x7fcb080b3e80 2024-09-17T02:52:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb10224640 1 -- 172.21.15.131:0/1812643074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb080cc900 con 0x7fcb080b3e80 2024-09-17T02:52:58.217 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcafe7fc640 1 -- 172.21.15.131:0/1812643074 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8005280 con 0x7fcb080b3e80 2024-09-17T02:52:58.218 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb10224640 1 -- 172.21.15.131:0/1812643074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0815ff60 con 0x7fcb080b3e80 2024-09-17T02:52:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcb10224640 1 -- 172.21.15.131:0/1812643074 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcb080d0410 con 0x7fcb080b3e80 2024-09-17T02:52:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcafe7fc640 1 -- 172.21.15.131:0/1812643074 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fcaf8005420 con 0x7fcb080b3e80 2024-09-17T02:52:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.221+0000 7fcafe7fc640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcaec073200 0x7fcaec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.225+0000 7fcafe7fc640 1 -- 172.21.15.131:0/1812643074 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7fcaf8095110 con 0x7fcb080b3e80 2024-09-17T02:52:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.225+0000 7fcafe7fc640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x7fcaec079830 0x7fcaec07bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.219 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.225+0000 7fcafe7fc640 1 -- 172.21.15.131:0/1812643074 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcaf8010e50 con 0x7fcaec079830 2024-09-17T02:52:58.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.225+0000 7fcafe7fc640 1 -- 172.21.15.131:0/1812643074 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7fcaf80955a0 con 0x7fcb080b3e80 2024-09-17T02:52:58.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.225+0000 7fcb0dfc0640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcaec073200 0x7fcaec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.225+0000 7fcb0e7c1640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x7fcaec079830 0x7fcaec07bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.225+0000 7fcb0e7c1640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x7fcaec079830 0x7fcaec07bcc0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:58.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.225+0000 7fcb0dfc0640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcaec073200 0x7fcaec075710 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcb04001650 tx=0x7fcb04009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:58.220 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.225+0000 7fcafe7fc640 1 -- 172.21.15.131:0/1812643074 <== osd.5 v2:172.21.15.189:6826/3175183672 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcaf8010e50 con 0x7fcaec079830 2024-09-17T02:52:58.233 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb10224640 1 -- 172.21.15.131:0/1812643074 --> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcb081fae90 con 0x7fcaec079830 2024-09-17T02:52:58.236 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcafe7fc640 1 -- 172.21.15.131:0/1812643074 <== osd.5 v2:172.21.15.189:6826/3175183672 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcb081fae90 con 0x7fcaec079830 2024-09-17T02:52:58.237 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:58.237 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T02:52:58.237 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:58.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb10224640 1 -- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x7fcaec079830 msgr2=0x7fcaec07bcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb10224640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:6826/3175183672,v1:172.21.15.189:6827/3175183672] conn(0x7fcaec079830 0x7fcaec07bcc0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb10224640 1 -- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcaec073200 msgr2=0x7fcaec075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb10224640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fcaec073200 0x7fcaec075710 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcb04001650 tx=0x7fcb04009040 comp rx=0 tx=0).stop 2024-09-17T02:52:58.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb10224640 1 -- 172.21.15.131:0/1812643074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb080b3e80 msgr2=0x7fcb0806f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb10224640 1 --2- 172.21.15.131:0/1812643074 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fcb080b3e80 0x7fcb0806f150 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800e7f0 tx=0x7fcaf800ecb0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb0e7c1640 1 -- 172.21.15.131:0/1812643074 reap_dead start 2024-09-17T02:52:58.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb10224640 1 -- 172.21.15.131:0/1812643074 shutdown_connections 2024-09-17T02:52:58.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb10224640 1 -- 172.21.15.131:0/1812643074 >> 172.21.15.131:0/1812643074 conn(0x7fcb0806d010 msgr2=0x7fcb0815a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:58.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb10224640 1 -- 172.21.15.131:0/1812643074 shutdown_connections 2024-09-17T02:52:58.239 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.241+0000 7fcb10224640 1 -- 172.21.15.131:0/1812643074 wait complete. 2024-09-17T02:52:58.254 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T02:52:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.409+0000 7faf76279640 1 -- 172.21.15.131:0/1383685427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf5c003a60 msgr2=0x7faf5c003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.409+0000 7faf76279640 1 --2- 172.21.15.131:0/1383685427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf5c003a60 0x7faf5c003ed0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7faf600016f0 tx=0x7faf6001c7b0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.409+0000 7faf76279640 1 -- 172.21.15.131:0/1383685427 shutdown_connections 2024-09-17T02:52:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.409+0000 7faf76279640 1 --2- 172.21.15.131:0/1383685427 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf5c003a60 0x7faf5c003ed0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.409+0000 7faf76279640 1 -- 172.21.15.131:0/1383685427 >> 172.21.15.131:0/1383685427 conn(0x7faf700cff40 msgr2=0x7faf700d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:58.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.409+0000 7faf76279640 1 -- 172.21.15.131:0/1383685427 shutdown_connections 2024-09-17T02:52:58.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.409+0000 7faf76279640 1 -- 172.21.15.131:0/1383685427 wait complete. 2024-09-17T02:52:58.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.409+0000 7faf76279640 1 Processor -- start 2024-09-17T02:52:58.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.409+0000 7faf76279640 1 -- start start 2024-09-17T02:52:58.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf76279640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf5c003a60 0x7faf701e9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf76279640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf70142d10 0x7faf701fd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf76279640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf701f9790 0x7faf701f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf76279640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7faf700ce030 con 0x7faf70142d10 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf76279640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7faf700cdeb0 con 0x7faf5c003a60 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf76279640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7faf700ce1b0 con 0x7faf701f9790 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf70142d10 0x7faf701fd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf70142d10 0x7faf701fd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45208/0 (socket says 172.21.15.131:45208) 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6f7fe640 1 -- 172.21.15.131:0/134422545 learned_addr learned my addr 172.21.15.131:0/134422545 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6ffff640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf5c003a60 0x7faf701e9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf74816640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf701f9790 0x7faf701f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6f7fe640 1 -- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf701f9790 msgr2=0x7faf701f9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6f7fe640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf701f9790 0x7faf701f9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6f7fe640 1 -- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf5c003a60 msgr2=0x7faf701e9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6f7fe640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf5c003a60 0x7faf701e9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6f7fe640 1 -- 172.21.15.131:0/134422545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf701fd610 con 0x7faf70142d10 2024-09-17T02:52:58.408 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf74816640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf701f9790 0x7faf701f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6f7fe640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf70142d10 0x7faf701fd0d0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e6b0 tx=0x7faf5800eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6cff9640 1 -- 172.21.15.131:0/134422545 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58004220 con 0x7faf70142d10 2024-09-17T02:52:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6cff9640 1 -- 172.21.15.131:0/134422545 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf58005020 con 0x7faf70142d10 2024-09-17T02:52:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf76279640 1 -- 172.21.15.131:0/134422545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf70143120 con 0x7faf70142d10 2024-09-17T02:52:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6cff9640 1 -- 172.21.15.131:0/134422545 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58005470 con 0x7faf70142d10 2024-09-17T02:52:58.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf76279640 1 -- 172.21.15.131:0/134422545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf701eb650 con 0x7faf70142d10 2024-09-17T02:52:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf76279640 1 -- 172.21.15.131:0/134422545 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf701eb0e0 con 0x7faf70142d10 2024-09-17T02:52:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6cff9640 1 -- 172.21.15.131:0/134422545 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7faf5800c850 con 0x7faf70142d10 2024-09-17T02:52:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6cff9640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf50073200 0x7faf50075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.413+0000 7faf6cff9640 1 -- 172.21.15.131:0/134422545 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7faf58094700 con 0x7faf70142d10 2024-09-17T02:52:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.417+0000 7faf6ffff640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf50073200 0x7faf50075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.410 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.417+0000 7faf6ffff640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf50073200 0x7faf50075710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faf600016c0 tx=0x7faf600a8040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:58.414 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.421+0000 7faf6cff9640 1 -- 172.21.15.131:0/134422545 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7faf701fd610 con 0x7faf70142d10 2024-09-17T02:52:58.586 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf76279640 1 -- 172.21.15.131:0/134422545 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7faf7015f040 con 0x7faf50073200 2024-09-17T02:52:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf6cff9640 1 -- 172.21.15.131:0/134422545 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7faf7015f040 con 0x7faf50073200 2024-09-17T02:52:58.587 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T02:52:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf76279640 1 -- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf50073200 msgr2=0x7faf50075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.590 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf76279640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf50073200 0x7faf50075710 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faf600016c0 tx=0x7faf600a8040 comp rx=0 tx=0).stop 2024-09-17T02:52:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf76279640 1 -- 172.21.15.131:0/134422545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf70142d10 msgr2=0x7faf701fd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf76279640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf70142d10 0x7faf701fd0d0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7faf5800e6b0 tx=0x7faf5800eb70 comp rx=0 tx=0).stop 2024-09-17T02:52:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf76279640 1 -- 172.21.15.131:0/134422545 shutdown_connections 2024-09-17T02:52:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf76279640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7faf50073200 0x7faf50075710 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf76279640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7faf701f9790 0x7faf701f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf76279640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7faf70142d10 0x7faf701fd0d0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf76279640 1 --2- 172.21.15.131:0/134422545 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7faf5c003a60 0x7faf701e9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.593+0000 7faf76279640 1 -- 172.21.15.131:0/134422545 >> 172.21.15.131:0/134422545 conn(0x7faf700cff40 msgr2=0x7faf700b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.597+0000 7faf76279640 1 -- 172.21.15.131:0/134422545 shutdown_connections 2024-09-17T02:52:58.592 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.597+0000 7faf76279640 1 -- 172.21.15.131:0/134422545 wait complete. 2024-09-17T02:52:58.610 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T02:52:58.610 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T02:52:58.769 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.773+0000 7ffa26a6c640 1 -- 172.21.15.131:0/2467001409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa200d0a90 msgr2=0x7ffa20160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.773+0000 7ffa26a6c640 1 --2- 172.21.15.131:0/2467001409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa200d0a90 0x7ffa20160080 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7ffa200ccef0 tx=0x7ffa1401c830 comp rx=0 tx=0).stop 2024-09-17T02:52:58.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.773+0000 7ffa26a6c640 1 -- 172.21.15.131:0/2467001409 shutdown_connections 2024-09-17T02:52:58.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.773+0000 7ffa26a6c640 1 --2- 172.21.15.131:0/2467001409 >> v2:172.21.15.189:3300/0 conn(0x7ffa201605c0 0x7ffa2006ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.773+0000 7ffa26a6c640 1 --2- 172.21.15.131:0/2467001409 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa200d0a90 0x7ffa20160080 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.773+0000 7ffa26a6c640 1 -- 172.21.15.131:0/2467001409 >> 172.21.15.131:0/2467001409 conn(0x7ffa2006d010 msgr2=0x7ffa2006d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:58.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.773+0000 7ffa26a6c640 1 -- 172.21.15.131:0/2467001409 shutdown_connections 2024-09-17T02:52:58.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.773+0000 7ffa26a6c640 1 -- 172.21.15.131:0/2467001409 wait complete. 2024-09-17T02:52:58.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa26a6c640 1 Processor -- start 2024-09-17T02:52:58.771 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa26a6c640 1 -- start start 2024-09-17T02:52:58.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa26a6c640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa200cd5b0 0x7ffa2014df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa26a6c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa201605c0 0x7ffa20157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa26a6c640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffa2014c150 0x7ffa2014c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa26a6c640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20169e70 con 0x7ffa201605c0 2024-09-17T02:52:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa26a6c640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20169cf0 con 0x7ffa200cd5b0 2024-09-17T02:52:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa26a6c640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ffa20169ff0 con 0x7ffa2014c150 2024-09-17T02:52:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa201605c0 0x7ffa20157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa201605c0 0x7ffa20157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45218/0 (socket says 172.21.15.131:45218) 2024-09-17T02:52:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1ffff640 1 -- 172.21.15.131:0/459150666 learned_addr learned my addr 172.21.15.131:0/459150666 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa24808640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa200cd5b0 0x7ffa2014df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa25009640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffa2014c150 0x7ffa2014c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.774 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1ffff640 1 -- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffa2014c150 msgr2=0x7ffa2014c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1ffff640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ffa2014c150 0x7ffa2014c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1ffff640 1 -- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa200cd5b0 msgr2=0x7ffa2014df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1ffff640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ffa200cd5b0 0x7ffa2014df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1ffff640 1 -- 172.21.15.131:0/459150666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa201459b0 con 0x7ffa201605c0 2024-09-17T02:52:58.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1ffff640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa201605c0 0x7ffa20157bb0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7ffa140b0040 tx=0x7ffa14004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:58.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1d7fa640 1 -- 172.21.15.131:0/459150666 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa140b2040 con 0x7ffa201605c0 2024-09-17T02:52:58.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1d7fa640 1 -- 172.21.15.131:0/459150666 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa14005020 con 0x7ffa201605c0 2024-09-17T02:52:58.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa1d7fa640 1 -- 172.21.15.131:0/459150666 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14005470 con 0x7ffa201605c0 2024-09-17T02:52:58.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa26a6c640 1 -- 172.21.15.131:0/459150666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa20159000 con 0x7ffa201605c0 2024-09-17T02:52:58.775 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.777+0000 7ffa26a6c640 1 -- 172.21.15.131:0/459150666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa20145c50 con 0x7ffa201605c0 2024-09-17T02:52:58.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.781+0000 7ffa26a6c640 1 -- 172.21.15.131:0/459150666 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa20160080 con 0x7ffa201605c0 2024-09-17T02:52:58.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.781+0000 7ffa1d7fa640 1 -- 172.21.15.131:0/459150666 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ffa140046d0 con 0x7ffa201605c0 2024-09-17T02:52:58.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.781+0000 7ffa1d7fa640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffa0c073230 0x7ffa0c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.781+0000 7ffa1d7fa640 1 -- 172.21.15.131:0/459150666 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7ffa1412df30 con 0x7ffa201605c0 2024-09-17T02:52:58.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.781+0000 7ffa1d7fa640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x7ffa0c079860 0x7ffa0c07bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.776 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.781+0000 7ffa1d7fa640 1 -- 172.21.15.131:0/459150666 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa140bde50 con 0x7ffa0c079860 2024-09-17T02:52:58.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.781+0000 7ffa1d7fa640 1 -- 172.21.15.131:0/459150666 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7ffa14130550 con 0x7ffa201605c0 2024-09-17T02:52:58.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.781+0000 7ffa24808640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffa0c073230 0x7ffa0c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.781+0000 7ffa25009640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x7ffa0c079860 0x7ffa0c07bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.781+0000 7ffa25009640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x7ffa0c079860 0x7ffa0c07bcf0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:58.777 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.781+0000 7ffa24808640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffa0c073230 0x7ffa0c075740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000b810 tx=0x7ffa10008040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:58.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.785+0000 7ffa1d7fa640 1 -- 172.21.15.131:0/459150666 <== osd.6 v2:172.21.15.189:6802/952142225 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa140bde50 con 0x7ffa0c079860 2024-09-17T02:52:58.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa26a6c640 1 -- 172.21.15.131:0/459150666 --> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ffa20158db0 con 0x7ffa0c079860 2024-09-17T02:52:58.792 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa1d7fa640 1 -- 172.21.15.131:0/459150666 <== osd.6 v2:172.21.15.189:6802/952142225 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ffa20158db0 con 0x7ffa0c079860 2024-09-17T02:52:58.793 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:58.793 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T02:52:58.793 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa26a6c640 1 -- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x7ffa0c079860 msgr2=0x7ffa0c07bcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa26a6c640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:6802/952142225,v1:172.21.15.189:6803/952142225] conn(0x7ffa0c079860 0x7ffa0c07bcf0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa26a6c640 1 -- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffa0c073230 msgr2=0x7ffa0c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa26a6c640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ffa0c073230 0x7ffa0c075740 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000b810 tx=0x7ffa10008040 comp rx=0 tx=0).stop 2024-09-17T02:52:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa26a6c640 1 -- 172.21.15.131:0/459150666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa201605c0 msgr2=0x7ffa20157bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa26a6c640 1 --2- 172.21.15.131:0/459150666 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ffa201605c0 0x7ffa20157bb0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7ffa140b0040 tx=0x7ffa14004040 comp rx=0 tx=0).stop 2024-09-17T02:52:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa25009640 1 -- 172.21.15.131:0/459150666 reap_dead start 2024-09-17T02:52:58.794 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa26a6c640 1 -- 172.21.15.131:0/459150666 shutdown_connections 2024-09-17T02:52:58.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa26a6c640 1 -- 172.21.15.131:0/459150666 >> 172.21.15.131:0/459150666 conn(0x7ffa2006d010 msgr2=0x7ffa2015a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:58.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa26a6c640 1 -- 172.21.15.131:0/459150666 shutdown_connections 2024-09-17T02:52:58.795 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.797+0000 7ffa26a6c640 1 -- 172.21.15.131:0/459150666 wait complete. 2024-09-17T02:52:58.811 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T02:52:58.953 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.957+0000 7f26977e1640 1 -- 172.21.15.131:0/1168575094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f267c003a40 msgr2=0x7f267c003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.957+0000 7f26977e1640 1 --2- 172.21.15.131:0/1168575094 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f267c003a40 0x7f267c003eb0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f268c00c260 tx=0x7f268c01cf40 comp rx=0 tx=0).stop 2024-09-17T02:52:58.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.957+0000 7f2695d7e640 1 -- 172.21.15.131:0/1168575094 reap_dead start 2024-09-17T02:52:58.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.957+0000 7f26977e1640 1 -- 172.21.15.131:0/1168575094 shutdown_connections 2024-09-17T02:52:58.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.957+0000 7f26977e1640 1 -- 172.21.15.131:0/1168575094 >> 172.21.15.131:0/1168575094 conn(0x7f26900d43a0 msgr2=0x7f26900d47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:58.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.957+0000 7f26977e1640 1 -- 172.21.15.131:0/1168575094 shutdown_connections 2024-09-17T02:52:58.954 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.957+0000 7f26977e1640 1 -- 172.21.15.131:0/1168575094 wait complete. 2024-09-17T02:52:58.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.961+0000 7f26977e1640 1 Processor -- start 2024-09-17T02:52:58.956 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.961+0000 7f26977e1640 1 -- start start 2024-09-17T02:52:58.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.961+0000 7f26977e1640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f267c003a40 0x7f2690164e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.961+0000 7f26977e1640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26901668f0 0x7f2690166d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.961+0000 7f26977e1640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26901fbe50 0x7f26901fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.961+0000 7f26977e1640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f269016aa10 con 0x7f26901668f0 2024-09-17T02:52:58.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.961+0000 7f26977e1640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f269016a890 con 0x7f267c003a40 2024-09-17T02:52:58.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.961+0000 7f26977e1640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f269016ab90 con 0x7f26901fbe50 2024-09-17T02:52:58.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.961+0000 7f2694d7c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26901668f0 0x7f2690166d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.961+0000 7f2694d7c640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26901668f0 0x7f2690166d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45238/0 (socket says 172.21.15.131:45238) 2024-09-17T02:52:58.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.961+0000 7f2694d7c640 1 -- 172.21.15.131:0/735389755 learned_addr learned my addr 172.21.15.131:0/735389755 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:58.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f269557d640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f267c003a40 0x7f2690164e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.960 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2695d7e640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26901fbe50 0x7f26901fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2694d7c640 1 -- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26901fbe50 msgr2=0x7f26901fc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2694d7c640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26901fbe50 0x7f26901fc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2694d7c640 1 -- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f267c003a40 msgr2=0x7f2690164e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2694d7c640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f267c003a40 0x7f2690164e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2694d7c640 1 -- 172.21.15.131:0/735389755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26901fc7a0 con 0x7f26901668f0 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2695d7e640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26901fbe50 0x7f26901fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2694d7c640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26901668f0 0x7f2690166d00 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f2680004f10 tx=0x7f2680001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2685ffb640 1 -- 172.21.15.131:0/735389755 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680015020 con 0x7f26901668f0 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2685ffb640 1 -- 172.21.15.131:0/735389755 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2680011960 con 0x7f26901668f0 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f26977e1640 1 -- 172.21.15.131:0/735389755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2690167620 con 0x7f26901668f0 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2685ffb640 1 -- 172.21.15.131:0/735389755 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680011d90 con 0x7f26901668f0 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f26977e1640 1 -- 172.21.15.131:0/735389755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2690167be0 con 0x7f26901668f0 2024-09-17T02:52:58.961 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f26977e1640 1 -- 172.21.15.131:0/735389755 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26900b7840 con 0x7f26901668f0 2024-09-17T02:52:58.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2685ffb640 1 -- 172.21.15.131:0/735389755 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f2680018020 con 0x7f26901668f0 2024-09-17T02:52:58.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2685ffb640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2674073200 0x7f2674075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:58.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f2685ffb640 1 -- 172.21.15.131:0/735389755 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f2680094b80 con 0x7f26901668f0 2024-09-17T02:52:58.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.965+0000 7f269557d640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2674073200 0x7f2674075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:58.962 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.969+0000 7f269557d640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2674073200 0x7f2674075710 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f268c0ac000 tx=0x7f268c0b0040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:58.964 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:58.969+0000 7f2685ffb640 1 -- 172.21.15.131:0/735389755 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f268001f020 con 0x7f26901668f0 2024-09-17T02:52:59.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.125+0000 7f26977e1640 1 -- 172.21.15.131:0/735389755 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f26901653c0 con 0x7f2674073200 2024-09-17T02:52:59.120 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.125+0000 7f2685ffb640 1 -- 172.21.15.131:0/735389755 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f26901653c0 con 0x7f2674073200 2024-09-17T02:52:59.120 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T02:52:59.122 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 -- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2674073200 msgr2=0x7f2674075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2674073200 0x7f2674075710 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f268c0ac000 tx=0x7f268c0b0040 comp rx=0 tx=0).stop 2024-09-17T02:52:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 -- 172.21.15.131:0/735389755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26901668f0 msgr2=0x7f2690166d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26901668f0 0x7f2690166d00 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f2680004f10 tx=0x7f2680001640 comp rx=0 tx=0).stop 2024-09-17T02:52:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 -- 172.21.15.131:0/735389755 shutdown_connections 2024-09-17T02:52:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f2674073200 0x7f2674075710 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f26901fbe50 0x7f26901fc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f26901668f0 0x7f2690166d00 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 --2- 172.21.15.131:0/735389755 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f267c003a40 0x7f2690164e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 -- 172.21.15.131:0/735389755 >> 172.21.15.131:0/735389755 conn(0x7f26900d43a0 msgr2=0x7f26900b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 -- 172.21.15.131:0/735389755 shutdown_connections 2024-09-17T02:52:59.123 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.129+0000 7f26977e1640 1 -- 172.21.15.131:0/735389755 wait complete. 2024-09-17T02:52:59.141 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T02:52:59.142 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T02:52:59.289 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.293+0000 7fe9f74f5640 1 -- 172.21.15.131:0/1056704681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9f006f950 msgr2=0x7fe9f00d02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.293+0000 7fe9f74f5640 1 --2- 172.21.15.131:0/1056704681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9f006f950 0x7fe9f00d02e0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009840 tx=0x7fe9e401c6d0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.293+0000 7fe9f74f5640 1 -- 172.21.15.131:0/1056704681 shutdown_connections 2024-09-17T02:52:59.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.293+0000 7fe9f74f5640 1 --2- 172.21.15.131:0/1056704681 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9f006f950 0x7fe9f00d02e0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.293+0000 7fe9f74f5640 1 -- 172.21.15.131:0/1056704681 >> 172.21.15.131:0/1056704681 conn(0x7fe9f006d010 msgr2=0x7fe9f006d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:59.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.293+0000 7fe9f74f5640 1 -- 172.21.15.131:0/1056704681 shutdown_connections 2024-09-17T02:52:59.290 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.293+0000 7fe9f74f5640 1 -- 172.21.15.131:0/1056704681 wait complete. 2024-09-17T02:52:59.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f74f5640 1 Processor -- start 2024-09-17T02:52:59.291 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f74f5640 1 -- start start 2024-09-17T02:52:59.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f74f5640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f006f540 0x7fe9f00b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f74f5640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe9f006f950 0x7fe9f013a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f74f5640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9f00d0820 0x7fe9f015e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f74f5640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f016aee0 con 0x7fe9f00d0820 2024-09-17T02:52:59.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f74f5640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f016ad60 con 0x7fe9f006f540 2024-09-17T02:52:59.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f74f5640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fe9f016b060 con 0x7fe9f006f950 2024-09-17T02:52:59.294 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f5a92640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9f00d0820 0x7fe9f015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f5a92640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9f00d0820 0x7fe9f015e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45258/0 (socket says 172.21.15.131:45258) 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f5a92640 1 -- 172.21.15.131:0/946598261 learned_addr learned my addr 172.21.15.131:0/946598261 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f5291640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f006f540 0x7fe9f00b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f4a90640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe9f006f950 0x7fe9f013a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f5a92640 1 -- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe9f006f950 msgr2=0x7fe9f013a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f5a92640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe9f006f950 0x7fe9f013a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f5a92640 1 -- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f006f540 msgr2=0x7fe9f00b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f5a92640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fe9f006f540 0x7fe9f00b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f5a92640 1 -- 172.21.15.131:0/946598261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f015a0b0 con 0x7fe9f00d0820 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f4a90640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fe9f006f950 0x7fe9f013a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f5a92640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9f00d0820 0x7fe9f015e240 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000b9e0 tx=0x7fe9e000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9edffb640 1 -- 172.21.15.131:0/946598261 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0003b80 con 0x7fe9f00d0820 2024-09-17T02:52:59.295 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9edffb640 1 -- 172.21.15.131:0/946598261 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9e0003e10 con 0x7fe9f00d0820 2024-09-17T02:52:59.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9edffb640 1 -- 172.21.15.131:0/946598261 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000cb80 con 0x7fe9f00d0820 2024-09-17T02:52:59.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f74f5640 1 -- 172.21.15.131:0/946598261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f015a390 con 0x7fe9f00d0820 2024-09-17T02:52:59.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.297+0000 7fe9f74f5640 1 -- 172.21.15.131:0/946598261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f0143a40 con 0x7fe9f00d0820 2024-09-17T02:52:59.296 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.301+0000 7fe9f74f5640 1 -- 172.21.15.131:0/946598261 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9f00d02e0 con 0x7fe9f00d0820 2024-09-17T02:52:59.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.301+0000 7fe9edffb640 1 -- 172.21.15.131:0/946598261 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fe9e0006d10 con 0x7fe9f00d0820 2024-09-17T02:52:59.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.301+0000 7fe9edffb640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe9dc073230 0x7fe9dc075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.301+0000 7fe9edffb640 1 -- 172.21.15.131:0/946598261 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7fe9e0094900 con 0x7fe9f00d0820 2024-09-17T02:52:59.297 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.301+0000 7fe9edffb640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x7fe9dc079860 0x7fe9dc07bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.301+0000 7fe9edffb640 1 -- 172.21.15.131:0/946598261 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9e000eea0 con 0x7fe9dc079860 2024-09-17T02:52:59.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.301+0000 7fe9edffb640 1 -- 172.21.15.131:0/946598261 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_get_version_reply(handle=1 version=127) ==== 24+0+0 (secure 0 0 0) 0x7fe9e0094cf0 con 0x7fe9f00d0820 2024-09-17T02:52:59.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.301+0000 7fe9f5291640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe9dc073230 0x7fe9dc075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.301+0000 7fe9f4a90640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x7fe9dc079860 0x7fe9dc07bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.301+0000 7fe9f4a90640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x7fe9dc079860 0x7fe9dc07bcf0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:59.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.301+0000 7fe9f5291640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe9dc073230 0x7fe9dc075740 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000f70 tx=0x7fe9e8009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:59.298 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.305+0000 7fe9edffb640 1 -- 172.21.15.131:0/946598261 <== osd.7 v2:172.21.15.189:6810/3586470400 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9e000eea0 con 0x7fe9dc079860 2024-09-17T02:52:59.311 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f74f5640 1 -- 172.21.15.131:0/946598261 --> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe9f0159e00 con 0x7fe9dc079860 2024-09-17T02:52:59.312 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9edffb640 1 -- 172.21.15.131:0/946598261 <== osd.7 v2:172.21.15.189:6810/3586470400 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe9f0159e00 con 0x7fe9dc079860 2024-09-17T02:52:59.312 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-17T02:52:59.312 INFO:teuthology.orchestra.run.smithi131.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T02:52:59.312 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-17T02:52:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f74f5640 1 -- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x7fe9dc079860 msgr2=0x7fe9dc07bcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f74f5640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:6810/3586470400,v1:172.21.15.189:6811/3586470400] conn(0x7fe9dc079860 0x7fe9dc07bcf0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f74f5640 1 -- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe9dc073230 msgr2=0x7fe9dc075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f74f5640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fe9dc073230 0x7fe9dc075740 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000f70 tx=0x7fe9e8009040 comp rx=0 tx=0).stop 2024-09-17T02:52:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f74f5640 1 -- 172.21.15.131:0/946598261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9f00d0820 msgr2=0x7fe9f015e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.313 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f74f5640 1 --2- 172.21.15.131:0/946598261 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fe9f00d0820 0x7fe9f015e240 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000b9e0 tx=0x7fe9e000bea0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f5a92640 1 -- 172.21.15.131:0/946598261 reap_dead start 2024-09-17T02:52:59.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f74f5640 1 -- 172.21.15.131:0/946598261 shutdown_connections 2024-09-17T02:52:59.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f74f5640 1 -- 172.21.15.131:0/946598261 >> 172.21.15.131:0/946598261 conn(0x7fe9f006d010 msgr2=0x7fe9f015f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:59.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f74f5640 1 -- 172.21.15.131:0/946598261 shutdown_connections 2024-09-17T02:52:59.314 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.317+0000 7fe9f74f5640 1 -- 172.21.15.131:0/946598261 wait complete. 2024-09-17T02:52:59.330 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T02:52:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.477+0000 7f4ca7069640 1 -- 172.21.15.131:0/2966953146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca015f960 msgr2=0x7f4ca00ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.477+0000 7f4ca7069640 1 --2- 172.21.15.131:0/2966953146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca015f960 0x7f4ca00ca590 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f4c98009bc0 tx=0x7f4c9801cba0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.477+0000 7f4ca7069640 1 -- 172.21.15.131:0/2966953146 shutdown_connections 2024-09-17T02:52:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.477+0000 7f4ca7069640 1 --2- 172.21.15.131:0/2966953146 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4ca00cc4a0 0x7f4ca0074610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.473 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.477+0000 7f4ca7069640 1 --2- 172.21.15.131:0/2966953146 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca015f960 0x7f4ca00ca590 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.477+0000 7f4ca7069640 1 -- 172.21.15.131:0/2966953146 >> 172.21.15.131:0/2966953146 conn(0x7f4ca00cff40 msgr2=0x7f4ca00d0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:59.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.477+0000 7f4ca7069640 1 -- 172.21.15.131:0/2966953146 shutdown_connections 2024-09-17T02:52:59.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.477+0000 7f4ca7069640 1 -- 172.21.15.131:0/2966953146 wait complete. 2024-09-17T02:52:59.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca7069640 1 Processor -- start 2024-09-17T02:52:59.474 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca7069640 1 -- start start 2024-09-17T02:52:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca7069640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ca00cdb10 0x7f4ca015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca7069640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca015f960 0x7f4ca00c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca7069640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4ca006f830 0x7f4ca00c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca7069640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca0168830 con 0x7f4ca015f960 2024-09-17T02:52:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca7069640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f4ca01686b0 con 0x7f4ca00cdb10 2024-09-17T02:52:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca7069640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f4ca01689b0 con 0x7f4ca006f830 2024-09-17T02:52:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca015f960 0x7f4ca00c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9ffff640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca015f960 0x7f4ca00c9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45270/0 (socket says 172.21.15.131:45270) 2024-09-17T02:52:59.477 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9ffff640 1 -- 172.21.15.131:0/1189880958 learned_addr learned my addr 172.21.15.131:0/1189880958 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca4e05640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ca00cdb10 0x7f4ca015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca5606640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4ca006f830 0x7f4ca00c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9ffff640 1 -- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4ca006f830 msgr2=0x7f4ca00c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9ffff640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4ca006f830 0x7f4ca00c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9ffff640 1 -- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ca00cdb10 msgr2=0x7f4ca015bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9ffff640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ca00cdb10 0x7f4ca015bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9ffff640 1 -- 172.21.15.131:0/1189880958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ca01fb920 con 0x7f4ca015f960 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9ffff640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca015f960 0x7f4ca00c9250 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400c460 tx=0x7f4c9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9d7fa640 1 -- 172.21.15.131:0/1189880958 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94016020 con 0x7f4ca015f960 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9d7fa640 1 -- 172.21.15.131:0/1189880958 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c94012940 con 0x7f4ca015f960 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca7069640 1 -- 172.21.15.131:0/1189880958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ca01fbc00 con 0x7f4ca015f960 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4c9d7fa640 1 -- 172.21.15.131:0/1189880958 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94012d90 con 0x7f4ca015f960 2024-09-17T02:52:59.478 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca7069640 1 -- 172.21.15.131:0/1189880958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ca01fc1c0 con 0x7f4ca015f960 2024-09-17T02:52:59.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.481+0000 7f4ca7069640 1 -- 172.21.15.131:0/1189880958 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ca0074610 con 0x7f4ca015f960 2024-09-17T02:52:59.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.485+0000 7f4c9d7fa640 1 -- 172.21.15.131:0/1189880958 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f4c94019020 con 0x7f4ca015f960 2024-09-17T02:52:59.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.485+0000 7f4c9d7fa640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4c8c073230 0x7f4c8c075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.485+0000 7f4c9d7fa640 1 -- 172.21.15.131:0/1189880958 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f4c94094290 con 0x7f4ca015f960 2024-09-17T02:52:59.479 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.485+0000 7f4ca4e05640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4c8c073230 0x7f4c8c075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.480 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.485+0000 7f4ca4e05640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4c8c073230 0x7f4c8c075740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4c980af000 tx=0x7f4c9801b3e0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:59.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.489+0000 7f4c9d7fa640 1 -- 172.21.15.131:0/1189880958 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f4c94020030 con 0x7f4ca015f960 2024-09-17T02:52:59.638 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.645+0000 7f4ca7069640 1 -- 172.21.15.131:0/1189880958 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f4ca00cd660 con 0x7f4c8c073230 2024-09-17T02:52:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.645+0000 7f4c9d7fa640 1 -- 172.21.15.131:0/1189880958 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4ca00cd660 con 0x7f4c8c073230 2024-09-17T02:52:59.639 INFO:teuthology.orchestra.run.smithi131.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T02:52:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.645+0000 7f4ca7069640 1 -- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4c8c073230 msgr2=0x7f4c8c075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.645+0000 7f4ca7069640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4c8c073230 0x7f4c8c075740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4c980af000 tx=0x7f4c9801b3e0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.645+0000 7f4ca7069640 1 -- 172.21.15.131:0/1189880958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca015f960 msgr2=0x7f4ca00c9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.645+0000 7f4ca7069640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca015f960 0x7f4ca00c9250 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f4c9400c460 tx=0x7f4c9400c920 comp rx=0 tx=0).stop 2024-09-17T02:52:59.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.649+0000 7f4ca7069640 1 -- 172.21.15.131:0/1189880958 shutdown_connections 2024-09-17T02:52:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.649+0000 7f4ca7069640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f4c8c073230 0x7f4c8c075740 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.649+0000 7f4ca7069640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f4ca006f830 0x7f4ca00c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.649+0000 7f4ca7069640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f4ca015f960 0x7f4ca00c9250 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.649+0000 7f4ca7069640 1 --2- 172.21.15.131:0/1189880958 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f4ca00cdb10 0x7f4ca015bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.649+0000 7f4ca7069640 1 -- 172.21.15.131:0/1189880958 >> 172.21.15.131:0/1189880958 conn(0x7f4ca00cff40 msgr2=0x7f4ca0141200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.649+0000 7f4ca7069640 1 -- 172.21.15.131:0/1189880958 shutdown_connections 2024-09-17T02:52:59.643 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.649+0000 7f4ca7069640 1 -- 172.21.15.131:0/1189880958 wait complete. 2024-09-17T02:52:59.661 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:52:59.800 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.805+0000 7f25c832a640 1 -- 172.21.15.131:0/842276537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c00ca5c0 msgr2=0x7f25c015c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.805+0000 7f25c832a640 1 --2- 172.21.15.131:0/842276537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c00ca5c0 0x7f25c015c5e0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f25c00cd610 tx=0x7f25b801ccc0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.805+0000 7f25c832a640 1 -- 172.21.15.131:0/842276537 shutdown_connections 2024-09-17T02:52:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.805+0000 7f25c832a640 1 --2- 172.21.15.131:0/842276537 >> v2:172.21.15.189:3300/0 conn(0x7f25c00cc980 0x7f25c00ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.805+0000 7f25c832a640 1 --2- 172.21.15.131:0/842276537 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c00ca5c0 0x7f25c015c5e0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.805+0000 7f25c832a640 1 -- 172.21.15.131:0/842276537 >> 172.21.15.131:0/842276537 conn(0x7f25c006eed0 msgr2=0x7f25c006f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.805+0000 7f25c832a640 1 -- 172.21.15.131:0/842276537 shutdown_connections 2024-09-17T02:52:59.801 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.805+0000 7f25c832a640 1 -- 172.21.15.131:0/842276537 wait complete. 2024-09-17T02:52:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c832a640 1 Processor -- start 2024-09-17T02:52:59.802 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c832a640 1 -- start start 2024-09-17T02:52:59.804 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c832a640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c00ca5c0 0x7f25c00b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c832a640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c00cc980 0x7f25c013d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c832a640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25c00cff00 0x7f25c0155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c832a640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f25c0169c50 con 0x7f25c00ca5c0 2024-09-17T02:52:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c832a640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f25c00b60b0 con 0x7f25c00cc980 2024-09-17T02:52:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c832a640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f25c0169dd0 con 0x7f25c00cff00 2024-09-17T02:52:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c60c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c00ca5c0 0x7f25c00b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c60c6640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c00ca5c0 0x7f25c00b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:45296/0 (socket says 172.21.15.131:45296) 2024-09-17T02:52:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c60c6640 1 -- 172.21.15.131:0/3352541107 learned_addr learned my addr 172.21.15.131:0/3352541107 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:52:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c58c5640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c00cc980 0x7f25c013d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c60c6640 1 -- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25c00cff00 msgr2=0x7f25c0155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.805 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c68c7640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25c00cff00 0x7f25c0155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c60c6640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25c00cff00 0x7f25c0155830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c60c6640 1 -- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c00cc980 msgr2=0x7f25c013d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c60c6640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c00cc980 0x7f25c013d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c60c6640 1 -- 172.21.15.131:0/3352541107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c01525c0 con 0x7f25c00ca5c0 2024-09-17T02:52:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c58c5640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c00cc980 0x7f25c013d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:52:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c60c6640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c00ca5c0 0x7f25c00b1a90 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f25b801a8f0 tx=0x7f25b8005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25b2ffd640 1 -- 172.21.15.131:0/3352541107 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b80b2040 con 0x7f25c00ca5c0 2024-09-17T02:52:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25b2ffd640 1 -- 172.21.15.131:0/3352541107 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25b8004040 con 0x7f25c00ca5c0 2024-09-17T02:52:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25b2ffd640 1 -- 172.21.15.131:0/3352541107 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8004430 con 0x7f25c00ca5c0 2024-09-17T02:52:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c832a640 1 -- 172.21.15.131:0/3352541107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c01528a0 con 0x7f25c00ca5c0 2024-09-17T02:52:59.806 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c832a640 1 -- 172.21.15.131:0/3352541107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c0152e60 con 0x7f25c00ca5c0 2024-09-17T02:52:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.809+0000 7f25c832a640 1 -- 172.21.15.131:0/3352541107 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c015c5e0 con 0x7f25c00ca5c0 2024-09-17T02:52:59.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.813+0000 7f25b2ffd640 1 -- 172.21.15.131:0/3352541107 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f25b80bd030 con 0x7f25c00ca5c0 2024-09-17T02:52:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.813+0000 7f25b2ffd640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25a4073230 0x7f25a4075740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:52:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.813+0000 7f25b2ffd640 1 -- 172.21.15.131:0/3352541107 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f25b812e510 con 0x7f25c00ca5c0 2024-09-17T02:52:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.813+0000 7f25c58c5640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25a4073230 0x7f25a4075740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:52:59.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.813+0000 7f25c58c5640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25a4073230 0x7f25a4075740 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005fd0 tx=0x7f25b4005de0 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:52:59.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.817+0000 7f25b2ffd640 1 -- 172.21.15.131:0/3352541107 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f25b80c4020 con 0x7f25c00ca5c0 2024-09-17T02:52:59.965 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.969+0000 7f25c832a640 1 -- 172.21.15.131:0/3352541107 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f25c00d0560 con 0x7f25a4073230 2024-09-17T02:52:59.966 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25b2ffd640 1 -- 172.21.15.131:0/3352541107 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13455 (secure 0 0 0) 0x7f25c00d0560 con 0x7f25a4073230 2024-09-17T02:52:59.967 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:52:59.967 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 -- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25a4073230 msgr2=0x7f25a4075740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25a4073230 0x7f25a4075740 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005fd0 tx=0x7f25b4005de0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 -- 172.21.15.131:0/3352541107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c00ca5c0 msgr2=0x7f25c00b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c00ca5c0 0x7f25c00b1a90 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f25b801a8f0 tx=0x7f25b8005ae0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 -- 172.21.15.131:0/3352541107 shutdown_connections 2024-09-17T02:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f25a4073230 0x7f25a4075740 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f25c00cff00 0x7f25c0155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f25c00cc980 0x7f25c013d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 --2- 172.21.15.131:0/3352541107 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f25c00ca5c0 0x7f25c00b1a90 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:52:59.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 -- 172.21.15.131:0/3352541107 >> 172.21.15.131:0/3352541107 conn(0x7f25c006eed0 msgr2=0x7f25c00b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:52:59.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 -- 172.21.15.131:0/3352541107 shutdown_connections 2024-09-17T02:52:59.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:52:59.973+0000 7f25c832a640 1 -- 172.21.15.131:0/3352541107 wait complete. 2024-09-17T02:52:59.988 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-09-17T02:52:59.416598+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263024,"kb_used_data":13512,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711696,"statfs":{"total":773094113280,"available":772824776704,"internally_reserved":0,"allocated":13836288,"data_stored":7095000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.273261"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":314,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T02:52:51.171022+0000","last_change":"2024-09-17T02:40:43.289807+0000","last_active":"2024-09-17T02:52:51.171022+0000","last_peered":"2024-09-17T02:52:51.171022+0000","last_clean":"2024-09-17T02:52:51.171022+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:52:51.171022+0000","last_undegraded":"2024-09-17T02:52:51.171022+0000","last_fullsized":"2024-09-17T02:52:51.171022+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T02:40:41.948675+0000","last_clean_scrub_stamp":"2024-09-17T02:40:41.948675+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:45.966820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509591,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2174976,"data_stored":1329651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509590,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509591,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33316,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2174976,"data_stored":1329651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509592,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32736,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:52:59.992 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T02:40:41.948675+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=2, tm_min=40, tm_sec=41, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=2, tm_min=52, tm_sec=54, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T02:52:59.992 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T02:53:16.146 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:53:16.149 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:53:19.993 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T02:53:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.160+0000 7ff440f63640 1 -- 172.21.15.131:0/2130407674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff424004810 msgr2=0x7ff424004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.160+0000 7ff440f63640 1 --2- 172.21.15.131:0/2130407674 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff424004810 0x7ff424004c20 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c310 tx=0x7ff42c01cf90 comp rx=0 tx=0).stop 2024-09-17T02:53:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.160+0000 7ff43ad9d640 1 -- 172.21.15.131:0/2130407674 reap_dead start 2024-09-17T02:53:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.160+0000 7ff440f63640 1 -- 172.21.15.131:0/2130407674 shutdown_connections 2024-09-17T02:53:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.160+0000 7ff440f63640 1 -- 172.21.15.131:0/2130407674 >> 172.21.15.131:0/2130407674 conn(0x7ff43c06eed0 msgr2=0x7ff43c06f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:53:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.160+0000 7ff440f63640 1 -- 172.21.15.131:0/2130407674 shutdown_connections 2024-09-17T02:53:20.158 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.160+0000 7ff440f63640 1 -- 172.21.15.131:0/2130407674 wait complete. 2024-09-17T02:53:20.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff440f63640 1 Processor -- start 2024-09-17T02:53:20.159 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff440f63640 1 -- start start 2024-09-17T02:53:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff440f63640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff424004810 0x7ff43c154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff440f63640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff43c148d80 0x7ff43c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff440f63640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff43c155c10 0x7ff43c151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff440f63640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c167920 con 0x7ff43c155c10 2024-09-17T02:53:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff440f63640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c1677a0 con 0x7ff424004810 2024-09-17T02:53:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff440f63640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7ff43c167aa0 con 0x7ff43c148d80 2024-09-17T02:53:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff43ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff43c155c10 0x7ff43c151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:53:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff43ad9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff43c155c10 0x7ff43c151380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47706/0 (socket says 172.21.15.131:47706) 2024-09-17T02:53:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff43ad9d640 1 -- 172.21.15.131:0/3452901957 learned_addr learned my addr 172.21.15.131:0/3452901957 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:53:20.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff43a59c640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff424004810 0x7ff43c154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:53:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff43ad9d640 1 -- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff43c148d80 msgr2=0x7ff43c150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff439d9b640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff43c148d80 0x7ff43c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:53:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff43ad9d640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff43c148d80 0x7ff43c150e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff43ad9d640 1 -- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff424004810 msgr2=0x7ff43c154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff43ad9d640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff424004810 0x7ff43c154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff43ad9d640 1 -- 172.21.15.131:0/3452901957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c155420 con 0x7ff43c155c10 2024-09-17T02:53:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff43ad9d640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff43c155c10 0x7ff43c151380 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0cda80 tx=0x7ff43400fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:53:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.164+0000 7ff42affd640 1 -- 172.21.15.131:0/3452901957 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff434003d90 con 0x7ff43c155c10 2024-09-17T02:53:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.168+0000 7ff42affd640 1 -- 172.21.15.131:0/3452901957 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff434017070 con 0x7ff43c155c10 2024-09-17T02:53:20.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.168+0000 7ff42affd640 1 -- 172.21.15.131:0/3452901957 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43400d900 con 0x7ff43c155c10 2024-09-17T02:53:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.168+0000 7ff440f63640 1 -- 172.21.15.131:0/3452901957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c157580 con 0x7ff43c155c10 2024-09-17T02:53:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.168+0000 7ff440f63640 1 -- 172.21.15.131:0/3452901957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c157b40 con 0x7ff43c155c10 2024-09-17T02:53:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.168+0000 7ff440f63640 1 -- 172.21.15.131:0/3452901957 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43c1572f0 con 0x7ff43c155c10 2024-09-17T02:53:20.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.168+0000 7ff42affd640 1 -- 172.21.15.131:0/3452901957 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7ff43400daa0 con 0x7ff43c155c10 2024-09-17T02:53:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.168+0000 7ff42affd640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff41c073200 0x7ff41c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.168+0000 7ff42affd640 1 -- 172.21.15.131:0/3452901957 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7ff434094870 con 0x7ff43c155c10 2024-09-17T02:53:20.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.168+0000 7ff43a59c640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff41c073200 0x7ff41c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:53:20.166 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.172+0000 7ff43a59c640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff41c073200 0x7ff41c075710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff430005d90 tx=0x7ff43000c500 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:53:20.169 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.172+0000 7ff42affd640 1 -- 172.21.15.131:0/3452901957 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7ff43405f470 con 0x7ff43c155c10 2024-09-17T02:53:20.338 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.340+0000 7ff440f63640 1 -- 172.21.15.131:0/3452901957 --> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff43c0ccea0 con 0x7ff41c073200 2024-09-17T02:53:20.339 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff42affd640 1 -- 172.21.15.131:0/3452901957 <== mgr.4129 v2:172.21.15.189:6800/674320934 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13365 (secure 0 0 0) 0x7ff43c0ccea0 con 0x7ff41c073200 2024-09-17T02:53:20.339 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:53:20.339 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-17T02:53:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 -- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff41c073200 msgr2=0x7ff41c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff41c073200 0x7ff41c075710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff430005d90 tx=0x7ff43000c500 comp rx=0 tx=0).stop 2024-09-17T02:53:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 -- 172.21.15.131:0/3452901957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff43c155c10 msgr2=0x7ff43c151380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff43c155c10 0x7ff43c151380 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0cda80 tx=0x7ff43400fed0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 -- 172.21.15.131:0/3452901957 shutdown_connections 2024-09-17T02:53:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7ff41c073200 0x7ff41c075710 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7ff43c155c10 0x7ff43c151380 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7ff43c148d80 0x7ff43c150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 --2- 172.21.15.131:0/3452901957 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7ff424004810 0x7ff43c154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.342 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 -- 172.21.15.131:0/3452901957 >> 172.21.15.131:0/3452901957 conn(0x7ff43c06eed0 msgr2=0x7ff43c14f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:53:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 -- 172.21.15.131:0/3452901957 shutdown_connections 2024-09-17T02:53:20.343 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.344+0000 7ff440f63640 1 -- 172.21.15.131:0/3452901957 wait complete. 2024-09-17T02:53:20.363 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-17T02:53:19.422489+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263012,"kb_used_data":13512,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711708,"statfs":{"total":773094113280,"available":772824788992,"internally_reserved":0,"allocated":13836288,"data_stored":7095000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003618"},"pg_stats":[{"pgid":"1.0","version":"127'74","reported_seq":324,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-09-17T02:52:56.639424+0000","last_change":"2024-09-17T02:52:56.639424+0000","last_active":"2024-09-17T02:52:56.639424+0000","last_peered":"2024-09-17T02:52:56.639424+0000","last_clean":"2024-09-17T02:52:56.639424+0000","last_became_active":"2024-09-17T02:40:43.287850+0000","last_became_peered":"2024-09-17T02:40:43.287850+0000","last_unstale":"2024-09-17T02:52:56.639424+0000","last_undegraded":"2024-09-17T02:52:56.639424+0000","last_fullsized":"2024-09-17T02:52:56.639424+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'74","last_scrub_stamp":"2024-09-17T02:52:56.639330+0000","last_deep_scrub":"127'74","last_deep_scrub_stamp":"2024-09-17T02:52:56.639330+0000","last_clean_scrub_stamp":"2024-09-17T02:52:56.639330+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:24:14.882315+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509595,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509595,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2174976,"data_stored":1329651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509594,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509595,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509595,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2174976,"data_stored":1329651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32736,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1581056,"data_stored":739283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T02:53:20.364 DEBUG:teuthology.orchestra.run.smithi131:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T02:53:20.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.528+0000 7fdd6557f640 1 -- 172.21.15.131:0/620044140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60160150 msgr2=0x7fdd60160560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.528+0000 7fdd6557f640 1 --2- 172.21.15.131:0/620044140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60160150 0x7fdd60160560 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fdd60071ba0 tx=0x7fdd5801cd80 comp rx=0 tx=0).stop 2024-09-17T02:53:20.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.528+0000 7fdd6557f640 1 -- 172.21.15.131:0/620044140 shutdown_connections 2024-09-17T02:53:20.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.528+0000 7fdd6557f640 1 --2- 172.21.15.131:0/620044140 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60160150 0x7fdd60160560 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.528+0000 7fdd6557f640 1 --2- 172.21.15.131:0/620044140 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdd6006eeb0 0x7fdd6006f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.528+0000 7fdd6557f640 1 --2- 172.21.15.131:0/620044140 >> v2:172.21.15.189:3300/0 conn(0x7fdd60073680 0x7fdd6015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.528+0000 7fdd6557f640 1 -- 172.21.15.131:0/620044140 >> 172.21.15.131:0/620044140 conn(0x7fdd600b1c40 msgr2=0x7fdd600b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:53:20.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.528+0000 7fdd6557f640 1 -- 172.21.15.131:0/620044140 shutdown_connections 2024-09-17T02:53:20.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.528+0000 7fdd6557f640 1 -- 172.21.15.131:0/620044140 wait complete. 2024-09-17T02:53:20.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd6557f640 1 Processor -- start 2024-09-17T02:53:20.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd6557f640 1 -- start start 2024-09-17T02:53:20.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd6557f640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdd6006eeb0 0x7fdd6015e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd6557f640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd60073680 0x7fdd600ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd6557f640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60160150 0x7fdd600cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd6557f640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7fdd600cb970 con 0x7fdd60160150 2024-09-17T02:53:20.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd6557f640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7fdd600cb7f0 con 0x7fdd60073680 2024-09-17T02:53:20.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd6557f640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7fdd600cbaf0 con 0x7fdd6006eeb0 2024-09-17T02:53:20.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60160150 0x7fdd600cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:53:20.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd5f7fe640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60160150 0x7fdd600cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47722/0 (socket says 172.21.15.131:47722) 2024-09-17T02:53:20.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd5f7fe640 1 -- 172.21.15.131:0/1291427485 learned_addr learned my addr 172.21.15.131:0/1291427485 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:53:20.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd5e7fc640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd60073680 0x7fdd600ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:53:20.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd5f7fe640 1 -- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdd6006eeb0 msgr2=0x7fdd6015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd5f7fe640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdd6006eeb0 0x7fdd6015e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd5f7fe640 1 -- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd60073680 msgr2=0x7fdd600ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd5f7fe640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd60073680 0x7fdd600ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd5f7fe640 1 -- 172.21.15.131:0/1291427485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd601fc4f0 con 0x7fdd60160150 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd5e7fc640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd60073680 0x7fdd600ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd5f7fe640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60160150 0x7fdd600cf3f0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fdd5801b430 tx=0x7fdd58005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd477fe640 1 -- 172.21.15.131:0/1291427485 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd580bc070 con 0x7fdd60160150 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd477fe640 1 -- 172.21.15.131:0/1291427485 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd58004330 con 0x7fdd60160150 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd6557f640 1 -- 172.21.15.131:0/1291427485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd601fc7d0 con 0x7fdd60160150 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd477fe640 1 -- 172.21.15.131:0/1291427485 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd580b2040 con 0x7fdd60160150 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd6557f640 1 -- 172.21.15.131:0/1291427485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd601fcd90 con 0x7fdd60160150 2024-09-17T02:53:20.531 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.532+0000 7fdd6557f640 1 -- 172.21.15.131:0/1291427485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd6015c630 con 0x7fdd60160150 2024-09-17T02:53:20.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.536+0000 7fdd477fe640 1 -- 172.21.15.131:0/1291427485 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7fdd580ad440 con 0x7fdd60160150 2024-09-17T02:53:20.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.536+0000 7fdd477fe640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdd4c073200 0x7fdd4c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.536+0000 7fdd477fe640 1 -- 172.21.15.131:0/1291427485 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7fdd5812e260 con 0x7fdd60160150 2024-09-17T02:53:20.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.536+0000 7fdd5effd640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdd4c073200 0x7fdd4c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:53:20.533 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.536+0000 7fdd5effd640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdd4c073200 0x7fdd4c075710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdd500058d0 tx=0x7fdd50005860 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:53:20.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.540+0000 7fdd477fe640 1 -- 172.21.15.131:0/1291427485 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7fdd580c5080 con 0x7fdd60160150 2024-09-17T02:53:20.705 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.708+0000 7fdd6557f640 1 -- 172.21.15.131:0/1291427485 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fdd6015aa90 con 0x7fdd60160150 2024-09-17T02:53:20.711 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.716+0000 7fdd477fe640 1 -- 172.21.15.131:0/1291427485 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v46)=0 v46) ==== 123+0+0 (secure 0 0 0) 0x7fdd6006f800 con 0x7fdd60160150 2024-09-17T02:53:20.713 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.716+0000 7fdd477fe640 1 -- 172.21.15.131:0/1291427485 <== mon.0 v2:172.21.15.131:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fdd580f8870 con 0x7fdd60160150 2024-09-17T02:53:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 -- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdd4c073200 msgr2=0x7fdd4c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdd4c073200 0x7fdd4c075710 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdd500058d0 tx=0x7fdd50005860 comp rx=0 tx=0).stop 2024-09-17T02:53:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 -- 172.21.15.131:0/1291427485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60160150 msgr2=0x7fdd600cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60160150 0x7fdd600cf3f0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fdd5801b430 tx=0x7fdd58005b10 comp rx=0 tx=0).stop 2024-09-17T02:53:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 -- 172.21.15.131:0/1291427485 shutdown_connections 2024-09-17T02:53:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7fdd4c073200 0x7fdd4c075710 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7fdd60160150 0x7fdd600cf3f0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7fdd60073680 0x7fdd600ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 --2- 172.21.15.131:0/1291427485 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7fdd6006eeb0 0x7fdd6015e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 -- 172.21.15.131:0/1291427485 >> 172.21.15.131:0/1291427485 conn(0x7fdd600b1c40 msgr2=0x7fdd600d07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:53:20.715 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 -- 172.21.15.131:0/1291427485 shutdown_connections 2024-09-17T02:53:20.716 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.720+0000 7fdd6557f640 1 -- 172.21.15.131:0/1291427485 wait complete. 2024-09-17T02:53:20.736 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T02:53:20.736 DEBUG:teuthology.orchestra.run.smithi131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T02:53:20.893 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.896+0000 7f7abcf21640 1 -- 172.21.15.131:0/3861028640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab806d550 msgr2=0x7f7ab8073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.896+0000 7f7abcf21640 1 --2- 172.21.15.131:0/3861028640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab806d550 0x7f7ab8073e10 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009900 tx=0x7f7aac01c8e0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.896+0000 7f7abcf21640 1 -- 172.21.15.131:0/3861028640 shutdown_connections 2024-09-17T02:53:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.896+0000 7f7abcf21640 1 --2- 172.21.15.131:0/3861028640 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab806d550 0x7f7ab8073e10 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.896+0000 7f7abcf21640 1 -- 172.21.15.131:0/3861028640 >> 172.21.15.131:0/3861028640 conn(0x7f7ab80b2dc0 msgr2=0x7f7ab80b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:53:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.896+0000 7f7abcf21640 1 -- 172.21.15.131:0/3861028640 shutdown_connections 2024-09-17T02:53:20.894 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.896+0000 7f7abcf21640 1 -- 172.21.15.131:0/3861028640 wait complete. 2024-09-17T02:53:20.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7abcf21640 1 Processor -- start 2024-09-17T02:53:20.895 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7abcf21640 1 -- start start 2024-09-17T02:53:20.897 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7abcf21640 1 --2- >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ab8074350 0x7f7ab813aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7abcf21640 1 --2- >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ab815d790 0x7f7ab814ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7abcf21640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab815b570 0x7f7ab815b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7abcf21640 1 -- --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab816afe0 con 0x7f7ab815b570 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7abcf21640 1 -- --> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab816ae60 con 0x7f7ab815d790 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7abcf21640 1 -- --> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] -- mon_getmap magic: 0 -- 0x7f7ab816b160 con 0x7f7ab8074350 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab815b570 0x7f7ab815b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab6d9d640 1 --2- >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab815b570 0x7f7ab815b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.131:3300/0 says I am v2:172.21.15.131:47742/0 (socket says 172.21.15.131:47742) 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab6d9d640 1 -- 172.21.15.131:0/2142800049 learned_addr learned my addr 172.21.15.131:0/2142800049 (peer_addr_for_me v2:172.21.15.131:0/0) 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab5d9b640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ab815d790 0x7f7ab814ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab659c640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ab8074350 0x7f7ab813aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab6d9d640 1 -- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ab8074350 msgr2=0x7f7ab813aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab6d9d640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ab8074350 0x7f7ab813aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.898 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab6d9d640 1 -- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ab815d790 msgr2=0x7f7ab814ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab6d9d640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ab815d790 0x7f7ab814ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab6d9d640 1 -- 172.21.15.131:0/2142800049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab815bec0 con 0x7f7ab815b570 2024-09-17T02:53:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab5d9b640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ab815d790 0x7f7ab814ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T02:53:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7ab6d9d640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab815b570 0x7f7ab815b980 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800b3e0 tx=0x7f7aa800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:53:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7aa6ffd640 1 -- 172.21.15.131:0/2142800049 <== mon.0 v2:172.21.15.131:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8015020 con 0x7f7ab815b570 2024-09-17T02:53:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7aa6ffd640 1 -- 172.21.15.131:0/2142800049 <== mon.0 v2:172.21.15.131:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f7aa8011850 con 0x7f7ab815b570 2024-09-17T02:53:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7abcf21640 1 -- 172.21.15.131:0/2142800049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab814cde0 con 0x7f7ab815b570 2024-09-17T02:53:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7aa6ffd640 1 -- 172.21.15.131:0/2142800049 <== mon.0 v2:172.21.15.131:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8011cc0 con 0x7f7ab815b570 2024-09-17T02:53:20.899 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.900+0000 7f7abcf21640 1 -- 172.21.15.131:0/2142800049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab814f500 con 0x7f7ab815b570 2024-09-17T02:53:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.904+0000 7f7abcf21640 1 -- 172.21.15.131:0/2142800049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ab814f270 con 0x7f7ab815b570 2024-09-17T02:53:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.904+0000 7f7aa6ffd640 1 -- 172.21.15.131:0/2142800049 <== mon.0 v2:172.21.15.131:3300/0 4 ==== mgrmap(e 5) ==== 97234+0+0 (secure 0 0 0) 0x7f7aa8018020 con 0x7f7ab815b570 2024-09-17T02:53:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.904+0000 7f7aa6ffd640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a98073200 0x7f7a98075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T02:53:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.904+0000 7f7aa6ffd640 1 -- 172.21.15.131:0/2142800049 <== mon.0 v2:172.21.15.131:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5507+0+0 (secure 0 0 0) 0x7f7aa8093fe0 con 0x7f7ab815b570 2024-09-17T02:53:20.900 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.904+0000 7f7ab659c640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a98073200 0x7f7a98075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T02:53:20.901 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.904+0000 7f7ab659c640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a98073200 0x7f7a98075710 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00012f0 tx=0x7f7ab0009040 comp rx=0 tx=0).ready entity=mgr.4129 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T02:53:20.904 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:20.908+0000 7f7aa6ffd640 1 -- 172.21.15.131:0/2142800049 <== mon.0 v2:172.21.15.131:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187803 (secure 0 0 0) 0x7f7aa801f020 con 0x7f7ab815b570 2024-09-17T02:53:21.108 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.112+0000 7f7abcf21640 1 -- 172.21.15.131:0/2142800049 --> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ab8073e10 con 0x7f7ab815b570 2024-09-17T02:53:21.109 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.112+0000 7f7aa6ffd640 1 -- 172.21.15.131:0/2142800049 <== mon.0 v2:172.21.15.131:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+3922 (secure 0 0 0) 0x7f7ab815bec0 con 0x7f7ab815b570 2024-09-17T02:53:21.109 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-17T02:53:21.109 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":115,"btime":"2024-09-17T02:52:48:978887+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5975,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6835/3040702607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6834","nonce":3040702607},{"type":"v1","addr":"172.21.15.189:6835","nonce":3040702607}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":5978,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.189:6837/1334693612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.189:6836","nonce":1334693612},{"type":"v1","addr":"172.21.15.189:6837","nonce":1334693612}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":8487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6834/309254167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6833","nonce":309254167},{"type":"v1","addr":"172.21.15.131:6834","nonce":309254167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":9051,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.131:6835/2755805715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.131:6832","nonce":2755805715},{"type":"v1","addr":"172.21.15.131:6835","nonce":2755805715}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115}],"filesystems":[]} 2024-09-17T02:53:21.110 INFO:teuthology.orchestra.run.smithi131.stderr:dumped fsmap epoch 115 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 -- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a98073200 msgr2=0x7f7a98075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a98073200 0x7f7a98075710 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7ab00012f0 tx=0x7f7ab0009040 comp rx=0 tx=0).stop 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 -- 172.21.15.131:0/2142800049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab815b570 msgr2=0x7f7ab815b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab815b570 0x7f7ab815b980 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800b3e0 tx=0x7f7aa800b8a0 comp rx=0 tx=0).stop 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 -- 172.21.15.131:0/2142800049 shutdown_connections 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:6800/674320934,v1:172.21.15.189:6801/674320934] conn(0x7f7a98073200 0x7f7a98075710 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.131:3300/0,v1:172.21.15.131:6789/0] conn(0x7f7ab815b570 0x7f7ab815b980 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:3300/0,v1:172.21.15.189:6789/0] conn(0x7f7ab815d790 0x7f7ab814ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 --2- 172.21.15.131:0/2142800049 >> [v2:172.21.15.189:3301/0,v1:172.21.15.189:6790/0] conn(0x7f7ab8074350 0x7f7ab813aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 -- 172.21.15.131:0/2142800049 >> 172.21.15.131:0/2142800049 conn(0x7f7ab80b2dc0 msgr2=0x7f7ab815dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 -- 172.21.15.131:0/2142800049 shutdown_connections 2024-09-17T02:53:21.112 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-17T02:53:21.116+0000 7f7abcf21640 1 -- 172.21.15.131:0/2142800049 wait complete. 2024-09-17T02:53:21.131 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-17T02:53:21.131 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T02:53:21.131 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T02:53:21.131 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.146 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T02:53:21.147 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T02:53:21.147 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.208 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T02:53:21.209 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T02:53:21.209 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.235 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T02:53:21.235 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T02:53:21.235 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.296 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T02:53:21.296 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T02:53:21.296 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T02:53:21.297 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.364 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T02:53:21.365 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T02:53:21.365 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.450 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T02:53:21.450 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T02:53:21.450 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.547 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T02:53:21.547 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T02:53:21.547 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.647 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T02:53:21.648 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T02:53:21.648 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.744 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T02:53:21.744 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T02:53:21.744 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.820 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T02:53:21.821 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T02:53:21.821 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.900 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T02:53:21.901 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T02:53:21.901 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:21.997 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T02:53:21.998 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T02:53:21.998 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T02:53:21.998 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:22.033 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T02:53:22.033 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T02:53:22.033 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:22.064 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T02:53:22.064 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T02:53:22.064 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T02:53:22.064 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:22.095 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T02:53:22.096 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T02:53:22.096 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:22.124 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T02:53:22.124 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T02:53:22.124 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T02:53:22.183 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T02:53:22.183 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T02:53:22.184 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | egrep -v 'object missing on disk' | egrep -v 'Invalid tag char' | head -n 1 2024-09-17T02:53:22.203 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:53:22.203 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T02:53:22.553 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:53:22.553 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T02:53:22.733 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:53:22.733 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T02:53:22.890 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:53:22.890 DEBUG:teuthology.orchestra.run.smithi131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T02:53:23.030 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:53:23.031 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T02:53:23.320 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:53:23.320 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T02:53:23.530 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:53:23.530 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T02:53:23.714 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:53:23.714 DEBUG:teuthology.orchestra.run.smithi189:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T02:53:23.897 INFO:tasks.ceph:Archiving mon data... 2024-09-17T02:53:23.898 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354/data/mon.a.tgz 2024-09-17T02:53:23.898 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-09-17T02:53:23.904 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.Agaek9DkSb 2024-09-17T02:53:23.904 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Agaek9DkSb 2024-09-17T02:53:24.049 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.Agaek9DkSb 2024-09-17T02:53:24.111 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.Agaek9DkSb is 448KB 2024-09-17T02:53:24.172 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.Agaek9DkSb 2024-09-17T02:53:24.177 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354/data/mon.b.tgz 2024-09-17T02:53:24.177 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-09-17T02:53:24.183 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.8g0HitM1SI 2024-09-17T02:53:24.183 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.8g0HitM1SI 2024-09-17T02:53:24.336 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.8g0HitM1SI 2024-09-17T02:53:24.400 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.8g0HitM1SI is 465KB 2024-09-17T02:53:24.462 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.8g0HitM1SI 2024-09-17T02:53:24.468 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354/data/mon.c.tgz 2024-09-17T02:53:24.468 DEBUG:teuthology.orchestra.run.smithi189:> mktemp 2024-09-17T02:53:24.520 INFO:teuthology.orchestra.run.smithi189.stdout:/tmp/tmp.Iixbw5FdC7 2024-09-17T02:53:24.520 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Iixbw5FdC7 2024-09-17T02:53:24.676 DEBUG:teuthology.orchestra.run.smithi189:> sudo chmod 0666 /tmp/tmp.Iixbw5FdC7 2024-09-17T02:53:24.740 DEBUG:teuthology.orchestra.remote:smithi189:/tmp/tmp.Iixbw5FdC7 is 465KB 2024-09-17T02:53:24.803 DEBUG:teuthology.orchestra.run.smithi189:> rm -fr /tmp/tmp.Iixbw5FdC7 2024-09-17T02:53:24.808 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T02:53:24.808 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T02:53:24.811 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T02:53:24.871 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T02:53:24.913 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T02:53:24.913 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T02:53:24.914 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354/remote/smithi131/crash 2024-09-17T02:53:24.915 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T02:53:24.930 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354/remote/smithi189/crash 2024-09-17T02:53:24.931 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T02:53:24.969 INFO:tasks.ceph:Shutting down logrotate 2024-09-17T02:53:24.969 DEBUG:teuthology.orchestra.run.smithi131:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:53:24.976 DEBUG:teuthology.orchestra.run.smithi189:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T02:53:25.052 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T02:53:25.065 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T02:53:25.107 INFO:tasks.ceph:Compressing logs... 2024-09-17T02:53:25.108 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T02:53:25.111 DEBUG:teuthology.orchestra.run.smithi189:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T02:53:25.130 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43958.log 2024-09-17T02:53:25.131 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36331.log 2024-09-17T02:53:25.131 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35466.log 2024-09-17T02:53:25.132 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43958.log.gz 2024-09-17T02:53:25.132 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51636.log 2024-09-17T02:53:25.132 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36331.log.gz 2024-09-17T02:53:25.132 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39935.log 2024-09-17T02:53:25.134 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35466.log: /var/log/ceph/ceph-client.admin.51636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35023.log 2024-09-17T02:53:25.135 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51636.log.gz 2024-09-17T02:53:25.135 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35202.log 2024-09-17T02:53:25.135 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39935.log.gz 2024-09-17T02:53:25.135 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35466.log.gz 2024-09-17T02:53:25.135 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35023.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35023.log.gz 2024-09-17T02:53:25.135 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51336.log 2024-09-17T02:53:25.135 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52602.log 2024-09-17T02:53:25.135 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35202.log.gz 2024-09-17T02:53:25.137 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.51336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36533.log 2024-09-17T02:53:25.137 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48556.log 2024-09-17T02:53:25.137 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36533.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52602.log.gz 2024-09-17T02:53:25.137 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38915.log 2024-09-17T02:53:25.137 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36533.log.gz 2024-09-17T02:53:25.137 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48524.log 2024-09-17T02:53:25.137 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.38915.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48556.log.gz 2024-09-17T02:53:25.137 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38324.log 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29547.log 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48524.log.gz 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40466.log 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38324.log.gz 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39156.log 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29547.log.gz 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48186.log 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40466.log.gz 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51022.log 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39156.log.gz 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39909.log 2024-09-17T02:53:25.140 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48186.log.gz 2024-09-17T02:53:25.141 INFO:teuthology.orchestra.run.smithi131.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.0.51336.log.gz 2024-09-17T02:53:25.141 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39459.log 2024-09-17T02:53:25.144 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51022.log.gz 2024-09-17T02:53:25.144 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39909.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39459.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39909.log.gz 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.39459.log.gz 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42241.log 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50666.log 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.42241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37966.log 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.38915.log.gz 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42241.log.gz 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39055.log 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37966.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50666.log.gz 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50931.log 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37966.log.gz 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41195.log 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39055.log.gz 2024-09-17T02:53:25.145 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39085.log 2024-09-17T02:53:25.146 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41195.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50931.log.gz 2024-09-17T02:53:25.146 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43700.log 2024-09-17T02:53:25.146 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41195.log.gz 2024-09-17T02:53:25.146 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50090.log 2024-09-17T02:53:25.146 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39085.log.gz 2024-09-17T02:53:25.146 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36124.log 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35944.log 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36124.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43700.log.gz 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40221.log 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50090.log.gz 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36124.log.gz 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46307.log 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53201.log 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35944.log.gz 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46307.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40221.log.gz 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40195.log 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46307.log.gz 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50326.log 2024-09-17T02:53:25.152 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53201.log.gz 2024-09-17T02:53:25.153 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40195.log: -- /var/log/ceph/ceph-client.admin.36516.log 2024-09-17T02:53:25.153 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50326.log: 87.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.40195.log.gz -5 2024-09-17T02:53:25.153 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40273.log 2024-09-17T02:53:25.153 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50326.log.gz 2024-09-17T02:53:25.153 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45978.log 2024-09-17T02:53:25.153 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36516.log.gz 2024-09-17T02:53:25.153 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48491.log 2024-09-17T02:53:25.153 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40273.log: /var/log/ceph/ceph-client.admin.45978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39642.log 2024-09-17T02:53:25.153 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40273.log.gz 2024-09-17T02:53:25.153 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45978.log.gz 2024-09-17T02:53:25.153 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38467.log 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48491.log.gz 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45818.log 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39642.log: /var/log/ceph/ceph-client.admin.38467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48899.log 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39642.log.gz 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38467.log.gz 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49623.log 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45818.log.gz 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48283.log 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48899.log: /var/log/ceph/ceph-client.admin.49623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37372.log 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48899.log.gz 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48283.log: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49623.log.gz 2024-09-17T02:53:25.154 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48283.log.gz 2024-09-17T02:53:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48082.log 2024-09-17T02:53:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28969.log 2024-09-17T02:53:25.169 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37372.log.gz 2024-09-17T02:53:25.169 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39879.log 2024-09-17T02:53:25.169 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48082.log.gz 2024-09-17T02:53:25.170 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.28969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38272.log 2024-09-17T02:53:25.170 INFO:teuthology.orchestra.run.smithi131.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28969.log.gz 2024-09-17T02:53:25.170 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44024.log 2024-09-17T02:53:25.170 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39879.log: /var/log/ceph/ceph-client.admin.38272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39850.log 2024-09-17T02:53:25.170 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39879.log.gz 2024-09-17T02:53:25.170 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38272.log.gz 2024-09-17T02:53:25.170 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36125.log 2024-09-17T02:53:25.170 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44024.log.gz 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39616.log 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39850.log.gz 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45236.log 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36125.log: /var/log/ceph/ceph-client.admin.39616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50562.log 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36125.log.gz 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45236.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39616.log.gz 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53128.log 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45236.log.gz 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39258.log 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47705.log 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53128.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50562.log.gz 2024-09-17T02:53:25.171 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48335.log 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53128.log.gz 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39258.log.gz 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.47705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37730.log 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39564.log 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48335.log.gz 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37730.log: 92.3% -- replaced with /var/log/ceph/ceph-client.0.47705.log.gz 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37398.log 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39564.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37730.log.gz 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35729.log 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39564.log.gz 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41592.log 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37398.log.gz 2024-09-17T02:53:25.172 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46366.log 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35729.log: /var/log/ceph/ceph-client.admin.41592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41540.log 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35729.log.gz 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41592.log.gz 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46615.log 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46366.log.gz 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41774.log 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46615.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41540.log.gz -5 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48413.log 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37888.log 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46615.log.gz 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41774.log.gz 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53409.log 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39200.log 2024-09-17T02:53:25.173 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37888.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48413.log.gz 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42499.log 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37888.log.gz 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39200.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53409.log.gz 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28921.log 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39200.log.gz 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42499.log: /var/log/ceph/ceph-client.admin.49966.log 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.28921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45623.log 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42499.log.gz 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr: 76.8% -- replaced with /var/log/ceph/ceph-client.admin.28921.log.gz 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42041.log 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42575.log 2024-09-17T02:53:25.174 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45623.log.gz 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.49966.log.gz 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36544.log 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42575.log: gzip -5 89.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.42041.log.gz -- 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.38246.log 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42575.log.gz 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48361.log 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36544.log.gz 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52847.log 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46185.log 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38246.log.gz 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48361.log.gz 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36535.log 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52847.log.gz 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49551.log 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36535.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46185.log.gz 2024-09-17T02:53:25.175 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49910.log 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36535.log.gz 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43269.log 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41800.log 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49551.log.gz 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43269.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49910.log.gz 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50849.log 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43269.log.gz 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40958.log 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41800.log.gz 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45463.log/var/log/ceph/ceph-client.admin.50849.log: 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40958.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50849.log.gz 2024-09-17T02:53:25.176 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52576.log 2024-09-17T02:53:25.177 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40958.log.gz 2024-09-17T02:53:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52434.log 2024-09-17T02:53:25.177 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45463.log.gz 2024-09-17T02:53:25.177 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48212.log 2024-09-17T02:53:25.177 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52576.log.gz 2024-09-17T02:53:25.197 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T02:53:25.197 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29037.log 2024-09-17T02:53:25.197 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-17T02:53:25.197 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.29037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29110.log 2024-09-17T02:53:25.197 INFO:teuthology.orchestra.run.smithi189.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29037.log.gz 2024-09-17T02:53:25.197 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34626.log 2024-09-17T02:53:25.197 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.29110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29085.log 2024-09-17T02:53:25.197 INFO:teuthology.orchestra.run.smithi189.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29110.log.gz 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.3.34626.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.29085.log: 88.2% -- replaced with /var/log/ceph/ceph-client.3.34626.log.gz 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29085.log.gz 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29061.log 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29134.log 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.29061.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29061.log.gz 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34507.log 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.29134.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29134.log.gz 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.1.34507.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28989.log 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: 88.5% -- replaced with /var/log/ceph/ceph-client.1.34507.log.gz 2024-09-17T02:53:25.198 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T02:53:25.199 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.28989.log: gzip -5 --verbose 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28989.log.gz 2024-09-17T02:53:25.199 INFO:teuthology.orchestra.run.smithi189.stderr: -- /var/log/ceph/ceph.audit.log 2024-09-17T02:53:25.199 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-17T02:53:25.199 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29158.log 2024-09-17T02:53:25.199 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-mds.d.log: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T02:53:25.199 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-17T02:53:25.199 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-client.admin.29158.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.29158.log.gz 2024-09-17T02:53:25.199 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T02:53:25.199 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-mds.b.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T02:53:25.201 INFO:teuthology.orchestra.run.smithi189.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T02:53:25.219 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29013.log 2024-09-17T02:53:25.227 INFO:teuthology.orchestra.run.smithi189.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.29013.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29013.log.gz 2024-09-17T02:53:25.257 INFO:teuthology.orchestra.run.smithi189.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-17T02:53:25.259 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37313.log 2024-09-17T02:53:25.259 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50217.log 2024-09-17T02:53:25.259 INFO:teuthology.orchestra.run.smithi131.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48212.log.gz 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45210.log 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37313.log.gz 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47377.log 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45210.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.50217.log.gz 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.53487.log 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45210.log.gz 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.52434.log.gz 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52008.log 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47377.log: /var/log/ceph/ceph-client.admin.53487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46057.log 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.52008.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53487.log.gz 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38193.log 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36332.log 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38193.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46057.log.gz 2024-09-17T02:53:25.260 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45900.log 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38193.log.gz 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.47377.log.gz 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36332.log.gz 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50725.log 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45900.log: /var/log/ceph/ceph-client.admin.36123.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45900.log.gz 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37992.log 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38783.log 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50725.log: gzip -5/var/log/ceph/ceph-client.admin.37992.log: --verbose -- /var/log/ceph/ceph-client.admin.28908.log 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50725.log.gz 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.52008.log.gz 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38783.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37992.log.gz 2024-09-17T02:53:25.261 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44937.log 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38783.log.gz 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.28908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45132.log 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28908.log.gz 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.44937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37142.log 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41115.log 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45132.log: /var/log/ceph/ceph-client.admin.37142.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45132.log.gz 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.0.44937.log.gz 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37142.log.gz 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50823.log 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37862.log 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41115.log: /var/log/ceph/ceph-client.admin.50823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46665.log 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41115.log.gz 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37862.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50823.log.gz 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38076.log 2024-09-17T02:53:25.262 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37862.log.gz 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44146.log 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38076.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46665.log.gz 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43358.log 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38076.log.gz 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43981.log 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40664.log 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44146.log.gz 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43981.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43358.log.gz 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49003.log 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40664.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24053.log 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43981.log.gz 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49003.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40664.log.gz 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35256.log 2024-09-17T02:53:25.263 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49003.log.gz 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.tmp-client.admin.24053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44990.log 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24053.log.gz 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37700.log 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35256.log.gz 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28878.log 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47293.log 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44990.log.gz 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37700.log.gz 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.28878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28878.log.gz 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40349.log 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47940.log 2024-09-17T02:53:25.264 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40349.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47293.log.gz 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.37237.log 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40349.log.gz 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50692.log 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.2.37237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29037.log 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47940.log.gz 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42770.log 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50692.log.gz 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29037.log: 89.6% -- replaced with /var/log/ceph/ceph-client.2.37237.log.gz 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29037.log.gz 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39023.log 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42770.log.gz 2024-09-17T02:53:25.265 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49331.log 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35148.log 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39023.log.gz 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52080.log 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49331.log.gz 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50482.log 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35148.log.gz 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51952.log 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52080.log.gz 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46589.log 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50482.log.gz 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45649.log 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51952.log.gz 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53331.log 2024-09-17T02:53:25.266 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46589.log.gz 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38044.log 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45649.log.gz 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45489.log 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53331.log.gz 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49726.log 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38044.log.gz 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42317.log 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45489.log.gz 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40039.log 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49726.log.gz 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52230.log 2024-09-17T02:53:25.267 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42317.log.gz 2024-09-17T02:53:25.268 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43414.log 2024-09-17T02:53:25.268 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40039.log.gz 2024-09-17T02:53:25.268 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49552.log 2024-09-17T02:53:25.268 INFO:teuthology.orchestra.run.smithi131.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52230.log.gz 2024-09-17T02:53:25.268 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28946.log 2024-09-17T02:53:25.268 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43414.log.gz 2024-09-17T02:53:25.268 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50194.log 2024-09-17T02:53:25.268 INFO:teuthology.orchestra.run.smithi131.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.49552.log.gz 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.28946.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28946.log.gz 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35786.log 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35864.log 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35786.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50194.log.gz 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35786.log.gz 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47761.log 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48925.log 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35864.log: /var/log/ceph/ceph-client.admin.47761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46559.log 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7%/var/log/ceph/ceph-client.admin.48925.log: -- replaced with /var/log/ceph/ceph-client.admin.35864.log.gz 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40854.log 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48925.log.gz 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47761.log.gz 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50019.log 2024-09-17T02:53:25.269 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35388.log 2024-09-17T02:53:25.270 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46559.log.gz 2024-09-17T02:53:25.270 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40854.log.gz 2024-09-17T02:53:25.270 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40247.log 2024-09-17T02:53:25.270 INFO:teuthology.orchestra.run.smithi131.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.50019.log.gz 2024-09-17T02:53:25.270 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50534.log 2024-09-17T02:53:25.270 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35388.log: /var/log/ceph/ceph-client.admin.40247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41403.log 2024-09-17T02:53:25.270 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35388.log.gz 2024-09-17T02:53:25.270 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50534.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40247.log.gz 2024-09-17T02:53:25.270 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40540.log 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50534.log.gz 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48108.log 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41403.log.gz 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40540.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.40540.log.gz 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44062.log 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48108.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40405.log 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48108.log.gz 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46135.log 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44062.log.gz 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.40405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50404.log 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50456.log 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46135.log: /var/log/ceph/ceph-client.admin.50404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37808.log 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50404.log.gz 2024-09-17T02:53:25.271 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50456.log: gzip 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46135.log.gz 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50456.log.gz 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr: -- /var/log/ceph/ceph-client.admin.38757.log 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37808.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37808.log.gz 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35812.log 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47573.log 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38757.log: /var/log/ceph/ceph-client.admin.35812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39961.log 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38757.log.gz 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35812.log.gz 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.40405.log.gz 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35174.log 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39961.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47573.log.gz 2024-09-17T02:53:25.272 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 87.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.39961.log.gz -- 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr: /var/log/ceph/ceph-client.admin.36776.log 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52953.log 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36776.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35174.log.gz 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53357.log 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36776.log.gz 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37836.log 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53357.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52953.log.gz 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36967.log 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53357.log.gz 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38220.log 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37836.log.gz 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52821.log 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36967.log.gz 2024-09-17T02:53:25.273 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29022.log 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38220.log: /var/log/ceph/ceph-client.admin.52821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51466.log 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38220.log.gz 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52821.log.gz 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47189.log 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29022.log.gz 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50640.log 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45023.log 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.51466.log.gz 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47189.log.gz 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36074.log 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45023.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50640.log.gz 2024-09-17T02:53:25.274 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48817.log 2024-09-17T02:53:25.275 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45023.log.gz 2024-09-17T02:53:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29109.log 2024-09-17T02:53:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36074.log: /var/log/ceph/ceph-client.admin.48817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34671.log 2024-09-17T02:53:25.275 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36074.log.gz 2024-09-17T02:53:25.275 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48817.log.gz 2024-09-17T02:53:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41167.log 2024-09-17T02:53:25.275 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29109.log.gz 2024-09-17T02:53:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.34671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52164.log 2024-09-17T02:53:25.275 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34671.log.gz 2024-09-17T02:53:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41566.log 2024-09-17T02:53:25.275 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41167.log: /var/log/ceph/ceph-client.admin.52164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42608.log 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41167.log.gz 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52164.log.gz 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45675.log 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46083.log 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41566.log.gz 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42608.log: /var/log/ceph/ceph-client.admin.45675.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42608.log.gz 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45281.log 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45675.log.gz 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53539.log 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46083.log.gz 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50757.log 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36543.log 2024-09-17T02:53:25.276 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53539.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45281.log.gz 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35414.log 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53539.log.gz 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50757.log.gz 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49486.log 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36543.log.gz 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38298.log 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35414.log.gz 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44422.log 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49486.log: /var/log/ceph/ceph-client.admin.38298.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49486.log.gz 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35122.log 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38298.log.gz 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48056.log 2024-09-17T02:53:25.277 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38018.log 2024-09-17T02:53:25.278 INFO:teuthology.orchestra.run.smithi131.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44422.log.gz 2024-09-17T02:53:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50966.log 2024-09-17T02:53:25.278 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35122.log.gz 2024-09-17T02:53:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38018.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48056.log.gz 2024-09-17T02:53:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39486.log 2024-09-17T02:53:25.278 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38018.log.gz 2024-09-17T02:53:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39590.log 2024-09-17T02:53:25.278 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50966.log.gz 2024-09-17T02:53:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43648.log 2024-09-17T02:53:25.278 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39486.log: /var/log/ceph/ceph-client.admin.39590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53773.log 2024-09-17T02:53:25.278 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39486.log.gz 2024-09-17T02:53:25.279 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39590.log.gz 2024-09-17T02:53:25.279 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46241.log 2024-09-17T02:53:25.279 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43648.log.gz 2024-09-17T02:53:25.279 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35542.log 2024-09-17T02:53:25.279 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53773.log: /var/log/ceph/ceph-client.0.46241.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53773.log.gz 2024-09-17T02:53:25.279 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39362.log 2024-09-17T02:53:25.279 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43332.log 2024-09-17T02:53:25.279 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39362.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35542.log.gz 2024-09-17T02:53:25.279 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44550.log 2024-09-17T02:53:25.279 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43332.log: gzip -5 87.6% --verbose -- /var/log/ceph/ceph-client.admin.42978.log 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39362.log.gz 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43332.log.gz 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37528.log 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44550.log: /var/log/ceph/ceph-client.admin.42978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41063.log 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50875.log 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44550.log.gz 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42978.log.gz 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37528.log.gz 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41063.log: 92.5% -- replaced with /var/log/ceph/ceph-client.0.46241.log.gz 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41063.log.gz 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40611.log 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45703.log 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50875.log.gz 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45437.log 2024-09-17T02:53:25.280 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53617.log 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45703.log.gz 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45437.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40611.log.gz 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45158.log 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45437.log.gz 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47811.log 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45158.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53617.log.gz 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37476.log 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45158.log.gz 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41462.log 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38653.log 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37476.log.gz 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47811.log.gz 2024-09-17T02:53:25.281 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49079.log 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28964.log 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38653.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41462.log.gz 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49079.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38653.log.gz 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41852.log 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49079.log.gz 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.28964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38378.log 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28964.log.gz 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42952.log 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53102.log 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41852.log.gz 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38378.log.gz 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37092.log 2024-09-17T02:53:25.282 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42952.log.gz 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29085.log 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50508.log 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8%/var/log/ceph/ceph-client.0.37092.log: -- replaced with /var/log/ceph/ceph-client.admin.53102.log.gz 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41644.log 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29085.log.gz 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46109.log 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.0.37092.log.gz 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50508.log.gz 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44341.log 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46109.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41644.log.gz 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41826.log 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46109.log.gz 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47083.log 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44341.log.gz 2024-09-17T02:53:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45926.log 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41826.log.gz 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50018.log 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36724.log 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45926.log.gz 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47083.log.gz 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48439.log 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35996.log 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36724.log.gz 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1%/var/log/ceph/ceph-client.admin.48439.log: -- replaced with /var/log/ceph/ceph-client.admin.50018.log.gz 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43518.log 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48439.log.gz 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39824.log 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35996.log.gz 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38575.log 2024-09-17T02:53:25.284 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43518.log.gz 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38705.log 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39824.log.gz 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43596.log 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38575.log.gz 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41434.log 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38705.log.gz 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42449.log 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43596.log.gz 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42393.log 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41434.log.gz 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52875.log 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36750.log 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1%/var/log/ceph/ceph-client.admin.52875.log: -- replaced with /var/log/ceph/ceph-client.admin.42449.log.gz 2024-09-17T02:53:25.285 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42393.log.gz 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36750.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52875.log.gz 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42900.log 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36750.log.gz 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48024.log 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35492.log 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42900.log.gz 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52790.log 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48024.log.gz 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49884.log 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41089.log 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% 91.2% -- replaced with /var/log/ceph/ceph-client.admin.35492.log.gz 2024-09-17T02:53:25.286 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49884.log: -- replaced with /var/log/ceph/ceph-client.admin.52790.log.gz 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53383.log 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49884.log.gz 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44676.log 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41089.log: /var/log/ceph/ceph-client.admin.53383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46459.log 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41089.log.gz 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53383.log.gz 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36309.log 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44676.log.gz 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53279.log 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48791.log 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7%/var/log/ceph/ceph-client.admin.53279.log: -- replaced with /var/log/ceph/ceph-client.admin.36309.log.gz 91.6% -- replaced with /var/log/ceph/ceph-client.admin.46459.log.gz 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-17T02:53:25.287 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43091.log 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53279.log.gz 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41011.log 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48791.log: /var/log/ceph/ceph-client.admin.43091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36313.log 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48791.log.gz 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43091.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39178.log 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41011.log: /var/log/ceph/ceph-client.admin.36313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29049.log 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41011.log.gz 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36313.log.gz 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53461.log 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39178.log: /var/log/ceph/ceph-client.admin.29049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36022.log 2024-09-17T02:53:25.288 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39178.log.gz 2024-09-17T02:53:25.289 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29049.log.gz 2024-09-17T02:53:25.289 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29061.log 2024-09-17T02:53:25.289 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53461.log.gz 2024-09-17T02:53:25.289 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35230.log 2024-09-17T02:53:25.289 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36022.log.gz 2024-09-17T02:53:25.289 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38809.log 2024-09-17T02:53:25.289 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29061.log.gz 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36854.log 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2%/var/log/ceph/ceph-client.admin.38809.log: -- replaced with /var/log/ceph/ceph-client.admin.35230.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38493.log 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38809.log.gz 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36854.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41670.log 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45766.log: /var/log/ceph/ceph-client.admin.38493.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38493.log.gz 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40714.log 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47914.log 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41670.log.gz 2024-09-17T02:53:25.290 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45333.log 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40714.log.gz 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45385.log 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45333.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47914.log.gz 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46721.log 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45333.log.gz 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42089.log 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45385.log.gz 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.46721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45523.log 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45731.log 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38350.log 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42089.log.gz 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45523.log.gz 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-17T02:53:25.291 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45731.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45731.log.gz 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49749.log 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.0.46721.log.gz 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38549.log 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38350.log.gz 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36318.log 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38549.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.49749.log.gz/var/log/ceph/ceph-client.admin.36318.log: 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52186.log 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38549.log.gz 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36318.log.gz 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45792.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52520.log 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53723.log 2024-09-17T02:53:25.292 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52520.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52186.log.gz 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35056.log 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52520.log.gz 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44729.log 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53723.log.gz 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35056.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35056.log.gz 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39310.log 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49676.log 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr: 91.2%/var/log/ceph/ceph-client.admin.39310.log: -- replaced with /var/log/ceph/ceph-client.admin.44729.log.gz 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29157.log 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39310.log.gz 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49676.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.49676.log.gz 2024-09-17T02:53:25.293 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50144.log 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41221.log 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29157.log.gz 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35310.log 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38859.log 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41221.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50144.log.gz 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35310.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41221.log.gz 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35310.log.gz 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44090.log 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41351.log 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38859.log.gz 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37782.log 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:gzip 87.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.42640.log 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44090.log.gz 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41351.log: /var/log/ceph/ceph-client.admin.37782.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41351.log.gz 2024-09-17T02:53:25.294 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37782.log.gz 2024-09-17T02:53:25.295 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51048.log 2024-09-17T02:53:25.295 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53695.log 2024-09-17T02:53:25.316 INFO:teuthology.orchestra.run.smithi189.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-17T02:53:25.349 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42640.log.gz 2024-09-17T02:53:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50378.log 2024-09-17T02:53:25.349 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51048.log.gz 2024-09-17T02:53:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53695.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53695.log.gz 2024-09-17T02:53:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40091.log 2024-09-17T02:53:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46533.log 2024-09-17T02:53:25.349 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50378.log.gz/var/log/ceph/ceph-client.admin.40091.log: 2024-09-17T02:53:25.349 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36116.log 2024-09-17T02:53:25.349 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40091.log.gz 2024-09-17T02:53:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46533.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46533.log.gz 2024-09-17T02:53:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29640.log 2024-09-17T02:53:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40737.log 2024-09-17T02:53:25.350 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7%/var/log/ceph/ceph-client.admin.29640.log: -- replaced with /var/log/ceph/ceph-client.admin.36116.log.gz 2024-09-17T02:53:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35670.log 2024-09-17T02:53:25.350 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29640.log.gz 2024-09-17T02:53:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40737.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40737.log.gz 2024-09-17T02:53:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51665.log 2024-09-17T02:53:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43544.log 2024-09-17T02:53:25.350 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49459.log 2024-09-17T02:53:25.350 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51665.log.gz 2024-09-17T02:53:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43544.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43544.log.gz 2024-09-17T02:53:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41514.log 2024-09-17T02:53:25.351 INFO:teuthology.orchestra.run.smithi131.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.35670.log.gz/var/log/ceph/ceph-client.admin.49459.log: 2024-09-17T02:53:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36534.log 2024-09-17T02:53:25.351 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49459.log.gz 2024-09-17T02:53:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47266.log 2024-09-17T02:53:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41514.log: /var/log/ceph/ceph-client.admin.36534.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41514.log.gz 2024-09-17T02:53:25.351 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36534.log.gz 2024-09-17T02:53:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41722.log 2024-09-17T02:53:25.351 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29493.log 2024-09-17T02:53:25.352 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47266.log.gz/var/log/ceph/ceph-client.admin.41722.log: 2024-09-17T02:53:25.352 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51409.log 2024-09-17T02:53:25.352 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41722.log.gz 2024-09-17T02:53:25.352 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29493.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29493.log.gz 2024-09-17T02:53:25.352 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51782.log 2024-09-17T02:53:25.352 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51204.log 2024-09-17T02:53:25.352 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4%/var/log/ceph/ceph-client.admin.51782.log: -- replaced with /var/log/ceph/ceph-client.admin.51409.log.gz 2024-09-17T02:53:25.352 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41902.log 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51782.log.gz 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51204.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51204.log.gz 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35970.log 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52137.log 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41902.log.gz/var/log/ceph/ceph-client.admin.35970.log: 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47033.log 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35970.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49516.log 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52137.log: /var/log/ceph/ceph-client.admin.47033.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52137.log.gz 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48977.log 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.47033.log.gz 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42848.log 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49516.log.gz 2024-09-17T02:53:25.353 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41958.log 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48977.log: /var/log/ceph/ceph-client.admin.42848.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48977.log.gz 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42848.log.gz 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48873.log 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37588.log 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.41958.log: /var/log/ceph/ceph-client.admin.48873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52482.log 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37588.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48873.log.gz 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37588.log.gz 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.0.41958.log.gz 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38996.log 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29133.log 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52482.log: /var/log/ceph/ceph-client.admin.38996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51126.log 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5%/var/log/ceph/ceph-client.admin.29133.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52482.log.gz 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38996.log.gz 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29133.log.gz 2024-09-17T02:53:25.354 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44372.log 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46004.log 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51126.log: /var/log/ceph/ceph-client.admin.44372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40489.log 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46004.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51126.log.gz 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44372.log.gz 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46004.log.gz 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36524.log 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52979.log 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40489.log: /var/log/ceph/ceph-client.admin.36524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48309.log 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3%/var/log/ceph/ceph-client.admin.52979.log: -- replaced with /var/log/ceph/ceph-client.admin.36524.log.gz 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40489.log.gz 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52979.log.gz 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41254.log 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40299.log 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6%/var/log/ceph/ceph-client.admin.41254.log: -- replaced with /var/log/ceph/ceph-client.admin.48309.log.gz 2024-09-17T02:53:25.355 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40802.log 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41254.log.gz 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48134.log 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40299.log: /var/log/ceph/ceph-client.admin.40802.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40802.log.gz 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40299.log.gz 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38731.log 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48160.log 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48134.log.gz/var/log/ceph/ceph-client.admin.38731.log: 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53050.log 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38731.log.gz 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49306.log 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48160.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48160.log.gz 2024-09-17T02:53:25.356 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53050.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53050.log.gz 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40828.log 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49306.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40828.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49306.log.gz 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48847.log 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40828.log.gz 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44626.log 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37424.log 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44626.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48847.log.gz 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43570.log 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr: 90.8%/var/log/ceph/ceph-client.admin.37424.log: -- replaced with /var/log/ceph/ceph-client.admin.44626.log.gz 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37424.log.gz 2024-09-17T02:53:25.357 INFO:teuthology.orchestra.run.smithi131.stderr: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43440.log 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53172.log 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43570.log: /var/log/ceph/ceph-client.admin.43440.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51178.log 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43570.log.gz 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43440.log.gz 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39668.log 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35838.log 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53172.log.gz 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39668.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51178.log.gz 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50299.log 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39668.log.gz 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41286.log 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35838.log: /var/log/ceph/ceph-client.admin.50299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43492.log 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35838.log.gz 2024-09-17T02:53:25.358 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41286.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50299.log.gz 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41286.log.gz 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41748.log 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50614.log 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43492.log: /var/log/ceph/ceph-client.admin.41748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53305.log 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50614.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41748.log.gz 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43492.log.gz 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50614.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51100.log 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37346.log 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53305.log: /var/log/ceph/ceph-client.admin.51100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52305.log 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51100.log.gz 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53305.log.gz 2024-09-17T02:53:25.359 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37346.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37346.log.gz 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44855.log 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38167.log 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4%/var/log/ceph/ceph-client.admin.44855.log: -- replaced with /var/log/ceph/ceph-client.admin.52305.log.gz 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51846.log 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44855.log.gz 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38523.log 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38167.log: /var/log/ceph/ceph-client.admin.51846.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38167.log.gz 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51846.log.gz 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39134.log 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45952.log 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38523.log: /var/log/ceph/ceph-client.admin.39134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36536.log 2024-09-17T02:53:25.360 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45952.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38523.log.gz 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39134.log.gz 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45952.log.gz 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43674.log 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46827.log 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36536.log: /var/log/ceph/ceph-client.admin.43674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50430.log 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46827.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43674.log.gz 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36536.log.gz 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45106.log 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37673.log 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46827.log.gz 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50430.log: /var/log/ceph/ceph-client.admin.45106.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50430.log.gz 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45106.log.gz 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52546.log 2024-09-17T02:53:25.361 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47324.log 2024-09-17T02:53:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37673.log: gzip/var/log/ceph/ceph-client.admin.52546.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45848.log 2024-09-17T02:53:25.362 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52546.log.gz 2024-09-17T02:53:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47324.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37673.log.gz 2024-09-17T02:53:25.362 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47324.log.gz 2024-09-17T02:53:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40169.log 2024-09-17T02:53:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51436.log 2024-09-17T02:53:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45848.log: /var/log/ceph/ceph-client.admin.40169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51757.log 2024-09-17T02:53:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51436.log: 87.4% 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45848.log.gz 2024-09-17T02:53:25.362 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40169.log.gz 2024-09-17T02:53:25.362 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51436.log.gz 2024-09-17T02:53:25.362 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36048.log 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37914.log 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51757.log: /var/log/ceph/ceph-client.admin.36048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47007.log 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37914.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36048.log.gz 89.4% 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51757.log.gz 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37914.log.gz 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44805.log 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28941.log 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35890.log 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44805.log: /var/log/ceph/ceph-client.admin.28941.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47007.log.gz 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr: 77.3% -- replaced with /var/log/ceph/ceph-client.admin.28941.log.gz 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45080.log 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44805.log.gz 2024-09-17T02:53:25.363 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49794.log 2024-09-17T02:53:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35890.log: /var/log/ceph/ceph-client.admin.45080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39284.log 2024-09-17T02:53:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49794.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35890.log.gz 2024-09-17T02:53:25.364 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45080.log.gz 2024-09-17T02:53:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46495.log 2024-09-17T02:53:25.364 INFO:teuthology.orchestra.run.smithi131.stderr: 90.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39987.log 2024-09-17T02:53:25.364 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49794.log.gz 2024-09-17T02:53:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39284.log: /var/log/ceph/ceph-client.admin.46495.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39284.log.gz 2024-09-17T02:53:25.364 INFO:teuthology.orchestra.run.smithi131.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46495.log.gz 2024-09-17T02:53:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42267.log 2024-09-17T02:53:25.364 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48387.log 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39987.log: /var/log/ceph/ceph-client.admin.42267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36128.log 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48387.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39987.log.gz 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37756.log 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42267.log.gz 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48387.log.gz 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36128.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36128.log.gz 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53005.log 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37940.log 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37756.log: /var/log/ceph/ceph-client.admin.53005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38406.log 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37940.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37756.log.gz 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53005.log.gz 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37940.log.gz 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49940.log 2024-09-17T02:53:25.365 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36115.log 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38406.log: /var/log/ceph/ceph-client.admin.49940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48238.log 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36115.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49940.log.gz 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38406.log.gz 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42666.log 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36115.log.gz 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41141.log 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48238.log: /var/log/ceph/ceph-client.admin.42666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48621.log 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48238.log.gz 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41141.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42666.log.gz 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41141.log.gzgzip 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45577.log 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51230.log 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48621.log: /var/log/ceph/ceph-client.admin.45577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40512.log 2024-09-17T02:53:25.366 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51230.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48621.log.gz 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45577.log.gz 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36121.log 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr: 91.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39798.log 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51230.log.gz 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40512.log: /var/log/ceph/ceph-client.admin.36121.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36121.log.gz 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40512.log.gz 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44600.log 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39733.log 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48465.log 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44600.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.39798.log.gz 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44600.log.gz 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39733.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39733.log.gz 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45307.log 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39232.log 2024-09-17T02:53:25.367 INFO:teuthology.orchestra.run.smithi131.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48465.log.gz/var/log/ceph/ceph-client.admin.45307.log: 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29097.log 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45307.log.gz 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45184.log 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39232.log.gz 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50992.log 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29097.log: /var/log/ceph/ceph-client.admin.45184.log: 86.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36111.log 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29097.log.gz 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45184.log.gz 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50992.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50992.log.gz 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47517.log 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35918.log 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36111.log: /var/log/ceph/ceph-client.admin.47517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44253.log 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35918.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47517.log.gz 2024-09-17T02:53:25.368 INFO:teuthology.orchestra.run.smithi131.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36111.log.gz 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35918.log.gz 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44881.log 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52927.log 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.44253.log: /var/log/ceph/ceph-client.admin.44881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39112.log 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52927.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44881.log.gz 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52927.log.gz 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.44253.log.gz 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50588.log 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43806.log 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39112.log: /var/log/ceph/ceph-client.admin.50588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42343.log 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.43806.log: 87.7% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39112.log.gz 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50588.log.gz 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47422.log 2024-09-17T02:53:25.369 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52901.log 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42343.log: /var/log/ceph/ceph-client.admin.47422.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42343.log.gz 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52901.log: 92.5% 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52901.log.gz 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.0.43806.log.gz 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.47422.log.gz 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53591.log 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.53591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36906.log 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mds.c.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53591.log.gz 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42139.log 2024-09-17T02:53:25.370 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36802.log 2024-09-17T02:53:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42139.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.36906.log.gz 2024-09-17T02:53:25.371 INFO:teuthology.orchestra.run.smithi131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42139.log.gz 2024-09-17T02:53:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29073.log 2024-09-17T02:53:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36802.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36802.log.gz 2024-09-17T02:53:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-17T02:53:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42796.log 2024-09-17T02:53:25.371 INFO:teuthology.orchestra.run.smithi131.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29073.log.gz 2024-09-17T02:53:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40985.log 2024-09-17T02:53:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42796.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42796.log.gz 2024-09-17T02:53:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29145.log 2024-09-17T02:53:25.371 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40985.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40985.log.gz 2024-09-17T02:53:25.372 INFO:teuthology.orchestra.run.smithi189.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T02:53:25.379 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40770.log 2024-09-17T02:53:25.379 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29145.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29145.log.gz 2024-09-17T02:53:25.379 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39414.log 2024-09-17T02:53:25.379 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40770.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40770.log.gz 2024-09-17T02:53:25.379 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52208.log 2024-09-17T02:53:25.379 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39414.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39414.log.gz 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45411.log 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52208.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52208.log.gz 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29121.log 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45411.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45411.log.gz 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44196.log 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29121.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29121.log.gz 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52708.log 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44196.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44196.log.gz 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50352.log 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.52708.log: 93.3% -- replaced with /var/log/ceph/ceph-client.0.52708.log.gz 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41377.log 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50352.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50352.log.gz 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53643.log 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41377.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41377.log.gz 2024-09-17T02:53:25.380 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42744.log 2024-09-17T02:53:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53643.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53643.log.gz 2024-09-17T02:53:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47139.log 2024-09-17T02:53:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42744.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42744.log.gz 2024-09-17T02:53:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44474.log 2024-09-17T02:53:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47139.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47139.log.gz 2024-09-17T02:53:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52652.log 2024-09-17T02:53:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44474.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44474.log.gz 2024-09-17T02:53:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49387.log 2024-09-17T02:53:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52652.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52652.log.gz 2024-09-17T02:53:25.381 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49135.log 2024-09-17T02:53:25.382 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.49387.log: 94.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42018.log 2024-09-17T02:53:25.386 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.49135.log: 94.1% -- replaced with /var/log/ceph/ceph-client.0.49387.log.gz 2024-09-17T02:53:25.386 INFO:teuthology.orchestra.run.smithi131.stderr: -- replaced with /var/log/ceph/ceph-client.0.49135.log.gz 2024-09-17T02:53:25.386 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40880.log 2024-09-17T02:53:25.387 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42018.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42018.log.gz 2024-09-17T02:53:25.387 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41696.log 2024-09-17T02:53:25.388 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40880.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40880.log.gz 2024-09-17T02:53:25.388 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46957.log 2024-09-17T02:53:25.389 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41696.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41696.log.gz 2024-09-17T02:53:25.390 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53076.log 2024-09-17T02:53:25.391 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46957.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46957.log.gz 2024-09-17T02:53:25.391 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35362.log 2024-09-17T02:53:25.392 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53076.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53076.log.gz 2024-09-17T02:53:25.393 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48673.log 2024-09-17T02:53:25.393 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35362.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35362.log.gz 2024-09-17T02:53:25.394 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40932.log 2024-09-17T02:53:25.395 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48673.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48673.log.gz 2024-09-17T02:53:25.395 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39701.log 2024-09-17T02:53:25.396 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40932.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40932.log.gz 2024-09-17T02:53:25.396 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43240.log 2024-09-17T02:53:25.397 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39701.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39701.log.gz 2024-09-17T02:53:25.398 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49283.log 2024-09-17T02:53:25.398 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43240.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43240.log.gz 2024-09-17T02:53:25.399 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43030.log 2024-09-17T02:53:25.400 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49283.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49283.log.gz 2024-09-17T02:53:25.400 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46777.log 2024-09-17T02:53:25.401 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43030.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43030.log.gz 2024-09-17T02:53:25.401 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51820.log 2024-09-17T02:53:25.402 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46777.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46777.log.gz 2024-09-17T02:53:25.403 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53253.log 2024-09-17T02:53:25.404 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51820.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51820.log.gz 2024-09-17T02:53:25.404 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47649.log 2024-09-17T02:53:25.405 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53253.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53253.log.gz 2024-09-17T02:53:25.405 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28981.log 2024-09-17T02:53:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47649.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47649.log.gz 2024-09-17T02:53:25.406 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48647.log 2024-09-17T02:53:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.28981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28981.log.gz 2024-09-17T02:53:25.407 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51152.log 2024-09-17T02:53:25.408 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48647.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48647.log.gz 2024-09-17T02:53:25.409 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36330.log 2024-09-17T02:53:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51152.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51152.log.gz 2024-09-17T02:53:25.410 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48730.log 2024-09-17T02:53:25.411 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36330.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36330.log.gz 2024-09-17T02:53:25.411 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51572.log 2024-09-17T02:53:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48730.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48730.log.gz 2024-09-17T02:53:25.412 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53565.log 2024-09-17T02:53:25.418 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.0.51572.log: 94.3% -- replaced with /var/log/ceph/ceph-client.0.51572.log.gz 2024-09-17T02:53:25.418 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43928.log 2024-09-17T02:53:25.419 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53565.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53565.log.gz 2024-09-17T02:53:25.420 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53669.log 2024-09-17T02:53:25.420 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43928.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43928.log.gz 2024-09-17T02:53:25.421 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49821.log 2024-09-17T02:53:25.422 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53669.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53669.log.gz 2024-09-17T02:53:25.422 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35282.log 2024-09-17T02:53:25.423 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49821.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49821.log.gz 2024-09-17T02:53:25.423 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36880.log 2024-09-17T02:53:25.424 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35282.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35282.log.gz 2024-09-17T02:53:25.424 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41037.log 2024-09-17T02:53:25.426 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36880.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36880.log.gz 2024-09-17T02:53:25.426 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39512.log 2024-09-17T02:53:25.426 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41037.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41037.log.gz 2024-09-17T02:53:25.427 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43622.log 2024-09-17T02:53:25.428 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39512.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39512.log.gz 2024-09-17T02:53:25.428 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51902.log 2024-09-17T02:53:25.430 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43622.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43622.log.gz 2024-09-17T02:53:25.430 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43188.log 2024-09-17T02:53:25.432 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51902.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.51902.log.gz 2024-09-17T02:53:25.432 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45359.log 2024-09-17T02:53:25.432 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43188.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43188.log.gz 2024-09-17T02:53:25.433 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51516.log 2024-09-17T02:53:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45359.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45359.log.gz 2024-09-17T02:53:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47599.log 2024-09-17T02:53:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51516.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51516.log.gz 2024-09-17T02:53:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43004.log 2024-09-17T02:53:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47599.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47599.log.gz 2024-09-17T02:53:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53435.log 2024-09-17T02:53:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43004.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43004.log.gz 2024-09-17T02:53:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38441.log 2024-09-17T02:53:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53435.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53435.log.gz 2024-09-17T02:53:25.439 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28993.log 2024-09-17T02:53:25.439 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38441.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38441.log.gz 2024-09-17T02:53:25.440 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45054.log 2024-09-17T02:53:25.441 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.28993.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28993.log.gz 2024-09-17T02:53:25.441 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42525.log 2024-09-17T02:53:25.442 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45054.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45054.log.gz 2024-09-17T02:53:25.442 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46334.log 2024-09-17T02:53:25.443 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42525.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42525.log.gz 2024-09-17T02:53:25.443 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46931.log 2024-09-17T02:53:25.444 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46334.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46334.log.gz 2024-09-17T02:53:25.445 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40117.log 2024-09-17T02:53:25.446 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46931.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46931.log.gz 2024-09-17T02:53:25.446 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36336.log 2024-09-17T02:53:25.447 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40117.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40117.log.gz 2024-09-17T02:53:25.447 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52379.log 2024-09-17T02:53:25.448 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36336.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36336.log.gz 2024-09-17T02:53:25.448 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51698.log 2024-09-17T02:53:25.450 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52379.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.52379.log.gz 2024-09-17T02:53:25.451 INFO:teuthology.orchestra.run.smithi189.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T02:53:25.452 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45874.log 2024-09-17T02:53:25.452 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51698.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51698.log.gz 2024-09-17T02:53:25.452 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40143.log 2024-09-17T02:53:25.453 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.45874.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45874.log.gz 2024-09-17T02:53:25.453 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47215.log 2024-09-17T02:53:25.454 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40143.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40143.log.gz 2024-09-17T02:53:25.454 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37502.log 2024-09-17T02:53:25.455 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47215.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47215.log.gz 2024-09-17T02:53:25.456 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37561.log 2024-09-17T02:53:25.456 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37502.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37502.log.gz 2024-09-17T02:53:25.457 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52328.log 2024-09-17T02:53:25.458 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37561.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37561.log.gz 2024-09-17T02:53:25.458 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40065.log 2024-09-17T02:53:25.459 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52328.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.52328.log.gz 2024-09-17T02:53:25.460 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51074.log 2024-09-17T02:53:25.461 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40065.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40065.log.gz 2024-09-17T02:53:25.461 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43123.log 2024-09-17T02:53:25.462 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51074.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51074.log.gz 2024-09-17T02:53:25.462 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38141.log 2024-09-17T02:53:25.463 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43123.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43123.log.gz 2024-09-17T02:53:25.463 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48951.log 2024-09-17T02:53:25.464 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38141.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.38141.log.gz 2024-09-17T02:53:25.465 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47479.log 2024-09-17T02:53:25.465 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48951.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48951.log.gz 2024-09-17T02:53:25.466 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43384.log 2024-09-17T02:53:25.467 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47479.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47479.log.gz 2024-09-17T02:53:25.467 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38601.log 2024-09-17T02:53:25.468 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43384.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43384.log.gz 2024-09-17T02:53:25.468 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43058.log 2024-09-17T02:53:25.469 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38601.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38601.log.gz 2024-09-17T02:53:25.469 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52255.log 2024-09-17T02:53:25.470 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43058.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43058.log.gz 2024-09-17T02:53:25.471 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46881.log 2024-09-17T02:53:25.472 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52255.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52255.log.gz 2024-09-17T02:53:25.472 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39336.log 2024-09-17T02:53:25.474 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46881.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46881.log.gz 2024-09-17T02:53:25.474 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40013.log 2024-09-17T02:53:25.475 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39336.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39336.log.gz 2024-09-17T02:53:25.475 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44120.log 2024-09-17T02:53:25.476 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40013.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40013.log.gz 2024-09-17T02:53:25.476 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42064.log 2024-09-17T02:53:25.477 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44120.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44120.log.gz 2024-09-17T02:53:25.478 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35440.log 2024-09-17T02:53:25.479 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42064.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42064.log.gz 2024-09-17T02:53:25.479 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42191.log 2024-09-17T02:53:25.480 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35440.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35440.log.gz 2024-09-17T02:53:25.480 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53747.log 2024-09-17T02:53:25.481 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42191.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42191.log.gz 2024-09-17T02:53:25.482 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46030.log 2024-09-17T02:53:25.483 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53747.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53747.log.gz 2024-09-17T02:53:25.483 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48765.log 2024-09-17T02:53:25.484 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46030.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46030.log.gz 2024-09-17T02:53:25.484 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43750.log 2024-09-17T02:53:25.485 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48765.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48765.log.gz 2024-09-17T02:53:25.485 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42692.log 2024-09-17T02:53:25.486 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43750.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43750.log.gz 2024-09-17T02:53:25.487 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42822.log 2024-09-17T02:53:25.488 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42692.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42692.log.gz 2024-09-17T02:53:25.488 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35336.log 2024-09-17T02:53:25.488 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42822.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42822.log.gz 2024-09-17T02:53:25.489 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47354.log 2024-09-17T02:53:25.490 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35336.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35336.log.gz 2024-09-17T02:53:25.490 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49194.log 2024-09-17T02:53:25.491 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47354.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47354.log.gz 2024-09-17T02:53:25.491 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51876.log 2024-09-17T02:53:25.492 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49194.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49194.log.gz 2024-09-17T02:53:25.493 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29005.log 2024-09-17T02:53:25.499 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51876.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51876.log.gz 2024-09-17T02:53:25.499 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43214.log 2024-09-17T02:53:25.499 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29005.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29005.log.gz 2024-09-17T02:53:25.499 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50273.log 2024-09-17T02:53:25.499 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43214.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43214.log.gz 2024-09-17T02:53:25.499 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46419.log 2024-09-17T02:53:25.499 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50273.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50273.log.gz 2024-09-17T02:53:25.499 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53513.log 2024-09-17T02:53:25.499 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46419.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46419.log.gz 2024-09-17T02:53:25.499 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36329.log 2024-09-17T02:53:25.501 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53513.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53513.log.gz 2024-09-17T02:53:25.501 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44314.log 2024-09-17T02:53:25.501 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36329.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36329.log.gz 2024-09-17T02:53:25.501 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42874.log 2024-09-17T02:53:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44314.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44314.log.gz 2024-09-17T02:53:25.502 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50903.log 2024-09-17T02:53:25.503 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42874.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.42874.log.gz 2024-09-17T02:53:25.503 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40539.log 2024-09-17T02:53:25.504 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.50903.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50903.log.gz 2024-09-17T02:53:25.505 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49846.log 2024-09-17T02:53:25.506 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40539.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40539.log.gz 2024-09-17T02:53:25.506 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44779.log 2024-09-17T02:53:25.507 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49846.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49846.log.gz 2024-09-17T02:53:25.507 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39538.log 2024-09-17T02:53:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44779.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44779.log.gz 2024-09-17T02:53:25.509 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38627.log 2024-09-17T02:53:25.510 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39538.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39538.log.gz 2024-09-17T02:53:25.510 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42926.log 2024-09-17T02:53:25.511 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38627.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38627.log.gz 2024-09-17T02:53:25.511 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43870.log 2024-09-17T02:53:25.512 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42926.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42926.log.gz 2024-09-17T02:53:25.512 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52107.log 2024-09-17T02:53:25.513 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43870.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43870.log.gz 2024-09-17T02:53:25.514 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40906.log 2024-09-17T02:53:25.514 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52107.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52107.log.gz 2024-09-17T02:53:25.515 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37287.log 2024-09-17T02:53:25.516 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.40906.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40906.log.gz 2024-09-17T02:53:25.516 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37450.log 2024-09-17T02:53:25.517 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37287.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37287.log.gz 2024-09-17T02:53:25.517 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46396.log 2024-09-17T02:53:25.518 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37450.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37450.log.gz 2024-09-17T02:53:25.519 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36828.log 2024-09-17T02:53:25.519 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.46396.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46396.log.gz 2024-09-17T02:53:25.520 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48702.log 2024-09-17T02:53:25.521 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.36828.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36828.log.gz 2024-09-17T02:53:25.521 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42718.log 2024-09-17T02:53:25.522 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.48702.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48702.log.gz 2024-09-17T02:53:25.523 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51280.log 2024-09-17T02:53:25.523 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.42718.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42718.log.gz 2024-09-17T02:53:25.523 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37645.log 2024-09-17T02:53:25.525 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.51280.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51280.log.gz 2024-09-17T02:53:25.525 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35756.log 2024-09-17T02:53:25.525 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37645.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37645.log.gz 2024-09-17T02:53:25.526 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49029.log 2024-09-17T02:53:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.35756.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35756.log.gz 2024-09-17T02:53:25.527 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49221.log 2024-09-17T02:53:25.528 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49029.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49029.log.gz 2024-09-17T02:53:25.529 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29469.log 2024-09-17T02:53:25.530 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49221.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49221.log.gz 2024-09-17T02:53:25.530 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47864.log 2024-09-17T02:53:25.531 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.29469.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29469.log.gz 2024-09-17T02:53:25.531 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44524.log 2024-09-17T02:53:25.532 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47864.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.47864.log.gz 2024-09-17T02:53:25.533 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53227.log 2024-09-17T02:53:25.533 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.44524.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44524.log.gz 2024-09-17T02:53:25.534 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41618.log 2024-09-17T02:53:25.535 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.53227.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53227.log.gz 2024-09-17T02:53:25.535 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41488.log 2024-09-17T02:53:25.536 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41618.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41618.log.gz 2024-09-17T02:53:25.536 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43297.log 2024-09-17T02:53:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.41488.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41488.log.gz 2024-09-17T02:53:25.537 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49253.log 2024-09-17T02:53:25.538 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43297.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43297.log.gz 2024-09-17T02:53:25.539 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47990.log 2024-09-17T02:53:25.539 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.49253.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49253.log.gz 2024-09-17T02:53:25.540 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43897.log 2024-09-17T02:53:25.541 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47990.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47990.log.gz 2024-09-17T02:53:25.541 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47449.log 2024-09-17T02:53:25.542 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43897.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43897.log.gz 2024-09-17T02:53:25.543 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52407.log 2024-09-17T02:53:25.545 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.47449.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.47449.log.gz 2024-09-17T02:53:25.545 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39388.log 2024-09-17T02:53:25.548 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.52407.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.52407.log.gz 2024-09-17T02:53:25.548 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37619.log 2024-09-17T02:53:25.551 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.39388.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39388.log.gz 2024-09-17T02:53:25.551 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43466.log 2024-09-17T02:53:25.551 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.37619.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37619.log.gz 2024-09-17T02:53:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38679.log 2024-09-17T02:53:25.552 INFO:teuthology.orchestra.run.smithi131.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-17T02:53:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.43466.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43466.log.gz 2024-09-17T02:53:25.552 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47543.log 2024-09-17T02:53:25.553 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/ceph-client.admin.38679.log: /var/log/ceph/ceph-client.admin.47543.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38679.log.gz 2024-09-17T02:53:25.553 INFO:teuthology.orchestra.run.smithi131.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47543.log.gz 2024-09-17T02:53:25.583 INFO:teuthology.orchestra.run.smithi131.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-17T02:53:25.659 INFO:teuthology.orchestra.run.smithi189.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T02:53:26.161 INFO:teuthology.orchestra.run.smithi131.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T02:53:26.507 INFO:teuthology.orchestra.run.smithi189.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T02:53:26.528 INFO:teuthology.orchestra.run.smithi131.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T02:53:26.564 INFO:teuthology.orchestra.run.smithi189.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T02:53:26.600 INFO:teuthology.orchestra.run.smithi189.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T02:53:26.603 INFO:teuthology.orchestra.run.smithi131.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T02:53:26.632 INFO:teuthology.orchestra.run.smithi131.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T02:53:26.706 INFO:teuthology.orchestra.run.smithi189.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T02:53:26.708 INFO:teuthology.orchestra.run.smithi189.stderr: 2024-09-17T02:53:26.708 INFO:teuthology.orchestra.run.smithi189.stderr:real 0m1.549s 2024-09-17T02:53:26.708 INFO:teuthology.orchestra.run.smithi189.stderr:user 0m6.487s 2024-09-17T02:53:26.708 INFO:teuthology.orchestra.run.smithi189.stderr:sys 0m0.194s 2024-09-17T02:53:26.864 INFO:teuthology.orchestra.run.smithi131.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T02:53:26.866 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-17T02:53:26.866 INFO:teuthology.orchestra.run.smithi131.stderr:real 0m1.752s 2024-09-17T02:53:26.866 INFO:teuthology.orchestra.run.smithi131.stderr:user 0m7.576s 2024-09-17T02:53:26.866 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m0.423s 2024-09-17T02:53:26.866 INFO:tasks.ceph:Archiving logs... 2024-09-17T02:53:26.866 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354/remote/smithi131/log 2024-09-17T02:53:26.867 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T02:53:28.215 DEBUG:teuthology.misc:Transferring archived files from smithi189:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354/remote/smithi189/log 2024-09-17T02:53:28.215 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T02:53:29.031 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T02:53:29.040 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T02:53:29.041 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T02:53:29.043 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T02:53:29.121 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T02:53:29.130 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T02:53:29.131 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T02:53:29.133 DEBUG:teuthology.orchestra.run.smithi189:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T02:53:29.144 INFO:teuthology.orchestra.run.smithi131.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T02:53:29.145 INFO:teuthology.orchestra.run.smithi131.stdout:============================================================================== 2024-09-17T02:53:29.145 INFO:teuthology.orchestra.run.smithi131.stdout:*hv01.front.sepi 67.205.162.81 3 u 66 64 377 0.091 -8.813 2.469 2024-09-17T02:53:29.145 INFO:teuthology.orchestra.run.smithi131.stdout:+hv02.front.sepi 63.231.80.2 3 u 4 64 377 0.095 -5.433 2.490 2024-09-17T02:53:29.145 INFO:teuthology.orchestra.run.smithi131.stdout:+hv03.front.sepi 74.6.168.72 3 u 62 64 377 0.108 -4.540 2.942 2024-09-17T02:53:29.145 INFO:teuthology.orchestra.run.smithi131.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-17T02:53:29.147 INFO:teuthology.orchestra.run.smithi189.stdout: remote refid st t when poll reach delay offset jitter 2024-09-17T02:53:29.147 INFO:teuthology.orchestra.run.smithi189.stdout:============================================================================== 2024-09-17T02:53:29.147 INFO:teuthology.orchestra.run.smithi189.stdout:*hv01.front.sepi 67.205.162.81 3 u 2 64 377 0.102 -7.452 1.365 2024-09-17T02:53:29.147 INFO:teuthology.orchestra.run.smithi189.stdout:+hv02.front.sepi 63.231.80.2 3 u 66 64 377 0.075 -0.347 2.641 2024-09-17T02:53:29.147 INFO:teuthology.orchestra.run.smithi189.stdout:+hv03.front.sepi 74.6.168.72 3 u 1 64 377 0.091 -0.755 1.965 2024-09-17T02:53:29.147 INFO:teuthology.orchestra.run.smithi189.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-09-17T02:53:29.148 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T02:53:29.158 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T02:53:29.159 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T02:53:29.169 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T02:53:29.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T02:53:29.194 INFO:teuthology.task.internal:Duration was 1443.082651 seconds 2024-09-17T02:53:29.194 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T02:53:29.205 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T02:53:29.205 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T02:53:29.208 DEBUG:teuthology.orchestra.run.smithi189:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T02:53:29.245 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T02:53:29.246 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-09-17T02:53:29.246 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T02:53:29.294 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi189.front.sepia.ceph.com 2024-09-17T02:53:29.295 DEBUG:teuthology.orchestra.run.smithi189:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T02:53:29.305 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T02:53:29.305 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T02:53:29.339 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T02:53:29.376 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T02:53:29.376 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T02:53:29.411 DEBUG:teuthology.orchestra.run.smithi189:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T02:53:29.486 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T02:53:29.496 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T02:53:29.496 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T02:53:29.507 DEBUG:teuthology.orchestra.run.smithi189:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T02:53:29.517 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T02:53:29.527 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T02:53:29.555 DEBUG:teuthology.orchestra.run.smithi189:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T02:53:29.561 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-09-17T02:53:29.569 INFO:teuthology.orchestra.run.smithi189.stdout:kernel.core_pattern = core 2024-09-17T02:53:29.581 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T02:53:29.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:53:29.617 DEBUG:teuthology.orchestra.run.smithi189:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T02:53:29.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T02:53:29.627 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T02:53:29.638 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T02:53:29.638 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354/remote/smithi131 2024-09-17T02:53:29.639 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T02:53:29.684 DEBUG:teuthology.misc:Transferring archived files from smithi189:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908354/remote/smithi189 2024-09-17T02:53:29.684 DEBUG:teuthology.orchestra.run.smithi189:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T02:53:29.704 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T02:53:29.704 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T02:53:29.723 DEBUG:teuthology.orchestra.run.smithi189:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T02:53:29.748 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T02:53:29.760 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T02:53:29.760 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T02:53:29.771 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T02:53:29.771 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T02:53:29.774 DEBUG:teuthology.orchestra.run.smithi189:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T02:53:29.777 INFO:teuthology.orchestra.run.smithi131.stdout: 658640 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 02:53 /home/ubuntu/cephtest 2024-09-17T02:53:29.797 INFO:teuthology.orchestra.run.smithi189.stdout: 658638 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 02:53 /home/ubuntu/cephtest 2024-09-17T02:53:29.798 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T02:53:29.808 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T02:53:29.835 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/forward-scrub} duration: 1443.082650899887 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-17T02:53:29.835 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T02:53:29.921 INFO:teuthology.run:pass